Zooming While Shooting Video: Everything You Need to Know About High-Quality, Lossless Zooming

This article is targeted at advanced iPhone 5, 5c, and 5s users using iOS 7; video shooters; programmers coding video apps; and Apple's engineers in charge of bug fixing.

iPhones produce really decent video footage under ideal circumstances. Sure, they have their share of problems; for example, the lack of stereo audio recording, optical image stabilization (OIS), and wide field-of-view lens, all three supported by other flagships like the LG G2 running Android or the Nokia 92x/1020 running Windows Phone 8. However, under good lighting, and if you shoot on a tripod, you can easily have significantly better image quality than the LG G2 or most, if not all, Android phones. (Tripods make it possible to avoid camera blur, which is detrimental to the image quality.) Similarly, all iPhones since the iPhone 4 generally produce much better-detailed footage than most dedicated point-and-shoot cameras. They can, in video quality, easily beat even large-sensor DSLRs or ILC cameras because there's no aliasing or, unless you zoom in, moire in the image—unlike imost large-sensor cameras. Of course, a large-sensor camera will always have better dynamic range and less noise, meaning the iPhone is only preferable in ideal lighting circumstances or when you don't have any other camera with you.

iOS 7 added an absolutely excellent feature to all iPhones starting with the iPhone 5: lossless zooming. It allows for recording much better-quality video when using the zoom. Lossless zoom, up until now, has only been supported by smartphones specifically meant to be used as true cameras, like Nokia's PureView phones.

In this article, I explain how lossless zoom can be utilized in the stock Camera app, what to pay attention to when using third-party apps, and the small bug making the current version of the stock Camera app less than ideal. In addition, I also present some source code to programmers—a full video recorder client with zooming capabilities.

You may have heard the built-in, stock Camera app of the iPhone is also capable of zooming. All you need to do is “pinch” before or while recording. A zoom slider will then be displayed, which you can use to set any zoom level between 1x and 3x. The following screenshot shows it set at 3x (as always click on the image to see the full-sized version):

(annotated by a red rectangle)

You may also have heard of contradicting information on iOS 7 adding not only zoom, but also doing it in an almost lossless way because of over sampling. This has, so far, been the territory of Nokia's high-end PureView handsets only. With iOS 7, iPhone 5, 5c, and 5s can play catch-up with Nokia's camera phone models, at least in this respect. 

Well, I have some good news for you. Zooming in iOS 7 DOES work when properly(!) programmed. The bad news: the built-in, stock Camera app still doesn't properly support this feature (assuming you only zoom moderately and not over the lossless limit), not even as late as iOS 7.0.2, more than two weeks after the first official, final iOS 7 release. In addition, the most popular third-party apps with support for shooting video, FiLMiC Pro and KitCam, aren't updated with the new iOS 7 features and don't deliver quality image when zoomed in, either.

With this article, I hope I can convince Apple to fix this bug in their stock Camera app and developers to implement proper zoom functionality in their apps. To help the developers with the this, I provide the full sources of a proof-of-concept, simple-but-working video recorder recording proper, lossless footage.

Why do you need to know all this?

Recently, I traveled to a conference with only my iPhone 5 (and, just to be on the safe side, a tripod.) Once there, I decided to record the entire conference on video. Knowing the limitations and range of the lossless zoom gave me much more liberty in placing my iPhone. As I could use a zoom up to 1.45x, I could place the phone a bit farther from the speaker, knowing there would be almost no image quality degradation. Without lossless zoom, I would have had to place the camera significantly closer.

This is what 1.45x zoom is capable of, framing-wise, assuming the same camera and subject locations. When not zoomed in at all, the subject only fills part of the screen (and the output frame):

and this is after zooming in to the maximum possible lossless zoom level:

(Both screenshots show the stock Camera app, with its zoom slider visible so that you can also see the current zoom factor.)

As you can see, with all iPhone models starting with the 5, even in the default 1080p there is some way for zooming without sacrificing any quality (or having an optical zoom, of course). It is not as large as with Nokia's 41 Mpixel camera phones, but still better than nothing. In addition, the lossless zoom helps a lot with zoom factors over the 1.4545x limit—as we'll see when comparing the image quality acquirable with 3x zoom using both the new partly lossless and the old fully lossy approach.

How should it work?

If you have read the excellent comparison of the effective resolution at different zoom levels of the Nokia 808 (the first camera phone to have true lossless zoom), you'll know lossless isn't really lossless. Oversampling will always(!) result in better pixel-level detail and less moire and aliasing. The quality deterioration, however, will still be much lower than with traditional, digital zooming—as we'll promptly see.

Let's see how this all works! Let me present you with three ISO 12233 resolution chart (reschart) shots. (The inline images are crops. You'll want to check out the original, full-resolution ones instead. Please consult my earlier articles on how these reschart shots should be evaluated.)

Let's start with 720p video framegrabs as they offer a whopping 2.18x almost-lossless zoom. Now, we'll see the vast quality improvement lossless zoom provides us! (Note that, for an introduction, I've chosen 720p over 1080p as it much more readily presents the vast quality difference between lossless and lossy zoom as it can zoom in up to 2.18x, while the default 1080p mode can only zoom up to 1.4545x losslessly. Later on, I'll also present lossless 1080p zoomed framegrabs.)

Here's a 720p framegrab crop (again, click the thumbnail for the full ISO 12233 chart shot) showing the effective resolution of the recorded footage without zooming in at all:

Note: in these crops, as with my previous, camera-related articles, I used a red line to denote the point after which, towards higher resolutions, the horizontal lines can't reliably be counted any more. (There are nine such lines on the right. If, at a given position, you can still easily count them, then that resolution is still truly rendered by the camera.) For example, in the above crop, it's exactly at 7.2 because 7.2 (720 lines) is the Nyquist threshold or 720p video modes, and all iPhones are capable of recording that much true detail when not zoomed in.

Now, let's take a look at a framegrab from a video shot at the maximum (for 720p) 2.18x lossless zoom:

There is significantly more color moire in the image (which is pretty common with such dynamic oversampling-based zoom methods) but, otherwise, the nine lines on the right can still be easily counted at even 700 lines or resolution. With traditional, lossy digital zoom, the value would be less(!) than half (after all, the zoom factor is more than two) of this.

Now, let's see how the effective resolution drops and the image quality becomes worse when entering the already-lossy zoom range, which, with 720p, is over 2.18x. Take a look at the same resolution chart with 3x zoom:

Ouch! While the zoom factor (0.82) between the previous 2.18 and the current 3 is significantly lower than between the first two (1.18), the difference in image quality is staggering. The resolution dropped a lot; now the nine lines are absolutely uncountable on the right and can only be reliably counted on the left up to around 480 lines of resolution. (See the red annotation, which is at around the 4.8 mark.) That's a HUGE drop in resolution, much more than between the first two images!

We'll soon see the same setup with 1080p (the native recording format of all iPhones starting with the 4S). First, however, let me introduce the test setup, and the zooming app I've written specifically for this article.

Test setup

I've used a well-lit ISO 12233 resolution chart printed in A4, with a specialized video LED reflector. Being very well-lit is important so that the camera can record at base ISO and won't introduce detail smearing because of noise reduction. I've continuously moved the iPhone, fastened to a pro tripod, to a more distant position when zooming in:

When testing third-party apps where it's not possible to zoom right to the end of the lossless zoom range, I started my camera client (see below), set it to either the maximum lossless zoom factor with a single touch on the “1.4x” button, carefully positioned the tripod, and then switched to the to-be-tested app and zoomed with its controls until I got the entire reschart properly framed. This way, I could be absolutely sure I used exactly the same zoom ratios in all the tested apps.

Note that I also checked for the electronic image stabilization (EIS for short) of each app. With FiLMiC Pro, it's disabled by default (and can be enabled); with the other apps (including the stock Camera client), it's always on. With the latter, before I positioned the tripod by moving it closer to the reschart or farther from it, I enabled the EIS switch in my video recorder app so that I could account for the different field of view when EIS is enabled.

My application

As has already been stated, I've written an app specifically for the purposes of this article: to reliably test whether the iOS 7 zoom is indeed lossless, and to switch between zoom levels and the enabled and disabled EIS state as easily as possible.

It's a really simple app (I didn't bother implementing touch-to-focus, GUI rotation, or even exporting to the Camera Roll to keep the code as simple as possible) but can be used for even true video shooting if you need lossless zoom now. It's available as an Xcode project HERE. If you don't know how it should be compiled and deployed on your own iPhone, ask a friend. There are a lot of iOS programmers—you may know one that you can quickly give your iPhone to for a quick installation. Needless to say, it's a source-only distribution, feel free to examine the code.

Here's the main interface:

Let's start with the upper left corner and introduce the controls clockwise.

The “Start” button in the upper left corner (annotated by a green rectangle) starts video recording. While it's recording, the title changes to “Stop.” Tap the latter to stop recording.

The “1x 1.4x 3x Max” control (annotated by a red rectangle) sets the zoom level. It can be done any time, even during recording. 1x doesn't use any zoom;  1.4x uses the maximum available lossless zoom (1.4545454545x for 1080p, 2.18x for 720p etc.); 3x zooms in three-fold and, finally, “Max” zooms in as much as possible. With 1080p, you can zoom up to 108 times (programmers: the value of activeFormat.videoMaxZoomFactor). (Of course, there isn't much point in zooming in THAT much. No wonder the stock Camera client tops up at 3x zoom and third-party App Store apps don't go over 5-6x zoom either.)

At the bottom of the screen, the pink rectangle shows the zoom currently used. In the screenshot above, it shows "1.454545" because I've, tapped the 1.4x button (see above) beforehand to zoom in to the maximal lossless zoom level.

The back rectangle has an “IS:” label and a switch. With the latter, you can quickly enable or disable EIS, even (as with the zoom level) during recording. Finally, the slider at the bottom (annotated by a blue rectangle) dynamically zooms between 1x and 3x, should you need continuous, arbitrary zooming.

Below, I use section numbering to denote a section being a subsection of another one. This means you'll be able to easily skip entire sections. For example, if you aren't interested in the question of “should I always disable Image Stabilization?” just skip the entire section and go straight to Section 2, where I explain what you can expect of currently available camera apps, starting with the stock Camera app.

1. The effects of the image stabilization on the image quality

As has also been explained in my dedicated article, none of the current iOS devices have optical image stabilization (OIS). Electronic  image stabilization (the one they use), on the other hand, use the outermost 10percent of all the pixels for some rudimentary stabilization—as do all electronic image stabilization (EIS) methods on all non-OIS-capable cameras and phones (including flagships like the Samsung GS4).

Should you want to do these comparisons yourself, feel free to use the zoom and IS controls of my app. To switch it back to 720p mode (see section 1.2 below), you'll also need to change the selectedAVCaptureDeviceFormatIdx = 15 assignment to selectedAVCaptureDeviceFormatIdx = 11 in VideoZoomerWithEISSwitchViewController. (11 is the full mode of 720p, while the original 15 is that of 1080p.)

Fortunately, should you need to use EIS (because you're shooting handheld and not on a tripod), it won't have any negative effect on the effectivity of the lossless zoom. Let me prove this. First, a set of 1080p reschart images.

1.1 1080p tests

1, 1x zoom, IS OFF:

2, 1x zoom, IS ON:

As you can see, there is absolutely no difference between the IS-enabled/disabled states in the non-zoomed-in state.

Let's continue with zoomed-in images. Start with the maximal lossless zoom, that is, with 1080p, 1.454545:

3, max. lossless (1.45x) zoom, IS OFF:

4, max. lossless (1.45x) zoom, IS ON:







While the latter image has a little bit more color moire, the resolution is exactly the same as that of the non-IS shot.

Now, let's apply 3x zoom, which is way beyond the non-lossless range (which, again, is  1.454545x in the default 1080p mode).

5, 3x zoom, IS OFF:

 

6, 3x zoom, IS ON:

The non-IS version is a little bit sharper, but don't let this mislead you, it doesn't have better effective resolution, it's just sharpened. That is, you can use both modes equally well. Just don't forget that, if you zoom, you'd better use a tripod for the best results. (Nevertheless, if you already carry a tripod with you, you could also take a dedicated camera or camcorder. Even the worst, cheapest ones would produce better resolution around 3x+ zoom levels, assuming the camera does have optical zoom.)



1.2 720p tests

Now, let's take a look at something only achievable via third-party apps (the stock Camera app only records in 1080p): comparing IS-eabled to IS-disabled framegrabs taken from 720p footage (recorded using my video recorder).

As in the very beginning, I've already provided three IS-disabled 720p framegrabs, I only give you their  IS-enabled counterparts, in exactly the same order. Scroll up so that directly compare them to the IS-disabled versions.

1x zoom:

2.18x (maximal lossless at 720p) zoom:

3x (already lossy) zoom:

While the second two IS-enabled shots are somewhat better & sharper than the IS-disabled counterparts, the difference is still well beyond variable test variance. That is, both modes can work equally.

2. What about the other apps (including the stock Camera app) on iPhones?

So far, I've only provided you with framegrabs right from my dedicated (and very simple) video recorder. In this section, I shed light on how stock and App Store apps record videos.



2.1 The stock Camera app

I have some mixed news. It doesn't seem to properly support the lossless zoom. Let me show you an example of this.

Compare the following stock Camera app shot made at 1.4545x zoom factor (again, the maximum lossless zoom factor) to the above-presented 1080p shots made with my client in bullets 3 and 4 of section “1.1 1080p tests”. 

The difference is obvious: while in the above (inline) crop, my app could still resolve truly 1080p of horizontal resolution. This is why all vertical lines are visible in the lower right corner up to the mark “10”. With the stock Camera app's shot, there are no distinguishable vertical lines to the right of the mark “8”. That is, while my app did deliver full 1080p worth or resolution, the stock Camera app only delivered around 810-820p.

The latter result is still better than that of simple, traditional digital zoom. With digital zoom, the  resolution would maximize out at 1080p / 1.45 = 745p. (Later, we'll see current App Store apps not making use of lossless zoom at all are indeed only capable of true 745p at the 1.4545x zoom.) This all means that the stock Camera client does lossless zoom, but for some reason, it still delivers worse-than-expected results in the lossless zone. But these results are still better than those of apps that haven't been (properly) updated to support the new iOS 7 zooming API.

Note that outside the lossless area (and at 1x zoom) the two apps produce exactly the same resolution. This is a 1x (that is, no-) zoom framegrab of the stock Camera app:

and this is the 3x one:

The latter should be compared to the IS-enabled version of my camera app. Again, the IS-disabled one has somewhat sharper (but not more detailed!) output. The effective resolution of the shots taken with my and the stock Camera app are identical.

By the way, this is the app's GUI when recording at exactly 1.45x zoom factor:

Again, I used my app to properly move the tripod first so that I could be absolutely sure I was setting the slider properly. After all, the Camera app doesn't display the current zoom ratio.

2.1.1 Why does the stock Camera app's delivering sub-par results when compared to the ideal case?

Frankly, I don't know. I've dones tons of tests in order to be absolutely certain that the above-presented results are correct. They are. In my app, I'm doing a simple AVCaptureDevice.videoZoomFactor assignment in control callbacks to change the zoom factor on the fly.

Finally, note that direct Full HD framegrabs while shooting video (see the “still shot” icon during video recording) will behave in exactly the same way as a framegrab grabbed from the video footage during post processing (after shooting), zoomed-image quality wise.

3. Third-party apps

Now that we've realized the stock Camera app is buggy and, when it comes to zooming inside the lossless range, delivers worse-than-expected results, let's take a look at the output of zooming-capable, popular third-party apps: KitCam, FiLMiC Pro and, when shooting in high-framerate modes only, SlowCam.

3.1  KitCam

KitCam (no longer in App Store) supports zooming if you drag your finger at the bottom of the screen. Then, a zoom slider will be displayed, along with a label displaying the zoom level. I've annotated both in the following screenshot:

Unfortunately, the zoom it has is fully lossy. This is perfectly understandable as KitCam was removed from the App Store way before iOS 7 (and the new videoZoomFactor property needed for lossless zooming) became available.

Let's start with the 1.4545 zoom (again, the highest lossless zoom factor available for 1080p shooting):

As you can see, the resolution is somewhat worse than that of the stock Camera client (about 820p) and much worse than that of my app (full 1080p). The vertical lines start to entirely be blurred at around 745p. This means there's absolutely no lossless zooming, as the traditional digital zoom would simple use 1080p / 1.4545 = 745p resolution. (As we'll see, the current version of FiLMiC Pro doesn't make use of  lossless zoom either.)

Now, take a look at the 3x zoom:

Awful, isn't it? MUCH worse than that of either the stock Camera client or my app (the latter two, again, produce exactly the same image quality). The effective resolution is 360p only, which you can easily see from the shot above. This 360p, incidentally, is one-third of 1080p, showing all zooming has been done digitally, not making use of the lossless region.

Finally, as was easy to predict, at 1x zoom, KitCam produces exactly the same, excellent results (true 1080p resolution) as all the other camera apps:

All in all, stay away from KitCam on your lossless zoom-capable (iPhone 5+ with iOS 7) phones if you ever plan to zoom! Even the stock Camera client has better quality (particularly over the lossless limit) and this won't ever be fixed!

3.2 FiLMiC Pro

FiLMiC Pro ($4.99) is also capable of zooming. I've annotated with blue rectangles the two (+ / -) icons you can use for zooming in the following screenshot:

I've tested the current, 3.3 version, released two months before the official release of iOS 7. This also means it cannot use the new zoom API using lossless zoom, which is certainly visible in the following two test shots:

This is the 1.4545x zoom resolution:

And this is that of the 3x zoom:

(Note that THIS flickr set also has EIS-enabled shots of FiLMiC and 24p ones as well (as opposed to the 30p ones above.) Just look for images starting with “filmic,” After the first hyphen, the frame rate is listed (either 30p or 24p), and at the end, after the zoom factor (1.4x or 3x) whether IS was on or off (EIS on or EIS off). I don't elaborate on them separately as both 30p/24p and  EIS on/EIS off delivers the same image quality.

4. What about slow-mo recording?

One of the major selling points of the iPhone 5s is its new Slow Motion mode, also available on the iPhone 5 when running iOS 7 and 5c. (And, I assume, the 4S too running on iOS 7, but I couldn't confirm this myself. Note that the 4S can't do lossless zoom in iOS 7 at all, regardless of the video mode.)

While there is lossless zoom in the 720p60 (iPhone 5 / 5c) / 720p120 (iPhone 5s only) slow-mo too, its range is significantly smaller than that of the regular 720p mode and its 2.18x lossless zoom: 1.08x. That is, it's practically nonexistent. And, as all these iPhones use pixel binning (halving the originally 720p resolution) when shooting in  720p60 / 720p120 mode, the originally already low resolution will become painfully low if you start zooming. You should never do that. If you still want to zoom while shooting slow-mo videos, consider getting a separate point-and-shoot camera with high-speed video recording. For example, even the really cheap Nikon P300 (released in early 2011 to very good reviews) is capable of 120 fps shooting at 640*480 and the effective resolution doesn't decrease if you zoom in.

Nevertheless, should you still zoom in, let me present you with some reschart shots from both my app and SlowCam ($1.99), one of the iOS 7-compliant and pretty cool slow-mo apps (review; pay special attention to my ('Menneisyys') comments under the article):

1, 1x zoom, my app (to be able to shoot 720p60 / 720p120 footage, make sure you change the selectedAVCaptureDeviceFormatIdx = 15 assignment to selectedAVCaptureDeviceFormatIdx = 13 in VideoZoomerWithEISSwitchViewController. (13 is the full mode of 720p60, while the original 15 is that of 1080p.)):

And that of SlowCam:

As you can see, there's no difference: both apps deliver way inferior results because of the pixel binning (meaning, again, 360p effective resolution).

2, 3x mode:

My app:

SlowCam:

No differences here either. The effective resolution is much worse than that of even 360p.

Nevertheless, at least the latter test shows you'll have exactly the same (bad) image quality using third-party, updated-for-iOS 7 slow-mo apps like SlowCam than using my app or waiting for a third-party developer to come out with a solution.

UPDATE (Oct. 7): I've also scrutinized a new photo & video shooter in App Store, ProCam (free), which at the moment is currently available for free and is definitely a must-have at this price point. It allows for changing the video recording resolution, framerate, and, this is very important if you need as wide a field of view as possible without sacrificing resolution and supported by VERY few apps, also allows for disabling the stabilization (read this dedicated article on the advantages of doing so). I've annotated this in the next screenshot:

While the app (current version: 3.1.1) was last updated on in September, it still doesn't make use of the new iOS 7 zoom features, including being lossless. Instead, it always shoots at the default resolution (whatever you set) and downsamples afterwards, while saving the video. This not only adds a lengthy post-processing encoding to the video shooting process, but also, as it's just downsampling the 1080p original, works in exactly the same way as simple digital zooms, meaning inferior image quality.

The app warns users of the post-processing requirements when using the zoom while shooting video:

and here's a screenshot of its post-processing zoomed footage after shooting:

The zoom lever is around the shutter icon; I've annotated it below (red rectangle). The app also displays the current zoom factor. The latter, as the app doesn't make use of the iOS 7 lossless zoom, is of not of much use in setting the maximum possible, but still-lossless, zoom without using third-party calibration like I've set other apps shoot at the lossless threshold calibrating the position of the camera beforehand using my zoomer app.

Unfortunately, the image quality of the zoomed-in footage is equally bad as with other, non-iOS 7-updated apps (KitCam, FiLMiC Pro) and, consequently, is to be avoided on lossless zooming-capable iOS 7 handsets (iPhone 5, 5c, and  5s). Two examples:

1.4545x zoom (lossless threshold at 1080p):

3x zoom (way beyond the lossless threshold at 1080p):

UPDATE (Oct. 9): an update to Better Camcorder has just been released. It's the first camera client in App Store to support the new way of lossless zooming. Please read THIS article for more info.