You might have heard of 2K video. It delivers (horizontally – but not vertically, at least not in the widescreen mode!) a little bit more resolution (2048) than Full HD (1920). As 2K is getting increasingly popular among movie buffs and a lot of filmmakers are actively discussing (see for example THIS thread) playing back 2K on the iPad 3, I decided to chime in the discussion so that the voice of hardcore iOS programmers, who also knows the intricacies of video formats pretty well (after all, my previous H.264 bible, back in 2007, was frontpaged on several leading Symbian / Windows Mobile tech sites), can also be heard.
First and foremost, let me present you the following test videos, all created from THIS (2:1 aspect ratio) extra-resolution PNG resolution chart. Note that, for my previous, Full HD article, I used THIS 16:9 image an the input. For today's tests, I needed to remove the topmost and bottommost parts of the originally 16:9 chart because both 2K and 4K videos use the 2:1 aspect ratio when exported from Final Cut Pro X I used for creating the shot (BTW, see Chapter 5 "Addendum – how did I create the test videos?" for more info on creating and exporting 2k videos from FCPX). Importing a 16:9 shot into a 2K (or 4K) video would have resulted in a thick black bar on both sides of the image, making it very hard to compare to yesterday's Full HD playback results.
A 2K (2048*1080 (here, vertically, because of the above-mentioned restriction of FCPX, 1024, but it has no effect on the results) pixels) H.264 AVC (part 10) version of the resolution chart
The above video re-compressed using the standard (ASP) MPEG-4 encoder
Finally, an anamorphic H.264 (level 10) video with true resolution of 1920 (so that the hardware decoder plays it back), stretched to 2048. (The stretching parameters I used can be seen in THIS screenshot. As there was no point in stretching vertically, I haven't increased the vertical size to 1024*2048/1920 = 1092. This way, the chart occupied exactly the same screen area in all cases.)
I've thoroughly tested the playback of all of these files from a third-party player (as iTunes wouldn't have synchronized these files to the stock Videos.) In the following three chapters, I elaborate on them all.
1. 2K H.264 AVC (part 10) playback (first video)
The H.264 AVC (part 10) format is by far the most widely used, very effective video format; no matter most cameras as of today use it by default; so does Blue-Ray discs, (more advanced) HD DVB broadcasters etc. Also, the movie rip scene exclusively uses this format because of its very good storage requirements and with current hardware, acceptable CPU / GPU usage.
Unfortunately, the iPad 3 isn't able to play back any H.264 AVC (part 10) videos larger than Full HD, even if vertically it's considerably smaller (1024 vs. 1080 in this case). Therefore, if you try to play it back, the player will just show a black screen and, practically, crashes (doesn't show the controls if you tap the screen and the whole app needs to be killed).
2. The same 2K video transcoded with the (ASP) MPEG-4 encoder
The plain MPEG-4 encoder is rarely used today because it delivers far worse image quality than the just-discussed at the H.264 AVC (part 10) codec at the same bitrate. (Incidentally, you can also check this out yourself: in the ASP version of the resolution chart I've provided, there is visible “blocking” around contrasty edges. This is a very common problem with lower-quality codecs like MPEG-4 ASP.)
However, there's a definite advantage of ASP encoding: the iPad 3 happily plays back not only true 2K videos encoded with ASP, but also videos with considerably higher vertical resolution: this is the subject of the next section.
2.1 Further ASP tips, demo videos
You WILL want to read THIS and THIS threads in their entirety. They (see for example THIS and THIS post) contain a LOT of benchmark reports of different vertical resolutions and quality settings. I, myself, do not consider ASP a viable encoder (whenever possible, use H.264 AVC for vastly superior image quality!) so I don't elaborate on these results.
2.2 Encoding
You can easily transcode your videos to plain MPEG4 (ASP): use Handbrake, select the Regular / High Profile in the Presets tab and change the (by default, H.264) Video Codec to MPEG-4.
3. Anamorphic video
Anamorphic videos (Wiki; please read it if you don't know how it works so that you understand why it's able to stretch video even on the iPad), which are also supported by iOS, have a neat trick: you can give your iPad a, physically, 1920-pixel-wide video file (of arbitrary height), and tell the system to stretch it horizontally, and only horizontally.
What advantages does this have?
1, you can stay with using the, compared to ASP, vastly superior part 10 AVC codec (don't need to use ASP).
2, while the horizontal resolution will suffer exactly as much as with the traditional 1920 > 2048 upsizing (because of stretching), the vertical resolution will remain the same. All this without any kind of any kind of windowed playback. (As long as you tell the system not to stretch it vertically.)
This means this video mode, quality-wise, is halfway between
- my windowed Full HD playback solution not utilizing the entire screen estate (leaving 64 pixel-wide black bars on both sides) and, as there's no stretching in either dimension, delivering by far the best image quality
- the approach Apple takes in its stock Video app (or any third-party app using the Full Screen mode as opposed to a windowed one), which stretches in both dimensions, resulting in considerably worse image quality.
I've created a difference image of the ASP MP4 and the anamorphic 2k video mode. It's HERE and is REALLY-REALLY interesting for tech buffs. (BTW, HERE is a tutorial on how such images can be created and HERE (starting in the middle of the article) are some practical samples of a RED user, Bill Barnes, also showing difference images on the iPad 3.) How you can clearly see there isn't image quality degradation (stretching) vertically, while there is horizontally? Easy. The parts of the image having (nearly) vertical lines placed close to each other are far “whiter” than the parts that sport horizontal, closely-spaced lines. In the following annotated version, I've denoted the vertical lines with red rectangles and the horizontal lines with red ovals.
(Click the link for the full-sized image! Turn up the backlight on your monitor to see the underlying differential picture!)
BTW, this also shows the MPEG-4 conversion isn't exactly the same as using a more advanced video codec (like AVC) – there are a lot of compression artifacts. This is why there are differences everywhere on the image, even with truly horizontal lines. However, these differences are in no way as pronounced as with vertical lines, letters, numbers etc., which, again, are heavily hit by the stretching process.
(Note that I couldn't create a similar comparison between the output of my windowed player and the output of the standard full-screen playback. As the latter stretches in both directions (1920 > 2048 and 1080 > 1152 pixels), both the horizontal and vertical resolution and contrast suffers heavily. However, as the two images are of different sizes (full screen stretched vs. windowed original size), they can't be compared with “simply” applying a difference image in Photoshop. Just downsizing the full screen image to become the same size as the windowed one (the one played back in my player) would have resulted even more resolution loss inherent with any kind of scaling, fully invalidating any meaningful comparison results.)
HOWEVER! The anamorphic mode can only meaningfully be used with 2k (or even higher-resolution) video material, NOT with Full HD. Think of this: you want to make use of the entire width of the screen, while not letting the video to be stretched vertically to keep the vertical resolution, contrast etc. intact (again, as opposed to the standard stretching applying upsizing in both directions). If you, therefore, keep the 1080 (or whatever vertical resolution the original movie had) lines' height while stretching (via explicit anamorphic mode) from 1920 to 2048 pixels, the aspect ratio will no longer be 16:9, but 17,06*9, which results in an already visible distortion. However, if you do try avoiding the distortion by not leaving the vertical size stay at 1080 but also apply stretching to 1152 (=1080*2048/1920), the vertical resolution will be affected because of stretching in exactly the same way as with the standard, automatic stretching of Full HD movies in full screen playback mode.
With 2k videos (or anything over 2k), as the display resolution is truly 2048 pixels (or - again - more), you can just stick with the original vertical size (with 2k videos, by default, 1080) without scarifying vertical resolution (no stretching takes place vertically, only horizontally, but the latter “only” results in detail loss and not in aspect ratio changes) or true 2:1 aspect ratio.
3.1 Setting up anamorphic encoding
Do everything as you would in encoding for full HD resolution. Click the “Picture Settings” icon at the top and select “Custom” in the Anamorphic drop-down list. After this, manually enter “2048” in the “Display width” text input field that will be dynamically shown. Make sure the “Width” (topmost, left) field stays at 1920 every time!
The following shot shows this:
And this is a shot of the main Handbrake window after applying the above settings. Pay particular attention to the “Picture Size” row at the bottom. It states the source video is of size 2k (2048 wide; note that, in this test, I've used an 1024-pixel-high image and not an 1080 pixel one), the target videos' physical (encoded) size (“Output”) is (1920*1024; again, 1920 is needed so that AVC-encoded material can be played back by the iPad) and, finally, the “Anamorphic” section states displaying the content will take a screen area of 2048*1024 – that is, the same as that of the source.
3.2 Summary of anamorphic encoding
Again,
- we only need to use anamorphic encoding because iDevices refuse to play back anything wider than 1920 pixels if they use the high-quality AVC encoder (as opposed to the standard, low(er)-quality MPEG-4 aka ASP, the subject of Chapter 2 above.)
- anamorphic encoding should ONLY be used to encode 2k (or larger) videos and NOT for originally Full HD content (for the reasons outlined previously).
- 2k videos (as opposed to even wider video sources – for example, 4k videos), where the frame width of the video material is exactly the same as that of the iPad, are EXTREMELY well suited for anamorphic encoding, as opposed to transcoding to non-anamorphic Full HD resolution (to 1920x1012 (=1080*1920/2048)). In the latter case, a resizing will take place during the encoding process, heavily reducing (vertical) resolution and (vertical) contrast. Even if you play back that material using my windowed Full HD player (which applies no stretching and, consequently, no additional image quality degradation), the image quality will be considerably worse than with using anamorphic encoding.
4. Hardware upscaling
Speaking of whether there's any advantages / disadvantages of using windowed (non-scaled) Full HD playback, yesterday night, I've run some battery (power consumption) tests to find out how my player behaves. For this, I've used the 1080p version of the Avatar BD rip, which is my standardized test video for power consumption measurement.
The power consumption was roughly the same as playing back the same video at full screen and probably a little bit lower than doing the same over Wi-Fi, playing back the video from an UPnP source: about 17-18% an hour, running at maximum brightness. (My UPnP Wi-Fi measurements have shown about 19% an hour battery charge decrease, which isn't considerably larger than that of local playback.)
That is, the hardware scaler works very effectively and doesn't have much (if any) effect on battery life. This also means you will NOT want to use explicit anamorphic video stretching for (only; again, for 2k+, anamorphic video results in keeping the true vertical resolution intact while using AVC encoding and not the low-quality ASP one) Full HD videos (or other, similar tricks) to save battery life.
5. Addendum – how did I create the test videos? + Tips for disabling the Ken Burns effect
Creating videos over the Full HD resolution isn't a trivial task as most (consumer-grade; think of e.g. iMovie) video applications don't support exporting in higher resolutions (not even in 2K). You'll definitely need a prosumer or even pro app for exporting.
For example, in Final Cut Pro X 10.0.3, create a new project under File > New Project... (leave everything on their default). Drop the full-res 2:1 image in the pane occupying the entire half bottom of the screen and set 2K when it states it can't recognize the format. Then, you can export the video file right away (Share > Export Media) – just remember to set “H.264” as the video codec.
With more consumer-grade editors, you can only export movies up to Full HD resolution. In some of them, you can't disable the Ken Burns effect at all, making these apps useless for our test purposes (that is, creating a test movie out of a high-resolution chart image). An example of these is the iOS version of iMovie. The Mac OS X version of iMovie already supports disabling the Ken Burns effect: after having created a new project; go to File / Project Properties, select “Fit in Frame” (or Crop) in the "Initial Photo Placement" drop-down list instead of the default Ken Burns. Then, drop the image in the project (as you'd do on iOS), finalize it (File / Finalize Project) and, finally, export at 1080p (only) at Share / Export Movie. Also note that QuickTime Player 7, which I generally use for quickly extracting sections of MP4 files (with all the subtitles and closed captions, if any), when creating a MOV from the chart PNG file (after, using Finder, duplicating it several times, so that there are some 400-500 copies of it to cover some 5-10 seconds at least) using File > Open Image Sequence, it exports movies with the codec PNG. The latter is played back by the desktop VLC without problems but not the iDevice.
6. Some other screenshots
In this section, I provide some additional screenshots of all these modes so that you can easily assess the quality of each of them.
First, let's compare the native (non-stretched) ASP (MPEG-4) 2k mode to the anamorphic 1920*1080 mode. The following pair of crop does show the anamorphic source is stretched horizontally (but not vertically!), resulting in decreased contrast and generally considerably worse resolution. In this case, the vertical, long part of the “1” (sorry, I'm not a typographer so I don't know the exact expression for it) is represented by 4 pixel columns in the anamorphic shot (because of the resizing in runtime) and only 3 in the native, non-stretched 2k shot (again, because there isn't resolution- and contrast-lowering stretching):
(anamorphic version)
(2k MPEG-4 ASP)
Full resolution chart screenshots in anamorphic encoding and MPEG-4 ASP-encoded, non-stretched video (encoding source: 2k video). (Click the links!) These were used to generate the (also annotated) difference image in section 3.
Another pair of MPEG-4 ASP vs. anamorphic comparison shots showing all the above:
(MPEG-4 ASP; notice the very ugly compression artifacts, one of the reasons I don't really recommend MPEG-4 ASP)
(anamorphic)
UPDATE (05/27/2012): it's possible to fully disable the Ken Burns effect in the iOS version of iMovie too, as opposed to what I've stated in the original article (I've looked for some kind of a menu item or setting, which are indeed missing). Just tap the picture in the timeline so that its borders get yellow (and the cutting / stretching handles appear). Then, the image in the upper right corner start to display "Start" (in the upper left corner) and "End" (upper right corner), semi-transparent buttons. Tap Start and zoom out entirely; then, just click End and also zoom out entirely. Tap Done (lower right corner) when ready.
UPDATE (06.07/2012): to do the same in the OS X version of iMovie, read THIS.
