By Werner Ruotsalainen on Sun, 07/10/2011
I've long been working on my forthcoming full(!) tutorial and comparison of everything video playback on iDevices. Now that I'm also writing a full comparison of all iDevice file accessor and media synchronizing tools, I've also very thoroughly tested the video (among other things, MPEG2 TS) conversion of the, otherwise, excellent and highly recommended SyncPod (by iSkysoft) tool. This – and the recent, great jailbreaking and XBMC releases – made me publish a separate, dedicated article. This all means that, as usual, my article contains tons of never-before-published information on generic video conversion, cutting etc. related information. Feel free to ask (in the comment section) if you'd like a more thorough explanation on any of the discussed (sub)subjects!
TS files aren't strangers for people that have some kind of a digital TV receiver card / dongle / unit or are a subscriber to online services (like TVKaista in Finland) offering recorded TV sessions with direct, non-converted stream saving capabilities. It's the standard format for transmission and storage of audio, video, and data, and is used in broadcast systems such as DVB and ATSC – that is, almost everywhere in the world where TV broadcasting has moved to digital.
Signalling the dynamic changes to the receiver
Among other things, TS files have support for dynamic metadata changes. For example, imagine an old, 4:3 TV program. When it's broadcast, it's best to notify the receivers to switch to 4:3 with the best possible way (letterboxing on the sides etc.) and not to horizontally stretch the image. When the program ends and the (now) standard 16:9 broadcast resumes, receivers need to be notified of starting to (again) stretch the image to fill up the entire horizontal screen estate. Similar signals notify the receivers of new audio tracks becoming available or becoming unavailable. An example: in a direct broadcast from a sports event, an additional Swedish in addition to the standard Finnish (you can also think of a DVD / Blue-Ray with more than one sound track / language). The same stands for subtitles, which can also dynamically change: generally, at least in Finland, some programmes have full subtitles (even Finnish-language conversation is subtitled in Finnish) in addition to the standard “only the translation of speech in other languages are subtitled” approach. In this article, I elaborate the conversion / playback problems of all of these questions. Note that these problems are not specifically Finland-related – I've used samples from Finland because I have a gigantic archive of Finnish TV broadcasts and, therefore it was comparably easy to search real-world examples for each of the discussed problems. This would have been much more problematic with other countries' TV broadcasts as they are, generally, not (freely) available over the Net in their original format (just, if at all, a dumbed-down YouTube version or, in a better case, MKV rip).
All this is signaled dynamically at the point when the new stream becomes available or there is some other type of change and, unless your player can read the changes, may render playback pretty much useless.
Unfortunately, on iOS, even some of the players that can play back TS files without transcoding are “dumb” and can't really track / changes. So is, unfortunately, the SyncPod app I've mentioned in the introduction. Just take a look at THIS (upscaled HD) or THIS (non-HD for devices prior to 4th generation small-screen ones ) movie clip and compare it to the original available HERE. (If you can't play back the latter file, download VLC and use it for playback.) See the difference? Yes, all the three videos start with 16:9 aspect ratio. Then, the TS version switches to 4:3 at around 0:03; the two, converted versions, on the other hand, don't. All the playback from there will remain assuming the source was all the time 16:9, resulting in a really ugly and annoying stretching.
BTW, it's not the only problem of SyncPod (or, for that matter, a lot of other video converter tools). Check out THIS or THIS short video. Both have two audio channels; the first (an excerpt from the excellent Finnish movie Mother of Mine), in addition to a Finnish + Swedish one, has a track for the visually impaired, where the originally Swedish language text is repeated in Finnish. The second (an ice hockey match from the recent World Championship) has a completely separate Finnish and Swedish channel. Now, let's take a look at the output of SyncPod's converter! Yes, it only has one sound track. And no subtitle tracks, I should add – the original had two: either Finnish text for Swedish speech or Finnish text for both Finnish and Swedish text. I've deliberately selected a scene of the movie which demonstrates the use of both.
Now, what you can do to avoid major problems like this?
First, you can just avoid using iPod (iPhone / iPod touch in pre-iOS5) / Videos (iPad and all devices under iOS5) and use a third-party media player that is able to handle multiple audio and/or subtitle tracks and/or dynamic TS changing. This has some of the following advantages:
- in many cases, free (no [pretty expensive!] purchase of SyncPod)
- no quality decrease because of the recompression
- no size increase when deploying on a newer-generation (hi-res small-screen device or any iPads) device (SyncPod upsizes everything to 720p, even low-res original videos, when it deploys to new hardware)
- no time / effort needed for recompression
- no need to add the title to the iTunes library (taking up a lot of storage space on your desktop)
- of course, all the metadata and audio tracks remain accessible and selectable
Some of the cons are as follows:
- third-party and TS-compliant apps are generally more complicated to use. For example, selecting a subtitle track is a real pain in the back in GoodPlayer, one of the most recommended TS players and the Cydia-only app, XBMC (more on it later), has even worse usability and is, in general, very hard to use.
- they may have much higher battery usage than the stock, built-in video player (I will have to check this)
- the free XBMC requires jailbreak (not an issue now that the current iOS version, 4.3.3, is jailbreakable on all models)
I'll elaborate on TS-capable, third-party players in my forthcoming article. Nevertheless, I'll devote bit more time below to XBMC now that it received major upgrades and it's now become really easy to jailbreak.
What if you already have tons of converted videos and would like to stick with them because, for example, they take far less storage than the original TS files (if and only if they have originally been converted for low-resolution, small-screen devices) or you've long ago deleted the source TS files. Then, to fix the problem of stretched image, you'll need a player that allows you to force a new aspect ratio. Unfortunately, very few of them do.
Playing back videos encoded with the wrong aspect ratio
AVPlayerHD by EPLAYWORKS.Co.Ltd
This is an excellent $3 (iPhone version sold separately, also for $3) player with some really unique features (e.g., time stretching) not available in any else iOS player, not even in the otherwise excellent XBMC. It also supports aspect ratio forcing. Example shots (as usual, click the articles for the original, true PNG – meaning no JPG compression artifacts! - screenshots!):
(this is how it should look like; got here by tapping the two-window icon in the upper right corner some two or three times)
The excellent, free but, unfortunately, Cydia-only XBMC 10.0-9:
Either manually set Pixel ratio to 0.75 (4:3):
Or, even easier (no need to struggle with the mouse-optimized interface), by setting the predefined menu item View Mode to Stretch 4:3:
Note that there is another third-party player, BUZZ Player HD by BUGUN Software, which also allows overriding the aspect ratio. However, in this particular case (a “too wide”, horizontally overstretched video needs to be “compressed” horizontally) it doesn't help. Let me show you how it renders the three aspect ratio overrides (1:1, 4:3, 16:9) it supports:
As you can see, none of them has resulted in the horizontal compression of the image. The 16:9 aspect ratio setting, for example, further stretched it horizontally (by compressing it vertically).
Pre-cutting the TS videos
If you don't mind recompressing your videos (because you've already purchased SyncPod and want to stick with it) but want to avoid stretching problems because of the dynamic switching, you'll want to extract the parts you would like to recompress with ProjectX (additional info). Below, I thoroughly explain how it should be used.
Note that I've tested almost all the TS-capable video converters / editors / cutters out there. The most recent versions of the following:
Avidemux (Mac), HDTVtoMPEG2 (home (additional remarks HERE and HERE; Win only; just drag the slider at the bottom and click “Exclude” to exclude the remaining part. Then, click “Process” at the top right to create the file.), TSSniper (additional info; Win only; this one didn't work, just displayed “DirectShow problem – Failure while trying to render filtergraph”, just like with some other people), DVBCut (another page; use e.g. THIS (another commercial app HERE) to decompress; Win only); VideoRedo TVSuite (Win only; commercial). None of them supported subtitles, some even didn't correctly copy / create the audio track language information (the exceptions being HDTVtoMPEG2 and VideoRedo).
Back to ProjectX. No, it's not the iOS reincarnation of the excellent Amiga game Project-X, but an (also) excellent video file manipulator tool. It's very easy to use: download the file and double-click ProjectX.jar. After accepting the agreement, tap “Cut Control” at the bottom (see red oval in the screenshot below) and load the video via File / Add. (Here, I explain how THIS already-introduced video is freed of the first three seconds still using the 16:9 aspect ratio.)
Now, drag the slider (see red oval) to a position where already the final aspect ratio is used (in this case, around 0:03):
Click the “Add cut point” icon (red oval below) once; the rest of the timeline remains green (while the previous becomes red), showing the next export operation will only save the video starting at the current point:
Now, click the “Prepare” button in the top left (just below QuickStart) and, in the new window that opens, check in the “1:1 binary copy” checkbox (red oval below):
Then, just click the Go / Pause button in the upper left corner:
Congratulations, the new file is in the same directory as the source file! Now, you can freely compress / transfer it via SyncPod (or any “dumb” converter): as there's no aspect ratio change in the file any more, the 4:3 video won't be stretched to 16:9.
Note that by googling for “edit ts files”, you'll find a lot of dedicated threads; as many of these are available right on the first two result pages, I won't link to them separately.
I've already mentioned the Cydia-only XBMC is a highly recommended media player. It has some features no other iOS app is capable of; for example, the ability to navigate and play back DVD menus(!). Now, with the recent release of 10.0-9, it has become “highly recommended” in my notebook. (Previously, I've thoroughly tested version 10.0-7 and didn't particularly like it: it, for example, wasn't able to follow the dynamic configuration changes in TS files and so on. Now these are all fixed.)
To install it, first, you'll need to jailbreak your device. It's, now, really easy (before Apple releases a new iOS version – hurry up!) - just navigate HERE from your mobile device and do if you just “purchased” the app(-alike) from AppStore. Then, when Cydia is installed, do what is explained HERE.
I've made some tests to compare its new deinterlacing algorithm to that of 10.0-7. Back then, it used the most simple “blending” algorithm (see THIS and THIS), which halves both the vertical and the temporal resolution. The latter isn't that big a problem on iOS as the hardware won't play videos with 60 fps at all, unlike the desktop Mac. (I've checked this with my special 60 fps counter video [original video file HERE; warning, it's 120 Mbyte in size!; dedicated article [programmatic creation of the file] HERE), along with my Nikon P300 camera capable of recording video at a whopping 120 fps.) The former, however, is, and it's can be clearly seen in the following (again, 10.0-7-time) screenshot (I recommend downloading the full version, along with a non-bleded one, and comparing it directly to a better one):
Here's how the same screen looks without deinterlacing:
Finally, this is the output of the 10.0-9 deinterlacing:
Unfortunately, the new deinterlacing algorithm is taxing the hardware far too much – even that of the iPad2. Even on it the video is played back much slower than the audio, let alone older, slower iDevices.
This can easily be spotted in the following iPad1 video (playing back my well-known 60 fps counter and shooting the entire process with my 120 fps Nikon P300; note that the video I shot is played back at 30 fps, that is, four times slower than it was shot):
(iPad1 10.0-9 deinterlacing speed benchmark; direct link; non-deinterlaced playback from around 0:16; deinterlaced from 1:58)
To show the real effects of this (instead of a “simple” counter benchmark), here's a shot of THIS already well-known test video on the (considerably faster) iPad 2:
(iPad2 10.0-9 deinterlacing speed benchmark; direct link; non-deinterlaced playback from around 3:13; non-deinterlaced from 4:57. The first half of the video shows the [negligable] speed difference bilinear (it's right from the start) vs. nearest neighbour (starting at around 1:40) stretching)
All in all, don't even think of using runtime deinterlacing in the latest XBMC version (as opposed to the previous one)! It'll make video playback impossible – even on the superfast, dual core iPad 2! For comparison, here's the 10.0-7 benchmark:
(original HERE; as you can see, it's WAY faster than 10.0-9)
Bilinear vs. nearest neighbour filtering
There's some other feature only XBMC has: you can set whether it should use bilinear or nearest neighbour filtering when stretching images.
Theoretically, the latter has worse image quality but better performance. However, there doesn't seem to be much difference when stretching PAL SD content (720*576) to XGA (1024*768) – neither visual quality, nor performance-wise; not even on the slower iPad1.
Click the followning thumbnails to get the full framegrabs; pay special attention to the outlines of letters (e.g., “N” or “A”). As you can see, bilinear filtering is of a little bit better quality:
The following iPad1 test video (using my 60 fps counter + 120 fps camera combo) show there is no visible performance difference between the two solutions (note that I haven't tested CPU / battery usage!):