iPhone Life magazine

Don't forget: the new YouTube client supports Closed Captions at last! (+ some Java / iOS / YouTube API programming)

With Google's own YouTube client, the biggest problem has always been the lack of Closed Caption (CC; a.k.a. subtitle) support. It was in no way possible to display subtitles in a video, not even in the latest iOS version (still) having the client built-in. Not so with the recently-released, updated, iPad- and widescreen-friendly, free(!!) YouTube client (AppStore link), which shows a CC icon, making it possible to activate CC's, in Landscape orientation on all models and Portrait orientation on iPads:

(iPad, Portrait, CC icon annotated; as with all the other images in this article, click for a much larger and better-quality version!)

(after tapping the icon, you can select the language you'd like to see the subs in)

(iPhone 5, Landscape; CC icon annotated)

(iPhone 5, Portrait; no CC button; however, CC's you select / activate in Landscape mode are rendered here too, as you can see in the screenshot showing "Es un...")

For example, THIS demo video has CC tracks in several languages. Desktop browsers render them just fine but, alas, not the (past, stock) YouTube clients. An example screenshot from the stock, built-in iPad client of iOS 5.1.1 (note the lack of any "CC" icon!):




Fortunately, in iOS versions prior to 6.x, the new client can co-exist with the built-in one! That is, if you stick with 5.1.1 (because your device can't be upgraded to iOS 6 or want to keep jailbreak), you can still install the new YouTube client and enjoy, among other things, subtitles.

Upon tapping a YouTube link, the old, stock, non-CC-capable version will be invoked; however, if you just copy-paste the full YouTube URL of the video to the “Search” field in the new client (after manually starting it), it'll find it just fine. This (pasting the above URL to the field) is shown in the following screenshot:



With CC-enabled videos like this one and with the new client, on the iPad, the “CC” icon is even visible in Portrait mode, not only in Landscape, as has already been shown in the first screenshot above.

Some related programming (strictly for advanced users / geeks / programmers!)

Before Google's own client, I had to code my YouTube client of my own, which, first, downloaded the list of subtitles from the server and, then, started rendering them on top of a UIView. The latter contained a UIWebView, with an object tag in the HTML body   – as is also recommended in Google's own “how-to” iOS programming guide.

Unfortunately, doing this wasn't as straightforward as doing the same with local, iOS-native files. After all, if you use hardware acceleration, you can just use currentPlaybackTime for handling user scrubbing, MPMoviePlaybackState for catching pausing / resuming etc. situations to stop / resume subtitle rendering in an overlay. Not so with UIWebView.

What is even worse, this all has become useless in iOS 6+, as is also mentioned both in the last-but-one user comment (wiibart, September 24, 2012 2:09 PM) under the above-linked official Google article and THIS  StackOverflow discussion. That is, you will no longer want to use this code. (I wonder if Google does come up with an updated, iOS 6-friendly recommendation or at least posts a quick update in the article telling people not to use the code any more.)

Nevertheless, getting and displaying the subtitles was really easy, even without using Google's own, official API (see THIS; note that the old, non-JSON-based one, Gdata, is outdated) but relying on direct XML parsing. For example, the conceptually very simple Google2SRT project does the latter. Should you want to know how the actual subtitle download works, it's as follows. The main class containing almost all the interesting code is Network. It, in addition to the standard packages, only uses org.jdom – that is, nothing Google-specific.

It, in List<NetSubtitle> getSubtitles(String URL) invoked when the “Read” button is clicked in the main GUI, first, creates a URL of the form “http://video.google.com/timedtext?type=list&v={VIDEOID}” (for example, http://video.google.com/timedtext?type=list&v=c8RGPpcenZY) from the user's plain YouTube file entered - this is how you can get the list of available CC languages from Google. Incidentally, this URL also works in a browser; just copy it to the address bar of any browser to see what it returns with.

Then, the actual network communication takes place in readListURL (the parameter of which is the above-built link), which returns a standard org.jdom.Document object, already containing Google's answer. From that DOM object, it's getListSubs()  that creates a java.util.List of NetSubtitle objects. (NetSubtitle is a simple container for the data listed in the table in the GUI; there's no fancy code there.)

When, after selecting the language(s) to download the SRT's in, the user clicks “Go!” (“jbutConvertir” in GUI.java), jbutConvertirActionPerformed() is called back. It's there that the real networking code is:

- a URL is built up with the selected language(s). This is only slightly different from the previously constructed URL; now, instead of just listing the available languages, we get the full subtitle of a specific one. This is why there's a new “lang” attribute in the URL and the “type” one has been changed from “list” to “track”. An example of getting the English track of the above video is http://video.google.com/timedtext?type=track&name=&lang=en&v=c8RGPpcenZY. Give it a try by just clicking it – your browser will nicely render the returned document. (You can also test the URL with other YouTube videos with CC. Remember to change the value of the “v” attribute – and also the “lang” attribute if needed (non-English subtitle).)

- Converter.run() is called directly, which just invokes iniciar() in the same class. (Note that the code doesn't use threading. By making Converter implement Runnable and start()ing an instance of it as a real thread, there would be no possible GUI lockdowns. You should always decouple probably long-running code from the GUI thread in both Java and OS X / iOS programming.) The latter just creates a well-formed, standard SRT file based on the timestamps and titles in the input XML document.

Rendering on iOS

Rendering a timed subtitle is also very easy. Let me present you my full SRT parser code, now, in true Objective-C, using plain C arrays and quick searching for as good performance as possible:

//
//  WBAOverlayViewAppDelegate.m
//  WBAOverlayView
//
//  Created by WR on 11/22/10.
//

#import "WBAOverlayViewAppDelegate.h"

//@interface SubtitleDataRecord : NSObject {}
//@property (nonatomic, retain) NSString* subtitleText;
//@property (nonatomic, retain) NSDate* date;
//@end
//@implementation SubtitleDataRecord
//@synthesize subtitleText, date;
//@end;

@implementation WBAOverlayViewAppDelegate

@synthesize window;

float upscaleFactor = 3.0/2;
float downscaleFactor = 2.0/3;
UIScreen* screen;
#define kVerticalSizeOfLabel 70
#define kHorizontalSizeOfLabel 1024
MPMoviePlayerController* mp;
UILabel* lab;

NSTimer* scrubHandlerTimer;
//NSMutableArray* startingTimeStamps;
//NSMutableDictionary* recordDataDict;

#define kCArraysize 10000
double startingTimeStampsCArray[kCArraysize];
double endingTimeStampsCArray[kCArraysize];
NSString* textCArray[kCArraysize];
int kRealNumberOfElementsInArrays;

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {    
    
    [self.window makeKeyAndVisible];
    // [[UIApplication sharedApplication] setStatusBarHidden:YES animated:NO];
    
    screen = [UIScreen mainScreen];
    if (screen.bounds.size.height == 1024)
    {
        upscaleFactor = 4.0/3;
        downscaleFactor = 3.0/4;  
    }
    
    NSString *videoFilePath = [[NSBundle mainBundle] pathForResource:@"Movie" ofType:@"m4v"];
    mp = [[MPMoviePlayerController alloc]
                                   initWithContentURL:[NSURL fileURLWithPath:videoFilePath]];
    if ([mp respondsToSelector:@selector(view)])
    {
        [self.window addSubview:[mp view]];
        [mp setFullscreen:YES animated:YES]; // must start in full screen over 3.2 so that the text remains visible all the time!        
    }
    [mp play];

    UIWindow *moviePlayerWindow = [[UIApplication sharedApplication] keyWindow];
    if (moviePlayerWindow==nil) moviePlayerWindow = window; // 3.2+
    UIScreen* screen = [UIScreen mainScreen];
    lab = [[UILabel alloc] initWithFrame:CGRectMake(0, screen.bounds.size.height-kVerticalSizeOfLabel, screen.bounds.size.width - 20, kVerticalSizeOfLabel)];
    lab.numberOfLines = 2;
    lab.alpha = 0.5;
    [moviePlayerWindow addSubview:lab];
    
    [[UIDevice currentDevice] beginGeneratingDeviceOrientationNotifications]; // needed for pre-3.2 OS versions - not any more in 4.x

    window.backgroundColor = [UIColor blackColor]; // to avoid white background in pre-3.2 Portrait modes
    SEL mySelector = @selector(deviceRotated:);
    if (![mp respondsToSelector:@selector(view)])
    {
        mySelector = @selector(deviceRotatedPre32:);
        lab.transform =CGAffineTransformRotate(lab.transform, M_PI/2);
        lab.frame = CGRectMake(0, 0, kVerticalSizeOfLabel, kHorizontalSizeOfLabel); // 90 degree default ls left rotation: the upper left point becomes the lower left one
    }
    [[NSNotificationCenter defaultCenter]
         addObserver:self
         selector:mySelector
         name:UIDeviceOrientationDidChangeNotification
         object:[UIDevice currentDevice]];
    for (int i=1; i<10000; i++)
    {
//        NSDate *nextStartTime = [NSDate dateWithTimeIntervalSince1970:1.0*i/10.0];
//        NSDate *nextStopTime =  [NSDate dateWithTimeIntervalSince1970:1.0*i/10.0+0.05];
        NSDate *nextStartTime = [NSDate dateWithTimeIntervalSince1970:10.0*i];
        NSDate *nextStopTime =  [NSDate dateWithTimeIntervalSince1970:10.0*i+9];
        
        startingTimeStampsCArray[i]=[nextStartTime timeIntervalSince1970];
        endingTimeStampsCArray[i]=[nextStopTime timeIntervalSince1970];
        NSString* textRow1 = [[NSString alloc] initWithFormat:@"%i", i];
        textCArray[i]=textRow1;
    }
        kRealNumberOfElementsInArrays = 10000;
        scrubHandlerTimer = [[NSTimer scheduledTimerWithTimeInterval:5
                                                                 target:self
                                                               selector:@selector(scrubHandler)
                                                               userInfo:nil
                                                        repeats:YES] retain];
        return YES;

}
- (void) scrubHandler
{
    double currentPlaybackTime = mp.currentPlaybackTime;
    if (currentPlaybackTime<=0) return;
    int minIndex = 0, maxIndex = kRealNumberOfElementsInArrays, midIndex;
    while (YES) {
        midIndex = (minIndex+maxIndex)/2;
        if (currentPlaybackTime < startingTimeStampsCArray[midIndex])
        {
            if (midIndex > 0 && currentPlaybackTime > startingTimeStampsCArray[midIndex-1]) {  
                    // outer if: if currentPlaybackTime is smaller than startingTimeStampsCArray[midIndex] but
                    // larger than startingTimeStampsCArray[midIndex-1], we display the latter (midIndex-1),
                    // hence the  midIndex-- that follows, along with the break.
                midIndex--;
                break;
            }  
            else
                // currentPlaybackTime is smaller than startingTimeStampsCArray[midIndex] and isn't bigger
                // than the previous array element: we need to search elements with smaller indices. (ALL larger indices will be
                // larger than currentPlaybackTime; this is why we abandon them all)
                maxIndex = midIndex-1;
        }
        else // currentPlaybackTime is larger than startingTimeStampsCArray[midIndex] - we need to go on examining *larger* indices to
             // find out the midIndex where the midIndex-1 is larger than currentPlaybackTime, but midIndex is still smaller
             // (see the first double if's)
                minIndex = midIndex+1;                        
    }
    if (endingTimeStampsCArray[midIndex] > currentPlaybackTime)
        lab.text = textCArray[midIndex];
    else
        lab.text = @"";
}
- (void)deviceRotated: (id) sender{
    UIDeviceOrientation orientation = [[UIDevice currentDevice] orientation];
    if (orientation == UIDeviceOrientationPortraitUpsideDown || orientation == UIDeviceOrientationPortrait)
    {
        window.transform =CGAffineTransformRotate(CGAffineTransformIdentity,
                                                  (orientation == UIDeviceOrientationPortraitUpsideDown) ? M_PI : 0);
        lab.transform = CGAffineTransformMakeScale(1, 1);
        lab.frame = CGRectMake( 0, screen.bounds.size.height-kVerticalSizeOfLabel,
                               kHorizontalSizeOfLabel, kVerticalSizeOfLabel);
    }
    else if (orientation == UIDeviceOrientationLandscapeLeft ||  orientation == UIDeviceOrientationLandscapeRight)
    {
        window.transform =CGAffineTransformMakeScale(upscaleFactor, upscaleFactor);        
        window.transform =CGAffineTransformRotate(window.transform,
                                                  (orientation == UIDeviceOrientationLandscapeLeft) ? M_PI/2 : -M_PI/2);        
        lab.transform = CGAffineTransformMakeScale(downscaleFactor, downscaleFactor);
        lab.frame = CGRectMake( 0,
                               screen.bounds.size.height-kVerticalSizeOfLabel
                               -((screen.bounds.size.height == 1024) ? 60:15)
                               - (screen.bounds.size.height-screen.bounds.size.width)/2*upscaleFactor,
                               kHorizontalSizeOfLabel, kVerticalSizeOfLabel);
    }
}
- (void)deviceRotatedPre32: (id) sender{
    UIDeviceOrientation orientation = [[UIDevice currentDevice] orientation];
    UIWindow* keyWindow = [[UIApplication sharedApplication] keyWindow];
    if (orientation == UIDeviceOrientationPortraitUpsideDown || orientation == UIDeviceOrientationPortrait)
    {
        CGAffineTransform transform =CGAffineTransformScale(CGAffineTransformIdentity, downscaleFactor, downscaleFactor);        
        keyWindow.transform =CGAffineTransformRotate(transform,
                                                     (orientation == UIDeviceOrientationPortraitUpsideDown) ? M_PI/2 : -M_PI/2);

        lab.frame = CGRectMake(-240+kVerticalSizeOfLabel, 0, kVerticalSizeOfLabel, kHorizontalSizeOfLabel);      
    }
    else if (orientation == UIDeviceOrientationLandscapeLeft ||  orientation == UIDeviceOrientationLandscapeRight)
    {
        keyWindow.transform =CGAffineTransformRotate(CGAffineTransformIdentity,
                            (orientation == UIDeviceOrientationLandscapeLeft) ? 0 : M_PI);
        lab.frame = CGRectMake(0, 0, kVerticalSizeOfLabel, kHorizontalSizeOfLabel);
    }
}
[...] // standard generated code here
@end



 

Want to master your iPhone and iPad? Sign up here to get our tip of the day delivered right to your inbox.
Topics:
Email icon
Want more? Get our weekly newsletter:

Werner Ruotsalainen is an iOS and Java programming lecturer who is well-versed in programming, hacking, operating systems, and programming languages. Werner tries to generate unique articles on subjects not widely discussed. Some of his articles are highly technical and are intended for other programmers and coders.

Werner also is interested in photography and videography. He is a frequent contributor to not only mobile and computing publications, but also photo and video forums. He loves swimming, skiing, going to the gym, and using his iPads. English is one of several languages he speaks.