Encoding h.264 for iOS with main and high profile

June 24, 2013

In this post, the focus is on encoding video for iOS devices with the h.264 main and high profiles. Typically, an video encoded for iOS would target the baseline profile, since the baseline profile makes it possible to run a video on old iPhone 3 devices as well as newer iPhone 4/5 and iPad devices. But, the baseline profile has some drawbacks that a developer should consider.

First, the baseline profile typically generates a larger file than the main or high profile. This is because the main and high profiles include advanced encode time analysis and CABAC compression approaches. If the h.264 videos will be included in the app resources of your iOS app, then every bit of space savings helps. Here are the byte size results from a very simple animation video that will be examined in this post:


36960 HomerSanta_baseline.m4v
34412 HomerSanta_main.m4v
34391 HomerSanta_high.m4v
40386 HomerSanta_high422.m4v
32034 HomerSanta_high444.m4v

The first thing to notice about these results is that the main profile output is smaller than the baseline output. For this example, the difference is only about a 7% savings when using the main profile. For a longer and more complex video the space savings will be larger. There is not much space savings in this example moving from main to high profile, the CABAC compression enabled by the main profile is likely the majority of the file size improvement.

The second thing to notice about the output files is that using the high profile and YUV 4:2:2 pixels results is a larger file as compared to baseline/main/high. This is expected since the baseline/main/high profiles use YUV 4:2:0 pixels that contain less information than 4:2:2 pixels would. See Chroma Subsampling on wikipedia for detailed information about YUV pixel formats. The last interesting thing about the output file sizes is that the high YUV 4:4:4 format actually compresses the best even though there is more information in each pixel than the 4:2:2 and 4:2:0 formats.

While file size is an important factor the actual colors that appear in the output file can be a major issue as well. For certain applications, the exact colors that appear on screen can be very important. A developer can read more about Color Management at wikipedia. The basic issue with colors when encoding h.264 is taking care to ensure that the colors that appear on the iOS device screen closely match the colors the video or graphic artist intended. This topic is complex and difficult and will be ignored for the remainder of this post. In addition, no current iOS hardware seems to support playback of h.264 content with YUV 4:2:2 or 4:4:4 pixels.

Source Video:

A test video consisting of a single image at 640 x 480 was created as a example. The video displays this one animated frame for 2 seconds. This video content was selected because it is animated content with large regions that are either exactly the same color or colors that are very close to each other.


HomerSanta_lossless

The original 640x480 lossless Quicktime movie encoded with the Apple Animation codec can be downloaded here:

The h.264 encoded versions are available here:


While h.264 encoding is very good, the image data is degraded a bit by the encoding process. Edges are blurred. The blurring can be hard to see, so here is a zoomed in animation that shows the lines on homer's hat. The "Lossless" image is the original PNG and the "Baseline" image shows the same region after encoding with the baseline profile.


HomerHat_3x


Encoding h.264 with x264:

These videos were encoded with x264 via ffmpeg. While there are other encoders around, the combination of ffmpeg+x264 is the best available. x264 produces the smallest files with the best quality of those I have tested. In addition, both ffmpeg and x264 are free software. But, it can be very difficult to actually find the correct command line arguments to actually create h.264 videos. The following ffmpeg command lines were used to create the h.264 encoded videos. You must use a recent version of ffmpeg and x264, old versions may not work.


ffmpeg -y -i HomerSanta.mov -c:v libx264
  -pix_fmt yuv420p -preset:v slow
  -profile:v baseline -tune:v animation -crf 23
  HomerSanta_baseline.m4v

ffmpeg -y -i HomerSanta.mov -c:v libx264
  -pix_fmt yuv420p -preset:v slow
  -profile:v main -tune:v animation -crf 23
  HomerSanta_main.m4v

ffmpeg -y -i HomerSanta.mov -c:v libx264
  -pix_fmt yuv420p -preset:v slow
  -profile:v high -tune:v animation -crf 23
  HomerSanta_high.m4v

ffmpeg -y -i HomerSanta.mov -c:v libx264
  -pix_fmt yuv422p -preset:v slow
  -profile:v high422 -tune:v animation -crf 23
  HomerSanta_high422.m4v

ffmpeg -y -i HomerSanta.mov -c:v libx264
  -pix_fmt yuv444p -preset:v slow
  -profile:v high444 -tune:v animation -crf 23
  HomerSanta_high444.m4v

iOS hardware playback

The baseline h.264 video will play on old iPhone 3 devices as well as newer iPhone 4/5 models and all iPads.

The main profile video should play on iPhone 4/5 models and on all iPad devices.

The high profile video should play on iPhone 4S/5 models and on iPad 2 and newer models.

The high422 and high444 videos do not currently play on any known iOS hardware. Some applications that have strict color requirements could benefit from support for these h.264 profiles, but until iOS hardware supports decoding these h.264 formats it is a mute point. For apps with specific color needs, lossless video support from a library like AVAnimator is suggested.

Given that the main profile provides a file size advantage and works on all iPhone and iPad devices except for the very old iPhone 3 and 3G models, a developer should consider using the main profile for video content that will be embedded inside an iPhone app.

Content for an iPad only app can make use of the main profile instead of the baseline profile. If the high profile offers an advantage in terms of video quality or file size then it could be considered for a new iPad app since only the iPad 1 would be excluded when encoding to the high profile.