H.264/MPEG-4 AVC
From Wikipedia, the free encyclopedia
This article may be too technical for a general audience. Please help improve this article by providing more context and better explanations of technical details to make it more accessible, without removing technical details. |
H.264 is a standard for video compression. It is also known as MPEG-4 Part 10, or MPEG-4 AVC (for Advanced Video Coding). It is one of the latest block-oriented motion-estimation-based codecs developed by the ITU-T Video Coding Experts Group (VCEG) together with the ISO/IEC Moving Picture Experts Group (MPEG) as the product of a partnership effort known as the Joint Video Team (JVT). The ITU-T H.264 standard and the ISO/IEC MPEG-4 Part 10 standard (formally, ISO/IEC 14496-10) are jointly maintained so that they have identical technical content. The final drafting work on the first version of the standard was completed in May 2003.
Contents |
[edit] Overview
The intent of the H.264/AVC project was to create a standard capable of providing good video quality at substantially lower bit rates than previous standards (e.g. half or less the bit rate of MPEG-2, H.263, or MPEG-4 Part 2), without increasing the complexity of design so much that it would be impractical or excessively expensive to implement. An additional goal was to provide enough flexibility to allow the standard to be applied to a wide variety of applications on a wide variety of networks and systems, including low and high bit rates, low and high resolution video, broadcast, DVD storage, RTP/IP packet networks, and ITU-T multimedia telephony systems.
The standardization of the first version of H.264/AVC was completed in May of 2003. The JVT then developed extensions to the original standard that are known as the Fidelity Range Extensions (FRExt). These extensions enable higher quality video coding by supporting increased sample bit depth precision and higher-resolution color information, including sampling structures known as YUV 4:2:2 and YUV 4:4:4. Several other features are also included in the Fidelity Range Extensions project, such as adaptive switching between 4×4 and 8×8 integer transforms, encoder-specified perceptual-based quantization weighting matrices, efficient inter-picture lossless coding, and support of additional color spaces. The design work on the Fidelity Range Extensions was completed in July of 2004, and the drafting work on them was completed in September of 2004.
Further recent extensions of the standard have included adding five new profiles intended primarily for professional applications, adding extended-gamut color space support, defining additional aspect ratio indicators, defining two additional types of "supplemental enhancement information" (post-filter hint and tone mapping), and deprecating one of the prior FRExt profiles that industry feedback indicated should have been designed differently.
Scalable Video Coding as specified in Annex G of H.264/AVC allows the construction of bitstreams that contain sub-bitstreams that conform to H.264/AVC. For temporal bitstream scalability, i.e., the presence of a sub-bitstream with a smaller temporal sampling rate than the bitstream, complete access units are removed from the bitstream when deriving the sub-bitstream. In this case, high-level syntax and inter prediction reference pictures in the bitstream are constructed accordingly. For spatial and quality bitstream scalability, i.e. the presence of a sub-bitstream with lower spatial resolution or quality than the bitstream, NAL units are removed from the bitstream when deriving the sub-bitstream. In this case, inter-layer prediction, i.e., the prediction of the higher spatial resolution or quality signal by data of the lower spatial resolution or quality signal, is typically used for efficient coding. The Scalable Video Coding extension was completed in November 2007.
The H.264 name follows the ITU-T naming convention, where the standard is a member of the H.26x line of VCEG video coding standards; the MPEG-4 AVC name relates to the naming convention in ISO/IEC MPEG, where the standard is part 10 of ISO/IEC 14496, which is the suite of standards known as MPEG-4. The standard was developed jointly in a partnership of VCEG and MPEG, after earlier development work in the ITU-T as a VCEG project called H.26L. It is thus common to refer to the standard with names such as H.264/AVC, AVC/H.264, H.264/MPEG-4 AVC, or MPEG-4/H.264 AVC, to emphasize the common heritage. The name H.26L, referring to its ITU-T history, is less common, but still used. Occasionally, it is also referred to as "the JVT codec", in reference to the Joint Video Team (JVT) organization that developed it. (Such partnership and multiple naming is not uncommon—for example, the video codec standard known as MPEG-2 also arose from the partnership between MPEG and the ITU-T, where MPEG-2 video is known to the ITU-T community as H.262.[1])
[edit] Features
H.264/AVC/MPEG-4 Part 10 contains a number of new features that allow it to compress video much more effectively than older standards and to provide more flexibility for application to a wide variety of network environments. In particular, some such key features include:
- Multi-picture inter-picture prediction including the following features:
- Using previously-encoded pictures as references in a much more flexible way than in past standards, allowing up to 16 reference frames (or 32 reference fields, in the case of interlaced encoding) to be used in some cases. This is in contrast to prior standards, where the limit was typically one; or, in the case of conventional "B pictures", two. This particular feature usually allows modest improvements in bit rate and quality in most scenes. But in certain types of scenes, such as those with repetitive motion or back-and-forth scene cuts or uncovered background areas, it allows a significant reduction in bit rate while maintaining clarity.
- Variable block-size motion compensation (VBSMC) with block sizes as large as 16×16 and as small as 4×4, enabling precise segmentation of moving regions. The supported luma prediction block sizes include 16×16, 16×8, 8×16, 8×8, 8×4, 4×8, and 4×4, many of which can be used together in a single macroblock. Chroma prediction block sizes are correspondingly smaller according to the chroma subsampling in use.
- The ability to use multiple motion vectors per macroblock (one or two per partition) with a maximum of 32 in the case of a B macroblock constructed of 16 4x4 partitions. The motion vectors for each 8×8 or larger partition region can point to different reference pictures.
- The ability to use any macroblock type in B-frames, including I-macroblocks, resulting in much more efficient encoding when using B-frames. This feature was notably left out from MPEG-4 ASP.
- Six-tap filtering for derivation of half-pel luma sample predictions, for sharper subpixel motion-compensation. Quarter-pixel motion is derived by linear interpolation of the halfpel values, to save processing power.
- Quarter-pixel precision for motion compensation, enabling precise description of the displacements of moving areas. For chroma the resolution is typically halved both vertically and horizontally (see 4:2:0) therefore the motion compensation of chroma uses one-eighth chroma pixel grid units.
- Weighted prediction, allowing an encoder to specify the use of a scaling and offset when performing motion compensation, and providing a significant benefit in performance in special cases—such as fade-to-black, fade-in, and cross-fade transitions. This includes implicit weighted prediction for B-frames, and explicit weighted prediction for P-frames.
- Spatial prediction from the edges of neighboring blocks for "intra"coding, rather than the "DC"-only prediction found in MPEG-2 Part 2 and the transform coefficient prediction found in H.263v2 and MPEG-4 Part 2. This includes luma prediction block sizes of 16×16, 8×8, and 4×4 (of which only one type can be used within each macroblock).
- Lossless macroblock coding features including:
- A lossless PCM macroblock representation mode in which video data samples are represented directly, allowing perfect representation of specific regions and allowing a strict limit to be placed on the quantity of coded data for each macroblock.
- An enhanced lossless macroblock representation mode allowing perfect representation of specific regions while ordinarily using substantially fewer bits than the PCM mode.
- Flexible interlaced-scan video coding features, including:
- Macroblock-adaptive frame-field (MBAFF) coding, using a macroblock pair structure for pictures coded as frames, allowing 16×16 macroblocks in field mode (compared with 16×8 half-macroblocks in MPEG-2).[clarify]
- Picture-adaptive frame-field coding (PAFF or PicAFF) allowing a freely-selected mixture of pictures coded as MBAFF frames with pictures coded as individual single fields (half frames) of interlaced video.[clarify]
- New transform design features, including:
- An exact-match integer 4×4 spatial block transform, allowing precise placement of residual signals with little of the "ringing" often found with prior codec designs. This is conceptually similar to the well-known DCT design, but simplified and made to provide exactly-specified decoding.
- An exact-match integer 8×8 spatial block transform, allowing highly correlated regions to be compressed more efficiently than with the 4×4 transform. This is conceptually similar to the well-known DCT design, but simplified and made to provide exactly-specified decoding.
- Adaptive encoder selection between the 4×4 and 8×8 transform block sizes for the integer transform operation.
- A secondary Hadamard transform performed on "DC" coefficients of the primary spatial transform applied to chroma DC coefficients (and also luma in one special case) to obtain even more compression in smooth regions.
- A quantization design including:
- Logarithmic step size control for easier bit rate management by encoders and simplified inverse-quantization scaling.
- Frequency-customized quantization scaling matrices selected by the encoder for perceptual-based quantization optimization.
- An in-loop deblocking filter which helps prevent the blocking artifacts common to other DCT-based image compression techniques.
- An entropy coding design including:
- Context-adaptive binary arithmetic coding (CABAC), an algorithm to losslessly compress syntax elements in the video stream knowing the probabilities of syntax elements in a given context. CABAC compresses data more efficiently than CAVLC but requires considerably more processing to decode.
- Context-adaptive variable-length coding (CAVLC), which is a lower-complexity alternative to CABAC for the coding of quantized transform coefficient values. Although lower complexity than CABAC, CAVLC is more elaborate and more efficient than the methods typically used to code coefficients in other prior designs.
- A common simple and highly structured variable length coding (VLC) technique for many of the syntax elements not coded by CABAC or CAVLC, referred to as Exponential-Golomb coding (or Exp-Golomb).
- Loss resilience features including:
- A network abstraction layer (NAL) definition allowing the same video syntax to be used in many network environments, including features such as sequence parameter sets (SPSs) and picture parameter sets (PPSs) that provide more robustness and flexibility than provided in prior designs.
- Flexible macroblock ordering (FMO), also known as slice groups, and arbitrary slice ordering (ASO), which are techniques for restructuring the ordering of the representation of the fundamental regions (macroblocks) in pictures. Typically considered an error/loss robustness feature, FMO and ASO can also be used for other purposes.
- Data partitioning (DP), a feature providing the ability to separate more important and less important syntax elements into different packets of data, enabling the application of unequal error protection (UEP) and other types of improvement of error/loss robustness.
- Redundant slices (RS), an error/loss robustness feature allowing an encoder to send an extra representation of a picture region (typically at lower fidelity) that can be used if the primary representation is corrupted or lost.
- Frame numbering, a feature that allows the creation of "sub-sequences", enabling temporal scalability by optional inclusion of extra pictures between other pictures, and the detection and concealment of losses of entire pictures, which can occur due to network packet losses or channel errors.
- Switching slices, called SP and SI slices, allowing an encoder to direct a decoder to jump into an ongoing video stream for such purposes as video streaming bit rate switching and "trick mode" operation. When a decoder jumps into the middle of a video stream using the SP/SI feature, it can get an exact match to the decoded pictures at that location in the video stream despite using different pictures, or no pictures at all, as references prior to the switch.
- A simple automatic process for preventing the accidental emulation of start codes, which are special sequences of bits in the coded data that allow random access into the bitstream and recovery of byte alignment in systems that can lose byte synchronization.
- Supplemental enhancement information (SEI) and video usability information (VUI), which are extra information that can be inserted into the bitstream to enhance the use of the video for a wide variety of purposes.[clarify]
- Auxiliary pictures, which can be used for such purposes as alpha compositing.
- Support of monochrome, 4:2:0, 4:2:2, and 4:4:4 chroma subsampling (depending on the selected profile).
- Support of sample bit depth precision ranging from 8 to 14 bits per sample (depending on the selected profile).
- The ability to encode individual color planes as distinct pictures with their own slice structures, macroblock modes, motion vectors, etc., allowing encoders to be designed with a simple parallelization structure (supported only in the three 4:4:4-capable profiles).
- Picture order count, a feature that serves to keep the ordering of the pictures and the values of samples in the decoded pictures isolated from timing information, allowing timing information to be carried and controlled/changed separately by a system without affecting decoded picture content.
These techniques, along with several others, help H.264 to perform significantly better than any prior standard under a wide variety of circumstances in a wide variety of application environments. H.264 can often perform radically better than MPEG-2 video—typically obtaining the same quality at half of the bit rate or less, especially on high bit rate and high resolution situations.[2]
Like other ISO/IEC MPEG video standards, H.264/AVC has a reference software implementation that can be freely downloaded[3]. Its main purpose is to give examples of H.264/AVC features, rather than being a useful application per se. Some reference hardware design work is also under way in the Moving Picture Experts Group. The above mentioned are complete features of H.264/AVC covering all profiles of H.264. A profile for a codec is a set of features of that codec identified to meet certain set of specifications of intended applications. This means that many of the features listed are not supported in some profiles. Various profiles of H.264/AVC are discussed in next section.
[edit] Profiles
The standard includes the following seven sets of capabilities, which are referred to as profiles, targeting specific classes of applications:
- Baseline Profile (BP): Primarily for lower-cost applications with limited computing resources, this profile is used widely in videoconferencing and mobile applications.
- Main Profile (MP): Originally intended as the mainstream consumer profile for broadcast and storage applications, the importance of this profile faded when the High profile was developed for those applications.
- Extended Profile (XP): Intended as the streaming video profile, this profile has relatively high compression capability and some extra tricks for robustness to data losses and server stream switching.
- High Profile (HiP): The primary profile for broadcast and disc storage applications, particularly for high-definition television applications (this is the profile adopted into HD DVD and Blu-ray Disc, for example).
- High 10 Profile (Hi10P): Going beyond today's mainstream consumer product capabilities, this profile builds on top of the High Profile—adding support for up to 10 bits per sample of decoded picture precision.
- High 4:2:2 Profile (Hi422P): Primarily targeting professional applications that use interlaced video, this profile builds on top of the High 10 Profile—adding support for the 4:2:2 chroma subsampling format while using up to 10 bits per sample of decoded picture precision.
- High 4:4:4 Predictive Profile (Hi444PP): This profile builds on top of the High 4:2:2 Profile—supporting up to 4:4:4 chroma sampling, up to 14 bits per sample, and additionally supporting efficient lossless region coding and the coding of each picture as three separate color planes.
In addition, the standard contains four additional all-Intra profiles, which are defined as simple subsets of other corresponding profiles. These are mostly for professional (e.g., camera and editing system) applications:
- High 10 Intra Profile: The High 10 Profile constrained to all-Intra use.
- High 4:2:2 Intra Profile: The High 4:2:2 Profile constrained to all-Intra use.
- High 4:4:4 Intra Profile: The High 4:4:4 Profile constrained to all-Intra use.
- CAVLC 4:4:4 Intra Profile: The High 4:4:4 Profile constrained to all-Intra use and to CAVLC entropy coding (i.e., not supporting CABAC).
As a result of the Scalable Video Coding extension, the standard contains three additional scalable profiles, which are defined as a combination of the H.264/AVC profile for the base layer (2nd word in scalable profile name) and tools that achieve the scalable extension:
- Scalable Baseline Profile: Primarily targeting video conferencing, mobile, and surveillance applications, this profile builds on top of a constrained version of the H.264/AVC Baseline profile to which the base layer (a subset of the bitstream) must conform. For the scalability tools, a subset of the available tools is enabled.
- Scalable High Profile: Primarily targeting broadcast and streaming applications, this profile builds on top of the H.264/AVC High Profile to which the base layer must conform.
- Scalable High Intra Profile: Primarily targeting production applications, this profile is the Scalable High Profile constrained to all-Intra use.
Baseline | Extended | Main | High | High 10 | High 4:2:2 | High 4:4:4
Predictive |
|
---|---|---|---|---|---|---|---|
I and P Slices | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
B Slices | No | Yes | Yes | Yes | Yes | Yes | Yes |
SI and SP Slices | No | Yes | No | No | No | No | No |
Multiple Reference Frames | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
In-Loop Deblocking Filter | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
CAVLC Entropy Coding | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
CABAC Entropy Coding | No | No | Yes | Yes | Yes | Yes | Yes |
Flexible Macroblock Ordering (FMO) | Yes | Yes | No | No | No | No | No |
Arbitrary Slice Ordering (ASO) | Yes | Yes | No | No | No | No | No |
Redundant Slices (RS) | Yes | Yes | No | No | No | No | No |
Data Partitioning | No | Yes | No | No | No | No | No |
Interlaced Coding (PicAFF, MBAFF) | No | Yes | Yes | Yes | Yes | Yes | Yes |
4:2:0 Chroma Format | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Monochrome Video Format (4:0:0) | No | No | No | Yes | Yes | Yes | Yes |
4:2:2 Chroma Format | No | No | No | No | No | Yes | Yes |
4:4:4 Chroma Format | No | No | No | No | No | No | Yes |
8 Bit Sample Depth | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
9 and 10 Bit Sample Depth | No | No | No | No | Yes | Yes | Yes |
11 to 14 Bit Sample Depth | No | No | No | No | No | No | Yes |
8x8 vs. 4x4 Transform Adaptivity | No | No | No | Yes | Yes | Yes | Yes |
Quantization Scaling Matrices | No | No | No | Yes | Yes | Yes | Yes |
Separate Cb and Cr QP control | No | No | No | Yes | Yes | Yes | Yes |
Separate Color Plane Coding | No | No | No | No | No | No | Yes |
Predictive Lossless Coding | No | No | No | No | No | No | Yes |
Baseline | Extended | Main | High | High 10 | High 4:2:2 | High 4:4:4
Predictive |
[edit] Levels
Level number | Max macroblocks per second | Max frame size (macroblocks) | Max video bit rate (VCL) for Baseline, Extended and Main Profiles | Max video bit rate (VCL) for High Profile | Max video bit rate (VCL) for High 10 Profile | Max video bit rate (VCL) for High 4:2:2 and High 4:4:4 Predictive Profiles | Examples for high resolution @ frame rate (max stored frames) in Level |
---|---|---|---|---|---|---|---|
1 | 1485 | 99 | 64 kbit/s | 80 kbit/s | 192 kbit/s | 256 kbit/s | 128x96@30.9 (8) 176x144@15.0 (4) |
1b | 1485 | 99 | 128 kbit/s | 160 kbit/s | 384 kbit/s | 512 kbit/s | 128x96@30.9 (8) 176x144@15.0 (4) |
1.1 | 3000 | 396 | 192 kbit/s | 240 kbit/s | 576 kbit/s | 768 kbit/s | 176x144@30.3 (9) 320x240@10.0 (3) 352x288@7.5 (2) |
1.2 | 6000 | 396 | 384 kbit/s | 480 kbit/s | 1152 kbit/s | 1536 kbit/s | 320x240@20.0 (7) 352x288@15.2 (6) |
1.3 | 11880 | 396 | 768 kbit/s | 960 kbit/s | 2304 kbit/s | 3072 kbit/s | 320x240@36.0 (7) 352x288@30.0 (6) |
2 | 11880 | 396 | 2 Mbit/s | 2.5 Mbit/s | 6 Mbit/s | 8 Mbit/s | 320x240@36.0 (7) 352x288@30.0 (6) |
2.1 | 19800 | 792 | 4 Mbit/s | 5 Mbit/s | 12 Mbit/s | 16 Mbit/s | 352x480@30.0 (7) 352x576@25.0 (6) |
2.2 | 20250 | 1620 | 4 Mbit/s | 5 Mbit/s | 12 Mbit/s | 16 Mbit/s | 352x480@30.7(10) 352x576@25.6 (7) 720x480@15.0 (6) 720x576@12.5 (5) |
3 | 40500 | 1620 | 10 Mbit/s | 12.5 Mbit/s | 30 Mbit/s | 40 Mbit/s | 352x480@61.4 (12) 352x576@51.1 (10) 720x480@30.0 (6) 720x576@25.0 (5) |
3.1 | 108000 | 3600 | 14 Mbit/s | 17.5 Mbit/s | 42 Mbit/s | 56 Mbit/s | 720x480@80.0 (13) 720x576@66.7 (11) 1280x720@30.0 (5) |
3.2 | 216000 | 5120 | 20 Mbit/s | 25 Mbit/s | 60 Mbit/s | 80 Mbit/s | 1280x720@60.0 (5) 1280x1024@42.2 (4) |
4 | 245760 | 8192 | 20 Mbit/s | 25 Mbit/s | 60 Mbit/s | 80 Mbit/s | 1280x720@68.3 (9) 1920x1088@30.1 (4) 2048x1024@30.0 (4) |
4.1 | 245760 | 8192 | 50 Mbit/s | 50 Mbit/s | 150 Mbit/s | 200 Mbit/s | 1280x720@68.3 (9) 1920x1088@30.1 (4) 2048x1024@30.0 (4) |
4.2 | 522240 | 8704 | 50 Mbit/s | 50 Mbit/s | 150 Mbit/s | 200 Mbit/s | 1920x1088@64.0 (4) 2048x1088@60.0 (4) |
5 | 589824 | 22080 | 135 Mbit/s | 168.75 Mbit/s | 405 Mbit/s | 540 Mbit/s | 1920x1088@72.3 (13) 2048x1024@72.0 (13) 2048x1088@67.8 (12) 2560x1920@30.7 (5) 3680x1536/26.7 (5) |
5.1 | 983040 | 36864 | 240 Mbit/s | 300 Mbit/s | 720 Mbit/s | 960 Mbit/s | 1920x1088@120.5 (16) 4096x2048@30.0 (5) 4096x2304@26.7 (5) |
Level number | Max macroblocks per second | Max frame size (macroblocks) | Max video bit rate (VCL) for Baseline, Extended and Main Profiles | Max video bit rate (VCL) for High Profile | Max video bit rate (VCL) for High 10 Profile | Max video bit rate (VCL) for High 4:2:2 and High 4:4:4 Predictive Profiles | Examples for high resolution @ frame rate (max stored frames) in Level |
[edit] Standardization committee and history
In early 1998 the Video Coding Experts Group (VCEG – ITU-T SG16 Q.6) issued a call for proposals on a project called H.26L, with the target to double the coding efficiency (which means halving the bit rate necessary for a given level of fidelity) in comparison to any other existing video coding standards for a broad variety of applications. VCEG was chaired by Gary Sullivan (Microsoft [formerly PictureTel], USA). The first draft design for that new standard was adopted in August 1999. In 2000, Thomas Wiegand (Heinrich Hertz Institute, Germany) became VCEG co-chair. In December of 2001, VCEG and the Moving Picture Experts Group (MPEG – ISO/IEC JTC 1/SC 29/WG 11) formed a Joint Video Team (JVT), with the charter to finalize the video coding standard. Formal approval of the specification came in March 2003. The JVT was (is) chaired by Gary Sullivan, Thomas Wiegand, and Ajay Luthra (Motorola, USA). In June 2004, the Fidelity range extensions (FRExt) project was finalized. From January 2005 to November 2007, the JVT was working on an extension of H.264/AVC towards scalability by an Annex called Scalable Video Coding (SVC). The JVT management team was extended by Jens-Reiner Ohm (Aachen University, Germany). Since July 2006, the JVT works on an extension of H.264/AVC towards multi-view video coding (MVC).
[edit] Versions
Versions of the H.264/AVC standard include the following completed revisions, corrigenda, and amendments (dates are final approval dates in ITU-T, while final "International Standard" approval dates in ISO/IEC are somewhat different and slightly later in most cases). Each version represents changes relative to the next lower version that is integrated into the text. Bold faced versions are published (or planned to be published).
- 1. Version (May 2003) First approved version of H.264/AVC containing Baseline, Extended, and Main profiles.
- 2. Version (May 2004) Corrigendum containing various minor corrections.
- 3. Version (March 2005) Major addition to H.264/AVC containing the first Amendment providing Fidelity Range Extensions (FRExt) containing High, High 10, High 4:2:2, and High 4:4:4 profiles.
- 4. Version (September 2005) Corrigendum containing various minor corrections and adding three aspect ratio indicators.
- 5. Version (June 2006) Amendment consisting of removal of prior High 4:4:4 profile (processed as a corrigendum in ISO/IEC)
- 6. Version (June 2006) Amendment consisting of minor extensions like extended-gamut color space support (bundled with above-mentioned aspect ratio indicators in ISO/IEC).
- 7. Version (April 2007) Amendment containing the addition of High 4:4:4 Predictive and four Intra-only profiles (High 10 Intra, High 4:2:2 Intra, High 4:4:4 Intra, and CAVLC 4:4:4 Intra).
- 8. Version (November 2007) Major addition to H.264/AVC containing the Amendment for Scalable Video Coding (SVC) containing Scalable Baseline, Scalable High, and Scalable High Intra profiles.
Planned additions:
- Corrigendum containing various minor corrections – not yet completed.
- Multi-view coding (MVC) – not yet completed.
[edit] Patent licensing
In countries where software patent regulations are upheld, the vendors of products which make use of H.264/AVC are expected to pay patent licensing royalties for the patented technology that their products use. A private organization known as MPEG LA, which is not affiliated in any way with the MPEG standardization organization, administers the licenses for patents applying to this standard, as well as the patent pools for MPEG-2 Part 1 Systems, MPEG-2 Part 2 Video, MPEG-4 Part 2 Video, and other technologies.
In January 2007, a U.S. District court jury gave an advisory opinion that one patent owned by Qualcomm should be invalidated.[4] Qualcomm had claimed that the patent had been incorporated in H.264 in violation of its patent.[5][6] The U.S. District Court judge has yet to rule on the verdict.[7]
[edit] Patents and GNU Free Software licenses
Discussions are often held regarding the legality of free software implementations of codecs like H.264, especially concerning the legal use of GNU LGPL and GPL implementations of H.264 and other patented codecs. Consensus in discussions is that the allowable use depends on the laws of local jurisdictions. If operating or shipping a product in a country or group of countries where none of the patents covering H.264 apply, then using, for example, an LGPL implementation of the codec is not a problem: There is no conflict between the software license and the (non-existent) patent license.
Conversely, shipping (not necessarily implementing) a product in the U.S. which includes an LGPL H.264 decoder/encoder would be in violation of the software license of the codec implementation. In simple terms, the LGPL and GPL licenses require that any rights held in conjunction with distributing the code also apply to anyone receiving the code, and no further restrictions are put on distribution or use. If there is a requirement for a patent license to be sought, this is a clear violation of both the GPL and LGPL terms. Thus, the right to distribute patent-encumbered code under those licenses as part of the product is revoked per the terms of the GPL and LGPL. However, if the initial implementor of the code did not hold the appropriate rights to build and distribute the code, the legal situation becomes less clear, and it is likely that all users of the implementation, whether LGPL or not, would be in breach of the relevant patents.
There have been no known court cases testing this legal interpretation to be correct; however, its interpretation fits best with statements regarding the topic made by the Free Software Foundation on this patent rights issue, in cases likely to use an expert/authoritative source on interpretation of the GPL and LGPL in a possible lawsuit.[citation needed]
[edit] Applications
H.264/AVC experienced widespread adoption within a few years of the completion of the standard. It is employed widely in applications ranging from television broadcast to video for mobile devices. In order to ensure compatibility and problem-free adoption of H.264/AVC, many standards bodies have amended or added to video standards so that users of these standards can employ H.264/AVC.
Both of the major candidate next-generation optical video disc rival formats deployed in 2006 include the H.264/AVC High Profile as a mandatory player feature—specifically:
- The Blu-ray Disc format of the Blu-ray Disc Association (BDA)
- The HD DVD format of the DVD Forum
The Digital Video Broadcast (DVB) standards body in Europe approved the use of H.264/AVC for broadcast television in Europe in late 2004. The Advanced Television Systems Committee (ATSC) standards body in the United States is considering the possibility of specifying one or two advanced video codecs for its optional Enhanced-VSB (E-VSB) transmission mode for use in U.S. broadcast television. It has included H.264/AVC and VC-1 into Candidate Standards as CS/TSG-659r2[8] and CS/TSG-658r1[9] respectively for this purpose. The status of terrestrial broadcast adoption in some specific countries is as follows:
- In Catalonia, Spain, TV3 is broadcasting a HDTV trial channel in DVB-T. Video is encoded in H.264 with PAFF interlacing.
- The prime minister of France announced the selection of H.264/AVC as a requirement for receivers of HDTV and pay TV channels for digital terrestrial broadcast television services (referred to as "TNT") in France in late 2004.
- The terrestrial broadcast systems in Lithuania, Estonia and Slovenia are expected to use H.264/AVC for all digital television services.
- The Brazilian terrestrial broadcast service uses H.264/AVC in ISDB-T.
- It is used for DVB-T transmission in Ukraine, Russia, Lithuania, Norway and other countries launching DVB-T broadcasting after 2006.
- In Poland test DVB-T transmissions (as of 2007 working reliably for several years) use MPEG-2, but commercial run scheduled after 2010 will be MPEG4-only.
- The Digital Multimedia Broadcast (DMB) service in the Republic of Korea will use H.264/AVC.
- Mobile-segment terrestrial broadcast services of ISDB-T in Japan will use the H.264/AVC codec, including major broadcasters such as NHK and Fuji Television.
- Norwegian NTV will use H.264/AVC for its national DVB-T broadcasting starting October 2007 in central southern areas of Norway. Norway will be among the first to use MPEG-4/AVC exclusively in all its terrestrial television broadcasts, when the analogue transponders are switched off in 2009.
- Hong Kong leading broadcaster,TVB, selected H.264 in for new digital services there, including HDTV service, in the China DMB-T/H system environment, starting from the end of 2007.[10] ATV has joined this decision and uses H.264 for its new digital services too (both broadcasters use MPEG-2 to encode the digital simulcast of the existing analogue channels.)
- New Zealand's Freeview service launched its DVB-T transmissions in March 2008 using H.264/AVC.
Direct broadcast satellite TV services will use the new standard, including:
- BBC HD uses H.264 for both its satellite (DVB-S) service on Astra 2D and its London terrestrial (DVB-T) trial service
- DirecTV (in the United States)
- Dish Network (in the United States)
- Euro1080 (in Europe)
- HD5 (MediaCorp TV in Singapore)
- Premiere (pay television network) (in Germany)
- ProSieben HD & Sat.1 HD (in Germany, ProSiebenSat.1 Media AG[11])
- Sky HD (in the United Kingdom and Ireland) – the company's standard definition service continues to use MPEG-2
- Sky Italia (in Italy)
- SVT HD (Sveriges Television in Sweden)
H.264 is used in a number of IPTV services. In particular, including:
- Australian ISP TPG Internet is broadcasting in H.264 for its IPTV service.
- Canadian Telco Bell Aliant as of April 10th 2008 is now broadcasting in H.264 for its IPTV service.
USDTV (now bankrupt) had announced plans to use H.264 for its pay-for-premium ATSC channels, which could only be decrypted by USDTV's set top boxes.
Inuk Networks the largest IPTV provider in the UK broadcasts in H.264.
The 3rd Generation Partnership Project (3GPP) has approved the inclusion of H.264/AVC as an optional feature in release 6 of its mobile multimedia telephony services specifications.
The North Atlantic Treaty Organisation (NATO) and the Motion Imagery Standards Board (MISB) of the United States Department of Defense (DoD) have adopted H.264/AVC as their preferred video codec for a broad variety of military applications.
The Internet Engineering Task Force (IETF) has completed a payload packetization format (RFC 3984) for carrying H.264/AVC video using its Real-time Transport Protocol (RTP).
The Internet Streaming Media Alliance (ISMA) has adopted H.264/AVC for its new ISMA 2.0 specifications.
Based on ITU-T H.32x standards, H.264/AVC is widely used for videoconferencing. Essentially all new videoconferencing products now support it.
The International Telecommunications Union-Radiocom. Sector (ITU-R) has adopted H.264/AVC in
- ITU-R BT.1687 "Video bit-rate reduction for real-time distribution of large-screen digital imagery applications for presentation in a theatrical environment" and
- ITU-R BT.1737 "Use of the ITU-T Recommendation H.264 (MPEG-4/AVC) video source-coding method to transport high definition TV programme material" for HDTV contribution, distribution, and satellite news gathering.
In October 2005, Apple Inc began selling H.264-encoded videos over the Internet through their iTunes Music Store.[12] Initially selling just television series and music videos, they expanded in September 2006 to sell films. On May 30, 2007 Apple announced plans to integrate streaming of YouTube videos into the Apple TV. In a later interview, Apple VP David Moody revealed that all of YouTube's videos are going to be transcoded to H.264 for higher compatibility and quality on the Apple TV. Starting in June, 2007, YouTube will be automatically encoding all new uploads with H.264. Their intention is to have the entire video catalog available in H.264 by autumn 2007. Apple's iPhone and iPod Touch support H.264 Baseline Profile, Levels 2.1 and 3, at resolutions up to 480x320 or 640x480 and bitrates up to 1.5 Mbit/s and is capable of playing the YouTube video content.[13]
Adobe supports H.264 in its Flash Player [14].
AVCHD is a high-definition recording format designed by Sony and Panasonic that uses H.264 (conforming to H.264 while adding additional application-specific features and constraints).
AVC-Intra is an intraframe compression only format, developed by Panasonic
[edit] Software encoder feature comparison
QuickTime | Nero Digital | x264 | Mainconcept | Elecard | Telestream | VSofts | ProCoder3 | |
---|---|---|---|---|---|---|---|---|
I and P Slices | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
B Slices | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
SI and SP Slices | No | No | No | No | No | No | No | No |
Multiple Reference Frames | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
In-Loop Deblocking Filter | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
CAVLC Entropy Coding | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
CABAC Entropy Coding | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Flexible Macroblock Ordering (FMO) | No | No | No | No | No | No | Yes | No |
Arbitrary Slice Ordering (ASO) | No | No | No | No | No | No | No | No |
Redundant Slices (RS) | No | No | No | No | No | No | No | No |
Data Partitioning | No | No | No | No | No | No | No | No |
Interlaced Coding (PicAFF, MBAFF) | No | No | Yes
(MBAFF) |
Yes | Yes | No | Yes
(MBAFF) |
Yes |
4:2:0 Chroma Format | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Monochrome Video Format (4:0:0) | No | No | No | No | Yes | No | No | No |
4:2:2 Chroma Format | No | No | No | No | No | Yes | Yes | No |
4:4:4 Chroma Format | No | No | No | No | No | No | Yes | No |
8 Bit Sample Depth | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
9 and 10 Bit Sample Depth | No | No | No | No | No | No | Yes | No |
11 to 14 Bit Sample Depth | No | No | No | No | No | No | No | No |
8x8 vs. 4x4 Transform Adaptivity | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Quantization Scaling Matrices | No | No | Yes | No | No | No | Yes | No |
Separate Cb and Cr QP control | No | No | Yes | Yes | Yes | No | No | No |
Separate Color Plane Coding | No | No | No | No | No | No | No | No |
Predictive Lossless Coding | No | No | Yes | No | Yes | No | No | No |
Film Grain Modeling | No | No | No | No | No | No | No | No |
QuickTime | Nero Digital | x264 | Mainconcept | Elecard | Telestream | VSofts | ProCoder3 |
[edit] See also
- Codec
- Comparison of H.264 and VC-1
- H.263
- H.264/MPEG-4 AVC Products and Implementations
- IPTV
- ISO/IEC Moving Picture Experts Group (MPEG)
- ITU-T Video Coding Experts Group (VCEG)
- List of devices that support H.264/MPEG-4 AVC
- MPEG-2
- MPEG-4
- x264 – Software encoder
[edit] References
- ^ H.262 : Information technology - Generic coding of moving pictures and associated audio information: Video. Retrieved on 2007-04-15.
- ^ Apple: H.264 FAQ
- ^ H.264/AVC JM Reference Software Download
- ^ SignOnSanDiego.com > News > Business -- Qualcomm loses its patent-rights case. Retrieved on 2007-04-15.
- ^ SignOnSanDiego.com News Business -- Qualcomm's patent case goes to jury. Retrieved on 2007-04-15.
- ^ Bloomberg.com: Technology. Retrieved on 2007-04-15.
- ^ Jury Sides with Broadcom in Qualcomm Patent Case. Retrieved on 2007-04-15.
- ^ ATSC Candidate Standard CS/TSG-659r2
- ^ ATSC Candidate Standard CS/TSG-658r1
- ^ Telecommunications Authority of Hong Kong. "Technical Standard for Digital Terrestrial Television Broadcasting", Telecommunications Authority of Hong Kong, June 4, 2007.
- ^ ProSiebenSat.1 Group to broadcast Sat.1 and ProSieben in HDTV. Retrieved on 2007-04-15.
- ^ Apple PR (October 12, 2005). Apple Announces iTunes 6 With 2,000 Music Videos, Pixar Short Films & Hit TV Shows. Apple Computer. Retrieved on 2006-12-17.
- ^ Apple PR (June 20, 2007). YouTube Live on Apple TV Today; Coming to iPhone on June 29. Apple Computer. Retrieved on 2007-06-21.
- ^ Adobe. Adobe Delivers Flash Player 9 with H.264 Video Support. Adobe.
[edit] External links
[edit] Introduction
- IEEE Overview of the H.264/AVC Video Coding Standard
- Overview and Introduction to the Fidelity Range Extensions
- Vcodex: H.264 Tutorials. Retrieved on 2007-04-15.
- MPEG-4 AVC/H.264 Information - Doom9's Forum. Retrieved on 2007-04-15.
[edit] The standard
- ITU-T publication page: H.264: Advanced video coding for generic audiovisual services. Retrieved on 2007-09-13.
- ISO publication page: ISO/IEC 14496-10:2005 – Information technology – Coding of audio-visual objects – Part 10: Advanced Video Coding. Retrieved on 2007-09-13.
[edit] Reference encoder/decoder
- IP Homepage - H.264/AVC JM Reference Software. Retrieved on 2007-04-15.
[edit] Standardization committee documents
- JVT document archive site. Retrieved on 2007-05-06.
[edit] HD content
- Apple - QuickTime - HD Gallery (R). Retrieved on 2007-04-15.
- Apple - Movie Trailers. Retrieved on 2007-04-15.
- Wuapi.com. Retrieved on 2008-05-15.
[edit] Miscellaneous
- Publications - Thomas Wiegand. Retrieved on 2007-06-23.
- Publications - Detlev Marpe. Retrieved on 2007-04-15.
- Fourth Annual H.264 video codecs comparison.
- Comparison between CoreAVC and FFmpeg.
|
|