MPEG

MPEG stands for the Motion Picture Expert Group. It along with the 3 letter MPG extension stands for the video/audio formats designed by the group.

The various iterations

MPEG-1

This is the earliest standard. It pioneered the digital format and opened the door for digital video transmission for such things as cable and satellite TV. It is the most popular video format in the world. The standard came out of the JPEG standard for video compression plus an earlier CIF standard used for video teleconferencing. Video uses a JPEG like compression for each frame. In addition the format takes advantage of the fact that not all of a picture changes with each frame so it defines macroblocks so that only part of the picture can be updated at a time. This means than the whole picture need not be transmitted in every frame, although you need to send the full image every once and a while to ensure that nothing gets out of sync. This results in a major decrease in the size of the video stream.

The CIF (Common Intermediate Format) standard, H.261, was 352 x 288 pixels and was the equivalent quality of a VHS tape recorder. The pixels were not square so a modern square pixel device would need 384 x 288 pixels to maintain a 4 x 3 ratio. Each macroblock was 16x16 pixels. The SIF (Standard Input format) adopted initially for MPEG-1 for American NTSC broadcast was modified to 352 x 240 to fit the screen better. Video CD's were made with this format.

MPEG-1 is designed to compress VHS-quality raw digital video and CD audio down to 1.5 Mbit/s (26:1 and 6:1 compression ratios respectively) without excessive quality loss. However, the MPEG-1 stream would support video as high as 4K x 4K Video at higher bit rates. Over time, the sound CODEC for MPEG was modified 3 times and we now know it as MP3 which has become an independent audio format.

MPEG-2

MPEG-1 has several deficiencies for TV use. Most notably it does not support interlaced video. It also has only 2 audio tracks. These and a few other features were added to produce MPEG-2. Note that a MPEG-2 decoder will still decode MPEG-1 data. The sound format was modernized to AAC and upped to be able to support 5.1 channels. Full progressive and interlace support is made available.

MPEG-2 (H.262) is used for DVD with NTSC resolution up to 720 × 480 (Pal 720 x 576) and defined aspect ratios of 4x3 and 16x9. The peak bit rate can be 10.08 Mbit/s. Note that 16x9 images are compressed horizontally (anamorphic) to fit in the frame. The DVD format is called VOB.

MPEG-2 video is also the defined format for American HD television (ATSC) with a maximum resolution of 1920 x 1080 interlaced but the sound is Dolby-AC3. There is a maximum bitrate of 19.4 Mbit/s for broadcast television and 38.8 Mbit/s for cable television. Blu-Ray can support 1920 x 1080 Progressive at 24 frames/sec if the TV supports it. Blu-Ray requires support for MPEG-2 part 2 and will usually have a .m2ts extension. About 2 hours of can be placed on a single disk in this format. However, this provides backward compatibility with DVD's which can be played on the same machine.

MPEG-3

This standard was developed for HDTV while the MPEG-2 standard was being developed. The codec was H.263 which still continues to handle the bitrates of 20-40Mb/s. However while the standard was underway it was discovered that HDTV could be rolled into the MPEG-2 standard and thus it was cancelled.

MPEG-4

This is the most recent video standard and uses elements of MPEG-1 and MPEG-2 while adding things like 3D support. It also provides for lower bit-rate video needed for Internet use. It adds the ability to define and use animated objects. It also has DRM support. The features and additions to MPEG-4 have not stopped so it is difficult to say if a particular CODEC (for example: H.264) can decode them all.

MPEG-4 part 14 (a subset of the full MPEG-4 standard) is also called MP4 and typically has an MP4 extension. Devices designed for this video format will often be called MP4 players, however some devices called MP4 players may not fully support the standard but are really MP3 players with some sort of video support. An MP4 compatible file with only sound elements will typically have an M4A file extension however as noted in MPEG-2 it is really just an AAC file. Other M4 extensions exist as well.

MP4 can be used for streaming Internet video and on-line broadcasting. The image format is almost identical to the MOV (QuickTime) format except for its added support for some extra MPEG features. To achieve support for possible lower bandwidths the format includes hints to help the decoder keep up.

Initially MPEG-4 audio was AAC, the same as MPEG-2, however and new HE-AAC format (hight efficiency AAC) is now standard. An HE-AAC codec can decode AAC but has improved sound qualities with higher compression, particularly at low bitrates and can be used successfully down to 48 Kbits/sec.

H.264 also known as AVC (Advanced Video Coding) is a particular codec and is identical with MPEG-4 part 10. MPEG-4 AVC is a required format for Blu-Ray disks. It is an advanced format that can achieve up to 4 hours on a standard disk at full resolution. Blu-Ray requires support for 3 video Codecs, in addition to AVC it supports MPEG-2 and VC-1. H.264 codec is often used for live recording using all I-frames for high quality (HD) video imaging. The newest codec is called H.265 also known as HEVC (High Efficiency Video Coding) and improves the compression by 2x for similar visual quality.

How it works

MPEG format is a succession of graphic images with or without embedded audio. Each image is time stamped and the decoder can typically handle images even if they are somewhat out of order. There are three kinds of images called I, P, and B. Because of the successive images in a motion picture it may be that only part the image is updated in subsequent frames.

I frame images are similar to JPEG images and contain the full image. The image is broken into blocks of data. A video could be made up only using I frames but the compression would not be very good and would require a high bitrate to display. (Similar to M-JPEG, Motion Jpeg).

P frame images contain a partial image comprising only data that was changed from the last I frame image or a previous P frame image. If you were to jump into a movie at an arbitrary point you might need to backup to the last I frame before the P frames would make any sense.

B frame, bi-directional, images achieve the most compression but are also the most complicated. Before talking about B frames it is important to understand the limitations of P frames. A feature of MPEG is that it can also handle an object that moves from frame to frame but is otherwise unchanged. For example a car moving across the screen. The subframe picture of the car can be transmitted once and then relocated in subsequent frames without needed to send the image again. This works well so long at the image covers up whatever image it moves in front of but is a problem in that it also uncovers a portion of the picture as is leaves. The idea of a B frame is that it can handle the uncovered portion of the image by knowing what the future image looks like.

The trick to knowing the future is that the images are sent out of order. For example, an I frame is sent with the current timestamp and another I frame is sent with a future timestamp and then one or more B frames are sent with timestamps between the two I frames. The decoder sees the future I frame and stores it away. When it needs to uncover an area of the screen it can use the portion of the future frame to know what it should look like. The distance between I frames is called the GOP (Group of pictures). The GOP defines the set of interdependent frames.

Creating B frames requires significant resources to generate the image frames in the first place but it very doable during the decode phase. It takes some dedicated hardware and powerful processors to be able to accomplish B frames in real time, for a sporting event for example, and thus may not even be used resulting in a much higher bandwidth requirement than would be needed for a movie that was prepared ahead of time and could be highly compressed with lots of B frames. Compression of 50 to 1 is possible.

Note in Mpeg-2 and later, frames can be divided into fields which are 1/2 of a frame interleaved with even and odd lines separated and are at twice the frame rate. This is called Interlaced.

There are systems and devices that capture images using only I frames. These can be played on MPEG players but are significantly larger (several times as big) than they would be with full MPEG support.