Slide1: Video Compression


Overview: Overview What is MPEG MPEG Standards MPEG Compression Basics MPEG Introduction MPEG Encoding & Decoding Intra frame Inter frame


What is MPEG: What is MPEG MPEG is an encoding and compression system for digital multimedia content defined by the Motion Pictures Expert Group (MPEG). MPEG reduces the amount of data needed to represent video many times over, but still manages to retain very high picture quality MPEG can compress both audio & video


MPEG Standards: MPEG Standards MPEG 1 Medium Bandwidth (up to 1.5Mbits/sec) Supports progressive images Non-interlaced video Supports only 352 * 240 resolution Optimized & used for storing movies on CD ROM


MPEG Standards: MPEG Standards MPEG 2 Originally designed for compressing broadcast quality video to 4 – 6 Mbps (PAL / NTSC) PAL (Phase Alternate Line) – Raw BW 216Mbps NTSC – Raw BW 168 bps Later expanded to support higher resolution including HDTV HDTV ( High Definition TV – Raw BW 1 Gbps Higher Bandwidth (up to 100Mbits/sec)


MPEG Standards: MPEG Standards MPEG 2 (cont.) Supports many resolution levels Low – 352 * 240 (For VCR) Main – 720 * 480 (NTSC broadcasting) High (for HDTV) 1440 * 1152 1920 * 1080 Can deal with interlaced video & progressive images MPEG 3 Originally designed for HDTV, but abandoned when it was discovered that MPEG-2 was sufficient for HDTV.


MPEG Standards: MPEG Standards MPEG 4 MPEG-4 defines coding of all types of multimedia information allows more interaction by the user includes some new video compression methods includes methods for audio and voice encoding scalable object based, allows scenes to be composed of natural and synthetic objects


MPEG Standards: MPEG Standards Goals of MPEG-4 reusable and reconfigurable content synthetic and natural content combined high level of user interaction scalability


MPEG Standards: MPEG Standards MPEG4 targets a wide range of av apps (from Digital TV to mobile multimedia) Provides interface points to existing systems Progressive and interlaced video formats Originally targeted for three bit-rate ranges: < 64Kbit/s 64 - 384 Kbit/s 384kbit - 4Mbit/s Also higher bit-rates are supported


MPEG Compression: MPEG Compression The MPEG-2 video compression algorithm achieves very high rates of compression by exploiting the redundancy in video information. Temporal redundancy - Arises when successive frames of video display images of the same scene Spatial redundancy - occurs because parts of the picture are often replicated (with minor changes) within a single frame of video.


MPEG Introduction: MPEG Introduction


MPEG Introduction: MPEG Introduction Video Sequence Begins with a sequence header. includes one or more groups of pictures, and ends with an end-of-sequence code. Group of Pictures (GOP) A Header and a series of one of more pictures intended to allow random access into the sequence


MPEG Introduction: MPEG Introduction Picture The primary coding unit of a video sequence. A picture consists of three rectangular matrices representing luminance (Y) and two chrominance (Cb and Cr) values. Slice Slice is a portion of a picture. Each slice is coded independently from the other slices of the picture. Slice are important in the handling of errors. If the bit stream contains an error, the decoder can skip to the next slice


MPEG Introduction: MPEG Introduction Macroblock A portion of image that consists oc 16x16 pixels and comprises 4 blocks of luminance component and 1 block each of the 2 chrominance components. At this layer, motion compensation and prediction are performed. Block A matrix of 8x8 elements.


Moving Picture Types: Moving Picture Types To achieve the requirement of random access, a set of pictures can be defined to form a Group of Picture (GOP), consisting of a minimum of one I-frame, which is the first frame, together with some P-frames and/or B-frames.


I-frame: I-frame Is a picture coded without reference to any picture except itself. It is a still image encoded in JPEG in real-time. Often, I pictures (I-frames) are used for random access and are used as references for the decoding of other pictures.


I-frame: I-frame


P-frame: P-frame The P-frames are forward predicted from the last I-frame or P-frame, it is impossible to reconstruct them without the data of another frame (I or P). Are coded with respect to the nearest previous I- or P-pictures. This technique is called forward prediction Uses motion compensation to provide more compression than I-frames


P-frame: P-frame Difference Huffman Code Motion vector Reference image (previous image) Encode: (a) motion vector - difference in spatial location of macroblocks (b) Small difference in content of the macro-blocks


MPEG Compression: MPEG Compression


Motion Vectors: Motion Vectors


Motion Vectors: Motion Vectors How to acquire motion vector? Do we need to find the motion vector for each pixel? How to find the motion vector?


Compromise between pixel and block: Compromise between pixel and block


Motion vectors computation: Motion vectors computation


Motion vectors block correspondence: Motion vectors block correspondence


Motion Vectors: Motion Vectors


Motion vectors: Motion vectors N frame N +1 frame Motion Vector


B-frame (Bidirectionally predictive coded frames): B-frame (Bidirectionally predictive coded frames) It requires information of the previous and following I-frame and/or P-frame for encoding and decoding. Three types of motion compensation techniques are used: Forward motion compensation uses past anchor frame information. Backward motion compensation uses future anchor frame information. Interpolative compensation uses the average of the past and future anchor frame information.


B-frame (Bidirectionally predictive coded frames): B-frame (Bidirectionally predictive coded frames) Two motion vectors are estimated (one to a past frame, one to a future frame). - [1/2 x ( + )] = DCT + Quant + RLE Huffman Code motion vectors (two) 000111010…... Past reference target future reference


I, B, P Basic Compression: I, B, P Basic Compression


I, B, P Basic Compression (forward prediction): I, B, P Basic Compression (forward prediction)


I, B, P Basic Compression (Bi- prediction): I, B, P Basic Compression (Bi- prediction)


I, B, P Frames (combine): I, B, P Frames (combine)


I, P, B Frames: I, P, B Frames I frames are independently encoded P frames are based on previous I, P frames B frames are based on previous and following I and P frames


Typical compression ratio: Typical compression ratio Type Size Compression --------------------- I 18 KB 7:1 P 6 KB 20:1 B 2.5 KB 50:1 Avg 4.8 KB 27:1 ---------------------


How to chose current frame: I, B, P?: How to chose current frame: I, B, P?


Some possibility in arranging I, B, P: Some possibility in arranging I, B, P


Picture Re-ordering: Picture Re-ordering In order to allow B frames to be decoded, frames are re-ordered when the mpeg file is created, so that when a frame is received, the decoder will already have the required reference frames. To encode the fames in display order I1B1B2P1B3B4P2B5B6P3B7B8P4 The ordering of the frames in the file would be I1P1B1B2P2B3B4P3B5B6P4B7B8 when the decoder receives the P frame, it decodes it, but it would delay displaying the picture, as the next frame is a B frame


Display Encoding/Decoding Order : Display Encoding/Decoding Order


Picture Re-ordering Example: Picture Re-ordering Example Give the encoded sequence of the following frames: I1P1P2B1B2B3B4P3B5I2B6B7B8P4 Answer I1P1P2P3B1B2B3B4I2B5P4B6B7B8


Bit Rate and Compression Ratio: Bit Rate and Compression Ratio Consider a video clip encoded in MPEG-1 with a frame rate of 30 frames per second and a group of pictures with sequence: IBBPBBPBBPBBPBB IBBPBBPBBPBBPBB ..... If the size of each I-frame, P-frame and B-frame is 12.5 KB, 6 KB and 2.5 KB respectively, calculate the average bit rate for the video clip. Suppose that the uncompressed frames are each of size 150 KB, find the compression ratio.


Bit Rate and Compression Ratio: Bit Rate and Compression Ratio In a GOP, there are 1 I-frame, 4 P-frames, and 10 B-frames. I-frame = 1 ×12.5 = 12.5 KB P-frame = 4 × 6 = 24 KB B-frame = 10 × 2.5 = 25 KB Size of a GOPs = 61.5 KB In 1 second, there are 30 frames = 2 GOPs = 2 × 61.5 KB = 123 KB Average bit rate = 123 × 1024 × 8 = 1007616 bit/s Overall compression ratio for the video stream = original/compressed = 15×150 / 61.5 = 36.59.


MPEG-1 Conclusion: MPEG-1 Conclusion Intraframe compression DCT Interframe compression Motion compression MPEG Hierarchy GoP, Picture, Slice, MB, Block I, B, P Frame


MPEG-2 : MPEG-2


MPEG-2 Summary: MPEG-2 Summary