Flexible Macroblock Ordering
Encyclopedia
H.264/MPEG-4 AVC
is a new standard for digital video compression jointly developed by ITU-T’s Video Coding Experts Group (VCEG
) and ISO/IEC’s Moving Picture Experts Group (MPEG).
Next to efficient coding algorithms, the H.264/AVC specifications define a set of error resilience tools. The most striking one is Flexible Macroblock Ordering or FMO.
as a sequence of macroblocks that belong to the same slice group. These macroblocks are processed in a scan order (left to right and top to bottom). A slice can be decoded independently.
FMO consists of deciding to which slice each macroblock of the image belongs.
Each macroblock can be assigned freely to a slice group using an MBAmap (MacroBlock Allocation map). The MBAmap consists of an identification number for each macroblock of the image that specifies to which slice group that macroblock belongs. The number of slice groups is limited to 8 for each picture to prevent complex allocation schemes.
If we deactivate the FMO, the images will be composed of a single slice group with the macroblocks in a scan order.
The use of FMO is totally compatible with any type of inter-frame prediction.
With this technique, we can correct errors easily by exploiting the spatial redundancy
of the images. It is a good idea to choose the slice groups in a way that no macroblock and its neighbors belong to the same group. Therefore, if a slice is lost during transmission
, it is possible to reconstruct the lost blocks with the information of the neighboring blocks. We have to consider the transmission characteristics of these slices: each slice is transmitted independently in separate units called packets. Each packet contains in its own header
the information to decode itself without any other packet’s information (if the images used as reference are the same in the encoder
and the decoder
side).
The use of FMO together with advanced error resilience tools can keep the visual quality even with a packet loss rate of 10%.
Note: Each color represents a slice group
▪ The FMO Type 1 can also be used in transmission environments with a high packet loss rate (example seen in Experimental Results).
▪ The FMO Type 2 can also be very useful. Let’s imagine we want to transmit a news bulletin video with a low bit rate
. If we encode the whole image with the bit rate distributed equally for all the macroblocks we'll get some bad results. Humans are particularly sensitive to image errors in faces, so we can mark the region of the newsreader’s face so it's encoded with more bits. Therefore, the background of the set will be encoded with less bits and the newsreader with a lot more bits (maintaining the total bit rate as we had before). This way, the subjective visual quality will be enhanced significantly.
In the images below we can see two frames of the Stefan.yuv sequence, the first one without using FMO and the second one using FMO Type 1:
We can clearly see in the bottom-right corner of the image that some packets were lost during transmission and the macroblocks of the image couldn’t be decoded. But when we use FMO Type 1, we have two slice groups dividing the image like a chess board. Therefore, when a packet of the slice group 0 is lost, we can recover that part of the image interpolating the neighboring macroblocks that have not been lost because they belong to slice group 1.
or MSE
), when the packets are being transmitted in an environment with a certain packet loss rate.
Even though we’ve been exposing the advantages of FMO, we also have to consider its main disadvantage: we have to send the information of which FMO we’re using in the header of the packet. That means we have a significant increase in the number of bits we have to send. Therefore, the use of FMO is only recommended when we know that the transmission environment is likely to have a rather high packet loss rate.
H.264/MPEG-4 AVC
H.264/MPEG-4 Part 10 or AVC is a standard for video compression, and is currently one of the most commonly used formats for the recording, compression, and distribution of high definition video...
is a new standard for digital video compression jointly developed by ITU-T’s Video Coding Experts Group (VCEG
VCEG
The Video Coding Experts Group or Visual Coding Experts Group is the informal name of Question 6 of Working Party 3 of Study Group 16 of the ITU-T. Its abbreviated title is ITU-T Q.6/SG 16...
) and ISO/IEC’s Moving Picture Experts Group (MPEG).
Next to efficient coding algorithms, the H.264/AVC specifications define a set of error resilience tools. The most striking one is Flexible Macroblock Ordering or FMO.
Description
One of the new characteristics of the H.264/AVC standard is the possibility of dividing an image in regions called slice groups. Each slice group can also be divided in several slices; therefore, we need to define the word sliceSlice
-Food:* A portion of bread, pizza, cake, or meat that is cut flat and thin, cf. sliced bread* Slice , a line of fruit-flavored drinks* Vanilla slice, a dessert* Mr. Slice, the mascot of Papa John's Pizza restaurant...
as a sequence of macroblocks that belong to the same slice group. These macroblocks are processed in a scan order (left to right and top to bottom). A slice can be decoded independently.
FMO consists of deciding to which slice each macroblock of the image belongs.
Each macroblock can be assigned freely to a slice group using an MBAmap (MacroBlock Allocation map). The MBAmap consists of an identification number for each macroblock of the image that specifies to which slice group that macroblock belongs. The number of slice groups is limited to 8 for each picture to prevent complex allocation schemes.
If we deactivate the FMO, the images will be composed of a single slice group with the macroblocks in a scan order.
The use of FMO is totally compatible with any type of inter-frame prediction.
With this technique, we can correct errors easily by exploiting the spatial redundancy
Redundancy (information theory)
Redundancy in information theory is the number of bits used to transmit a message minus the number of bits of actual information in the message. Informally, it is the amount of wasted "space" used to transmit certain data...
of the images. It is a good idea to choose the slice groups in a way that no macroblock and its neighbors belong to the same group. Therefore, if a slice is lost during transmission
Transmission (telecommunications)
Transmission, in telecommunications, is the process of sending, propagating and receiving an analogue or digital information signal over a physical point-to-point or point-to-multipoint transmission medium, either wired, optical fiber or wireless...
, it is possible to reconstruct the lost blocks with the information of the neighboring blocks. We have to consider the transmission characteristics of these slices: each slice is transmitted independently in separate units called packets. Each packet contains in its own header
Header
Header may refer to: Computers and engineering* Header , supplemental data at the beginning of a data block** E-mail header** HTTP header* Header file, a text file used in computer programming...
the information to decode itself without any other packet’s information (if the images used as reference are the same in the encoder
Encoder
An encoder is a device, circuit, transducer, software program, algorithm or person that converts information from one format or code to another, for the purposes of standardization, speed, secrecy, security, or saving space by shrinking size.-Media:...
and the decoder
Decoder
A decoder is a device which does the reverse operation of an encoder, undoing the encoding so that the original information can be retrieved. The same method used to encode is usually just reversed in order to decode...
side).
The use of FMO together with advanced error resilience tools can keep the visual quality even with a packet loss rate of 10%.
Type
When using FMO, the image can be divided in different scan patterns of the macroblocks. There are seven FMO map types, referred to as Type 0 through Type 6. Type 6 is the most general one and allows full flexibility to the user. The others use specific pattern rules. These patterns can be exploited to represent the MBAmap more efficiently:- Type 0: uses run lengths which are applied consecutively until the map is complete. Therefore only the run lengths are needed to rebuild the image on the decoder side.
- Type 1: also known as scattered slices; it uses a mathematical function, which is known in both the encoder and the decoder, to spread the macroblocks. The distribution in the figure, in which the macroblocks are spread forming a chess board, is one common case (see Applications).
- Type 2: is used to mark rectangular areas, so-called regions of interest. In this case the coordinates top-left and bottom-right of the rectangles are saved in the MBAmap.
- Type 3-5: are dynamic types that let the slice groups grow and shrink over the different pictures in a cyclic way. Only the growth rate, the direction and the position in the cycle have to be known.
Note: Each color represents a slice group
Applications
In this section we describe some of the possible applications of FMO in H.264/AVC video coding:▪ The FMO Type 1 can also be used in transmission environments with a high packet loss rate (example seen in Experimental Results).
▪ The FMO Type 2 can also be very useful. Let’s imagine we want to transmit a news bulletin video with a low bit rate
Bit rate
In telecommunications and computing, bit rate is the number of bits that are conveyed or processed per unit of time....
. If we encode the whole image with the bit rate distributed equally for all the macroblocks we'll get some bad results. Humans are particularly sensitive to image errors in faces, so we can mark the region of the newsreader’s face so it's encoded with more bits. Therefore, the background of the set will be encoded with less bits and the newsreader with a lot more bits (maintaining the total bit rate as we had before). This way, the subjective visual quality will be enhanced significantly.
Experimental Results
To do these experiments we have used reference software of the H.264/AVC standard [1].In the images below we can see two frames of the Stefan.yuv sequence, the first one without using FMO and the second one using FMO Type 1:
We can clearly see in the bottom-right corner of the image that some packets were lost during transmission and the macroblocks of the image couldn’t be decoded. But when we use FMO Type 1, we have two slice groups dividing the image like a chess board. Therefore, when a packet of the slice group 0 is lost, we can recover that part of the image interpolating the neighboring macroblocks that have not been lost because they belong to slice group 1.
Conclusions
This overview of FMO and the experimental results appearing in this article clearly show that FMO is a very useful error resilience tool. Its use can enhance visual quality, both subjective and objective (SNRSignal-to-noise ratio
Signal-to-noise ratio is a measure used in science and engineering that compares the level of a desired signal to the level of background noise. It is defined as the ratio of signal power to the noise power. A ratio higher than 1:1 indicates more signal than noise...
or MSE
Mean squared error
In statistics, the mean squared error of an estimator is one of many ways to quantify the difference between values implied by a kernel density estimator and the true values of the quantity being estimated. MSE is a risk function, corresponding to the expected value of the squared error loss or...
), when the packets are being transmitted in an environment with a certain packet loss rate.
Even though we’ve been exposing the advantages of FMO, we also have to consider its main disadvantage: we have to send the information of which FMO we’re using in the header of the packet. That means we have a significant increase in the number of bits we have to send. Therefore, the use of FMO is only recommended when we know that the transmission environment is likely to have a rather high packet loss rate.
External links
- [1] Software H.264: http://iphome.hhi.de/suehring/tml/download/
- T.Wiegand, G.J. Sullivan, G. Bjøntegaard, A.Luthra: Overview of the H.264/AVC Video Coding Standard. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, July 2003
- Y.Dhondt, P.Lambert: Flexible Macroblock Ordering, an error resilience tool in H.264/AVC. Fifth FTW PhD Symposium, Faculty of Engineering, Ghent University, December 2004, Paper No. 106