Video quality
Encyclopedia
Video quality is a characteristic of a video
passed through a video transmission/processing system, a formal or informal measure of perceived video degradation (typically, compared to the original video). Video processing systems may introduce some amounts of distortion or artifacts in the video signal, so video quality evaluation is an important problem.
using some traditional test signal (for example, a collection of color bars and circles).
Nowadays, digital video
systems are replacing analog ones, and evaluation methods have changed. Performance of a digital video processing system can vary significantly and depends on dynamic characteristics of input video signal (e.g. amount of motion or spatial details). That's why digital video quality should be evaluated on diverse video sequences, often from the user's database.
Therefore, they can be classified as Full Reference Methods (FR), Reduced Reference Methods (RR) and No-Reference Methods (NR). FR metrics compute the quality difference by comparing every pixel in each image of the distorted video to its corresponding pixel in the original video. RR metrics extract some features of both videos and compare them to give a quality score. They are used when all the original video is not available, e.g. in a transmission with a limited bandwidth. NR metrics try to assess the quality of a distorted video without any reference to the original video. These metrics are usually used when the video coding method is known.
The most traditional ways of evaluating quality of digital video processing system (e.g. video codec
like DivX
, Xvid
) are calculation of the signal-to-noise ratio
(SNR) and peak signal-to-noise ratio
(PSNR) between the original video signal and signal passed through this system.
PSNR is the most widely used objective video quality metric. However, PSNR values do not perfectly correlate with a perceived visual quality due to the non-linear behavior of the human visual system. Recently a number of more complicated and precise metrics were developed, for example UQI, VQM, PEVQ
, SSIM
, VQuad-HD and CZD
. Based on a benchmark by the Video Quality Experts Group (VQEG) in the course of the Multimedia Test Phase 2007-2008 some metrics were standardized as ITU-T Rec. J.246 (RR), J.247 (FR) in 2008 and J.341 (FR HD) in 2011.
The performance of an objective video quality metric is evaluated by computing the correlation between the objective scores and the subjective
test results. The latter is called mean opinion score
(MOS). The most frequently used correlation coefficients are : linear correlation coefficient
, Spearman's rank correlation coefficient
, kurtosis
, kappa coefficient and outliers ratio
.
When estimating quality of a video codec, all the mentioned objective methods may require repeating post-encoding tests in order to determine the encoding parameters that satisfy a required level of visual quality, making them time consuming, complex and impractical for implementation in real commercial applications.
For this reason, much research has been focused on developing novel objective evaluation methods which enable prediction of the perceived quality level of the encoded video before the actual encoding is performed http://www.springerlink.com/content/f8v2p4r8522664l5.
measurements” are described in ITU-T
recommendation BT.500. Their main idea is the same as in Mean Opinion Score
for audio: video sequences are shown to the group of viewers and then their opinion is recorded and averaged to evaluate the quality of each video sequence. However, details of testing may vary greatly.
Video
Video is the technology of electronically capturing, recording, processing, storing, transmitting, and reconstructing a sequence of still images representing scenes in motion.- History :...
passed through a video transmission/processing system, a formal or informal measure of perceived video degradation (typically, compared to the original video). Video processing systems may introduce some amounts of distortion or artifacts in the video signal, so video quality evaluation is an important problem.
From analog to digital video
Since the time when the world's first video sequence was recorded, many video processing systems have been designed. In the ages of analog video systems, it was possible to evaluate quality of a video processing system by calculating the system's frequency responseFrequency response
Frequency response is the quantitative measure of the output spectrum of a system or device in response to a stimulus, and is used to characterize the dynamics of the system. It is a measure of magnitude and phase of the output as a function of frequency, in comparison to the input...
using some traditional test signal (for example, a collection of color bars and circles).
Nowadays, digital video
Digital video
Digital video is a type of digital recording system that works by using a digital rather than an analog video signal.The terms camera, video camera, and camcorder are used interchangeably in this article.- History :...
systems are replacing analog ones, and evaluation methods have changed. Performance of a digital video processing system can vary significantly and depends on dynamic characteristics of input video signal (e.g. amount of motion or spatial details). That's why digital video quality should be evaluated on diverse video sequences, often from the user's database.
Objective video quality
Objective video evaluation techniques are mathematical models that approximate results of subjective quality assessment, but are based on criteria and metrics that can be measured objectively and automatically evaluated by a computer program. Objective methods are classified based on the availability of the original video signal, which is considered to be of high quality (generally not compressed).Therefore, they can be classified as Full Reference Methods (FR), Reduced Reference Methods (RR) and No-Reference Methods (NR). FR metrics compute the quality difference by comparing every pixel in each image of the distorted video to its corresponding pixel in the original video. RR metrics extract some features of both videos and compare them to give a quality score. They are used when all the original video is not available, e.g. in a transmission with a limited bandwidth. NR metrics try to assess the quality of a distorted video without any reference to the original video. These metrics are usually used when the video coding method is known.
The most traditional ways of evaluating quality of digital video processing system (e.g. video codec
Video codec
A video codec is a device or software that enables video compression and/or decompression for digital video. The compression usually employs lossy data compression. Historically, video was stored as an analog signal on magnetic tape...
like DivX
DivX
DivX is a brand name of products created by DivX, Inc. , including the DivX Codec which has become popular due to its ability to compress lengthy video segments into small sizes while maintaining relatively high visual quality.There are two DivX codecs; the regular MPEG-4 Part 2 DivX codec and the...
, Xvid
XviD
Xvid is a video codec library following the MPEG-4 standard, specifically MPEG-4 Part 2 Advanced Simple Profile . It uses ASP features such as b-frames, global and quarter pixel motion compensation, lumi masking, trellis quantization, and H.263, MPEG and custom quantization matrices.Xvid is a...
) are calculation of the signal-to-noise ratio
Signal-to-noise ratio
Signal-to-noise ratio is a measure used in science and engineering that compares the level of a desired signal to the level of background noise. It is defined as the ratio of signal power to the noise power. A ratio higher than 1:1 indicates more signal than noise...
(SNR) and peak signal-to-noise ratio
Peak signal-to-noise ratio
The phrase peak signal-to-noise ratio, often abbreviated PSNR, is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation...
(PSNR) between the original video signal and signal passed through this system.
PSNR is the most widely used objective video quality metric. However, PSNR values do not perfectly correlate with a perceived visual quality due to the non-linear behavior of the human visual system. Recently a number of more complicated and precise metrics were developed, for example UQI, VQM, PEVQ
PEVQ
PEVQ ' is a standardized end-to-end measurement algorithm to score the picture quality of a video presentation by means of a 5-point mean opinion score...
, SSIM
SSIM
The structural similarity index is a method for measuring the similarity between two images. The SSIM index is a full reference metric, in other words, the measuring of image quality based on an initial uncompressed or distortion-free image as reference...
, VQuad-HD and CZD
CZD
CZD as PSNR is a per-pixel quality metric . Described in literature as being “useful for comparing vectors with strictly non-negative elements” it measures the similarity among different samples...
. Based on a benchmark by the Video Quality Experts Group (VQEG) in the course of the Multimedia Test Phase 2007-2008 some metrics were standardized as ITU-T Rec. J.246 (RR), J.247 (FR) in 2008 and J.341 (FR HD) in 2011.
The performance of an objective video quality metric is evaluated by computing the correlation between the objective scores and the subjective
Subjective video quality
Subjective video quality is a subjective characteristic of video quality. It is concerned with how video is perceived by a viewer and designates his or her opinion on a particular video sequence...
test results. The latter is called mean opinion score
Mean Opinion Score
The Mean Opinion Score test has been used for decades in telephony networks to obtain the human user's view of the quality of the network. In multimedia especially when codecs are used to compress the bandwidth requirement , the mean opinion score ...
(MOS). The most frequently used correlation coefficients are : linear correlation coefficient
Correlation
In statistics, dependence refers to any statistical relationship between two random variables or two sets of data. Correlation refers to any of a broad class of statistical relationships involving dependence....
, Spearman's rank correlation coefficient
Spearman's rank correlation coefficient
In statistics, Spearman's rank correlation coefficient or Spearman's rho, named after Charles Spearman and often denoted by the Greek letter \rho or as r_s, is a non-parametric measure of statistical dependence between two variables. It assesses how well the relationship between two variables can...
, kurtosis
Kurtosis
In probability theory and statistics, kurtosis is any measure of the "peakedness" of the probability distribution of a real-valued random variable...
, kappa coefficient and outliers ratio
Outliers Ratio
In objective video quality assessment, the outliers ratio is a measure of the performance of an objective video quality metric. It is the ratio of "false" scores given by the objective metric to the total number of scores. The "false" scores are the scores that lie outside the intervalwhere MOS...
.
When estimating quality of a video codec, all the mentioned objective methods may require repeating post-encoding tests in order to determine the encoding parameters that satisfy a required level of visual quality, making them time consuming, complex and impractical for implementation in real commercial applications.
For this reason, much research has been focused on developing novel objective evaluation methods which enable prediction of the perceived quality level of the encoded video before the actual encoding is performed http://www.springerlink.com/content/f8v2p4r8522664l5.
Subjective video quality
The main goal of many objective video quality metrics is to automatically estimate average user (viewer) opinion on a quality of video processed by the system. Sometimes however, measurement of subjective video quality can also be challenging because it may require a trained expert to judge it. Many “subjective video qualitySubjective video quality
Subjective video quality is a subjective characteristic of video quality. It is concerned with how video is perceived by a viewer and designates his or her opinion on a particular video sequence...
measurements” are described in ITU-T
ITU-T
The ITU Telecommunication Standardization Sector is one of the three sectors of the International Telecommunication Union ; it coordinates standards for telecommunications....
recommendation BT.500. Their main idea is the same as in Mean Opinion Score
Mean Opinion Score
The Mean Opinion Score test has been used for decades in telephony networks to obtain the human user's view of the quality of the network. In multimedia especially when codecs are used to compress the bandwidth requirement , the mean opinion score ...
for audio: video sequences are shown to the group of viewers and then their opinion is recorded and averaged to evaluate the quality of each video sequence. However, details of testing may vary greatly.
See also
- Subjective video qualitySubjective video qualitySubjective video quality is a subjective characteristic of video quality. It is concerned with how video is perceived by a viewer and designates his or her opinion on a particular video sequence...
- Video codecs
- Mean Opinion ScoreMean Opinion ScoreThe Mean Opinion Score test has been used for decades in telephony networks to obtain the human user's view of the quality of the network. In multimedia especially when codecs are used to compress the bandwidth requirement , the mean opinion score ...
- PSNRPeak signal-to-noise ratioThe phrase peak signal-to-noise ratio, often abbreviated PSNR, is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation...
- SSIM
- Perceptual Evaluation of Video Quality (PEVQ, ITU-T J.247)PEVQPEVQ ' is a standardized end-to-end measurement algorithm to score the picture quality of a video presentation by means of a 5-point mean opinion score...
- MDIMedia Delivery IndexThe Media Delivery Index is a set of measures that can be used to monitor both the quality of a delivered video stream as well as to show system margin for IPTV systems by providing an accurate measurement of jitter and delay at network level , which are the main causes for quality loss...