Subjective video quality
Encyclopedia
Subjective video quality is a subjective characteristic of video quality
. It is concerned with how video is perceived by a viewer and designates his or her opinion on a particular video
sequence. Subjective video quality tests are quite expensive in terms of time (preparation and running) and human resources.
for audio
.
To evaluate the subjective video quality of a video processing system:
Many parameters of the viewing conditions can influence the results, such as room illumination, display type, brightness, contrast, resolution, viewing distance, and the age and educational level of experts.
recommendation BT.500. One of the standardized methods is DSIS - Double Stimulus Impairment Scale: the expert is presented with an unimpaired reference video, then with the same video impaired, and after that he is asked to vote on the second video using an impairment scale (from "impairments are imperceptible" to "impairments are very annoying").
Further refined testing methods to take into account low picture resolutions (VGA, CIF and QCIF), e.g. for mobile and multimedia applications are referred to in ITU-T Rec. P.910. Advanced setups for typical artifacts of high resolution (HDTV), e.g. in next generation networks incl. IPTV are also under development within the Video Quality Experts Group (VQEG).
. Additional procedures can be used for averaging, for example experts who give unstable results can be rejected (for instance, if their correlation with average opinion is small)...
Video quality
Video quality is a characteristic of a video passed through a video transmission/processing system, a formal or informal measure of perceived video degradation...
. It is concerned with how video is perceived by a viewer and designates his or her opinion on a particular video
Video
Video is the technology of electronically capturing, recording, processing, storing, transmitting, and reconstructing a sequence of still images representing scenes in motion.- History :...
sequence. Subjective video quality tests are quite expensive in terms of time (preparation and running) and human resources.
Measurement
The main idea of measuring subjective video quality is the same as in the Mean Opinion ScoreMean Opinion Score
The Mean Opinion Score test has been used for decades in telephony networks to obtain the human user's view of the quality of the network. In multimedia especially when codecs are used to compress the bandwidth requirement , the mean opinion score ...
for audio
Sound
Sound is a mechanical wave that is an oscillation of pressure transmitted through a solid, liquid, or gas, composed of frequencies within the range of hearing and of a level sufficiently strong to be heard, or the sensation stimulated in organs of hearing by such vibrations.-Propagation of...
.
To evaluate the subjective video quality of a video processing system:
- Choose video sequences for testing (they are often named SRC).
- Choose settings of system that you want to evaluate (often named HRC or Hypothetical Reference Circuit).
- Choose a test methodScientific methodScientific method refers to a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge. To be termed scientific, a method of inquiry must be based on gathering empirical and measurable evidence subject to specific principles of...
(how sequences are presented to experts and how their opinion is collected: DSCQS, SSCQE, DSCS). - Invite sufficient number of experts (18 or more is recommended).
- Carry out testing.
- Calculate average marks for each HRC based on experts' opinion.
Many parameters of the viewing conditions can influence the results, such as room illumination, display type, brightness, contrast, resolution, viewing distance, and the age and educational level of experts.
Testing methods
There are an enormous number of ways of showing video sequences to experts and to record their opinions. A few of them have been standardized. They are thoroughly described in ITU-RITU-R
The ITU Radiocommunication Sector is one of the three sectors of the International Telecommunication Union and is responsible for radio communication....
recommendation BT.500. One of the standardized methods is DSIS - Double Stimulus Impairment Scale: the expert is presented with an unimpaired reference video, then with the same video impaired, and after that he is asked to vote on the second video using an impairment scale (from "impairments are imperceptible" to "impairments are very annoying").
Further refined testing methods to take into account low picture resolutions (VGA, CIF and QCIF), e.g. for mobile and multimedia applications are referred to in ITU-T Rec. P.910. Advanced setups for typical artifacts of high resolution (HDTV), e.g. in next generation networks incl. IPTV are also under development within the Video Quality Experts Group (VQEG).
Analysis of results
Opinions of experts can be averaged; average mark is usually given with confidence intervalConfidence interval
In statistics, a confidence interval is a particular kind of interval estimate of a population parameter and is used to indicate the reliability of an estimate. It is an observed interval , in principle different from sample to sample, that frequently includes the parameter of interest, if the...
. Additional procedures can be used for averaging, for example experts who give unstable results can be rejected (for instance, if their correlation with average opinion is small)...
See also
- Mean Opinion ScoreMean Opinion ScoreThe Mean Opinion Score test has been used for decades in telephony networks to obtain the human user's view of the quality of the network. In multimedia especially when codecs are used to compress the bandwidth requirement , the mean opinion score ...
- Peak signal-to-noise ratioPeak signal-to-noise ratioThe phrase peak signal-to-noise ratio, often abbreviated PSNR, is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation...
(PSNR) - Perceptual Evaluation of Video Quality (PEVQPEVQPEVQ ' is a standardized end-to-end measurement algorithm to score the picture quality of a video presentation by means of a 5-point mean opinion score...
) - Media Delivery IndexMedia Delivery IndexThe Media Delivery Index is a set of measures that can be used to monitor both the quality of a delivered video stream as well as to show system margin for IPTV systems by providing an accurate measurement of jitter and delay at network level , which are the main causes for quality loss...
(MDI) - Structural similarity (SSIM)
- Absolute Category RatingAbsolute Category RatingAbsolute Category Rating is a test method used in quality tests. It has been standardized in ITU-T Recommendation P.910. In this method, a single test condition is presented to the viewers once only. They should then give a quality rating on an ACR scale. Test conditions should be presented in...
- Lumi maskingLumi maskingLumi masking is a technique used by video compression software, which reduces quality in very bright or very dark areas of the picture, as quality loss in these areas is less likely to be visible. It is also known as "psychovisual enhancements" or "adaptive quantization"...
- PsychophysicsPsychophysicsPsychophysics quantitatively investigates the relationship between physical stimuli and the sensations and perceptions they effect. Psychophysics has been described as "the scientific study of the relation between stimulus and sensation" or, more completely, as "the analysis of perceptual...
Further reading
- ITU-R BT.500-12 Recommendation: Methodology for the subjective assessment of the quality of television pictures
- Comparing subjective video quality testing methodologies
- IPTV MPEG and Quality of Experience Testing