In photography

Photography

Photography is the art, science and practice of creating durable images by recording light or other electromagnetic radiation, either electronically by means of an image sensor or chemically by means of a light-sensitive material such as photographic film...

 and image processing

Image processing

In electrical engineering and computer science, image processing is any form of signal processing for which the input is an image, such as a photograph or video frame; the output of image processing may be either an image or, a set of characteristics or parameters related to the image...

, color balance is the global adjustment of the intensities of the colors (typically red, green, and blue primary colors

Primary Colors

Primary Colors: A Novel of Politics is a roman à clef, a work of fiction that purports to describe real life characters and events — namely, Bill Clinton's first presidential campaign in 1992...

). An important goal of this adjustment is to render specific colors – particularly neutral colors – correctly; hence, the general method is sometimes called gray balance, neutral balance, or white balance. Color balance changes the overall mixture of colors in an image and is used for color correction; generalized versions of color balance are used to get colors other than neutrals to also appear correct or pleasing.

Image data acquired by sensors – either film

Photographic film

Photographic film is a sheet of plastic coated with an emulsion containing light-sensitive silver halide salts with variable crystal sizes that determine the sensitivity, contrast and resolution of the film...

 or electronic image sensor

Image sensor

An image sensor is a device that converts an optical image into an electronic signal. It is used mostly in digital cameras and other imaging devices...

s – must be transformed from the acquired values to new values that are appropriate for color reproduction or display. Several aspects of the acquisition and display process make such color correction essential – including the fact that the acquisition sensors do not match the sensors in the human eye, that the properties of the display medium must be accounted for, and that the ambient viewing conditions of the acquisition differ from the display viewing conditions.

The color balance operations in popular image editing

Image editing

Image editing encompasses the processes of altering images, whether they be digital photographs, traditional analog photographs, or illustrations. Traditional analog image editing is known as photo retouching, using tools such as an airbrush to modify photographs, or editing illustrations with any...

 applications usually operate directly on the red, green, and blue channel pixel

Pixel

In digital imaging, a pixel, or pel, is a single point in a raster image, or the smallest addressable screen element in a display device; it is the smallest unit of picture that can be represented or controlled....

 values, without respect to any color sensing or reproduction model. In shooting film, color balance is typically achieved by using color correction filters over the lights or on the camera lens.

Generalized color balance

Sometimes the adjustment to keep neutrals neutral is called white balance, and the phrase color balance refers to the adjustment that in addition makes other colors in a displayed image appear to have the same general appearance as the colors in an original scene. It is particularly important that neutral (gray, achromatic, white) colors in a scene appear neutral in the reproduction. Hence, the special case of balancing the neutral colors (sometimes gray balance, neutral balance, or white balance) is a particularly important – perhaps dominant – element of color balancing.

Normally, one would not use the phrase color balance to describe the adjustments needed to account for differences between the sensors and the human eye, or the details of the display primaries. Color balance is normally reserved to refer to correction for differences in the ambient illumination conditions. However, the algorithms for transforming the data do not always clearly separate out the different elements of the correction. Hence, it can be difficult to assign color balance to a specific step in the color correction process. Moreover, there can be significant differences in the color balancing goal. Some applications are created to produce an accurate rendering – as suggested above. In other applications, the goal of color balancing is to produce a pleasing rendering. This difference also creates difficulty in defining the color balancing processing operations.

Illuminant estimation and adaptation

Most digital cameras have a means to select a color correction based on the type of scene illumination, using either manual illuminant selection, or automatic white balance (AWB), or custom white balance. The algorithm that performs this analysis performs generalized color balancing, known as illuminant adaptation or chromatic adaptation

Chromatic adaptation

Chromatic adaptation is one aspect of vision that may fool someone into observing a color-based optical illusion, such as the same color illusion.An object may be viewed under various conditions. For example, it may be illuminated by sunlight, the light of a fire, or a harsh electric light...

.

Many methods are used to achieve color balancing. Setting a button on a camera is a way for the user to indicate to the processor the nature of the scene lighting. Another option on some cameras is a button which one may press when the camera is pointed at a gray card

Gray card

A gray card is a middle gray reference, typically used together with a reflective light meter, as a way to produce consistent image exposure and/or color in film and photography....

 or other neutral object. This "custom white balance" step captures an image of the ambient light, and this information is helpful in controlling color balance.

There is a large literature on how one might estimate the ambient illumination from the camera data and then use this information to transform the image data. A variety of algorithms have been proposed, and the quality of these have been debated. A few examples and examination of the references therein will lead the reader to many others. Examples are Retinex, an artificial neural network

Artificial neural network

An artificial neural network , usually called neural network , is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes...

 or a Bayesian method.

Color balance and chromatic colors

Color balancing an image affects not only the neutrals, but other colors as well. An image that is not color balanced is said to have a color cast, as everything in the image appears to have been shifted towards one color or another. Color balancing may be thought in terms of removing this color cast.

Color balance is also related to color constancy

Color constancy

Color constancy is an example of subjective constancy and a feature of the human color perception system which ensures that the perceived color of objects remains relatively constant under varying illumination conditions. A green apple for instance looks green to us at midday, when the main...

. Algorithms and techniques used to attain color constancy are frequently used for color balancing, as well. Color constancy is, in turn, related to chromatic adaptation

Chromatic adaptation

Chromatic adaptation is one aspect of vision that may fool someone into observing a color-based optical illusion, such as the same color illusion.An object may be viewed under various conditions. For example, it may be illuminated by sunlight, the light of a fire, or a harsh electric light...

. Conceptually, color balancing consists of two steps: first, determining the illuminant

Standard illuminant

A standard illuminant is a theoretical source of visible light with a profile which is published. Standard illuminants provide a basis for comparing images or colors recorded under different lighting.-CIE illuminants:...

 under which an image was captured; and second, scaling the components (e.g., R, G, and B) of the image or otherwise transforming the components so they conform to the viewing illuminant.

Viggiano found that white balancing in the camera's native RGB tended to produce less color inconstancy (i.e., less distortion of the colors) than in monitor RGB for over 4000 hypothetical sets of camera sensitivities. This difference typically amounted to a factor of more than two in favor of camera RGB. This means that it is advantageous to get color balance right at the time an image is captured, rather than edit later on a monitor. If one must color balance later, balancing the raw image data

RAW image format

A camera raw image file contains minimally processed data from the image sensor of either a digital camera, image scanner, or motion picture film scanner. Raw files are so named because they are not yet processed and therefore are not ready to be printed or edited with a bitmap graphics editor...

 will tend to produce less distortion of chromatic colors than balancing in monitor RGB.

Mathematics of color balance

Color balancing is sometimes performed on a three-component image (e.g., RGB

RGB color model

The RGB color model is an additive color model in which red, green, and blue light is added together in various ways to reproduce a broad array of colors...

) using a 3x3 matrix

Matrix (mathematics)

In mathematics, a matrix is a rectangular array of numbers, symbols, or expressions. The individual items in a matrix are called its elements or entries. An example of a matrix with six elements isMatrices of the same size can be added or subtracted element by element...

. This type of transformation is appropriate if the image were captured using the wrong white balance setting on a digital camera, or through a color filter.

Scaling monitor R, G, and B

In principle, one wants to scale all relative luminances in an image so that objects which are believed to be neutral

Grey

Grey or gray is an achromatic or neutral color.Complementary colors are defined to mix to grey, either additively or subtractively, and many color models place complements opposite each other in a color wheel. To produce grey in RGB displays, the R, G, and B primary light sources are combined in...

 appear so. If, say, a surface with was believed to be a white object, and if 255 is the count which corresponds to white, one could multiply all red

Red

Red is any of a number of similar colors evoked by light consisting predominantly of the longest wavelengths of light discernible by the human eye, in the wavelength range of roughly 630–740 nm. Longer wavelengths than this are called infrared , and cannot be seen by the naked eye...

 values by 255/240. Doing analogously for green

Green

Green is a color, the perception of which is evoked by light having a spectrum dominated by energy with a wavelength of roughly 520–570 nanometres. In the subtractive color system, it is not a primary color, but is created out of a mixture of yellow and blue, or yellow and cyan; it is considered...

 and blue

Blue

Blue is a colour, the perception of which is evoked by light having a spectrum dominated by energy with a wavelength of roughly 440–490 nm. It is considered one of the additive primary colours. On the HSV Colour Wheel, the complement of blue is yellow; that is, a colour corresponding to an equal...

 would result, at least in theory, in a color balanced image. In this type of transformation the 3x3 matrix is a diagonal matrix

Diagonal matrix

In linear algebra, a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero. The diagonal entries themselves may or may not be zero...

.

where , , and are the color balanced red, green, and blue components of a pixel

Pixel

In digital imaging, a pixel, or pel, is a single point in a raster image, or the smallest addressable screen element in a display device; it is the smallest unit of picture that can be represented or controlled....

 in the image; , , and are the red, green, and blue components of the image before color balancing, and , , and are the red, green, and blue components of a pixel which is believed to be a white surface in the image before color balancing. This is a simple scaling of the red, green, and blue channels, and is why color balance tools in Photoshop and the GIMP

GIMP

GIMP is a free software raster graphics editor. It is primarily employed as an image retouching and editing tool and is freely available in versions tailored for most popular operating systems including Microsoft Windows, Apple Mac OS X, and Linux.In addition to detailed image retouching and...

 have a white eyedropper tool. It has been demonstrated that performing the white balancing in the phosphor set assumed by sRGB tends to produce large errors in chromatic colors, even though it can render the neutral surfaces perfectly neutral.

Scaling X, Y, Z

If the image may be transformed into CIE XYZ tristimulus values

CIE 1931 color space

In the study of color perception, one of the first mathematically defined color spaces is the CIE 1931 XYZ color space, created by the International Commission on Illumination in 1931....

, the color balancing may be performed there. This has been termed a “wrong von Kries” transformation. Although it has been demonstrated to offer usually poorer results than balancing in monitor RGB, it is mentioned here as a bridge to other things. Mathematically, one computes:


where , , and are the color-balanced tristimulus values; , , and are the tristimulus values of the viewing illuminant (the white point to which the image is being transformed to conform to); , , and are the tristimulus values of an object believed to be white in the un-color-balanced image, and , , and are the tristimulus values of a pixel in the un-color-balanced image. If the tristimulus values of the monitor primaries are in a matrix so that:


where , , and are the un-gamma corrected

Gamma correction

Gamma correction, gamma nonlinearity, gamma encoding, or often simply gamma, is the name of a nonlinear operation used to code and decode luminance or tristimulus values in video or still image systems...

 monitor RGB, one may use:

Von Kries's method

Johannes von Kries

Johannes von Kries

Johannes Adolf von Kries was a German physiological psychologist who formulated the modern “duplicity” or “duplexity” theory of vision mediated by rod cells at low light levels and three types of cone cells at higher light levels...

, whose theory of rods

Rod cell

Rod cells, or rods, are photoreceptor cells in the retina of the eye that can function in less intense light than can the other type of visual photoreceptor, cone cells. Named for their cylindrical shape, rods are concentrated at the outer edges of the retina and are used in peripheral vision. On...

 and three color-sensitive cone

Cone cell

Cone cells, or cones, are photoreceptor cells in the retina of the eye that are responsible for color vision; they function best in relatively bright light, as opposed to rod cells that work better in dim light. If the retina is exposed to an intense visual stimulus, a negative afterimage will be...

 types in the retina

Retina

The vertebrate retina is a light-sensitive tissue lining the inner surface of the eye. The optics of the eye create an image of the visual world on the retina, which serves much the same function as the film in a camera. Light striking the retina initiates a cascade of chemical and electrical...

 has survived as the dominant explanation of color sensation for over 100 years, motivated the method of converting color to the LMS color space

LMS Color Space

LMS is a color space represented by the response of the three types of cones of the human eye, named after their responsivity at long, medium and short wavelengths....

, representing the effective stimuli for the Long-, Medium-, and Short-wavelength cone types that are modeled as adapting independently. A 3x3 matrix converts RGB or XYZ to LMS, and then the three LMS primary values are scaled to balance the neutral; the color can then be converted back to the desired final color space

Color space

A color model is an abstract mathematical model describing the way colors can be represented as tuples of numbers, typically as three or four values or color components...

:


where , , and are the color-balanced LMS cone tristimulus values; , , and are the tristimulus values of an object believed to be white in the un-color-balanced image, and , , and are the tristimulus values of a pixel in the un-color-balanced image.

Matrices to convert to LMS space were not specified by von Kries, but can be derived from CIE color matching functions and LMS color matching functions when the latter are specified; matrices can also be found in reference books.

Scaling camera RGB

By Viggiano's measure, and using his model of gaussian camera spectral sensitivities, most camera RGB spaces performed better than either monitor RGB or XYZ. If the camera's raw RGB values are known, one may use the 3x3 diagonal matrix:

and then convert to a working RGB space such as sRGB or Adobe RGB after balancing.

Preferred chromatic adaptation spaces

Comparisons of images balanced by diagonal transforms in a number of different RGB spaces have identified several such spaces that work better than others, and better than camera or monitor spaces, for chromatic adaptation, as measured by several color appearance models; the systems that performed statistically as well as the best on the majority of the image test sets used were the "Sharp", "Bradford", "CMCCAT", and "ROMM" spaces.

General illuminant adaptation

The best color matrix for adapting to a change in illuminant is not necessarily a diagonal matrix in a fixed color space. It has long been known that if the space of illuminants can be described as a linear model with N basis terms, the proper color transformation will be the weighted sum of N fixed linear transformations, not necessarily consistently diagonalizable.

External links

• White Balance - Intro at nikondigital.org
• Understanding White Balance - Tutorial
• Affine color balance with saturation, with code and on-line demonstration