Computational photography
Encyclopedia
Computational imaging refers to any image formation method that involves a digital computer. Computational photography refers broadly to computational imaging techniques that enhance or extend the capabilities of
digital photography
. The output of these techniques is an ordinary photograph, but one that could not have been taken by a traditional camera
.
Its current definition, which stems from a 2004 course at
Stanford University and a 2005 symposium at MIT (see links below),
has evolved to cover a number of
subject areas in computer graphics
, computer vision
, and applied
optics
. These areas are given below, organized according to a taxonomy
proposed by Shree K. Nayar
. Within each area is a list of techniques, and for
each technique one or two representative papers or books are cited.
Deliberately omitted from the
taxonomy are image processing
(see also digital image processing
)
techniques applied to traditionally captured
images in order to produce better images. Examples of such techniques are
image scaling
, dynamic range compression (i.e. tone mapping
),
color management
, image completion (a.k.a. inpainting or hole filling),
image compression
, digital watermarking
, and artistic image effects.
Also omitted are techniques that produce range data
,
volume data
, 3D models, 4D light fields
,
4D, 6D, or 8D BRDF
s, or other high-dimensional image-based representations.
to create new images. The applications include image-based relighting, image enhancement, image deblurring, geometry/material recovery and so forth.
Coded aperture
imaging was mainly applied in astronomy or X-ray imaging to boost the image quality. Instead of a single pin-hole, a pinhole pattern is applied in imaging, and deconvolution
is performed to recover the image. In coded exposure imaging, the on/off state of the shutter is coded to modify the kernel of motion blur
. In this way motion deblurring becomes a well-conditioned problem. Similarly, in a lens based coded aperture, the aperture can be modified by inserting a broadband mask. Thus, out of focus deblurring becomes a well-conditioned problem. The coded aperture can also improve the quality in light field acquisition using Hadamard transform optics.
techniques first appeared in the computer vision literature,
either under other names or within papers aimed at 3D shape analysis.
digital photography
Digital photography
Digital photography is a form of photography that uses an array of light sensitive sensors to capture the image focused by the lens, as opposed to an exposure on light sensitive film...
. The output of these techniques is an ordinary photograph, but one that could not have been taken by a traditional camera
Camera
A camera is a device that records and stores images. These images may be still photographs or moving images such as videos or movies. The term camera comes from the camera obscura , an early mechanism for projecting images...
.
Its current definition, which stems from a 2004 course at
Stanford University and a 2005 symposium at MIT (see links below),
has evolved to cover a number of
subject areas in computer graphics
Computer graphics
Computer graphics are graphics created using computers and, more generally, the representation and manipulation of image data by a computer with help from specialized software and hardware....
, computer vision
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
, and applied
optics
Optics
Optics is the branch of physics which involves the behavior and properties of light, including its interactions with matter and the construction of instruments that use or detect it. Optics usually describes the behavior of visible, ultraviolet, and infrared light...
. These areas are given below, organized according to a taxonomy
proposed by Shree K. Nayar
Shree K. Nayar
Shree K. Nayar is an engineer and computer scientist known for his work in the fields of computer vision, computer graphics and computational cameras. He is the T. C. Chang Professor of the at Columbia University. Nayar co-directs the and is the head of the , which develops advanced computer...
. Within each area is a list of techniques, and for
each technique one or two representative papers or books are cited.
Deliberately omitted from the
taxonomy are image processing
Image processing
In electrical engineering and computer science, image processing is any form of signal processing for which the input is an image, such as a photograph or video frame; the output of image processing may be either an image or, a set of characteristics or parameters related to the image...
(see also digital image processing
Digital image processing
Digital image processing is the use of computer algorithms to perform image processing on digital images. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing...
)
techniques applied to traditionally captured
images in order to produce better images. Examples of such techniques are
image scaling
Image scaling
In computer graphics, image scaling is the process of resizing a digital image. Scaling is a non-trivial process that involves a trade-off between efficiency, smoothness and sharpness. As the size of an image is increased, so the pixels which comprise the image become increasingly visible, making...
, dynamic range compression (i.e. tone mapping
Tone mapping
Tone mapping is a technique used in image processing and computer graphics to map one set of colors to another in order to approximate the appearance of high dynamic range images in a medium that has a more limited dynamic range...
),
color management
Color management
In digital imaging systems, color management is the controlled conversion between the color representations of various devices, such as image scanners, digital cameras, monitors, TV screens, film printers, computer printers, offset presses, and corresponding media.The primary goal of color...
, image completion (a.k.a. inpainting or hole filling),
image compression
Image compression
The objective of image compression is to reduce irrelevance and redundancy of the image data in order to be able to store or transmit data in an efficient form.- Lossy and lossless compression :...
, digital watermarking
Digital watermarking
Digital watermarking is the process of embedding information into a digital signal which may be used to verify its authenticity or the identity of its owners, in the same manner as paper bearing a watermark for visible identification. In digital watermarking, the signal may be audio, pictures, or...
, and artistic image effects.
Also omitted are techniques that produce range data
3D scanner
A 3D scanner is a device that analyzes a real-world object or environment to collect data on its shape and possibly its appearance . The collected data can then be used to construct digital, three dimensional models....
,
volume data
Voxel
A voxel is a volume element, representing a value on a regular grid in three dimensional space. This is analogous to a pixel, which represents 2D image data in a bitmap...
, 3D models, 4D light fields
Light field
The light field is a function that describes the amount of light faring in every direction through every point in space. Michael Faraday was the first to propose that light should be interpreted as a field, much like the magnetic fields on which he had been working for several years...
,
4D, 6D, or 8D BRDF
Bidirectional reflectance distribution function
The bidirectional reflectance distribution function is a four-dimensional function that defines how light is reflected at an opaque surface...
s, or other high-dimensional image-based representations.
Computational illumination
This is controlling photographic illumination in a structured fashion, then processing the captured images,to create new images. The applications include image-based relighting, image enhancement, image deblurring, geometry/material recovery and so forth.
Computational optics
This is capture of optically coded images, followed by computational decoding to produce new images.Coded aperture
Coded aperture
Coded Apertures or Coded-Aperture Masks are grids, gratings, or other patterns of materials opaque to various wavelengths of light. The wavelengths are usually high-energy radiation such as X-rays and gamma rays. By blocking and unblocking light in a known pattern, a coded "shadow" is cast upon a...
imaging was mainly applied in astronomy or X-ray imaging to boost the image quality. Instead of a single pin-hole, a pinhole pattern is applied in imaging, and deconvolution
Deconvolution
In mathematics, deconvolution is an algorithm-based process used to reverse the effects of convolution on recorded data. The concept of deconvolution is widely used in the techniques of signal processing and image processing...
is performed to recover the image. In coded exposure imaging, the on/off state of the shutter is coded to modify the kernel of motion blur
Motion blur
Motion blur is the apparent streaking of rapidly moving objects in a still image or a sequence of images such as a movie or animation. It results when the image being recorded changes during the recording of a single frame, either due to rapid movement or long exposure.- Photography :When a camera...
. In this way motion deblurring becomes a well-conditioned problem. Similarly, in a lens based coded aperture, the aperture can be modified by inserting a broadband mask. Thus, out of focus deblurring becomes a well-conditioned problem. The coded aperture can also improve the quality in light field acquisition using Hadamard transform optics.
Computational processing
This is processing of non-optically-coded images to produce new images.Computational sensors
These are detectors that combine sensing and processing, typically in hardware.Early work in computer vision
Although computational photography is a currently popular buzzword in computer graphics, many of itstechniques first appeared in the computer vision literature,
either under other names or within papers aimed at 3D shape analysis.
External links
- Computational Photography (Raskar, R., Tumblin, J.,), A.K. Peters. In press.
- Special issue on Computational Photography, IEEE Computer, August 2006.
- Camera Culture and Computational Journalism: Capturing and Sharing Visual Experiences, IEEE CG&A Special Issue, Feb 2011.