Demosaicing

From Wikipedia, the free encyclopedia

 This article does not cite any references or sources. (September 2007)
Please help improve this article by adding citations to reliable sources. Unverifiable material may be challenged and removed.

A demosaicing algorithm is a digital image process used to interpolate a complete image from the partial raw data received from the color-filtered image sensor (via a color filter array or CFA) internal to many digital cameras in form of a matrix of colored pixels. Also known as CFA interpolation or color reconstruction, another common spelling is demosaicking.

Many modern digital cameras provide a raw file with data in a filter-mosaic format; the user can demosaic it using software, rather than using the camera's built-in firmware.

Contents

[edit] Goal

The Bayer arrangement of color filters on the pixel array of an image sensor. Each two-by-two submosaic contains two green, one blue, and one red filter.
The Bayer arrangement of color filters on the pixel array of an image sensor. Each two-by-two submosaic contains two green, one blue, and one red filter.

There are a number of different ways the pixel filters are arranged in practice, the most common being the Bayer filter as in the image at the right, alternating values of Red (R) and Green (G) for odd rows and alternating values of Green (G) and Blue (B) for even rows.

Since each pixel of the sensor is behind a color filter, the output is an array of pixel values, each indicating a raw intensity of one of three primary colors. Thus, an algorithm is needed to estimate for each pixel the color levels for all color components, rather than a single component.

[edit] Tradeoffs

Some methods may produce better results for natural scenes, and some for printed material, for instance. This reflects the inherent problem in estimating pixels that we do not really know for certain.[citation needed]

Naturally there is also the ubiquitous tradeoff of speed versus quality of estimation.[citation needed]

[edit] Illustration

To reconstruct a color image from the data collected by the color filtering array, a form of interpolation is needed to fill in the blanks. The mathematics here is subject to individual implementation, and is called demosaicing. If you have a RAW image, you can use different demosaicing than is built into the camera, often yielding higher quality.

In this example, we use Adobe Photoshop's bicubic interpolation to simulate the circuitry of a Bayer filter device such as a digital camera. In a typical commercial implementation, low pass anti-alias filters will be added that make the artifacts shown here less pronounced, with a corresponding reduction of sharpness.

Below is a simulated image sampling taken by a Bayer filtered sensor array (the corresponding original image is shown at the end, by the demosaiced reconstruction, for comparison); each pixel only has a value of either R or G or B:

Bayer filter samples
Red Green Blue

A digital camera typically has means to reconstruct a whole RGB image using the above information. The resulting image could be something like this:

Original Reconstructed

The reconstructed image is typically accurate in uniform-colored areas, but has a loss of resolution (detail and sharpness) and has edge artifacts (for example, the edges of letters have visible color fringes and some roughness).

[edit] Algorithms

[edit] Quick (low-grade)

  • Nearest neighbor replication simply copies an adjacent pixel of the correct color component. It is unsuitable for any application where quality matters, but can be useful for continuous previews given limited computational resources.

[edit] Simple interpolation

These algorithms are examples of multivariate interpolation on a uniform grid, using relatively straightforward mathematical operations using only nearby instances of the same color component. The simplest is the bilinear interpolation method. In this method, the red value of a non-red pixel is computed as the average of the two or four adjacent red pixels, and similarly for blue and green. Slightly fancier methods that interpolate independently within each color plane include bicubic interpolation, spline interpolation, and Lanczos resampling.

[edit] Adaptive

These algorithms adapt their method of estimation (i.e. the estimation formula) depending on features of the area surrounding the pixel of interest.[citation needed]

  • Variable Number of Gradients interpolation computes gradients near the pixel of interest and uses the lower gradients (representing smoother and more similar parts of the image) to make an estimate. It is used in first versions of dcraw, and suffers from color artifacts .
  • Pixel Grouping uses assumptions about natural scenery in making estimates. It has fewer color artifacts on natural images than the Variable Number of Gradients method; it was introduced in dcraw from rel. 8.71 as "Patterned Pixel Grouping".
  • Adaptive homogeneity-directed interpolation selects the direction of interpolation so as to maximize a homogeneity metric, thus typically minimizing color artifacts.[1] It has been implemented in recent versions of dcraw.[2]

[edit] Video super-resolution/demosaicing

It has recently shown that the super-resolution and demosaicing are two faces of a same problem and it is reasonable to address them in a unified context.[3] Note that both these problems address the aliasing issue. Therefore, specially in case of video (multi-frame) reconstruction, a joint super-resolution and demosaicing approach provides the optimal solution.

[edit] Proprietary

Various commercial products implement proprietary estimation methods about which little is publicly known, and which may or may not be similar to publicly known algorithms.[citation needed]

[edit] References

  1. ^ Keigo Hirakawa's home page. See AHD Demosaicing.
  2. ^ Decoding raw digital photos in Linux, Dave Coffin.
  3. ^ Sina Farsiu, Michael Elad, Peyman Milanfar (2006). "Multi-Frame Demosaicing and Super-Resolution of Color Images". IEEE Trans. on Image Processing 15 (1): 141-159. 

[edit] External links

Languages