Light-field camera

Front and back of a Lytro, the first consumer light-field camera, showing the front lens and LCD touchscreen.

A light-field camera, also called a plenoptic camera, is a camera that uses a microlens array to capture 4D light field information about a scene. Such light field information can be used to improve the solution of computer graphics and computer vision-related problems, and to make digital plenoptic pictures that can be refocused after they are taken.

Technology

The first light-field camera was proposed by Gabriel Lippmann in 1908, which used integral photography as the underlying technology. In 1992, Adelson and Wang proposed the design of a plenoptic camera that can be used to significantly reduce the correspondence problem in stereo matching.^[1] To achieve this, an array of microlenses is placed at the focal plane of the camera main lens. The image sensor is positioned slightly behind the microlenses. Using such images the displacement of image parts that are not in focus can be analyzed and depth information can be extracted.

Potentially, this camera system can be used to refocus an image virtually on a computer after the picture has been taken, as explained by Ng et al.^[2] The drawback of such a system is the low resolution that the final images have. As one microlens samples the light directions at one spatial point an increase in the number of image pixels can only be done by increasing the number of microlenses by the same amount.^[3]

To overcome this limitation, Lumsdaine and Georgiev describe a new design of a plenoptic camera, called focused plenoptic camera where the microlens array is positioned before or behind the focal plane of the main lens. This modification samples the light field in a different way that allows for a higher spatial resolution by having a lower angular resolution at the same time. With this design images can be refocused with a much higher spatial resolution. However, the low angular resolution can introduce some unwanted aliasing artifacts.

A different design using low-cost printed film (mask) instead of microlens array was proposed by researchers at MERL in 2007.^[4] This design overcomes several limitations of microlens array in terms of chromatic aberrations, loss of boundary pixels, and allows higher-spatial-resolution photos to be captured. However, a mask-based design reduces light compared to microlens arrays.

Image refocusing

In 2004, a team at Stanford University Computer Graphics Laboratory used a 16-megapixel camera with a 90,000-microlens array (meaning that each microlens covers about 175 pixels, and the final resolution is 90 kilopixels) to demonstrate that pictures can be refocused after they are taken.^[2]

Stereo with plenoptic cameras

Plenoptic cameras are good for following moving objects.^[5] A recording from a security camera based upon this technology could be used to produce an accurate 3D model of a suspect.^[6]

Manufacturers of light-field cameras

Cameras available for purchase

From 2010 on, Raytrix has sold several models of plenoptic camera for industrial and scientific applications, with resolutions starting from 1 megapixel.^[7][8]

Lytro was founded by Stanford University Computer Graphics Laboratory alumnus Ren Ng to commercialize the light-field camera he developed as a graduate student there. Lytro has developed a consumer light-field "digital camera capable of images using a plenoptic technique.^[9]

Other cameras

The Adobe light-field camera is a prototype 100-megapixel camera that takes a three-dimensional photo of the scene in focus using 19 uniquely configured lenses. Each lens will take a 5.2-megapixel photo of the entire scene around the camera and each image can be focused later in any way.^[10]

The CAFADIS camera is a plenoptic camera developed by the University of La Laguna (Spain).^[11] CAFADIS stands (in Spanish) for phase-distance camera, since it can be used for distance and optical wavefront estimation. From a single shot it can produce several images refocused at different distances, depth maps, all-in-focus images and stereo pairs. A similar optical design can also be used in adaptive optics in astrophysics, in order to correct the aberrations caused by atmospheric turbulence in telescope images. In order to perform these tasks, different algorithms, running on GPU and FPGA, operate on the raw image captured by the camera.

Mitsubishi Electric Research Laboratories's (MERL) light-field camera^[4] is based on the principle of optical heterodyning and uses a printed film (mask) placed close to the sensor. Any hand-held camera can be converted into a light-field camera using this technology by simply inserting a low-cost film on top of the sensor.^[12] A mask-based design avoids the problem of loss of resolution, since a high-resolution photo can be generated for the focused parts of the scene.

Modern approaches to light field acquisition include mask-based devices that capture an optimized optical projection of a light field and reconstruct the high-resolution light field via compressive sensing.^[13]

Cameras built using angle-sensitive pixels can perform single-lens, single-sensor 3D photography.^[14]

In smartphone

Nokia has invested in Pelican Imaging to produce a plenoptic camera system with 16-lens array camera and predicted will be implemented in smartphone in 2014.^[15]

Light-field microscope

Stanford University Computer Graphics Laboratory has developed a light-field microscope using a microlens array similar to the one used in the light-field camera developed by the lab. The prototype is built around a Nikon Eclipse transmitted light microscope/wide-field fluorescence microscope and standard CCD cameras. Light-field capturing ability is obtained by a module containing a microlens array and other optical components placed in the light path between the objective lens and camera, with the final multifocused image rendered using deconvolution.^[16][17][18] A later version of the prototype added a light-field illumination system consisting of a video projector (allowing computational control of illumination) and a second microlens array in the illumination light path of the microscope. The addition of a light-field illumination system both allowed for additional types of illumination (such as oblique illumination and quasi-dark-field) and correction for optical aberrations.^[17]

See also

• Angle-sensitive pixel
• Compound eye
• Integral imaging

References

External links

• Article by Ren Ng of Stanford (now at Lytro)
• Say Sayonara to Blurry Pics. Wired.
• Fourier slice photography
• Light Field Microscopy video by Stanford Computer Graphics Laboratory.
• IEEE Spectrum article May 2012 Lightfield photography revolutionizes imaging, with sample images and diagrams of operation, retrieved 2012 May 11