2D-plus-depth

2D image and its depth map, lighter depth map areas are regarded as closer to a viewer

2D-plus-Depth or also called 2D + Z format (not to be confused with 2D plus Delta or 2D-plus-DOT) is a Stereoscopic Video Coding format that is used for 3D displays, such as Philips WOWvx. Philips discontinued work on the WOWvx line in 2009, citing "current market developments".^[1] Currently, this Philips technology is used by Dimenco company, led by former key 3D engineers and scientists of Philips. They offer autostereoscopic 3D displays which use the 2D-plus-Depth format for 3D video input.^[2]

Overview

The 2D-plus-Depth format is described in a Philips' white paper^[3] and articles.^[4]

Each 2D image frame is supplemented with a greyscale depth map which indicates if a specific pixel in the 2D image needs to be shown in front of the display (white) or behind the screen plane (black). The 256 greyscales can build a smooth gradient of depth within the image. Processing within the monitor used this input to render the multiview images.

Supported by various companies across the display industry, 2D-plus-Depth has been standardized in MPEG as an extension for 3D filed under ISO/IEC FDIS 23002-3:2007(E).^[5]

There is also an extension on the 2D-plus-Depth format called the WOWvx Declipse format. It is described in the same Philips' white paper "3D Interface Specifications". In this advanced format, two more planes are added to the original 2D image and its depth map for each frame: the background areas covered by foreground objects and their respective depth map. So, each frame in the Declipse format is described with an image containing four parts, or quadrants. This extension improves potential visual quality by providing data for more correct and precise filling of the uncovered occlusion areas created by shifting foreground objects during the multiview generation process.

Advantages

2D-plus-Depth has the advantage that it has a limited bandwidth increase compared to 2D (compressed greyscale increases bandwidth 5–20%) so that it can be used in existing distribution infrastructures.

2D-plus-Depth offers flexibility and compatibility with existing production equipment and compression tools.^[6][7]

It allows applications to use different 3D display screen sizes and designs in the same system.

Another advantage is that depth maps are created in the course of 2D-to-stereo 3D conversion using almost any approach to this video transformation.^[8] That is why there is the respective 2D-plus-Depth representation of a converted stereo footage in almost all cases. Considering the lack of 3D content shot in stereo and the number of converted 3D films, this is a big benefit.

Disadvantages

2D-plus-Depth is not compatible with existing 2D or 3D-Ready displays. The format has been criticized due to the limited amount of depth that can be displayed in an 8-bit greyscale.

2d-plus-Depth cannot handle* transparency (semi-transparent objects in the scene) and occlusion (an object blocking the view of another). The 2d plus DOT format takes these factors into account.^[9] Additionally, it cannot handle reflection, refraction (beyond simple transparency) and other optical phenomena.

Creation of accurate 2D-plus-Depth can be costly and difficult, though recent advances in range imaging have made this process more accessible.^[10][11]

2d-plus-Depth lacks the potential increase in resolution of using two complete images.

Depth cannot be reliably estimated for a monocular video in most cases. Notable exceptions are camera motion scenes when object motion is static or almost absent, and landscape scenes when depth map can be approximated well enough with a gradient. This allows automatic depth estimation.^[12][13] In general case only semi-automatic approach is viable for 2D to 2D-plus-Depth conversion. Philips developed a 3D content creation software suite named BlueBox^[14] which includes semi-automated conversion of 2D content into 2D-plus-Depth format and automatic generation of 2D-plus-Depth from stereo. A similar semiautomatic approach to high quality 2D to 2D-plus-Depth conversion is implemented in YUVsoft's 2D to 3D Suite, available as a set of plugins for After Effects and NUKE video compositing software.^[15]

Stereoscopic to 2D-plus-Depth conversion involves several algorithms including scene change detection, segmentation, motion estimation and image matching. Automatic stereo to 2D+Depth conversion is now possible due to new high performance software and GPU technology, even in live real-time mode.^[16]

Alternatives

Other 3D formats are stereo (left-right or alternating frames) and multiview 3D format (such as Multiview Video Coding and Scalable Video Coding), and 2D plus Delta.

References