Head-mounted display

A head-mounted display (or helmet-mounted display, for aviation applications), both abbreviated HMD, is a display device, worn on the head or as part of a helmet, that has a small display optic in front of one (monocular HMD) or each eye (binocular HMD).

There is also an optical head-mounted display (OHMD), which is a wearable display that has the capability of reflecting projected images as well as allowing the user to see through it.

Overview

A typical HMD has either one or two small displays with lenses and semi-transparent mirrors embedded in a helmet, eyeglasses (also known as data glasses) or visor. The display units are miniaturised and may include CRT, LCDs, Liquid crystal on silicon (LCos), or OLED. Some vendors employ multiple micro-displays to increase total resolution and field of view.

Types

HMDs differ in whether they can display just a computer generated image (CGI), show live images from the real world or a combination of both.

• Most HMDs display only a computer-generated image, sometimes referred to as a virtual image
• Some HMDs allow a CGI to be superimposed on a real-world view. This is sometimes referred to as augmented reality or mixed reality. Combining real-world view with CGI can be done by projecting the CGI through a partially reflective mirror and viewing the real world directly. This method is often called Optical See-Through. Combining real-world view with CGI can also be done electronically by accepting video from a camera and mixing it electronically with CGI. This method is often called Video See-Through.

Optical HMD

An optical head-mounted display uses an optical mixer which is made of partly silvered mirrors. It has the capability of reflecting artificial images as well as letting real images to cross the lens and let the user to look through it.

Various techniques have existed for see-through HMD's. Most of these techniques can be summarized into two main families: “Curved Mirror” based and “Waveguide” based. The curved mirror technique has been used by Vuzix in their Star 1200 product and by Laster Technologies. Various waveguide techniques have existed for some time. These techniques include diffraction optics, holographic optics, polarized optics, and reflective optics.

Applications

Major HMD applications include military, governmental (fire, police, etc.) and civilian/commercial (medicine, video gaming, sports, etc.).

Aviation and Tactical / ground

Ruggedized HMDs are increasingly being integrated into the cockpits of modern helicopters and fighter aircraft. These are usually fully integrated with the pilot's flying helmet and may include protective visors, night vision devices and displays of other symbology.

Military, police and firefighters use HMDs to display tactical information such as maps or thermal imaging data while viewing the real scene. Recent applications have included the use of HMD for paratroopers.^[1] In 2005, the Liteye HMD was introduced for ground combat troops as a rugged, waterproof lightweight display that clips into a standard US PVS-14 military helmet mount. The self-contained color monocular OLED display replaces the NVG tube and connects to a mobile computing device. The LE has see-through capability and can be used as a standard HMD or for augmented reality applications. The design is optimized to provide high definition data under all lighting conditions, in covered or see-through modes of operation. The LE has a low power consumption, operating on four AA batteries for 35 hours or receiving power via standard USB connection.^[2]

DARPA continues to fund research in Augmented Reality HMDs as part of the Persistent Close Air Support (PCAS) Program. Vuzix is currently working on a system for PCAS that will use holographic waveguides to produce see-through augmented reality glasses that are only a few millimeters thick.^[3]

Engineering, science and medicine

Engineers and scientists use HMDs to provide stereoscopic views of CAD schematics. These systems are also used in the maintenance of complex systems, as they can give a technician what is effectively "x-ray vision" by combining computer graphics such as system diagrams and imagery with the technician's natural vision. There are also applications in surgery, wherein a combination of radiographic data (CAT scans and MRI imaging) is combined with the surgeon's natural view of the operation, and anesthesia, where the patient vital signs are within the anesthesiologist's field of view at all times.^[4]

Research universities often use HMDs to conduct studies related to vision, balance, cognition and neuroscience.

As of 2010, the use of predictive visual tracking measurement to identify mild traumatic brain injury was being studied. In visual tracking tests, a HMD unit with eye-tracking capability shows an object moving in a regular pattern. People without brain injury are able to track the moving object with smooth pursuit eye movements and correct trajectory. The test requires both attention and working memory which are difficult functions for people with mild traumatic brain injury. The question being studied, is whether results for people with brain injury will show visual-tracking gaze errors relative to the moving target.^[5]

Gaming and video

Low cost HMD devices are available for use with 3D games and entertainment applications.

One of the first commercially available HMDs was the Forte VFX-1 which was announced at CES in 1994.^[6] The VFX-1 had stereoscopic displays, 3-axis head-tracking, and stereo headphones.

Another pioneer in this field was Sony who released the Glasstron in 1997, which had as an optional accessory a positional sensor which permitted the user to view the surroundings, with the perspective moving as the head moved, providing a deep sense of immersion.

One novel application of this technology was in the game MechWarrior 2, which permitted users of the Sony Glasstron or Virtual I/O's iGlasses to adopt a new visual perspective from inside the cockpit of the craft, using their own eyes as visual and seeing the battlefield through their craft's own cockpit.

Sony has released the Personal 3D Viewer (or HMZ-T1), a fully surround-sound headset for 3D gaming and movies.^[7]

Sensics demonstrated at CES 2012 a gaming and entertainment goggle that included an on-board Android processor as well as hand tracking to facilitate natural interaction.^[8]

Many brands of video glasses can now be connected to video and DSLR cameras, making them applicable as a new age monitor. As a result of the glasses ability to block out ambient light, filmmakers and photographers are able to see clearer presentations of their live images.^[9]

The Oculus Rift is an upcoming virtual reality (VR) head-mounted display created by Palmer Luckey that the company Oculus VR is developing for virtual reality simulations and video games.^[10]

VR headsets are also planned for use with game consoles like the Xbox One and the PS4. The Xbox VR headset is reportedly named Fortaleza.^[11]

Sports

A HMD system has been developed for Formula One drivers by Kopin Corp. and the BMW Group. According to BMW, “The HMD is part of an advanced telemetry system approved for installation by the Formula One racing committee… to communicate to the driver wirelessly from the heart of the race pit.” The HMD will display critical race data while allowing the driver to continue focussing on the track. Pit crews control the data and messages sent to their drivers through two-way radio.^[12]

Recon Instruments released on 3 November 2011 two head mounted displays for ski goggles, MOD and MOD Live, the later based on an Android operating system.^[13]

Training and simulation

A key application for HMDs is training and simulation, allowing to virtually place a trainee in a situation that is either too expensive or too dangerous to replicate in real-life. Training with HMDs cover a wide range of applications from driving, welding and spray painting, flight and vehicle simulators, dismounted soldier training, medical procedure training and more.

Performance parameters

• Ability to show stereoscopic imagery. A binocular HMD has the potential to display a different image to each eye. This can be used to show stereoscopic images. It should be borne in mind that so-called 'Optical Infinity' is generally taken by flight surgeons and display experts as about 9 metres. This is the distance at which, given the average human eye rangefinder "baseline" (distance between the eyes or Inter-Pupillary Distance (IPD)) of between 2.5 and 3 inches (6 and 8 cm), the angle of an object at that distance becomes essentially the same from each eye. At smaller ranges the perspective from each eye is significantly different and the expense of generating two different visual channels through the Computer-Generated Imagery (CGI) system becomes worthwhile.
• Interpupillary Distance (IPD). This is the distance between the two eyes, measured at the pupils, and is important in designing Head-Mounted Displays.
• Field of view (FOV) – Humans have an FOV of around 180°, but most HMDs offer considerably less than this. Typically, a greater field of view results in a greater sense of immersion and better situational awareness. Most people do not have a good feel for what a particular quoted FOV would look like (e.g. 25°) so often manufacturers will quote an apparent screen size. Most people sit about 60 cm away from their monitors and have quite a good feel about screen sizes at that distance. To convert the manufacturer's apparent screen size to a desktop monitor position, just divide the screen size by the distance in feet, then multiply by 2. Consumer-level HMDs typically offer a FOV of about 30-40° whereas professional HMDs offer a field of view of 60° to 150°.
• Resolution – HMDs usually mention either the total number of pixels or the number of pixels per degree. Listing the total number of pixels (e.g. 1600×1200 pixels per eye) is borrowed from how the specifications of computer monitors are presented. However, the pixel density, usually specified in pixels per degree or in arcminutes per pixel, is also used to determine visual acuity. 60 pixels/° (1 arcmin/pixel) is usually referred to as eye limiting resolution, above which increased resolution is not noticed by people with normal vision. HMDs typically offer 10 to 20 pixels/°, though advances in micro-displays help increase this number.
• Binocular overlap - measures the area that is common to both eyes. Binocular overlap is the basis for the sense of depth and stereo, allowing humans to sense which objects are near and which objects are far. Humans have a binocular overlap of about 100° (50° to the left of the nose and 50° to the right). The larger the binocular overlap offered by an HMD, the greater the sense of stereo. Overlap is sometimes specified in degrees (e.g. 74°) or as a percentage indicating how much of the visual field of each eye is common to the other eye.
• Distant focus ('Collimation'). Optical techniques may be used to present the images at a distant focus, which seems to improve the realism of images that in the real world would be at a distance.
• On-board processing and operating system. Some HMD vendors offer on-board operating systems such as Android, allowing applications to run locally on the HMD and eliminating the need to be tethered to an external device to generate video. These are sometimes referred to as Smart Goggles.

Support of 3D video formats

Depth perception inside an HMD requires different images for the left and right eyes. There are multiple ways to provide these separate images:

• Use dual video inputs, thereby providing a completely separate video signal to each eye
• Time-based multiplexing. Techniques such as frame sequential combine two separate video signals into one signal by alternating the left and right images in successive frames.
• Side by side or top/bottom multiplexing. This method allocated half of the image to the left eye and the other half of the image to the right eye.

The advantage of dual video inputs is that it provides the maximum resolution for each image and the maximum frame rate for each eye. The disadvantage of dual video inputs is that it requires separate video outputs and cables from the device generating the content.

Time-based multiplexing preserves the full resolution per each image, but reduces the frame rate by half. For example, if the signal is presented at 60 Hz, each eye is receiving just 30 Hz updates. This may become an issue with accurately presenting fast-moving images.

Side-by-side and top/bottom multiplexing provide full-rate updates to each eye, but reduce the resolution presented to each eye. Many 3D broadcasts, such as ESPN, chose to provide side-by-side 3D which saves the need to allocate extra transmission bandwidth and is more suitable to fast-paced sports action relative to time-based multiplexing techniques.

Not all HMDs provide depth perception. Some lower-end modules are essentially bi-ocular devices where both eyes are presented with the same image.

3D video players sometimes allow maximum compatibility with HMDs by providing the user with a choice of the 3D format to be used.

Peripherals

• The most rudimentary HMDs simply project an image or symbology on a wearer’s visor or reticle. The image is not slaved to the real world (i.e., the image does not change based on the wearer’s head position).
• More sophisticated HMDs incorporate a positioning system that tracks the wearer’s head position and angle, so that the picture or symbology displayed is congruent with the outside world using see-through imagery.
• Head tracking – Slaving the imagery. Head-mounted displays may also be used with tracking sensors that allow changes of angle and orientation to be recorded. When such data is available in the system computer, it can be used to generate the appropriate computer-generated imagery (CGI) for the angle-of-look at the particular time. This allows the user to "look around" a virtual reality environment simply by moving the head without the need for a separate controller to change the angle of the imagery. In radio-based systems (compared to wires), the wearer may move about within the tracking limits of the system.
• Eye tracking – Eye trackers measure the point of gaze, allowing a computer to sense where the user is looking. This information is useful in a variety of contexts such as user interface navigation : by sensing the user's gaze, a computer can change the information displayed on a screen, bring additional details to attention, etc.
• Hand tracking - tracking hand movement from the perspective of the HMD allows natural interaction with content and a convenient game-play mechanism

HMD manufacturers (alphabetically)

Companies that have produced HMDs include:

• Altergaze (smartphone based HMD)
• Arisawa
• Atari (Jaguar VR headset)^[14][15][16]
• Canon
• Carl Zeiss cinemizer OLED
• Cinoptics (former Cybermind)
• eMagin (EMAN)
• Epson
• Fatshark Vision Systems
• HTC (is currently developing HTC Vive in collaboration with Valve Corporation)
• iTheater VisionTech (China, since 2005)
• Kopin Corporation (created Golden-i)
• The Link simulation division of L-3 Communications
• LASTER Technologies
• Motorola (wearable PC)
• MyVu (formerly MicroOptical)
• Nintendo (created the Virtual Boy video game platform)
• novero
• NVIS
• Oculus VR (in the form of their device the Oculus Rift)
• Olympus
• Optinvent^[17]
• Penny^[18] introduced C Wear Interactive Glasses 2008, 40 degrees FOV See Through Augmented Reality Glasses
• Philips (SCUBA) introduced 1998^[19][20]
• Recon Instruments
• Rockwell Collins Optronics (formerly Kaiser Electro-Optics). Note that SEOS, now part of Rockwell Collins, also produced HMDs in the UK.
• SA Photonics^[21]
• SEGA Sega VR
• Sensics
• SiliconMicroDisplay^[22] 1080p LCos microdisplays
• Sony (currently developing Project Morpheus; former developments: the Glasstron and the HMZ-T1).
• Takara (Dynovisor) introduced 1996^[23][24]
• TDVision
• Vuzix

Transparent glasses

List of optical head-mounted display manufacturers

See also

• Virtual reality (VR)
• Augmented reality (AR)
• Computer-mediated reality
• Eyetap
• Head-up display (HUD)
• Helmet mounted display
• lumus-optical
• Mixed reality (MR)
• Positioning technologies
• Screenless
• Stereoscopy
• Virtual retinal display

References

Bibliography

• Head Mounted Displays: Designing for the user; Melzer and Moffitt; McGraw Hill, 1997.
• O. Cakmakci and J.P. Rolland. Head-Worn Displays: A Review. IEEE Journal of Display Technology, Vol. 2, No. 3, September 2006..

Virtual reality · Mixed reality · Reality Concepts Virtuality Virtual cinematography Augmented reality Augmented virtuality Real life Projection augmented model Reality–virtuality continuum Artificial reality Simulated reality Ubiquitous computing Virtual world (Persistent world) Multimodal interaction Telepresence Immersion Technology Compositing Camera resectioning Haptic suit Head-mounted display (Optical head-mounted display) Head-up display Image-based modeling and rendering Real-time computer graphics Virtual retinal display Wearable computer Stereoscopy (Computer stereo vision, Computer vision) Chroma key Visual hull Free viewpoint television Omnidirectional treadmill Hidden surface determination Tracking Motion capture Tracking system Types Optical Inertial Magnetic Devices Wired glove Gametrak PlayStation Move Leap Motion Kinect Sixense TrueMotion Immersive devices Personal Google Cardboard HTC Vive Oculus Rift Samsung Gear VR Project Morpheus Rooms AlloSphere Cave TreadPort History Sensorama Virtual Boy Famicom 3D System Sword of Damocles Sega VR Virtuality Applications Pervasive game ARToolKit Interactive art (Virtual graffiti) See also Simulated reality in fiction

Stereoscopy Perception Binocular rivalry Binocular vision Chromostereopsis Convergence insufficiency Correspondence problem Peripheral vision Depth perception Epipolar geometry Kinetic depth effect Stereoblindness Stereopsis Stereopsis recovery Stereoscopic acuity Display technologies Active shutter 3D system Anaglyph 3D Autostereogram Autostereoscopy Bubblegram ChromaDepth Head-mounted display Holography Integral imaging Lenticular lens Multiscopy Parallax barrier Polarized 3D system Stereo display Stereoscope Vectograph Virtual retinal display Volumetric display Wiggle stereoscopy Other technologies 2D to 3D conversion 2D plus Delta 2D-plus-depth Computer stereo vision Multiview Video Coding Pseudoscope Stereo photography techniques Stereoautograph Stereoscopic depth rendition Stereoscopic rangefinder Stereoscopic spectroscopy Stereoscopic Video Coding Types of products 3D camcorder 3D film 3D television 3D-enabled mobile phones 4D film Blu-ray 3D Digital 3D Stereo camera Stereo microscope Stereoscopic video game Notable products Dolby 3D Fujifilm FinePix Real 3D MasterImage 3D Nintendo 3DS XL New Nvidia 3D Vision RealD 3D Sharp Actius RD3D View-Master XpanD 3D Miscellaneous International Stereoscopic Union Stereoscopic Displays and Applications

Headwear Hats Caps General Baseball Beanie Beret Bucket Casquette Conical Asian Cowboy Cricket Easter Fedora Fez Flat Gandhi Knit Kofia Newsboy Pillbox Pork pie Sombrero Stormy Kromer Straw Sun Top Tricorne Trucker Ushanka Military Boonie Campaign Forage Mitznefet Peaked Sailor Side Utility cover Wrapped Bonnet Do-rag Hood Turban Veil Crowns and bands Astral Circlet Consort Corolla Coronation Coronet Diadem Eastern Ferronnière Fillet Hoop Imperial "Queens" State Tiara Wreath (laurel) Helmets Combat Corinthian Imperial Barbute Close Great Morion Sallet Advanced Combat Enhanced Combat (US) Flight Mk 7 Paratrooper Stahlhelm Vocational Firefighter Hard hat Welding Sport Batting Bicycle Football Motorcycle Racing Other protective Balaclava Earmuffs Earplug Face shield Facekini Padded headgear Scarf Sports visor Eyewear Eyeglasses Pince-nez Browline GI Rimless Monocle Goggles Sunglasses Head-mounted display Hairwear and other items Barrette Deely bobber Dush-toh Hair drop Hair tie Hairnet Headband Headphones Headpiece Peineta Scrunchie Shpitzel List of headwear

Emerging technologies Technology Fields Agriculture Agricultural robot Closed ecological systems In vitro meat Genetically modified food Precision agriculture Vertical farming Architecture Arcology Building printing Contour Crafting D-Shape Domed city Biomedical Artificial uterus Ampakine Brain transplant Cryonics Cryoprotectant Vitrification De-extinction Cryopreservation Full genome sequencing Genetic engineering Gene therapy Head transplant Isolated brain Life extension Strategies for Engineered Negligible Senescence Nanomedicine Nanosensors Personalized medicine Regenerative medicine Stem cell treatments Tissue engineering Robotic surgery Suspended animation Synthetic biology Synthetic genomics Virotherapy Oncolytic Virus Tricorder Displays Next generation FED FLD iMoD Laser LPD OLED OLET QD-LED SED TPD TDEL TMOS Screenless Bionic contact lens Head-mounted display Head-up display Optical head-mounted display Virtual retinal display Other Autostereoscopy Flexible display Holographic display Computer-generated holography Multi-primary color display Ultra HD Volumetric display Electronics Electronic nose Electronic textile Flexible electronics Molecular electronics Nanoelectromechanical systems Memristor Spintronics Thermal copper pillar bump Energy Production Airborne wind turbine Artificial photosynthesis Biofuels Carbon negative fuel Concentrated solar power Fusion power Home fuel cell Hydrogen economy Methanol economy Molten salt reactor Nantenna Solar roadway Space-based solar power Vortex engine Storage Beltway battery Carbon neutral fuel Compressed air energy storage Flywheel energy storage Grid energy storage Lithium air battery Molten salt battery Nanowire battery Research in lithium-ion batteries Silicon air battery Thermal energy storage Ultracapacitor Other Smart grid Contactless energy transfer IT and communications Ambient intelligence Internet of Things Artificial intelligence Applications of artificial intelligence Progress in artificial intelligence Machine translation Machine vision Semantic Web Speech recognition Atomtronics Carbon nanotube field-effect transistor Cybermethodology Fourth-generation optical discs 3D optical data storage Holographic data storage GPGPU Memory CBRAM FRAM Millipede MRAM NRAM PRAM Racetrack memory RRAM SONOS Optical computing Quantum computing Quantum cryptography RFID Chipless RFID Three-dimensional integrated circuit Manufacturing 3D printing Claytronics Molecular assembler Utility fog Materials science Aerogel Amorphous metal Artificial muscle Conductive polymer Femtotechnology Fullerene Graphene High-temperature superconductivity High-temperature superfluidity Linear acetylenic carbon Metamaterials Metamaterial cloaking Multi-function structures Nanotechnology Carbon nanotubes Molecular nanotechnology Nanomaterials Picotechnology Programmable matter Quantum dots Silicene Superalloy Synthetic diamond Military Antimatter weapon Caseless ammunition Directed-energy weapon Laser Maser Particle beam weapon Sonic weapon Electromagnetic weapon Coilgun Railgun Plasma weapon Pure fusion weapon Stealth technology Vortex ring gun Neuroscience Artificial brain Blue Brain Project Brain–computer interface Electroencephalography Mind uploading Brain-reading Neuroinformatics Neuroprosthetics Bionic eye Brain implant Exocortex Retinal implant Robotics Domotics Nanorobotics Powered exoskeleton Self-reconfiguring modular robot Swarm robotics Unmanned vehicle Space science Launch Fusion rocket Non-rocket spacelaunch Mass driver Orbital ring Space elevator Space fountain Space tether Reusable launch system Propulsion Beam-powered propulsion Ion thruster Laser propulsion Plasma propulsion engine Helicon thruster VASIMR Project Orion Nuclear pulse propulsion Solar sail Warp drive Other Interstellar travel Propellant depot Transport Aerial Adaptive Compliant Wing Aeroscraft Backpack helicopter Delivery drone Flying car High-altitude platform Jet pack Pulse detonation engine Scramjet Spaceplane Skylon Supersonic transport Land Airless tire Tweel Alternative fuel vehicle Hydrogen vehicle Driverless car Ground effect train Maglev train Personal rapid transit Vactrain ET3 Global Alliance Hyperloop Vehicular communication systems Pipeline Pneumatic transport Automated vacuum collection Foodtubes Other Anti-gravity Cloak of invisibility Digital scent technology Force field Plasma window Immersive virtual reality Virtusphere Magnetic refrigeration Phased-array optics Quantum technology Quantum teleportation Topics Collingridge dilemma Differential technological development Ephemeralization Exploratory engineering Fictional technology Proactionary principle Technological change Technological unemployment Technological convergence Technological evolution Technological paradigm Technology forecasting Accelerating change Moore's law Technological singularity Technology scouting Technology readiness level Technology roadmap Transhumanism Category List