Optical see-through head-mounted display
More actions
Optical see-through head-mounted display (OST-HMD), also called Optical head-mounted display or OHMD, is a type of head-mounted display that projects images and allows the user to see through its display. OHMDs are used in augmented reality (AR). Unlike virtual reality HMDs that obscure the wearer's vision of the real world, OHMDs let the wearer see the surroundings while data and image overlays are shown in front of the eyes. In his 1997 survey of augmented reality, Ronald Azuma described the principle: optical see-through HMDs place optical combiners in front of the user's eyes that are "partially transmissive, so that the user can look directly through them to see the real world" and partially reflective, so that the user also sees virtual images bounced off the combiners from head-mounted monitors.[1]
The alternative is a video see-through HMD, which captures the real world with head-mounted cameras and combines the camera images with computer graphics electronically.[2] The optical approach dates to the 1960s and Ivan Sutherland's head-mounted display, and in a 2021 survey Yuta Itoh, Tobias Langlotz, Jonathan Sutton and Alexander Plopski wrote that OST-HMDs had recently seen "a reemergence, partially thanks to large investments from industry".[3] Commercial examples include Microsoft's HoloLens line, Magic Leap 1 and Magic Leap 2, Google Glass and the monocular Meta Ray-Ban Display.
HoloLens displays are fixed at an optical distance of approximately 2.0 m from the user, according to Microsoft's design guidance.[4] Magic Leap 1 switches between two focal planes using data from its eye tracker,[3] and Magic Leap recommends keeping content on Magic Leap 2 close to its focal plane at 0.74 m.[5]
Vuzix is a provider of OST-HMDs; the company develops waveguide optics, display systems and smart glasses, and sells enterprise smart glasses such as the M400.[6]
A number of companies have marketed waveguides for OST-HMDs, including Dispelix, Digilens, and Lumus. Dispelix describes its transparent waveguides as see-through displays for AR devices that "optically combine the real and virtual worlds in the user's field of vision".[7] Lumus makes geometric waveguides and lists products such as Maximus and Z-Lens with a 50 degree field of view.[8] DigiLens produces waveguides with an inkjet printing and holographic contact copy process and also sells its own ARGO smart glasses.[9]
An optical head-mounted display can cover only one eye, as monocular devices such as Google Glass and Meta Ray-Ban Display do, or both eyes.[10][11] Wearers can interact with the projected digital content through input methods such as voice commands, gestures and controllers.
How it works
In the basic design, the real world is seen through half-transparent mirrors placed in front of the eyes, and the same mirrors reflect the computer-generated images into the eyes, so that the real and virtual views are combined optically.[2] Azuma compared the approach to the head-up displays used in military aircraft, except that the combiners are attached to the head, and noted that OST-HMDs had sometimes been described as a "HUD on a head".[1] Boeing researchers Thomas Caudell and David Mizell listed the basic elements of such a head-up display in 1992 as an image source, relay optics and a beam splitter, where the beam splitter lets the viewer see the real world and the graphics at the same time.[12]
Because the combiner has to reflect display light as well as pass light from the environment, it usually reduces how much of the real world the user sees. Azuma gave the example of an HMD that transmitted about 30 percent of incoming light, and wrote that most optical see-through HMDs of the time acted "like a pair of sunglasses when the power is cut off".[1] In a modern AR display the part that produces the image is usually called the light engine and the part that relays it to the eye while transmitting ambient light is the optical combiner. The combiner largely determines the field of view, eye box and image sharpness, while brightness, efficiency and size depend on both parts.[13] Light engines used in AR include micro-LED, micro-OLED, liquid-crystal-on-silicon (LCoS), digital micromirror devices and laser beam scanning.[13]
Combiner designs
Itoh and colleagues group the optical designs used in commercial OST-HMDs into half mirrors, birdbath combiners, free-form prisms, waveguides and retinal scanning, with light-field, pinlight and holographic designs found in research prototypes.[3]
| Design | How it works | Trade-offs noted in the literature |
|---|---|---|
| Half mirror | A tilted partially reflective mirror in front of the eye reflects the display image | Trade-off between form factor and eye box size[3] |
| Birdbath | A beam splitter and a curved mirror fold the optical path | Larger field of view, but light passes the half mirrors several times, lowering efficiency, and the design tends to be larger[3] |
| Free-form prism | A prism with free-form surfaces reflects the image by total internal reflection; a compensator cancels the prism's optical power for the see-through view | One design by Cheng et al. reached a 54 degree diagonal field of view with an 8 mm exit pupil[13] |
| Geometric (reflective) waveguide | Light travels through a thin plate by total internal reflection and an array of partial mirrors couples it out toward the eye | Good image sharpness and color uniformity, but gaps can appear in the eye box[13] |
| Diffractive or holographic waveguide | Gratings or holographic optical elements couple light into and out of the plate | Different wavelengths diffract at different angles, which the design must account for[3] |
| Retinal scanning | A laser source projects the image directly into the eye[3] | In Maxwellian-view designs the image stays in focus on the retina regardless of the eye's focus[13] |
Waveguide combiners expand the exit pupil by coupling part of the trapped light out at every internal reflection, which enlarges the eye box.[13] Designs based on conventional tilted half mirrors find it hard to reach a flat, eyeglasses-like shape because the mirror has to be angled.[13]
History
Rolland and Fuchs trace see-through HMDs to the 1960s. Ivan Sutherland's 1965 and 1968 optical see-through stereo HMDs were the first computer graphics HMDs, using miniature cathode ray tubes (CRTs), a mechanical head tracker and a hand-tracking device.[2] In his 1968 paper, Sutherland described a headset whose optics presented a virtual image of each CRT about eighteen inches in front of each eye with a 40 degree field of view. "Half-silvered mirrors in the prisms through which the user looks allow him to see both the images from the cathode ray tubes and objects in the room simultaneously," he wrote, so that displayed material could "hang disembodied in space or ... coincide with maps, desk tops, walls, or the keys of a typewriter".[14] The system is known as the Sword of Damocles.[3]
Most later development of see-through HMDs was aimed at military applications, while work on 3D scientific and medical visualization began at the University of North Carolina at Chapel Hill in the 1980s.[2] In 1985 Warren Robinett and colleagues at UNC built a see-through display with color LCDs, half-silvered mirrors and magnifying lenses.[12] At Boeing, Caudell and Mizell prototyped a "heads-up, see-through, head-mounted display" they called the HUDset, meant to superimpose wiring diagrams and drill locations on aircraft parts and replace templates and formboard diagrams; their 1992 paper referred to the technology as "augmented reality".[12]
Google stopped selling Glass Enterprise Edition 2 on 15 March 2023 and ended support on 15 September 2023.[10] Microsoft ended HoloLens 2 production in October 2024 and said the headset would receive updates for critical security issues and software regressions until 31 December 2027.[15] At Connect 2024, Meta showed Orion, a 98 g prototype with a 70 degree diagonal field of view that uses silicon carbide lenses and advanced waveguides; Meta said it would not be sold because each unit cost on the order of US$10,000 to produce.[16] Meta Ray-Ban Display, with a monocular full-color display in the right lens, went on sale at a limited number of US retailers on 30 September 2025 at US$799.[11]
| Device | Status | Display notes |
|---|---|---|
| Sutherland's HMD (1968) | Research system | Binocular; half-silvered mirrors; 40 degree field of view[14] |
| Boeing HUDset (1992) | Research prototype | Head-tracked see-through display for aircraft manufacturing[12] |
| HoloLens 2 | Production ended October 2024; updates until 31 December 2027[15] | 52 degree field of view; fixed focal distance of about 2.0 m[16][4] |
| Magic Leap 1 | Support and cloud services ended 31 December 2024[17] | Two focal planes switched using eye tracking[3] |
| Magic Leap 2 | Direct sales ended 31 March 2026; authorized resellers may deliver units through 31 December 2026[18] | 70 degree diagonal field of view; focal plane at 0.74 m; global and segmented dimming[16][5][19] |
| Google Glass Enterprise Edition 2 | Sales ended 15 March 2023 | Single heads-up display[10] |
| Meta Orion | Prototype, not sold | 70 degree diagonal field of view; silicon carbide waveguides; micro-LED projectors[16] |
| Meta Ray-Ban Display | On sale from 30 September 2025 | Monocular 600 x 600 pixel display, 20 degree field of view, 42 pixels per degree[11] |
Optical versus video see-through
Azuma listed four advantages of optical see-through over video see-through.[1] Optical blending is simpler and cheaper, because only the graphics have to be processed; the real world is seen directly with a delay of a few nanoseconds, while each video stream has delays in the tens of milliseconds. Resolution of the real world is not degraded, since only the graphics are limited by the display. Safety is better, because when power is removed the user "still has a direct view of the real world", whereas the wearer of an unpowered video see-through HMD is "effectively blind". Finally, there is no eye offset: video cameras usually sit away from the eyes and shift the user's vantage point, a problem the optical design avoids.[1] Rolland and Fuchs made a similar point about failure modes: a failed OST-HMD may leave the user without computer-generated images but with the real-world view, while a failed video see-through HMD may remove both views.[2] Itoh and colleagues add that video see-through decouples the user from reality and prevents mutual eye contact.[3]
Video see-through has its own advantages. Azuma listed more flexible composition (video can replace real pixels with virtual ones, while optical combiners let light from both sources through), easier correction of distortion for wide fields of view, the ability to delay the real-world video so it matches the graphics, extra registration information from the camera image, and easier matching of real and virtual brightness.[1] He noted that most mechanical assembly and repair prototypes of the time used optical see-through, possibly for cost and safety reasons, while most medical prototypes used video.[1]
| Property | Optical see-through | Video see-through |
|---|---|---|
| View of the real world | Direct, at full natural resolution[1] | Limited to camera and display resolution[1] |
| Delay of the real-world view | A few nanoseconds[1] | Tens of milliseconds; can be matched to the graphics[1] |
| Behavior on power loss | Real world stays visible[1] | User is effectively blind[1] |
| Occlusion of real objects by virtual ones | Hard; virtual objects look semi-transparent[1] | Pixel-by-pixel compositing[2] |
| Viewpoint | The user's own eyes[1] | Camera position, which must be matched to the eyes[2] |
Technical challenges
Itoh and colleagues group the obstacles to making virtual content look real on an OST-HMD into spatial realism (registration and distortion), temporal realism (latency and flicker) and visual realism (color, occlusion, focus, resolution and dynamic range).[3]
Registration and latency
Virtual objects must be registered with the real scene, which requires calibrating the tracker to the HMD optics. Rolland and Fuchs identified lag, the time between measuring the head pose and showing the image rendered for that pose, as the largest source of registration error in the systems of their time, typically 60 to 180 ms. During that delay virtual objects can "swim" away from the real objects they are attached to.[2] Because an optical system cannot delay the real scene, it has to be built for low latency, perhaps under 60 ms where predictive tracking can help, and users may otherwise have to limit themselves to slow head motions.[2] Itoh and colleagues state that see-through AR can require even lower latency than VR, where studies suggest about 5 ms or less is needed for the most sensitive subjects, since in AR latency also produces visible registration errors.[3] Microsoft recommends that HoloLens apps render at a minimum of 60 frames per second; the system predicts where the head will be when the frame is displayed and adjusts the image to account for prediction error.[4]
Calibration
An OST-HMD also needs to know where the user's eyes are relative to the display, because the virtual image is only correct from the eye's viewpoint. Jens Grubert and colleagues, in a 2017 survey, note that researchers had proposed calibration methods for this problem for more than 20 years.[20] The best-known manual method is the Single Point Active Alignment Method (SPAAM) introduced by Mihran Tuceryan and Nassir Navab, in which the user aligns an on-screen crosshair with a tracked real point several times so the system can solve for the eye-display projection.[20][3] Automatic methods such as INDICA, proposed by Itoh and Gudrun Klinker, separate an offline display calibration from an online estimate of the eye position, so the user does not have to repeat the procedure each time the headset is put on.[3] On HoloLens 2, interpupillary distance is estimated during the device's eye calibration.[4]
Occlusion, contrast and color
Optical combiners add display light to light from the scene, so virtual objects cannot fully hide real ones. Azuma explained that a filter to block real-world light selectively would have to be placed where the scene is in focus inside the optics, which makes the design much more complex; without it, "the virtual objects appear ghost-like and semi-transparent".[1] Rolland and Fuchs wrote that the designer's main control is the ratio of reflectance to transmittance of the beam splitter, chosen to match display brightness to expected ambient light.[2] Itoh and colleagues give the example of a combiner that is 50 percent transparent and 50 percent reflective, where the background always contributes 50 percent of each pixel's color and intensity; many commercial devices, including HoloLens, add shades to darken the background.[3] They describe Kiyoshi Kiyokawa's ELMO-1 as seminal work on occlusion-capable OST-HMDs; it placed an LCD at an internal focal plane to block background light.[3]
Magic Leap describes its displays as additive and added a low-resolution dimmer panel to Magic Leap 2 that "selectively subtracts photons in the environment". Global dimming tints the whole view, and segmented dimming darkens only the areas behind virtual content so that it appears more opaque.[19]
Contrast in bright surroundings is measured as the ambient contrast ratio. Xiong and colleagues give a minimum of 3:1 for recognizable images, 5:1 for adequate readability and 10:1 or more for outstanding readability, and estimate that reaching 10:1 on a sunny day (about 3,000 nits ambient) needs a display brightness of at least 30,000 nits before optical losses.[13] Azuma had described the same problem in 1997: if the real environment is too bright it washes out the virtual image, and if it is too dark the virtual image washes out the real world.[1]
Focus and the vergence-accommodation conflict
Most OST-HMDs show the virtual image on a single fixed focal plane, while real objects lie at many distances, so the user may not be able to see a real object and a virtual one sharply at the same time.[1] The fixed focal plane also causes the vergence-accommodation conflict.[3] Microsoft advises placing HoloLens content between 1.25 m and 5 m, fading content out at 40 cm and clipping it at 30 cm, and notes that discomfort is greatest when gaze switches between distances.[4] Magic Leap sets the default near clipping plane on Magic Leap 2 at 0.37 m.[5] Research approaches include multifocal and varifocal displays, light-field displays with stacked LCDs or microlens arrays, pinlight displays and holographic displays; multifocal designs can only approximate focus cues with a limited number of planes and may add flicker.[3]
Field of view
Azuma noted that wide field-of-view displays are harder to build with optical see-through, because any distortion of the real-world view must be corrected optically rather than digitally, and complex optics add cost and weight.[1] Rolland and Fuchs noted that optical see-through HMDs were typically designed open enough for users to see around the device with peripheral vision, bringing the total real-world field of view close to natural vision, although the mounts of the see-through mirror usually leave a ring of obstruction, much like an eyeglass frame.[2] The human visual field reaches up to about 180 degrees horizontally,[3] while UploadVR lists a 52 degree field of view for HoloLens 2 and 70 degrees diagonal for Magic Leap 2 and Orion.[16] For waveguides, Meta says the silicon carbide used in Orion has a refractive index of 2.7, which it describes as the highest known for an optical application.[21]
Applications
Early OST-HMD research concentrated on manufacturing and medicine. At Boeing, the HUDset was intended to guide workers assembling wire bundles and drilling parts by overlaying the diagram or hole location on the workpiece.[12] At UNC, an error analysis of an optical see-through HMD for craniofacial surgery aimed to overlay CT skull data on the patient, and researchers at the University of Central Florida developed an optical see-through tool for visualizing knee-joint motion over a tracked leg model.[2] Azuma wrote that the choice between optical and video see-through depends on the application's requirements.[1] Rolland and Fuchs noted that because an optical system adds no artificial delay to the real scene, real objects "will always be where they are perceived to be", which may matter for a broad range of applications.[2]
See also
References
- ↑ 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.10 1.11 1.12 1.13 1.14 1.15 1.16 1.17 1.18 1.19 Ronald T. Azuma (1997-08). "A Survey of Augmented Reality". Presence: Teleoperators and Virtual Environments, vol. 6, no. 4, pp. 355-385. MIT Press. https://direct.mit.edu/pvar/article/6/4/355/18336/A-Survey-of-Augmented-Reality. Retrieved 2026-09-27.
- ↑ 2.00 2.01 2.02 2.03 2.04 2.05 2.06 2.07 2.08 2.09 2.10 2.11 2.12 Jannick P. Rolland, Henry Fuchs (2000-06). "Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization". Presence: Teleoperators and Virtual Environments, vol. 9, no. 3, pp. 287-309. MIT Press. doi:10.1162/105474600566808. https://henryfuchs.web.unc.edu/wp-content/uploads/sites/4964/2013/05/Optical-versus-Video-See-Through-Head-Mounted-Displays-in-Medical-Visualization.pdf. Retrieved 2026-09-27.
- ↑ 3.00 3.01 3.02 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 Yuta Itoh, Tobias Langlotz, Jonathan Sutton, Alexander Plopski (2021-07). "Towards Indistinguishable Augmented Reality: A Survey on Optical See-through Head-mounted Displays". ACM Computing Surveys, vol. 54, no. 6, article 120. doi:10.1145/3453157. https://doi.org/10.1145/3453157. Retrieved 2026-09-27.
- ↑ 4.0 4.1 4.2 4.3 4.4 "Comfort". Microsoft Learn. Microsoft. https://learn.microsoft.com/en-us/windows/mixed-reality/design/comfort. Retrieved 2026-09-27.
- ↑ 5.0 5.1 5.2 "Comfort and Content Placement Guidelines". Magic Leap Developer Documentation. Magic Leap. https://developer-docs.magicleap.cloud/docs/guides/best-practices/comfort-content-placement/. Retrieved 2026-09-27.
- ↑ "Vuzix". Vuzix. https://www.vuzix.com/. Retrieved 2026-09-27.
- ↑ "Dispelix". Dispelix. https://www.dispelix.com/. Retrieved 2026-09-27.
- ↑ "Lumus". Lumus. https://lumus.com/. Retrieved 2026-09-27.
- ↑ "DigiLens". DigiLens. https://www.digilens.com/. Retrieved 2026-09-27.
- ↑ 10.0 10.1 10.2 Scott Hayden (2023-03-16). "Google Discontinues Glass Enterprise Edition Smartglasses". Road to VR. https://www.roadtovr.com/google-discontinues-glass-enterprise/. Retrieved 2026-09-27.
- ↑ 11.0 11.1 11.2 Scott Hayden (2025-09-17). "Meta Unveils Ray-Ban Smart Glasses with Display, Launching for $800 This Month". Road to VR. https://roadtovr.com/meta-ray-ban-smart-glasses-display-price-release-date-specs/. Retrieved 2026-09-27.
- ↑ 12.0 12.1 12.2 12.3 12.4 Thomas P. Caudell, David W. Mizell (1992). "Augmented Reality: An Application of Heads-Up Display Technology to Manual Manufacturing Processes". Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences. IEEE. doi:10.1109/HICSS.1992.183317. https://www.editionmultimedia.fr/wp-content/uploads/2022/11/Augmented-Reality-Tom-Caudell-et-David-Mizell-IEEE1992.pdf.
- ↑ 13.0 13.1 13.2 13.3 13.4 13.5 13.6 13.7 Jianghao Xiong, En-Lin Hsiang, Ziqian He, Tao Zhan, Shin-Tson Wu (2021-10-25). "Augmented reality and virtual reality displays: emerging technologies and future perspectives". Light: Science & Applications, vol. 10, article 216. doi:10.1038/s41377-021-00658-8. https://www.nature.com/articles/s41377-021-00658-8. Retrieved 2026-09-27.
- ↑ 14.0 14.1 Ivan E. Sutherland (1968). "A head-mounted three dimensional display". AFIPS Fall Joint Computer Conference. pp. 757-764. doi:10.1145/1476589.1476686. https://web.stanford.edu/class/ee267/notes/sutherland_hmd.pdf.
- ↑ 15.0 15.1 David Heaney (2024-10-01). "Microsoft Is Discontinuing HoloLens 2, With No Replacement". UploadVR. https://www.uploadvr.com/microsoft-discontinuing-hololens-2/. Retrieved 2026-09-27.
- ↑ 16.0 16.1 16.2 16.3 16.4 David Heaney (2024-09-25). "Meta's 'Orion' Prototype AR Glasses Have 70 Degree FOV And A Wireless Compute Puck". UploadVR. https://www.uploadvr.com/meta-connect-2024-orion-prototype-ar-glasses/. Retrieved 2026-09-27.
- ↑ Scott Hayden (2023-09-01). "Unicorn AR Startup Magic Leap is Killing Its First Headset Next Year". Road to VR. https://roadtovr.com/magic-leap-1-shut-down-2024/. Retrieved 2026-09-27.
- ↑ "Terms of Sale for Magic Leap 2". Magic Leap. Magic Leap. https://www.magicleap.com/legal/terms-of-sale. Retrieved 2026-09-27.
- ↑ 19.0 19.1 "Global/Segmented Dimmer". Magic Leap Developer Documentation. Magic Leap. https://developer-docs.magicleap.cloud/docs/guides/features/dimmer-feature/. Retrieved 2026-09-27.
- ↑ 20.0 20.1 Jens Grubert, Yuta Itoh, Kenneth Moser, J. Edward Swan II (2017-09-13). "A Survey of Calibration Methods for Optical See-Through Head-Mounted Displays". arXiv, 1709.04299. https://arxiv.org/abs/1709.04299. Retrieved 2026-09-27.
- ↑ "Crystal Clear: Our Silicon Carbide Waveguides & the Path to Orion's Large FoV". Meta Quest Blog. Meta. 2025-03-06. https://www.meta.com/blog/orion-silicon-carbide-waveguides-ar-glasses-large-field-of-view/. Retrieved 2026-09-27.