Augmented reality headset
More actions
An augmented reality headset is a type of head-worn gadget that enables the wearer to have an augmented reality experience, which is virtual objects in the real world environment.
In the research literature the device is usually called a see-through head-mounted display (HMD). Ronald Azuma's 1997 survey of the field contrasts it with the closed-view HMDs used for virtual reality: a closed-view HMD blocks any direct view of the real world, while a see-through HMD lets the user see the real world with virtual objects superimposed by optical or video technologies.[1] The two approaches, optical see-through and video see-through, divide AR headsets into two broad families with different strengths and weaknesses.[1]
Examples of AR headsets are the Microsoft Hololens, the Magic Leap 1, and Project North Star.
An AR headset in a glasses form factor is called AR glasses.
One of the earliest see-through head-mounted displays was described by Ivan Sutherland in 1968, and the term "augmented reality" dates from work on a head-worn display for aircraft wiring at Boeing in 1990.[2][3] Microsoft's HoloLens, a self-contained headset, shipped to developers in 2016; it was followed by the Magic Leap and HoloLens 2 devices, video see-through headsets such as the Apple Vision Pro, and glasses-style prototypes such as Meta Orion.[4][5][6]
Definition
Azuma defines augmented reality as any system with three characteristics: it combines real and virtual, it is interactive in real time, and it is registered in 3-D. He notes that some researchers define AR in a way that requires head-mounted displays, and he deliberately avoids that restriction so that monitor-based and other combining technologies also qualify.[1] An AR headset is therefore one way of delivering augmented reality, not a requirement of it. In the same survey Azuma describes AR as a "middle ground" between completely synthetic virtual environments and completely real telepresence, citing Paul Milgram's work on what became the reality-virtuality continuum.[1]
The difference from a virtual reality headset lies in what the wearer sees of the physical surroundings. Azuma writes that virtual environment technologies "completely immerse a user inside a synthetic environment" in which the user cannot see the real world, whereas AR "supplements reality, rather than completely replacing it".[1] Xiong, Hsiang, He, Zhan and Wu make the same distinction in their 2021 review of AR and VR displays: VR offers a totally immersive experience, while AR displays virtual images "while remaining see-through capability".[7]
How it works
Every AR headset has to solve two problems at once: showing a computer-generated image to each eye, and merging that image with the wearer's view of the real world so that virtual objects appear fixed in place. Azuma's conceptual diagrams of both headset types share the same core parts: a head tracker that reports the position of the head, a scene generator that renders the graphics for that viewpoint, and head-mounted displays that present the result. They differ in how the real world reaches the eye.[1]
Optical see-through
An optical see-through headset places optical combiners in front of the eyes. Azuma describes these combiners as partially transmissive, so the wearer looks directly through them at the real world, and partially reflective, so the wearer also sees virtual images reflected from head-mounted displays. He compares the design to the head-up displays of military aircraft with the combiner attached to the head, and notes that such headsets had been called a "HUD on a head".[1] Because a half-silvered combiner lets in only part of the outside light, most optical see-through headsets of the period dimmed the real world; one headset he cites transmitted about 30% of incoming light, and such a device acts "like a pair of sunglasses when the power is cut off".[1]
Modern optical see-through headsets pair a small microdisplay light engine with a combiner. Xiong et al. list the light engines used in AR as micro-LED, micro-OLED, liquid crystal on silicon (LCoS), digital micromirror devices, and MEMS-based laser beam scanning, and group combiners into two main types: free-space combiners and waveguide combiners.[7]
Free-space combiners use geometric optics. The simplest uses a single freeform half-mirror to collimate the image toward the eye, which can reach a field of view of up to 90 degrees but causes image distortion known as pupil swim and a relatively bulky form factor. Birdbath optics add optics on the display side and a beam splitter to fold the light path, which gives room for aberration correction and reduces the size to some extent.[7] Leap Motion's open-source Project North Star headset of 2018 used large birdbath-style reflectors of this kind, which Road to VR compared to the approach of the Meta 2.[8]
Waveguide combiners trap light inside a thin substrate by total internal reflection and couple part of it out at each bounce. Xiong et al. identify this exit pupil expansion (EPE) process, which effectively enlarges the system etendue and spreads the image over a large eyebox, as a distinctive feature of waveguide combiners. They divide waveguides into diffractive designs, which use gratings or holographic elements, and achromatic designs such as the geometric (partial-mirror) reflective waveguide, which usually gives better image quality (modulation transfer function) and color uniformity than diffractive designs but can leave gaps in the eyebox.[7] Microsoft lists the optics of HoloLens 2 as "See-through holographic lenses (waveguides)".[9]
Video see-through
A video see-through headset starts from a closed-view HMD and adds one or two head-mounted video cameras. Azuma describes how the camera video of the real world is combined with the rendered graphics and the combined result is shown on the displays in front of the eyes.[1] The combination can be done by chroma-keying, in which a reserved background color in the graphics is replaced with camera video, or, if depth is known for each pixel, by a per-pixel depth comparison that lets real objects cover virtual ones and the reverse.[1]
Apple Vision Pro and Samsung Galaxy XR show the real world this way, a technique usually called passthrough. Apple states that Apple Vision Pro uses 12 cameras, five sensors and six microphones, that its R1 chip "streams new images to the displays within 12 milliseconds", and that its two micro-OLED displays have 23 million pixels in total.[5] In UploadVR's review, David Heaney wrote that "in all but the brightest lighting conditions" the headset's passthrough "does not feel anything like a transparent optic", described it as slightly grainy and slightly blurry, and found that it "is not depth-correct": its scale and perspective are slightly off.[10] Azuma had noted in 1997 that video see-through cameras are usually not located exactly where the user's eyes are, which displaces what the user sees from what they expect to see.[1] Galaxy XR, which went on sale in the United States on 21 October 2025 for US$1,799.99,[11] uses 6.5-megapixel cameras for passthrough video and has an open periphery that leaves the real world visible to the sides and below unless the light blockers included in the box are attached, according to UploadVR.[12]
Optical versus video
Azuma's survey sets out the trade-offs between the two approaches, and concludes that "both optical and video technologies have their roles", with the choice depending on the application. In 1997 many mechanical assembly and repair prototypes used optical designs, possibly because of cost and safety, while most medical prototypes used video, probably for its blending flexibility and extra registration strategies.[1] Jannick Rolland and Henry Fuchs compared the two approaches for 3-D medical visualization in a 2000 paper, from both a technology and a human-factors point of view, and pointed to eye tracking, multifocus planes and hybrid optical/video technology as promising future developments.[13]
| Property | Optical see-through | Video see-through |
|---|---|---|
| View of the real world | Seen directly through the combiners, with a delay of a few nanoseconds | Seen through cameras; video streams have delays in the tens of milliseconds, and digitizing usually adds at least one frame |
| Real-world resolution | Not degraded | Limited to the resolution of the display |
| Loss of power | The headset becomes "a pair of heavy sunglasses" and the wearer can still see | The wearer is "effectively blind" |
| Eye offset | Generally not a difficult design problem | Cameras sit away from the eyes, shifting the viewpoint unless mirrors fold the camera path |
| Occlusion | Virtual objects appear "ghost-like and semi-transparent" because the combiner passes light from both sources | Compositor can choose real, virtual or a blend for each pixel |
| Delay matching | Near-instant real world but delayed virtual image | Real-world video can be delayed to match the graphics |
| Registration | Relies on the head tracker alone | The camera image gives an additional source of registration data |
Source: Azuma (1997), section 3.2.[1]
Tracking and sensing
Azuma calls registration, keeping real and virtual objects properly aligned, "one of the most basic problems currently limiting Augmented Reality applications". Errors that go unnoticed in VR are obvious in a see-through headset, where the conflict is between two things the eye sees, such as a real hand and a virtual one.[1] Self-contained headsets therefore carry their own sensors for inside-out tracking and spatial mapping. Microsoft's specification for HoloLens 2 lists four visible light cameras for head tracking, two infrared cameras for eye tracking, a 1-megapixel time-of-flight depth sensor and an inertial measurement unit, and describes the device as providing world-scale six-degrees-of-freedom tracking, a real-time environment mesh and fully articulated two-handed hand tracking.[9]
Design challenges
Several limits of AR headsets come from the optics and from sharing the view with the real world.
Occlusion and opacity
In an optical design, a filter that selectively blocks real-world light would have to sit where the image is in focus, which makes the optics much more complex; Azuma noted in 1997 that no existing optical see-through HMD did this, so virtual objects looked semi-transparent.[1] Xiong et al. describe dynamically adjusting transmittance with a tunable dimmer as one way to improve contrast.[7] Magic Leap 2 included a "dynamic dimming" feature that TechCrunch described as letting the device "selectively transition between full passthrough and full opacity".[14] See also occlusion.
Brightness and contrast
Xiong et al. measure the contrast of a see-through display by its ambient contrast ratio, and estimate ambient luminance at about 30 nits in a normal living room, 150 nits in an office, 300 nits outdoors on an overcast day and 3,000 nits on a sunny day. They give 3:1 as the minimum for recognizable images and 10:1 or more for outstanding readability; reaching 10:1 on a sunny day requires a display brightness of at least 30,000 nits before optical losses.[7] Azuma had described the same problem: if the environment is too bright it washes out the virtual image, and if it is too dark the virtual image washes out the real world.[1]
Focus and depth cues
In optical designs the virtual image is projected at one distance, often fixed, so the wearer may not be able to see a real object and a virtual object at different distances sharply at the same time.[1] Xiong et al. note that the vergence-accommodation conflict "may be more serious in AR than VR" because the virtual image is laid directly over a real world that has correct depth cues.[7]
Field of view and eyebox
In geometric optical systems, conservation of etendue creates a trade-off between field of view and eyebox, and increasing etendue requires larger optics.[7] Magic Leap's documentation gives the display field of view of Magic Leap 2 as 45 degrees horizontal by 55 degrees vertical (70 degrees diagonal), and warns that it may be significantly smaller if the headset does not fit correctly.[15] Meta states that the silicon carbide waveguides in its Orion prototype, with a refractive index of 2.7, allow a field of view of approximately 70 degrees.[16]
Portability
Azuma observed that VR users mostly stay in one place, while AR requires the user to be where the task is, so AR systems "place a premium on portability", especially the ability to walk around outdoors.[1]
History
Early research
Ivan Sutherland described his head-mounted three-dimensional display in a paper at the 1968 Fall Joint Computer Conference. Special spectacles held two miniature cathode ray tubes, and a mechanical or an ultrasonic sensor measured the position of the user's head. The optics presented a virtual image about eighteen inches in front of each eye with a 40 degree field of view, and half-silvered mirrors in the prisms let the user see the displayed images and objects in the room at the same time, so displayed material could be made "either to hang disembodied in space or to coincide with maps, desk tops, walls, or the keys of a typewriter".[2] The Computer History Museum records that the display was jokingly called "The Sword of Damocles" because of the large overhead beam that supported its weight, and that Sutherland's student Bob Sproull helped design it.[17]
In 1990 Thomas Caudell, a scientist at Boeing, looked for a way to help workers assemble wire bundles for the Boeing 777 without constantly switching between instructions and the work. With his colleague David Mizell he conceived a head-worn see-through display that would superimpose guidance lines on the assembly board, and Caudell named this kind of digital vision "augmented reality". MIT Technology Review reported that tracking was not responsive enough and wearable computers not powerful enough to make the system practical at the time.[3] Caudell and Mizell presented the work at the 1992 Hawaii International Conference on System Sciences as "Augmented reality: an application of heads-up display technology to manual manufacturing processes".[18] By 1997 Azuma's survey was describing Boeing's wire-bundle prototype, a laser printer maintenance application built by Steve Feiner's group at Columbia, and see-through headsets made by Hughes Electronics.[1]
Commercial headsets
Microsoft introduced HoloLens and Windows Holographic at a Windows 10 event on 21 January 2015, describing a headset with see-through lenses, spatial sound, advanced sensors and a holographic processing unit (HPU) that needed no wires and no phone or PC connection.[19] The HoloLens Development Edition went on pre-order on 29 February 2016 for US$3,000 and began shipping on 30 March 2016 in the United States and Canada; Alex Kipman called it "the first and only fully untethered holographic computer".[4] Its successor, HoloLens 2, was announced on 24 February 2019 at US$3,500; TechCrunch reported that its field of view was twice as large as the original's.[20] Microsoft confirmed to UploadVR in October 2024 that HoloLens 2 production had ended, with security updates continuing until 31 December 2027.[21]
Magic Leap released the Magic Leap One Creator Edition on 8 August 2018 for US$2,295, initially with in-person delivery in six US cities. The system was split into a Lightwear headset, a Lightpack computing unit and a controller.[22] In a guest review for Road to VR, Lucas Rizzotto wrote that the headset's see-through lenses were "considerably darker to compensate for lack of image brightness", which could remove the wearer too much from the environment.[23] In July 2022 Magic Leap set a 30 September 2022 release for Magic Leap 2 in three editions (US$3,299 base, US$4,099 Developer Pro and US$4,999 Enterprise), aimed at professional users rather than consumers.[14][24] UploadVR described its dynamic dimming as blocking outside light to make virtual objects more visible in a wider range of lighting conditions.[24] In June 2018 Leap Motion published the designs for Project North Star, an AR headset prototype with a combined field of view of over 100 degrees and a pair of fast-switching LCD displays giving a resolution of 1,600 x 1,400 per eye at 120 frames per second, under the Apache 2.0 license.[8]
Apple announced Apple Vision Pro on 5 June 2023, calling it a "spatial computer", at US$3,499.[5] It went on sale in the United States on 2 February 2024, starting at US$3,499 with 256 GB of storage.[25] On 25 September 2024 Meta showed Orion, a pair of AR glasses it called a "purposeful product prototype"; Meta said Orion would not be sold to consumers and would be used by Meta employees and select external audiences.[6]
| Headset | Maker | See-through method | Key dates | Launch price |
|---|---|---|---|---|
| Head-mounted three-dimensional display | Ivan Sutherland | Optical (half-silvered mirrors) | Described 1968 | Research system |
| HoloLens Development Edition | Microsoft | Optical (see-through lenses) | Announced 21 January 2015; shipped 30 March 2016 | US$3,000 |
| Magic Leap One Creator Edition | Magic Leap | Optical (tinted see-through lenses) | Released 8 August 2018 | US$2,295 |
| Project North Star | Leap Motion | Optical (birdbath-style reflectors) | Designs published June 2018 | Open-source design |
| HoloLens 2 | Microsoft | Optical (waveguides) | Announced 24 February 2019; production ended 2024 | US$3,500 |
| Magic Leap 2 | Magic Leap | Optical, with dynamic dimming | Release set for 30 September 2022 | From US$3,299 |
| Apple Vision Pro | Apple | Video (passthrough cameras) | Announced 5 June 2023; US release 2 February 2024 | US$3,499 |
| Meta Orion | Meta Platforms | Optical (silicon carbide waveguides) | Shown 25 September 2024 | Not sold (prototype) |
| Samsung Galaxy XR | Samsung | Video (passthrough cameras) | US release 21 October 2025 | US$1,799.99 |
Sources: see the History and How it works sections above.
Applications
Azuma's 1997 survey identified six classes of AR applications that had been explored: medical visualization, maintenance and repair, annotation, robot path planning, entertainment, and military aircraft navigation and targeting.[1] Manufacturing was the origin of the term, through Boeing's wire-bundle assembly work.[3] Medicine was an early focus for video see-through designs,[1] and Rolland and Fuchs reviewed medical AR research efforts in the United States and Europe in their comparison of the two headset types.[13]
Later commercial headsets were sold mainly to businesses. HoloLens 2 ships with Microsoft's Dynamics 365 Guides, which helps employees learn new skills, and Dynamics 365 Remote Assist, which lets technicians work with remote collaborators, among its preinstalled software.[9] TechCrunch reported that Magic Leap 2 targeted professional users exclusively, a shift from its predecessor.[14] See Enterprise AR and Consumer AR for the two markets.
See also
References
- ↑ 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.10 1.11 1.12 1.13 1.14 1.15 1.16 1.17 1.18 1.19 1.20 Ronald T. Azuma (1997-08). "A Survey of Augmented Reality". Presence: Teleoperators and Virtual Environments, vol. 6, no. 4, pp. 355-385. MIT Press. https://doi.org/10.1162/pres.1997.6.4.355. Retrieved 2026-09-27.
- ↑ 2.0 2.1 Ivan E. Sutherland (1968). "A head-mounted three dimensional display". Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, part I (AFIPS '68), pp. 757-764. Association for Computing Machinery. https://doi.org/10.1145/1476589.1476686. Retrieved 2026-09-27.
- ↑ 3.0 3.1 3.2 Rachel Metz (2014-02-24). "Augmented Reality Gets to Work". MIT Technology Review. https://www.technologyreview.com/2014/02/24/173872/augmented-reality-gets-to-work/. Retrieved 2026-09-27.
- ↑ 4.0 4.1 Alex Kipman (2016-02-29). "Announcing Microsoft HoloLens Development Edition open for pre-order, shipping March 30". Windows Blogs. Microsoft. https://blogs.windows.com/devices/2016/02/29/announcing-microsoft-hololens-development-edition-open-for-pre-order-shipping-march-30/. Retrieved 2026-09-27.
- ↑ 5.0 5.1 5.2 "Introducing Apple Vision Pro: Apple's first spatial computer". Apple Newsroom. Apple. 2023-06-05. https://www.apple.com/newsroom/2023/06/introducing-apple-vision-pro/. Retrieved 2026-09-27.
- ↑ 6.0 6.1 "Introducing Orion, Our First True Augmented Reality Glasses". Meta Newsroom. Meta Platforms. 2024-09-25. https://about.fb.com/news/2024/09/introducing-orion-our-first-true-augmented-reality-glasses/. Retrieved 2026-09-27.
- ↑ 7.0 7.1 7.2 7.3 7.4 7.5 7.6 7.7 Jianghao Xiong, En-Lin Hsiang, Ziqian He, Tao Zhan, Shin-Tson Wu (2021-10-25). "Augmented reality and virtual reality displays: emerging technologies and future perspectives". Light: Science and Applications, vol. 10, article 216. doi:10.1038/s41377-021-00658-8. https://pmc.ncbi.nlm.nih.gov/articles/PMC8546092/. Retrieved 2026-09-27.
- ↑ 8.0 8.1 Ben Lang (2018-06-06). "Leap Motion Open-sources Project North Star, An AR Headset Prototype With Impressive Specs". Road to VR. https://www.roadtovr.com/leap-motion-reveals-project-north-star-an-open-source-wide-fov-ar-headset-dev-kit/. Retrieved 2026-09-27.
- ↑ 9.0 9.1 9.2 "HoloLens 2 hardware". Microsoft Learn. Microsoft. 2023-02-02. https://learn.microsoft.com/en-us/hololens/hololens2-hardware. Retrieved 2026-09-27.
- ↑ David Heaney (2024-02-16). "Apple Vision Pro Review: A Heavy Portable Cinema & Monitor With A Promising Spatial OS". UploadVR. https://www.uploadvr.com/apple-vision-pro-review/. Retrieved 2026-09-27.
- ↑ "Introducing Galaxy XR: Opening New Worlds". Samsung Newsroom U.S.. Samsung Electronics. 2025-10-21. https://news.samsung.com/us/introducing-galaxy-xr-opening-new-worlds/. Retrieved 2026-09-27.
- ↑ David Heaney (2025-10-21). "Samsung Galaxy XR With Android XR Out Now For $1800, Controllers $250". UploadVR. https://www.uploadvr.com/samsung-galaxy-xr-google-android-xr-out-now/. Retrieved 2026-09-27.
- ↑ 13.0 13.1 Jannick P. Rolland, Henry Fuchs (2000-06). "Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization". Presence: Teleoperators and Virtual Environments, vol. 9, no. 3, pp. 287-309. MIT Press. https://doi.org/10.1162/105474600566808. Retrieved 2026-09-27.
- ↑ 14.0 14.1 14.2 Lucas Matney (2022-07-12). "Magic Leap details pricing for its next augmented reality device". TechCrunch. https://techcrunch.com/2022/07/12/magic-leap-details-pricing-for-its-next-augmented-reality-device. Retrieved 2026-09-27.
- ↑ "Field of View". Magic Leap Developer Documentation. Magic Leap. https://developer-docs.magicleap.cloud/docs/device/hardware/fov/. Retrieved 2026-09-27.
- ↑ "Crystal Clear: Our Silicon Carbide Waveguides and the Path to Orion's Large FoV". Meta. Meta Platforms. 2025-03-06. https://www.meta.com/blog/orion-silicon-carbide-waveguides-ar-glasses-large-field-of-view/. Retrieved 2026-09-27.
- ↑ "The Sword of Damocles: Early head-mounted display". Revolution: The First 2000 Years of Computing. Computer History Museum. https://www.computerhistory.org/revolution/input-output/14/356/1888. Retrieved 2026-09-27.
- ↑ T. P. Caudell, D. W. Mizell (1992). "Augmented reality: an application of heads-up display technology to manual manufacturing processes". Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences, vol. 2, pp. 659-669. IEEE. https://doi.org/10.1109/HICSS.1992.183317. Retrieved 2026-09-27.
- ↑ "Windows 10 announcements on Jan 21, 2015". Microsoft Learn (archived MSDN blog). Microsoft. 2015-01-21. https://learn.microsoft.com/en-gb/archive/blogs/jennifer/windows-10-announcements-on-jan-21-2015. Retrieved 2026-09-27.
- ↑ Frederic Lardinois (2019-02-24). "Say hello to Microsoft's new $3,500 HoloLens with twice the field of view". TechCrunch. https://techcrunch.com/2019/02/24/say-hello-to-microsofts-new-hololens-with-twice-the-field-of-view/. Retrieved 2026-09-27.
- ↑ David Heaney (2024-10-01). "Microsoft Is Discontinuing HoloLens 2 As Production Ends". UploadVR. https://www.uploadvr.com/microsoft-discontinuing-hololens-2/. Retrieved 2026-09-27.
- ↑ Jon Fingas (2018-08-08). "Magic Leap's $2,295 mixed reality headset is available now". Engadget. https://www.engadget.com/2018-08-08-magic-leap-one-creator-edition-available.html. Retrieved 2026-09-27.
- ↑ Lucas Rizzotto (2018-08-17). "Magic Leap One Developer Review - An Ambitious Headset with Untapped Potential". Road to VR. https://roadtovr.com/magic-leap-one-review/. Retrieved 2026-09-27.
- ↑ 24.0 24.1 Ian Hamilton (2022-07-12). "Magic Leap 2 On Sale To General Public In September For $3,299". UploadVR. https://www.uploadvr.com/magic-leap-price-starts-3k/. Retrieved 2026-09-27.
- ↑ "Apple Vision Pro available in the U.S. on February 2". Apple Newsroom. Apple. 2024-01-08. https://www.apple.com/newsroom/2024/01/apple-vision-pro-available-in-the-us-on-february-2/. Retrieved 2026-09-27.