Toggle search
Toggle menu
5.5K
6.1K
231
53.6K
VR & AR Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Augmented reality

vrarwiki.com · VR & AR Wiki
More actions
(Redirected from AR)

Augmented reality (AR) combines a view of the physical environment with computer-generated content. Ronald Azuma's 1997 research definition requires real and virtual content, interaction in real time, and registration in three dimensions. Registration gives virtual content a spatial relationship to the physical scene, beyond a caption fixed to a screen. This definition includes handheld and monitor displays as well as head-mounted displays.[1]

In Paul Milgram and Fumio Kishino's 1994 taxonomy of visual displays, AR is the part of the reality-virtuality continuum in which a predominantly real environment contains virtual objects. Augmented virtuality describes the converse, a predominantly virtual environment containing real elements. Mixed reality covers the combinations between the completely real and completely virtual endpoints. The continuum classifies what a display presents; the authors also distinguish direct viewing from camera-mediated viewing, world knowledge, reproduction fidelity and the sense of being present in the displayed world.[2]

An immersive virtual reality display can instead present a completely synthetic environment.[2] Azuma's definition excludes prerecorded effects and unregistered two-dimensional overlays. His survey also discusses augmentation of sound and touch.[1] A 2001 follow-up survey by Azuma and five co-authors stated the same three properties: combining real and virtual objects in a real environment, running interactively and in real time, and registering real and virtual objects with each other. It did not restrict AR to head-mounted displays or to sight, and it treated the removal of real objects from the perceived environment, which some researchers call mediated or diminished reality, as a subset of AR.[3]

Apple's and Google's phone and tablet AR platforms reached the public in 2017 and 2018. Apple's iOS 11, released on 19 September 2017, brought ARKit applications to iPhone and iPad, and Google moved ARCore out of preview as version 1.0 on 23 February 2018.[4][5][6] Head-worn AR ranges from optical see-through headsets such as Microsoft HoloLens 2, whose production Microsoft confirmed had ended in October 2024,[7][8] to lighter display glasses. In September 2026 Meta began selling its Meta Ray-Ban Display glasses outside the United States, in Canada and the United Kingdom, and opened pre-orders in France, Italy and Germany with availability set for 13 October 2026.[9] Snap's consumer Specs AR glasses have been on pre-order since June 2026 for shipping in fall 2026;[10] on 17 October 2026 the order site still listed them as a pre-order without a shipping date.[11]

Reviewed 17 October 2026. New and changed claims (2001-2026 timeline rows, HoloLens 2, Meta Ray-Ban Display, Orion, Snap Specs status, OpenXR, WebXR AR Module, Microsoft focus guidance, IVAS, FDA list, Azuma 1997 and 2001, Hoffman 2008, PTAM, ISMAR reviews, Speicher 2019) checked against the cited sources. About review dates.

Technologies

Display design and spatial tracking are separate choices. Kato and Billinghurst's 1999 system distinguished optical and video see-through displays while treating camera, display and eye calibration as additional registration problems.[12] Azuma and colleagues' 2001 survey grouped the displays used for AR into head-worn, handheld and projective types, and divided head-worn displays into optical see-through and video see-through designs.[3]

Display arrangement View of the physical environment How content is combined
Phone, tablet or monitor AR A camera image shown on a screen Graphics are composited with camera images. ARCore supplies camera pose estimates for phone AR.[13]
Optical see-through head-mounted display Directly through transparent optics An optical combiner adds display light while transmitting light from the surroundings.[1]
Video see-through head-mounted display Cameras supply the view on the headset's screens Camera images and graphics are composited. Viewpoint and display calibration affect the resulting view.[12]
Projection or spatial AR The physical surfaces themselves Projectors place imagery on real surfaces. Perspective-correct three-dimensional imagery can require the viewer's position and a model of the projection surfaces.[14]

Optical see-through head-mounted displays

Modern optical see-through designs include waveguides. For example, the fifth-generation Spectacles announced in September 2024 use liquid crystal on silicon micro-projectors and waveguides to produce their see-through AR display.[15]

The optical systems discussed by Azuma add display light without erasing light from real objects. This makes hiding a physical object behind an opaque virtual object difficult. Image contrast against the surroundings is another constraint.[1]

Focus is a further constraint. In most head-mounted displays the eyes accommodate to the fixed focal distance of the display while converging on the apparent depth of the virtual object; Microsoft's design guidance says that breaking this natural link, the vergence-accommodation conflict, leads to visual discomfort or fatigue.[16] For HoloLens (1st gen) and HoloLens 2, Microsoft states that the displays are fixed at an optical distance of approximately 2.0 m, gives 1.25 m to 5 m as the optimal zone for hologram placement, and recommends against ever presenting holograms closer than 40 cm.[16] In a laboratory study using a display that could present correct or nearly correct focus cues, David Hoffman, Ahna Girshick, Kurt Akeley and Martin Banks found that correct focus cues shortened the time needed to identify a stereoscopic stimulus, improved stereoacuity in a time-limited task, reduced distortions in perceived depth and reduced fatigue and discomfort. Their display was a research apparatus, not an AR headset.[17] Outdoor use adds other problems. Writing in 2001 from their experience building outdoor AR systems, Azuma and colleagues reported that displays such as the Sony Glasstron washed out completely in bright sunlight and had a fixed focus several feet away, often closer than the outdoor landmarks.[3]

Input varies by device and application. Spectacles' documented interaction tools include hand gestures, voice interaction and a phone used as a secondary controller; these are ways to select or manipulate content, rather than substitutes for tracking its spatial position.[18]

Phone and tablet AR

ARCore combines visual features and inertial measurements to estimate camera pose. Rendering from that estimate makes content appear spatially attached to the camera view.[13] Apple's ARKit provides device motion tracking, camera capture and scene processing for iOS and iPadOS applications, as documented in its ARKit 4 developer session.[19]

Using computer vision to track geometric features is different from recognizing an object's identity. ARCore detects planes from feature points and warns that surfaces without texture may not be detected properly.[13] Scene understanding can also supply a mesh for environment interactions, as demonstrated by Apple's visionOS RealityKit tools.[20]

Video see-through displays

Kato and Billinghurst's conferencing prototype used an optical see-through HMD and a camera for marker tracking. Their comparison explains that video see-through displays can present a camera viewpoint displaced from the eye and that low frame rates affect interaction. Optical see-through calibration additionally requires camera, display and eye relationships.[12]

Projection and spatial AR

Projection mapping can align imagery with physical surfaces. In their 1998 spatial AR paper, Raskar, Welch and Fuchs described a projector-based prototype that used surface geometry, projector calibration and the viewer's position to render virtual objects on irregular surfaces. The method made the environment a display surface, so the user did not need to wear an HMD.[14]

That prototype also illustrates the limits of projection. Dark or highly reflective surfaces and ambient lighting affected image contrast; a person could cast a shadow into a front projection. Its perspective-correct rendering supported one active tracked viewpoint at a time, with multiplexing proposed for additional viewers. These are constraints of the described arrangement, rather than specifications for every projection system.[14]

Tracking, registration and limits

Tracking estimates the pose of a device or another tracked object. Registration concerns how the rendered content aligns with the physical scene from the viewer's position. Azuma and Bishop's 1994 optical see-through research separated errors from optics, mechanical alignment, tracking and viewing parameters from errors caused by system delay during head movement. A scene can therefore be tracked while its virtual content is still visibly misaligned.[21]

Technique or representation Role in an AR system Limitation or distinction
Fiducial markers Kato and Billinghurst used known-size square patterns and template matching for pose estimation.[12] Visible markers supplied coordinate references; display and eye calibration remained necessary.[12]
SLAM and visual-inertial tracking ARCore combines camera features and inertial measurements to estimate pose.[13] The environment estimate can change; plane detection needs usable features.[13]
Spatial anchors An anchor provides a pose to which virtual content is attached. ARCore updates attached content when its world-space estimate changes.[22] Separate anchors can adjust independently, causing objects to shift relative to one another. Local, Cloud and Geospatial anchors have different validity and placement scopes.[22]
Plane detection and meshes Surfaces can support placement, collision tests and occlusion. Magic Leap 2's real-time sensing provides planes and triangle meshes.[23] Its documentation recommends mostly static environments. Moving objects, thin structures and pose drift can produce incomplete or misaligned reconstructed geometry.[23]
Depth maps and occlusion A depth estimate can determine which pixels of a virtual object should be hidden by a nearer physical object.[24] ARCore's depth-from-motion results depend on device movement, scene features and device support. Dedicated depth sensors are optional, and Depth must be explicitly enabled.[24]

Calibration and latency

Calibration establishes the relationships needed to render from the correct viewpoint. These can include camera projection parameters, tracker-to-eye position and orientation, and display geometry. In Caudell and Mizell's manufacturing prototype, a registration device located the eye relative to the tracker and work platform. The authors also described how a headset moving on the wearer's face could introduce new registration errors and require readjustment.[25]

Latency matters when the viewer moves between a pose measurement and the display of the corresponding image. Azuma and Bishop tested head-motion prediction using an optical tracker, inertial sensors and their laboratory rendering system. They showed reduced dynamic error in that setup, but warned that prediction cannot substitute for reducing system delay. Neither that result nor a manufacturer's latency figure establishes the registration accuracy of another device or application.[21]

Persistence and environmental change

An anchor is a spatial reference within a tracking system; persistence across sessions needs a mechanism to recover that reference. For example, Magic Leap 2 distinguishes its temporary session map from saved Spaces. Tracking can continue without successful localization into a saved Space, and localization can fail in darkness, with too few distinctive features or in a moving crowd. The system may create a new session origin after unrecoverable head-tracking loss.[26]

Depth and geometry also remain estimates. Google's ARCore guidance says its depth data is most accurate at roughly 0.5-5 metres and improves as the device observes the scene from different positions. Its documentation identifies featureless surfaces as a source of imprecise depth. The range is ARCore guidance, rather than a universal AR operating range.[24]

Augmented reality history timeline

Studierstube (Image: www.informit.com)
The Touring Machine (Image: www.informit.com)
ARQuake (Image: www.informit.com)
The Invisible Train (Image: www.informit.com)
Date or publication Example What the source documents
Fictional precursor L. Frank Baum's The Master Key The story's Character Marker spectacles make letters indicating a person's character appear on their forehead. This is a fictional device, rather than a functioning display or a technical demonstration.[27]
1962 patent Morton Heilig's Sensorama US patent 3,050,870 was filed on 10 January 1961 and granted on 28 August 1962. It describes a multisensory simulator. The patent documents a simulator, rather than an interactive overlay registered with the physical scene.[28]
1968 Sutherland's head-mounted display Ivan Sutherland's paper describes perspective wireframe drawings viewed through half-silvered optics. It reports both mechanical and ultrasonic head-position sensors. The displayed perspective changed with the observer's head position.[29]
1985 publication VIDEOPLACE Myron Krueger, Thomas Gionfriddo and Katrin Hinrichsen described combining a participant's live video image with a computer graphic world whose objects reacted to the participant's movement. The paper calls the approach artificial reality.[30]
1992 Manufacturing overlays and the AR name Thomas Caudell and David Mizell's Boeing paper uses the term augmented reality for a display combined with head tracking and workplace registration. Their demonstrations included wiring formboards, connector assembly and composite cloth layup. They described alignment, field-of-view and delay limitations of the demonstration hardware.[25]
Report dated September 1992 Virtual Fixtures Louis Rosenberg's Armstrong Laboratory report introduced computer-generated perceptual overlays for a telepresence workspace. The report considered haptic and auditory aids. Its identifier is AL/CF-TR-1994-0089 despite the September 1992 report date.[31]
1992 Ultrasound imagery within a patient Michael Bajura, Henry Fuchs and Ryutarou Ohbuchi reported overlaying ultrasound slices in a video see-through HMD. Their January 1992 experiment scanned a volunteer who was 38 weeks pregnant. The authors reported that conflicting depth cues initially made the imagery appear pasted onto her rather than inside her; this was an exploratory visualization, not evidence of clinical efficacy.[32]
1993 KARMA Steven Feiner, Blair MacIntyre and Dorée Seligmann described Knowledge-based Augmented Reality for Maintenance Assistance. Their laser-printer testbed generated explanatory graphics for simple maintenance actions and tracked the user's head and printer parts. The paper identifies focus, calibration and display limitations.[33]
1993 Chameleon George Fitzmaurice's spatially aware palmtop prototype combined a small display with a six-degree-of-freedom sensor. It explored an information lens associated with physical objects, including a wall map. The prototype was tethered for video and tracking.[34]
1995 NaviCam Jun Rekimoto and Katashi Nagao's portable video see-through device recognized color-code IDs in the environment and overlaid situation-sensitive information. The paper describes a camera-equipped display connected to a workstation.[35]
1996 Studierstube Dieter Schmalstieg, Anton Fuhrmann, Zsolt Szalavári and Michael Gervautz's technical report presented an architecture for multiple users sharing virtual objects through see-through HMDs.[36]
1997 The Touring Machine Feiner, MacIntyre, Tobias Höllerer and Anthony Webster described a mobile campus information prototype using a see-through headworn display, differential GPS and orientation sensing. A backpack computer generated graphics, while a handheld computer supplied a complementary interface.[37]
1999 Marker tracking and display calibration Kato and Billinghurst documented marker tracking and HMD calibration for an AR conferencing system. Known square markers supplied coordinate references for virtual participant displays and a shared whiteboard.[12]
2000 ARQuake The Wearable Computer Lab's outdoor AR adaptation of Quake used a head-mounted display, a mobile computer, head tracking and GPS. Its project archive lists the October 2000 ISWC paper and describes alignment, outdoor tracking and lighting problems.[38]
2003 Self-contained handheld AR Daniel Wagner and Dieter Schmalstieg reported video see-through rendering and ARToolKit marker tracking running on an HP iPAQ PDA with an attached camera. An optional server could offload tracking. This was a PDA prototype with hardware limits, rather than a smartphone release.[39]
2004 The Invisible Train The collaborative handheld game displayed virtual trains on a real wooden track through PDA video displays and synchronized game state over a wireless network. It was presented at SIGGRAPH 2004's Emerging Technologies show.[40][41]
2007 Parallel Tracking and Mapping (PTAM) Georg Klein and David Murray's ISMAR paper estimated camera pose in an unknown scene by splitting tracking and mapping into two threads on a dual-core computer: one tracked hand-held camera motion while the other built a 3D map of point features from earlier video frames.[42]
2011 KinectFusion Richard Newcombe and colleagues' ISMAR paper mapped and tracked indoor environments in real time with a low-cost RGB-D camera such as the Microsoft Kinect. A 2018 review identified it as the most highly cited ISMAR paper of 2008-2017.[43]
2015 HoloLens announcement Microsoft introduced Microsoft HoloLens at its Windows 10 event on 21 January 2015, describing an untethered device for displaying digital content in the surroundings. The announcement is distinct from later device shipments.[44]
2016 HoloLens Development Edition Microsoft opened pre-orders on 29 February 2016 and said devices would start shipping in the United States and Canada on 30 March 2016 at US$3,000, to developers who had applied and received an invitation.[45]
2016 Pokémon GO launch Niantic announced Pokémon GO availability on the App Store and Google Play in Australia, New Zealand and the United States on 6 July 2016, with other countries to follow.[46]
2017 ARCore preview and ARKit in iOS 11 Google released an ARCore preview on 29 August 2017 for the Pixel and Samsung Galaxy S8 running Android 7.0 or later, with motion tracking, horizontal surface detection and light estimation.[47] Apple released iOS 11 on 19 September 2017 and said that hundreds of millions of people could then use AR apps on iPhone and iPad.[5][4]
2018 ARCore 1.0 Google took ARCore out of preview on 23 February 2018, allowing developers to publish AR apps on the Play Store. It supported 13 device models at launch; Google said ARCore worked on 100 million Android smartphones.[6]
2019 HoloLens 2 and OpenXR 1.0 Microsoft introduced Microsoft HoloLens 2 at MWC Barcelona on 24 February 2019 at US$3,500, saying it had more than doubled the field of view while keeping the same pixel density.[48] The Khronos Group released the OpenXR 1.0 API standard for VR and AR devices on 29 July 2019.[49]
2024 Orion prototype and HoloLens 2 last-time buy Meta presented Orion on 25 September 2024 as what it called its first true AR glasses, a product prototype for employees and selected external audiences that would not be sold to consumers.[50] In October 2024 Microsoft announced a last-time buy for HoloLens 2 and said updates for critical security issues and major regressions would continue through 31 December 2027.[8]
2025 Meta Ray-Ban Display Meta announced glasses with a full-color display and the Meta Neural Band wristband, starting at US$799 for both, with US retail availability from 30 September 2025.[51]
2026 Snap Specs pre-orders Snap opened pre-orders for Snap Specs on 16 June 2026 at US$2,195, with a US$200 refundable deposit, for shipping in the fall in the United States, the United Kingdom and France.[10]

Hardware and software platforms

Platform or framework Role Dated context
ARKit and RealityKit Apple's tracking and scene-processing framework, with RealityKit providing rendering and interaction tools. ARKit 4 documents iOS and iPadOS AR capabilities; Apple's WWDC25 RealityKit session documents newer access to ARKit data and environment interaction on visionOS.[19][20]
visionOS Apple's spatial application platform. Apple's 2025 presentation describes shared world anchors and content persistence in visionOS 26. These are version-specific capabilities.[52]
ARCore Google's mobile AR SDK, including motion tracking, planes, anchors and optional depth. Device and feature support varies. The Depth API documentation, checked on 13 October 2026, requires supported hardware and explicit activation.[13][24]
Android XR A platform with developer tools for XR devices. ARCore for Jetpack XR documents perception features such as planes and spatial anchors. Its session requires an Android XR device or a supported ARCore device; features may need configuration.[53]
Snap OS and Lens Studio Runtime and creation tools for Spectacles experiences. Snap introduced Snap OS with fifth-generation Spectacles in September 2024. Developer documentation checked in October 2026 still identifies Spectacles (2024) and its interaction tools.[15][18]
WebXR A web API for sessions, tracked poses and input from XR devices. The June 2026 Candidate Recommendation Draft identifies immersive AR sessions and refers their behavior to the AR Module. Applications can query session support; a positive query does not guarantee that a session request succeeds.[54] The separate WebXR Augmented Reality Module, a Candidate Recommendation Draft dated 25 April 2025, defines the immersive-ar session mode and an environmentBlendMode attribute: alpha-blend for pass-through displays, additive for additive-light displays, and opaque, which must not be used for immersive-ar sessions.[55]
OpenXR A Khronos Group API standard for cross-platform access to VR, AR and mixed reality devices. Khronos released OpenXR 1.0 on 29 July 2019. OpenXR 1.1, released on 15 April 2024, moved widely used extensions such as a local floor reference space and a palm-pose grip surface into the core specification.[49][56]
Windows Mixed Reality A historical Windows XR platform and headset ecosystem. Microsoft removed its PC components in Windows 11 version 24H2. Its published guidance allows existing devices to work with Steam through November 2026 on Windows 11 version 23H2 and says support ends after November 2026. This PC platform deadline should not be applied to HoloLens solely because both use mixed reality terminology.[57][58]

Related technology and company articles include SmartEyeglass, Magic Leap and Snap.

AR headsets

See also: AR Glasses

Smart glasses can provide camera and audio functions, as in Ray-Ban Stories, which Meta described as a step toward AR glasses.[59] The table distinguishes documented display, tracking and input functions. A dated list of released AR glasses and headsets is at List of AR Glasses and Headsets.

Android XR's device guidance distinguishes camera-mediated headsets, transparent wired XR glasses, and audio or display glasses. Audio glasses provide sound without a visual overlay; display glasses add a visual output. World registration depends on the application and tracking arrangement.[60]

Device articles Documented arrangement Classification context
Microsoft HoloLens 2 See-through waveguide lenses, six-degree-of-freedom tracking and spatial mapping.[58] An optical see-through device with environmental tracking; its Windows Holographic operating system is distinct from the removed PC Windows Mixed Reality components.[58][57] Microsoft has ended production and plans updates for critical security issues through 31 December 2027.[7][8]
Magic Leap 2 Device and developer documentation describes six-degree-of-freedom tracking, saved Spaces and spatial anchors.[26] A spatial AR system whose content placement depends on tracking and localization. Its saved maps and runtime sensing are different representations.[26][23]
Snap Spectacles 5 Standalone see-through AR display with cameras used by the Snap Spatial Engine.[15] Snap's fifth-generation device, announced in 2024 for its developer program. That source does not establish the capabilities of earlier camera-only Spectacles.[15]
Xreal One Pro A tethered virtual-screen display with native three-degree-of-freedom modes; XREAL's Japanese product listing specifies that six-degree-of-freedom anchoring requires its separately sold Eye accessory.[61][62] Screen anchoring and the supported accessory configuration matter. A virtual screen is not evidence that every application reconstructs or interacts with the surroundings.[61][62]
Meta Ray-Ban Display An information display for messages, captions and previews, with a wristband for input. Walking navigation was announced as a beta in selected cities.[51] The 2025 announcement describes information-display functions; it does not document rendering world-registered three-dimensional objects.[51] Sales expanded beyond the United States in September 2026.[9]
Meta Orion Transparent lenses with what Meta calls holographic displays.[50] A 2024 product prototype shown to employees and selected external audiences; Meta said it would not reach consumers.[50]
Snap Specs Standalone see-through glasses with in-house LCoS displays and waveguides, a 51-degree diagonal field of view, Snap OS, and head tracking, hand tracking, environment meshing and spatial anchoring handled by a dedicated Snapdragon chip, according to UploadVR's report of Snap's announcement.[10] A consumer see-through AR display with spatial anchoring. Pre-orders opened in June 2026; it had not shipped as of 17 October 2026.[10][11]
Ray-Ban Stories and Oakley Meta HSTN Camera capture and audio output are documented functions.[59][63] Camera and audio smart glasses belong in the wider eyewear directory; these functions should not be presented as a spatial AR display. Meta itself described Stories as a step toward AR glasses.[59][63]

Further eyewear articles

Further reading on eyewear and historical devices:

Application areas

Azuma's 1997 survey identified at least six classes of AR applications that had been explored: medical visualization, maintenance and repair, annotation, robot path planning, entertainment, and military aircraft navigation and targeting.[1] It expected aircraft manufacturing, where Boeing was exploring optical approaches and McDonnell Douglas video approaches, to be among the first deployed head-mounted AR uses. It also predicted that stringent registration requirements and the consequences of mistakes would delay routine medical use, and that AR would probably be used for medical training before it became common in surgery.[1]

Area Documented example Scope and limits
Military aviation Azuma's survey describes head-up displays and helmet-mounted sights that superimpose vector graphics on a pilot's view, sometimes registered with targets in the environment to aim weapons.[1] A description of practice as of 1997.[1]
Dismounted soldiers The US Army's Integrated Visual Augmentation System (IVAS) includes a heads-up display originally based on Microsoft's commercially available HoloLens. The Army approved transferring Microsoft's IVAS contract to Anduril Industries on 10 April 2025 and later awarded Soldier Borne Mission Command (SBMC) contracts to Anduril (US$159 million) and Rivet (US$195 million), reportedly covering an 18-month rapid prototyping effort.[64] The July 2026 Congressional Research Service update quotes the Government Accountability Office as stating that none of the 10,000 developed IVAS units will be fielded.[64]
Medical devices The US Food and Drug Administration defines AR as overlaying or mixing simulated digital imagery with the real world, viewed through a camera or a display such as a smartphone, head-mounted display or heads-up display. Its list of authorized AR and VR medical devices includes surgical navigation and planning systems; Augmedics' xvision Spine System has several entries, with final decisions ranging from 29 November 2019 (K192186) to 3 October 2025 (K251639).[65] The FDA states that the list is not a comprehensive resource of AR/VR medical devices.[65]
Film and television At SIGGRAPH '95, exhibitors showed virtual sets that composited actors, filmed by a tracked motion-control camera in front of a blue screen, into 3D virtual backgrounds in real time.[1] Azuma's survey says the entertainment industry saw virtual sets as a way to reduce production costs.[1]

Apps

See also: Augmented reality use cases

These examples cover camera-based encounters, model previews and approximate measurement, with links to existing application articles.[66][67][68]

Example Use Scope and limits
Pokémon GO Location-based play and camera-based AR encounters.[46] The developer's help page describes enabling AR encounters on supported devices and detecting the surroundings with the camera.[66]
AR Quick Look Viewing and placing USDZ objects in the physical surroundings. Apple's gallery documents iPhone, iPad and Apple Vision Pro viewing and shows browser integration examples; capability depends on the supported device and software.[67]
IKEA Place Previewing furniture models in a room. IKEA's September 2019 account describes the app's 2017 launch and later update. It is a historical example, not a statement of present app-store availability.[69]
Measure on iPhone Estimating dimensions through the camera view. Apple describes the measurements as approximate.[68]

Research

Terminology

Definitions of the neighbouring terms still vary. In a 2019 CHI paper, Maximilian Speicher, Brian D. Hall and Michael Nebeling interviewed ten AR and VR experts from academia and industry and surveyed 68 papers. They found no universally agreed definition of mixed reality and identified six partially competing notions. They then proposed a framework that characterizes applications by seven dimensions: number of environments, number of users, level of immersion, level of virtuality, degree of interaction, input and output.[70]

Research trends

Azuma and colleagues wrote that several AR conferences began in the late 1990s, including the International Workshop and Symposium on Augmented Reality and the International Symposium on Mixed Reality.[3] Kim and colleagues describe the later International Symposium on Mixed and Augmented Reality (ISMAR) as the leading academic conference for AR.[43] In a 2008 review of the conference's first ten years, Feng Zhou, Henry Been-Lirn Duh and Mark Billinghurst concentrated on tracking, interaction and display research.[71]

Kim, Billinghurst, Gerd Bruder, Duh and Gregory Welch repeated the method for ISMAR papers from 2008 to 2017 in a 2018 review. They report that five of the top ten papers of the first decade concerned tracking, and that tracking accounted for 20.1% of 313 topic classifications in the earlier review. Among 264 papers of the second decade, tracking remained the most frequent topic at 19.4% of 439 classifications in the original eleven categories; evaluation rose to 16.4%, applications and rendering each reached 12.5%, and calibration dropped out of the five most frequent topics. The authors suggest that the growth in evaluation papers may reflect AR systems getting closer to real users and more rigorous review.[43]

Kim and colleagues also trace the spread of tracking that needs no prepared environment. Their review notes that Klein and Murray's PTAM work received the ISMAR best paper award in 2007 and an ISMAR impact paper award in 2017, that they ran PTAM on a camera phone in 2009, and that SLAM then became widely used for AR tracking in ISMAR papers.[43] ARCore, for example, describes its motion tracking as simultaneous localization and mapping that combines visual information with inertial measurements from the device's IMU.[13]

Open problems

In 2001 Azuma and colleagues grouped the obstacles to wider use into technological limitations, user interface limitations and social acceptance. Technological limits included displays, trackers and systems that needed to become more accurate, lighter, cheaper and less power hungry, tracking in unprepared environments, and calibration procedures too complicated for end users. Interface research had concentrated on low-level perceptual issues such as depth perception and latency, rather than on what information to show and how users should query it. Social acceptance covered fashion and privacy, since the tracking needed to display information could also be used for monitoring and recording.[3] Privacy questions specific to current devices are covered in Privacy in virtual and augmented reality.

Developer resources

Developer APIs

Wagner and Schmalstieg's 2003 handheld system used ARToolKit for marker tracking directly on a PDA.[39] ARKit and ARCore supply mobile AR facilities, while RealityKit and ARCore for Jetpack XR expose platform-specific scene and perception tools.[19][13][20][53]

Developers must distinguish a session's available features from a platform's overall feature list. WebXR supplies a support query before an application requests an immersive session. ARCore Depth needs device support and explicit activation; Android XR perception features may also be disabled until configured. These checks affect whether an experience can run and what environmental information it can use.[54][24][53]

See also

References

  1. ↑ 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 Azuma, Ronald T. (1997). "A Survey of Augmented Reality". Presence: Teleoperators and Virtual Environments. pp. 355-385. doi:10.1162/pres.1997.6.4.355. https://www.ronaldazuma.com/papers/ARpresence.pdf. Retrieved 2026-10-13.
  2. ↑ 2.0 2.1 Milgram, Paul; Kishino, Fumio (1994-12). "A Taxonomy of Mixed Reality Visual Displays". IEICE Transactions on Information and Systems. pp. 1321-1329. https://scispace.com/pdf/a-taxonomy-of-mixed-reality-visual-displays-54v3x9pjl3.pdf. Retrieved 2026-10-13.
  3. ↑ 3.0 3.1 3.2 3.3 3.4 Ronald Azuma, Yohan Baillot, Reinhold Behringer, Steven Feiner, Simon Julier, Blair MacIntyre (2001). "Recent Advances in Augmented Reality". IEEE Computer Graphics and Applications, vol. 21, no. 6, pp. 34-47. doi:10.1109/38.963459. https://www.ronaldazuma.com/papers/cga2001.pdf. Retrieved 2026-10-17.
  4. ↑ 4.0 4.1 Cherlynn Low (2017-09-12). "iOS 11 comes to your iPhone and iPad on September 19th". Engadget. https://www.engadget.com/2017-09-12-ios-11-sept-11-public.html. Retrieved 2026-10-17.
  5. ↑ 5.0 5.1 "iOS 11 is available tomorrow". Apple Newsroom. Apple. 2017-09-18. https://www.apple.com/newsroom/2017/09/ios-11-available-tomorrow/. Retrieved 2026-10-17.
  6. ↑ 6.0 6.1 Anuj Gosalia (2018-02-23). "Announcing ARCore 1.0 and new updates to Google Lens". The Keyword, Google. https://blog.google/products/arcore/announcing-arcore-10-and-new-updates-google-lens/. Retrieved 2026-10-17.
  7. ↑ 7.0 7.1 David Heaney (2024-10-01). "Microsoft Is Discontinuing HoloLens 2 As Production Ends". UploadVR. https://www.uploadvr.com/microsoft-discontinuing-hololens-2/. Retrieved 2026-10-17.
  8. ↑ 8.0 8.1 8.2 Scott Hayden (2024-10-01). "Microsoft Discontinues HoloLens 2, Support to End in 2027 with No Successor in Sight". Road to VR. https://www.roadtovr.com/microsoft-hololens-2-discontinued-support-2027-hololens-3/. Retrieved 2026-10-17.
  9. ↑ 9.0 9.1 Scott Hayden (2026-09-23). "Meta Ray-Ban Display is Finally Rolling Out Internationally Starting Today". Road to VR. https://www.roadtovr.com/meta-ray-ban-display-int-launch-connect-2026/. Retrieved 2026-10-17.
  10. ↑ 10.0 10.1 10.2 10.3 David Heaney (2026-06-16). "Snap Opens Preorders For Specs, True AR Glasses Shipping This Fall For $2195". UploadVR. https://www.uploadvr.com/snap-specs-design-revealed-preorders-open-price/. Retrieved 2026-10-17.
  11. ↑ 11.0 11.1 "SPECS". SPECS. Snap Inc.. https://www.specs.com/. Retrieved 2026-10-17.
  12. ↑ 12.0 12.1 12.2 12.3 12.4 12.5 Kato, Hirokazu; Billinghurst, Mark (1999). "Marker Tracking and HMD Calibration for a Video-based Augmented Reality Conferencing System". Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality. doi:10.1109/IWAR.1999.803809. https://www.hitl.washington.edu/artoolkit/Papers/IWAR99.kato.pdf. Retrieved 2026-10-13.
  13. ↑ 13.0 13.1 13.2 13.3 13.4 13.5 13.6 13.7 "Fundamental concepts". Google for Developers. https://developers.google.com/ar/develop/fundamentals. Retrieved 2026-10-13.
  14. ↑ 14.0 14.1 14.2 Raskar, Ramesh; Welch, Greg; Fuchs, Henry (1998-11-01). "Spatially Augmented Reality". First International Workshop on Augmented Reality. https://www.researchgate.net/publication/2502539_Spatially_Augmented_Reality. Retrieved 2026-10-13.
  15. ↑ 15.0 15.1 15.2 15.3 "SPS 2024 | Introducing New Spectacles and Snap OS: The Next Frontier of AR Glasses". Snap Newsroom. 2024-09-17. https://newsroom.snap.com/sps-2024-spectacles-snapos. Retrieved 2026-10-13.
  16. ↑ 16.0 16.1 "Comfort - Mixed Reality". Microsoft Learn. Microsoft. https://learn.microsoft.com/en-us/windows/mixed-reality/design/comfort. Retrieved 2026-10-17.
  17. ↑ David M. Hoffman, Ahna R. Girshick, Kurt Akeley, Martin S. Banks (2008-03-28). "Vergence-accommodation conflicts hinder visual performance and cause visual fatigue". Journal of Vision, vol. 8, no. 3, article 33. doi:10.1167/8.3.33. https://doi.org/10.1167/8.3.33. Retrieved 2026-10-17.
  18. ↑ 18.0 18.1 "Introduction to Spectacles". Snap for Developers. https://developers.snap.com/spectacles/get-started/introduction. Retrieved 2026-10-13.
  19. ↑ 19.0 19.1 19.2 "Explore ARKit 4". Apple Developer, WWDC20. 2020. https://developer.apple.com/videos/play/wwdc2020/10611/. Retrieved 2026-10-13.
  20. ↑ 20.0 20.1 20.2 "What's new in RealityKit". Apple Developer, WWDC25. 2025. https://developer.apple.com/videos/play/wwdc2025/287/. Retrieved 2026-10-13.
  21. ↑ 21.0 21.1 Azuma, Ronald; Bishop, Gary (1994). "Improving Static and Dynamic Registration in an Optical See-through HMD". Computer Graphics Proceedings, Annual Conference Series, SIGGRAPH 1994. pp. 197-204. https://www.ronaldazuma.com/papers/sig94paper.pdf. Retrieved 2026-10-13.
  22. ↑ 22.0 22.1 "Working with Anchors". Google for Developers. https://developers.google.com/ar/develop/anchors. Retrieved 2026-10-13.
  23. ↑ 23.0 23.1 23.2 "Real-time World Sensing". Magic Leap Developer Documentation. 2025-11-11. https://developer-docs.magicleap.cloud/docs/guides/features/spatial-mapping/. Retrieved 2026-10-13.
  24. ↑ 24.0 24.1 24.2 24.3 24.4 "Depth adds realism". Google for Developers. https://developers.google.com/ar/develop/depth. Retrieved 2026-10-13.
  25. ↑ 25.0 25.1 Caudell, Thomas P.; Mizell, David W. (1992). "Augmented Reality: An Application of Heads-Up Display Technology to Manual Manufacturing Processes". Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences. pp. 659-669. doi:10.1109/HICSS.1992.183317. https://www.editionmultimedia.fr/wp-content/uploads/2022/11/Augmented-Reality-Tom-Caudell-et-David-Mizell-IEEE1992.pdf. Retrieved 2026-10-13.
  26. ↑ 26.0 26.1 26.2 "Spatial Mapping & Localization". Magic Leap Developer Documentation. 2025-11-11. https://developer-docs.magicleap.cloud/docs/guides/features/spaces/. Retrieved 2026-10-13.
  27. ↑ L. Frank Baum. "The Master Key". Project Gutenberg. https://www.gutenberg.org/files/436/436-h/436-h.htm. Retrieved 2026-10-13.
  28. ↑ Morton L. Heilig (1962-08-28). "Sensorama simulator, US3050870A". Google Patents, United States patent text. https://patents.google.com/patent/US3050870A/en. Retrieved 2026-10-13.
  29. ↑ Sutherland, Ivan E. (1968). "A head-mounted three dimensional display". AFIPS Fall Joint Computer Conference. pp. 757-764. https://web.stanford.edu/class/ee267/notes/sutherland_hmd.pdf. Retrieved 2026-10-13.
  30. ↑ Krueger, Myron W.; Gionfriddo, Thomas; Hinrichsen, Katrin (1985-04). "VIDEOPLACE—an artificial reality". ACM SIGCHI Bulletin 16 (4): 35-40. Retrieved 2026-10-13.
  31. ↑ Louis B. Rosenberg (1992-09). "The Use of Virtual Fixtures as Perceptual Overlays to Enhance Operator Performance in Remote Environments". USAF Armstrong Laboratory, report AL/CF-TR-1994-0089, author-uploaded copy. https://www.researchgate.net/publication/235116787_The_Use_of_Virtual_Fixtures_as_Perceptual_Overlays_to_Enhance_Operator_Performance_in_Remote_Environments. Retrieved 2026-10-13.
  32. ↑ Bajura, Michael; Fuchs, Henry; Ohbuchi, Ryutarou (1992). "Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient". Computer Graphics, SIGGRAPH 1992. pp. 203-210. https://henryfuchs.web.unc.edu/wp-content/uploads/sites/4964/2013/05/Merging-Virtual-Objects-with-the-Real-World-Seeing-Ultrasound-Imagery-within-the-Patient.pdf. Retrieved 2026-10-13.
  33. ↑ Feiner, Steven; MacIntyre, Blair; Seligmann, Dorée (1993-07). "Knowledge-based augmented reality". Communications of the ACM 36 (7): 53-62. doi:10.1145/159544.159587. Retrieved 2026-10-13.
  34. ↑ Fitzmaurice, George W. (1993-07). "Situated Information Spaces and Spatially Aware Palmtop Computers". Communications of the ACM 36 (7): 39-49. Retrieved 2026-10-13.
  35. ↑ Rekimoto, Jun; Nagao, Katashi (1995). "The World through the Computer: Computer Augmented Interaction with Real World Environments". Proceedings of UIST 1995. doi:10.1145/215585.215639. https://3dvar.com/Rekimoto1995The.pdf. Retrieved 2026-10-13.
  36. ↑ Dieter Schmalstieg, Anton Fuhrmann, Zsolt Szalavári and Michael Gervautz (1996-04). "'Studierstube' - An Environment for Collaboration in Augmented Reality". TU Wien, technical report TR-186-2-96-17. https://www.cg.tuwien.ac.at/research/publications/1996/schmalstieg-1996-SEC/. Retrieved 2026-10-13.
  37. ↑ Feiner, Steven; MacIntyre, Blair; Höllerer, Tobias; Webster, Anthony (1997). "A Touring Machine: Prototyping 3D Mobile Augmented Reality Systems for Exploring the Urban Environment". Proceedings of the First International Symposium on Wearable Computers. https://sites.cs.ucsb.edu/~holl/pubs/feiner-1997-iswc.pdf. Retrieved 2026-10-13.
  38. ↑ "ARQuake: Interactive Outdoor Augmented Reality Collaboration System". Wearable Computer Lab project archive, 1998-2006. https://www.tinmith.net/arquake/. Retrieved 2026-10-13.
  39. ↑ 39.0 39.1 Wagner, Daniel; Schmalstieg, Dieter (2003). "First Steps Towards Handheld Augmented Reality". Proceedings of the Seventh IEEE International Symposium on Wearable Computers. doi:10.1109/ISWC.2003.1241402. https://www.researchgate.net/publication/4041797_First_steps_towards_handheld_augmented_reality. Retrieved 2026-10-13.
  40. ↑ "The Invisible Train: A Handheld Augmented Reality Game". Interactive Media Systems, TU Wien project archive. https://www.ims.tuwien.ac.at/projects/invisible-train. Retrieved 2026-10-13.
  41. ↑ Daniel Wagner, Thomas Pintaric and Dieter Schmalstieg (2004-08-08). "The Invisible Train: Handheld Collaborative Augmented Reality". Graz University of Technology publication record. https://tugraz.elsevierpure.com/en/publications/the-invisible-train-handheld-collaborative-augmented-reality. Retrieved 2026-10-13.
  42. ↑ Georg Klein, David Murray (2007-11). "Parallel Tracking and Mapping for Small AR Workspaces". 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2007). IEEE. doi:10.1109/ISMAR.2007.4538852. https://www.robots.ox.ac.uk/~gk/publications/KleinMurray2007ISMAR.pdf. Retrieved 2026-10-17.
  43. ↑ 43.0 43.1 43.2 43.3 Kangsoo Kim, Mark Billinghurst, Gerd Bruder, Henry Been-Lirn Duh, Gregory F. Welch (2018-11). "Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008-2017)". IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 11, pp. 2947-2962. doi:10.1109/TVCG.2018.2868591. https://sreal.ucf.edu/wp-content/uploads/2018/08/Kim2018b.pdf. Retrieved 2026-10-17.
  44. ↑ Terry Myerson (2015-01-21). "The next generation of Windows: Windows 10". Windows Insider Blog. https://blogs.windows.com/windows-insider/2015/01/21/the-next-generation-of-windows-windows-10/. Retrieved 2026-10-13.
  45. ↑ Alex Kipman (2016-02-29). "Announcing Microsoft HoloLens Development Edition open for pre-order, shipping March 30". Windows Blogs. Microsoft. https://blogs.windows.com/devices/2016/02/29/announcing-microsoft-hololens-development-edition-open-for-pre-order-shipping-march-30/. Retrieved 2026-10-17.
  46. ↑ 46.0 46.1 John Hanke and the Niantic team (2016-07-06). "Break out the sneakers and Poké Balls!". Pokémon GO. https://pokemongo.com/news/launch. Retrieved 2026-10-13.
  47. ↑ Dave Burke (2017-08-29). "ARCore: Augmented reality at Android scale". The Keyword, Google. https://blog.google/products/arcore/arcore-augmented-reality-android-scale/. Retrieved 2026-10-17.
  48. ↑ Julia White (2019-02-24). "Microsoft at MWC Barcelona: Introducing Microsoft HoloLens 2". Official Microsoft Blog. Microsoft. https://blogs.microsoft.com/blog/2019/02/24/microsoft-at-mwc-barcelona-introducing-microsoft-hololens-2/. Retrieved 2026-10-17.
  49. ↑ 49.0 49.1 "Khronos Releases OpenXR 1.0 Specification Establishing a Foundation for the AR and VR Ecosystem". The Khronos Group. 2019-07-29. https://www.khronos.org/news/press/khronos-releases-openxr-1.0-specification-establishing-a-foundation-for-the-ar-and-vr-ecosystem. Retrieved 2026-10-17.
  50. ↑ 50.0 50.1 50.2 "Introducing Orion, Our First True Augmented Reality Glasses". Meta Newsroom. 2024-09-25. https://about.fb.com/news/2024/09/introducing-orion-our-first-true-augmented-reality-glasses/. Retrieved 2026-10-17.
  51. ↑ 51.0 51.1 51.2 "Meta Ray-Ban Display: AI Glasses With an EMG Wristband". Meta Newsroom. 2025-09-17. https://about.fb.com/news/2025/09/meta-ray-ban-display-ai-glasses-emg-wristband/. Retrieved 2026-10-13.
  52. ↑ "Platforms State of the Union". Apple Developer, WWDC25. 2025. https://developer.apple.com/videos/play/wwdc2025/102/. Retrieved 2026-10-13.
  53. ↑ 53.0 53.1 53.2 "Enhance app experiences with perception using ARCore for Jetpack XR". Android Developers. https://developer.android.com/develop/xr/jetpack-xr-sdk/arcore. Retrieved 2026-10-13.
  54. ↑ 54.0 54.1 "WebXR Device API". World Wide Web Consortium. https://www.w3.org/TR/webxr/. Retrieved 2026-10-13.
  55. ↑ "WebXR Augmented Reality Module - Level 1". World Wide Web Consortium, Candidate Recommendation Draft. 2025-04-25. https://www.w3.org/TR/webxr-ar-module-1/. Retrieved 2026-10-17.
  56. ↑ "Khronos Releases OpenXR 1.1 to Further Streamline Cross-Platform XR Development". The Khronos Group. 2024-04-15. https://www.khronos.org/news/press/khronos-releases-openxr-1.1-to-further-streamline-cross-platform-xr-development. Retrieved 2026-10-17.
  57. ↑ 57.0 57.1 "Features and functionality removed in Windows client". Microsoft Learn. https://learn.microsoft.com/en-us/windows/whats-new/removed-features. Retrieved 2026-10-13.
  58. ↑ 58.0 58.1 58.2 "About HoloLens 2". Microsoft Learn. 2023-03-13. https://learn.microsoft.com/en-us/hololens/hololens2-hardware. Retrieved 2026-10-13.
  59. ↑ 59.0 59.1 59.2 "Ray-Ban Stories: How We Built Our First Smart Glasses". Meta Newsroom. 2023-02-16. https://about.fb.com/news/2023/02/ray-ban-stories-how-we-built-our-first-smart-glasses/. Retrieved 2026-10-13.
  60. ↑ "Understand the types of Android XR devices". Android Developers. 2026-05-19. https://developer.android.com/develop/xr/devices. Retrieved 2026-10-13.
  61. ↑ 61.0 61.1 "XREAL One Pro". XREAL. https://www.xreal.com/us/one-pro. Retrieved 2026-10-13.
  62. ↑ 62.0 62.1 "XREAL One Pro". XREAL Japan store. https://jp.shop.xreal.com/en/products/xreal-one-pro. Retrieved 2026-10-13.
  63. ↑ 63.0 63.1 "Introducing Oakley Meta Glasses, a New Category of Performance AI Glasses". Meta Newsroom. 2025-06-20. https://about.fb.com/news/2025/06/introducing-oakley-meta-glasses-a-new-category-of-performance-ai-glasses/. Retrieved 2026-10-13.
  64. ↑ 64.0 64.1 "Army's Integrated Visual Augmentation System (IVAS)/Soldier Borne Mission Command (SBMC)". Congressional Research Service, In Focus IF13022 (EveryCRSReport copy). 2026-07-20. https://www.everycrsreport.com/files/2026-07-20_IF13022_06226ecfaadb2bc19b6b6fd748e6d6d76f511305.html. Retrieved 2026-10-17.
  65. ↑ 65.0 65.1 "Augmented Reality and Virtual Reality in Medical Devices". U.S. Food and Drug Administration. 2026-09-04. https://www.fda.gov/medical-devices/digital-health-center-excellence/augmented-reality-and-virtual-reality-medical-devices. Retrieved 2026-10-17.
  66. ↑ 66.0 66.1 "Catching Pokémon in AR mode". Pokémon GO Help Center. https://niantic.helpshift.com/hc/en/6-pokemon-go/faq/28-catching-pokemon-in-ar-mode/. Retrieved 2026-10-13.
  67. ↑ 67.0 67.1 "AR Quick Look gallery". Apple Developer. https://developer.apple.com/quick-look-gallery/. Retrieved 2026-10-13.
  68. ↑ 68.0 68.1 "Measure dimensions with iPhone". Apple Support. https://support.apple.com/en-euro/guide/iphone/iphd8ac2cfea/ios. Retrieved 2026-10-13.
  69. ↑ "IKEA Home sparks ideas and inspiration through artificial intelligence". IKEA Global. 2019-09-24. https://www.ikea.com/global/en/newsroom/innovation/ikea-sparks-home-furnishing-ideas-and-inspiration-through-artificial-intelligence-190924/. Retrieved 2026-10-13.
  70. ↑ Maximilian Speicher, Brian D. Hall, Michael Nebeling (2019-05). "What is Mixed Reality?". Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19), pp. 1-15. ACM. doi:10.1145/3290605.3300767. https://doi.org/10.1145/3290605.3300767. Retrieved 2026-10-17.
  71. ↑ Feng Zhou, Henry Been-Lirn Duh, Mark Billinghurst (2008-09). "Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR". 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR 2008), pp. 193-202. IEEE. doi:10.1109/ISMAR.2008.4637362. https://doi.org/10.1109/ISMAR.2008.4637362. Retrieved 2026-10-17.
Retrieved from "https://vrarwiki.com/index.php?title=Augmented_reality&oldid=52482"