Extended reality

The reality-virtuality continuum, which spans the range from a fully real environment to a fully virtual one with augmented reality and augmented virtuality in between.

Extended reality (XR) is an umbrella term that covers the spectrum of technologies which merge or replace a user's perception of the physical world with computer-generated content. It groups together Virtual Reality (VR), Augmented reality (AR) and Mixed reality (MR) under a single label, along with any related immersive or spatial-computing approaches that combine real and virtual environments.

The "X" in XR is commonly read as a variable or placeholder. It can stand in for the "V" of virtual, the "A" of augmented or the "M" of mixed, which lets the term refer collectively to whichever immersive technology is being discussed, including future ones that do not yet exist. Because of this flexibility, XR is used both in research and in marketing as a convenient way to describe immersive technology in general rather than committing to a single category.

Definition and scope

Extended reality is not a single device or technique. It is a collective descriptor for technologies that alter a person's experience of reality by adding digital elements to the physical world, by replacing the physical world with a virtual one, or by blending the two. The three established domains gathered under the term are:

• Virtual Reality (VR): the user is fully immersed in a computer-generated environment, typically through a head-mounted display that blocks out the physical surroundings and replaces them with a synthetic scene.
• Augmented reality (AR): computer-generated content is overlaid on a view of the real world, so that digital information appears alongside the user's actual surroundings rather than replacing them.
• Mixed reality (MR): real and virtual content coexist and can interact, combining elements of both AR and VR so that virtual objects can be anchored to, and respond to, the physical environment.

Because these categories sit on a continuum rather than in fully separate boxes, the boundaries between them are not always sharp. Many modern devices are capable of more than one mode: a VR Headset with passthrough cameras, for example, can present a fully virtual scene (VR) or composite virtual objects onto a live camera feed of the room (a form of MR), which is one reason the single umbrella term XR has become useful.

The reality-virtuality continuum

The conceptual foundation most often used to organise extended reality predates the term itself. In 1994, Paul Milgram and Fumio Kishino published "A Taxonomy of Mixed Reality Visual Displays", which simultaneously introduced the notion of the reality-virtuality (RV) continuum and the term "mixed reality".

The continuum is anchored at two endpoints. One end is a purely real environment consisting solely of real objects, and the other is a purely virtual environment consisting solely of virtual objects. Everything between those poles is described as mixed reality. Within that middle region, augmented reality refers to cases where the display of an otherwise real environment is augmented with virtual content, while augmented virtuality refers to cases where a largely virtual environment includes some awareness or inclusion of real-world objects.

Position on the continuum	Description
Real environment	The world as perceived directly, consisting solely of real objects.
Augmented reality (AR)	A predominantly real view augmented with virtual content.
Augmented virtuality (AV)	A predominantly virtual environment that includes some real-world elements.
Virtual environment	A fully computer-generated world consisting solely of virtual objects.

Milgram and Kishino's original paper was explicitly concerned with visual displays and display hardware, and it identified several distinct classes of mixed-reality display systems. Since its publication the RV continuum has been cited thousands of times and has become one of the standard frameworks for situating VR, AR and MR relative to one another. Extended reality, as the term is used today, is often understood as the whole of this spectrum, from the completely real to the completely virtual.

Relationship and boundaries between VR, AR and MR

The three component technologies differ mainly in how much of the user's perceived environment is real versus synthetic, and in how the real and virtual layers interact:

• In VR, the physical surroundings are entirely substituted by a virtual scene, giving a strong sense of presence inside a digital world.
• In AR, the physical surroundings remain dominant and digital content is layered on top, usually without the virtual elements being aware of or reacting to the real geometry.
• In MR, virtual and real content are registered to the same space so that they can occlude, anchor to, and interact with one another, drawing on capabilities from both AR and VR.

In practice the line between AR and MR is the most contested, and different vendors and writers draw it in different places. The umbrella term XR sidesteps the dispute by referring to the whole family at once, which is part of why it has been adopted across the industry.

Hardware and use cases

Extended reality experiences are delivered through a range of devices. Fully immersive VR typically uses an opaque head-mounted display, while AR and MR are delivered through see-through smart glasses and headsets or through video passthrough on otherwise opaque headsets. These devices are commonly paired with supporting hardware such as motion-tracked controllers, hand tracking, eye tracking and six-degrees-of-freedom tracking that places virtual content relative to the user and the surrounding space.

Reported applications of XR span training and simulation, design and engineering visualisation, healthcare, education, remote collaboration, retail and product visualisation, and entertainment and gaming. The shared characteristic is that a digital layer is combined with, or substituted for, the user's perception of the physical environment to support tasks that are difficult, expensive or impossible to carry out with conventional 2D displays.

Industry usage of the "XR" label

The XR label is widely used by hardware and platform companies as a single banner for their immersive technology efforts. Qualcomm, for example, markets its mobile XR silicon under the Snapdragon XR brand and describes its Snapdragon XR2 family as platforms for extended reality that target both AR and VR (and, in later generations, mixed reality) experiences. Here the "X" again works as a cross-cutting variable, signalling that one platform is intended to serve the whole range of immersive form factors rather than a single one.

The term is also used in product names, job titles, research centres and standards work. Its convenience as a catch-all is real, but the same breadth means it is sometimes used loosely in marketing, where "XR" can be applied to almost any product that touches AR or VR.

Standards

The most prominent cross-platform standard for extended reality is OpenXR, a royalty-free open standard developed by the Khronos Group. OpenXR provides a common set of application programming interfaces (APIs) so that XR applications can run across a wide range of AR and VR hardware without being rewritten for each vendor's proprietary software development kit, which reduces fragmentation in the ecosystem.

Khronos announced the OpenXR initiative in early 2017 and released a provisional 0.90 specification in March 2019. The OpenXR 1.0 specification was released on July 29, 2019, and a 1.1 revision that folded widely used extensions into the core specification followed on April 15, 2024. The standard is supported across much of the industry, including runtimes from Microsoft, Meta, Valve (SteamVR), HTC, Magic Leap and others, and Khronos itself describes the AR and VR platforms it targets as "collectively known as XR".

See also

• Virtual Reality
• Augmented reality
• Mixed reality
• OpenXR
• Head-mounted display
• Qualcomm Snapdragon XR2