Reality-virtuality continuum
The reality-virtuality continuum is a conceptual framework that places displays and environments on a spectrum running from a completely real environment at one end to a completely virtual environment at the other, with the blended region between the two extremes described as mixed reality. It was introduced by Paul Milgram and Fumio Kishino in 1994 and is one of the most frequently cited frameworks used to position virtual reality (VR), augmented reality (AR), and related technologies relative to one another.[1][2]
In the original formulation, the continuum spans from a real environment, defined as one "consisting solely of real objects," to a virtual environment, defined as one "consisting solely of virtual objects."[1] Everything in between, where real and virtual objects are presented together within a single display, is mixed reality. The same 1994 work introduced the term mixed reality and named the two intermediate regions augmented reality (closer to the real end) and augmented virtuality (closer to the virtual end).[1][2]
Origin
The framework comes from two related publications by Paul Milgram and his collaborators in 1994. The first is the journal paper "A Taxonomy of Mixed Reality Visual Displays," an invited paper in the IEICE Transactions on Information and Systems special issue on Networked Reality.[1] The second is a conference paper, "Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum," by Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino, presented in the SPIE volume Telemanipulator and Telepresence Technologies.[3]
At the time, Milgram was with the Department of Industrial Engineering at the University of Toronto, and Kishino was with the ATR Communication Systems Research Laboratories in Kyoto, Japan.[1] Milgram had been an invited researcher at ATR in Kyoto during 1993 and 1994, the period in which the work was produced.[4] The stated objective was to bring order to what the authors described as inexact terminology and unclear conceptual boundaries among researchers working with hybrid real-and-virtual environments.[1]
The 1994 papers treated mixed reality as a subset of VR-related technologies rather than something separate from it. Milgram and Kishino wrote that they focused on "a particular subclass of VR related technologies that involve the merging of real and virtual worlds, which we refer to generically as Mixed Reality (MR)."[1]
The continuum
Milgram and Kishino represented the continuum as a horizontal line. The left end is the real environment and the right end is the virtual environment; the region in between is mixed reality, with augmented reality nearer the real end and augmented virtuality nearer the virtual end.[1] The authors described the most direct way to view a mixed reality environment as "one in which real world and virtual world objects are presented together within a single display, that is, anywhere between the extrema of the virtuality continuum."[1]
The paper defined the two named intermediate classes by which side of the world predominates. Augmented reality was given as an operational definition referring to "any case in which an otherwise real environment is 'augmented' by means of virtual (computer graphic) objects."[1] Augmented virtuality was proposed for the converse case, in which a primarily computer-generated world has real (for example video) content added to it.[1] The authors noted that as technology advanced it might become harder to tell whether a given experience was predominantly real or predominantly virtual, which could weaken the case for the separate AR and AV labels but would not affect the more general mixed reality term covering the middle of the continuum.[1]
The terms "reality-virtuality continuum" and "virtuality continuum" are used interchangeably for the same framework; the 1994 journal figure labels it the "virtuality continuum," while the companion SPIE paper title uses "reality-virtuality continuum."[1][3]
Classes of mixed reality displays
Before proposing a taxonomy, the journal paper identified six classes of hybrid display environment that fit the mixed reality definition.[1]
| Class | Description |
|---|---|
| 1 | Monitor-based (non-immersive) "window-on-the-world" video displays with computer-generated images overlaid electronically or digitally. |
| 2 | Video displays as in Class 1, but presented through immersive head-mounted displays (HMDs) rather than desktop monitors. |
| 3 | See-through HMDs on which computer graphics are optically superimposed, using half-silvered mirrors, onto directly viewed real-world scenes. |
| 4 | As in Class 3, but using video rather than optical viewing of the outside world, producing a video see-through system that maps orthoscopically onto the immediate surroundings. |
| 5 | Completely graphic display environments to which video "reality" is added. |
| 6 | Completely graphic, partially immersive environments (for example large-screen displays) in which real physical objects in the user's surroundings interact with the computer-generated scene. |
Milgram and Kishino argued that these six classes appear distinct at first glance but become hard to separate once concepts such as real versus virtual, direct versus indirect viewing, egocentric versus exocentric viewpoint, and orthoscopic scaling are considered, which is why a classification framework was needed rather than a simple list.[1]
The three dimensions
Rather than relying on the discrete display classes, the paper proposed an approximately three-dimensional taxonomy for positioning mixed reality displays.[1][3] The three dimensions are summarized below using the authors' own framing questions.
| Dimension | Guiding question |
|---|---|
| Extent of World Knowledge (EWK) | How much is known about the world being displayed, from completely unmodelled to completely modelled? |
| Reproduction Fidelity (RF) | How realistically can the world be displayed? |
| Extent of Presence Metaphor (EPM) | What is the extent of the illusion that the observer is present within that world? |
These dimensions let displays that fall in the same place on the simple left-to-right continuum still be distinguished by how much the system models the world, how convincing the imagery is, and how immersed the viewer feels.[1]
Scope and limitations
The original framework was explicitly restricted to visual displays. Milgram and Kishino wrote that they focused "exclusively on mixed reality visual displays," while noting that many of the same concepts also apply to other modalities such as auditory and haptic displays.[1] Later commentary has emphasized this limitation, pointing out that the continuum as first proposed did not account for sound, touch, smell, or taste.[2]
In a 2021 review, "Revisiting Milgram and Kishino's Reality-Virtuality Continuum," Richard Skarbez, Missie Smith, and Mary C. Whitton noted that the original paper had been cited thousands of times and remained one of the most-used frameworks in the field, but proposed revisions to it.[2] Their argument was that the continuum is better understood as discontinuous rather than a smooth line, that mixed reality should be taken to encompass conventional virtual reality, and that a revised taxonomy should include factors based on user experience rather than only on the capabilities of the hardware.[2]
Relationship to mixed reality and XR
The continuum is the source of the term mixed reality and is commonly used to define it: mixed reality covers any environment in which real and virtual objects are combined within a single display, occupying the span of the continuum between, but not including, the two endpoints.[1][5] Reference material on the framework places augmented reality near the real end, augmented virtuality in the center-right region, virtual reality at the virtual end, and uses the Microsoft HoloLens as an example of a mixed reality device.[5]
The framework is also used to explain the umbrella term extended reality (XR), which is described as covering AR, augmented virtuality, mixed reality, and VR, that is, any technology situated at a point along the continuum that blends physical and digital content.[5] Because the 1994 framework predates these later marketing and product categories, the way specific commercial systems are slotted onto it varies between sources, but the underlying spectrum from real to virtual remains the common reference point.[2][5]
References
- ↑ 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.10 1.11 1.12 1.13 1.14 1.15 1.16 1.17 1.18
- Kishino, Fumio(December 1994). "A Taxonomy of Mixed Reality Visual Displays".{Template:Journal. E77-D(12)
- 1321-1329. https://www.alice.id.tue.nl/references/milgram-kishino-1994.pdf. Retrieved 2026-06-15.
- ↑ 2.0 2.1 2.2 2.3 2.4 2.5
- Smith, Missie(2021). "Revisiting Milgram and Kishino's Reality-Virtuality Continuum".{Template:Journal. 2. doi
- 10.3389/frvir.2021.647997. https://www.frontiersin.org/journals/virtual-reality/articles/10.3389/frvir.2021.647997/full. Retrieved 2026-06-15.
- ↑ 3.0 3.1 3.2 Milgram, Paul; Takemura, Haruo; Utsumi, Akira; Kishino, Fumio (1994). "Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum". 2351. Telemanipulator and Telepresence Technologies. SPIE. pp. 282-292. Template:Hide in printTemplate:Only in print. https://www.spiedigitallibrary.org/conference-proceedings-of-spie/2351/1/Augmented-reality--a-class-of-displays-on-the-reality/10.1117/12.197321.short.
- ↑ "Paul Milgram". https://www.mie.utoronto.ca/faculty_staff/milgram/.
- ↑ 5.0 5.1 5.2 5.3 "What Is the Virtuality Continuum?". https://ixdf.org/literature/topics/virtuality-continuum.