Augmented Reality

Augmented reality (AR) is a dynamic computer generated experience that enables a user to interact with and view spatially registered overlays of virtual content onto parts of the real world. It enhances perception, allowing environments to be enriched in new ways. A basic characteristic of AR is that it merges the real and the virtual worlds. The technology aims to enhance our perception of reality through the incorporation of computer generated data and simulations into our senses, creating a reality-based interface ^[1].

The goal of AR devices is to supplement the real world with virtual objects by overlaying digital imageries and information on top of physical objects and enabling the users of the devices to seamlessly interact with the digital content. Through the use of computer vision and object recognition, digital information about the real world around us can not only be viewed but also manipulated in real-time ^[1].

In general, the technology combines real and virtual objects, aligns real and virtual objects with each other, and runs interactively in real-time. Furthermore, it is not restricted to a specific type of display technology, like an HMD, and can potentially be applied to other senses beside sight ^[1].

In the mixed reality spectrum, AR is closer to a real environment. Therefore, unlike Virtual Reality, Augmented Reality does not replace the real world with a virtual one. AR simply enhances and modifies the real world ^[1].

In 2007, MIT recognized AR as one of ten emerging technologies, reporting that this type of human-computer interaction is on the verge of major adoption ^[1].

Technologies

Optical see-through head-mounted displays

Augmented Reality devices are transparent glasses-like wearables called optical head-mounted displays (OHMDs). These devices have displays with small projectors that create digital information and rendered images on top of objects in the physical world. These devices have built-in cameras that uses computer vision and object recognition to identify objects and decipher the physical environment around the devices. Information and data about the surroundings can be streamed into the display in real time. Users can interact and manipulate the information through various input methods such as voice commands, hand and body gestures, touchpads and more.

Augmented Reality history timeline

The historical development of AR technologies intersects with that of virtual reality. During the initial stages of its evolution, the terms augmented reality and virtual reality had not been coined and, consequently, there wasn’t a clear distinction between the two ^[2].

1901- A Concept of AR

Frank L Baum writes a novel in which there is a concept that can be equated to AR: a set of electronic glasses called “character marker” that were used to map data onto people ^[2].

1957 - The Sensorama

The cinematographer Morton Heilig invented the Sensorama. This machine delivered visuals, sound, vibration, and smell to the viewer. It was not controlled by a computer but, nevertheless, it was an attempt at adding additional data to an experience. The machine was patented in 1961, and it looked like an arcade machine ^[2] ^[3].

1968 - The Sword of Damocles HMD

Ivan Sutherland and Bob Sproull created a head-mounted display system at Harvard University and the University of Utah. The device presented simple wireframe graphics, used see-through optics, and was held to the ceiling by a mechanical arm which tracked the head movements of the user. This iteration of the technology would prove to be impractical for mass use. Sutherland also postulated the concept of the “Ultimate Display” in 1965 and would have a great impact in the VR and AR fields of study ^[2] ^[4] ^[5] ^[6] ^[1].

1975 - Videoplace

The videoplace was developed by the American computer artist Myron Krueger. It was an interface that allowed users to manipulate and interact with virtual objects in real-time. It combined projectors, video-cameras and special purpose hardware, as well as onscreen silhouettes of the users ^[2] ^[3] ^[4].

1980 - Wearable computing

The computational photography researcher Steve Mann creates the first example of wearable computing ^[2].

1990 - Augmented Reality

Professor Thomas P. Caudell, a researcher at Boeing, coined the term augmented reality. The term was in reference to a HMD that guided workers through assembling electrical wires in aircrafts ^[2] ^[3] ^[1].

1992 - Virtual Fixtures

Virtual Fixtures is developed at USAF Armstrong’s Research Lab by Louis Rosenberg. According to some sources, it can be considered the first properly functioning AR system. It was a system that overlaid sensory information on a workspace to improve human productivity ^[2].

1993 - KARMA

Feiner and colleagues introduced KARMA - Knowledge-based augmented reality for maintenance assistance. KARMA was capable of inferring instructions sequences for repair and maintenance procedures ^[4]. During the same year, Fitzmaurice created the first handheld spatially aware display called Chameleon - a precursor to handheld AR. It consisted of a tethered handheld LCD screen that showed the video output of an SGI graphics workstation and was spatially tracked using a magnetic tracking device. The system was capable of providing information to the user such as providing information about a location on a wall-mounted map ^[4].

1994 - Medical AR

At the University of North Carolina, State and colleagues presented a medical AR application. It was capable of allowing a physician to observe a fetus directly within a pregnant woman (Figure 1) ^[4].

1995 - NaviCam

Rekimoto and Nagao developed a true handheld AR display, although it was still tethered to a workstation. The NaviCam had a forward-facing camera, and from its video feed it could detect color-coded markers, displaying information on a video see-through view ^[4].

1996 - Studierstube

The first collaborative AR system is developed by Schmalstieg and colleagues. The Studierstube allowed for multiple users to experience virtual objects in the same shared space through the use of HMDs. Each user from their individual viewpoint could see an image in correct perspective ^[4].

1997 - The Touring Machine

Feiner and colleagues create the first outdoor AR system, at Columbia University. The Touring Machine (Figure 3) had a see-through HMD, GPS, and orientation tracking. The system needed a backpack with a computer to deliver mobile 3D graphics, various sensors, and an earlier version of a tablet computer for input ^[4].

2000 - ARQuake

The AR version of the Quake game is developed by Bruce Thomas, at the University of South Australia (Figure 4). It was an outdoor mobile version of the game developed by Id Software ^[3].

2003 - First autonomous handheld AR system

Wagner and Schmalstieg presented a precursor to the current smartphones - a handheld AR system that ran autonomously on a “personal digital assistant.” In 2004, a multiplayer handheld AR game called Invisible Train (Figure 5) was shown at the SIGGRAPH Emerging Technologies show floor ^[4].

2008 - A commercial AR application

The first commercial AR application is developed by German agencies in Munich for advertising. It consisted of a printed magazine ad of a model BMW mini. When held in front of a computer’s camera, a user could manipulate the virtual car on the screen and move it around to view different angles ^[5] ^[7]. During the same year, the Wikitude AR Travel Guide was released for the G1 Android phone ^[3].

2009 - ARToolkit

A design tool, ARToolkit, is made available in Adobe Flash ^[2].

2013 - Google Glass

The open beta of Google Glass is announced ^[2].

2015 - HoloLens

Microsoft announced AR support for the company’s AR headset HoloLens ^[2].

2016 - Pokémon Go

Pokémon Go is released and becomes a major success. It is considered an achievement for the AR industry. The game hit its peak in August 2016 with almost 46 million users. While the game has failed to maintain high levels of engagement, it showed the potential of AR ^[5] ^[7].