Toggle search
Toggle menu
5.5K
6.1K
232
53.7K
VR & AR Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Spatial computing

vrarwiki.com · VR & AR Wiki
More actions
See also: Terms and Technical Terms

Spatial computing is a paradigm where human-computer interaction (HCI) moves beyond traditional desktop or mobile screens, allowing digital information and processes to be perceived and manipulated as if they exist within three-dimensional physical space. The definition most often cited comes from Simon Greenwold's 2003 master's thesis at the Massachusetts Institute of Technology, which described spatial computing as "human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces".[1][2] It involves machines understanding and interacting with the geometry and semantics of the surrounding environment, enabling users to interact with digital content using natural modalities like gestures, gaze, and voice, often overlaying this content onto their view of the real world.[3][4] In practice, the term covers technologies like virtual reality (VR), augmented reality (AR), and mixed reality (MR) that anchor digital content to real-world locations and objects, so that virtual elements can be perceived as part of the surrounding environment, collectively sometimes referred to as Extended Reality (XR).[5][4]

The phrase has more than one lineage. Geographic information science uses it for computing with locations and geospatial data, while HCI and XR research use it for systems that place digital content in a user's physical surroundings; a 2025 review by Wang, Luo, Cho and Yu describes the field as conceptually fragmented across these disciplines.[6] The term reached a general audience in June 2023, when Apple introduced Apple Vision Pro as "Apple's first spatial computer" and visionOS as a "spatial operating system".[3] Apple's developer guidelines tell app makers to describe a Vision Pro app as a spatial computing app rather than as AR, VR, XR or MR.[7]

Reviewed 17 October 2026. All 77 cited sources re-opened and checked (Greenwold thesis, DOI papers via Crossref, Apple, Google, Samsung, Meta, Microsoft and Magic Leap primary pages, press reports) for quotations, dates, prices and device status as of 17 October 2026. About review dates.

Definitions

Greenwold's thesis gave the definition quoted above and added in its abstract that spatial computing "is an essential component for making our machines fuller partners in our work and play".[1] In the body of the thesis he wrote that the real objects and spaces should ideally have prior significance to the user, and he separated spatial computing from 3D modeling and digital design because it requires the forms and spaces it handles to already exist in the world; a screen showing a virtual space was not enough unless that space was meaningfully related to an actual place.[1] His definition did not require a 3D display. He gave an auto-flushing toilet that senses a user moving away as a trivial example that still qualifies, an example the Associated Press repeated in 2024.[1][8]

In a 2016 Communications of the ACM review centred on location-based computing, Shashi Shekhar, Steven Feiner and Walid Aref wrote that "spatial computing encompasses the ideas, solutions, tools, technologies, and systems that transform our lives by creating a new understanding of locations". Their examples included GPS navigation, virtual globes such as Google Earth, and site selection, asset tracking and logistics in large organizations.[9] Wang and colleagues trace two threads from these roots: a geospatial thread that grew out of geographic information systems, and a mixed reality thread that runs from Paul Milgram's reality-virtuality continuum to Greenwold. They identify two schools of thought about what "spatial" means: space as context that guides interaction with the physical world, and space as a mixed physical and digital environment for embodied interaction.[6]

Industry definitions are broader. NVIDIA describes spatial computing as merging "digital data with the physical world in real time" through AR and VR headsets or smart glasses,[4] and TechTarget describes it as the processes and tools used to capture, process and interact with three-dimensional data.[5] Writing in Harvard Business Review in 2023, Cathy Hackl called it "an evolving form of computing that blends our physical world and virtual experiences".[10]

History

The concept of blending computation with physical space has roots in multiple fields. Wang and colleagues date the technological foundations of the geospatial thread to the first geographic information systems of the 1960s, and note that GIS tools and in-car GPS reached wider use from the mid-1980s.[6] Influential precursors on the interaction side include Ivan Sutherland's Sketchpad, the subject of his 1963 MIT doctoral thesis, and his 1968 head-mounted three-dimensional display, which presented the user with a perspective image that changed as the head moved.[11][12] Mark Weiser's vision of ubiquitous computing at Xerox PARC, published for a general audience in Scientific American in 1991, imagined computers woven into the fabric of everyday life; Greenwold's thesis dates Weiser's first statement of the idea to 1988.[13][1]

In the early 1990s, researchers at the University of Washington's Human Interface Technology Lab (HIT Lab), founded by VR pioneer Thomas A. Furness III, explored advanced 3D interfaces.[14] Bob Jacobson, who joined the lab in 1989 and served as its associate director, also led the Seattle company Worldesign Inc.; the AWE XR Hall of Fame credits him with helping to introduce the term "spatial computing" there in the 1990s.[15] In a 1995 newsgroup post, Jacobson wrote that Worldesign had built a projection chamber it called the Virtual Environment Theater and that its first virtual world was a tour of the Giza Plateau based on old archaeological maps.[16] In 1997 World Scientific published an edited volume, Spatial Computing: Issues in Vision, Multimedia and Visualization Technologies, edited by T. Caelli, P. Lam and H. Bunke.[17][18]

Greenwold wrote his thesis at the MIT Media Lab, in John Maeda's Aesthetics + Computation Group, and later writers credit it as the source of the term's modern definition.[1][10][5] Greenwold defined it as "human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces," emphasizing machines becoming "fuller partners in our work and play."[1] The thesis placed the work within existing research on HCI, ubiquitous computing, invisible computing and tangible interfaces.[1] Research on context-aware computing, systems that examine and react to a person's changing context, had been under way since a 1994 paper by Bill Schilit, Norman Adams and Roy Want.[19] Wang and colleagues state that Greenwold "integrated MR with the concept of spatial computing".[6]

Commercial developments accelerated in the 2010s:

  • Microsoft's Kinect launched in North America on 4 November 2010 and let Xbox 360 players control games and entertainment with gestures and voice commands, with no controller; Microsoft reported 1 million units sold in its first 10 days.[20]
  • Google's Project Tango (2014) brought AR to phones with a depth-sensing infrared camera and dedicated motion sensors. The extra hardware made it expensive for manufacturers, and Google ended Tango support on 1 March 2018 in favor of ARCore.[21]
  • Microsoft HoloLens (unveiled 2015, Development Edition shipped 30 March 2016 at US$3,000 in the US and Canada) was a self-contained "mixed reality" headset performing real-time spatial mapping and anchoring holograms to the environment. Microsoft called it "the first and only fully untethered holographic computer".[22][23]
  • Magic Leap, which became a corporation in May 2011,[24] used the term "spatial computing" in marketing its Magic Leap One Creator Edition, which went on sale on 8 August 2018 at US$2,295; SiliconANGLE reported that the company presented its new Lumin OS as providing a spatial computing platform.[25]
  • Apple's ARKit, announced in June 2017,[26] and Google's ARCore, previewed in August 2017,[27] brought basic spatial computing to ordinary smartphones using their built-in cameras and motion sensors, with no extra hardware; Google listed motion tracking and horizontal surface detection among ARCore's core capabilities.[26][27] Pokémon GO, released in summer 2016, added an ARKit-based AR+ mode in December 2017 that fixed Pokémon to a location in the real world.[28]
  • Room-scale VR systems (HTC Vive, Oculus Rift) tracked the user's movement through a room, and later standalone headsets (Oculus Quest, released 21 May 2019) incorporated inside-out spatial tracking for environmental awareness; the Quest's Insight system used cameras on the headset itself to track the room.[29]

The early 2020s saw further mainstreaming. Facebook rebranded to Meta in October 2021 and said its focus would be "to bring the metaverse to life".[30] Apple unveiled the Apple Vision Pro on 5 June 2023, explicitly branding it as a "spatial computer."[3] Apple CEO Tim Cook said that "Today marks the beginning of a new era for computing," and that "Just as the Mac introduced us to personal computing, and iPhone introduced us to mobile computing, Apple Vision Pro introduces us to spatial computing."[3] The headset went on sale in the United States on 2 February 2024 from US$3,499, with Apple's announcement headed "The era of spatial computing is here".[31] On launch day Cook told employees in a memo that Vision Pro began "an entirely new era of spatial computing", comparing the moment with the Mac and iPhone launches.[32] The Associated Press described spatial computing that day as "an esoteric mode of technology that Apple executives and their marketing gurus are trying to thrust into the mainstream", while noting that the concept "has been around for at least 20 years".[8]

Competing platforms followed. Google announced Android XR for headsets and glasses on 12 December 2024,[33] and Samsung released the first Android XR headset, the Samsung Galaxy XR, in the United States on 21 October 2025 for US$1,799.99.[34] Apple updated Vision Pro with the M5 chip on 15 October 2025, saying the new model set "a new standard for what's possible in spatial computing".[35]

Core concepts

Spatial computing typically combines several components:

  • Machine perception of space: Devices must understand the physical environment in 3D. This involves technologies like Simultaneous Localization and Mapping (SLAM), which tracks the device's position and orientation while building a map of the space.[36] Depth sensors (like LiDAR or Time-of-Flight cameras) and RGB cameras capture geometric and visual information. Computer vision algorithms, often powered by artificial intelligence (AI), interpret this data to recognize surfaces, objects (for example walls, tables, chairs), people, and potentially understand scene semantics.[4][5]
  • Persistence and context: Digital objects or information placed within the spatial environment can maintain their position and state relative to the physical world, even when the user looks away or leaves and returns. Microsoft's mixed reality documentation describes a spatial anchor as "an important point in the world that the system tracks over time", and notes that anchors can be saved and loaded so that an application finds the same real-world location across sessions.[37] Depth information lets virtual objects appear behind real ones (occlusion); Google's ARCore Depth API, for example, creates depth images that apps use for occlusion and for interactions between virtual objects and real surfaces.[38]
  • Natural user interaction: Input moves beyond the keyboard and mouse. Common interaction methods include Hand tracking (recognizing hand shapes and gestures), Eye tracking (using gaze as a pointer or input trigger), Voice commands, and sometimes specialized controllers. Apple describes Vision Pro's interface as a fully three-dimensional user interface controlled by the user's eyes, hands and voice,[3] and NVIDIA describes hand tracking as letting users interact with virtual objects in a natural way.[4]
  • Blending digital and physical realities: Spatial computing often manifests as AR or MR, where digital information is integrated with the user's view of the real world through Optical see-through displays (like HoloLens, Magic Leap) or Video pass-through displays (like Meta Quest 3, Apple Vision Pro). Ronald Azuma's 1997 survey of augmented reality set out the tradeoffs between these optical and video blending approaches.[39] It also applies to fully immersive VR experiences that create complex, interactive 3D environments where the user's physical movements are tracked and reflected. Spatial audio adds 3D sound cues anchored to locations in the environment; Apple says Vision Pro's Spatial Audio system makes sounds seem to come from the environment around the user and that its audio pods adapt sound to the environment.[3]

Enabling technologies and core components

Spatial computing systems rely on a combination of hardware and software:

Hardware

  • Sensors and cameras: Inertial Measurement Units (IMUs) for orientation, RGB cameras for visual data, depth sensors (LiDAR, Structured Light, Time-of-Flight) for 3D geometry capture, and Microphones for voice input.[5] Eye tracking cameras inside headsets monitor gaze. Apple's R1 chip in Vision Pro processes input from 12 cameras, five sensors and six microphones,[3] and Samsung lists two pass-through cameras, six world-facing tracking cameras, four eye-tracking cameras, five IMUs and one depth sensor on the Galaxy XR.[34]
  • Displays: High-resolution, high-refresh-rate micro-displays (Micro-OLED, MicroLED) render sharp images. Vision Pro uses micro-OLED technology with 23 million pixels across two displays,[3] and the Galaxy XR uses micro-OLED panels of 3,552 x 3,840 pixels with a field of view of 109 degrees horizontal and 100 degrees vertical.[34] Waveguides or other novel optics are used in optical see-through AR glasses; Magic Leap's 2026 business, for example, centers on supplying surface-relief diffractive waveguides to other glasses makers.[40] Wide field-of-view (FOV) lenses are common in VR/MR headsets.
  • Processing units: Energy-efficient Systems-on-Chip (SoCs) with CPUs, GPUs, and often dedicated AI/NPUs or co-processors handle sensor fusion, computer vision tasks, and real-time 3D rendering on-device. Apple says the R1 co-processor in Vision Pro streams new images to the displays within 12 milliseconds,[3] and the Galaxy XR pairs a Qualcomm Snapdragon XR2+ Gen 2 platform with a Hexagon NPU for on-device AI.[34]
  • Input devices: Beyond integrated tracking (hand, eye, voice), some systems use handheld controllers providing buttons, joysticks, and haptic feedback; Samsung sells a Galaxy XR Controller separately for US$249.99, for example.[34]

Software

  • Spatial mapping algorithms: Primarily SLAM and related techniques (for example visual-inertial odometry) to create real-time 3D environmental maps and track device pose.[36] ARCore combines camera feature points with IMU data to determine the phone's position and orientation.[27]
  • Computer vision and ML: Algorithms for object recognition, gesture detection, scene understanding, semantic segmentation, and rendering. NVIDIA describes AI and machine learning as letting spatial computing devices interpret and contextualize sensor data from the real world, and lists Gaussian splats and NeRFs as techniques for rapid 3D scene reconstruction.[4]
  • Rendering engines: Tools like Unity and Unreal Engine provide frameworks for developing 3D environments, handling physics, and supporting AR/VR application development; both engines document XR development for virtual, mixed and augmented reality devices.[41][42]
  • Operating systems and SDKs: Specialized OSs (for example Apple visionOS, Windows Holographic, Android variants such as Android XR) manage spatial tasks. In visionOS an app launches by default in the Shared Space, where several apps run side by side, and can move to a Full Space in which it is the only app running.[43] SDKs (for example ARKit, ARCore, OpenXR, MRTK) provide APIs for developers to build spatial applications; the Khronos Group describes OpenXR as a royalty-free, open standard API for XR applications that run across AR and VR devices.[44]
  • Cloud and Edge computing: Used to offload heavy computation (rendering, AI processing, large-scale mapping), enable collaborative multi-user experiences (for example shared spatial anchors, "AR Cloud" concepts), and stream content. NVIDIA describes hybrid rendering, in which some tasks run in the cloud and others locally, and edge or cloud streaming of complex scenes to headsets.[4][37]
  • Connectivity: High-bandwidth, low-latency wireless links such as Wi-Fi 6E and 5G carry the streamed content when rendering is offloaded from the headset.

Relationship to VR, AR, and MR

Spatial computing underlies advanced forms of VR, AR, and MR (often grouped under the umbrella term XR). While closely related and sometimes used interchangeably in marketing, there are nuances. Paul Milgram and colleagues placed AR on a reality-virtuality continuum running from a fully real environment to a fully virtual one, and called the displays between the two extremes "mixed reality" displays.[45] Unity's documentation uses similar categories for application developers.[41]

  • Virtual Reality (VR): Creates a fully immersive digital environment replacing the user's real-world view. Spatial computing principles apply within this virtual space for tracking user movement (room-scale VR), environmental awareness (for example safety boundaries based on real walls), and interacting with virtual objects using tracked hands or controllers.
  • Augmented Reality (AR): Overlays digital information onto the real world, typically via smartphones, tablets, or simpler smart glasses. Azuma's survey defines AR systems by three characteristics: they combine real and virtual, are interactive in real time, and are registered in 3-D.[39] Mobile AR uses spatial computing for plane detection and tracking.[27]
  • Mixed Reality (MR): A more advanced form of AR where digital objects are integrated more realistically into the physical environment, appearing anchored to and potentially interacting with real surfaces and objects. Users can interact with both physical and virtual elements simultaneously. MR relies on spatial computing for real-time mapping, understanding, occlusion, and interaction. Meta calls the Quest 3 a "mixed reality headset",[46] and Microsoft used the same term for HoloLens.[22]

Wang and colleagues treat the mixed reality thread as one of two strands of spatial computing, alongside the geospatial strand, and propose spatial computing as a broader paradigm that links environment, computation and human experience.[6] Hackl notes that professionals in AR, VR, XR and AI have worked on spatial computing for years.[10]

Relation to other computing paradigms

Spatial computing builds upon and overlaps with several earlier computing paradigms:

  • Ubiquitous computing (pervasive computing): Envisions computers embedded everywhere, becoming invisible parts of daily life. Weiser called virtual reality the notion "most diametrically opposed" to his vision because it "attempts to make a world inside the computer".[13] Greenwold agreed with Weiser's critique of existing interfaces but wrote that, unlike ubiquitous and invisible computing, spatial computing does not advocate invisibility of the machine in every circumstance and does not shun spatial representation by computers.[1]
  • Ambient computing: Emphasizes calm, background operation responsive to user presence, often without traditional screens (for example smart speakers, automated lighting). In 1995 Weiser and John Seely Brown described "calm technology" as technology that engages both the center and the periphery of a person's attention and moves between the two.[47] Spatial computing can be ambient (for example AR glasses providing subtle cues), but often involves explicit visual overlays.
  • Context-aware computing: Systems that adapt based on current context (location, time, user activity). Schilit, Adams and Want described such systems in 1994 as ones that "examine and react to an individual's changing context", and grouped context-aware applications into proximate selection, automatic contextual reconfiguration, contextual information and commands, and context-triggered actions.[19] Spatial computing focuses specifically on real-time spatial context (geometry, pose, environment), which Wang and colleagues call the "contextual understanding of space" school.[6]

In this framing, spatial computing systems are typically context-aware and can be part of ubiquitous or ambient computing scenarios; what distinguishes them is real-time 3D spatial understanding and interaction that places digital content in the user's perceived physical space.[6][1]

Applications and use cases

Spatial computing has applications across numerous sectors:

  • Design and manufacturing: Visualizing 3D CAD models in context, collaborative design reviews in shared virtual spaces, remote expert assistance for repairs, and creating Digital Twins of factories or products. NVIDIA cites automotive design through virtual prototyping and digital twins, and training simulations in manufacturing.[4]
  • Healthcare: Surgical planning using 3D patient models, AR overlays during surgery for navigation, immersive medical training simulations, rehabilitation exercises using AR/VR, and visualizing complex medical data (MRI/CT scans) in 3D. A 2017 review of 102 studies by Vávra and colleagues reported increasing interest among surgeons in using augmented reality.[48] HealthTech Magazine described presurgical anatomy viewing, clinical training simulations and 3D views of anatomical images among the uses of spatial computing headsets in 2024.[49] For stroke rehabilitation, a 2017 Cochrane review found virtual reality no more beneficial than conventional therapy for upper limb function, but possibly beneficial when added to usual care.[50]
  • Education and training: Immersive learning experiences (virtual field trips, science labs), visualizing complex concepts (molecules, historical events) in 3D, and complex task training (aircraft maintenance, emergency response) with AR guidance. A 2014 systematic review by Bacca and colleagues analysed 32 studies of AR in education published between 2003 and 2013.[51]
  • Collaboration and communication: Virtual meetings with spatial presence (avatars in shared spaces), remote collaboration on 3D projects, and shared digital workspaces (for example virtual whiteboards, multiple virtual monitors). Mark Billinghurst and Hirokazu Kato wrote in 2002 that collaborative AR interfaces let users see each other along with virtual objects, allowing communication more like face-to-face than screen-based collaboration.[52] On Vision Pro, FaceTime shows a user as a Persona, a digital representation that reflects face and hand movements.[3]
  • Retail and e-commerce: Virtually trying on clothes or accessories (AR mirrors), or placing virtual furniture or appliances in a room using mobile AR apps before purchase. IKEA Place, announced on 12 September 2017 and built on ARKit, offered more than 2,000 IKEA products at launch as 3D, true-to-scale models that users could place in their own rooms.[53]
  • Entertainment and gaming: Immersive VR games with room-scale tracking, location-based AR games blending virtual elements with the real world, interactive spatial storytelling, and spatial viewing of 360-degree and volumetric content.[28]
  • Navigation and information access: Contextual information overlaid on the real world (for example AR directions in streets or airports, information about landmarks) and indoor navigation aids. Google Maps' Live View, released as a beta on 8 August 2019, overlays arrows, directions and street names on the camera view, using the phone's camera, location data and Street View imagery.[54]
  • Architecture and construction: Visualizing architectural designs on-site using AR, and virtual walkthroughs of buildings in VR before construction. A 2013 review by Wang, Kim, Love and Kang classified 120 built-environment AR articles published between 2005 and 2011.[55]

Industry adoption and notable devices and platforms

Several major technology companies have built spatial computing products (status as of October 2026):

  • Microsoft: An early mover with Microsoft HoloLens (2016) and HoloLens 2 (2019, US$3,500 at its February 2019 announcement), primarily targeting enterprise and industrial MR use cases.[22][56] Microsoft confirmed in October 2024 that HoloLens 2 production had ended with no replacement; it promised security and regression updates until 31 December 2027.[57] The platform includes Windows Holographic; its collaborative service Microsoft Mesh was discontinued on 1 December 2025 and replaced by immersive events in Microsoft Teams.[58]
  • Apple: Explicitly entered the market branding Apple Vision Pro (announced 2023, released 2024) as its first "spatial computer," running visionOS.[3][31] An updated model with the M5 chip and Dual Knit Band went on sale on 22 October 2025 at US$3,499, rendering 10 percent more pixels and running at up to 120Hz.[35] visionOS 27, announced at WWDC in June 2026,[59] was released for both Vision Pro models on 14 September 2026.[60][61] The device focuses on productivity (virtual displays), entertainment (immersive video), and spatial FaceTime.[3]
  • Meta: Sells the consumer Meta Quest line. Quest Pro (2022) added full-color passthrough,[62] Quest 3 (October 2023, from US$499.99) was billed as a mixed reality headset,[46] and Quest 3S followed in 2024 from US$299.99. Meta's developer blog describes its Horizon OS as designed for spatial computing and offers the Meta Spatial SDK for Android developers.[63] It focuses on gaming, social VR (Horizon Worlds), and productivity (Horizon Workrooms). Its AR glasses prototype Orion, previously codenamed Project Nazare, was shown in September 2024 as a product prototype that would not be sold to consumers.[64] Meta Ray-Ban Display glasses, controlled with the Meta Neural Band EMG wristband, went on sale in the US on 30 September 2025 from US$799,[65] and Meta's product page lists Meta VR Glasses at US$1,299, "Coming Spring 2027", with no pre-order.[66]
  • Magic Leap: Early player focused on high-end AR/MR headsets (Magic Leap 1 in 2018, Magic Leap 2 on 30 September 2022 from about US$3,300) and used the term "spatial computing" for its platform.[25][67] Magic Leap 1 stopped functioning after its cloud services ended on 31 December 2024.[68] Direct sales of Magic Leap 2 ended on 31 March 2026, with resellers allowed to deliver units through 31 December 2026,[69] and in July 2026 the company announced it was moving away from its own devices to supply waveguides and integration expertise to partners building AR glasses.[40]
  • Google: Develops the ARCore platform for Android mobile AR. Explored early concepts with Project Tango and Google Glass. Current efforts include Google Maps Live View (AR navigation),[54] Google Beam (the 3D video communication platform, formerly Project Starline, renamed in May 2025),[70] and Android XR. At Google I/O in May 2026 Google said that audio-only Android XR glasses made with Samsung, Gentle Monster and Warby Parker would arrive in fall 2026, ahead of display glasses.[71]
  • Samsung: Builds the Galaxy XR headset on Android XR with a Snapdragon XR2+ Gen 2 chip; at launch it was sold in the United States and South Korea.[34][72]
  • Others: Companies like Nvidia provide underlying technologies (GPUs, and Nvidia Omniverse libraries for high-fidelity rendering of large models and digital twins in spatial applications),[4] while engine providers like Unity and Unreal Engine offer development tools for XR.[41][42] Numerous startups focus on specific applications or hardware components.

Challenges, criticisms, and terminology confusion

Spatial computing faces several hurdles:

  • Technical limitations: Constraints remain in field of view (especially for optical see-through AR), display resolution and brightness, device weight and ergonomic comfort, Battery life, and on-device processing power. A 2021 review of AR and VR displays by Xiong and colleagues states that matching the performance of human vision while keeping the near-eye display module compact and lightweight "imposes unprecedented challenges on optical engineering".[73] Azuma identified registration and sensing errors as two of the biggest problems in building effective AR systems.[39] Battery life on current headsets is measured in hours: Apple rates the M5 Vision Pro at up to two and a half hours of general use,[35] and Samsung rates the Galaxy XR at up to two hours of general use.[34]
  • Cost: High-end devices like HoloLens 2 (US$3,500 at announcement) and Apple Vision Pro (US$3,499 at launch) are expensive.[56][31] Newer devices are cheaper: the Galaxy XR launched at US$1,799.99 and Meta lists its VR Glasses at US$1,299.[34][66]
  • User experience (UX) and adoption: Issues like motion sickness or visual fatigue can affect some users; Steven LaValle's textbook Virtual Reality devotes a chapter to evaluating VR systems that covers VR sickness.[74] Social acceptance of wearing head-mounted devices in public is still evolving.
  • Privacy and security: Devices constantly scanning the user's environment with cameras and sensors (potentially including eye tracking and hand tracking) raise privacy concerns regarding data collection, storage, and use. The Electronic Frontier Foundation wrote in 2022 that the sensors on AR devices can record everyone and everything nearby, posing risks to both users and bystanders, and that eye tracking can capture involuntary reactions.[75]
  • Definition ambiguity and buzzword status: The term "spatial computing" has been described as marketing language, particularly following Apple's push. The Associated Press called it a mode of technology that Apple executives and marketers were trying to bring into the mainstream while avoiding the more widely used terms "augmented reality" and "virtual reality".[8] Apple's own developer guidelines tell developers not to describe their Vision Pro apps as AR, VR, XR or MR.[7] In a January 2024 commentary, Tomislav Bezmalinovic of MIXED argued that "technically, Vision Pro is still a VR headset" and that Apple's advertising and naming did not change that.[76] In academic work, Wang and colleagues describe the field as conceptually fragmented across HCI, geographic information science and computer science.[6] While rooted in academic work, its current usage encompasses a broad, sometimes inconsistent, range of technologies.

Outlook and perspectives

Companies and researchers have described several directions, mostly as plans or research problems rather than shipping products:

  • Hardware evolution: Lighter, smaller devices resembling standard eyeglasses. Meta said in 2024 that it planned to begin shipping consumer AR glasses "in the near future" after Orion,[64] Google said its first Android XR glasses would be audio-only, with display glasses to follow,[71] and Magic Leap's 2026 strategy is to supply waveguides for partners' AI display glasses.[40] Display research continues on holographic optical elements, micro-LEDs and metasurfaces.[73]
  • AI integration: Assistants that interpret the user's surroundings and generative AI for content creation within spatial environments. NVIDIA describes generative AI that creates 3D and 4D models from text descriptions,[4] Google built Android XR around its Gemini assistant,[33] and visionOS 27 added Visual Intelligence for asking questions about physical and virtual surroundings.[59][61]
  • Ecosystem development: Standardization of platforms and protocols through OpenXR, which Khronos describes as reducing the time and cost of adapting applications to individual XR platforms.[44]
  • Enhanced interaction: Sensor-based inputs such as electromyography (EMG) wristbands, which translate electrical motor nerve signals into digital commands. Facebook (now Meta) described its EMG wrist research in 2021 and distinguished it from implanted brain-computer interfaces;[77] the resulting Meta Neural Band shipped with Meta Ray-Ban Display in 2025.[65]

Tim Cook has described spatial computing as a new era of computing, comparing Vision Pro with the Mac and iPhone.[3][32] Cathy Hackl told the Associated Press in 2024 that "Spatial computing will enable devices to understand the world in ways they never have been able to do before."[8]

See also

References

  1. ↑ 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 Simon Greenwold (2003-06). "Spatial Computing". Master of Science thesis, Program in Media Arts and Sciences, Massachusetts Institute of Technology. https://acg.media.mit.edu/people/simong/thesis/SpatialComputing.pdf. Retrieved 2026-10-17.
  2. ↑ Simon Greenwold (2003). "In form". DSpace@MIT (catalog record of the S.M. thesis, Program in Media Arts and Sciences; abstract identical to the "Spatial Computing" thesis). Massachusetts Institute of Technology. https://dspace.mit.edu/handle/1721.1/61547. Retrieved 2026-10-17.
  3. ↑ 3.00 3.01 3.02 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.10 3.11 3.12 "Introducing Apple Vision Pro: Apple's first spatial computer". Apple Newsroom. Apple. 2023-06-05. https://www.apple.com/newsroom/2023/06/introducing-apple-vision-pro/. Retrieved 2026-10-17.
  4. ↑ 4.00 4.01 4.02 4.03 4.04 4.05 4.06 4.07 4.08 4.09 "What Is Spatial Computing?". NVIDIA Glossary. NVIDIA. https://www.nvidia.com/en-us/glossary/spatial-computing/. Retrieved 2026-10-17.
  5. ↑ 5.0 5.1 5.2 5.3 5.4 Alexander S. Gillis, George Lawton (2024-02-12). "What is spatial computing?". TechTarget. https://www.techtarget.com/searchcio/definition/spatial-computing. Retrieved 2026-10-17.
  6. ↑ 6.0 6.1 6.2 6.3 6.4 6.5 6.6 6.7 Yibo Wang, Yuhan Luo, Janghee Cho, Junnan Yu (2025-08-29). "What is "Spatial" about Spatial Computing?". arXiv preprint 2508.20477 (cs.HC). https://arxiv.org/abs/2508.20477. Retrieved 2026-10-17.
  7. ↑ 7.0 7.1 "Submit your apps to the App Store for Apple Vision Pro". Apple Developer. Apple. https://developer.apple.com/visionos/submit/. Retrieved 2026-10-17.
  8. ↑ 8.0 8.1 8.2 8.3 Associated Press (2024-02-02). "Why Apple is pushing the term 'spatial computing' along with its new Vision Pro headset". Spectrum News NY1 (AP Top News). https://ny1.com/nyc/all-boroughs/ap-top-news/2024/02/02/why-apple-is-pushing-the-term-spatial-computing-along-with-its-new-vision-pro-headset. Retrieved 2026-10-17.
  9. ↑ Shashi Shekhar, Steven K. Feiner, Walid G. Aref (2016-01). "Spatial Computing". Communications of the ACM, vol. 59, no. 1, pp. 72-81. https://doi.org/10.1145/2756547. Retrieved 2026-10-17.
  10. ↑ 10.0 10.1 10.2 Cathy Hackl (2023-11-10). "What leaders need to know about spatial computing". MIT Media Lab (originally published in Harvard Business Review). https://www.media.mit.edu/articles/what-leaders-need-to-know-about-spatial-computing/. Retrieved 2026-10-17.
  11. ↑ Ivan Edward Sutherland (1963). "Sketchpad, a man-machine graphical communication system". DSpace@MIT (Ph.D. thesis, Massachusetts Institute of Technology). https://dspace.mit.edu/handle/1721.1/14979. Retrieved 2026-10-17.
  12. ↑ Ivan E. Sutherland (1968-12). "A head-mounted three dimensional display". Proceedings of the AFIPS Fall Joint Computer Conference, part I. https://doi.org/10.1145/1476589.1476686. Retrieved 2026-10-17.
  13. ↑ 13.0 13.1 Mark Weiser (1991-09). "The Computer for the 21st Century". Scientific American, vol. 265, no. 3 (author's pre-publication text, hosted by Stanford University). https://graphics.stanford.edu/courses/cs428-03-spring/Papers/readings/General/Weiser_SciAm91.htm. Retrieved 2026-10-17.
  14. ↑ "Thomas A. Furness". University of Washington Industrial & Systems Engineering. https://ise.washington.edu/facultyfinder/thomas-a-furness. Retrieved 2026-10-17.
  15. ↑ "Bob Jacobson: HIT Lab Associate Director and Architect of First VR Forum". AWE XR Hall of Fame. AWE. https://www.awexr.com/hall-of-fame/135-bob-jacobson. Retrieved 2026-10-17.
  16. ↑ Robert Jacobson (1995-02-22). "Re: ANNOUNCE: Looking for New Products". sci.virtual-worlds (Usenet), archived by Lysator/Ludd. https://ftp.ludd.ltu.se/pub/misc/virtual-worlds/commercial/Worldesign-2.95. Retrieved 2026-10-17.
  17. ↑ T. Caelli, P. Lam, H. Bunke (eds.) (1997-08). "Spatial Computing: Issues in Vision, Multimedia and Visualization Technologies". World Scientific. https://doi.org/10.1142/3342. Retrieved 2026-10-17.
  18. ↑ "Spatial Computing: Issues in Vision, Multimedia and Visualization Technologies". Open Library. Internet Archive. https://openlibrary.org/works/OL27986275W. Retrieved 2026-10-17.
  19. ↑ 19.0 19.1 Bill N. Schilit, Norman Adams, Roy Want (1994-12). "Context-Aware Computing Applications". Proceedings of the First Workshop on Mobile Computing Systems and Applications (WMCSA), pp. 85-90. https://doi.org/10.1109/WMCSA.1994.16. Retrieved 2026-10-17.
  20. ↑ "Kinect for Xbox 360 Hits Million Mark in Just 10 Days". Microsoft News. Microsoft. 2010-11-15. https://news.microsoft.com/source/2010/11/15/kinect-for-xbox-360-hits-million-mark-in-just-10-days/. Retrieved 2026-10-17.
  21. ↑ Adam Westlake (2017-12-16). "Google Kills Project Tango In Favor Of ARCore". SlashGear. https://www.slashgear.com/google-kills-project-tango-in-favor-of-arcore-16512026. Retrieved 2026-10-17.
  22. ↑ 22.0 22.1 22.2 Alex Kipman (2016-02-29). "Announcing Microsoft HoloLens Development Edition open for pre-order, shipping March 30". Windows Blog. Microsoft. https://blogs.windows.com/devices/2016/02/29/announcing-microsoft-hololens-development-edition-open-for-pre-order-shipping-march-30/. Retrieved 2026-10-17.
  23. ↑ "Microsoft HoloLens". Microsoft Learn. Microsoft. https://www.microsoft.com/en-us/hololens. Retrieved 2026-10-17.
  24. ↑ Sean Hollister (2014-11-19). "How Magic Leap Is Secretly Creating a New Alternate Reality". Gizmodo. https://gizmodo.com/how-magic-leap-is-secretly-creating-a-new-alternate-rea-1660441103. Retrieved 2026-10-17.
  25. ↑ 25.0 25.1 Kyt Dotson (2018-08-08). "Magic Leap One augmented reality goggles ship today for $2,295". SiliconANGLE. https://siliconangle.com/2018/08/08/magic-leap-one-augmented-reality-goggles-ship-today-2295-price-tag/. Retrieved 2026-10-17.
  26. ↑ 26.0 26.1 "iOS 11 brings powerful new features to iPhone and iPad this fall". Apple Newsroom. Apple. 2017-06-05. https://www.apple.com/newsroom/2017/06/ios-11-brings-new-features-to-iphone-and-ipad-this-fall/. Retrieved 2026-10-17.
  27. ↑ 27.0 27.1 27.2 27.3 Dave Burke (2017-08-29). "ARCore: Augmented reality at Android scale". The Keyword. Google. https://blog.google/products/arcore/arcore-augmented-reality-android-scale/. Retrieved 2026-10-17.
  28. ↑ 28.0 28.1 Scott Hayden (2017-12-21). "'Pokémon GO' Now Uses Apple's ARKit on iOS 11, Bringing Pokémon Closer to Reality". Road to VR. https://roadtovr.com/pokemon-go-now-uses-apples-arkit-ios-11-bringing-pokemon-closer-reality/. Retrieved 2026-10-17.
  29. ↑ David Jagneaux (2019-04-30). "Oculus Quest And Rift S Launch May 21, Pre-Orders Now Available". UploadVR. https://uploadvr.com/oculus-quest-rift-s-launch-pre-orders/. Retrieved 2026-10-17.
  30. ↑ "Introducing Meta: A Social Technology Company". Meta Newsroom. Meta. 2021-10-28. https://about.fb.com/news/2021/10/facebook-company-is-now-meta/. Retrieved 2026-10-17.
  31. ↑ 31.0 31.1 31.2 "Apple Vision Pro available in the U.S. on February 2". Apple Newsroom. Apple. 2024-01-08. https://www.apple.com/newsroom/2024/01/apple-vision-pro-available-in-the-us-on-february-2/. Retrieved 2026-10-17.
  32. ↑ 32.0 32.1 Filipe Espósito (2024-02-02). "Tim Cook compares Vision Pro launch to iPhone launch in memo to employees". 9to5Mac. https://9to5mac.com/2024/02/02/tim-cook-vision-pro-launch-memo/. Retrieved 2026-10-17.
  33. ↑ 33.0 33.1 Shahram Izadi (2024-12-12). "Android XR: The Gemini era comes to headsets and glasses". The Keyword. Google. https://blog.google/products/android/android-xr/. Retrieved 2026-10-17.
  34. ↑ 34.0 34.1 34.2 34.3 34.4 34.5 34.6 34.7 "Introducing Galaxy XR: Opening New Worlds". Samsung Newsroom U.S.. Samsung Electronics. 2025-10-21. https://news.samsung.com/us/introducing-galaxy-xr-opening-new-worlds/. Retrieved 2026-10-17.
  35. ↑ 35.0 35.1 35.2 "Apple Vision Pro upgraded with the powerful M5 chip and comfortable Dual Knit Band". Apple Newsroom. Apple. 2025-10-15. https://www.apple.com/newsroom/2025/10/apple-vision-pro-upgraded-with-the-m5-chip-and-dual-knit-band/. Retrieved 2026-10-17.
  36. ↑ 36.0 36.1 Hugh Durrant-Whyte, Tim Bailey (2006-06). "Simultaneous localization and mapping: part I". IEEE Robotics & Automation Magazine, vol. 13, no. 2, pp. 99-110. https://doi.org/10.1109/MRA.2006.1638022. Retrieved 2026-10-17.
  37. ↑ 37.0 37.1 "Spatial anchors". Microsoft Learn (Mixed Reality documentation). Microsoft. https://learn.microsoft.com/en-us/windows/mixed-reality/design/spatial-anchors. Retrieved 2026-10-17.
  38. ↑ "Depth adds realism". ARCore, Google for Developers. Google. https://developers.google.com/ar/develop/depth. Retrieved 2026-10-17.
  39. ↑ 39.0 39.1 39.2 Ronald T. Azuma (1997-08). "A Survey of Augmented Reality". Presence: Teleoperators and Virtual Environments, vol. 6, no. 4, pp. 355-385. doi:10.1162/pres.1997.6.4.355. https://www.ronaldazuma.com/papers/ARpresence.pdf. Retrieved 2026-10-17.
  40. ↑ 40.0 40.1 40.2 "Magic Leap pivots to AR waveguide supply". optics.org. 2026-07-13. https://optics.org/news/magic-leap-pivots-to-ar-waveguide-supply. Retrieved 2026-10-17.
  41. ↑ 41.0 41.1 41.2 "XR". Unity Manual. Unity Technologies. https://docs.unity3d.com/Manual/XR.html. Retrieved 2026-10-17.
  42. ↑ 42.0 42.1 "Developing for XR Experiences in Unreal Engine". Unreal Engine Documentation. Epic Games. https://dev.epicgames.com/documentation/en-us/unreal-engine/developing-for-xr-experiences-in-unreal-engine. Retrieved 2026-10-17.
  43. ↑ "Designing for visionOS". Human Interface Guidelines. Apple. https://developer.apple.com/design/human-interface-guidelines/designing-for-visionos. Retrieved 2026-10-17.
  44. ↑ 44.0 44.1 "OpenXR". Khronos Group. https://www.khronos.org/openxr/. Retrieved 2026-10-17.
  45. ↑ Paul Milgram, Haruo Takemura, Akira Utsumi, Fumio Kishino (1995). "Augmented reality: a class of displays on the reality-virtuality continuum". Proceedings of SPIE, vol. 2351, Telemanipulator and Telepresence Technologies, pp. 282-292. https://doi.org/10.1117/12.197321. Retrieved 2026-10-17.
  46. ↑ 46.0 46.1 "Meet Meta Quest 3, Our Mixed Reality Headset Starting at $499.99". Meta Newsroom. Meta. 2023-09-27. https://about.fb.com/news/2023/09/meet-meta-quest-3-mixed-reality-headset/. Retrieved 2026-10-17.
  47. ↑ Mark Weiser, John Seely Brown (1995-12-21). "Designing Calm Technology". Xerox PARC (archived by the Internet Archive). https://web.archive.org/web/20180604010109/http://www.ubiq.com:80/hypertext/weiser/calmtech/calmtech.htm. Retrieved 2026-10-17.
  48. ↑ P. Vávra, J. Roman, P. Zonča, P. Ihnát, M. Němec, J. Kumar, N. Habib, A. El-Gendi (2017). "Recent Development of Augmented Reality in Surgery: A Review". Journal of Healthcare Engineering, vol. 2017, article 4574172. https://doi.org/10.1155/2017/4574172. Retrieved 2026-10-17.
  49. ↑ Erin Laviola (2024-04-03). "Spatial Computing: What Do Healthcare IT Leaders Need to Know?". HealthTech Magazine. https://healthtechmagazine.net/article/2024/04/spatial-computing-healthcare-perfcon. Retrieved 2026-10-17.
  50. ↑ Kate E. Laver, Belinda Lange, Stacey George, Judith E. Deutsch, Gustavo Saposnik, Maria Crotty (2017-11-20). "Virtual reality for stroke rehabilitation". Cochrane Database of Systematic Reviews, CD008349. https://doi.org/10.1002/14651858.CD008349.pub4. Retrieved 2026-10-17.
  51. ↑ Jorge Bacca, Silvia Baldiris, Ramon Fabregat, Sabine Graf, Kinshuk (2014). "Augmented Reality Trends in Education: A Systematic Review of Research and Applications". Educational Technology & Society, vol. 17, no. 4, pp. 133-149. https://dugi-doc.udg.edu/handle/10256/17763. Retrieved 2026-10-17.
  52. ↑ Mark Billinghurst, Hirokazu Kato (2002-07). "Collaborative augmented reality". Communications of the ACM, vol. 45, no. 7, pp. 64-70. https://doi.org/10.1145/514236.514265. Retrieved 2026-10-17.
  53. ↑ "Launch of new IKEA Place app". IKEA Global Newsroom. Inter IKEA Systems. 2017-09-12. https://www.ikea.com/global/en/newsroom/innovation/ikea-launches-ikea-place-a-new-app-that-allows-people-to-virtually-place-furniture-in-their-home-170912/. Retrieved 2026-10-17.
  54. ↑ 54.0 54.1 Natashah Hitti (2019-08-13). "Google Maps launches AR walking directions". Dezeen. https://www.dezeen.com/2019/08/13/google-maps-ar-live-view-tech/. Retrieved 2026-10-17.
  55. ↑ Xiangyu Wang, Mi Jeong Kim, Peter E.D. Love, Shih-Chung Kang (2013-07). "Augmented Reality in built environment: Classification and implications for future research". Automation in Construction, vol. 32, pp. 1-13. https://doi.org/10.1016/j.autcon.2012.11.021. Retrieved 2026-10-17.
  56. ↑ 56.0 56.1 Lewis Painter (2019-02-24). "HoloLens 2 release date, pricing, features and specs news". Tech Advisor. https://www.techadvisor.com/article/729527/hololens-2-release-date-pricing-features-and-specs-news.html. Retrieved 2026-10-17.
  57. ↑ David Heaney (2024-10-01). "Microsoft Is Discontinuing HoloLens 2, With No Replacement". UploadVR. https://www.uploadvr.com/microsoft-discontinuing-hololens-2/. Retrieved 2026-10-17.
  58. ↑ Richard Speed (2025-12-02). "Microsoft mops up Mesh after another metaverse misfire". The Register. https://www.theregister.com/software/2025/12/02/microsoft-mops-up-mesh-after-another-metaverse-misfire/2385405. Retrieved 2026-10-17.
  59. ↑ 59.0 59.1 David Heaney (2026-06-14). "visionOS 27 Is A Much Bigger Update Than Apple's Keynote Suggested". UploadVR. https://uploadvr.com/visionos-27-announced-apple-vision-pro-wwdc-26. Retrieved 2026-10-17.
  60. ↑ Ryan Christoffel (2026-09-09). "visionOS 27 release date: Here's when the new update launches". 9to5Mac. https://9to5mac.com/2026/09/09/visionos-27-release-date-heres-when-the-new-update-launches/. Retrieved 2026-10-17.
  61. ↑ 61.0 61.1 Shalom Levytam (2026-09-14). "Apple Officially Releases visionOS 27". iClarified. https://www.iclarified.com/102219/apple-officially-releases-visionos-27-download. Retrieved 2026-10-17.
  62. ↑ Sam Rutherford (2022-10-31). "Meta Quest Pro review: A next-gen headset for the VR faithful". Engadget. https://www.engadget.com/meta-quest-pro-review-a-next-gen-headset-for-the-vr-faithful-specs-price-130045313-130045431.html. Retrieved 2026-10-17.
  63. ↑ "Start Building with Meta Quest 3 and 3S". Meta for Developers Blog. Meta. 2024-10-15. https://developers.meta.com/horizon/blog/start-building-meta-quest-3s-3-launch-spatial-sdk-2D-mixed-reality/. Retrieved 2026-10-17.
  64. ↑ 64.0 64.1 "Introducing Orion, Our First True Augmented Reality Glasses". Meta Newsroom. Meta. 2024-09-25. https://about.fb.com/news/2024/09/introducing-orion-our-first-true-augmented-reality-glasses/. Retrieved 2026-10-17.
  65. ↑ 65.0 65.1 "Meta Ray-Ban Display: AI Glasses With an EMG Wristband". Meta Newsroom. Meta. 2025-09-17. https://about.fb.com/news/2025/09/meta-ray-ban-display-ai-glasses-emg-wristband/. Retrieved 2026-10-17.
  66. ↑ 66.0 66.1 "Meta VR Glasses". Meta. Meta. https://www.meta.com/vr-glasses/. Retrieved 2026-10-17.
  67. ↑ Ben Lang (2022-07-12). "Magic Leap 2 Starting Price Positioned Slightly Less Than HoloLens 2, Release Date Set for September". Road to VR. https://www.roadtovr.com/magic-leap-2-release-date-price-specs/. Retrieved 2026-10-17.
  68. ↑ David Heaney (2023-08-30). "Magic Leap 1 AR Headsets Will "Cease To Function" After 2024". UploadVR. https://www.uploadvr.com/magic-leap-1-cease-to-function-2025/. Retrieved 2026-10-17.
  69. ↑ "Terms of Sale for Magic Leap 2". Magic Leap. 2026-02-19. https://www.magicleap.com/legal/terms-of-sale. Retrieved 2026-10-17.
  70. ↑ Andrew Nartker (2025-05-20). "Google Beam: Our AI-first 3D video communication platform". The Keyword. Google. https://blog.google/technology/research/project-starline-google-beam-update/. Retrieved 2026-10-17.
  71. ↑ 71.0 71.1 Shahram Izadi (2026-05-19). "Intelligent eyewear is coming this fall". The Keyword. Google. https://blog.google/products-and-platforms/platforms/android/android-xr-io-2026/. Retrieved 2026-10-17.
  72. ↑ David Heaney (2025-10-21). "Samsung Galaxy XR With Android XR Out Now For $1800, Controllers $250". UploadVR. https://www.uploadvr.com/samsung-galaxy-xr-google-android-xr-out-now/. Retrieved 2026-10-17.
  73. ↑ 73.0 73.1 Jianghao Xiong, En-Lin Hsiang, Ziqian He, Tao Zhan, Shin-Tson Wu (2021-10-25). "Augmented reality and virtual reality displays: emerging technologies and future perspectives". Light: Science & Applications, vol. 10, article 216. https://doi.org/10.1038/s41377-021-00658-8. Retrieved 2026-10-17.
  74. ↑ Steven M. LaValle (2023). "Virtual Reality". Cambridge University Press (free online edition). https://lavalle.pl/vr/. Retrieved 2026-10-17.
  75. ↑ Katitza Rodriguez, Rory Mir (2022-12-24). "Pivotal Year for the Metaverse and Extended Reality: 2022 in Review". Electronic Frontier Foundation. https://www.eff.org/deeplinks/2022/12/pivotal-year-metaverse-and-extended-reality. Retrieved 2026-10-17.
  76. ↑ Tomislav Bezmalinovic (2024-01-09). "Apple's marketing aside, Vision Pro is still a VR headset". MIXED. https://mixed-news.com/en/apple-vision-pro-is-a-vr-headset/. Retrieved 2026-10-17.
  77. ↑ Lauren Goode (2021-03-18). "Facebook Finally Explains Its Mysterious Wrist Wearable". WIRED. https://www.wired.com/story/facebook-wrist-wearable-human-computer-interactions/. Retrieved 2026-10-17.
Retrieved from "https://vrarwiki.com/index.php?title=Spatial_computing&oldid=52419"