Spatial anchors: Difference between revisions

'''Spatial anchors''' are persistent reference points in the real world that [[augmented reality]] (AR) and [[mixed reality]] (MR) systems use to lock virtual objects to a fixed location in physical space.<ref name="MagicLeap">[https://developer-docs.magicleap.cloud/docs/guides/unity/perception/anchors/spatial-anchors-overview/ Magic Leap Developer Docs – Spatial Anchors Overview (2025)]</ref><ref name="MSLearn">[https://learn.microsoft.com/en-us/windows/mixed-reality/design/spatial-anchors Microsoft Learn – Spatial Anchors (2025)]</ref> A spatial anchor establishes a world-locked frame of reference that defines a point in the environment with a unique coordinate frame, capturing a complete '''6 degrees of freedom (6DOF)''' representation—three translational coordinates (X, Y, Z position) and three rotational coordinates (pitch, yaw, roll orientation).<ref name="OpenXR">[https://registry.khronos.org/OpenXR/specs/1.1/man/html/XrSpatialAnchorMSFT.html Khronos OpenXR – XR_MSFT_spatial_anchor Extension Specification]</ref><ref name="BrownWiki">[https://www.vrwiki.cs.brown.edu/vr-development-software/unity/spatial-anchors VR Software Wiki – Spatial Anchors in Unity]</ref> The AR device continuously tracks this anchor over time, so that any digital content attached to it remains accurately '''world-locked''' (tied to a real-world position and orientation) rather than floating or drifting as the user moves.<ref name="ARKitAnchor">[https://www.captechconsulting.com/blogs/visualizing-surfaces-detected-by-arkit CapTech Consulting – ARAnchor ARKit Overview (2019)]</ref>

By rendering virtual objects relative to a spatial anchor's coordinate system, those objects appear fixed in the real world with minimal drift or deviation, even as the user changes their viewpoint or returns to the scene later.<ref name="MagicLeap"/><ref name="OpenXR"/> This capability is essential for creating believable and immersive experiences where digital elements appear to be a natural part of the user's surroundings, solving the fundamental AR problem of '''drift'''—where virtual objects can appear to float away from their intended positions as the system's understanding of the environment updates.<ref name="RecreateFAQ">[https://recreate.nl/faq-items/what-is-a-spatial-anchor/ Recreate – What is a spatial anchor?]</ref>

The AR system continuously updates its understanding of the environment through its sensors, making small, ongoing adjustments to the anchor's pose (position and orientation) to keep it perfectly aligned with the real world. These micro-corrections are what keep the attached virtual content world-locked and stable.<ref name="MSLearn"/> A critical characteristic is that the world coordinate system is not static—as the AR system's understanding of the environment changes, it adjusts its model of the world to keep things consistent, with the anchor's transformation automatically updated each frame to compensate for changes in the system's world model.<ref name="ARCoreConcepts">[https://developers.google.com/ar/reference/c/group/concepts Google ARCore – Concepts Documentation]</ref>

Spatial anchors are fundamentally based on '''trackable feature points''' detected in the environment through [[computer vision]] algorithms.<ref name="Reko3D">[https://reko3d.com/blog/spatial-anchors/ Reko3D XR Glossary – Spatial Anchors (2024)]</ref> The AR platform detects distinctive visual features in camera images—such as corners, edges, T-junctions, and texture patterns—using algorithms like ORB (Oriented FAST and Rotated BRIEF), [[SIFT]] (Scale-Invariant Feature Transform), or SURF (Speeded Up Robust Features).<ref name="JaklAnalysis">[https://www.andreasjakl.com/basics-of-ar-anchors-keypoints-feature-detection/ Andreas Jakl – Basics of AR Anchors and Feature Detection]</ref>

In some cases, spatial anchors can be defined using geospatial data such as GPS coordinates and maps, allowing virtual content to be tied to a specific latitude and longitude in the real world.<ref name="Reko3D"/> AR platforms now support '''geospatial anchors''' that let developers place content at global positions—anchoring virtual objects by latitude, longitude, and altitude without needing to scan the immediate surroundings.<ref name="ARCoreGeo">[https://developers.googleblog.com/en/make-the-world-your-canvas-with-the-arcore-geospatial-api Google Developers Blog – ARCore Geospatial API Announcement (2022)]</ref> These anchors leverage [[Visual Positioning System]] (VPS) technology that uses pre-captured imagery databases (such as [[Google Street View]]) with machine learning to extract 3D points and match device camera feeds against VPS models, providing centimeter-level accuracy where available.<ref name="NianticVPS">[https://lightship.dev/docs/ardk/3.6/features/lightship_vps/ Niantic Lightship VPS Documentation – Persistent Location Anchors]</ref>

Spatial anchors are implemented on top of the device's environmental tracking capabilities, using techniques like [[simultaneous localization and mapping]] (SLAM) to identify visual feature points or surface geometry in the environment.<ref name="Reko3D"/> Modern AR systems achieve robust tracking through '''Visual-Inertial Odometry (VIO)''', which fuses data from camera sensors with [[Inertial Measurement Unit]] (IMU) sensors—combining accelerometers (measuring linear acceleration) and gyroscopes (measuring rotational velocity).<ref name="VSLAM_MDPI">[https://www.mdpi.com/1424-8220/24/4/1161 MDPI Sensors – Enhancing Outdoor Location-Based AR Anchors Using Visual SLAM]</ref>

Once created, a spatial anchor remains fixed at its real-world location and is not meant to be moved arbitrarily. Anchors are generally used for virtual elements intended to stay in one place (such as a holographic sign positioned on a wall) rather than for moving objects.<ref name="MSLearn2">[https://learn.microsoft.com/en-us/windows/mixed-reality/design/spatial-anchors Microsoft Learn – Spatial Anchors Usage Guidelines (2025)]</ref> Many AR frameworks treat anchors as relatively heavy-weight tracking objects—the system will continually adjust and refine the anchor's position to keep it aligned with the environment, which can incur computational cost.<ref name="UnityAnchor">[https://docs.unity3d.com/Packages/[email protected]/manual/anchor-manager.html Unity AR Foundation Documentation – ARAnchorManager]</ref>

Developers are advised to use spatial anchors sparingly and only for content that truly needs persistent world-locking, and to avoid attaching fast-moving or dynamic objects to anchors.<ref name="UnityAnchor"/><ref name="MSLearn2"/> A critical limitation documented by Microsoft is that '''holograms greater than 3 meters from anchor origin experience noticeable positional errors''' due to lever-arm effects—angular errors are small when close to the anchor but become magnified at distance.<ref name="MSLearn2"/> The recommended solution is creating new anchors for distant objects rather than using a single anchor, treating the 3-meter range as an effective radius for anchor accuracy.<ref name="MSLearn2"/>

Early AR relied on '''fiducial markers'''—QR codes and printed patterns that required pre-placed markers in environments. Software libraries like '''ARToolKit''', first released in 2000, allowed AR applications to recognize specific physical markers and use them as stable anchor points for virtual content.<ref name="ARToolKit">[https://www.assemblrworld.com/blog/history-of-augmented-reality Assemblr – The History of Augmented Reality]</ref> This marker-based approach was robust but limited the AR experience to locations where these predefined markers could be placed.

'''Azure Spatial Anchors''' (ASA) by Microsoft provided a cloud backend to which an application could upload local anchors and later retrieve them on another device (or by another user), enabling collaborative mixed reality across [[HoloLens]], iOS, and Android.<ref name="ASA"/> ASA established a common coordinate frame for shared experiences without requiring any QR codes or prior environmental setup—every device that located the Azure anchor would align its content to the exact same physical spot in the world.<ref name="ASA"/> The service worked by creating a cloud-based representation of an anchor's surrounding environment using feature descriptors (not actual images) that could be accessed by other devices via a unique ID.<ref name="ASA"/>

Initially, Cloud Anchors had a 24-hour time limit (anchors would expire after a day), but in 2020 Google launched '''persistent Cloud Anchors''' which can last much longer (on the order of 1 to 365 days) to support content that users can return to over time.<ref name="GoogleBlog2018"/> Using ARCore's cloud service, multiple people running the same app can place and see shared AR objects in a scene—for example, leaving AR graffiti that others can discover later at the same real-world location.

ARKit automatically generates some anchors (for example, when planar surfaces or images are detected, it creates plane anchors or image anchors), and developers can add their own anchors at arbitrary positions. ARKit does not have a built-in cloud anchor service, but it allows multi-user sharing by merging AR sessions: an app can share a map (which contains anchors) with another device to synchronize their coordinate space. In ARKit 4, Apple introduced '''Location Anchors (ARGeoAnchor)''', which use high-resolution Apple Maps data in certain cities to anchor content to real-world coordinates (latitude, longitude, altitude)—enabling experiences where AR content is tied to specific landmarks or locations.<ref name="AppleLocation">[https://www.apple.com/newsroom/2020/06/ios-14-offers-new-features/ Apple Newsroom – ARKit 4 Location Anchors (2020)]</ref>

Locally, HoloLens also allowed saving anchors to disk and exporting/importing them (via a WorldAnchorTransferBatch in Unity) to persist holograms between app runs.<ref name="MSLearn3"/> The HoloLens system has a limit on how far content can be from an anchor before stability decreases—Microsoft recommends keeping holograms within a few meters of their anchor for best accuracy, specifically noting that holograms greater than 3 meters from anchor origin experience noticeable positional errors.<ref name="MSLearn2"/>

Best practices emphasize that anchors are resource-intensive on most platforms, recommending avoiding multiple anchors within 2 meters of each other, removing anchors when no longer needed, and considering disabling plane detection after initial scan. Platform-specific behaviors vary—Meta Quest planes are static (not dynamically updated) while ARKit/ARCore planes dynamically update at runtime, and different platforms may have varying anchor limits.<ref name="UnityAnchor"/>

'''Minecraft Earth''' utilized Azure Spatial Anchors and PlayFab integration to create life-size AR experiences, allowing players to build and share persistent structures in the real world. The game featured "Adventures"—small slices of Minecraft worlds rendered in life-size AR on sidewalks and parks. Though later retired, it demonstrated the technical feasibility of large-scale collaborative AR gaming.<ref name="MinecraftEarth">[https://news.microsoft.com/features/minecraft-earth-azure-spatial-anchors/ Microsoft News – Minecraft Earth Technology]</ref>

'''Feature-scarce environments''' present the primary challenge—empty white walls, uniform floors, and large glass areas lack visual features for anchor creation. Anchors fail to create or match reliably in these conditions. The practical impact is severe in modern office buildings with minimalist design.<ref name="Qualium_Challenges"/><ref name="XREAL_Limitations">[https://xreal.gitbook.io/nrsdk/development/spatial-anchor XREAL Developer Docs – Spatial Anchor Limitations]</ref>

'''Dynamic environments''' with moving objects (people, equipment) occlude reference features, causing tracking issues and anchor instability. This is particularly problematic in crowded spaces or busy warehouses where the environment constantly changes. '''Surface requirements''' exclude transparent, semi-transparent, or reflective surfaces—mirrors, glass, and glossy surfaces prevent the system from detecting and tracking features effectively.<ref name="Qualium_Challenges"/>

Sharing anchors between devices can introduce additional alignment errors, where each user sees the virtual content in a slightly different position—displacement can be subtle (4–5 cm) but often significant enough (up to 20 cm in some cases) to break the illusion of stability, especially for applications requiring high precision.<ref name="Meta_Drift">[https://communityforums.atmeta.com/discussions/dev-unity/spatial-anchors-issues Meta Community Forums – Spatial Anchors Issues]</ref>

'''Latency issues''' in anchor stabilization affect user experience—matching saved data to real-time visuals should occur in under 5 seconds ideally. Poor performance leads to user frustration and abandonment. Platform variance shows ARKit is typically faster than cloud anchors (ARCore/Azure Spatial Anchors) for initial anchor establishment.<ref name="Qualium_Challenges"/>

The discontinuation of Azure Spatial Anchors—a major cross-platform solution—further underscores the risks for developers who rely on a single proprietary, centralized service for this critical functionality.<ref name="ASA_Sunset"/><ref name="MS_Sunset_Impact">[https://www.mdpi.com/2076-3417/15/13/6959 MDPI – Cross-Platform Framework for Synchronizing Spatial Anchors]</ref>

'''Network dependencies''' mean cloud anchors require stable internet connectivity—hosting and resolving anchors fails without connection. This limits applicability in industrial environments with restricted network access or outdoor locations with poor coverage. Hybrid approaches combining local anchors for offline scenarios with cloud anchors for cross-device sharing provide partial mitigation.<ref name="Qualium_Challenges"/>

For persistent anchors to function, the user's device must successfully "re-localize"—that is, recognize the current environment by matching it to a previously saved map. This process can be a significant point of friction and failure. If the user starts the application from a different position than where the map was created, or if the environment has changed too much, re-localization may fail, and the persistent content will not appear.<ref name="SABIAT_Paper">[https://benswift.me/assets/documents/preprints/he_et_al_2021_spatial_anchor_based_indoor_asset_tracking.pdf CSIRO Research – Spatial-Anchor-Based Indoor Asset Tracking]</ref>

Major platforms have implemented privacy protections. Azure Spatial Anchors followed three principles: '''Data segregation''' (each customer controls their own spatial anchor data store with no data sharing between subscriptions), '''Image-less anchor creation''' (images processed locally on edge devices with only derived format feature descriptors transmitted to cloud—original images never stored), and '''Pose-only anchor queries''' (returning only 6DoF pose by default with no image data returned to querying devices).<ref name="ASA_Privacy"/>

Niantic characterizes the AR Cloud as "one of the most important infrastructures in the history of computing" and the current moment as a "land grab" similar to early internet—presence, visibility, and brand relevance in the physical world will define market positions.<ref name="Niantic_Enterprise"/>

'''On-device machine learning''' in spatial computing enables Apple's Create ML to train object tracking models directly from command line—converting 3D model files through ML training for visionOS tracking. '''Multimodal AI''' bridges spatial and business data, processing text, image, audio, and spatial data types simultaneously.<ref name="Deloitte_Trends">[https://www2.deloitte.com/us/en/insights/focus/tech-trends/2025/tech-trends-future-of-spatial-computing.html Deloitte Tech Trends 2025 – Future of Spatial Computing]</ref>