Spatial mapping: Difference between revisions
Xinreality (talk | contribs) No edit summary |
Xinreality (talk | contribs) m Text replacement - "e.g.," to "for example" Tags: Mobile edit Mobile web edit |
||
| Line 52: | Line 52: | ||
This was followed by mobile AR frameworks: Apple's [[ARKit]] in June 2017 integrated visual-inertial odometry (VIO) for iOS devices, revolutionizing mobile AR by solving monocular Visual-Inertial Odometry without requiring depth sensors, instantly enabling 380 million devices.<ref name="AndreasJakl">{{cite web |url=https://www.andreasjakl.com/basics-of-ar-slam-simultaneous-localization-and-mapping/ |title=Basics of AR: SLAM – Simultaneous Localization and Mapping |publisher=Andreas Jakl |date=2018-08-14 |access-date=2025-10-27}}</ref> Google's [[ARCore]] in 2017 brought SLAM to Android, using similar depth-from-motion algorithms that compare images from different angles combined with IMU measurements to generate depth maps on standard hardware.<ref name="AndreasJakl"/> Meta's Oculus Quest (2019) incorporated inside-out tracking with SLAM for standalone VR/AR, eliminating external sensors.<ref name="MetaAnchorsDev">{{cite web |url=https://developers.meta.com/horizon/documentation/unity/unity-spatial-anchors-overview/ |title=Spatial Anchors Overview |publisher=Meta for Developers |date=2024-05-15 |access-date=2025-10-27}}</ref> | This was followed by mobile AR frameworks: Apple's [[ARKit]] in June 2017 integrated visual-inertial odometry (VIO) for iOS devices, revolutionizing mobile AR by solving monocular Visual-Inertial Odometry without requiring depth sensors, instantly enabling 380 million devices.<ref name="AndreasJakl">{{cite web |url=https://www.andreasjakl.com/basics-of-ar-slam-simultaneous-localization-and-mapping/ |title=Basics of AR: SLAM – Simultaneous Localization and Mapping |publisher=Andreas Jakl |date=2018-08-14 |access-date=2025-10-27}}</ref> Google's [[ARCore]] in 2017 brought SLAM to Android, using similar depth-from-motion algorithms that compare images from different angles combined with IMU measurements to generate depth maps on standard hardware.<ref name="AndreasJakl"/> Meta's Oculus Quest (2019) incorporated inside-out tracking with SLAM for standalone VR/AR, eliminating external sensors.<ref name="MetaAnchorsDev">{{cite web |url=https://developers.meta.com/horizon/documentation/unity/unity-spatial-anchors-overview/ |title=Spatial Anchors Overview |publisher=Meta for Developers |date=2024-05-15 |access-date=2025-10-27}}</ref> | ||
The introduction of LiDAR to consumer devices began with iPad Pro in March 2020 and iPhone 12 Pro in October 2020, using Vertical Cavity Surface Emitting Laser technology with direct Time-of-Flight measurement. This enabled ARKit 3.5's Scene Geometry API for instant AR with triangle mesh classification into semantic categories.<ref name="AppleDeveloper">{{cite web |url=https://developer.apple.com/documentation/arkit/arkit_scene_reconstruction |title=ARKit Scene Reconstruction |publisher=Apple Developer Documentation |date=2020 |access-date=2025-10-27}}</ref> The 2020s have seen refinements, such as HoloLens 2's Scene Understanding SDK (2019), which builds on spatial mapping for semantic environmental analysis.<ref name="MicrosoftDoc"/> Advancements in LiDAR ( | The introduction of LiDAR to consumer devices began with iPad Pro in March 2020 and iPhone 12 Pro in October 2020, using Vertical Cavity Surface Emitting Laser technology with direct Time-of-Flight measurement. This enabled ARKit 3.5's Scene Geometry API for instant AR with triangle mesh classification into semantic categories.<ref name="AppleDeveloper">{{cite web |url=https://developer.apple.com/documentation/arkit/arkit_scene_reconstruction |title=ARKit Scene Reconstruction |publisher=Apple Developer Documentation |date=2020 |access-date=2025-10-27}}</ref> The 2020s have seen refinements, such as HoloLens 2's Scene Understanding SDK (2019), which builds on spatial mapping for semantic environmental analysis.<ref name="MicrosoftDoc"/> Advancements in LiDAR (for example iPhone 12 Pro, 2020) and AI-driven feature detection have further democratized high-fidelity mapping.<ref name="AndreasJakl"/> | ||
Microsoft launched HoloLens 2 in 2019 with improved Azure Kinect sensors, and Meta Quest 3 arrived in 2023 with full-color passthrough, depth sensing via IR patterned light projector, and sophisticated Scene API with semantic labeling. Apple Vision Pro launched in 2024, representing the current state-of-the-art in spatial computing with advanced eye tracking and hand tracking. Today, spatial mapping is integral to spatial computing, with ongoing research in collaborative SLAM for multi-user experiences.<ref name="WikipediaSLAM"/> | Microsoft launched HoloLens 2 in 2019 with improved Azure Kinect sensors, and Meta Quest 3 arrived in 2023 with full-color passthrough, depth sensing via IR patterned light projector, and sophisticated Scene API with semantic labeling. Apple Vision Pro launched in 2024, representing the current state-of-the-art in spatial computing with advanced eye tracking and hand tracking. Today, spatial mapping is integral to spatial computing, with ongoing research in collaborative SLAM for multi-user experiences.<ref name="WikipediaSLAM"/> | ||
| Line 98: | Line 98: | ||
| '''[[Mapping Range]]''' || Controls the maximum distance from the sensor at which depth data is incorporated into the map. || 2 m – 20 m <ref name="StereolabsDocsS2"/> || High (longer range = more data to process = higher resource usage) || Moderate (longer range can map large areas faster but may reduce accuracy at the farthest points) | | '''[[Mapping Range]]''' || Controls the maximum distance from the sensor at which depth data is incorporated into the map. || 2 m – 20 m <ref name="StereolabsDocsS2"/> || High (longer range = more data to process = higher resource usage) || Moderate (longer range can map large areas faster but may reduce accuracy at the farthest points) | ||
|- | |- | ||
| '''[[Mesh Filtering]]''' || Post-processing to reduce polygon count (decimation) and clean mesh artifacts ( | | '''[[Mesh Filtering]]''' || Post-processing to reduce polygon count (decimation) and clean mesh artifacts (for example fill holes). || Presets (for example Low, Medium, High) <ref name="StereolabsDocsS2"/> || Low (reduces polygon count, leading to significant performance improvement in rendering) || Moderate (aggressive filtering can lead to loss of fine geometric detail) | ||
|- | |- | ||
| '''[[Mesh Texturing]]''' || The process of applying camera images to the mesh surface to create a photorealistic model. || On / Off <ref name="StereolabsDocsS2"/> || High (requires storing and processing images, creating a texture map, and using more complex shaders for rendering) || High (dramatically increases visual realism) | | '''[[Mesh Texturing]]''' || The process of applying camera images to the mesh surface to create a photorealistic model. || On / Off <ref name="StereolabsDocsS2"/> || High (requires storing and processing images, creating a texture map, and using more complex shaders for rendering) || High (dramatically increases visual realism) | ||
| Line 111: | Line 111: | ||
=== Essential Sensor Technologies === | === Essential Sensor Technologies === | ||
Modern XR devices rely on [[sensor fusion]], the process of combining data from multiple sensors to achieve a result that is more accurate and robust than could be achieved by any single sensor alone.<ref name="SLAMSystems">{{cite web |url=https://www.sbg-systems.com/glossary/slam-simultaneous-localization-and-mapping/ |title=SLAM - Simultaneous localization and mapping |publisher=SBG Systems |access-date=2025-10-23}}</ref><ref name="MilvusSensors">{{cite web |url=https://milvus.io/ai-quick-reference/what-sensors-eg-accelerometer-gyroscope-are-essential-in-ar-devices |title=What sensors ( | Modern XR devices rely on [[sensor fusion]], the process of combining data from multiple sensors to achieve a result that is more accurate and robust than could be achieved by any single sensor alone.<ref name="SLAMSystems">{{cite web |url=https://www.sbg-systems.com/glossary/slam-simultaneous-localization-and-mapping/ |title=SLAM - Simultaneous localization and mapping |publisher=SBG Systems |access-date=2025-10-23}}</ref><ref name="MilvusSensors">{{cite web |url=https://milvus.io/ai-quick-reference/what-sensors-eg-accelerometer-gyroscope-are-essential-in-ar-devices |title=What sensors (for example accelerometer, gyroscope) are essential in AR devices? |publisher=Milvus |access-date=2025-10-23}}</ref> The essential sensor suite includes: | ||
==== Depth Cameras ==== | ==== Depth Cameras ==== | ||
| Line 153: | Line 153: | ||
* '''[[Visual SLAM]] (vSLAM)''': Uses one or more cameras to track visual features.<ref name="MathWorksSLAM"/> | * '''[[Visual SLAM]] (vSLAM)''': Uses one or more cameras to track visual features.<ref name="MathWorksSLAM"/> | ||
* '''[[LiDAR SLAM]]''': Uses a LiDAR sensor to build a precise geometric map.<ref name="MathWorksSLAM"/> | * '''[[LiDAR SLAM]]''': Uses a LiDAR sensor to build a precise geometric map.<ref name="MathWorksSLAM"/> | ||
* '''[[Multi-Sensor SLAM]]''': Fuses data from various sources ( | * '''[[Multi-Sensor SLAM]]''': Fuses data from various sources (for example cameras, IMU, LiDAR) for enhanced robustness and accuracy.<ref name="MathWorksSLAM"/> | ||
Spatial mapping is typically accomplished via SLAM algorithms, which build a map of the environment in real time while tracking the device's position within it.<ref name="Adeia">{{cite web |url=https://adeia.com/blog/spatial-mapping-empowering-the-future-of-ar |title=Spatial Mapping: Empowering the Future of AR |publisher=Adeia |date=2022-03-02 |access-date=2025-10-27}}</ref> | Spatial mapping is typically accomplished via SLAM algorithms, which build a map of the environment in real time while tracking the device's position within it.<ref name="Adeia">{{cite web |url=https://adeia.com/blog/spatial-mapping-empowering-the-future-of-ar |title=Spatial Mapping: Empowering the Future of AR |publisher=Adeia |date=2022-03-02 |access-date=2025-10-27}}</ref> | ||
| Line 237: | Line 237: | ||
The principles of spatial mapping extend to a planetary scale through [[geospatial mapping]]. Instead of headset sensors, this field uses data from satellites, aircraft, drones, and ground-based sensors to create comprehensive 3D maps of the Earth.<ref name="Matrack">{{cite web |url=https://matrackinc.com/geospatial-mapping/ |title=What is Geospatial Mapping and How does it Work? |publisher=Matrack Inc. |access-date=2025-10-23}}</ref><ref name="Spyrosoft">{{cite web |url=https://spyro-soft.com/blog/geospatial/what-is-geospatial-mapping-and-how-does-it-work |title=What is Geospatial Mapping and How Does It Work? |publisher=Spyrosoft |access-date=2025-10-23}}</ref> | The principles of spatial mapping extend to a planetary scale through [[geospatial mapping]]. Instead of headset sensors, this field uses data from satellites, aircraft, drones, and ground-based sensors to create comprehensive 3D maps of the Earth.<ref name="Matrack">{{cite web |url=https://matrackinc.com/geospatial-mapping/ |title=What is Geospatial Mapping and How does it Work? |publisher=Matrack Inc. |access-date=2025-10-23}}</ref><ref name="Spyrosoft">{{cite web |url=https://spyro-soft.com/blog/geospatial/what-is-geospatial-mapping-and-how-does-it-work |title=What is Geospatial Mapping and How Does It Work? |publisher=Spyrosoft |access-date=2025-10-23}}</ref> | ||
* This large-scale mapping is critical for urban planning, precision agriculture, environmental monitoring ( | * This large-scale mapping is critical for urban planning, precision agriculture, environmental monitoring (for example tracking deforestation or glacial retreat), and disaster management.<ref name="Matrack"/><ref name="Faro">{{cite web |url=https://www.faro.com/en/Resource-Library/Article/Past-Present-and-Future-of-Geospatial-Mapping |title=The Past, Present and Future of Geospatial Mapping |publisher=FARO |access-date=2025-10-23}}</ref><ref name="SurveyTransfer">{{cite web |url=https://surveytransfer.net/geospatial-applications/ |title=10 Key Industries Using Geospatial Applications |publisher=SurveyTransfer |access-date=2025-10-23}}</ref> | ||
* Projects like Google's AlphaEarth Foundations fuse vast quantities of satellite imagery, radar, and 3D laser mapping data into a unified digital representation of the planet, allowing scientists to track global changes with remarkable precision.<ref name="AlphaEarth">{{cite web |url=https://deepmind.google/discover/blog/alphaearth-foundations-helps-map-our-planet-in-unprecedented-detail/ |title=AlphaEarth Foundations helps map our planet in unprecedented detail |publisher=Google DeepMind |access-date=2025-10-23}}</ref> | * Projects like Google's AlphaEarth Foundations fuse vast quantities of satellite imagery, radar, and 3D laser mapping data into a unified digital representation of the planet, allowing scientists to track global changes with remarkable precision.<ref name="AlphaEarth">{{cite web |url=https://deepmind.google/discover/blog/alphaearth-foundations-helps-map-our-planet-in-unprecedented-detail/ |title=AlphaEarth Foundations helps map our planet in unprecedented detail |publisher=Google DeepMind |access-date=2025-10-23}}</ref> | ||
* Pokemon Go achieved unprecedented scale with 800+ million downloads and 600+ million active users, using Visual Positioning System with centimeter-level accuracy. Niantic built a Large Geospatial Model with over 50 million neural networks trained on location data comprising 150+ trillion parameters for planet-scale 3D mapping from pedestrian perspective.<ref name="niantic">{{cite web |url=https://nianticlabs.com/news/largegeospatialmodel |title=Large Geospatial Model |publisher=Niantic Labs |access-date=2025-10-27}}</ref> | * Pokemon Go achieved unprecedented scale with 800+ million downloads and 600+ million active users, using Visual Positioning System with centimeter-level accuracy. Niantic built a Large Geospatial Model with over 50 million neural networks trained on location data comprising 150+ trillion parameters for planet-scale 3D mapping from pedestrian perspective.<ref name="niantic">{{cite web |url=https://nianticlabs.com/news/largegeospatialmodel |title=Large Geospatial Model |publisher=Niantic Labs |access-date=2025-10-27}}</ref> | ||
| Line 363: | Line 363: | ||
The next major frontier for spatial mapping is the shift from purely geometric understanding (knowing ''where'' a surface is) to '''[[semantic understanding]]''' (knowing ''what'' a surface is).<ref name="SpatialAI"/><ref name="FutureDirections1">{{cite web |url=https://arxiv.org/html/2508.20477v1 |title=What is Spatial Computing? A Survey on the Foundations and State-of-the-Art |publisher=arXiv |access-date=2025-10-23}}</ref> This involves leveraging [[AI]] and [[machine learning]] algorithms to analyze the map data and automatically identify, classify, and label objects and architectural elements in real-time—for example, recognizing a surface as a "couch," an opening as a "door," or an object as a "chair."<ref name="MetaHelp"/><ref name="SpatialAI"/> | The next major frontier for spatial mapping is the shift from purely geometric understanding (knowing ''where'' a surface is) to '''[[semantic understanding]]''' (knowing ''what'' a surface is).<ref name="SpatialAI"/><ref name="FutureDirections1">{{cite web |url=https://arxiv.org/html/2508.20477v1 |title=What is Spatial Computing? A Survey on the Foundations and State-of-the-Art |publisher=arXiv |access-date=2025-10-23}}</ref> This involves leveraging [[AI]] and [[machine learning]] algorithms to analyze the map data and automatically identify, classify, and label objects and architectural elements in real-time—for example, recognizing a surface as a "couch," an opening as a "door," or an object as a "chair."<ref name="MetaHelp"/><ref name="SpatialAI"/> | ||
This capability, already emerging in platforms like Meta Quest's Scene API, will enable a new generation of intelligent and context-aware XR experiences. Virtual characters could realistically interact with the environment ( | This capability, already emerging in platforms like Meta Quest's Scene API, will enable a new generation of intelligent and context-aware XR experiences. Virtual characters could realistically interact with the environment (for example sitting on a recognized couch), applications could automatically adapt their UI to the user's specific room layout, and digital assistants could understand commands related to physical objects ("place the virtual screen on that wall").<ref name="FutureDirections1"/> | ||
=== Neural Rendering and AI-Powered Mapping === | === Neural Rendering and AI-Powered Mapping === | ||
| Line 435: | Line 435: | ||
<ref name="HoloLensYouTube">{{cite web |url=https://www.youtube.com/watch?v=zff2aQ1RaVo |title=HoloLens - What is Spatial Mapping? |publisher=Microsoft |access-date=2025-10-23}}</ref> | <ref name="HoloLensYouTube">{{cite web |url=https://www.youtube.com/watch?v=zff2aQ1RaVo |title=HoloLens - What is Spatial Mapping? |publisher=Microsoft |access-date=2025-10-23}}</ref> | ||
<ref name="SLAMSystems">{{cite web |url=https://www.sbg-systems.com/glossary/slam-simultaneous-localization-and-mapping/ |title=SLAM - Simultaneous localization and mapping |publisher=SBG Systems |access-date=2025-10-23}}</ref> | <ref name="SLAMSystems">{{cite web |url=https://www.sbg-systems.com/glossary/slam-simultaneous-localization-and-mapping/ |title=SLAM - Simultaneous localization and mapping |publisher=SBG Systems |access-date=2025-10-23}}</ref> | ||
<ref name="MilvusSensors">{{cite web |url=https://milvus.io/ai-quick-reference/what-sensors-eg-accelerometer-gyroscope-are-essential-in-ar-devices |title=What sensors ( | <ref name="MilvusSensors">{{cite web |url=https://milvus.io/ai-quick-reference/what-sensors-eg-accelerometer-gyroscope-are-essential-in-ar-devices |title=What sensors (for example accelerometer, gyroscope) are essential in AR devices? |publisher=Milvus |access-date=2025-10-23}}</ref> | ||
<ref name="MathWorksSLAM">{{cite web |url=https://www.mathworks.com/discovery/slam.html |title=What Is SLAM (Simultaneous Localization and Mapping)? |publisher=MathWorks |access-date=2025-10-23}}</ref> | <ref name="MathWorksSLAM">{{cite web |url=https://www.mathworks.com/discovery/slam.html |title=What Is SLAM (Simultaneous Localization and Mapping)? |publisher=MathWorks |access-date=2025-10-23}}</ref> | ||
<ref name="PressbooksSensors">{{cite web |url=https://pressbooks.pub/augmentedrealitymarketing/chapter/sensors-for-arvr/ |title=Sensors for AR/VR |publisher=Pressbooks |access-date=2025-10-23}}</ref> | <ref name="PressbooksSensors">{{cite web |url=https://pressbooks.pub/augmentedrealitymarketing/chapter/sensors-for-arvr/ |title=Sensors for AR/VR |publisher=Pressbooks |access-date=2025-10-23}}</ref> | ||