Toggle search
Toggle menu
5.1K
5.9K
229
50.3K
VR & AR Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Loop closure

From VR & AR Wiki
More actions

Loop closure (also called loop closing) is the part of simultaneous localization and mapping (SLAM) in which a robot, camera or headset recognizes that it has returned to a place it has already mapped, checks that the match is real, and uses the resulting constraint to correct the error that has built up in its estimated trajectory and map. The recognition step is usually called loop closure detection and is a form of place recognition.[1][2]

Loop closure is what separates full SLAM from odometry. In their 2016 survey of the field, Cadena and colleagues wrote that "if we sacrifice loop closures, SLAM reduces to odometry", and described visual-inertial navigation as a reduced SLAM system in which the loop closure (or place recognition) module is disabled.[1] Because a single wrong loop closure can corrupt the whole map, loop closure systems are designed and evaluated with a strong bias against false positives.[2][1]

In virtual reality (VR) and augmented reality (AR), loop closure matters wherever a tracking system keeps a map of the user's surroundings and the user moves away and comes back. Research on smartphone AR has linked it directly to whether virtual objects stay where they were placed.[3] Meta's documentation for its Project Aria research glasses distinguishes "open loop" odometry output from "closed loop" trajectories that are bundle adjusted with detected loop closures.[4] Public documentation for consumer platforms such as ARCore, ARKit and Oculus Insight describes drift correction, place recognition or saved maps, but not the loop closure algorithms themselves.

Reviewed 12 October 2026. Checked every claim against the cited papers (Cadena 2016, Tsintotas 2022, Durrant-Whyte and Bailey 2006, FAB-MAP, DBoW2, ORB-SLAM, ORB-SLAM3, VINS-Mono, RTAB-Map, Latif 2013, Grisetti 2010, PTAM, NetVLAD, Lowry 2016, Scargill 2021), their DOI metadata via Crossref, and the Meta, Google, Apple and Microsoft documentation pages. About review dates.

Definition

A device that estimates its motion only by integrating successive measurements (wheel encoders, cameras, an Inertial measurement unit, or a combination) accumulates error, so its position estimate drifts. Cadena et al. note that wheel odometry becomes unusable after a few meters, while modern visual and inertial odometry has much smaller drift but still drifts.[1] In the same survey they describe a robot that ignores loop closures as interpreting the world as an "infinite corridor" in which it keeps exploring new areas; a loop closure event tells it that the corridor keeps intersecting itself, so the robot learns the real topology of the environment and can find shortcuts between places.[1]

Loop closure was originally discussed as "the revisiting problem", the term used by Stewart, Ko, Fox and Konolige: whether the data just captured by the robot matches any data collected earlier at a previously visited place.[2] Hugh Durrant-Whyte and Tim Bailey's 2006 SLAM tutorial calls the loop-closure problem, "when a robot returns to reobserve landmarks after a large traverse", especially difficult, and notes that the standard EKF-SLAM formulation is especially fragile to incorrect association of observations to landmarks.[5] The same tutorial records that in the years after the acronym SLAM was first presented in 1995, research on what was then also called concurrent mapping and localization focused on computational efficiency and on "data association, or loop closure".[5]

The authors of ORB-SLAM3 sort the data association in a SLAM system into three kinds. Short-term association matches map elements seen in the last few seconds, which is all that most odometry systems use. Mid-term association matches nearby map elements whose accumulated drift is still small. Long-term association matches observations with elements of previously visited areas through place recognition "regardless of the accumulated drift": this covers loop detection, merging with a separate map, and relocalization after tracking is lost. In their account, long-term matching is what lets a system reset drift and correct the map with pose-graph optimization or bundle adjustment.[6]

In the common split of a SLAM system into a front end, which turns raw sensor data into constraints, and a back end, which estimates the trajectory and map from them, loop closure detection belongs to the front end. Rejection of false-positive loop closures has been treated as a front-end task (verification) and, in more recent work, also as a back-end task.[2][1]

How it works

A loop closure pipeline has three stages: finding candidate places that may match the current observation, verifying a candidate geometrically, and correcting the trajectory and map.[1][2]

Detection

Comparing the features of every new image with every feature seen before quickly becomes impractical as the map grows. Bag-of-words models avoid this by quantizing the feature space into "visual words", which allows more efficient search, and arranging the words in hierarchical vocabulary trees makes lookup efficient in large datasets.[1] In such systems an inverse (inverted) index is used for fast retrieval of earlier images that may be similar to the current one.[7] (See Bag of visual words.) Matching whole images in this way is called the image-to-image or appearance-based approach. A 2009 study by Brian Williams and colleagues compared one implementation each of three families of loop closure detection for monocular SLAM: map-to-map, image-to-image and image-to-map.[8] Gálvez-López and Tardós summarized its conclusion as follows: map-to-image methods performed well in small environments, but image-to-image methods such as FAB-MAP scaled better in large ones.[7]

Published systems include:

  • FAB-MAP (Mark Cummins and Paul Newman, 2008) treats place recognition probabilistically. It learns a generative model of place appearance, using a Chow Liu tree to capture which visual words tend to occur together, so that identical but indistinctive observations receive a low probability of coming from the same place. It can also decide that an observation comes from a new, unseen place, and its complexity is linear in the number of places in the map.[9]
  • DBoW2 (Dorian Gálvez-López and Juan D. Tardós, 2012) builds a vocabulary tree over binary BRIEF descriptors computed at FAST keypoints, adds a direct index to speed up the point correspondences needed for geometric checking, and verifies candidates for temporal and geometric consistency. The authors reported no false positives on very different datasets with the same vocabulary and settings, and 22 ms per frame, including feature extraction, on a sequence of 26,300 images.[7] DBoW2 was later used for place recognition in ORB-SLAM, ORB-SLAM3 and VINS-Mono.[10][6][11]
  • RTAB-Map (Mathieu Labbé and François Michaud) started as an appearance-based loop closure detector with memory management for large-scale, long-term operation. It quantizes visual features into an incremental vocabulary, scores each new location against the locations in its working memory with a TF-IDF approach, and feeds the scores into a Bayes filter; a loop closure is accepted when a hypothesis passes a fixed threshold and a transformation between the two locations can be computed.[12] (See RTAB-Map.)

Bag-of-words methods work well when a place is revisited under similar conditions but fail under severe illumination change, because the visual words no longer match. Later methods match sequences of images rather than single frames, combine several appearances of a place, or add spatial information.[1] The 2016 survey of visual place recognition by Stephanie Lowry and colleagues reviews place recognition methods for environments with and without appearance change, and the role of place recognition in pose graph SLAM, where it supplies loop closure candidates.[13]

Features learned by convolutional neural networks have also been used for visual loop closure detection, especially under large changes in appearance; Tsintotas et al. note that their computational cost often requires a GPU and limits real-time use.[2] One example is NetVLAD (Relja Arandjelović and colleagues, CVPR 2016), a trainable layer that aggregates convolutional features into a single image descriptor and was trained for place recognition on Google Street View Time Machine images.[14][2]

Loop closure is not limited to cameras. Laser-based SLAM front ends have used features extracted from 2D laser scans, and a laser loop closure can be checked by how well the current scan matches the existing map.[1] RTAB-Map adds a separate "proximity detection" step that uses laser scans to link nearby nodes, for example when a robot traverses the same corridor again in a different direction and the camera cannot be used to find loop closures.[12]

Verification

Appearance alone is not proof that two observations show the same place, because different places can look alike. Candidates therefore go through geometric verification; in vision systems this is commonly done with RANSAC, which looks for a geometric transformation consistent with enough feature matches.[1][7]

ORB-SLAM (2015) is a detailed published example. For each new keyframe it queries a DBoW2-based recognition database, discards candidates that score lower than the least similar of the keyframe's neighbors in the covisibility graph, and accepts a loop candidate only after three consecutive consistent detections. It then computes a similarity transformation between the two keyframes from 3D-to-3D point matches, using RANSAC and Horn's method; this transformation both measures the drift accumulated around the loop and serves as the geometric validation.[10] According to the ORB-SLAM3 paper, raw DBoW2 queries using only the first candidate reach precision and recall of roughly 50 to 80 percent, and the temporal and geometric checks move the working point to 100 percent precision with 30 to 40 percent recall. Because the temporal check delays place recognition by at least three keyframes, ORB-SLAM3 replaced it: candidate keyframes are first checked for geometric consistency and then for local consistency with three covisible keyframes, most of which are already in the map. Its authors report that this increases recall at a slightly higher computational cost.[6]

Correction

In graph-based SLAM, the robot's poses are nodes and the measured relations between them are edges. Odometry links consecutive poses; when the robot re-enters a known area and the current observation is matched to an earlier one, the system adds a new edge labeled with the relative transformation between them. Optimizing the graph then finds the configuration of poses most consistent with all the edges, which spreads the correction around the loop.[15] RTAB-Map's graph has the same structure, with separate neighbor, loop closure and proximity links; adding a loop closure link triggers an optimization that propagates the error through the graph.[12]

The number of quantities that can drift, and the transformation used to close a loop, depend on the sensors:

System Sensor setup Loop correction described in the paper
ORB-SLAM Single camera (monocular) The map can drift in seven degrees of freedom (three translations, three rotations and scale), so the loop is closed with a similarity transformation and a pose graph optimization over similarity constraints[10]
VINS-Mono Single camera plus IMU With the IMU, roll and pitch are observable, so drift accumulates only in x, y, z and yaw, and a 4-DOF pose graph optimization is used[11]
ORB-SLAM3 Monocular, stereo or RGB-D cameras; monocular and stereo also with an IMU The aligning transformation is a similarity (Sim(3)) in pure monocular mode or while a monocular-inertial map is still immature, and a rigid SE(3) transformation otherwise; with a mature visual-inertial map, a candidate must also pass a check of pitch and roll against the gravity direction[6]

After computing the correction, ORB-SLAM aligns both sides of the loop, fuses duplicated map points, and runs a pose graph optimization over its "Essential Graph", a sparse subgraph of the keyframe covisibility graph. ORB-SLAM3 also launches a full bundle adjustment in a separate thread after a loop correction so the refinement does not affect real-time tracking.[10][6]

Relocalization and map merging

Loop closure is related to, but distinct from, relocalization. Relocalization, also known as the "kidnapped robot problem", recovers the device's pose within an existing map after tracking has failed or the device has been moved without knowing how. Loop closure, by contrast, adds a connection between the current location and an earlier one that corrects the map's accumulated error.[2] In practice the same place recognition machinery often serves both. ORB-SLAM3 keeps a single DBoW2 database for relocalization, loop closing and map merging across its "Atlas" of maps: a match inside the active map triggers a loop correction, and a match with a different stored map merges the two maps.[6] VINS-Mono likewise uses its loop detection module for relocalization and then runs a global pose graph optimization.[11]

False loop closures

The main risk in loop closure is perceptual aliasing: distinct places that look similar to the sensor. Cadena et al. write that, despite front-end checks, wrong loop closures reaching the back end are "unavoidable" in the presence of perceptual aliasing and can severely corrupt the estimate.[1] Tsintotas et al. note that a single false positive can in many cases cause total failure of SLAM, which is why the most common benchmark for loop closure detectors is recall at 100 percent precision.[2]

Two lines of research address this. Front-end methods make detection more conservative (temporal consistency checks, geometric verification, probabilistic models of aliasing such as FAB-MAP's).[9][7] Robust back-end methods examine loop closure constraints during or before optimization. Yasir Latif, César Cadena and José Neira's RRR algorithm (2013) seeks consensus among all available loop closures and can detect and remove a past loop closing decision that new evidence shows was wrong. Their paper compares it with two contemporary approaches: Niko Sünderhauf and Peter Protzel's "switchable constraints", which penalize loop closure links that deviate from the constraints they suggest during graph optimization, and Olson and Agarwal's max-mixtures.[16][1] Cadena et al. also point out that metric information in the map makes place recognition more robust, because the map indicates where loop closures are plausible and allows spurious ones to be discarded.[1]

Applications in VR and AR

Research systems

Parallel Tracking and Mapping (PTAM), presented by Georg Klein and David Murray at ISMAR 2007, was designed to track a hand-held camera in a small AR workspace. Its authors wrote that the system was "not designed to close large loops in the SLAM sense"; they traced the limitation to the tracker's M-estimator not being informed of feature-map uncertainties and added: "In practical AR use, this is not an issue."[17] The ORB-SLAM paper later noted that PTAM's map points were useful for tracking but not for place recognition, and that PTAM did not detect large loops.[10] VINS-Mono (2018), from the Hong Kong University of Science and Technology, combines visual-inertial odometry with loop detection and 4-DOF pose graph optimization; its authors reported applying it to small-scale AR scenarios and released an iOS version alongside the PC version.[11]

Products and platforms

Meta's 2019 engineering description of Oculus Insight, the tracking system that Meta said powered both the original Oculus Quest and the Rift S, lists place recognition among its computer vision components alongside visual-inertial mapping and geometry reconstruction, and says that landmarks observed repeatedly let Insight compensate for drift. The article does not describe the loop closure step itself.[18]

Meta's Machine Perception Services (MPS), offered to Aria Research Kit partners as cloud post-processing of Project Aria recordings,[19] output two trajectories at the 1 kHz IMU rate. The "open loop" trajectory is odometry with good local accuracy but drift that grows with time and distance; the "closed loop" trajectory is fully bundle adjusted with detected loop closures, reducing that drift. The documentation warns that, because of the loop closure correction, the closed loop trajectory's relative and local accuracy within a short time span (seconds) might be worse.[4]

For phone AR, Google's ARCore documentation says ARCore uses SLAM and that the poses of tracked objects can change "as ARCore improves its understanding of its own position and its environment", which is why virtual content should be attached to anchors (Spatial anchors).[20] Apple's ARKit lets an app save its world map and restore it in a later session, so the same virtual content can be placed at the same positions if the app launches in the same physical environment; two devices can also share a world map.[21] Microsoft's World Locking Tools documentation for Microsoft HoloLens describes the underlying problem in plain terms: when the head travels from point A to point B and back, the tracking system is slightly wrong about the distance traveled each way, and the toolkit can recognize from the physical features around point A that the head is back and correct its coordinates.[22] None of these three pages describes the loop closure algorithm used.

Measured effect on virtual content

Tim Scargill, Jiasi Chen and Maria Gorlatova measured how far virtual objects ("holograms") moved on an iPhone 11 running ARKit 4 and on a Samsung Galaxy Note 10+ and a Nokia 7.1 running ARCore 1.23, across six user actions. In the "walk away" action, the user walked about 7 m away from a placed object and returned. Mean drift was 25.1 cm on ARKit compared with an average of 4.8 cm on ARCore, and ARCore did better in over 80 percent of scenes. The authors wrote that after the user returned, ARCore was "clearly able to achieve accurate loop closure (recognizing a previously visited place and adjusting the pose estimate accordingly) more reliably than ARKit".[3] The results apply to the software versions tested in 2021.

See also

References

  1. ↑ 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.10 1.11 1.12 1.13 Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, Jose Neira, Ian Reid, John J. Leonard (2016-12). "Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age". IEEE Transactions on Robotics, vol. 32, no. 6, pp. 1309-1332. doi:10.1109/TRO.2016.2624754. https://doi.org/10.1109/TRO.2016.2624754. Retrieved 2026-10-12.
  2. ↑ 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 Konstantinos A. Tsintotas, Loukas Bampis, Antonios Gasteratos (2022-11). "The Revisiting Problem in Simultaneous Localization and Mapping: A Survey on Visual Loop Closure Detection". IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 11, pp. 19929-19953. doi:10.1109/TITS.2022.3175656. https://doi.org/10.1109/TITS.2022.3175656. Retrieved 2026-10-12.
  3. ↑ 3.0 3.1 Tim Scargill, Jiasi Chen, Maria Gorlatova (2021-09-29). "Here To Stay: Measuring Hologram Stability in Markerless Smartphone Augmented Reality". arXiv preprint 2109.14757. https://arxiv.org/abs/2109.14757. Retrieved 2026-10-12.
  4. ↑ 4.0 4.1 "MPS output - Trajectory". Project Aria Tools documentation. Meta. https://facebookresearch.github.io/projectaria_tools/docs/data_formats/mps/slam/mps_trajectory. Retrieved 2026-10-12.
  5. ↑ 5.0 5.1 Hugh Durrant-Whyte, Tim Bailey (2006-06). "Simultaneous localization and mapping: part I". IEEE Robotics & Automation Magazine, vol. 13, no. 2, pp. 99-110. doi:10.1109/MRA.2006.1638022. https://doi.org/10.1109/MRA.2006.1638022. Retrieved 2026-10-12.
  6. ↑ 6.0 6.1 6.2 6.3 6.4 6.5 Carlos Campos, Richard Elvira, Juan J. Gomez Rodriguez, Jose M. M. Montiel, Juan D. Tardos (2021-12). "ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial, and Multimap SLAM". IEEE Transactions on Robotics, vol. 37, no. 6, pp. 1874-1890. doi:10.1109/TRO.2021.3075644. https://doi.org/10.1109/TRO.2021.3075644. Retrieved 2026-10-12.
  7. ↑ 7.0 7.1 7.2 7.3 7.4 Dorian Gálvez-López, Juan D. Tardós (2012-10). "Bags of Binary Words for Fast Place Recognition in Image Sequences". IEEE Transactions on Robotics, vol. 28, no. 5, pp. 1188-1197. doi:10.1109/TRO.2012.2197158. https://doi.org/10.1109/TRO.2012.2197158. Retrieved 2026-10-12.
  8. ↑ Brian Williams, Mark Cummins, José Neira, Paul Newman, Ian Reid, Juan Tardós (2009-12). "A comparison of loop closing techniques in monocular SLAM". Robotics and Autonomous Systems, vol. 57, no. 12, pp. 1188-1197. doi:10.1016/j.robot.2009.06.010. https://doi.org/10.1016/j.robot.2009.06.010. Retrieved 2026-10-12.
  9. ↑ 9.0 9.1 Mark Cummins, Paul Newman (2008-06). "FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance". The International Journal of Robotics Research, vol. 27, no. 6, pp. 647-665. doi:10.1177/0278364908090961. https://doi.org/10.1177/0278364908090961. Retrieved 2026-10-12.
  10. ↑ 10.0 10.1 10.2 10.3 10.4 Raul Mur-Artal, J. M. M. Montiel, Juan D. Tardos (2015-10). "ORB-SLAM: A Versatile and Accurate Monocular SLAM System". IEEE Transactions on Robotics, vol. 31, no. 5, pp. 1147-1163. doi:10.1109/TRO.2015.2463671. https://doi.org/10.1109/TRO.2015.2463671. Retrieved 2026-10-12.
  11. ↑ 11.0 11.1 11.2 11.3 Tong Qin, Peiliang Li, Shaojie Shen (2018-08). "VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator". IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1004-1020. doi:10.1109/TRO.2018.2853729. https://doi.org/10.1109/TRO.2018.2853729. Retrieved 2026-10-12.
  12. ↑ 12.0 12.1 12.2 Mathieu Labbé, François Michaud (2019-03). "RTAB-Map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation". Journal of Field Robotics, vol. 36, no. 2, pp. 416-446. doi:10.1002/rob.21831. https://doi.org/10.1002/rob.21831. Retrieved 2026-10-12.
  13. ↑ Stephanie Lowry, Niko Sünderhauf, Paul Newman, John J. Leonard, David Cox, Peter Corke, Michael J. Milford (2016-02). "Visual Place Recognition: A Survey". IEEE Transactions on Robotics, vol. 32, no. 1, pp. 1-19. doi:10.1109/TRO.2015.2496823. https://doi.org/10.1109/TRO.2015.2496823. Retrieved 2026-10-12.
  14. ↑ Relja Arandjelović, Petr Gronat, Akihiko Torii, Tomas Pajdla, Josef Sivic (2016-06). "NetVLAD: CNN Architecture for Weakly Supervised Place Recognition". 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5297-5307. doi:10.1109/CVPR.2016.572. https://doi.org/10.1109/CVPR.2016.572. Retrieved 2026-10-12.
  15. ↑ Giorgio Grisetti, Rainer Kümmerle, Cyrill Stachniss, Wolfram Burgard (2010). "A Tutorial on Graph-Based SLAM". IEEE Intelligent Transportation Systems Magazine, vol. 2, no. 4, pp. 31-43. doi:10.1109/MITS.2010.939925. https://doi.org/10.1109/MITS.2010.939925. Retrieved 2026-10-12.
  16. ↑ Yasir Latif, César Cadena, José Neira (2013-12). "Robust loop closing over time for pose graph SLAM". The International Journal of Robotics Research, vol. 32, no. 14, pp. 1611-1626. doi:10.1177/0278364913498910. https://doi.org/10.1177/0278364913498910. Retrieved 2026-10-12.
  17. ↑ Georg Klein, David Murray (2007-11). "Parallel Tracking and Mapping for Small AR Workspaces". 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2007), pp. 225-234. doi:10.1109/ISMAR.2007.4538852. https://doi.org/10.1109/ISMAR.2007.4538852. Retrieved 2026-10-12.
  18. ↑ Joel Hesch, Anna Kozminski, Oskar Linde (2019-08-22). "Powered by AI: Oculus Insight". Meta AI blog. Meta. https://ai.meta.com/blog/powered-by-ai-oculus-insight/. Retrieved 2026-10-12.
  19. ↑ "Machine Perception Services (MPS)". Project Aria Tools documentation. Meta. https://facebookresearch.github.io/projectaria_tools/docs/ARK/mps. Retrieved 2026-10-12.
  20. ↑ "Fundamental concepts". ARCore, Google for Developers. Google. https://developers.google.com/ar/develop/fundamentals. Retrieved 2026-10-12.
  21. ↑ "ARWorldMap". Apple Developer Documentation. Apple. https://developer.apple.com/documentation/arkit/arworldmap. Retrieved 2026-10-12.
  22. ↑ "The basic idea of World Locking Tools". Microsoft Learn (archived). Microsoft. 2021-10-06. https://learn.microsoft.com/en-us/previous-versions/mixed-reality/world-locking-tools/documentation/concepts/basicconcepts. Retrieved 2026-10-12.
Retrieved from "https://vrarwiki.com/index.php?title=Loop_closure&oldid=49937"