Visual positioning system

A visual positioning system (VPS) determines the position and orientation of a camera-equipped device by matching what the camera sees against a previously built map of visual features, instead of relying only on satellite signals. In augmented reality (AR), a VPS lets a phone or headset work out where it is and which way it faces with enough precision to attach virtual content to a specific street corner, building or room, and to show that content in the same place to other users and on later visits. Google describes its own VPS as matching "recognizable parts of the user's environment" against a 3D point cloud built from Street View imagery and then using "computer vision algorithms" to "compute the position and orientation of the device".

Major systems include Google's VPS, exposed to developers through the ARCore Geospatial API since May 2022, the location anchors in Apple's ARKit (introduced with ARKit 4 in 2020), and the VPS developed by Niantic for Lightship in 2022 and now run by the spun-off company Niantic Spatial. The technique is related to simultaneous localization and mapping (SLAM), which builds a map during a session, and to the AR Cloud idea of a shared, persistent map of the world.

Reviewed 27 September 2026. Checked every cited source against the lead, history, systems table, applications and research sections (Google VPS and Geospatial API, ARKit location anchors, Niantic Lightship and Niantic Spatial VPS, three arXiv papers). About review dates.

How it works

A VPS has two stages. In the mapping stage, the operator collects images whose locations are known and extracts distinctive visual features from them. Google software engineer Tilman Reinhardt wrote in 2019 that Google's VPS "first creates a map by taking a series of images which have a known location and analyzing them for key visual features, such as the outline of buildings or bridges, to create a large scale and fast searchable index". In the localization stage, the device captures camera frames and compares them with that index. Niantic Spatial describes its service in the same terms: the device sends "camera frames and basic sensor data to a localization service", which "matches the observed scene against available visual maps and other location signals" and returns a pose, meaning the device's position and orientation.

The result can be a full six degrees of freedom (6DoF) pose (three axes of position and three of rotation) or a reduced 3DoF result of position and heading. Google's Geospatial API combines VPS results with GPS and the device's other sensors, and it can also be used in areas without VPS coverage, where GPS alone may be enough for an accurate pose outdoors with few or no overhead obstructions.

The main motivation is the weakness of satellite positioning in cities. Reinhardt noted that in dense urban areas GPS suffers from "low visibility to the sky and signals reflecting off of buildings", which can place a user "on the wrong side of the street, or even a few blocks away", and that phone compass readings "can be inaccurate by up to 180 degrees". Google's VPS uses machine learning to decide which features to trust, "prioritizing features that are likely to be permanent parts of the scene and ignoring things like trees, dynamic light movement, and construction". Changes in lighting, weather and season remain the main technical difficulty; Reinhardt listed "late at night, in a snowstorm, or in torrential downpour" as conditions still to be solved.

Systems differ in where the matching runs. Apple states that ARKit's visual localization runs on the device, with "no processing in the cloud and no images sent back to Apple". Niantic Spatial's service, by contrast, receives camera frames from the device.

History

Google publicly introduced a "Visual Positioning Service" at its I/O developer conference on 17 May 2017. Clay Bavor, then Google's vice president of virtual reality, presented it as an indoor counterpart to GPS for phones running the Tango AR platform, able to find a device's position "within a few centimetres" by recognizing visual features of a building. Bavor's example was finding a particular screwdriver in a Lowe's store, summed up as "GPS can get you to the door, VPS can get you the exact item"; Google said the system was already working in partner museums and selected Lowe's stores.

Google then rebuilt VPS around Street View for outdoor navigation in Google Maps. Reinhardt's February 2019 description of this "global localization" approach said it drew on Street View data "from over 93 countries". The resulting Live View feature overlays walking directions on the camera image; Google moved it from alpha testing to a public beta in August 2019 for ARCore-compatible Android phones and ARKit-compatible iPhones in areas with Street View coverage.

Apple announced location anchors as part of ARKit 4 in June 2020. ARKit localizes the device against a "localization map that contains feature points of the surrounding area that can be seen from the street", together with high-resolution Apple Maps data and on-device machine learning. At launch the feature required an iPhone or iPad with an A12 Bionic chip or newer and GPS, and it covered the San Francisco Bay Area, New York, Los Angeles, Chicago and Miami.

On 11 May 2022 Google opened its VPS to developers through the ARCore Geospatial API for Android and iOS. Google said the service uses machine learning to "extract trillions of 3D points from Street View images", computes a device's position and orientation "in less than a second", and works "in over 87 countries" without the developer having to visit or scan the location. The same technology had powered Live View "since 2019". Later that month Niantic released Lightship VPS with about 30,000 public locations, concentrated in San Francisco, London, Tokyo, Los Angeles, New York City and Seattle, and claimed "centimeter-level" accuracy from a single camera frame. Unlike Google's Street View-based map, Niantic's maps were built from phone scans collected through its Wayfarer app and a surveyor program in the six launch cities. In September 2022 Niantic extended the service to browser-based AR through 8th Wall, with more than 100,000 VPS-activated locations at launch.

At I/O in May 2023 Google added features on top of the Geospatial API: the Streetscape Geometry API, which "provides a 3D mesh within a 100m radius of the user's mobile device location"; Rooftop anchors, which place content on building tops; Geospatial Depth, which combines the device's own depth measurements with Streetscape Geometry data to provide depth "up to 65m" (Android only); and the Geospatial Creator authoring tool for Unity and Adobe Aero.

In May 2025, after Niantic sold its games business to Scopely, its geospatial technology moved to a new company, Niantic Spatial, led by CEO John Hanke and CTO Brian McClendon and started with US$250 million in funding. The company said its VPS and a planned "Large Geospatial Model" were built on "over 30 billion posed images from millions of locations around the world". In June 2025 Snap Inc. invested in Niantic Spatial and announced a multi-year partnership to bring its VPS to Lens Studio, Snapchat and Spectacles. When 8th Wall shut down its hosted services and released its engine to developers in stages in early 2026, VPS was among the capabilities not included. On 7 April 2026 Niantic Spatial released VPS 2.0 together with its Scaniverse capture platform. The company says VPS 2.0 gives "6DoF localization with near-centimeter accuracy" in areas mapped with Scaniverse and, without prior scanning, can augment GPS to provide 3DoF positioning; the launch material was aimed mainly at robotics, industrial sites and GPS-degraded environments.

Major systems

System	Operator	Map source	Developer access	Notes
Visual Positioning System (Geospatial API)	Google	Street View imagery	ARCore Geospatial API for Android, Android NDK, iOS, Unity (AR Foundation) and Unreal Engine	Opened to developers 11 May 2022 in over 87 countries; also powers Google Maps Live View
Location anchors (geotracking)	Apple	Street-level localization map and Apple Maps data	ARKit 4 and later	Launched in 2020 in five US metropolitan areas; localization runs on the device
Lightship VPS, later Niantic Spatial VPS	Niantic, then Niantic Spatial	User and surveyor phone scans	Lightship ARDK, later Niantic Spatial SDK; integration with Lens Studio and Spectacles announced in 2025	About 30,000 locations at the May 2022 launch; VPS 2.0 released April 2026

Applications in VR and AR

The most visible consumer use is AR pedestrian navigation. Google Maps Live View draws arrows and street markers over the camera image after VPS has fixed the phone's position and heading. A VPS also gives spatial anchors a real-world address: an ARCore geospatial anchor can be placed by latitude, longitude and altitude (WGS84 anchors), relative to the ground (terrain anchors) or relative to a building's roof (rooftop anchors). Because content is tied to shared coordinates rather than to one device's session, it can persist between visits and be seen by several people at once. Road to VR described Niantic's VPS as the piece that lets multiple devices localize within one shared map for synchronized multiplayer and persistent content, which GPS is too imprecise to support.

Google's launch partners for the Geospatial API in 2022 included the micromobility companies Bird, Lime and WeMo, as well as the NBA, Snap, Lyft, Telstra and Accenture (for Marvel Stadium), DOCOMO and Curiosity. Snap and Niantic Spatial presented their 2025 partnership in terms of AR glasses: Lens developers would be able to anchor experiences "to millions of real-world locations with centimeter-level precision", and Snapchat users would help extend the underlying map. Niantic Spatial's 2026 positioning also targets machines, listing robots, drones, vehicles, headsets and field teams as clients of its localization service.

Research

In the computer vision literature the underlying problem is usually called visual localization or camera relocalization. Sattler and colleagues' 2018 benchmark tested state-of-the-art methods under day and night, weather and seasonal changes and concluded that "long-term localization is far from solved". PoseNet (Kendall, Grimes and Cipolla, 2015) showed that a convolutional neural network could regress a camera's 6DoF pose directly from a single RGB image in 5 ms, with accuracy of about 2 m and 6 degrees outdoors and 0.5 m and 10 degrees indoors. Sarlin, Cadena, Siegwart and Dymczyk's hierarchical method (CVPR 2019) uses a coarse-to-fine structure: a global image retrieval step first proposes candidate places, and local features are matched only within those candidates to compute the 6DoF pose.

See also

• AR Cloud
• Simultaneous localization and mapping
• Spatial anchors
• Visual-inertial odometry
• Inside-out tracking
• ARCore
• ARKit
• Niantic Lightship
• Computer vision