Outside-in tracking

This page is a stub, please expand it if you have more information.

See also: Markerless outside-in tracking and Positional tracking

An Oculus Rift CV1 external sensor, the infrared camera used for outside-in tracking

Introduction

Figure 1. Outside-in tracking (Image: www.wareable.com)

Figure 2. Inside-out vs. outside-in tracking (Image: Ishii, 2010)

Outside-in tracking is a form of positional tracking and, generally, a method of optical tracking. When referring to virtual reality (VR), tracking is the process of tracing the scene coordinates of moving objects in real time, such as head-mounted displays (HMDs) or motion controller peripherals.

Outside-in VR tracking uses cameras or other sensors placed in a stationary location and oriented towards the tracked object (for example a headset) that moves freely around a designated area defined by the intersecting visual ranges of the cameras (Figure 1). The object is therefore observed from the outside by the fixed tracking device. Usually, the tracked object carries a known set of fiducial markers that are used to calculate its position relative to the sensors. While this type of positional tracking can be achieved using the visible light spectrum, it is common to use infra-red (IR) markers and cameras that detect that type of light.

The accuracy and performance of outside-in VR tracking depend on factors such as the quality of the optical sensors, the tracking markers and targets, processing power, and the tracking algorithms, all of which can vary greatly.

Outside-in tracking using markers is a well-developed and researched technology. A group of researchers (Pustka et al., 2012) built a positional tracking system of this kind using only unmodified off-the-shelf mobile phones. An early two-camera tracking system using CCD cameras and LED beacons was described by Madritsch and Gervautz in 1996, and a system that used synchronized IR cameras, able to distinguish 6D targets, was described by Dorfmüller in 1999.

An outside-in tracking system needs room calibration after the cameras or sensors are placed, and the data acquired by the system is processed on a computer. Besides its application in VR, this type of tracking is used in motion capture.

Contrast with inside-out tracking

Outside-in tracking functions as the inverse of inside-out tracking (Figure 2). In outside-in tracking the sensors are placed in a stationary location and observe markers carried by the moving HMD or controllers; in inside-out tracking the sensors are placed on the headset and look outward, either at fiducial markers fixed in the environment or, more commonly in modern systems, at natural features of the surroundings. A practical consequence of placing the cameras outside is that markers can be distributed over several sides of the headset, so the system can keep tracking it through large rotations, provided the markers remain visible to at least one camera.

A second practical difference concerns where the pose is computed. In outside-in systems the heavy work of identifying markers and reconstructing position can run on a separate stationary computer, which keeps the load off the worn device. Inside-out systems must run their computer vision on the headset itself, which raises the headset's computational and thermal budget but removes the fixed external hardware.

How marker-based optical outside-in tracking works

In a marker-based optical outside-in system, one or more fixed cameras observe markers attached to the tracked device. The markers are either active (light-emitting, typically IR LEDs that blink in a known pattern) or passive (retroreflective spheres illuminated by IR emitters mounted around each camera, which reflect the light back to the sensor). Because the geometric arrangement of the markers on the device is known in advance, the system can identify the markers in each camera image, measure their two-dimensional positions, and reconstruct the device's three-dimensional position and orientation. Using two or more cameras with overlapping views allows three-dimensional reconstruction through stereo vision and the epipolar constraint; the Ribo et al. stereo rig, for example, used progressive-scan CCD cameras running at 30 Hz and tracked up to 25 independent targets. Adding cameras and placing them around the play area extends the tracked volume and reduces the chance that the user's body blocks the line of sight to the markers.

A recurring problem in active-marker systems is the correspondence, or matching, step: deciding which detected light blob in a camera image belongs to which physical LED on the device. Meta describes this for Constellation as the inverse of the perspective-n-point problem, solved on every frame with probabilistic matching that uses either an exhaustive search when no prior pose is available or a smaller search window seeded by the previous frame's pose. Knowing the LED layout in advance, and using multiple calibrated cameras, lets the system stay locked on even when only a few markers are visible.

A different optical approach, used by Valve's Lighthouse system, reverses the usual camera-and-marker arrangement. The fixed base stations contain no cameras; instead they sweep the room with structured infra-red laser planes, and photodiodes on the headset and controllers detect each sweep. Because the timing of the sweeps is known, each sensor can compute the angle from itself to a base station, and the device fuses these angles with an onboard inertial measurement unit to recover its own pose. The system is still outside-in because the position-defining reference (the base stations) is external and stationary.

Devices using outside-in tracking

The Oculus Rift CV1, released on 28 March 2016, used outside-in tracking through the Constellation system. The headset and the Oculus Touch controllers carry IR LEDs hidden under their surfaces that blink in a defined pattern, and one or more external IR sensors (the Oculus Sensor) observe these LEDs to compute pose. A single sensor was sufficient for seated head tracking, two forward-facing sensors were recommended once the Touch controllers shipped, and adding a third sensor improved tracking further. Production of the original Rift ended in March 2019, when Meta (then Facebook) replaced it with the inside-out tracked Oculus Rift S.

PlayStation VR, also released in 2016, used outside-in tracking with the PlayStation Camera. The headset carries nine positional LEDs on its surface, and the external camera tracks these LEDs, along with the illuminated spheres of the PlayStation Move controllers, to follow head and controller movement. Unlike Constellation, PlayStation VR tracks in the visible light spectrum rather than infra-red. Sony's successor headset, PlayStation VR2 (2023), abandoned this external-camera method in favour of inside-out tracking with cameras built into the headset, mirroring the wider industry move away from outside-in tracking on consumer hardware.

The HTC Link mobile headset used the Neon tracking system from Ximmerse, an outside-in system in which a small external stereo camera tracks visible-light marker orbs on the headset and on each controller, producing a front-facing tracking volume.

Valve's Lighthouse base stations remain the most widely used outside-in system on PC VR. They are required by the Valve Index, the HTC Vive family, Pimax headsets such as the Crystal, and the Bigscreen Beyond, among others; the Beyond ships without base stations and relies on a buyer's existing SteamVR setup. The second-generation Valve Index Base Station 2.0 sweeps its lasers 100 times a second, has a field of view of about 160 by 115 degrees and a range of around 7 m, and up to four of them can be combined to cover a play area of up to roughly 10 m by 10 m. Valve opened the underlying SteamVR Tracking technology to third-party hardware makers on 4 August 2016 on a royalty-free basis, which let companies build their own headsets and trackers around Lighthouse without paying Valve a per-unit fee. Body trackers built on the same system, such as the HTC Vive Tracker and the Tundra Tracker, attach to the waist, feet or props and are tracked by the same external base stations.

Outside the consumer market, professional optical motion capture systems such as those made by OptiTrack and Vicon are outside-in systems. They surround a capture volume with high-speed IR cameras that track retroreflective markers attached to objects or to a performer's body, and they are widely used for character animation, film, games and biomechanics because they offer high precision and low latency. OptiTrack cameras emit IR light that is reflected by passive retroreflective markers and detected by the camera sensor; the software computes each marker's 2D image position and reconstructs its 3D position from multiple views. OptiTrack states that its systems deliver sub-millimetre accuracy, with positional error of less than 0.3 mm in robot-tracking deployments and accuracy as low as sub-20 micrometres in optimal conditions. Vicon has built optical motion capture systems for over 40 years and serves life sciences, entertainment and engineering customers.

Tracking systems using outside-in tracking

Tracking system	Used by	Light spectrum
Constellation	Oculus Rift CV1 and Oculus Touch	Infra-red
Lighthouse (SteamVR Tracking)	HTC Vive, Valve Index, Pimax Crystal, Bigscreen Beyond	Infra-red
PlayStation Camera	PlayStation VR and PlayStation Move	Visible
Neon Tracking System (Ximmerse)	HTC Link	Visible
OptiTrack / Vicon	Professional motion capture	Infra-red

Accuracy

Outside-in systems can reach high precision because they use dedicated, fixed sensors. For Constellation, Oculus and contemporary coverage described sub-millimetre positional accuracy with near-zero latency. The Lighthouse system has been measured independently: VR researcher Oliver Kreylos, then at UC Davis, reported in July 2016 that a stationary HTC Vive headset showed jitter of about 0.3 mm when covered by two base stations, rising to about 2.1 mm along an axis seen by only one base station, and he estimated overall precision at around 1.5 mm RMS and accuracy at around 1.9 mm RMS. Professional motion capture systems from OptiTrack and Vicon target sub-millimetre accuracy over much larger volumes using many high-resolution cameras.

Advantages and disadvantages

The main reported advantages of outside-in tracking are high accuracy and precision, because the system can use dedicated, fixed sensors and specialized algorithms; low computational load on the headset, because the pose is calculated on a separate stationary computer rather than on the worn device; and the ability to keep controllers tracked through large rotations, as long as their markers stay visible to at least one camera.

The disadvantages stem from the need for external hardware. The cameras or base stations must be installed, positioned and calibrated, which requires planning and a dedicated play area. Tracking is confined to the fixed volume covered by the cameras, the tracked device must generally face the cameras, and tracking can be lost through occlusion when the user's body or furniture blocks the line of sight between a camera and the markers. Systems that rely on visible light, such as PlayStation VR, are also sensitive to room lighting conditions.

Industry shift toward inside-out tracking

For standalone consumer headsets the industry has largely moved from outside-in to inside-out tracking, which places the cameras on the headset and removes the need for external sensors. The Oculus Quest (2019) shipped as a standalone headset that needs no external sensors, using the Oculus Insight inside-out system, and the Oculus Rift S replaced the original Rift's Constellation sensors with the same inside-out approach. Inside-out tracking simplifies setup and removes the fixed tracking volume, at the cost of higher computational load on the headset and of controller tracking that can be lost when a controller leaves the field of view of the headset's own cameras. Sony followed the same path with PlayStation VR2 in 2023. Outside-in tracking has not disappeared, however: as of 2026 Lighthouse-based SteamVR Tracking remains in active use on tethered PC VR headsets such as the Valve Index, Bigscreen Beyond and Pimax models, where its precision and full-body tracking accessories are valued, and professional motion capture continues to rely almost entirely on outside-in optical systems.