Body tracking

Body tracking is the capture of the pose and movement of a user's body so that it can drive an avatar or interaction in virtual reality (VR) and augmented reality (AR). In consumer VR the head and the two hands are tracked directly by the headset and controllers, which gives three points of positional tracking; reproducing the rest of the body (torso, hips, knees and feet) requires either additional sensors worn on the body or software that estimates the missing joints from the available points.

When tracking is extended beyond the head and hands to follow the whole skeleton, it is usually called full-body tracking (FBT). Body tracking draws on techniques from motion capture, a field developed for film, games and biomechanics, and adapts them to run in real time and at consumer cost for social VR, fitness and embodied interaction.

How it works

The standard problem in consumer VR is that the headset reports head pose and the controllers report hand pose, leaving the rest of the body unknown. Two broad approaches add the missing information: measure more of the body with hardware, or infer it with software.

Hardware-based body tracking places trackers at extra points (commonly the waist and the two feet) and reads each tracker's position and orientation. A skeleton fitted to those points, together with the user's measured body proportions, then poses the avatar. Software-based body tracking takes the few points that are already tracked and applies inverse kinematics (IK), a method that computes the joint angles of a kinematic chain from the desired position of its end effectors (such as a hand or foot). IK lets a system place the elbow, shoulder, pelvis and legs so that the chain reaches the known head and hand positions, but the result is an estimate: with only three points the legs in particular are underdetermined, which can make feet slide or sitting and crouching look wrong.

A third approach replaces or supplements hand-written IK with learned models that generate plausible motion from sparse inputs. At CVPR 2023, Du and colleagues at Meta described AGRoL, a conditional diffusion model that synthesises full-body motion, including the lower body, from the head and two wrist signals available on standalone headsets, training on the AMASS motion-capture dataset; the authors report more realistic motion than prior methods while running in real time.

Methods

Body tracking systems used with VR and AR fall into several families that differ in the sensors they use and in whether they need external infrastructure.

Method	How it senses the body	External infrastructure	Notes
Dedicated optical trackers (Lighthouse)	Pucks worn on the body are tracked by SteamVR Lighthouse base stations using infrared laser sweeps	Requires base stations	High precision; example is the Vive Tracker
Self-tracking (inside-out) pucks	Each tracker carries its own cameras and computes its position from the environment	None	Example is the VIVE Ultimate Tracker
Inertial (IMU) trackers and suits	Accelerometer, gyroscope and sometimes magnetometer in each unit measure orientation; positions are derived from body proportions	None	Examples are SlimeVR and Xsens MVN suits
Markerless camera estimation	A depth or RGB camera observes the user and a model labels body parts and fits a skeleton	A camera facing the user	Example is the Microsoft Kinect
Sparse-point inference	IK or a learned model extends the headset and controller points to the whole body	None	Example is Meta's Movement SDK

Dedicated optical trackers

The most common consumer full-body setup attaches dedicated trackers to the waist and feet. The Vive Tracker 3.0 is a 75 g puck with up to 7.5 hours of battery life that is tracked by SteamVR Lighthouse base stations and pairs to the PC through a wireless dongle; HTC lists it at 149 USD and the number usable at once depends on available USB ports, application support and radio conditions. Third-party Lighthouse trackers such as Tundra Trackers work the same way, and applications like VRChat accept up to eight extra trackers in addition to the headset and controllers, mapping them to feet, knees, hips, chest, elbows and shoulders, with a calibration step in which the user stands upright and looks forward so the system can align the trackers to the avatar's proportions.

To remove the base stations, HTC released the VIVE Ultimate Tracker, announced in November 2023, which carries two wide field-of-view cameras and an onboard chipset so each puck performs its own inside-out positional tracking. HTC recommends at least three trackers for body tracking, a USB-C dongle supports up to five at once, and the system works over OpenXR with SteamVR and with the Vive XR Elite.

Inertial trackers and suits

Inertial systems put an IMU on each tracked body segment. The sensors measure orientation directly; absolute position is reconstructed from a biomechanical model of the body rather than measured optically, so the systems work outdoors and indoors with no cameras, emitters, markers, line-of-sight requirements or lighting constraints. Xsens MVN, a professional inertial motion-capture suit, is built on miniature inertial sensors, biomechanical models and sensor fusion algorithms.

In the consumer space, SlimeVR is a set of open hardware and open-source software for full-body tracking that needs no base stations. Each tracker measures its own rotation with an IMU and sends the data to a server on the PC or phone, which combines the trackers with the user's proportions and the headset position to place the body parts; five trackers (one on each thigh, one on each ankle and one at the waist) are enough for a basic setup. Drift from accumulated IMU error is corrected over time by skeletal calibration; SlimeVR's "Autobone" routine estimates bone lengths from movement.

Markerless camera estimation

Markerless systems observe the user with a camera and recover the skeleton in software, with no worn hardware. Inexpensive depth cameras, in particular the Microsoft Kinect, made monocular full-body skeletal tracking practical for research and hobbyist VR. A typical pipeline segments the user from the background, classifies each pixel into a body part with a machine-learning model, fits a kinematic skeleton to the classified pixels and applies temporal filtering to reduce jitter. The main weaknesses are occlusion (joints hidden from a single camera) and non-frontal poses, which researchers address by fusing two or more cameras placed at different angles. Markerless tracking improves comfort and cuts setup time compared with worn trackers, but generally at the cost of lower positional accuracy and higher latency.

Sparse-point inference on standalone headsets

On standalone headsets the trend has been to add body tracking without any extra hardware by inferring the body from the sensors the headset already has. Meta's Movement SDK exposes body, eye and face tracking through OpenXR; its body-tracking path reconstructs a body skeleton from three input points, the headset and the two hands or controllers, and is supported on Meta Quest 2, Quest 3, Quest 3S and Quest Pro. At Meta Connect 2023 the company added two features, released to developers on 20 December 2023 in the v60 update. Inside-Out Body Tracking (IOBT), exclusive to Meta Quest 3, uses the headset's side cameras to capture elbow, wrist and torso movement directly. Generative Legs, compatible with all Quest devices, uses an AI model to infer leg motion such as walking, jumping, ducking and squatting from the upper-body pose alone; Meta says this produces more natural avatars than traditional IK and, combined with IOBT, yields a plausible full body without external hardware.

Origins in motion capture

Body tracking for VR descends from motion capture, the recording of human movement for animation and analysis. One of the earliest worn systems was the Data Suit from VPL Research in the late 1980s, which used sensors connected by fibre-optic cables to a computer that updated the figure 15 to 30 times a second. Two families of professional mocap matured in parallel and still underpin body tracking today. Optical marker-based systems, made by companies such as OptiTrack and Vicon, surround a capture volume with high-speed infrared cameras that track reflective or light-emitting markers on a performer, giving high precision and low latency; this is an example of outside-in tracking adapted to the body. Inertial systems such as Xsens MVN instead put IMUs on the body and need no cameras at all. Consumer VR body tracking takes these ideas and trims them to a handful of trackers, or to no trackers, so they fit a living room and a consumer budget.

Uses in VR and AR

The most visible use is embodiment in social VR. In applications like VRChat, standard three-point tracking (head and two hands) drives the upper body, and adding waist and foot trackers lets the legs and hips follow the real body, which the platform handles through its IK system and a calibration step that aligns the user to their avatar's proportions. Better lower-body tracking improves the sense of presence and self-immersion, and supports actions such as dancing, kicking and sitting that three-point IK approximates poorly. Body tracking is also used in VR fitness and exercise titles, in dance and performance, in VTubing where a tracked performer animates a virtual character, and in research on embodiment and avatars. On AR and standalone hardware the same upper-body and leg-estimation methods let a headset animate a fuller avatar of the wearer for telepresence and social presence without any worn trackers.

Current status

As of 2026 there is no single dominant method; the choice depends on the platform and the user's tolerance for extra hardware. PC VR users who want accurate legs still rely on worn trackers, either Lighthouse pucks such as the Vive Tracker 3.0 or inertial trackers such as SlimeVR. Self-tracking pucks like the VIVE Ultimate Tracker remove the base-station requirement for that audience. Open hardware continues to develop: SlimeVR's Butterfly trackers, detailed during a Crowd Supply crowdfunding campaign in February 2026, are ultra-slim IMU trackers built on a Nordic nRF52833 microcontroller and a TDK ICM-45686 IMU, rated for over 48 hours of use and a latency under 15 ms, with a six-tracker set listed from 279 USD and shipping expected in August 2026. On standalone headsets the direction is hardware-free body tracking driven by on-device cameras and learned models, such as Meta's IOBT and Generative Legs, which trade some accuracy for the convenience of needing nothing beyond the headset.

See also

• Positional tracking
• Motion capture
• Inverse kinematics
• Hand tracking
• Avatar
• SlimeVR