Line of sight

Line of sight or LoS is a straight line along which the observer has unobstructed vision. It could also mean the entire visible area (unobstructed area) of the observer's field of view.

In virtual reality and augmented reality the term is used in three related senses. In vision science and eye tracking, the line of sight is a defined axis of the eye, and gaze trackers estimate its three-dimensional direction to find where a user is looking. In tracking, it is the clear path that optical and acoustic systems need between their emitters and sensors; when that path is blocked, the tracked object is occluded and its pose can be lost. In wireless VR, it is the unobstructed radio path that high-frequency links such as 60 GHz WiGig need between the transmitter and the headset.

Reviewed 27 September 2026. Checked every citation against its source (Arba Mosquera 2015, Villanueva 2008 chapter PDF, Welch and Foxlin 2002 PDF, EyeWiki, Microsoft Learn, Valve, HTC, Oculus/Meta, Magic Leap, Road to VR, UploadVR, Unreal and Unity docs), including specs, dates, quotes and headlines. About review dates.

Line of sight of the eye

Ophthalmic optics distinguishes several reference axes of the eye. In their 2015 review of centration in refractive surgery, Arba Mosquera, Verma and McAlinden define the line of sight as "the ray from the fixation point reaching the foveola via the PC" (pupil center). It is not a single straight line inside the eye: it runs from the fixation point to the pupil center and then reaches the foveola after refraction at each optical surface. The same review defines the pupillary axis as the line normal to the cornea that passes through the center of the entrance pupil, and describes the visual axis as a line from the fixation point to the foveola that does not necessarily pass through the pupil center. A task force of the Optical Society (OSA) formed in 1999 recommended that the line of sight be used as the reference axis for calculating and measuring the optical aberrations of the eye.

The angle between the pupillary axis and the line of sight is called angle lambda. EyeWiki describes it as nearly identical to angle kappa, the angle between the optical and visual axes, when the fixation point is not very close to the eye.

Gaze estimation

In gaze tracking the three-dimensional direction of gaze is defined as the line of sight, and the point of regard is the point where the line of sight meets the area being viewed, such as a screen. Villanueva, Cerrolaza and Cabeza note that a visual fixation is a stable position of the line of sight with an angular dispersion below 1 degree, which is why most gaze tracking designers try to keep estimation errors below 1 degree. They describe the fovea, the retinal area responsible for central vision, as about 1.2 degrees in diameter.

Geometry-based trackers such as the one the chapter describes find the optical axis of the eye first, from the pupil and the corneal reflections of infrared light sources, and then derive the line of sight from it. Because the fovea is offset from the back pole of the eyeball, the visual axis differs from the optical axis by a horizontal angle of about 5 degrees and a vertical angle of 2 to 3 degrees, figures the chapter takes from Guestrin and Eizenman (2006). The line of sight is approximated by the visual axis, the line connecting the fovea and the nodal point, and the tracker applies this angular offset to the optical axis to obtain it.

Mixed reality headsets use two forms of gaze as input. Microsoft's design guidance for HoloLens describes head-gaze as a ray pointing straight ahead from between the user's eyes, computed from the position and orientation of the head, and calls it "a fairly coarse approximation of where the user is looking". Headsets with eye tracking, such as HoloLens 2, produce an eye-gaze vector instead, which gives a finer measurement. Applications intersect the gaze ray with holograms or with the spatial mapping mesh to decide what the user is targeting.

Line of sight in tracking systems

In their 2002 survey of motion tracking, Greg Welch and Eric Foxlin list immunity to occlusion, "needing no clear line of sight to anything else", among the properties of an ideal tracker, and compare how each tracking technology meets it:

Technology	Line-of-sight requirement (Welch and Foxlin, 2002)
Optical	The primary disadvantage of all optical systems is that there must be a clear line of sight between the source and the sensor. Adding more sensor units increases overall visibility.
Acoustic	Requires a line of sight between emitters and receivers, but is somewhat more tolerant of occlusion than optical tracking because sound travels through and around obstacles more easily.
Magnetic	Magnetic fields pass through the human body, so there is no line-of-sight requirement; the authors call this "extremely convenient and unique to magnetic trackers". Ferromagnetic and conductive objects distort the field.
Inertial	Inertial sensors are self-contained and have no line-of-sight requirements and no emitters to install.

The paper also explains why a partly blocked optical target is hard to handle. With analog photosensors, a partial occlusion "may turn out to be the biggest problem, as it results in a signal that's plausible but incorrect", while image-forming cameras can recognize and reject partially occluded sources by their shape. Prior knowledge about how targets move relative to each other, such as markers on the shoulder and arm staying a fixed distance apart, can help a system survive momentary occlusions.

Lighthouse

Valve's Lighthouse system (SteamVR Tracking) is an optical outside-in system. In the version sold with the Valve Index, fixed lasers in each base station sweep the room 100 times a second and are picked up by photosensors on the headset and controllers; each base station has a 160 by 115 degree field of view and a 7 m range, and up to four base stations can cover a 10 m by 10 m space. Valve hardware engineer Alan Yates, quoted by Road to VR in 2015, said that "like all optical systems occlusion is a limiting factor" and that spatial redundancy helps. According to Yates, one base station is enough to track a rigid object that has an IMU, five sensors need to be visible to acquire a pose (though tracking can hold with fewer), and visibility to two or more base stations allows a single sensor to be tracked.

With the first-generation base stations used by the original HTC Vive, the two base stations also synchronize with each other optically. HTC's user guide tells users to make sure each one has an unobstructed view of the other, and recommends connecting them with a sync cable when a hanging object, or the user's own head or hands, blocks the base stations from seeing one another. SteamVR Tracking 2.0 replaced the original sync blinker with "sync-on-beam", in which data bursts are delivered through the laser and sync hits; Valve had said the blinker caused interference between base stations. The tracked devices still need to be in view: HTC's documentation for SteamVR Base Station 2.0 tells users to make sure the lines of sight between the base stations and VIVE Tracker (3.0) units are not obstructed.

Constellation

The Oculus Rift used Constellation, in which external camera sensors track infrared LED markers on the headset and Touch controllers. Road to VR explained the occlusion problem in 2016: a player could turn 180 degrees away from the camera, hold the controllers near their chest, and the controllers would become completely occluded from the camera. Oculus addressed this with additional cameras, recommending a three-sensor setup for full 360-degree room-scale use. Oculus's own 2017 setup guide stated that a single sensor offered ideal Touch tracking up to six feet away, good tracking up to eight feet, and "essentially won't track past 10 feet away", while two sensors that can both see a controller should give ideal tracking up to at least 10 feet. Citing "the risk of occlusion", it recommended maximum playable areas of five by five feet for two-sensor 360 setups and eight by eight feet for three-sensor roomscale setups. The guide also warned about furniture blocking the sensors' view and advised against using more than three sensors.

Inside-out and hybrid systems

Inside-out tracking moves the cameras onto the headset, so a controller has to be within the view of the headset's own cameras. In Oculus Insight, infrared LEDs in the controllers are detected by the headset cameras, which lets the system bound the position drift caused by integrating data from multiple IMUs; an extrapolation function predicts where the head and hands will move in the milliseconds ahead.

The Magic Leap 2 controller combines IR LED tracking with a lightweight SLAM tracker that runs on the controller itself, and Magic Leap's developer documentation says tracking "should still work when there is no direct line of sight". The documentation lists limits: the controller can lose tracking out of line of sight in low-light, low-texture or high dynamic range scenes; if the headset moves more than 5 meters without line of sight to the controller LEDs, the controller pose stops reporting; and if neither method can find the 6DoF position, the controller falls back to 3DoF orientation only, stopping position reports if 6DoF tracking is not re-established within one minute. Its predecessor, the Magic Leap One, used electromagnetic tracking for its controller: FCC filings described a transmitter generating three orthogonal AC magnetic fields at frequencies from 28.5 kHz to 42.42 kHz.

Camera-based hand tracking has the same constraint. Microsoft's HoloLens design documentation states that the device recognizes gestures from hands that are visible to it, and that the hand must be within a "gesture frame" that the gesture-sensing cameras can see, from nose to waist and between the shoulders. It recommends that an application stop, rather than reverse, a manipulation when the user's hand leaves the frame.

Line of sight in wireless VR

Wireless links in the 60 GHz band depend on line of sight. The HTC Vive Wireless Adapter, released on 24 September 2018, used Intel's 60 GHz WiGig technology with DisplayLink's XR codec. Road to VR explained that because 60 GHz is a high-frequency band, "it doesn't penetrate walls or other objects very well, and consistent line of sight to the transmitter is necessary"; the transmitter has a range of up to six meters and a 150 degree field of view. HTC's setup instructions say to mount the link box at least 125 cm (4 ft) from the ground and about 150 cm (5 ft) from the play area, facing the play area, with a clear line of sight between the user and the link box.

Wi-Fi streaming uses lower frequencies but is also sensitive to placement. For Air Link on the Quest headsets, Oculus recommended that the router "be in the same room as the headset or in line-of-sight, and at least 1m off the ground", with the PC connected to the router by Ethernet and the headset on the 5 GHz band.

Line-of-sight tests in software

Game engines used to build VR and AR applications offer visibility queries that check whether a straight line between two points is blocked. In Unreal Engine, the controller function Line Of Sight To "checks line to center and top of other actor" and returns true if the controller's pawn can see the other actor; the eye position the check starts from can be supplied or taken from the current view target. A comparable general query in Unity is Physics.Linecast, which returns true if any collider intersects the line between a start point and an end point.

See also

• Occlusion
• Field of view
• Eye tracking
• Gaze
• Outside-in tracking
• Optical tracking
• Positional tracking