Latency
More actions
Latency in virtual reality (VR) and augmented reality (AR) is the delay between an input or physical event and a corresponding system response. Motion-to-photon latency (MTP) measures the delay between movement of a tracked object and its visual representation on a display. Head movement, controller movement and changes in the camera view can follow different processing paths, so a headset does not have one latency value that describes every interaction.[1]
Delayed visual feedback can affect interaction and presence, and latency can contribute to cybersickness. The magnitude and perceptual effects depend on the movement, application, display and timing behavior. In AR, delayed virtual imagery can also move out of alignment with real objects. Useful latency measurements therefore identify their starting event, output endpoint and test conditions.[2][3]
Motion-to-photon latency
An end-to-end measurement compares a physical event with the light produced by the corresponding display update. It includes delays in sensing, communication, tracking, application processing, rendering and presentation. Measuring only the time spent rendering a frame, or the interval from an API call to presentation, leaves out parts of that path.[2][4]
For an event-based test, latency is the time of the visible response minus the time of the physical event. Continuous-motion tests instead compare trajectories, for example by estimating a phase shift between real and displayed motion. These approaches can give different results when the system uses predictive tracking. A predictable movement may be displayed with little apparent lag even though the physical processing pipeline still takes time.[5]
Distinguishing feedback paths
| Measure | Starting event and output | Interpretation |
|---|---|---|
| Head motion-to-photon | Head movement to the corresponding displayed viewpoint | Specify rotation or translation, movement profile and location measured on the display. A compositor may update the viewpoint after application rendering.[6] |
| Controller motion-to-photon | Movement of a tracked controller to the corresponding visible movement | Does not measure button response or necessarily share the head-view correction path.[5] |
| Hand tracking delay | Physical hand movement to estimated hand or finger data; alternatively, movement to a displayed virtual hand | A comparison of tracking data with a reference motion capture system is not automatically a measurement of display emission.[7] |
| Eye-to-image latency | Eye movement or eye-camera capture to the resulting gaze-dependent display change | Includes eye tracking and the image-update pipeline; relevant to foveated rendering.[8] |
| Passthrough photon-to-photon | A change in light in the environment to its appearance through the headset camera and display | Measures the real-world video path, distinct from virtual-content head motion-to-photon.[9] |
| Interaction-to-haptic or interaction-to-audio | A physical or virtual event to vibration, force or sound output | Separate hardware and software paths can give different delays, creating audiovisual or visual-haptic asynchrony.[10] |
Latency also differs from spatial tracking error. A pose can be late but accurate for its timestamp, or timely but spatially inaccurate. Motion prediction introduces another distinction: its displayed pose estimates a future state rather than reporting a newly measured physical state. Calibration and temporal alignment both matter when comparing a tracked hand or object with a reference.[4][7]
History
Early head-mounted display research
Latency was measured as a system-level problem in head-mounted display research by the early 1990s. In a technical report dated 23 March 1993, Mark R. Mine of the University of North Carolina at Chapel Hill defined end-to-end delay as the total time for the image in an HMD to change in response to movement of the user's head. He divided it into tracking system, application host, image generation and display system delays. For the tests, the tracking sensor was removed from the HMD and mounted on a pendulum arm, a photodiode and LED marked the arm's passage through a reference point, a second photodiode watched a conventional monitor used in place of the HMD screens, and a digitizing oscilloscope recorded the signals. Tracker delays including communication ranged from 10.95 ms for a single-unit Polhemus Fastrak to 61.65 ms for the older Ascension Bird. With the laboratory's Pixel-Planes 5 graphics system, a one-polygon test scene and NTSC displays, the delay after tracking data reached the host was approximately 70 ms. Mine concluded that most graphics systems were optimized for high throughput instead of minimum end-to-end delay.[11]
Researchers in the 1990s also changed display hardware and rendering systems to shorten or hide these delays. Matthew Regan and Ronald Pose's 1994 SIGGRAPH paper described an address recalculation pipeline, a display controller that performed orientation viewport mapping after rendering. In their design, head orientation did not need to be known accurately until less than a microsecond before the first pixel of a frame was sent to the HMD.[12] At UNC, Marc Olano, Jon Cohen, Mark Mine and Gary Bishop presented two approaches in 1995: SLATS, an experimental Pixel-Planes 5 rendering system that guaranteed a constant single NTSC field of latency, and "just-in-time pixels", which rendered the pixels of a scanned display according to their position in the scan.[13]
Ronald Azuma's 1997 survey of augmented reality described end-to-end delays of 100 ms as fairly typical of existing systems, with delays of 250 ms or more on slow, heavily loaded or networked systems. Citing Holloway, he described system delay as the largest single source of registration error in existing AR systems. The survey grouped countermeasures into four categories: reducing system lag, reducing apparent lag (for example, image deflection or image warping applied after rendering), matching the timing of real and virtual video streams in video see-through systems, and predicting future viewpoint and object locations.[14]
Consumer development kits
In a post on Valve's blog dated 29 December 2012, Michael Abrash wrote that from his experience "more than 20 ms is too much for VR and especially AR", and that research indicated the threshold might be 15 ms or even 7 ms. His illustration combined a 60-degree-per-second head turn with 50 ms of latency, which left virtual images about three degrees out of position. He estimated a then-current total of 36 ms (4 ms tracking, 16 ms rendering and up to 16 ms of 60 Hz scan-out, assuming zero persistence) and described sensor fusion of optical and inertial tracking, simpler scenes and "racing the beam" as ways to reduce it.[15]
John Carmack's essay on latency mitigation strategies, quoted at length in a WIRED post by Bruce Sterling on 24 February 2013, stated that absolute delays below approximately 20 ms are generally imperceptible. It also stated: "A total system latency of 50 milliseconds will feel responsive, but still subtly lagging." It recommended recording high-speed video that captures both the physical motion and the display update, then counting the video frames between them.[16] On 25 September 2013, Oculus VR opened pre-orders for a Latency Tester, a developer tool it said could "quickly and precisely measure motion-to-photons latency from any point in their game pipeline."[17] Engadget reported that the device replaced one of the development kit's two lens caps, connected by mini-USB, and showed a per-eye latency reading on its rear.[18]
Milestones
The following table lists dated developments described in this article. It is a selection, not a complete history of latency research.
| Year | Development | Notes |
|---|---|---|
| 1993 | Mine's UNC end-to-end delay measurements | Pendulum, photodiodes and oscilloscope; tracker, host, rendering and display delays separated.[11] |
| 1994 | Regan and Pose's address recalculation pipeline | Orientation viewport mapping performed after rendering, in the display controller.[12] |
| 1995 | SLATS and just-in-time pixels | Constant single-field rendering latency on Pixel-Planes 5; pixels rendered according to scan position.[13] |
| 1997 | Azuma's AR survey | Typical end-to-end delay of 100 ms; four categories of countermeasure.[14] |
| 2003-2004 | Latency discrimination studies by Adelstein, Ellis and colleagues | Reported JNDs below 17 ms and averaging about 15 ms or less for trained observers.[19][20] |
| 2012-2013 | Abrash and Carmack latency essays; Oculus Latency Tester | 20 ms discussed as an engineering target; Latency Tester offered to developers.[15][16][17] |
| 2015 | Asynchronous timewarp explanation | Orientation-only reprojection of a completed frame with a newer head pose.[6] |
| 2016 | Asynchronous spacewarp | Frame extrapolation; Oculus's example used 45 fps application output for a 90 Hz display.[21] |
| 2018 | SteamVR Motion Smoothing | Extrapolation from the last two delivered frames when an application misses frame rate.[22] |
| 2020 | Phase Sync on Quest | Adaptive frame timing; Meta reported a 10 ms reduction in Oculus Home on Quest 2.[23] |
| 2024 | OptoFidelity passthrough tests | Approximately 11 ms minimum photon-to-photon latency for Apple Vision Pro, 35-40 ms for three other headsets.[9] |
| 2025 | Meta's Quest 3S display paper | Motion-to-photon latency reported as 14% better than Quest 2 and 8% better than Quest 3.[24] |
Types and sources of latency
Components of the latency pipeline
The processing path includes both computation and waiting for other components. Overlapping pipeline stages make high throughput possible, but throughput alone does not establish how old the displayed input is. Buffering and synchronization can add delay even when a stage finishes its work quickly.[2]
| Stage | Sources of delay | Measurement or optimization consideration |
|---|---|---|
| Sensor acquisition | Sampling intervals, camera exposure and readout | Use the measurement timestamp, rather than treating data arrival as the moment the physical event occurred.[25] |
| Tracking and sensor fusion | Motion estimation, filtering, transport and processing of older measurements | Newer inertial data can update a pose before the next camera-based correction becomes available.[25] |
| Application simulation | Input sampling, physics, interaction logic and thread scheduling | A visually updated controller and a collision response can use poses from different points in the frame.[5] |
| CPU and GPU rendering | Command generation, GPU work, synchronization and queued frames | Profile both application and compositor work, including missed submissions and GPU scheduling effects.[26] |
| Compositing and correction | Image distortion, overlays, reprojection and frame synthesis | The displayed head pose may be newer than the application image; scene animation and interaction responses may remain older.[6] |
| Transmission and display | Image transfer, display buffering, scanout and pixel illumination | Identify the measured display location and whether the endpoint is the first, middle or another scanline.[27] |
Sensor and tracking latency
An inertial measurement unit supplies angular velocity and acceleration observations. High sampling frequency helps provide recent observations, but the sampling interval is not the complete sensor-to-display delay. The historical Oculus Rift Development Kit tracking paper reported inertial observations at 1,000 Hz and separately discussed transport, fusion, simulation, rendering and video output. It also distinguished estimating present orientation from forecasting the orientation needed when an image becomes visible.[4]
Camera images represent exposure intervals, and their age depends on the timestamp convention. Daniel Wagner's 2018 DAQRI pipeline explanation used a hypothetical 20 ms exposure timestamped at its midpoint: the image was already 10 ms old when exposure ended, before another assumed 5 ms of readout. Those values illustrate pose age in that example; they are not measurements of every inside-out tracking system. Integrating subsequent IMU observations can bring an older camera-based estimate forward, with limits from noise and drift.[25]
The visual hand model has its own detection and estimation stages. In a Quest 2 study published online in 2023, Abdlkarim and colleagues compared hand data with Qualisys motion capture for eight volunteers performing reaching and hand opening tasks. The headset, running build 33.0, was fixed above the participant's head and connected through Link to a custom Unity environment. For reaching, the authors reported a mean absolute tracking delay of 45.0 ms, obtained by adding a 7 ms reference-system correction to the measured relative delay of 38.0 ms. The correction came from a cited Qualisys latency test. This was a comparison of tracking data in that configuration, with participants limited to light skin tones; it did not measure emitted light from a displayed hand.[7]
Display latency
Refresh rate describes the number of display refresh cycles per second. The reciprocal gives the nominal interval between refreshes. For a rate f in hertz, this interval is 1,000 / f milliseconds. The following values are arithmetic conversions, rather than headset latency measurements:
| Refresh rate | Nominal refresh interval |
|---|---|
| 60 Hz | 16.67 ms |
| 72 Hz | 13.89 ms |
| 90 Hz | 11.11 ms |
| 120 Hz | 8.33 ms |
| 144 Hz | 6.94 ms |
The interval establishes a timing budget, but it does not give the total age of a displayed pose. The OpenXR specification allows the predicted application display period to differ from the hardware refresh cycle. Runtime frame pacing, queued work and reprojection affect when an application frame is displayed.[28]
Pixel transition time, scanout and persistence are separate properties. Transition time concerns a pixel's change in output; scanout concerns delivery or updating of different parts of an image; persistence concerns how long an image remains illuminated. Low persistence can reduce the blur produced while the eyes move across an illuminated image. It does not remove upstream sensing or simulation delays. An OLED or LCD label alone therefore does not determine end-to-end latency.[29][25]
Display tuning can trade latency against other artifacts. In a 2025 Society for Information Display paper, Meta engineers wrote that ghosting in LCD headsets occurs when the backlight turns on before the liquid crystal has fully settled, or when the duty cycle is too high, and that a longer settling time improves ghosting but inherently increases latency. For Meta Quest 3S, which uses the same display system design as Meta Quest 2, they described reducing backlight on-time, optimizing GPU rendering and applying temperature-dependent backlight timing, because liquid crystal responds faster at higher temperatures. They reported that the headset's motion-to-photon latency was 14% better than Quest 2 and 8% better than Meta Quest 3. The paper gives these as relative figures and does not state an absolute latency or a test method.[24]
Rendering latency
The renderer may finish a frame at a high rate while still displaying old input. Multi-stage rendering separates application simulation, CPU command submission, GPU execution and compositor presentation. Avoiding unnecessary queued frames and supplying a recent pose close to rendering can reduce the delay along this path. Front-buffer rendering is a historical approach that draws to the buffer read by the display; it requires careful synchronization between rendering and scanout.[29]
SteamVR's advanced frame timing graph separates CPU and GPU work and records events such as late starts, dropped frames and reprojection. Its application CPU graph does not capture every parallel task on other threads. GPU timings can include scheduling overhead, idle gaps or competing work. These distinctions help locate a bottleneck, but a low value in one graph is not proof of low motion-to-photon latency.[26]
Human perception and physiological effects
Perception thresholds
Latency detection is a task-dependent measurement. In a detection experiment, a point of subjective equality (PSE) identifies the stimulus level at a 50% detection rate. A just-noticeable difference (JND) is the stimulus change needed to move that rate by 25 percentage points. These fitted measures depend on the experiment and cannot by themselves establish a maximum acceptable system delay. In Jerald and Whitton's 2009 scene-motion study, six analyzable participants viewed a projected scene through a modified HMD casing while making prescribed head yaws. Across participants and head-acceleration bins, the mean latency JND was 16.6 ms and the smallest was 3.2 ms. The mean PSE was 38.9 ms. The authors explicitly limited transfer to different contrast, fields of view, experience and motion conditions.[30]
Earlier discrimination experiments reported thresholds of a similar size. Bernard Adelstein, Thomas Lee and Stephen Ellis tested whether observers detect latency through "image slip", the motion of the virtual scene caused by system lag. Forcing a change from constant-rate to randomly paced head motion shifted latency detection criteria and caused a minor degradation in discrimination; the authors inferred that observers use image slip at least in part, and that the JND of observers who had learned to discriminate latency remained below 17 ms.[19] Katerina Mania and colleagues presented head-tracked, stereoscopic HMD scenes ranging from simple geometric objects to a radiosity-rendered pair of interconnected rooms. Comparing the results with an earlier study that used only simple objects, they inferred that the JND for trained observers averages about 15 ms or less, independent of scene complexity and real-world meaning.[20] Allison and colleagues studied perceptual stability directly: subjects became significantly more likely to report oscillopsia, the perception that the visual world swims or oscillates, during head movements when end-to-end latency or head velocity increased.[31]
The frequently quoted 20 ms target also appears in historical engineering guidance. John Carmack's 2013 essay recommended reducing absolute head-view delays to approximately that level. This recommendation is useful context for latency optimization, but it is not a formal universal perceptual or comfort standard.[16][30] Michael Abrash's 2014 presentation described 20 ms motion-to-last-photon as working for Valve's prototypes and suggested 25 ms might be sufficient. He also treated presence as depending on several display, optical and tracking properties, rather than latency alone.[32]
Waltemate and colleagues studied a different question in 2016: ten participants copied upper-body movements while viewing a full-body virtual mirror in a CAVE. Avatar feedback had total delays of 45, 75, 125, 210 or 350 ms; the tracked-glasses perspective was not delayed. Motor accuracy worsened between 75 and 125 ms, while reported agency and body ownership declined later. The simultaneity comparison between 75 and 125 ms was a statistical trend, rather than a significant result in that pairwise test. These findings concern delayed avatar feedback during their movement-copying task, not a 75 ms head-view threshold for all VR.[33]
Presence and task performance
Meehan, Razzaque, Whitton and Brooks examined physiological and self-reported responses in a virtual environment with an ordinary room and a second room designed to evoke fear or stress. Participants were assigned to a low-latency group (about 50 ms) or a high-latency group (about 90 ms). The low-latency group reported a higher sense of presence and showed a statistically higher change in heart rate between the two rooms. The study found no significant relationship between latency and simulator sickness.[34]
Watson, Walker, Ribarsky and Spaulding studied variation in system responsiveness, which they defined as the elapsed time until a system responds to user control and described statistically by its mean and standard deviation. Their three within-subjects studies of grasp and placement tasks had 11, 12 and 10 participants; the first controlled mean responsiveness through mean frame rates of 9, 13 and 17 Hz. The standard deviation of responsiveness affected performance only when it was above 82 ms, and placement, which required more frequent visual feedback, was more sensitive to responsiveness than grasping.[35] The results concern grasp and placement performance, not latency detection thresholds or cybersickness.
Simulator sickness and cybersickness
Delayed responses can create disagreement between visual feedback and motion cues. Sensory mismatch is one explanation for cybersickness; other theories include postural instability and rest-frame accounts. The symptoms and their relationship to latency depend on the system and exposure. A latency number alone cannot predict an individual's symptoms or guarantee a comfortable experience.[1]
Timing variation also matters. In a 2018 HTC Vive search experiment, Stauffert and colleagues analyzed a final sample of 45 participants, comparing a group with injected HMD tracking jitter against a group without it. After excluding four high baseline sickness scores, their analysis found an effect on total Simulator Sickness Questionnaire score and disorientation, but not significant differences in the nausea and oculomotor subscales. The experiment supports examining spikes and irregular timing; it does not establish that every variable-delay system is worse than every higher constant-delay system.[36]
Latency reduction techniques
Software techniques
Prediction algorithms
Prediction estimates a pose for the time at which it will be used, rather than simply displaying the most recent measurement. Constant angular velocity and constant angular acceleration were among the models compared in the historical Rift head tracking paper. The prediction horizon and changing motion influence the resulting error. Filtering noisy sensor observations and predicting a future pose are related operations, but a particular filter name does not identify how much visual delay has been removed.[4]
Motion onset, impacts, reversals and stops are useful tests because they can invalidate the trend inferred from earlier samples. In Warburton and colleagues' instrumented controller experiment, the HTC Vive, Oculus Rift CV1, Oculus Rift S and Valve Index were tested with Unity 2019.4 LTS and SteamVR 1.14.16. Mean controller motion-to-photon delays across the tested setups were 21-42 ms at sudden movement onset and 2-13 ms during continuous movement. Prediction reduced apparent delay within approximately 25-58 ms of onset. These are dated controller-translation results, rather than measurements of pure rotational headset latency or a comparison of timewarp implementations.[5]
Late latching and frame scheduling
Late latching supplies newer head and controller transforms close to GPU execution. Meta's Unreal documentation, updated April 2026, describes sampling poses at the end of the render thread and patching uniform-buffer transforms. The benefit depends on how much time elapsed since the earlier pose sample; it is not a fixed reduction of a few milliseconds for every scene. The documented implementation requires Vulkan and Multi-View, and lists engine-specific incompatibilities.[37]
A later visual pose can also differ from the pose already used by gameplay and collision code. Meta describes the possibility of a head or hand briefly appearing through geometry even when earlier gameplay calculations prevent that collision. Improving visual pose timing therefore does not automatically update every simulation decision.[37]
In OpenXR, xrWaitFrame paces the application's frame loop and returns an anticipated display time. xrLocateViews provides views for a requested time, and a later call for the same target can use newer information. A pipelined application should compute its target display time using the predicted time and interval and keep that time consistent across its simulation and rendering stages. These API timing rules support prediction and scheduling; they do not guarantee a particular measured MTP value.[28]
Meta's Phase Sync is an adaptive frame-timing method for Quest applications. Meta's Unity documentation contrasts it with fixed-latency mode, in which frames are composited as early as possible to avoid missing the current frame; Phase Sync adapts frame timing to the application's workload, aiming to complete rendering just before the compositor needs the frame.[38] A December 2020 Meta developer blog post said the technology was first introduced in the Oculus PC SDK and reported latency reductions of 8 ms in Oculus Home on the original Quest and 10 ms on Quest 2. The same post said Phase Sync would be the default timing method in Meta's OpenXR implementation.[23] Both sources warn that an application whose workload fluctuates or spikes frequently may produce more stale frames, and the Unity guide states that Phase Sync and late latching often work well together.[38][23]
Asynchronous timewarp
Asynchronous timewarp (ATW) reprojects the latest completed image using a newer head pose. It can operate separately from application rendering and update a frame before display even when the application has missed a deadline. In the orientation-only version described in Oculus's 2015 explanation, rotational correction reduces view lag but does not account for translation or advance the animation of objects. Near objects can exhibit positional judder, and moving objects can remain at their earlier state.[6]
Depth-based positional reprojection can also correct viewpoint translation. It still has to handle newly exposed areas that were not present in the previous image. Reprojection thus changes the age of the viewpoint information without necessarily making all content in that image current.[6][39]
Spacewarp and synthesized frames
Asynchronous spacewarp extrapolates scene movement from earlier frames. Oculus's 2016 PC Rift explanation used an example of application frames generated at 45 fps with intermediate output for a 90 Hz display. It described artifacts from rapid brightness changes, repeated patterns, disocclusion and some translucent content. Synthetic frames are estimates of how an earlier scene should appear, rather than newly simulated application frames.[21]
The separate Quest feature, Application SpaceWarp, requires application-provided motion-vector and depth buffers. It permits rendering at half the display rate, with the compositor synthesizing intervening frames. Its implementation has additional work and memory requirements, and incorrect motion vectors can produce artifacts. Meta's guide also warns that controller input latency may remain higher than in an application rendering at the full rate, even when head-view latency is well compensated.[40][39]
Valve's SteamVR Motion Smoothing applies extrapolation on PC. Valve's announcement contrasted it with television motion smoothing, which interpolates between two existing frames and "also adds latency". When SteamVR sees that an application is not going to make frame rate, Motion Smoothing looks at the last two delivered frames, estimates motion and animation, and extrapolates a new frame. At 90 Hz on the Vive and Vive Pro, the application then needs to render only one of every two frames, and the system can scale further to two synthesized frames for every delivered frame. Valve said the feature was enabled for Windows 10 systems with NVIDIA GPUs and was not used with Oculus Rift or Windows Mixed Reality headsets, whose display drivers use different techniques when applications miss frame rate.[22] Game Developer had reported the opt-in beta in October 2018.[41]
Hardware solutions
Sensor acquisition, display electronics and processors affect the available timing budget. Hardware improvements are most useful when they shorten the path identified by measurement, rather than only increasing a component's nominal update rate. A display with fast transitions can still receive an old image, while a renderer with spare GPU capacity may be waiting on synchronization or input.[2][29]
Dedicated processing can shorten a particular feedback path. Apple's technical specification for the M5 Apple Vision Pro, read on 12 October 2026, lists an R1 chip and a 12 ms photon-to-photon latency. That specification describes the incoming-light camera/display path; it does not state that every application reaction, controller input or hand pose becomes visible after 12 ms.[42]
Research prototypes can use tighter coupling than general-purpose products. Peter Lincoln's 2017 dissertation describes a mechanically tracked optical see-through apparatus with FPGA-based post-transmission image correction and a rapidly updated digital micromirror display. Its six-bit grayscale implementation had a characterized average motion-to-photon delay of approximately 80 microseconds, with timing verified using encoder signals, a light sensor and an oscilloscope. The rig and limited correction operation are essential context; this was not a freely moving consumer headset or a general application pipeline.[43]
Measurement methods
High-speed camera methods
A camera can capture a tracked object and its displayed response in a common time reference. Event counting compares the frames containing an onset, reversal or stop. At 240 fps, the nominal sampling interval is approximately 4.17 ms; counting frames does not create arbitrarily finer event timing. Periodic-motion fitting can estimate a phase difference at sub-frame intervals, but depends on the movement model and signal quality.[2]
For an HMD, lenses and the small display area can make it difficult to film the real motion and its virtual representation together. Warburton and colleagues instead used a visible LED on a controller assembly and encoded the application's controller samples in display colors. Matching the camera footage to the recorded colors connected physical positions with samples actually shown by the headset. The technique measures controller movement without treating a desktop mirror window as the final visual endpoint.[5]
Exposure, rolling shutter, camera/display synchronization, screen persistence and event-detection rules affect uncertainty. A result should distinguish variation in the system from variation introduced by sampling or reconstruction. Friston and Steed compared automated frame counting, sinusoid fitting and photodiode-based approaches against a controllable mechanical simulator to examine such measurement accuracy.[2]
Light sensors and electronic instrumentation
Photodiodes or other light sensors detect changes in display output, while an encoder, physical switch or other reference marks the input event. An oscilloscope or synchronized acquisition system compares the signals. The method needs a test stimulus that maps the input to a detectable light change. Sensor response, thresholds and acquisition bandwidth contribute to the measurement and should be documented.[43][2]
Passthrough testing can instead compare an external changing light or clock with its appearance on the headset display. Gruen and colleagues used synchronized cameras and an electronic clock to measure this photon-to-photon path. A head-pose prediction measurement and a passthrough test therefore observe different inputs even when they use the same headset.[3]
Software tools
Software timing tools help identify processing costs and missed deadlines. Their metric names require careful interpretation:
| Tool or metric | What it describes | Important limit |
|---|---|---|
| Oculus Debug Tool and PC Performance HUD | Application timing, compositor timing and latency estimates | The documented App Tracking to Mid-Photon interval begins at the tracking-state API call, not at physical motion. TimeWarp to Mid-Photon uses the later timewarp pose, while Flip to Photon - Start concerns presentation and the first scanline.[27] |
| SteamVR Advanced Frame Timing | Application and compositor CPU/GPU timing, missed presentations and reprojection | Frame timing describes parts of the processing path and scheduling; it is not an external comparison of physical movement with emitted light.[26] |
| Game-engine profiling | Timing of application loops and rendering work | The loop in which tracked data drives an interaction can change its delay. A displayed pose and a gameplay response may have different update timing.[5] |
| CloudXR Framework live network statistics | Pose-to-frame-received timing and its variation | These metrics exclude subsequent dequeue, submission, reprojection and display time, and differ from an external end-to-end measurement.[44] |
Cognitive measurement
Gruen and colleagues' 2020 study compared response times with and without video see-through HMDs and checked the inferred delay against electronic-clock measurements. Sixteen participants performed a rapid button-release task after a displayed circle changed color. The inference assumes that a person's cognitive contribution stays sufficiently consistent across the compared conditions. It offers a way to study timing while a person uses the device, but human variability and baseline-display latency limit interpretation.[3]
Interpreting performance measurements
A published number is most informative when its endpoint and operating conditions are available. Comparisons need to distinguish physical delay from predicted visual response, startup from continuous movement, and minimum values from means or distributions. A test of one movement or display location cannot by itself establish the delay of every XR interaction.[2][5]
Dated measurement examples
| Publication or test | Reported result | Scope and limits |
|---|---|---|
| Warburton and colleagues, online 2022, journal volume 2023 | Mean delays across tested setups of 21-42 ms at movement onset and 2-13 ms during continuous movement | Controller translation with HTC Vive, Rift CV1, Rift S and Index; Unity 2019.4 LTS and SteamVR 1.14.16. Prediction affected the trajectory. Not rotational head latency.[5] |
| Abdlkarim and colleagues, 2023 | Mean absolute hand tracking delay of 45.0 ms during reaching | Eight participants; fixed Quest 2 running build 33.0, Unity and Link, with a 7 ms Qualisys reference correction. Data delay rather than display photon timing.[7] |
| OptoFidelity passthrough test, 14 February 2024 | Approximately 11 ms for Apple Vision Pro; HTC Vive XR Elite, Meta Quest 3 and Meta Quest Pro in the 35-40 ms range | Manufacturer of measurement equipment reported a minimum photon-to-photon value using an LED target and BUDDY 6. Minimums were deliberately selected; this is not a mean, current ranking or virtual-content MTP figure.[9] |
| OptoFidelity angular test, 15 February 2024 | Separate axis-wise latency medians and variation | Robot encoder compared with virtual markers during sinusoidal rotation. This measured virtual-content angular response, distinct from the preceding camera passthrough test.[45] |
| Meta engineers, SID Display Week 2025 | Quest 3S motion-to-photon latency 14% better than Quest 2 and 8% better than Quest 3 | Manufacturer's relative figures from display, backlight and rendering optimization; the paper states no absolute value or test method.[24] |
Reporting latency and jitter
Repeated measurements can reveal irregular spikes or periodic behavior that an average hides. The 2020 Stauffert review recommends going beyond a single mean when describing latency, including its variability and observed timing patterns. This also helps distinguish a consistently delayed response from intermittent missed deadlines.[1]
| Report item | Why it matters |
|---|---|
| Input and output endpoints | Head rotation, controller motion, tracking-data arrival and camera passthrough are different paths. |
| Movement profile and analysis | Sudden onset, continuous motion and cross-correlation can give different estimates with prediction. |
| Hardware and software versions | Engine, runtime, drivers, refresh mode and processing configuration affect the path. |
| Output location and instrumentation | Scanline location, capture interval and synchronization affect the timing estimate. |
| Distribution and time series | Mean, spread, percentiles, spikes and periodicity describe behavior hidden by one summary value. |
| Prediction and reprojection behavior | A corrected viewpoint can be newer than object animation or gameplay feedback. |
These reporting considerations follow the measurement analyses of Friston and Steed, Warburton and colleagues, and Stauffert and colleagues. Percentile statistics should also identify which interval was sampled: for example, a streaming system's pose-to-frame-received percentiles leave out display presentation.[2][5][1][44]
Special considerations
Differences between VR and AR
In optical see-through AR, the real environment is viewed through optics without a camera-to-display pipeline. Delayed virtual graphics can visibly slip relative to that direct view during head motion. The required alignment depends on motion and the tolerance for spatial error, so a single 5 ms figure does not describe every optical AR task. The low latency display research described above addresses this registration problem through late correction within display hardware.[43]
Azuma's 1997 survey gave a worked example of the scale involved. With a typical end-to-end lag of 100 ms and a moderate head rotation of 50 degrees per second, the angular dynamic error is 5 degrees, which at a 68 cm arm length is a registration error of almost 60 mm. At the same rotation rate, he wrote, system lag must be 10 ms or less to keep angular error below 0.5 degrees, while scanning out a 60 Hz frame buffer alone takes 16.67 ms.[14]
Video passthrough adds camera acquisition and image presentation to the real-world view. Its delay can be measured independently of virtual-content viewpoint updates. Rendering and warping may keep a virtual object responsive to head rotation while the captured image of a real object remains older. The endpoint distinction is demonstrated by separate optical-camera and virtual-marker tests, rather than by combining their numbers into one headset latency.[9][45]
Video see-through also allows the two delays to be matched. Azuma described adding delay to the camera video so that the real-world view stays synchronized with the slower virtual imagery; this reduces conflicts between real and virtual objects, but both are then delayed, and the added delay must change as scene-generation time varies.[14]
Network and cloud rendering
NVIDIA CloudXR separates server rendering from client display. Pose information travels toward the renderer and video returns to the client, with encoding, transport and decoding contributing work along the path. High network bandwidth does not remove these processing and synchronization steps.[46]
CloudXR Framework's statistics distinguish pose transmission, frame receipt, dejitter buffering and submission. Its pose-to-frame-received latency is therefore broader than a simple network ping but narrower than physical motion-to-photon. NVIDIA's network guide recommends inspecting high-percentile timing and jitter, while its quality-monitor documentation explicitly excludes display and reprojection time from the network quality interval. These vendor metrics are diagnostics for the documented streaming system, not universal comfort limits.[44][47]
Local wireless PC VR streaming also adds encoding, radio transmission and decoding to the path. Steam Frame includes a wireless adapter that provides a direct connection between the PC and the headset; Valve's developer setup guide says streaming over home Wi-Fi "tends to be more congested and less reliable" than that direct connection.[48] In Road to VR's September 2026 review, Scott Hayden wrote that with the dedicated adapter the average user would not be able to tell wireless PC streaming apart from a tethered connection by latency or fidelity, and also reported that the headset lost contact with the adapter quickly when he played farther from the PC. These are a vendor recommendation and a reviewer's impression, not measured latency values.[49]
Multimodal latency
Haptic, audio and visual responses have separate output paths. Reducing one delay may leave a mismatch between modalities. In a 2019 research account, a haptic-glove team measured physical taps, actuator vibration, headphone output and screen light using synchronized recordings. The team described how changing audio affected the apparent responsiveness of the haptic feedback; it treated its early three-modality perceptual results as preliminary.[50]
Di Luca and Mahnan's associated two-modality study tested nineteen participants tapping a virtual cube while seeing an articulated hand and receiving fingertip vibration. Visual-haptic simultaneity depended on the order of the signals, and participants tolerated different asynchronies when vibration followed rather than preceded visual contact. The experiment measured the timing of visual and vibrotactile feedback in that tapping task.[10]
Research directions
Low latency research includes prediction, display-side correction and gaze-dependent rendering, each with a different error measure. The most relevant question can be spatial registration, response at a sudden movement, or the visibility of a quality change after an eye movement.[4][43][8]
For example, Albert and colleagues' 2017 foveated-rendering study tested nine participants in an HTC Vive with an SMI eye tracker, comparing three image degradation techniques and several high-quality region sizes. Their HMD experiment found no significant difference among the 0, 10, 20 and 40 ms added-delay conditions. The 80 and 150 ms conditions reduced the acceptable amount of foveation in comparisons with several shorter-delay conditions. They suggested an eye-to-image latency of approximately 50-70 ms could be tolerated under the tested conditions. This concerns detection of foveation artifacts in their scene, not permission to delay head-view updates by that amount or to treat all peripheral content as temporally insensitive.[8]
See also
References
- ↑ 1.0 1.1 1.2 1.3 Jan-Philipp Stauffert; Florian Niebling; Marc Erich Latoschik (2020-11-26). "Latency and Cybersickness: Impact, Causes, and Measures. A Review". Frontiers in Virtual Reality 1: 582204. doi:10.3389/frvir.2020.582204. Retrieved 2026-10-12.
- ↑ 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 Sebastian Friston; Anthony Steed (2014). "Measuring Latency in Virtual Environments". IEEE Transactions on Visualization and Computer Graphics 20 (4): 616-625. doi:10.1109/TVCG.2014.30. Retrieved 2026-10-12.
- ↑ 3.0 3.1 3.2 Robert Gruen; Eyal Ofek; Anthony Steed; Ran Gal; Mike Sinclair; Mar Gonzalez-Franco (2020). "Measuring System Visual Latency through Cognitive Latency on Video See-Through AR devices". 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). pp. 791-799. doi:10.1109/VR46266.2020.00103. https://www.microsoft.com/en-us/research/wp-content/uploads/2020/01/ieee_vr_2020___latency.pdf. Retrieved 2026-10-12.
- ↑ 4.0 4.1 4.2 4.3 4.4 Steven M. LaValle; Anna Yershova; Max Katsev; Michael Antonov (2014). "Head Tracking for the Oculus Rift". 2014 IEEE International Conference on Robotics and Automation (ICRA). pp. 187-194. doi:10.1109/ICRA.2014.6906608. https://lavalle.pl/anna/papers/LavYerKatAnt14.pdf. Retrieved 2026-10-12.
- ↑ 5.0 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 Matthew Warburton; Mark Mon-Williams; Faisal Mushtaq; J Ryan Morehead (2023). "Measuring motion-to-photon latency for sensorimotor experiments with virtual reality systems". Behavior Research Methods 55 (7): 3658-3678. doi:10.3758/s13428-022-01983-5. Retrieved 2026-10-12.
- ↑ 6.0 6.1 6.2 6.3 6.4 Michael Antonov (2015-03-02). "Asynchronous Timewarp Examined". Oculus developer blog. https://developers.meta.com/vr/blog/asynchronous-timewarp-examined/. Retrieved 2026-10-12.
- ↑ 7.0 7.1 7.2 7.3 Diar Abdlkarim; Massimiliano Di Luca; Poppy Aves; Mohamed Maaroufi; Sang-Hoon Yeo; R. Chris Miall; Peter Holland; Joeseph M. Galea (2023). "A methodological framework to assess the accuracy of virtual reality hand-tracking systems: A case study with the Meta Quest 2". Behavior Research Methods. doi:10.3758/s13428-022-02051-8. Retrieved 2026-10-12.
- ↑ 8.0 8.1 8.2 Rachel Albert; Anjul Patney; David Luebke; Joohwan Kim (2017). "Latency Requirements for Foveated Rendering in Virtual Reality". ACM Transactions on Applied Perception 14 (4): Article 25. doi:10.1145/3127589. Retrieved 2026-10-12.
- ↑ 9.0 9.1 9.2 9.3 Tommi Björk (2024-02-14). "Apple Vision Pro Benchmark Test 1: See-Through Latency, Photon-to-Photon". OptoFidelity. https://www.optofidelity.com/insights/blogs/apple-vision-pro-benchmark-test-1-see-through-latency-photon-to-photon. Retrieved 2026-10-12.
- ↑ 10.0 10.1 Massimiliano Di Luca; Arash Mahnan (2019). "Perceptual Limits of Visual-Haptic Simultaneity in Virtual Reality Interactions". 2019 IEEE World Haptics Conference (WHC). https://massimilianodiluca.info/publication/di-luca-2019/Di-Luca-2019.pdf. Retrieved 2026-10-12.
- ↑ 11.0 11.1 Mark R. Mine (1993-03-23). "Characterization of End-to-End Delays in Head-Mounted Display Systems". University of North Carolina at Chapel Hill, Department of Computer Science, technical report TR93-001. https://www.cs.unc.edu/techreports/93-001.pdf. Retrieved 2026-10-17.
- ↑ 12.0 12.1 Matthew Regan, Ronald Pose (1994). "Priority rendering with a virtual reality address recalculation pipeline". Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '94), ACM, pp. 155-162. doi:10.1145/192161.192192. https://doi.org/10.1145/192161.192192. Retrieved 2026-10-17.
- ↑ 13.0 13.1 Marc Olano, Jon Cohen, Mark Mine, Gary Bishop (1995). "Combatting rendering latency". Proceedings of the 1995 Symposium on Interactive 3D Graphics (I3D '95), ACM. doi:10.1145/199404.199407. https://doi.org/10.1145/199404.199407. Retrieved 2026-10-17.
- ↑ 14.0 14.1 14.2 14.3 Ronald T. Azuma (1997-08). "A Survey of Augmented Reality". Presence: Teleoperators and Virtual Environments, vol. 6, no. 4, pp. 355-385. doi:10.1162/pres.1997.6.4.355. https://www.ronaldazuma.com/papers/ARpresence.pdf. Retrieved 2026-10-17.
- ↑ 15.0 15.1 Michael Abrash (2012-12-29). "Latency - the sine qua non of AR and VR". Valve blog (archived copy). https://web.archive.org/web/2013/http://blogs.valvesoftware.com/abrash/latency-the-sine-qua-non-of-ar-and-vr/. Retrieved 2026-10-17.
- ↑ 16.0 16.1 16.2 Bruce Sterling (2013-02-24). "John Carmack's Latency Mitigation Strategies". WIRED. https://www.wired.com/2013/02/john-carmacks-latency-mitigation-strategies/. Retrieved 2026-10-12.
- ↑ 17.0 17.1 Oculus VR (2013-09-25). "Latency Tester Pre-Orders Now Open". Oculus blog (Meta). https://www.meta.com/blog/latency-tester-pre-orders-now-open/. Retrieved 2026-10-17.
- ↑ Ben Gilbert (2013-09-25). "Oculus Rift latency tester now available for pre-order, knows better than your eyes". Engadget. https://www.engadget.com/2013/09/25/oculus-rift-latency-tester-pre-order/. Retrieved 2026-10-17.
- ↑ 19.0 19.1 Bernard D. Adelstein, Thomas G. Lee, Stephen R. Ellis (2003-10). "Head Tracking Latency in Virtual Environments: Psychophysics and a Model". Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 47, no. 20, pp. 2083-2087. doi:10.1177/154193120304702001. https://doi.org/10.1177/154193120304702001. Retrieved 2026-10-17.
- ↑ 20.0 20.1 Katerina Mania, Bernard D. Adelstein, Stephen R. Ellis, Michael I. Hill (2004-08-07). "Perceptual sensitivity to head tracking latency in virtual environments with varying degrees of scene complexity". Proceedings of the 1st Symposium on Applied Perception in Graphics and Visualization (APGV 2004), ACM, pp. 39-47. doi:10.1145/1012551.1012559. https://doi.org/10.1145/1012551.1012559. Retrieved 2026-10-17.
- ↑ 21.0 21.1 Dean Beeler; Ed Hutchins; Paul Pedriana (2016-11-10). "Asynchronous Spacewarp". Oculus developer blog. https://developers.meta.com/vr/blog/asynchronous-spacewarp/. Retrieved 2026-10-12.
- ↑ 22.0 22.1 "Introducing SteamVR Motion Smoothing". Steam News (SteamVR). Valve. 2018-11-27. https://steamcommunity.com/games/250820/announcements/detail/1705071932992003492. Retrieved 2026-10-17.
- ↑ 23.0 23.1 23.2 Jian Zhang, Loren McQuade, Neel Bedekar (2020-12-07). "Bringing Phase Sync to Mobile VR". Meta Horizon OS Developers blog. https://developers.meta.com/horizon/blog/bringing-phase-sync-to-mobile-vr/. Retrieved 2026-10-17.
- ↑ 24.0 24.1 24.2 Jie Xiang, Ruidong Zhu, Linghui Rao, Yongmin Park, et al. (2025). "Quest 3S Immersive Display with High Visual Fidelity". SID Symposium Digest of Technical Papers (Display Week 2025), invited paper 39-4, pp. 523-525. Society for Information Display. https://sid.org/Portals/sid/Files/SID-Digest/docs/39-4.pdf. Retrieved 2026-10-17.
- ↑ 25.0 25.1 25.2 25.3 Daniel Wagner (2018-08-20). "MOTION TO PHOTON LATENCY IN MOBILE AR AND VR". DAQRI on Medium. https://medium.com/@DAQRI/motion-to-photon-latency-in-mobile-ar-and-vr-99f82c480926. Retrieved 2026-10-12.
- ↑ 26.0 26.1 26.2 "SteamVR/Frame Timing". Valve Developer Community. 2024-01-21. https://developer.valvesoftware.com/wiki/SteamVR/Frame_Timing. Retrieved 2026-10-12.
- ↑ 27.0 27.1 "Performance Head-Up Display". Meta Horizon OS Developers. https://developers.meta.com/vr/documentation/native/pc/dg-hud/. Retrieved 2026-10-12.
- ↑ 28.0 28.1 "The OpenXR Specification: xrWaitFrame, XrFrameState and xrLocateViews". Khronos OpenXR Registry. https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#xrWaitFrame. Retrieved 2026-10-12.
- ↑ 29.0 29.1 29.2 Freddi Jeffries (2016-03-01). "Virtual Reality – Blatant Latency and how to Avoid it". Arm Community. https://developer.arm.com/community/arm-community-blogs/b/mobile-graphics-and-gaming-blog/posts/virtual-reality-blatant-latency-and-how-to-avoid-it. Retrieved 2026-10-12.
- ↑ 30.0 30.1 Jason Jerald; Mary Whitton (2009). "Relating Scene-Motion Thresholds to Latency Thresholds for Head-Mounted Displays". 2009 IEEE Virtual Reality Conference. pp. 211-218. doi:10.1109/VR.2009.4811025. https://pmc.ncbi.nlm.nih.gov/articles/PMC3095496/. Retrieved 2026-10-12.
- ↑ Robert S. Allison, Laurence R. Harris, Michael Jenkin, Urszula Jasiobedzka, James E. Zacher (2001). "Tolerance of temporal delay in virtual environments". Proceedings IEEE Virtual Reality 2001, pp. 247-254. doi:10.1109/VR.2001.913793. https://doi.org/10.1109/VR.2001.913793. Retrieved 2026-10-17.
- ↑ Michael Abrash (2014). "What VR Could, Should, and Almost Certainly Will Be within Two Years". Steam Dev Days 2014. https://media.steampowered.com/apps/steamdevdays/slides/vrshouldbe.pdf. Retrieved 2026-10-12.
- ↑ Thomas Waltemate; Irene Senna; Felix Hülsmann; Marieke Rohde; Stefan Kopp; Marc Ernst; Mario Botsch (2016). "The Impact of Latency on Perceptual Judgments and Motor Performance in Closed-loop Interaction in Virtual Reality". Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology (VRST '16). pp. 27-35. doi:10.1145/2993369.2993381. https://graphics.rocks/publications/2016-latency.pdf. Retrieved 2026-10-12.
- ↑ Michael Meehan, Sharif Razzaque, Mary C. Whitton, Frederick P. Brooks (2003). "Effect of latency on presence in stressful virtual environments". Proceedings IEEE Virtual Reality 2003, pp. 141-148. doi:10.1109/VR.2003.1191132. https://doi.org/10.1109/VR.2003.1191132. Retrieved 2026-10-17.
- ↑ Benjamin Watson, Neff Walker, William Ribarsky, Victoria Spaulding (1998-09). "Effects of Variation in System Responsiveness on User Performance in Virtual Environments". Human Factors, vol. 40, no. 3, pp. 403-414 (author copy on arXiv). doi:10.1518/001872098779591287. https://arxiv.org/pdf/2507.18085. Retrieved 2026-10-17.
- ↑ Jan-Philipp Stauffert; Florian Niebling; Marc Erich Latoschik (2018). "Effects of Latency Jitter on Simulator Sickness in a Search Task". 2018 IEEE Conference on Virtual Reality and 3D User Interfaces. pp. 121-127. doi:10.1109/VR.2018.8446195. https://downloads.hci.informatik.uni-wuerzburg.de/2018-ieeevr-latency-preprint.pdf. Retrieved 2026-10-12.
- ↑ 37.0 37.1 "Enable Late Latching". Meta Horizon OS Developers. 2026-04-14. https://developers.meta.com/vr/documentation/unreal/unreal-late-latching/. Retrieved 2026-10-12.
- ↑ 38.0 38.1 "Enable Phase Sync". Meta Horizon OS Developers. 2025-02-18. https://developers.meta.com/horizon/documentation/unity/enable-phase-sync/. Retrieved 2026-10-17.
- ↑ 39.0 39.1 "App Spacewarp". Meta Horizon OS Developers. 2024-08-16. https://developers.meta.com/vr/documentation/spatial-sdk/os-app-spacewarp/. Retrieved 2026-10-12.
- ↑ "App Spacewarp". Meta Horizon OS Developers. 2026-04-08. https://developers.meta.com/vr/essentials/app-spacewarp/. Retrieved 2026-10-12.
- ↑ Alissa McAloon (2018-10-18). "SteamVR now supports motion smoothing in beta". Game Developer. https://www.gamedeveloper.com/game-platforms/steamvr-now-supports-motion-smoothing-in-beta. Retrieved 2026-10-17.
- ↑ "Apple Vision Pro - Technical Specifications". Apple. https://www.apple.com/apple-vision-pro/specs/. Retrieved 2026-10-12.
- ↑ 43.0 43.1 43.2 43.3 Peter C. Lincoln (2017). "Low Latency Displays for Augmented Reality". University of North Carolina at Chapel Hill, doctoral dissertation, chapter 3. https://sreal.ucf.edu/wp-content/uploads/2018/02/dissertation_lincoln-op.pdf. Retrieved 2026-10-12.
- ↑ 44.0 44.1 44.2 "Network Test and Session Quality". NVIDIA CloudXR SDK documentation. https://docs.nvidia.com/cloudxr-sdk/latest/usr_guide/cloudxr_framework/network_quality.html. Retrieved 2026-10-12.
- ↑ 45.0 45.1 Tommi Björk (2024-02-15). "Apple Vision Pro Benchmark Test 2: Angular Motion-to-Photon Latency in VR". OptoFidelity. https://www.optofidelity.com/insights/blogs/apple-vision-pro-bencmark-test-2.-angular-motion-to-photon-latency-in-vr. Retrieved 2026-10-12.
- ↑ "NVIDIA CloudXR SDK Overview". NVIDIA CloudXR SDK documentation. https://docs.nvidia.com/cloudxr-sdk/latest/overview/overview.html. Retrieved 2026-10-12.
- ↑ "Network Setup". NVIDIA CloudXR SDK documentation. https://docs.nvidia.com/cloudxr-sdk/latest/requirement/network_setup.html. Retrieved 2026-10-12.
- ↑ "Setting up your Steam Frame for development". Steamworks Documentation. Valve. https://partner.steamgames.com/doc/steamhardware/steamframe/setup. Retrieved 2026-10-17.
- ↑ Scott Hayden (2026-09-16). "Steam Frame Review - Power-user's Playground". Road to VR. https://roadtovr.com/valve-steam-frame-review/. Retrieved 2026-10-17.
- ↑ Forrest Smith (2019-07-18). "Measuring the perception of latency with a haptic glove". Tech at Meta. https://tech.facebook.com/reality-labs/2019/7/measuring-the-perception-of-latency-with-a-haptic-glove/. Retrieved 2026-10-12.