Late latching

Late latching is a latency-reduction technique for real-time graphics in which tracking data, such as the pose of a head-mounted display or a hand controller, is supplied to the rendering pipeline as late as possible before the GPU uses it, so that the frame is drawn with a newer pose than the one available when the application began preparing it. Oculus chief architect Atman Binstock described it in 2015 as feeding updated head-tracking pose data "into the GPU by a side channel" and having the GPU "latch" this data before executing the queued commands, where "latch" means atomically copying the constantly updated pose data to a private buffer for later use.

In virtual reality (VR), the technique addresses a conflict between GPU efficiency and motion-to-photon latency. Letting the CPU queue frames ahead of the GPU keeps both processors busy, but every queued frame carries a pose that grows older while it waits. Late latching keeps the queue-ahead structure while replacing the stale pose shortly before use. Meta documents late latching as an option for Unity and Unreal Engine apps on Meta Quest 2, Meta Quest 3, Meta Quest 3S and Meta Quest Pro, stating that it can remove "up to 1 additional frame worth of latency" in head and controller poses. Google's Android XR documentation for Unity describes the same idea as letting the head pose be updated late in the frame generation pipeline.

Late latching changes when pose data is consumed rather than how the pose is estimated, so it is distinct from predictive tracking. It is also separate from post-render image correction such as Timewarp, although the two are often combined.

Reviewed 21 October 2026. Checked every claim against Binstock's 2015 Oculus post (archived), NVIDIA's GDC 2015 VR Direct notes, AMD's LiquidVR release, AnandTech, Meta's Unity and Unreal late latching docs, UE4 and Phase Sync blog posts and v12/v23/v38 release notes, Unity and Android XR API docs, Olano et al. 1995, Kim et al. 2020 and US 11,660,535 B2. About review dates.

Background: pipelining and latency

Pipelined graphics systems gain throughput at the expense of latency. In a 1995 paper on head-mounted display latency at the University of North Carolina, Marc Olano, Jon Cohen, Mark Mine and Gary Bishop wrote that a system able to produce a new image every frame may have more than one frame of latency because of pipelining, and that "The classic problem with pipelining is that it provides increased throughput at a cost in latency." They described lag as especially visible in see-through displays, where virtual objects appear to swim while the real world moves without delay.

Binstock's 2015 post described the same trade-off on contemporary PCs. To keep the GPU fully used, the application runs ahead of it, and draw commands are buffered in several stages before execution; on a Windows machine of the time this could add up to three frames of latency in the graphics pipeline, not counting application pipelining or display latency. In his idealized 100 Hz example, a pose sampled at 0 ms reached the display at 30 ms. NVIDIA's Game Developers Conference 2015 talk noted that the Oculus SDK of the time avoided this by forcing a synchronous model, flushing and waiting on the GPU each frame. That kept latency low, but the application's render thread sat idle waiting for the right moment to submit the distortion and timewarp pass, and the GPU then went idle waiting for the next frame from the CPU.

How it works

Late-latched warp

The simplest form described by Binstock latches only the orientation used by the final rotation warp. The application queues a GPU command that, when executed, copies the latest tracked pose into a buffer, and the warp pass reads its orientation from that buffer rather than from a value fixed when the CPU recorded the frame. In his example timeline, the GPU latches the orientation at 17 ms and the warp uses it at 18 ms, bringing orientation latency back to about 3 ms while the CPU still runs a frame ahead.

NVIDIA's GDC 2015 presentation made the same point in terms of constant buffers: the timewarp head pose is "latched in" when render commands are submitted, so the render thread must otherwise wait before submitting timewarp. If the constants can be updated after submission, timewarp can be queued early and the render thread can move on to the next frame. The talk said this could be done in OpenGL 4.4 with a persistently mapped buffer, letting the CPU write new head poses directly into video memory, for example from a thread that updates them every millisecond. It added that Direct3D 12 also supports persistently mapped buffers, while Direct3D 11 does so only indirectly by violating the NO_OVERWRITE contract when mapping constant buffers.

Late-latched position and scene rendering

Late latching can also supply the position used to render the scene itself. Because the rendered image depends on the head position and there is no simple 2D warp that can correct position after rendering, the position has to be latched before the GPU starts the frame. In Binstock's example this happens at 9 ms, so position latency stays higher than orientation latency, which he considered a reasonable compromise because heads change position more slowly than they rotate.

This form is harder to integrate. The final view matrix is not available to the CPU when the engine records the frame, so CPU code cannot combine it into model-view matrices or use it for view-space lighting, and any shader that uses the view matrix or works in view space may need to be changed. Engines still need an approximate pose for culling.

Uniform buffer patching on Quest

Meta's Quest implementations take a different route. Instead of having shaders read a separate pose buffer, the engine samples head and controller poses at the end of the render thread, just before the command list is sent to the GPU, then recalculates every uniform buffer value derived from those poses and writes the new values into the same uniform buffer memory. Examples given by Meta are the model matrices of objects attached to the hand anchors and a new global view-projection matrix for the camera.

A 2019 Meta developer blog post by Jian Zhang, Loren McQuade and Cass Everitt explained the Vulkan details for Unreal Engine 4. The view uniform buffer is created as host-visible and host-coherent memory, mapped once, and then modified by the CPU at any time. For engines treated as a black box, the authors described guarding the update with Vulkan events (vkCmdWaitEvents before rendering and vkSetEvent after the buffer is re-updated). For the UE4 mobile renderer, which normally has no mid-frame flushing and a single queue, they used a simpler path that updates the buffer with new poses right before vkQueueSubmit, and aborts late latching for a frame if a mid-frame submission is detected. Because UE4 packs all draw-call data into one large uniform buffer, the implementation tracks and patches each view-matrix location; the authors measured a CPU cost of 0.1 to 0.3 ms for most apps they tested. Controller late latching works the same way at the graphics API level but has to walk each controller's transform hierarchy.

The same post argued that the technique helps more on Oculus Quest than on Oculus Rift because the Quest's Adreno 540 GPU uses a tiled architecture that needs the whole frame's draw call information before it starts, which raises CPU-to-GPU latency.

History

Binstock wrote in March 2015 that the basic idea of delivering input late into a pipelined graphics system "has been around forever", citing its use on several generations of game consoles and in hardware mouse cursors. His post said Oculus was pursuing late latching with Unity and Unreal and planned SDK support for custom engines, and that Oculus was sharing the work to encourage GPU makers such as AMD and NVIDIA to adopt it.

Both GPU makers presented related features at GDC 2015. NVIDIA's VR Direct talk, given by Nathan Reed, a developer technology engineer at NVIDIA, and Dean Beeler, a software engineer at Oculus, described late-latched timewarp constants and pointed to the Oculus blog post. AMD announced its LiquidVR SDK on 3 March 2015 with a feature called Latest Data Latch, which AMD described as "a programming mechanism that helps get head tracking data from the head-mounted display to the GPU as quickly as possible". AnandTech reported that Latest Data Latch and AMD's asynchronous shaders were meant to work together, with the latch grabbing the latest tracking data that the shader-based time warp then used.

For the standalone Quest, Oculus shipped an Unreal Engine 4 implementation as an experimental feature in the 1.41 release of its Unreal Engine integration, targeting UE4.23 with Vulkan; the December 2019 blog post asked why no published apps had yet enabled it. Meta's v12 release notes in December 2019 listed improvements to late latching, and the v23 notes in December 2020 said late latching was now automatically disabled when occlusion culling is active in UE4. In Unity, the Oculus XR Plugin documentation for version 3.4.1 listed late latching as an experimental Android setting, disabled by default, under active development and Vulkan-only. Meta's v38 release notes of March 2022 added the ability to enable late latching from the Unity Inspector tab. The API reference for version 4.4 of the plugin describes the setting as reducing "tracked rendering latency by updating head and controller poses as late as possible before rendering" and is still Vulkan-only.

Implementations

Platform or engine	Mechanism as documented	Requirements and notes
Oculus PC pipeline (2015 design)	GPU command latches the latest pose into a buffer used by the warp, and optionally by scene rendering	Position latching requires engine shaders to read the view matrix from the latched buffer
NVIDIA VR Direct (2015)	CPU thread keeps updating timewarp constants in a persistently mapped buffer after commands are submitted	Described for OpenGL 4.4 and Direct3D 12; Direct3D 11 only indirectly
AMD LiquidVR (2015)	Latest Data Latch, used together with asynchronous shaders for time warp	Part of LiquidVR SDK 1.0
Meta XR plugin for Unreal Engine	Poses sampled at the end of the render thread; pose-derived uniform buffer values rewritten in place	Multi-View and Vulkan required; not to be combined with occlusion culling in UE4 or with Emulated Uniform Buffer in UE5
Oculus XR Plugin and Meta XR (OpenXR) settings for Unity	Same uniform buffer approach; option "Late Latching (Vulkan)" in the Android settings, also available with OpenXR under the Meta Quest feature group	Multi-View and Vulkan required
Unity XR display subsystem	MarkTransformLateLatched marks a GameObject transform, and its descendants, to be updated with the latest VR pose before rendering is submitted to the GPU	Node types Head, LeftHand and RightHand
Android XR Extensions for Unity	Enabled at runtime through MarkTransformLateLatched with the Head node	Google states it reduces input latency "by close to a whole frame time"; no measurements are given on the page

Measured benefit

Meta's documentation gives the upper bound as one frame and explains that the real saving depends on how long the app's main and render threads take, since a "frame" on those threads is not guaranteed to last exactly 1/72 of a second at 72 frames per second. Developers can check the effect by comparing the Prd value in the VrApi log, which Meta describes as the render latency measured by the runtime, with and without late latching; Meta's example shows 45 ms without and 35 ms with it, a 10 ms saving.

The saving also interacts with Phase Sync, Meta's frame-timing feature for Quest apps. Phase Sync schedules the main and render threads to start just before the GPU needs the frame; in Meta's example of a simple app that spends 1 ms on the main thread and 3 ms on the render thread, late latching then reduces latency by only about 4 ms. Meta still describes the two features as complementary and recommends using both. A 2020 Meta blog post on Phase Sync said Phase Sync can save more latency when an app runs fast, while late latching helps more when an app has an expensive render thread. The 2019 UE4 post gave no single figure, saying the saving depends on the app's render thread workload, and listed the expected effects as less TimeWarp "black pulling", more spatially stable near-field objects and a better controller experience.

Limitations

Because only rendering receives the newer pose, the image can show a head or hand position that game code has not processed. Meta warns that this "could allow players to put their heads or hands through a wall for a frame" even when gameplay code prevents that collision, because physics and gameplay calculations still use the earlier gameplay pose. Binstock called this the deepest "rabbit hole" of the technique: rendering ends up running ahead of the game code. He also listed the difficulty of making late latching work reliably across operating systems and diverse hardware, and frame-timing details, as open problems.

NVIDIA's 2015 talk noted that late-latched timewarp constants remove many idle gaps in the CPU-GPU pipeline but still rely on the app rendering consistently at the display's refresh rate; if the app hitches, the headset shows a stuck or stuttering image. The talk presented asynchronous timewarp, which decouples the warp from the app's main rendering, as the next step. Meta's current implementations are tied to Vulkan and multiview rendering, and the Unreal version lists engine features it should not be combined with.

Research and uses outside VR

The idea of sampling input late and correcting the image after rendering has been studied for non-VR games. In a 2020 paper in the Proceedings of the ACM on Computer Graphics and Interactive Techniques, Joohwan Kim, Pyarelal Knowles, Josef Spjut, Ben Boudaoud and Morgan McGuire of NVIDIA adapted image-warping and simulation-warping techniques "from virtual reality" to a first-person aiming task with simulated remote-rendering latency. Their late-warp updated the image from late-latched player input; the authors chose not to evaluate input prediction because it adds error. With nine experienced players, a fixed 60 Hz frame rate and 80 ms of added latency, the warp methods removed up to 80% of the task-completion-time penalty.

All five authors are among the inventors on NVIDIA's US patent 11,660,535, "Hardware acceleration and event decisions for late latch and warp in interactive computer products", filed 30 July 2020 and granted 30 May 2023. The patent defines late latching as "sampling a user input or inputs later in the image generation/presentation pipeline" and states that a late-latch input can reduce input-output latency by 5 to 20 ms for both cloud and local gaming.

See also

• Motion-to-photon latency
• Latency
• Timewarp
• Asynchronous Spacewarp
• Predictive tracking
• Pose
• Single-pass stereo rendering
• Graphics pipeline