Timewarp

See also: Asynchronous Spacewarp, Application SpaceWarp and Motion Smoothing

Timewarp (also written time warp or time warping, and also known as reprojection) is a technique in VR that warps the rendered image before sending it to the display to correct for the head movement that occurred after the rendering. The system renders a frame using a predicted head pose, then, a few milliseconds before the frame is shown, reads the latest head-tracking data and transforms the finished image so that it matches where the head actually is. This lowers the perceived latency of head motion. When the warp runs on its own thread, independently of the application's rendering, it is called asynchronous timewarp (ATW) and can also fill in display refreshes that the application missed, which reduces the judder caused by dropped frames.

The idea is older than the name. Research systems in the 1990s already remapped or re-rendered images after rendering to hide latency. John Carmack described "time warping" for head-mounted displays in February 2013, and Timewarp became a feature of the Oculus SDK. It was initially released in version 0.3.1 of the SDK in April 2014. Positional Timewarp, which also corrects for head translation, appeared in Oculus PC SDK 0.6.0 in May 2015.

In current systems the warp is performed by the VR runtime or the operating system's compositor. Meta describes Time Warp as one of the three main functions of the compositor in its headsets, Microsoft HoloLens has hardware-assisted reprojection and Windows Mixed Reality immersive headsets support depth-based reprojection, and the OpenXR standard lets applications submit depth so that runtimes can reproject more accurately. Different runtimes perform timewarp differently, especially when an application cannot keep up with the display.

Reviewed 17 October 2026. Every cited source re-opened (Antonov and Carmack archives, Oculus/Meta developer posts and docs, Road to VR, UploadVR, Valve GDC 2016 slides, Microsoft, OpenXR 1.1 spec, Apple docs, Sony coverage, paper abstracts and Mark 1999 report) and each claim checked against it. About review dates.

How it works

Carmack's 2013 description sets out the basic steps. After a frame is drawn with the best information available, the system does not display it directly. It fetches the latest input, generates updated view parameters and calculates a transformation that warps the rendered image to approximately where it would be with those parameters. Because this post-processing pass takes a fairly predictable amount of time, it can be scheduled late in the frame, which ordinary scene rendering cannot be.

The simplest form is orientation-only timewarp, which corrects only the rotational change in head pose. Michael Antonov, then chief software architect at Oculus, noted that it is a 2D warp, so it costs little performance when it is combined with the lens distortion pass. In Oculus SDK 0.3.1 the scene was re-projected "to a more recently measured orientation during the distortion rendering phase". J. M. P. van Waveren of Oculus VR likewise describes the time warp as a pass that both corrects the optical aberration of the headset lenses and transforms the stereoscopic images using the latest head-tracking data. Meta's later terminology calls this variant Orientation Timewarp (OTW, or 3-DoF TW).

Correcting for head translation as well requires per-pixel depth. Carmack pointed out that any screen pixel, together with its depth buffer value, can be converted back to a world-space position and re-transformed to a new screen location for a changed set of view parameters. Variants that do this are called positional timewarp or positional reprojection; they use depth maps that the application submits along with its color images (see below).

Carmack also noted two practical points. Two-dimensional overlays that must stay fixed on screen have to be composited after the warp, or they move incorrectly. A warp to a destination with the same field of view leaves some corners or edges undefined; rendering a larger field of view than needed avoids this, although he found that leaving the unrendered pixels black was "surprisingly unobtrusive" in a wide-field-of-view HMD.

How timewarp reduces latency

Without timewarp, the image on the display is based on the head pose that was sampled before rendering started. On a 60 Hz display a new frame appears every 16.7 milliseconds, so that pose is always somewhat out of date by the time the image is shown. Meta's compositor documentation states that an application "had to commit to a head pose prediction tens of milliseconds before it will be displayed", and that the compositor corrects for this just a few milliseconds before display. Timewarp therefore works together with predictive tracking: the application renders with a predicted pose, and the late warp corrects the rotational part of the prediction error.

With timewarp, the first steps are the same: the system samples the head pose and renders the image. Before the image is displayed, it samples the head pose again, and the rendered image is transformed to fit the newer data. The resulting image more accurately depicts the position of the head at the time of display than the image initially rendered. In the timing diagrams of Carmack's 2013 article, a late-scheduled time warp gave a latency of 2 to 18 milliseconds, compared with 12 to 28 milliseconds for late input sampling with view bypass, in which the newest input only updates the view parameters before rendering. Timewarp only works well for small corrections: Microsoft notes that the image updates work best with small changes and cannot completely fix motion parallax, and Meta calls rotational reprojection a good solution as long as eye translation is small.

UploadVR summarized the original, synchronous form by saying that "Standard Timewarp in itself did not actually help with framerate, nor was it intended to"; its purpose was to lower perceived latency. In the OpenXR API, the runtime returns a predicted display time for each frame, and the specification notes that an accurate prediction "is very important to avoid black pull-in by reprojection and to reduce motion judder in case the runtime does not implement a translational reprojection".

Asynchronous Timewarp (ATW)

Asynchronous Timewarp or ATW is when timewarp occurs on another thread in parallel (asynchronously) with rendering. Before every vsync, the ATW thread generates a new timewarped frame from the latest frame completed by the rendering thread. If the application has not submitted a new frame in time, ATW warps the last completed frame to reflect the head motion since it was rendered. The result is not exactly correct, but it is adjusted for head rotation, so it judders less than the unchanged frame that the display would otherwise show a second time. ATW fills in the missed frames and reduces judder. Van Waveren adds that asynchronous operation can improve overall GPU utilization, because the stereoscopic rendering no longer has to be synchronized with the display refresh cycle.

ATW first shipped on the Samsung Gear VR Innovator Edition, which had no positional tracking and mostly avoided near-field content, so many of the artifacts described below were reduced there. Bringing ATW to the PC was harder. It needs the GPU to interrupt (preempt) the application's rendering work shortly before scan-out; Antonov estimated that at 90 Hz, with a frame lasting about 11 milliseconds, preemption granularity of around 2 milliseconds or better was needed, while a single large draw call could take 10 milliseconds. Windows versions before Windows 8 preempted only at the coarse granularity of batch queues. Van Waveren's ATW test utilities, copyrighted by Oculus VR and licensed under the Apache License 2.0, list the requirements on hardware that cannot run graphics tasks concurrently: context priorities (queue priorities in the Vulkan version), fine-grained and low-latency priority-based task switching, accurate timing of the display refresh, and predictable latency and throughput of the warp itself.

Oculus announced ATW for the Oculus Rift at GDC 2016, and in a March 2016 post it stated that the Oculus PC SDK 1.3 implements ATW on Windows; ATW reached the PC with the consumer Rift launch that month. In its account, the work required changes to operating-system GPU scheduling, GPU command-processor microcode and kernel driver design in cooperation with Microsoft, NVIDIA and AMD; "Graphics driver VR extensions in the form of AMD's Liquid VR and NVIDIA's VRWorks were developed to support this." Oculus reported that early measurements of Rift launch titles without ATW showed apps missing about 5% of their frames, and that ATW filled in for most of these misses, "resulting in judder reduction of 20-100x". It also stated that ATW has no performance cost to the application and requires no code changes. UploadVR wrote that ATW's reliance on hardware features added in recent GPUs was one of the reasons the Rift does not support "GeForce 7-series cards or AMD cards predating the R9 series".

Timewarp and frame rate

Timewarp does not make the application render faster. What ATW changes is the rate at which the display receives updated images: the van Waveren paper says that, when run asynchronously, the time warp "can be used to increase the perceived frame rate and to smooth out inconsistent frame rates". In a game engine limited to 50 frames per second, a new frame is ready only once every 20 milliseconds, while a 60 Hz display refreshes every 16.7 milliseconds. With ATW, each refresh that has no new frame shows the last completely rendered frame (not the one still being rendered), warped with the latest head pose.

Oculus warned against relying on this. In 2015 Antonov wrote that many had hoped ATW would let engines render at a lower frame rate and "artificially fill in dropped frames", but that there are intrinsic limits, and he called ATW "pothole insurance" for occasional skipped frames. He recommended that ATW run at a fixed fraction of the game frame rate: at a 90 Hz refresh rate, an application should either hit 90 Hz or fall to 45 Hz, because an intermediate rate such as 65 Hz produces a constantly changing number of images on the retina, which is a worse artifact. Valve's Alex Vlachos quoted the same passage at GDC 2016 and called every-other-frame reprojection "a good tradeoff". Oculus's 2016 guidance was still to target a consistent 90 fps.

Research systems had already used image warping to raise the apparent frame rate. Mark, McMillan and Bishop reported in 1997 that re-rendering from previously computed views using rendered images and their Z-buffers "allows an order-of-magnitude increase in apparent frame rate" over conventional rendering alone for the small viewpoint changes that occur within a fraction of a second.

Limitations of timewarp

Timewarp works well in a rotational-only tracking system. When positional (translational) tracking is introduced, orientation-only timewarp does not work as well and presents a few problems. When it is used with positional tracking, only the rotational change is reflected in the resulting image. The positional change is ignored, causing positional judder on objects close to the viewer; even a pure head rotation translates the eyes. Antonov wrote that in an environment with near-field detail, translating the head makes the positional judder "nearly as bad as running without ATW", while mid-to-far-field scenes can be viewed without noticeable judder. This is addressed by Positional Timewarp.

Moving objects such as an animated character can also cause judder, because they are effectively frozen in the warped frames and appear as multiple images. How noticeable the judder is depends on the size, number and speed of the moving objects. Large, fast-moving objects can cause very noticeable judder. Specular highlights and reflections depend on the eye vector, so they can also judder after a warp, and parallax mapping and relief mapping show similar problems.

Vlachos listed further side effects of rotation-only reprojection in 2016. Judder comes from camera translation, animation and objects moved by tracked controllers. Rotation-only reprojection is eye-centered rather than head-centered, so it reprojects from the wrong location, and the inter-camera distance narrows during reprojection depending on the amount of rotation. He still considered it "good enough" as a last-resort safety net for missed frames.

Positional Timewarp

Positional Timewarp (PTW) extends timewarp to head translation as well as rotation. Meta defines it as "A version of TW reprojection that reduces the latency of both rotation and translation", also called 6-DoF timewarp. Antonov explained in 2015 that positional warping "needs to consider the depth of the original rendered frame", displacing parts of the image by different amounts. This creates dis-occlusion artifacts at object edges, where the original frame has no data, and the technique cannot easily handle translucency. Carmack had described the same problem in 2013: translation warping of realistic scenes produces smears or gaps along silhouette edges, and first-person hands and weapons are a prominent case. Vlachos called positional reprojection "still an unsolved problem" in 2016, because a traditional renderer stores only one depth per pixel, which makes particle systems and other translucent content hard to represent.

Road to VR's coverage of Oculus PC SDK 0.6.0 in May 2015 described positional timewarp as adding positional support to what could now be called "rotational" timewarp, and noted that it "is only designed to smooth over relatively small dips in FPS". Meta's developers later wrote that PTW was polished for Oculus Dash. In April 2019 Oculus released ASW 2.0 for the Rift and Rift S, which combines Asynchronous Spacewarp (ASW) with PTW. Applications must submit their depth buffer each frame, otherwise the runtime falls back to ASW 1.0; Unity and Unreal Engine submit depth by default with their Oculus integrations. Meta states that, unlike ASW, "PTW helps lower HMD motion latency as it executes during the final stage of composition".

Positional Timewarp and ASW are separate techniques. ASW, available since the Oculus 1.10 runtime in November 2016, builds on ATW: it tracks animation and movement within the scene to extrapolate synthetic frames, so that, according to Oculus, applications that run at 90 fps on recommended-spec systems can typically run at 45 fps on minimum-spec systems. ATW, in contrast, is capable of extrapolating "only a single thing: the user's head rotation". PTW corrects head motion using depth, while ASW corrects for motion inside the scene. ASW expands upon ATW, which only applies to the rotational tracking of the head.

On standalone Meta headsets, positional timewarp is more expensive than rotational reprojection and is only enabled for an application when Application SpaceWarp (AppSW) is enabled; the compositor then expects both a depth buffer and a motion vector buffer from the application.

Oculus Rift vs. SteamVR vs. PSVR

The runtimes of the first consumer headsets handled missed frames in different ways.

At the Oculus Rift launch, the Oculus runtime used asynchronous timewarp to pick up the latest available application frame and warp it before each 90 Hz display scan-out, whatever the application's own frame rate. ATW re-warps old frames; it does not interpolate new ones. Since the Oculus 1.10 runtime of November 2016, ASW turns on automatically during sustained frame drops and runs the application at 45 fps, half the Rift's refresh rate, while synthesizing the intermediate frames.

The SteamVR runtime first used interleaved reprojection. When an application consistently missed its frame rate, SteamVR ran it at 45 fps and generated every other frame. Valve's Vlachos described it at GDC 2016 as every-other-frame, rotation-only reprojection that an application can request through an OpenVR API hint, or that the VR system can enable automatically as a safety net; the application then gets about 18 milliseconds per frame to render. Valve added asynchronous reprojection to the SteamVR beta on 25 October 2016 and to the general release in November 2016. It initially required an NVIDIA GPU. AMD support followed in April 2017. In November 2018 Valve added Motion Smoothing, a feature similar to ASW; UploadVR wrote that with its release "Interleaved Reprojection became obsolete".

Both approaches have trade-offs. Asynchronous reprojection keeps a slight dip below 90 fps from halving the frame rate, but an intermediate frame rate produces the changing pattern of retinal images that Antonov described. A fixed half rate gives a stable pattern but halves the application's frame rate; Road to VR wrote that SteamVR's interleaved reprojection, by reprojecting every other frame, hurt positional movement and animation. UploadVR wrote that "Interleaved Reprojection actually had some perceptual advantages over asynchronous reprojection".

Sony planned reprojection for PlayStation VR before its launch. When it showed the updated Project Morpheus prototype in March 2015, Sony said that developers could still render their games at 60 fps while the headset output 120 fps, by creating another frame between two given frames, a technique Sony called reprojection. At GDC 2016 Sony engineer Chris Norden told developers that PSVR games must never drop below 60 fps. PlayStation VR2 (2023) offers developers native 120 Hz, native 90 Hz and a mode that reprojects 60 fps rendering to 120 Hz output; UploadVR's technical analysis named Horizon Call of the Mountain and Resident Evil 8 as titles using the reprojection mode and described its double-image or "ghosting" side effect. Some developers later replaced it: a March 2023 update to The Light Brigade replaced reprojection with native 90 Hz support on PS VR2. According to UploadVR, a November 2024 PS5 Pro patch for Gran Turismo 7 achieved smoother video playback in VR "through the VR Positional Reprojection function".

Other platforms

Standalone headsets and streaming

Meta's documentation for its standalone headsets describes Time Warp as the compositor rotating "the most recently completed frame to account for head rotation since the frame was generated". The system measures the application's render time and unblocks it just in time for composition. Meta notes that pure camera rotation introduces no parallax, which makes rotational reprojection effective and inexpensive, and that it is a good solution as long as eye translation is small.

Timewarp also helps hide the extra latency of streaming. Oculus Link relies on the PC runtime's ATW, which, in Meta's words, "guarantees that a frame is always ready for the upcoming display period", with the timing adjusted for the added streaming latency. UploadVR explained in 2020 that Link's latency is masked during head rotation because the headset warps the decoded frame in the direction the head rotated, while positional movement is not masked in the same way. In a 1999 University of North Carolina report, William Mark had already described a real-time system in which an image warper at the remote display location compensates for network latency.

HoloLens and Windows Mixed Reality

In augmented reality, the same correction keeps virtual content registered to the real world. Microsoft describes HoloLens as predicting where the user's head will be when an image is shown, then using hardware to adjust the rendered image for the difference between the predicted and actual head position; the adjustment works best for small changes and cannot completely fix motion parallax. Microsoft's developer documentation lists four kinds of reprojection:

Mode	How it works (per Microsoft)	Availability
Depth reprojection	The application submits its depth buffer every frame, and each part of the scene is stabilized independently based on its distance from the user	HoloLens 2 and immersive headsets
Planar reprojection	The application sets a stabilization plane each frame (for example with SetFocusPoint); content on that plane is the most stable	All Windows Mixed Reality platforms
Automatic planar reprojection	The system sets the stabilization plane using information in the submitted depth buffer	HoloLens (1st gen) and HoloLens 2
None	If the application does nothing, planar reprojection is used with a plane fixed 2 meters along the user's head gaze	Default

For desktop immersive headsets, Microsoft advises against setting a stabilization plane, because providing the depth buffer for per-pixel, depth-based reprojection gives better visual quality. Microsoft also defined an OpenXR vendor extension, XR_MSFT_composition_layer_reprojection, for passing extra reprojection information; the OpenXR registry now lists it as deprecated without replacement.

OpenXR

The Khronos Group's XR_KHR_composition_layer_depth extension lets applications submit depth images along with the color images in projection layers. The specification states that the runtime "may use this information to perform more accurate reprojections taking depth into account". The extension's contributors came from Oculus, Microsoft, Arm and Qualcomm.

Apple Vision Pro

Apple's visionOS render pipeline documentation for Apple Vision Pro does not use the term timewarp. It says that on Vision Pro "the system renders updates to the images the device displays in order to reposition the UI relative to changes in head position", and it tells Metal developers to query a new pose prediction for each frame immediately before encoding GPU work and to match the display refresh rate, usually 90 Hz.

History

Date	Event
1994	Matthew Regan and Ronald Pose present an address recalculation pipeline at SIGGRAPH, a display controller that performs viewport mapping for head orientation after rendering
1997	William Mark, Leonard McMillan and Gary Bishop publish post-rendering 3D warping, which uses depth to correct for viewpoint translation as well as rotation
February 2013	John Carmack describes late-scheduled "time warping" in "Latency Mitigation Strategies"
April 2014	Oculus SDK 0.3.1 introduces Timewarp
Late 2014	Asynchronous timewarp ships on the Samsung Gear VR Innovator Edition
March 2015	Oculus publishes "Asynchronous Timewarp Examined" on ATW's limits on PC
May 2015	Oculus PC SDK 0.6.0 adds positional timewarp
March 2016	ATW comes to the PC with Oculus PC SDK 1.3 and the consumer Oculus Rift
March 2016	Valve presents interleaved reprojection at GDC 2016
October-November 2016	SteamVR adds asynchronous reprojection for NVIDIA GPUs
November 2016	Oculus runtime 1.10 adds Asynchronous Spacewarp
April 2017	SteamVR asynchronous reprojection supports AMD GPUs
November 2018	SteamVR adds Motion Smoothing
April 2019	ASW 2.0 combines ASW with positional timewarp on the Rift
February 2023	PlayStation VR2 offers developers a 60 fps to 120 Hz reprojection mode

Research

Academic work on post-rendering warping predates consumer VR. Regan and Pose's 1994 address recalculation pipeline was "a graphics display controller specifically designed for use with head mounted virtual reality systems". Because it performed orientation viewport mapping after rendering, the head orientation did not need to be known accurately until less than a microsecond before the first pixel of an update frame was sent to the display, which kept perceived latency to head rotation minimal. Combined with image composition, it allowed different objects to be rendered at different rates, which the authors called priority rendering.

Mark, McMillan and Bishop (1997) showed that a pair of rendered images and their Z-buffers contain almost all the information needed to re-render from nearby viewpoints. They used McMillan and Bishop's image warping algorithm so that viewpoint translation as well as rotation could be corrected, and warped two reference images and composited the results to avoid occlusion artifacts. Mark's 1999 dissertation extended the approach to a system that renders every Nth frame conventionally and generates the in-between frames with an image warper; he noted that earlier work such as Regan and Pose's had taken the same general approach without per-pixel depth. In 2008, Ferdi Smit, Robert van Liere and Bernd Fröhlich described a multi-GPU VR architecture with a programmable display layer that generated intermediate display frames by per-pixel depth-image warping, replacing the default behavior of repeating application frames until an update is available.

J. M. P. van Waveren of Oculus VR presented a paper on ATW at the ACM VRST conference in 2016. It describes how the time warp operates as close as possible to the display refresh and transforms a stereoscopic pair of images from the view at the time of rendering to the correct view at the time of display, and discusses the trade-offs of implementing a fast, high-quality time warp with predictable latency and throughput on consumer hardware.

Reprojection has also been applied to video see-through AR. Jann Philipp Freiwald, Nicholas Katzakis and Frank Steinicke's Camera Time Warp (CamWarp, 2018) extends reprojection techniques to camera images in video see-through head-mounted displays to reduce the registration error between captured video and rendered virtual images. In their experiments, the results suggested that CamWarp could reduce discomfort and cybersickness symptoms for all tested camera configurations and improved speed and accuracy in a physical object-moving task.

See also

• Asynchronous Spacewarp
• Application SpaceWarp
• Motion Smoothing
• Motion-to-photon latency
• Predictive tracking
• Judder
• Latency
• Refresh rate