Graphics processing unit

A graphics processing unit (GPU) is a specialized electronic processor designed to accelerate the creation of images and to perform large numbers of mathematical operations in parallel. It was developed to take over the calculations needed to draw three-dimensional graphics, work that early personal computers handled on the central processing unit (CPU) until the demands of graphics-intensive software outstripped what a general-purpose CPU could deliver.

A GPU contains many small cores that each handle part of a task at the same time, in contrast to a CPU, which has a smaller number of cores optimized for sequential work. This parallel design suits graphics, where the colour of every pixel can be computed largely independently, and it also makes GPUs effective for other highly parallel workloads such as machine learning and scientific computing.

GPUs are central to virtual reality (VR) and augmented reality (AR) because a headset must draw two separate views, one per eye, at a high refresh rate and with low latency. The hardware that performs this work falls into two broad classes: powerful discrete GPUs from companies such as Nvidia and AMD that drive PC VR systems, and lower-power mobile GPUs integrated into system-on-chip processors that run standalone VR headsets.

History

The term "GPU" predates its modern meaning. Sony used it in 1994 for the 32-bit graphics processor, designed by Toshiba, in the original PlayStation games console. The term entered wide use in 1999, when NVIDIA launched the GeForce 256 and marketed it as "the world's first GPU".

Announced on 31 August 1999 and released on 11 October 1999, the GeForce 256 was built around Nvidia's NV10 chip. It moved transform and lighting calculations, previously run on the CPU, onto the graphics hardware itself. Nvidia defined a GPU as a single-chip processor with integrated transform, lighting, triangle setup and clipping, and rendering engines capable of processing a minimum of 10 million polygons per second, a threshold the GeForce 256 was built to meet. Integrating transform-and-lighting hardware into a consumer card brought capability that had been confined to expensive professional systems into mainstream price ranges.

Programmability followed. In October 2002 ATI introduced the Radeon 9700 (codenamed R300), the first Direct3D 9.0 accelerator with flexible pixel and vertex shaders that supported looping and floating-point math. Programmable shaders let developers write small programs that ran on the GPU to control how surfaces were lit and coloured, and they also opened the door to using the GPU for work unrelated to graphics. Nvidia's GeForce 8 series, released in the mid-2000s, turned the GPU into a more general computing device, and in 2007 Nvidia released CUDA (Compute Unified Device Architecture), described as the earliest widely adopted programming model for GPU computing. This use of graphics hardware for non-graphics computation is known as general-purpose computing on graphics processing units (GPGPU).

How a GPU works

A GPU achieves its speed through parallelism. Where a CPU has a few cores tuned for fast sequential execution, a GPU has hundreds or thousands of simpler cores that execute the same operation across many data elements at once. Rendering a frame involves computing the colour of each pixel, and because those computations are largely independent, the work can be split across many cores simultaneously. The same architecture that suits pixels also suits the large matrix and vector operations used in machine learning, which is why GPUs became common in that field.

GPUs are commonly divided into integrated and discrete types. An integrated GPU is built into the same package as the CPU and has no memory of its own; it shares the computer's system RAM. A discrete GPU is a separate card with its own dedicated video memory (VRAM), typically high-speed GDDR or HBM modules placed close to the GPU cores, connected to the rest of the system through an expansion slot. Discrete GPUs supply far more rendering power and are what high-end PC VR depends on; integrated and mobile GPUs trade raw performance for low power draw and small size, the constraints that matter inside a battery-powered headset.

Role in VR and AR rendering

Rendering for a head-mounted display is more demanding than rendering for a flat monitor. The system must produce two images at once, one for each eye, from slightly offset viewpoints, which is what gives the stereoscopic sense of depth. Both images must be drawn at a high and steady frame rate, generally at least 90 frames per second, and at the lowest possible latency, because dropped frames and lag between head movement and the displayed image are a leading cause of motion sickness. Each eye view is also rendered at high resolution to reduce the visible pixel grid, which multiplies the pixel count the GPU must shade. Because of these combined demands, a GPU intended for VR generally needs a high-performance graphics core and a substantial amount of video memory; guidance cited by Database Mart recommends at least 6 GB of VRAM, with 8 GB or more for complex scenes.

When a GPU cannot sustain the target frame rate, headset runtimes fall back on reprojection, which generates an intermediate frame from already rendered frames and the latest head-tracking data so the view still responds to motion. Oculus's Asynchronous Spacewarp (ASW) activates when an application is consistently dropping frames: it forces the application to run at half the headset's refresh rate, for example 36 frames per second on a 72 Hz headset or 45 on a 90 Hz one, and synthesizes every second frame by extrapolating the motion between previous frames. Because it works from colour information without scene depth, it can introduce visible artefacts, particularly if rendering performance falls further. A related technique, Stereo Shading Reprojection, reuses shaded pixels from one eye to help fill the other eye's image and then closes the gaps with an extra pass; Meta reported it saving more than 20% of GPU cost per frame in pixel-shader-heavy scenes.

Foveated rendering is another way to spend GPU resources more efficiently. When a headset can track the user's gaze, it renders the small region the eye is looking at in full detail and the periphery at lower detail, where the eye's acuity is low. PlayStation VR2 uses eye-tracked foveated rendering on the PlayStation 5, which Sony said can run scenes substantially faster by leveraging the console's RDNA 2 geometry engine.

GPU makers have built VR-specific rendering features directly into their hardware and software. Nvidia's VRWorks software development kit, expanded in 2016 with the Pascal architecture, added Single Pass Stereo, which lets the GPU run one geometry pass for both eyes rather than two, and Lens Matched Shading, which renders closer to the distorted shape the headset lens requires so the GPU shades fewer pixels that would later be discarded. Nvidia said Single Pass Stereo could roughly double geometry throughput and Lens Matched Shading could raise pixel-shading throughput by up to 50%.

Desktop and console GPUs for VR

PC VR headsets rely on discrete GPUs installed in a host computer. When the Oculus Rift CV1 shipped in 2016, Oculus set recommended graphics requirements of an Nvidia GeForce GTX 970 or an AMD Radeon R9 290 or better, alongside an Intel Core i5-4590 CPU and 8 GB of RAM. To help buyers identify capable machines, both GPU vendors ran certification programs in 2016: Nvidia's "GeForce GTX VR Ready" badge required at least a GTX 970 in desktops, while AMD offered "Radeon VR Ready Premium" and "Radeon VR Ready Creator" tiers, paralleling Oculus's own "Oculus Ready" PC program.

Game consoles use fixed GPU hardware. PlayStation VR2, released on 22 February 2023, runs on the PlayStation 5 and its custom AMD RDNA 2 GPU, and pairs eye tracking with foveated rendering so the console can drive the headset's per-eye OLED panels at 90 Hz and 120 Hz.

Mobile GPUs for standalone VR and AR

Standalone headsets carry the GPU inside the device, so they use the lower-power mobile GPUs found in system-on-chip processors. Most dedicated VR and AR headsets in this class use Qualcomm Snapdragon XR chips, whose graphics is handled by Qualcomm's Adreno GPU line. The Qualcomm Snapdragon XR2 Gen 2, built on a 4 nm process and first used in the Meta Quest 3 in 2023, includes an Adreno GPU that Qualcomm rated at up to 2.5 times the peak performance and double the performance per watt of the first-generation XR2. The chip supports per-eye displays up to 3K by 3K, according to Qualcomm, and provides hardware acceleration for tasks specific to headsets, including positional tracking, camera passthrough, SpaceWarp motion extrapolation, and super-resolution sharpening.

Some headsets use general-purpose mobile or desktop-class silicon rather than a dedicated XR chip. The Apple Vision Pro, released in 2024, runs on Apple's M2 processor, which has a 10-core GPU, supported by a second Apple-designed R1 chip that handles sensor input with a 12-millisecond photon-to-photon latency. The Vision Pro uses eye-tracked foveated rendering to limit full-resolution rendering to the region the user is looking at.

GPU classes used in VR and AR

Class	Where it runs	Example GPU / chip	Used by
Discrete desktop GPU	Host PC, separate card with own VRAM	Nvidia GeForce GTX 970, AMD Radeon R9 290	Oculus Rift and other PC VR headsets
Console GPU	Fixed hardware in a games console	AMD RDNA 2 (PlayStation 5)	PlayStation VR2
Mobile GPU in an XR chip	Inside the headset, system-on-chip	Adreno (Qualcomm Snapdragon XR2 Gen 2)	Meta Quest 3 and most standalone VR headsets
General-purpose mobile GPU	Inside the headset, system-on-chip	Apple M2 10-core GPU	Apple Vision Pro