Pupil swim

This page is a stub, please expand it if you have more information.

See also: Fresnel lens, Pancake lens and Eye tracking

Pupil swim is an optical artifact in virtual reality (VR) and augmented reality (AR) head-mounted displays (HMDs) in which the rendered scene appears to warp, stretch, or wobble as the eye moves or rotates behind the lens. It arises because the geometric distortion introduced by a viewing lens changes with the position of the eye's pupil in the eye box, while the software distortion correction applied to the image is usually computed for a single fixed, nominal eye position. When the pupil moves away from that nominal position, the residual distortion no longer cancels, so straight lines in the virtual world bend and objects seem to swim.

The effect is one of several geometric distortions that make a static virtual environment feel unstable when the user looks around. A perceived discrepancy between the visual motion of the scene and the user's actual eye and head motion is a recognized trigger for simulator sickness. Pupil swim can be reduced by careful optical design, but completely removing it generally requires eye tracking combined with dynamic distortion correction, which updates the correction in real time to match the measured position of each pupil.

Cause

VR optics magnify a small display panel placed a few centimeters from the eye. To fill a wide field of view this magnification is strong and is not uniform across the lens, so the lens introduces substantial geometric distortion (typically chromatic and pincushion or barrel distortion). HMD software counteracts this by pre-warping the rendered image with the inverse distortion, so that after the light passes through the lens the user sees a geometrically correct scene.

That pre-warp is exact only for one assumed pupil location, the point for which the lens and the correction were designed. The human eye does not stay fixed: it rotates in its socket to look around, and the head can also shift the eye relative to the optics. As the gaze angle changes, the optical path between the pupil and the lens changes, and the geometric distortion the lens applies changes with it. The patent literature describes this as a "ray crossing point" that floats, or swims, across the lens, producing a movement of the exit pupil and a "change of a visible image distortion as the gaze angle of the eye changes, which may cause eye strain and discomfort." Because the rendered correction is static while the real distortion varies with eye position, a mismatch remains, and that mismatch is perceived as pupil swim.

The size of the effect depends on the slope of the distortion curve, that is, how quickly distortion changes with field angle. It is largest toward the edge of the lens, where distortion becomes strongly non-linear and software correction matches the physical lens least well; the Rotlex optics group describes the result as a static room that "feels like it is made of jelly."

Relation to lens type

Pupil swim is a property of the viewing optic, and different lens architectures exhibit it to different degrees.

Fresnel lens designs, used in many first- and second-generation consumer headsets such as the Oculus Rift CV1, collapse a thick lens into concentric grooves to save weight, but they still distort the image as a function of gaze and can show pupil swim even when the lens is optimized for spot size across viewing angles. Pancake lens optics, which fold the light path with a partially reflective surface and a reflective polarizer, are used in thinner modern headsets and can be designed to reduce field curvature and pupil swim; Oculus patent material notes that a pancake assembly's extra reflective surface can be used to lower pupil swim relative to a purely refractive lens with only two optical surfaces, though pupil swim can still appear in a well-optimized pancake design. No common HMD lens type is inherently free of the artifact.

Local pupil swim

A related defect, distinct from the broad warping described above, is local pupil swim, also reported as "local ripples" or an "orange peel" effect. Here small regions of the virtual image distort locally as the head moves, rather than the whole scene warping smoothly. In a 2023 paper in the Journal of the Society for Information Display, researchers from Meta (Jia, Chan, Lian, and Rio) characterized local pupil swim as a common visual defect in VR and AR HMDs and presented a method to identify its root cause and quantify its perceptual impact using a combination of optical simulation, measurement, and perceptual modeling.

Mitigation

There are two broad approaches to reducing pupil swim.

The first is optical. Lens designers optimize the lens so that distortion changes as little as possible across the range of gaze angles, for example by using a merit function that equalizes the distortion at different viewing angles so the residual error is similar wherever the eye looks. This makes the static software correction a closer match across the whole eye box. In 2015 Oculus co-founder Palmer Luckey said the company had "been able to greatly reduce" pupil swim through custom optical work, while acknowledging that optics alone do not eliminate it.

The second approach is computational and depends on eye tracking. Eye-tracked dynamic distortion correction (DDC) generates a fresh distortion correction every frame from the measured three-dimensional position of the pupil, so the pre-warp tracks the eye instead of assuming a fixed position. Because the artifact is driven by the changing pupil location, this is the only method that can fully correct it. DDC requires the system to know the pupil's position in three dimensions rather than only the gaze direction, which is more demanding than the eye tracking used for foveated rendering.

DDC is not trivial to deploy, because neither the eye tracker nor the optical distortion model is perfectly accurate (for instance because of manufacturing tolerances), and an over-aggressive or laggy correction can introduce its own motion. The 2022 SIGGRAPH study by Guan and colleagues at Meta Reality Labs built a VR display simulator that reproduces gaze-contingent distortion and ran user studies to find how accurate (in eye-position bias) and how low-latency an eye tracker must be for DDC to improve, rather than harm, visual comfort.

Status in products

As of 2026, real-time dynamic distortion correction for pupil swim has mostly been demonstrated in research prototypes rather than widely shipped in consumer headsets. Meta's Butterscotch Varifocal research prototype, shown in 2023, used two eye-tracking cameras per eye and applied dynamic distortion correction; reviewers reported an image free of the geometric distortion and pupil swim familiar from many production headsets. Several shipping consumer and professional headsets include eye tracking (for example the Meta Quest Pro, Apple Vision Pro, and headsets from Varjo and Pimax), which is a prerequisite for DDC, but the primary defense against pupil swim in most consumer products has remained optical design, in particular the move to pancake lens optics in thinner headsets.