Edge computing

Edge computing is a distributed computing model that places computation and data storage close to where data is produced and consumed, rather than in a distant centralized cloud data center. By shortening the physical and network distance between an application and its server, edge computing reduces round-trip delay and the amount of traffic carried across wide-area networks, which makes it relevant to latency-sensitive workloads such as virtual reality (VR), augmented reality (AR), and the broader category of extended reality (XR).

In an XR context, edge computing is most often discussed as a way to offload heavy graphics rendering and scene processing from a lightweight headset or pair of glasses to a nearby server, an approach called split rendering or remote rendering. The headset keeps the time-critical tasks (head and controller tracking, final reprojection of the image) on-device, while a more powerful edge server renders the demanding 3D graphics and streams the result back over a fast wireless link. The goal is to give a small, low-power, untethered device access to graphics quality it could not produce on its own, without the wires that earlier high-end VR systems required.

Background and terminology

The general idea of moving computation toward the edge of the network grew out of content delivery networks (CDNs) in the late 1990s and 2000s, which cached web content on servers near users to cut load times. As those caching nodes were extended to run general application code, the term edge computing came into use to describe the broader architectural shift away from purely centralized clouds.

The version of edge computing most relevant to mobile XR is standardized as multi-access edge computing (MEC). The European Telecommunications Standards Institute (ETSI) created an Industry Specification Group for the concept in 2014 under the name Mobile Edge Computing, defining it as a way to provide IT and cloud-computing capabilities at the edge of the mobile network, inside or close to the radio access network. In March 2017 ETSI renamed the effort Multi-access Edge Computing, keeping the MEC abbreviation, to reflect that the work would also cover fixed broadband and Wi-Fi access in addition to cellular networks such as LTE and 5G. Placing MEC servers at or near cellular base stations is often described as a building block of 5G, because it lets operators host applications where the network can reach them in a few milliseconds.

Why latency matters for XR

The connection between edge computing and XR comes from the latency requirements of head-mounted displays. The relevant figure is motion-to-photon latency: the time between a user moving their head and the display updating to match. If this delay is too large, the virtual image lags behind the real motion, which breaks the sense of presence and can cause motion sickness.

Researchers and VR engineers converged on a target of roughly 20 milliseconds or less of motion-to-photon latency for comfortable VR. Michael Abrash, who led VR research at Valve and later at Meta's Reality Labs, argued that latency is essential to convincing VR and that AR may demand even tighter budgets, on the order of 15 ms or below, because virtual objects are seen directly against the real world where any lag is obvious. This budget is the constraint that makes a distant cloud impractical for the most timing-sensitive parts of XR: light traveling to a faraway data center and back, plus the rendering and encoding time, can easily exceed the whole budget on its own. Edge computing addresses this by keeping the server geographically close, so that the network portion of the delay stays small enough to leave room for rendering and display.

Split rendering and remote rendering

Split rendering divides an XR application's work between a client device and a server connected by a low-latency link such as 5G or Wi-Fi. Computationally heavy operations, principally rendering the 3D scene, are offloaded to an edge or cloud server, while latency-critical tasks such as head tracking, hand tracking, controller tracking, and the final warp of the image to the user's current head pose remain on the headset. Because the rendered frame is produced for a head pose that is already a few milliseconds old by the time it arrives, split-rendering systems use techniques such as late-stage reprojection on the device to correct for the head movement that occurred during the round trip.

A related academic line of work studies multi-tier designs that place rendering across both edge and cloud servers. Mehrabi, Siekkinen, Kamarainen, and Yla-Jaaski described a Multi-Tier CloudVR system that leverages edge computing for remote-rendered virtual reality, balancing image quality against the delay of reaching servers at different distances. Surveys of MEC for immersive applications, such as the 2022 review by Yitong Wang and Jun Zhao, frame edge computing as a way to relieve the limited computation, storage, and battery of XR devices, and explicitly point to Qualcomm's Boundless XR and NVIDIA's CloudXR as example architectures.

Use in VR and AR

Qualcomm: split rendering and Boundless XR

Qualcomm has promoted split rendering as the route to slim AR glasses, under the banner of Boundless XR. Its early AR reference design, based on the Snapdragon XR1 chip, used a wired connection between a smartphone and the glasses so that rendering could be divided between the two devices. On 20 May 2022 the company introduced a Wireless AR Smart Viewer reference design built on the Qualcomm Snapdragon XR2 processor and the FastConnect 6900 chip, which keeps the glasses dependent on a host (a smartphone, PC, or processing puck) for most rendering but removes the wire, using Wi-Fi 6E with a stated latency of less than 3 ms between the headset and the host. Qualcomm's longer-term framing of Boundless XR extends this to 5G, where the host is replaced by an edge or cloud server, so that photorealistic rendering can be streamed to a lightweight headset over a mobile network.

To support tethered AR development, Qualcomm announced the Snapdragon Spaces XR Developer Platform at AWE in November 2021 and made it generally available in spring 2022. Snapdragon Spaces let developers build head-worn AR experiences that run across a phone and connected smart glasses. Qualcomm has since directed developers to migrate from Snapdragon Spaces to Google's Android XR platform, providing migration guides for its hand-tracking and interaction components.

NVIDIA CloudXR

NVIDIA CloudXR is a software framework that renders VR and AR content on a GPU-equipped server and streams the result to a client, including standalone headsets, Android devices, and Windows PCs. NVIDIA announced it at MWC Los Angeles in October 2019, with support for SteamVR and OpenVR applications so that existing PC VR titles could be streamed without modification. Commentary on the announcement noted that, because low latency depends on the physical distance between the data center and the headset as much as on the network, having the rendering server within range of an edge computing node is as important to streamed XR as 5G itself. CloudXR remains an active product, and NVIDIA has continued to develop it, including a browser-based CloudXR.js client SDK for streaming XR directly to device browsers.

Telecom and cloud edge platforms

Network operators and cloud providers have positioned MEC as infrastructure for XR. Ericsson described a system combining a Snapdragon XR2 5G headset, NVIDIA RTX GPUs running CloudXR on the edge, and its own distributed 5G core, with the user-plane function placed as close to the user as possible to meet VR's latency requirements. Amazon Web Services offers remote rendering for AR through AWS Wavelength, which embeds GPU-equipped compute inside carriers' public 5G networks. AWS states that remote rendering of this kind requires less than 20 ms of round-trip latency to the XR device, with roughly 10 Kbps of uplink and 20 Mbps of downlink per device, and cites a deployment in which Holo-Light's rendering software processed more than 100 million polygons at 40 to 60 frames per second, far beyond what a self-contained pair of data goggles could handle on its own.

Limitations

Offloading rendering across a network adds delay that a fully on-device system does not have, and any added latency between the time a head pose is captured and the time the matching image is displayed has to fit inside the XR latency budget. Split-rendering systems therefore depend on a fast, reliable connection; congestion, signal drops, or distance to the edge server can push latency past the comfortable threshold and reintroduce the lag and discomfort the approach is meant to avoid. These constraints have kept fully cloud-rendered consumer XR limited compared with standalone devices: modern standalone VR and standalone AR headsets render on-device, while edge and split rendering are used mainly for tethered AR glasses, enterprise visualization, and experimental 5G deployments.