Vision processing unit

Load video

YouTube

YouTube might collect personal data. Privacy Policy

ContinueDismiss

A vision processing unit or VPU is a processing unit that is responsible for image recognition and related computer vision tasks. The architecture of VPU is very different from the traditional CPU. Movidius VPUs, for example, combine an array of programmable vector processors, a set of fixed-function hardware accelerators for imaging and vision, and a shared on-chip memory fabric that keeps data movement to a minimum. Its power consumption when recognizing images and faces is far lower than that of regular CPUs; in a 2018 study of the Myriad 2, researchers at KTH Royal Institute of Technology, the Barcelona Supercomputing Center and Intel found that a multi-VPU setup achieved image-classification performance comparable to reference CPU and GPU implementations while reducing thermal design power by up to eight times. Later VPUs were also built to run neural network computations on the device itself; in 2016 Movidius said its Myriad 2 MA2450 could "perform complex neural network computations in ultra-compact form factors".

According to Rivas-Gomez and colleagues, the word "vision" in the name reflects the processor's original purpose, which was to accelerate computer vision applications on the "edge". For virtual and augmented reality hardware, the appeal of a VPU is that camera-based work such as eye tracking, hand tracking, depth sensing and positional tracking can run at the same time on a small, low-power dedicated chip. Movidius vision processors were used in Google's Project Tango augmented reality platform and later in the Intel RealSense T265 tracking camera, which runs its entire SLAM algorithm on a Myriad 2 VPU.

Reviewed 27 September 2026. Checked every specification, date, quote and attribution in the expanded text against the 19 cited sources (Intel, Movidius, Inuitive and RealSense documents, the Rivas-Gomez arXiv paper, Crossref metadata for Barry et al., and press coverage). About review dates.

Architecture

Intel, which acquired Movidius in 2016, describes Movidius VPUs as merging three architectural elements: an array of programmable VLIW (very long instruction word) vector processors with an instruction set tuned to computer vision and deep learning workloads; a collection of hardware accelerators for image signal processing, computer vision and deep learning inference; and a commonly accessible memory fabric that minimizes data movement on the chip. Brendan Barry and colleagues described the Myriad 2 in the 2015 IEEE Micro paper "Always-on Vision Processing Unit for Mobile Applications", which Rivas-Gomez and colleagues cite as a primary source for the chip.

Rivas-Gomez and colleagues summarize the Myriad 2 as a 28 nm co-processor that dissipates less than 1 W. Its programmable core is 12 vector processors called Streaming Hybrid Architecture Vector Engines (SHAVE), each with wide register files and several functional units controlled by variable-length long instruction word packets. The SHAVE processors exchange data through a shared, multi-ported 2 MB memory block called the Connection Matrix (CMX), made up of 16 blocks of 128 KB. Next to them sits a Streaming Image Processing Pipeline (SIPP) of hardware-accelerated kernels for common image operations, including tone mapping, a Harris corner detector, a histogram of oriented gradients (HoG) edge operator, and luminance and chrominance denoising. The manufacturer's claimed maximum theoretical performance is 1000 Gflops using FP16 arithmetic.

The third-generation Myriad X added a dedicated on-chip block for deep neural networks. Intel calls it the Neural Compute Engine and rates it at 1 TOPS (trillion operations per second) of deep neural network inference, based on its peak floating-point throughput; Intel's architectural calculation for the whole chip is more than 4 TOPS. The Myriad X has 16 programmable 128-bit VLIW vector processors, 16 MIPI lanes that can connect up to 8 HD resolution RGB cameras, over 20 hardware accelerators for tasks such as optical flow and stereo depth, and 2.5 MB of on-chip memory with up to 450 GB per second of internal bandwidth. In an October 2017 analysis, BDTI listed the Myriad X as a 16 nm TSMC part, against 28 nm for the Myriad 2.

History

Movidius and Project Tango

Movidius was a chip company headquartered in San Mateo, California, with a design center in Dublin, Ireland; SiliconANGLE described it in 2016 as a 10-year-old company with about 180 employees. Its first-generation Myriad 1 vision processor was inside the prototype phone that Google's Advanced Technology and Projects group revealed as Project Tango in February 2014. There it worked as a co-processor beside the main CPU, handling motion detection and tracking, depth mapping, and the recording and interpretation of spatial and motion data. TechCrunch reported that the Myriad 1 operated in the range of a couple of hundred milliwatts, compared with more than 1 W for competing 3D-sensing systems such as Microsoft's Kinect.

Myriad 2 and the Google agreement

The Myriad 2 MA2450 is a VPU by Movidius. On January 27, 2016, Google and Movidius announced that Google would source Movidius processors and the complete Movidius software development environment and would contribute to Movidius' neural network technology roadmap, and that Google would use the MA2450, Movidius' flagship Myriad 2 chip, in next-generation devices. TechCrunch reported that the agreement covered the Myriad 2 family, which included the MA2150 and MA2450 chips and their software development kits. Google has previously used the processors in Project Tango devices. In the announcement Movidius called the MA2450 the most powerful version of the Myriad 2 family, and Blaise Agüera y Arcas, head of Google's machine intelligence group, said: "By working with Movidius, we're able to expand this technology beyond the data center and out into the real world".

A month later, at Mobile World Congress 2016, Movidius showed the Myriad 2 to the press as a chip for AR and VR headsets. Road to VR reported that the chip fits into a 500 mW power envelope, and that a prototype running four demos at once from a single image sensor used, according to the Movidius team, only 22 percent of its array of 12 vector processors. Jack Dashwood, the company's marketing communications director, said: "The idea is end customers are looking for gesture, depth, eye tracking, all at the same time, and all very fast and at low latency." He gave an example split of the 12 processors: two for depth, two for positional tracking, one for eye tracking, and the rest depending on the customer. Dashwood said Movidius was working with several VR and AR headset manufacturers, which he did not name, and that products using the VPU might appear in late 2016 or early 2017.

Intel acquisition and Myriad X

Intel announced its purchase of Movidius on September 6, 2016. Intel did not disclose the price; SiliconANGLE reported an estimated value of $400 million. Josh Walden, head of Intel's New Technology Group, said that "Movidius' technology optimizes, enhances and brings RealSense capabilities to fruition", linking the deal to Intel's RealSense depth-camera line. At the time, Movidius chips were used for obstacle detection and avoidance in drones made by DJI and in Google's Tango augmented reality platform.

Intel launched the first-generation Neural Compute Stick, a USB accelerator built on the Myriad 2 MA2450, in July 2017. On August 28, 2017, it introduced the Myriad X, which it called the world's first system-on-chip shipping with a dedicated Neural Compute Engine, and listed drones, robotics, smart cameras and virtual reality among its target product categories. The Neural Compute Stick 2, based on the Myriad X, followed on November 14, 2018. At its AI Summit on November 12, 2019, Intel described a third-generation Movidius VPU code-named Keem Bay, which it said would deliver about 10 times the performance of the previous generation and launch in the first half of 2020 in PCI Express and M.2 form factors.

Discontinued standalone parts and integration into PC processors

Intel's product database lists the Myriad 2 Vision Processing Unit 4GB, the configuration with 4 GB of stacked memory that Rivas-Gomez and colleagues identify as the MA2450, as discontinued. Intel also discontinued the Neural Compute Stick 2: the last order date was February 28, 2022, technical support ran until June 30, 2023, and warranty support until June 30, 2024.

The architecture moved into Intel's PC processors instead. In May 2023, Intel said that every Meteor Lake client processor would include a VPU based on the Movidius technology. In Intel's plan the VPU takes "sustained AI and AI offload" work, while the CPU keeps simple low-latency inference jobs and the GPU takes jobs that need parallel throughput. In an April 2024 analysis of the shipping block, by then described as a neural processing unit (NPU), Chips and Cheese traced its design to Movidius' digital signal processors: SHAVE cores sit beside large multiply-accumulate arrays and handle machine learning steps that cannot be mapped onto those arrays.

Movidius VPU generations

Chip	Introduced	Published details
Myriad 1	Shown in the Project Tango prototype phone, February 2014	Co-processor for motion tracking and depth mapping; in the range of a couple of hundred milliwatts
Myriad 2 (MA2150, MA2450)	Intel lists a Q3 2016 launch for the 4 GB part	28 nm; 12 SHAVE vector processors; 2 MB CMX memory; less than 1 W
Myriad X	Announced August 28, 2017	16 nm TSMC; 16 SHAVE vector processors; Neural Compute Engine rated at 1 TOPS; more than 4 TOPS total; up to 8 HD cameras; 2.5 MB on-chip memory
Keem Bay	Described November 12, 2019	About 10 times the performance of the previous generation (Intel claim); planned for the first half of 2020

Applications in VR and AR

Tracking and mapping

The main VR and AR use of a VPU is inside-out tracking and scene understanding, which depend on processing several camera streams with low latency. The Intel RealSense T265 tracking camera is a documented example. It combines wide field-of-view stereo fisheye cameras (about 165 degree circular field of view, captured by monochrome global shutter sensors of about 800 pixel diameter), a Bosch BMI055 inertial measurement unit with a 200 Hz gyroscope and a 62.5 Hz accelerometer, and a Myriad 2 VPU. The VPU runs the whole SLAM algorithm on board, analyzes the stereo images and fuses all sensor information into 6DoF tracking, which the camera reports over USB at 200 Hz while using less than 1.5 W. RealSense's documentation notes that most AR and VR headsets, including the Microsoft HoloLens, Oculus Quest and Magic Leap One, use their own versions of visual-inertial odometry, and positions the T265 as a standalone SLAM system for AR and VR headsets, robots and drones.

Other vision processors for headsets

Other chipmakers sold dedicated vision processors for the same market. In September 2017 the Israeli company Inuitive introduced the NU4000, which it marketed for augmented reality, virtual reality, drones and robots and described as defining a "Vision Processor Unit" category. Inuitive stated that the chip has on-chip SLAM with key-point extraction at 120 fps from two cameras, a depth engine with 120 megapixels per second of throughput supporting stereo and structured light streams, a CNN processor of more than 2 TOPS within a total above 8 TOPS, connections for 6 cameras and 2 displays in a 7 x 8 mm package, and a time-warp engine that it said reduces motion-to-photon latency to 1 ms.

Some headset makers designed their own vision co-processors rather than buying a VPU. The first Microsoft HoloLens pairs a 14 nm Intel Atom "Cherry Trail" system-on-chip with a custom chip Microsoft calls the Holographic Processing Unit (HPU). Microsoft engineer Nick Baker described it at the Hot Chips conference in August 2016 as a 28 nm TSMC part with 24 Tensilica DSP cores in 12 clusters, 8 MB of SRAM and a power draw of under 10 W, which aggregates sensor data and handles the headset's environment sensing and gesture processing in hardware. The Register reported that the design was meant to spare the x86 CPU from extra processing of the information coming from the HPU.

See also

• Computer vision
• Movidius Myriad 2
• Project Tango
• Intel RealSense
• Inside-out tracking
• Simultaneous localization and mapping
• Visual-inertial odometry
• Machine learning