Toggle search
Toggle menu
5.1K
5.9K
229
50.6K
VR & AR Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Neural processing unit

From VR & AR Wiki
More actions

A neural processing unit (NPU) is a processor, usually a block inside a system on a chip (SoC), that is designed specifically to run neural network inference at low power. Qualcomm describes its NPU as "built from the ground-up specifically for accelerating AI inference at low power", with hardware for the scalar, vector and tensor math that neural network layers are made of.[1] Chipmakers use different names for the same class of block: Apple calls its version the Neural Engine,[2] Qualcomm the Hexagon NPU,[1] and Huawei used the name Neural Processing Unit for the block in its Kirin 970 smartphone chip in 2017.[3]

NPUs matter to virtual and augmented reality hardware because headsets and smart glasses run several machine learning models at the same time (hand tracking, eye tracking, depth sensing, speech recognition and object detection) inside tight power and heat limits.[4] The chips in the Meta Quest 3, the Ray-Ban Meta glasses and the Apple Vision Pro all include one.[5][6][7][8]

Reviewed 13 October 2026. Every cited figure, quote, date and attribution checked against the Qualcomm, Apple, Huawei and Microsoft sources, the press reports and the original research papers (authors, venues and DOIs). About review dates.

How it works

A neural network layer is mostly multiplications and additions on large arrays of numbers, followed by a non-linear activation function. Qualcomm's 2024 white paper sorts this work into scalar, vector and tensor math and argues that a general-purpose CPU or GPU in a battery-powered device has other "day jobs" (the operating system, games and apps), while the NPU "is built specifically for AI" and trades some ease of programmability for peak performance, power efficiency and area efficiency.[1] The same paper gives the reason dedicated tensor hardware saves energy: multiplying two N x N matrices reads 2N² values but performs 2N³ operations, so a tensor unit does about N operations for every value it fetches from memory, far more than scalar or vector units.[1]

The core of most NPUs is therefore a large array of multiply-accumulate (MAC) units next to on-chip memory, usually working at reduced numerical precision. Google's first Tensor Processing Unit, a datacenter inference chip described in 2017, is one example: its heart is a matrix unit of 65,536 8-bit MACs with a peak of 92 trillion operations per second (TOPS) and 28 MiB of software-managed on-chip memory.[9] Mobile NPUs follow the same idea at much lower power. The Hexagon NPU in the Qualcomm Snapdragon AR1 Gen 1 for smart glasses, for instance, has scalar, vector and tensor accelerators and supports INT4 and INT8 precision.[7] In Intel's Meteor Lake PC processors, the NPU 3720 splits its MAC array across two Neural Compute Engine tiles that can perform 4,096 INT8 MACs per cycle, and Movidius-derived SHAVE vector DSPs handle the steps the array cannot.[10]

Memory access is a central concern in these designs. In their 2014 paper on the DianNao accelerator, Tianshi Chen and colleagues wrote that recent convolutional and deep neural networks were "characterized by their large size" and designed their chip with "a special emphasis on the impact of memory" on performance and energy.[11] Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang and Joel Emer's tutorial and survey in the Proceedings of the IEEE (2017) covers these hardware platforms and the joint hardware and algorithm design used to cut the cost of deep neural network processing.[12]

Related processors

An NPU is distinct from a Vision processing unit (VPU), which began as a processor for camera and computer vision pipelines, and from the custom Holographic Processing Unit in the first Microsoft HoloLens. The categories overlap in practice: Intel's PC NPU is based on designs from Movidius, which began as flexible digital signal processors; Intel acquired Movidius in 2016 and added large arrays of MAC units.[10] In Qualcomm's chips the NPU sits next to other blocks: the AR1 Gen 1 also has an "AR-grade Engine for Visual Analytics" and a separate micro NPU in its Sensing Hub for audio and sensors.[7]

History

Research origins

In a paper at the 45th IEEE/ACM International Symposium on Microarchitecture (MICRO) in December 2012, Hadi Esmaeilzadeh, Adrian Sampson, Luis Ceze and Doug Burger used the name "neural processing unit" (NPU) for a low-power accelerator tightly coupled to a processor's pipeline. Their compiler trained a neural network to imitate a region of approximate code and then replaced that code with calls to the NPU, which the authors reported gave an average whole-application speedup of 2.3x and energy savings of 3.0x across their benchmarks.[13][14]

Accelerators aimed at deep neural networks themselves followed. The DianNao design from the Institute of Computing Technology in China and Inria reached 452 GOP/s in 3.02 mm² and 485 mW after layout at 65 nm; the authors reported it as 117.87x faster than a 128-bit 2 GHz SIMD processor while cutting total energy by 21.08x.[11] Google said its Tensor Processing Unit had been running in its datacenters since 2015. In the ISCA 2017 paper, its engineers reported that on production inference workloads the TPU was on average about 15 to 30 times faster than a contemporary server CPU or GPU, with 30 to 80 times higher TOPS per watt.[9]

Smartphone NPUs

In 2017 Huawei and Apple both announced phone chips with dedicated neural hardware. On 2 September 2017, at IFA in Berlin, Huawei presented the Kirin 970 as its "first mobile AI computing platform featuring a dedicated Neural Processing Unit (NPU)"; Huawei claimed its heterogeneous architecture gave up to 25 times the performance and 50 times the efficiency of a quad-core Cortex-A73 CPU cluster on AI tasks.[3] Ten days later Apple introduced the A11 Bionic, whose "neural engine" was a dual-core design that performed up to 600 billion operations per second and enabled Face ID and Animoji.[2]

Qualcomm traces its NPU to the first Hexagon DSP, launched on Snapdragon in 2007. According to the company, the Snapdragon 820 (2015) was the first chip with the Qualcomm AI Engine, the Hexagon Tensor Accelerator was added in the Snapdragon 855 in 2018, and in 2020 the scalar, vector and tensor accelerators were fused into one architecture with a large shared memory.[1] A 2019 benchmark study by Andrey Ignatov and colleagues at the ICCV workshops found that the performance of mobile AI accelerators had been "evolving rapidly in the past two years".[15] Apple's Neural Engine moved into its Mac chips; the Apple M2, announced in June 2022, has a Neural Engine that Apple says processes up to 15.8 trillion operations per second.[16]

Applications in VR and AR

XR machine learning workloads

Tracking and perception in current headsets rely on neural networks. As researchers at Facebook (now Meta Platforms) described in September 2019, hand tracking on the Oculus Quest uses deep neural networks on the headset's four cameras, with a quantized network framework that "enables real-time hand-tracking on a mobile processor" without depth sensors and without taking resources dedicated to applications.[17]

Researchers at Facebook and partner universities have argued that XR machine learning workloads are harder for accelerators to support than conventional single-task, single-model workloads. In a 2021 HPCA paper, Hyoukjun Kwon and colleagues from Georgia Tech, Facebook and NVIDIA noted that AR and VR applications run several deep neural networks for sub-tasks such as object detection, image segmentation, eye tracking and speech recognition, and that their layers vary widely in operation and shape. Their proposed accelerator, which combines sub-accelerators with different dataflows, showed 65.3% lower latency and 5.0% lower energy than the best fixed-dataflow design in their evaluation.[18] The XRBench suite (MLSys 2023), with authors from Meta, the University of California, Irvine, Georgia Tech and Harvard University, calls these "real-time multi-task multi-model" workloads. It defines usage scenarios such as social interaction, outdoor activity, an AR assistant and VR and AR gaming, built from unit tasks including hand tracking, eye segmentation, gaze estimation, keyword detection, speech recognition, semantic segmentation, object detection, depth estimation and plane detection, run in cascaded, concurrent and cascaded-concurrent patterns.[4]

A 2022 paper by Meta Reality Labs Research proposed spreading the compute across the glasses instead: processing placed on the image sensors themselves, combined with dense 3D-IC interconnects and STT-MRAM memory. For a machine-learning-based hand tracking algorithm, its simulation framework showed that this distributed on-sensor design could reduce system power compared with a centralized processor, with added benefits in latency and privacy.[19]

Microsoft HoloLens

Microsoft announced in July 2017 that the second version of its HoloLens holographic processing unit would include an AI coprocessor to "natively and flexibly implement DNNs". Marc Pollefeys, director of science for HoloLens, wrote that the coprocessor was designed to run continuously off the HoloLens battery, and that Harry Shum had shown an early spin of the chip running hand segmentation live during a keynote at CVPR 2017.[20]

Qualcomm XR platforms

Qualcomm's XR chips include a Hexagon AI block. The company's product brief for the Qualcomm Snapdragon XR2 Gen 2 lists a Hexagon Processor with a fused AI accelerator architecture, Hexagon Vector eXtensions, a Hexagon Scalar Accelerator and Hexagon Matrix eXtensions, and claims "8x more performant AI" and improved INT8 acceleration. The brief lists hand, head, controller, facial expression, depth estimation and 3D reconstruction among its concurrent perception tasks.[21] When the chip was announced on 27 September 2023, UploadVR reported that its NPU offers up to 4x peak AI performance and 8x AI performance per watt for INT8; Qualcomm's brief states that its comparisons are with the first-generation XR2.[21] UploadVR wrote that the gain can be used for new use cases such as dynamic object recognition and scene classification. The chip debuted in the Meta Quest 3.[5][22]

The Snapdragon AR1 Gen 1, announced the same day for smart glasses, is described in its product brief as "powered by the 3rd Generation Qualcomm Hexagon NPU" together with the AR-grade Engine for Visual Analytics. Qualcomm says this AI hardware improves image and audio quality and supports visual search and real-time translation.[7] The chip powers the Ray-Ban Meta smart glasses,[6] and a TechInsights teardown identified it as the main processor of the Meta Ray-Ban Display.[23]

At Augmented World Expo on 10 June 2025, Qualcomm announced the Snapdragon AR1+ Gen 1. Engadget reported that its 3rd-gen Hexagon NPU supports small language models of about 1 billion parameters on the glasses without a phone or cloud connection, and UploadVR reported that Qualcomm demonstrated prototype glasses running Meta's Llama-3.2-1B fully on-device. UploadVR noted that a model of this size is much smaller than the large cloud models behind services such as ChatGPT.[24][25]

Apple Vision Pro

The original Apple Vision Pro, announced on 5 June 2023, pairs an Apple M2 with the Apple R1 sensor-processing chip.[26] Apple's technical specifications list a 16-core Neural Engine as part of that M2.[8] The model updated with the Apple M5, announced on 15 October 2025, also has a 16-core Neural Engine. Apple says it makes AI-powered system features run up to 50 percent faster than the previous generation, citing capturing a Persona and turning photos into spatial scenes as examples, and third-party apps up to 2x faster.[27]

NPUs in XR chips

Chip Used in NPU details (manufacturer statements)
Qualcomm Snapdragon XR2 Gen 2 Meta Quest 3 Hexagon Processor with fused AI accelerator architecture; "8x more performant AI"[21]; up to 4x peak AI performance and 8x AI performance per watt (INT8) versus the first XR2[5]
Qualcomm Snapdragon AR1 Gen 1 Ray-Ban Meta[6], Meta Ray-Ban Display[23] 3rd Generation Hexagon NPU with scalar, vector and tensor accelerators, INT4 and INT8; separate micro NPU in the Sensing Hub[7]
Snapdragon AR1+ Gen 1 Announced at AWE 2025, shown in prototype glasses[25] 3rd-gen Hexagon NPU; small language models of about 1 billion parameters on the glasses[24]
Apple M2 Apple Vision Pro (original model) 16-core Neural Engine[8]; up to 15.8 trillion operations per second[16]
Apple M5 Apple Vision Pro (M5 model) 16-core Neural Engine; AI system features up to 50 percent faster than the previous generation[27]

See also

References

  1. ↑ 1.0 1.1 1.2 1.3 1.4 "Unlocking on-device generative AI with an NPU and heterogeneous computing". Qualcomm. Qualcomm Technologies. 2024-02. https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/Unlocking-on-device-generative-AI-with-an-NPU-and-heterogeneous-computing.pdf. Retrieved 2026-10-13.
  2. ↑ 2.0 2.1 "The future is here: iPhone X". Apple Newsroom. Apple. 2017-09-12. https://www.apple.com/newsroom/2017/09/the-future-is-here-iphone-x/. Retrieved 2026-10-13.
  3. ↑ 3.0 3.1 "Huawei Reveals the Future of Mobile AI at IFA 2017". Huawei. Huawei Technologies. 2017-09-02. https://www.huawei.com/au/press-events/news/au/2017/huawei-reveals-the-future-of-mobile-ai-at-ifa-2017. Retrieved 2026-10-13.
  4. ↑ 4.0 4.1 Hyoukjun Kwon, Krishnakumar Nair, Jamin Seo, Jason Yik, Debabrata Mohapatra, Dongyuan Zhan, Jinook Song, Peter Capak, Peizhao Zhang, Peter Vajda, Colby Banbury, Mark Mazumder, Liangzhen Lai, Ashish Sirasao, Tushar Krishna, Harshit Khaitan, Vikas Chandra, Vijay Janapa Reddi (2023). "XRBench: An Extended Reality (XR) Machine Learning Benchmark Suite for the Metaverse". Proceedings of Machine Learning and Systems 5 (MLSys 2023). arXiv:2211.08675. https://proceedings.mlsys.org/paper_files/paper/2023/hash/07b7c2f8a6978e376f9656d69619bfbb-Abstract-mlsys2023.html. Retrieved 2026-10-13.
  5. ↑ 5.0 5.1 5.2 David Heaney (2023-09-27). "Quest 3's New Snapdragon XR2 Gen 2 Chipset Has More Than Double The GPU Power". UploadVR. https://www.uploadvr.com/snapdragon-xr2-gen-2/. Retrieved 2026-10-13.
  6. ↑ 6.0 6.1 6.2 Hugo Swart (2023-11-29). "Qualcomm and Meta are Expanding Your Reality: Here's How". Edge AI and Vision Alliance. Qualcomm Technologies. https://www.edge-ai-vision.com/2023/11/qualcomm-and-meta-are-expanding-your-reality-heres-how/. Retrieved 2026-10-13.
  7. ↑ 7.0 7.1 7.2 7.3 7.4 "Snapdragon AR1 Gen 1 Platform Product Brief (87-86507-1 Rev B)". Qualcomm. Qualcomm Technologies. https://docs.qualcomm.com/doc/87-86507-1/87-86507-1_REV_B_Snapdragon_AR1_Gen_1_Platform_Product_Brief.pdf. Retrieved 2026-10-13.
  8. ↑ 8.0 8.1 8.2 "Apple Vision Pro - Tech Specs". Apple Support. Apple. https://support.apple.com/en-us/117810. Retrieved 2026-10-13.
  9. ↑ 9.0 9.1 Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, et al. (2017-06). "In-Datacenter Performance Analysis of a Tensor Processing Unit". Proceedings of the 44th Annual International Symposium on Computer Architecture (ISCA 2017), pp. 1-12. https://doi.org/10.1145/3079856.3080246. Retrieved 2026-10-13.
  10. ↑ 10.0 10.1 Chester Lam (2024-04-22). "Intel Meteor Lake's NPU". Chips and Cheese. https://chipsandcheese.com/p/intel-meteor-lakes-npu. Retrieved 2026-10-13.
  11. ↑ 11.0 11.1 Tianshi Chen, Zidong Du, Ninghui Sun, Jia Wang, Chengyong Wu, Yunji Chen, Olivier Temam (2014). "DianNao: A Small-Footprint High-Throughput Accelerator for Ubiquitous Machine-Learning". Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2014), pp. 269-284. https://doi.org/10.1145/2541940.2541967. Retrieved 2026-10-13.
  12. ↑ Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, Joel S. Emer (2017-12). "Efficient Processing of Deep Neural Networks: A Tutorial and Survey". Proceedings of the IEEE, vol. 105, no. 12, pp. 2295-2329. https://doi.org/10.1109/JPROC.2017.2761740. Retrieved 2026-10-13.
  13. ↑ Hadi Esmaeilzadeh, Adrian Sampson, Luis Ceze, Doug Burger (2012-12). "Neural Acceleration for General-Purpose Approximate Programs". 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 449-460. https://doi.org/10.1109/MICRO.2012.48. Retrieved 2026-10-13.
  14. ↑ "Neural Processing Units". SAMPA, University of Washington. University of Washington. https://sampa.cs.washington.edu/new/research/approximation/npu.html. Retrieved 2026-10-13.
  15. ↑ Andrey Ignatov, Radu Timofte, Andrei Kulik, Seungsoo Yang, Ke Wang, Felix Baum, Max Wu, Lirong Xu, Luc Van Gool (2019-10). "AI Benchmark: All About Deep Learning on Smartphones in 2019". 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3617-3635. https://doi.org/10.1109/ICCVW.2019.00447. Retrieved 2026-10-13.
  16. ↑ 16.0 16.1 "Apple unveils M2 with breakthrough performance and capabilities". Apple Newsroom. Apple. 2022-06-06. https://www.apple.com/newsroom/2022/06/apple-unveils-m2-with-breakthrough-performance-and-capabilities/. Retrieved 2026-10-13.
  17. ↑ Shangchen Han, Beibei Liu, Tsz Ho Yu, Randi Cabezas, Peizhao Zhang, Peter Vajda, Eldad Isaac, Robert Wang (2019-09-25). "Using deep neural networks for accurate hand-tracking on Oculus Quest". Meta AI Blog. Meta. https://ai.meta.com/blog/hand-tracking-deep-neural-networks/. Retrieved 2026-10-13.
  18. ↑ Hyoukjun Kwon, Liangzhen Lai, Michael Pellauer, Tushar Krishna, Yu-Hsin Chen, Vikas Chandra (2021-02). "Heterogeneous Dataflow Accelerators for Multi-DNN Workloads". 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pp. 71-83. https://doi.org/10.1109/HPCA51647.2021.00016. Retrieved 2026-10-13.
  19. ↑ Jorge Gomez, Saavan Patel, Syed Shakib Sarwar, Ziyun Li, Raffaele Capoccia, Zhao Wang, Reid Pinkham, Andrew Berkovich, Tsung-Hsun Tsai, Barbara De Salvo, Chiao Liu (2022-03-14). "Distributed On-Sensor Compute System for AR/VR Devices: A Semi-Analytical Simulation Framework for Power Estimation". arXiv preprint 2203.07474 (TinyML Research Symposium). https://arxiv.org/abs/2203.07474. Retrieved 2026-10-13.
  20. ↑ Marc Pollefeys (2017-07-23). "Second version of HoloLens HPU will incorporate AI coprocessor for implementing DNNs". Microsoft Research Blog. Microsoft. https://www.microsoft.com/en-us/research/blog/second-version-hololens-hpu-will-incorporate-ai-coprocessor-implementing-dnns/. Retrieved 2026-10-13.
  21. ↑ 21.0 21.1 21.2 "Snapdragon XR2 Gen 2 Platform Product Brief (87-73689-1 Rev A)". Qualcomm. Qualcomm Technologies. https://docs.qualcomm.com/doc/87-73689-1/87-73689-1_REV_A_Snapdragon_XR2_Gen_2_Platform_Product_Brief.pdf. Retrieved 2026-10-13.
  22. ↑ Sam Rutherford (2023-09-27). "Qualcomm announces two new Snapdragon chips for next-gen headsets and smart glasses". Engadget. https://www.engadget.com/qualcomm-announces-two-new-snapdragon-chips-for-next-gen-headsets-and-smart-glasses-180010526.html. Retrieved 2026-10-13.
  23. ↑ 23.0 23.1 "Meta Ray-Ban Display Teardown Reveals More Than Meets the Eye". TechInsights. 2026-06-10. https://www.techinsights.com/blog/meta-ray-ban-display-teardown-reveals-more-meets-eye. Retrieved 2026-10-13.
  24. ↑ 24.0 24.1 Steve Dent (2025-06-10). "Qualcomm says its new AR1+ Gen 1 chip can handle AI directly on smart glasses". Engadget. https://www.engadget.com/wearables/qualcomm-says-its-new-ar1-gen-1-chip-can-handle-ai-directly-on-smart-glasses-163002410.html. Retrieved 2026-10-13.
  25. ↑ 25.0 25.1 David Heaney (2025-06-10). "Qualcomm Reveals Snapdragon AR1+ Chip For High-End Smart Glasses". UploadVR. https://www.uploadvr.com/qualcomm-snapdragon-ar1-plus-smart-glasses-chip/. Retrieved 2026-10-13.
  26. ↑ "Introducing Apple Vision Pro: Apple's first spatial computer". Apple Newsroom. Apple. 2023-06-05. https://www.apple.com/newsroom/2023/06/introducing-apple-vision-pro/. Retrieved 2026-10-13.
  27. ↑ 27.0 27.1 "Apple Vision Pro upgraded with the M5 chip and Dual Knit Band". Apple Newsroom. Apple. 2025-10-15. https://www.apple.com/newsroom/2025/10/apple-vision-pro-upgraded-with-the-m5-chip-and-dual-knit-band/. Retrieved 2026-10-13.
Retrieved from "https://vrarwiki.com/index.php?title=Neural_processing_unit&oldid=50202"