Smartphone

A smartphone is a handheld mobile device that combines cellular telephony with a general-purpose computer, a touchscreen display, wireless data connectivity, and the ability to install third-party software applications. Beyond its role as a phone, the modern smartphone packs a high-resolution display, one or more cameras, a battery, a system-on-chip (SoC) processor, and a cluster of sensors, including a gyroscope, accelerometer, and magnetometer, that together form an inertial measurement unit. That combination of components made the smartphone the substrate on which the first wave of mass-market consumer virtual reality (VR) and augmented reality (AR) was built.

For roughly the period from 2014 to 2019, smartphones were the most common way for ordinary people to experience VR. A phone slotted into a simple lensed holder, such as Google Cardboard or the Samsung Gear VR, became a low-cost headset whose screen, processor, and motion sensors did all the work. On the AR side, the phone never went away: software frameworks such as Apple's ARKit and Google's ARCore turned hundreds of millions of existing handsets into AR devices using only the built-in camera and motion sensors. The same economies of scale that drove down the price of mobile displays, sensors, and chips also made the standalone VR headset possible, since devices such as the Oculus Quest run on processors and panels originally engineered for phones.

History

The device generally cited as the first smartphone is the IBM Simon Personal Communicator, manufactured by Mitsubishi Electric and sold through BellSouth Cellular beginning on 16 August 1994. The Simon combined a cellular phone with personal digital assistant (PDA) functions, ran on a 4.5 inch resistive touchscreen, and could load additional software. It cost 899 US dollars on a two-year contract or 1,099 US dollars without one, and about 50,000 units sold before it was discontinued in February 1995. The category remained a niche of business-oriented devices, including Palm and BlackBerry handsets, for more than a decade.

The form factor that defines the modern smartphone arrived with the first Apple iPhone, unveiled by Steve Jobs on 9 January 2007 and released on 29 June 2007. It used a 3.5 inch capacitive multi-touch display operated by finger gestures rather than a stylus or physical keyboard. Google's Android platform reached consumers the following year on the HTC Dream (sold in the United States as the T-Mobile G1), which HTC announced in September 2008. The capacitive touchscreen, fast mobile processors, and increasingly capable sensors that these devices standardized are the same components that consumer VR and AR would later depend on.

Components relevant to VR and AR

Several smartphone subsystems matter directly for VR and AR. The table summarizes the main ones and their role.

Component	Function in a phone	Role in VR / AR
Display (LCD or OLED/AMOLED)	Screen for apps and media	Supplies the image; in phone-based VR it is split into two views, one per eye
Inertial measurement unit (gyroscope + accelerometer + magnetometer)	Screen rotation, step counting, compass	Rotational head tracking; sensor fusion for 3DOF and, with the camera, 6DOF
Rear camera (CMOS sensor)	Photography and video	Captures the scene for AR; provides the feature points used in computer-vision tracking
System-on-chip (CPU + GPU + DSP)	General computing and graphics	Renders the VR/AR scene and runs tracking algorithms
Depth sensor (time-of-flight or LiDAR, select models)	Autofocus, portrait effects	Measures distance to surfaces for instant AR object placement
Wireless radios (Wi-Fi, Bluetooth, cellular)	Connectivity	Streaming content, pairing controllers, location services

The motion sensors are central. A VR headset needs at least one MEMS inertial measurement unit for three-degrees-of-freedom (3DOF) rotational tracking, and the mass production of these tiny gyroscopes and accelerometers for phones is what made them cheap enough to put in inexpensive headsets.

Smartphone-based virtual reality

Between 2014 and 2019, the dominant entry point to VR was a smartphone placed inside a viewer with two lenses. The phone provided the display, the processor, and the motion tracking; the viewer provided the optics and a way to hold the handset at the right distance from the eyes. A compatible app splits the screen into two images, one for each eye, and applies barrel distortion that the lenses then correct, producing a single stereoscopic view with a wide field of view.

Google Cardboard

Google Cardboard was introduced at the Google I/O conference on 25 June 2014. It was developed by David Coz and Damien Henry, engineers at Google's Cultural Institute in Paris. The viewer was a folded cardboard holder with two 45 mm lenses; the first version used a magnet as a button (read through the phone's compass sensor), and the second version, shown at Google I/O 2015, replaced the magnet with a conductive lever that triggers a touch event on the phone's screen. Version 1 fit screens up to 5.7 inches and version 2 up to 6 inches. Google released software development kits for Android and iOS plus a Unity plugin. By November 2019 more than 15 million Cardboard viewers had shipped. Google stopped selling the viewer through its store on 3 March 2021, and on 6 November 2019 it announced it would open-source the platform's SDK.

Samsung Gear VR

The Samsung Gear VR was developed by Samsung in partnership with Oculus and unveiled at the Samsung press conference at IFA Berlin on 3 September 2014. Unlike Cardboard, it worked only with specific Samsung Galaxy flagship phones, which slid into the front and acted as both the display and the processor. The headset itself contained the optics and a custom inertial measurement unit for rotational tracking, and it connected to the phone through micro-USB or, on later models, USB-C. The first Innovator Edition (SM-R320) appeared in October 2014, a consumer edition (SM-R322) followed on 20 November 2015, and the final model (SM-R325) shipped on 15 September 2017. Supported phones spanned the Galaxy Note 4 through the Galaxy S10 series; the Galaxy Note10 and later were not supported. Samsung ended XR service support on 30 September 2020.

Google Daydream

Google Daydream was announced at Google I/O in May 2016 as a higher-quality phone-VR platform than Cardboard. It required a certified "Daydream-ready" phone and shipped with a small wireless controller that had a touchpad and motion sensors and charged over USB-C. The first Daydream View headset was released on 10 November 2016 at 79 US dollars, and a second generation followed on 19 October 2017 at 99 US dollars. Google named eight hardware partners (Samsung, HTC, LG, Xiaomi, Huawei, ZTE, Asus, and Alcatel), and supported phones included the Pixel line and several Galaxy models. On 15 October 2019 Google stopped selling the headset and ceased certifying new phones, and Android 11 (2020) dropped platform support entirely; Google cited the lack of broad consumer and developer adoption.

Decline of phone-based VR

Phone-based VR faded around 2019 to 2020 as both major platforms ended support. Several limitations drove the decline. The sensors in a phone give only 3DOF rotational tracking, so the user can look around but cannot move through the virtual space, which best-in-class VR provides through 6DOF tracking. Running VR also generated heat that throttled the phone's processor, and it tied up the device the user relied on all day. Standalone headsets then offered a more self-contained experience: the Oculus Go, released on 1 May 2018 at 199 US dollars, packed its own screen, processor, and battery into the headset and ran independently without a phone or PC.

Smartphone-based augmented reality

While phone VR rose and fell, AR on the phone grew and remained the most widely deployed form of consumer AR. The phone's rear camera shows the live scene, software analyzes that video together with the motion sensors to work out where the device is in space, and virtual content is drawn on top of the camera feed. This requires no extra hardware beyond what a modern phone already contains.

ARKit

Apple announced ARKit at its Worldwide Developers Conference in June 2017 and shipped it with iOS 11. ARKit is an API that lets developers build AR apps using the device's camera, processor, and motion sensors, and it requires an Apple A9 or later chip. Its tracking incorporates computer-vision and SLAM technology developed in part by Metaio, a company Apple acquired in May 2015. The core technique is visual-inertial odometry: ARKit recognizes notable features in the camera image, tracks how their positions change across frames, and compares that with the motion-sensor data to build a high-precision model of the device's position and movement, enabling content to be placed in the world with six degrees of freedom.

ARCore

Google announced ARCore on 29 August 2017, shortly after ARKit, and released version 1.0 on 23 February 2018. It enables AR apps on Android 7.0 and later devices that have a suitable camera, motion sensors, and enough processing power. ARCore rests on three capabilities: motion tracking, which uses SLAM with visual feature points to follow the device's position; environmental understanding, which detects flat surfaces; and light estimation, which measures lighting so virtual objects can be lit to match. A later Depth API builds a depth map from the single moving camera. ARCore is software-locked to devices that pass Google's certification process.

Depth sensing and LiDAR

Some high-end phones add a dedicated depth sensor to speed up and sharpen AR. Apple added a LiDAR scanner to the iPad Pro on 25 March 2020 and to the iPhone 12 Pro and 12 Pro Max on 23 October 2020. The scanner emits pulses of infrared laser light and times how long they take to return, measuring distance directly at ranges up to about five meters. Because it captures depth in effectively a single reading rather than by comparing camera images over time, LiDAR allows "instant AR," placing virtual objects on real surfaces with no scanning delay.

Pokemon Go

The clearest demonstration of smartphone AR's reach is Pokemon Go, a location-based AR game developed by Niantic in partnership with Nintendo and The Pokemon Company. It launched on 6 July 2016 in Australia, New Zealand, and the United States, using the phone's GPS to place game elements in the real world and the camera to overlay creatures on the live view. It became one of the most downloaded apps of 2016, passing 500 million installs, and is widely credited with popularizing location-based AR among the general public.

Influence on standalone headsets

The smartphone industry's scale shaped VR and AR hardware well beyond phone-based viewers. The affordable high-resolution displays, low-power chips, and sensors that consumer VR depends on were engineered first for phones rather than for headsets. Standalone headsets mostly adopted chipsets originally designed for smartphones. The original Oculus Quest, released on 21 May 2019, runs a Qualcomm Snapdragon 835, a mobile SoC first used in phones, paired with 4 GB of LPDDR4X memory and dual PenTile OLED panels at 1440 by 1600 pixels per eye; it operates without a PC, though a phone running the companion app is needed only for first-time setup. In 2018 Qualcomm introduced the Snapdragon XR1, its first chipset purpose-built for AR and VR rather than repurposed from a phone, signaling the point at which XR hardware began to diverge from smartphone parts.

Phone display technology continued to set the ceiling on early VR image quality. Smartphone AMOLED panels top out around 600 pixels per inch, which is visible as a screen-door effect when magnified by VR lenses, so newer high-end headsets such as the Apple Vision Pro moved to Micro-OLED microdisplays exceeding 3,000 pixels per inch.

Current status

As of 2026 the smartphone's roles in VR and AR have separated. Dedicated phone-based VR is effectively over: Google Cardboard, the Samsung Gear VR, and Google Daydream are all discontinued, and consumer VR has shifted to standalone headsets. Smartphone AR remains active and widely used through ARKit and ARCore, which reach a large installed base of existing phones and underpin AR features in shopping, navigation, social media filters, and games. Phones also still serve as companion devices for headset setup, account management, and casting.