Automated testing of VR applications

Automated testing of VR applications is the use of software tools and techniques to check virtual reality (VR) software with less manual play in a head-mounted display. Documented approaches include unit and integration tests run with a game engine's test framework, desktop simulators that stand in for a headset and its controllers, recorded input sessions that are replayed, research tools that explore 3D scenes and trigger interactions on their own, test agents trained with machine learning, and large language models that analyse screenshots of a running application.

In a 2023 study of 314 open-source Unity VR projects, 79% had no automated tests. A 2025 systematic mapping study of research on testing extended reality (XR) applications identified 34 primary studies published between January 2000 and July 2024. As of 2026, Meta considers apps for distribution in the Meta Horizon Store only if they meet its Virtual Reality Checks (VRCs), which include requirements that an app install and run without crashes, freezes or extended unresponsive states and that immersive apps maintain a minimum rendering rate.

According to the mapping study, XR applications involve "virtually infinite interaction possibilities", "the specific oracles needed to validate functionality remain unclear and vary by system", and automated approaches to usability problems such as cybersickness detection still depend largely on user involvement. Meta's documentation states that its desktop simulator, Meta XR Simulator, "does not emulate device hardware".

Reviewed 1 November 2026. Checked study figures and tool descriptions against the cited papers, and vendor claims against the cited Unity, Meta, Google, Apple, Khronos, AltTester and CORDIS pages. About review dates.

Why VR applications are hard to test

Rzig and colleagues attribute the difficulty of testing VR applications to the physical interactivity that VR requires and to its reliance on specific hardware systems. Andrade, Nunes and Delamaro list images, sounds, videos and differentiated interaction as features of VR software that can become sources of new kinds of faults. The mapping study by Ruizhen Gu, José Miguel Rojas and Donghwan Shin contrasts Android apps, which have finite interaction paths when navigating between screens, with XR apps, in which even moving toward a virtual object and interacting with it requires accounting for countless variations in interaction sequences.

The same study separates three activities. Test automation only automates the execution of tests; test input generation also creates the input data but not the test oracles (the expected results); test generation creates both. Test generation was the least explored of these, appearing in only three studies. In a 2026 journal article, first posted as a preprint in January 2026, the same authors state that existing VR testing approaches enable scene navigation and interaction activation but cannot automatically synthesise realistic 3D user inputs, such as grab and trigger actions with hand-held controllers, and that existing metrics fail to robustly capture diverse interaction coverage.

Testing practice in VR projects

The studies of VR testing practice summarised below all analysed Unity projects.

Study	Venue	Sample	Main findings
Andrade, Nunes and Delamaro	SVR 2019; extended arXiv preprint, 2020	119 Unity VR projects, compared with 107 non-VR C# projects that had unit tests	6 VR projects (5.04%) had tests, 154 unit tests in total; about 12.2% of VR classes estimated to be fault-prone, against 8.9% of non-VR classes
Rzig, Iqbal, Attisano, Qin and Hassan	ISSTA 2023	314 open-source Unity VR projects	Tests in 61 projects (79% had none); average assertion density 17.63%; on average 38.43% of a project's tests had at least one of six test smells
Gurramkonda, Chakroborty, Maxim, Mkaouer and Hassan	ASE 2025	Unity VR test code	VRTestSniffer, a static analysis tool that detects 17 test smell categories, with a reported F1-score of 95.61%

For the extended version of their study, Andrade and colleagues catalogued 151 VR projects found by searching GitHub and analysed 119. Code coverage could not be measured because Unity lacked an out-of-the-box coverage tool at the time. Code smells were common, especially implementation smells, and an unsupervised clustering approach based on code metrics flagged 2,627 of 21,508 VR classes as fault-prone. The authors conclude that the lack of testing is a significant risk to the projects' success.

In their abstract, Rzig and colleagues write that "we still know little about VR application testing". They found test-to-code ratios more than 14 times lower than those reported in a similar analysis by Vidacs and colleagues. Problems ranging from compilation errors to incompatibility between coverage measurement and some Unity versions prevented code coverage reports, so they measured assertion density instead. Most tests were unit and integration tests, and the authors found no evidence of search-based or goal-directed testing in their set. From a random sample of 370 of the 8,723 tests, drawn from 36 projects, they built a taxonomy of 13 main categories and 14 subcategories. They define a "VR-attention" test as "a test that involves VR devices and VR experience, which are less likely to appear in non-VR applications", and found four VR-attention categories: GUI, input, performance and device tests. Examples include a test that creates a sequence of hand movements to simulate touch and scrolling, a test that checks that the control layout of a particular manufacturer's XR device loads, a test of how an eye tracking system captures a target object among other distracting objects, and a performance test that sets a large number of control points in a VR sketching application. VRTestSniffer, whose authors include Foyzul Hassan of the ISSTA study, builds on the test smell categories of the earlier tsDetect tool and analyses abstract syntax trees and control-flow and data-flow graphs of Unity VR test code.

Test frameworks and device simulators

Engine test frameworks

According to Unity's documentation for Unity 6.3, the Unity Test Framework is a core package distributed as part of the Unity Editor installation. Tests for Edit mode or Play mode code run from the Test Runner window, from the command line or from code. The framework integrates a custom version of NUnit based on version 3.5, and its Unity tests can interact with frames and the application loop, running as coroutines in Play mode. Unity's package documentation calls Unity Test Framework a more descriptive name for the toolset otherwise known as the Unity Test Runner, and says that tests can also run on target platforms such as Standalone, Android and iOS. As of November 2026, the package changelog listed version 1.9.0, dated 14 August 2026, as its newest release.

The mapping study lists the framework among the industrial tools used or referenced in the studies it reviewed, together with Airtest, a visual UI test automation framework that uses screenshot-based locators, and AltUnity Tester, whose test scripts reach Unity elements through identifiers such as object names and tags. AltUnity Tester was renamed AltTester Unity SDK with version 1.8.0, announced in October 2022.

Simulated headsets and controllers

According to the mapping study, many XR platforms include simulation capabilities for testing and debugging apps without a physical headset, such as Meta XR Simulator and Unity's XR Device Simulator. The following simulators are documented by their vendors as of November 2026.

Simulator	Developer	Environment	Simulated input	Documented limits
XR Interaction Simulator, a sample in the XR Interaction Toolkit	Unity Technologies	Unity Editor	Key presses from a keyboard and mouse or a controller drive the headset, controllers or hands	Described as primarily an Editor-only testing tool; the older XR Device Simulator remains as a legacy sample
Meta XR Simulator	Meta	OpenXR runtime on Windows and macOS	Keyboard and mouse, Quest controllers through data forwarding, or an Xbox controller on Windows	Does not emulate device hardware; no OpenGL or OpenGL ES rendering
Android XR Emulator	Google	Android Studio (latest Canary build)	Mouse pointer drives hand raycasts, a right-hand pinch and eye gaze	Not supported for Unity or OpenXR apps; no left-hand controls or grab and poke gestures
Simulated visionOS device in Device Hub	Apple	Device Hub, an app that ships alongside Xcode 27	Pointer for gaze and gestures; keyboard, trackpad or game controller for movement	Cannot interact with an app running on a physical device
Immersive Web Emulator	Meta	Extension for Chromium-based browsers, for WebXR	Emulated Meta Quest devices, controllers and hands, with a synthetic environment module	Lighting estimation and DOM overlays unsupported; repository archived on 1 October 2026

The XR Interaction Simulator turns "plain key presses (be it from a keyboard and mouse combo or a controller)" into simulated input: Tab cycles between an FPS mode and a device mode, the bracket keys activate the left, right or both controllers, and pressing a bracket key twice switches between controller and hand mode. The documentation notes that the simulator does not move the camera or controllers of the XR Origin directly but "drives them indirectly through simulated input".

Meta describes Meta XR Simulator as "a lightweight OpenXR runtime that runs on your computer", with no operating system image and no Android layer. Applications request OpenXR 1.0 or 1.1 and render with Vulkan, Direct3D 11 or 12, or Metal. One device profile is active at a time: Meta Quest 2, Meta Quest Pro, Oculus Rift S, Meta Quest 3 (the default), Meta Quest 3S or Meta VR Glasses. Synthetic rooms supply passthrough, scene data, spatial anchors and, on Windows for some profiles, environment depth, and more than one application client can connect to the simulator to test a shared session. According to Meta's setup guide, the simulator is a standalone application with installers for Windows and macOS, its package in the Unity Asset Store is deprecated, and Unity projects that still contain the com.meta.xr.simulator package should remove it.

Google's emulator for Android XR simulates hand tracking by aiming the virtual hands' raycasts at the mouse pointer and treating a drag as a pinch, and simulates gaze from the pointer's position in 3D space. Google's OpenXR guide states that the emulator "is not supported for Unity or OpenXR apps". For visionOS, Apple's documentation shows an Apple Vision Pro simulator in Device Hub. The Mac's pointer controls where the simulated user's eyes look, clicks and drags trigger taps and pinches, built-in simulated scenes with different room layouts, furniture and lighting simulate passthrough, and a FaceTime session with simulated spatial participants is used to test SharePlay features. Meta's Immersive Web Emulator emulates Meta Quest devices for WebXR development and supports the gamepad, hand input, hit test, plane detection, mesh detection and anchors modules; Meta archived its GitHub repository on 1 October 2026.

Record and replay

Record and replay captures a sequence of inputs once and plays it back in later runs. The mapping study describes it as a common testing approach for GUI apps that simplifies the automation of complex usage scenarios, but notes poor maintainability: a captured test frequently breaks when the app's user interface changes and then needs substantial manual updates. Meta XR Simulator records sessions to VRS files and replays them with the same head and input motion.

Meta's Unity guide to automated testing with the simulator, last updated in July 2024, describes three approaches. Zero-code automation uses record and replay with snapshot testing; Meta says it "requires no modifications to your app or game" and is "particularly advantageous for smoke testing". Programmatic tests written in code handle cases that screenshots cannot capture, such as an object obscured behind another object, and dynamic situations such as randomly spawning game objects. A hybrid approach combines the two and is described as useful for testing areas of an application that are not accessible at the beginning of gameplay. The guide recommends running such tests regularly in continuous integration (CI) pipelines, for example per pull request, hourly or nightly, says they can run on cloud infrastructure without physical Meta XR devices, and notes that multiplayer client interactions can be recorded and replayed in CI.

Meta's no-code testing guide for native apps, updated in September 2026, gives a four-step workflow: record each test case to its own VRS file, replay it to confirm the behaviour, copy the app and the recording to a test machine, and launch the app there with a session_capture block in the simulator's persistent_data.json file. Meta states that "Replay does not judge the outcome: it starts playback and requests session exit, and nothing more." The CI harness decides the result from a timeout, the process exit status and the runtime logs. The app must handle the exit request, and the block triggers replay at every launch until it is removed.

Automated scene exploration

Research on automated VR testing has produced tools that explore a VR scene on their own and interact with the objects they find.

Tool	Authors and venue	Approach	Reported evaluation
VRTest	Xiaoyin Wang (University of Texas at San Antonio); ICSE 2022 demonstrations track	Reads information from a VR scene and controls the user camera to explore the scene and interact with virtual objects; built-in strategies VRMonkey (random exploration) and VRGreed (greedy algorithm)	Tool demonstration
VRGuide	Xiaoyin Wang, Tahmid Rafi and Na Meng; ASE 2023	Adapts the computational geometry technique Cut Extension to optimise camera routes for covering all interactable objects; the authors present it as a response to challenges such as object occlusion and movement, which earlier testing strategies could not address	Compared with VRTest on eight projects: higher coverage within the time limit on two, saturation coverage with on average about 31% less testing time on the other six; four previously unknown bugs confirmed by developers, one of which VRTest also found
VRExplorer	Zhengyang Zhu, Hong-Ning Dai, Hanyang Guo, Zeqin Liao and Zibin Zheng; ASE 2025	Entity, Action and Task (EAT) models of interactions; an agent navigates with path-finding on Unity's NavMesh and chooses interactions with a probabilistic finite state machine	On 11 projects, average gains over VRGuide of 122.8% in coverage of executable lines of code and 52.8% in method coverage; two functional bugs and one non-functional bug found
XRintTest	Ruizhen Gu, José Miguel Rojas and Donghwan Shin; Automated Software Engineering, 2026 (preprint January 2026)	An Interaction Flow Graph of targets, actions and conditions guides scene exploration and the execution of interactions	93% coverage of fire, manipulate and socket interactions on the ten-scene XRBench3D benchmark (456 interactions); 12 times more effective and 6 times more efficient than random exploration

The VRTest abstract states that quality assurance techniques for VR software had not been well studied and that validation relied mostly on manual testing. The VRExplorer authors describe VRTest as an early attempt that explores scenes by controlling the camera's orientation, and VRGuide as an improvement that routes exploration around occluded objects, but state that VRGuide "can only test virtual objects with the 'click' interaction". VRExplorer's framework is meant to be reusable across projects built with different Unity versions and interaction plug-ins, such as the XR Interaction Toolkit, SteamVR and the Mixed Reality Toolkit. Its authors kept 102 of 971 Unity VR projects collected from GitHub after manual quality checks, and report that manual exploration took 30 to 90 minutes per project, depending on the engineer's proficiency. The authors of XRintTest derived its four interaction types (fire, manipulate, socket and custom) from nine open-source VR projects. Besides runtime exceptions, the tool detected non-exception interaction issues, including subtle configuration defects.

Machine-learning and language-model test agents

Agent-based testing

The European Union's Horizon 2020 project iv4XR (Intelligent Verification/Validation for Extended Reality Based Systems) ran from 1 October 2019 to 31 December 2022, coordinated by INESC ID in Portugal. Its project description states that XR's "fine grained and high level of interactivity and realism make such systems very hard and expensive to test". The project set out to build AI-based verification and validation tools, with test agents that explore and test virtual worlds automatically and socio-emotional AI for assessing the quality of user experience. The mapping study describes iv4XR as a suite with frameworks for agent-based, model-based and reinforcement learning-based testing, as well as user experience testing.

Andrade, Nunes and Delamaro's 2023 paper "Exploiting deep reinforcement learning and metamorphic testing to automatically test virtual reality applications" combines metamorphic testing, agent-based testing and machine learning. Metamorphic relations detect collision and camera faults while intelligent agents generate test data. In an experiment on four VR applications, the approach reached an accuracy between 93% and 69%, depending on the application's complexity. Citing this work, the mapping study describes Unity's ML-Agents toolkit, with more than 17 example environments, as well suited to evaluating XR testing approaches, including agent-based testing.

Imitation learning

VRScout, described in a September 2025 arXiv preprint by Yurun Wu, Yousong Sun, Burkhard Wunsche, Jia Wang and Elliott Wen, is an agent that learns to play VR games from human demonstrations (imitation learning); its authors position it for quality assurance and safety auditing. It uses an enhanced Action Chunking Transformer that predicts multi-step action sequences, with a sliding horizon that adapts at runtime. A virtual device driver built on Valve's OpenVR SDK presents a virtual headset and two virtual controllers to SteamVR, so the agent can play commercial games without modifying them. Its inputs are RGB frames sampled at 30 Hz and encoded with an ImageNet-pretrained ResNet-18, together with the poses and button states of the headset and controllers.

Trained on 3.5 hours of demonstrations per game from a single player, the agent cleared an Expert-level map in Beat Saber with rank A and reached rank C in Pistol Whip; results for SUPERHOT VR are reported only qualitatively. Inference ran at 60 Hz on an RTX 4090 and, with reduced smoothing, at 35 Hz on an RTX 3060; training used an A100. The authors note that the agent relies primarily on RGB images, without signals such as spatial audio, object categories or scene graphs, and that its adaptive window may not generalise well to slower, strategy-focused genres.

Language models

Zhenyu Qi and colleagues tested GPT-4o for field-of-view analysis in VR exploration testing, in a case study posted to arXiv in January 2025 and later published in Automated Software Engineering. The study used screenshots of The Break-In, a thief-simulation game installed from Meta's store and run on a Meta Quest 2, with 216 labelled entities. GPT-4o correctly identified 41.67% of the entities with a basic prompt and 71.30% with chain-of-thought prompting plus one refinement round. Its descriptions of entity features were at least 90% accurate, combining colour, placement and shape gave the best F1-score (0.70) for matching the same entity across fields of view, and scene recognition reached 85.18%, but the model failed to label the identified entities. Among the threats to the study's validity, the authors list its use of a single language model and a single cartoon-like game with simplified scenes.

The mapping study points to multimodal language models, which process both text and images, as a possible route to test oracles for graphics-heavy XR software, and notes that language models have been used to generate unit tests for Unity-based game development.

Performance and store compliance testing

Meta states that Meta Horizon Store apps "must meet or exceed Virtual Reality Check (VRC) guidelines to be considered for distribution". It groups the checks into areas such as packaging, audio, performance, functional, security, tracking, input, accessibility, privacy policy, content and publishing, and marks each check as required or recommended. VRC.Quest.Functional.1 states: "App must install and run without crashes, freezes, or extended unresponsive states." Meta links test plans for the checks but warns that the VRCs listed in them "are slightly out of date".

VRC.Quest.Performance.1, required for immersive apps and recommended for panel apps, states that "The app must run at an allowed refresh rate and maintain a rendering rate (fps) of at least 60 fps". Interactive apps must use a refresh rate of 72, 80, 90, 96, 100 or 120 Hz, and may render at half the refresh rate during portions of the experience that use Application SpaceWarp. Meta's test steps are to use the app for the length of its content or 45 minutes, whichever is shorter, then check the OVR Metrics Tool's FPS graph for extended periods below 60 fps, or 30 fps with Application SpaceWarp. Meta lowered the minimum rendering rate to 60 fps, or 30 fps with Application SpaceWarp, on 7 August 2024, added 120 Hz on 29 October 2024 and added 96 and 100 Hz on 22 October 2025, saying that these two rates were not yet available and were planned for the Quest 2, Quest 3 and Quest 3S in a future OS version.

The OVR Metrics Tool, which Meta describes as "a performance monitoring solution for Meta Quest headsets and Meta VR Glasses", reports frame rate, stale frames, screen tears, CPU and GPU levels and utilisation, app GPU time and memory. It can record sessions to CSV files, overlay live statistics in a heads-up display and take screenshots when dropped frames exceed a set limit within a time window, and these functions can be switched on and off with Android Debug Bridge (adb) broadcast commands.

Meta's RenderDoc AI Tools page, updated in September 2026, describes skills bundled with RenderDoc Meta Fork that let an AI agent capture a frame from an app on a connected headset, rank draw calls by GPU cost, hot-swap shaders and re-measure, and simplify geometry. Meta states that "Every number the agent reports comes from a real capture of your app running on real hardware", and that a capture should be replayed on the same headset model that captured it. The page names Claude Code as an example of an agent that can read the skill directories.

At the API level, the Khronos Group announced an OpenXR Best Practices Validation Layer on 13 November 2025. It flags problems in the frame loop, such as beginning a new frame while the previous one is still in flight or calling xrSyncActions and xrLocateSpace before xrWaitFrame, and checks the data passed to xrEndFrame, including non-zero field-of-view values and field-of-view and pose data that match those from xrLocateViews for projection layers. The layer is available in the OpenXR-SDK-Source repository, and Khronos states that it "is intended for use in development and debugging workflows and should not be used in production deployments".

Research overview

For their mapping study, Gu, Rojas and Shin searched the OpenAlex database, which returned 1,167 studies; after selection and backward snowballing, 34 primary studies remained. Automated testing was the most common topic (seven studies, 21%), followed by usability testing and security testing (six each); stakeholder surveys were the least studied topic, with one study. About 59% of the studies concerned VR and about 26% augmented reality. Twenty-five studies (74%) proposed new testing solutions, 15 of them with only basic validation, and the first empirical evaluation study was published in 2019. Functionality (17 studies) and usability (8) were the main test objectives; user interaction was the most frequent functional target (five studies) and cybersickness the most frequent usability target (three). System testing was the most prominent test level (22 studies), and machine learning-based testing (nine studies) and model-based testing (four) were the only techniques used by more than one study. The authors also report that StereoID, a research tool for detecting stereoscopic visual inconsistencies linked to cybersickness, was not accessible.

Limitations

The mapping study gives collision detection and object misplacement as examples of failures that may need distinct test oracles, and proposes studying which oracle types, such as assertions, contracts or metamorphic relationships, suit different XR testing scenarios. Meta's replay workflow leaves the pass or fail decision to the CI harness, and Meta's Unity guide states that an object obscured behind another cannot be captured through screenshots alone.

The mapping study also states that human behaviour in XR environments "cannot be mathematically modelled to guarantee predictable outcomes", and that automated approaches to usability issues such as cybersickness detection rely on manual testing or live user data collection, which is time-consuming and costly.

The evaluations reported for these research tools each covered between one and eleven applications or scenes: VRGuide was tested on eight projects; VRExplorer on 11 projects; XRintTest on a ten-scene benchmark; VRScout on three commercial games; and the GPT-4o case study on one game. VRExplorer targets Unity applications, a choice its authors explain by Unity's dominant role in VR and mixed reality markets, and they note that it still needs manual scene analysis and customised interfaces, which add work and a learning curve for test engineers. Both studies of open-source projects were unable to measure code coverage in Unity projects.

See also

• How to develop VR games with AI
• Immersive Web SDK
• Oculus Debug Tool