360 Video
More actions
- Main article: VR Apps
360 Video is a type of video especially adapted to virtual reality. It is fundamentally a 3DOF form of immersive video. It allows the user to look around the scenery in 3DOF and sometimes in limited cases 6DOF.
360-degree video is also called panoramic, spherical or omnidirectional video. Its content lies on a sphere that covers the whole 360 by 180 degree viewing range around the camera, unlike conventional 2D video, which covers only a limited plane.[1] A viewer wearing a head-mounted display chooses which part of the sphere to look at by turning their head; on phones the view is changed by moving the device, and on computers by dragging with a mouse.[1][2] Because the camera position is fixed at capture time, standard 360 video responds to head rotation but not to translation of the viewing position; formats and camera systems that record depth to support limited movement are covered under Volumetric video.[3][4]
How it works
Capture and stitching
360 video is recorded with cameras or rigs that capture every direction of a scene at the same time.[5] The footage from the individual cameras is then stitched into a single spherical frame. Facebook's open Surround 360 reference design of 2016 used industrial cameras with global shutters and exported 4K, 6K or 8K video for each eye at 30 or 60 frames per second; its software corrected lens distortion and computed optical flow between pairs of cameras to derive left-eye and right-eye stereo disparity, which Facebook said cut stitching time "from weeks to overnight".[6] Google's Jump system took a similar approach: its research paper describes a rig and a fully automatic stitching pipeline that reduces stitching to pairwise image interpolation followed by compositing, using optical flow.[7]
Monoscopic and stereoscopic 360 video
Monoscopic 360 video stores a single view of the sphere, so both eyes see the same image. Stereoscopic ("3D-360") video stores a separate view for each eye. Google's open Spherical Video V2 metadata specification for MP4 and WebM files defines a monoscopic mode and several stereoscopic modes, including ones in which the left-eye and right-eye images are packed top-bottom or left-right in each frame.[8] Jump captures stereo 360 video in the omnidirectional stereo (ODS) format; the authors analysed the distortions inherent to ODS when shown in a VR headset and found them small enough for a wide variety of scenes.[7] Road to VR noted in 2015 that GoPro's smaller six-camera rig captured only monoscopic content; the 16-camera rig GoPro was building for Jump was part of a program aimed at capturing and sharing 3D VR video.[9]
Projection
Video codecs work on flat rectangular frames, so the sphere is projected onto a 2D plane for storage, processing and transmission. Codecs designed for conventional 2D video do not fit these projected frames well because of their spherical characteristics.[1] The most common layouts are:
| Projection | How it maps the sphere | Notes |
|---|---|---|
| Equirectangular (ERP) | Longitude and latitude become the horizontal and vertical axes of one rectangle, with the forward direction at the centre of the frame[8] | The poles get many pixels and the equator relatively few, although most important content sits near the horizon[10] |
| Cubemap (CMP) | The sphere is projected onto the six faces of a cube[11] | Less geometric distortion than ERP;[1] Facebook reported that its cube map output held the same information as the equirectangular input with 25 percent fewer pixels per frame[11] |
| Equi-angular cubemap (EAC) | A cubemap in which pixel samples are spaced evenly by angle rather than by distance on the cube face[10] | Developed jointly by YouTube and Google's Daydream team; Google reported more uniform pixel density than ERP or a standard cubemap and a visible quality improvement for 360 stereo video[10] |
The Spherical Video V2 specification also defines a mesh projection type for other layouts, and it stores the projection and stereo mode in the file's metadata so that players know how to display it.[8] When YouTube launched 360 video, it provided documentation and a metadata script for creators to format their files correctly.[2]
Delivery and standards
Covering the full sphere at high fidelity requires very high resolutions, and 360 video also needs high frame rates to avoid motion sickness, which places heavy demands on storage and bandwidth.[1] In 2015 Facebook engineers wrote that incoming 360 files were 4K or higher, with bit rates that could exceed 50 Mb per second (22 GB per hour), doubled for 3D stereo versions.[11] At any moment the viewer sees only the viewport, a small part of the sphere, so the region outside it is invisible and its encoded bits are largely redundant.[1] Viewport-dependent streaming exploits this: the MPEG Omnidirectional Media Format (OMAF, ISO/IEC 23090-2) specifies tile-based streaming of 360 video using HEVC tiles, which Fraunhofer HHI describes as allowing "significantly higher resolution at the end device".[12]
OMAF is the first international standard for storage and distribution of 360 video.[12] Its first edition was completed in October 2017 and covered 360 video, images and audio for three-degrees-of-freedom rendering. The second edition, finalized in October 2020, added multiple viewpoints (each corresponding to one omnidirectional camera) and overlays, and supports six-degrees-of-freedom rendering in the sense that movement of the viewing position affects how overlays are drawn over the omnidirectional background.[3]
History
| Date | Event |
|---|---|
| 13 March 2015 | YouTube announces support for uploading and viewing 360-degree video, on youtube.com in Chrome and in the YouTube app for Android, with iPhone and iPad support to follow. Compatible cameras named at launch include Bublcam, Giroptic's 360cam, IC Real Tech's Allie, Kodak's SP360 and Ricoh's Theta.[2] |
| 28 May 2015 | Clay Bavor announces Google's Jump program at the Google I/O keynote, including a 16-camera rig built by GoPro from Hero 4 cameras.[9] |
| 23 September 2015 | Facebook adds 360 video to News Feed on the web and Android, with iOS following on 12 November 2015; owners of the Samsung Gear VR could play the videos in the headset.[5] |
| 15 October 2015 | Facebook engineers describe converting 360 uploads from equirectangular to cube map layout.[11] |
| 12 April 2016 | Facebook presents Surround 360, an open 3D-360 capture system whose camera design and stitching code it planned to publish on GitHub.[6] |
| 18 April 2016 | YouTube introduces 360-degree live streaming and spatial audio for on-demand videos.[13] |
| 14 March 2017 | Google describes the equi-angular cubemap projection, developed jointly by YouTube and Daydream.[10] |
| 19 April 2017 | Facebook announces the Surround 360 x24 and x6 cameras (24 and 6 cameras), which record per-pixel depth for six-degrees-of-freedom video, and plans to license the designs to commercial partners.[4] |
| June 2017 | YouTube CEO Susan Wojcicki announces VR180 at VidCon: stereoscopic 180-degree video from a new class of cameras made with Lenovo, LG and YI Technology, viewable on Google Cardboard, Daydream and PlayStation VR and as ordinary 16:9 video on flat screens.[14] |
| October 2017 | First edition of MPEG OMAF completed.[3] |
| 2020 | Second edition of OMAF finalized (October);[3] Google researchers present a light field video system that records with 46 synchronized cameras on a 92 cm hemispherical dome and produces 6DOF video with an 80 cm viewing baseline.[15] |
VR180 trades the full sphere for a forward-facing hemisphere. Road to VR reported that it delivers higher resolution than 3D 360 video at similar or smaller file sizes.[14]
Research
A 2020 survey by Mai Xu, Chen Li, Shanyi Zhang and Patrick Le Callet groups the work into three areas: datasets and models of visual attention (where viewers look on the sphere), subjective and objective visual quality assessment, and compression methods that use either the spherical geometry or visual attention models.[1] The authors note that quality assessment methods built for 2D video are not appropriate for 360 content because of sphere-to-plane projection, and that quality loss is more noticeable in 360 video because the viewer concentrates on the small viewport.[1] Standardization projects aimed at more efficient 360 compression include MPEG-I and JPEG 360.[1]
List of 360 Videos:
See also
References
- ↑ 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 Mai Xu, Chen Li, Shanyi Zhang, Patrick Le Callet (2020). "State-of-the-art in 360° Video/Image Processing: Perception, Assessment and Compression". IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 1, pp. 5-26 (arXiv:1905.00161). doi:10.1109/JSTSP.2020.2966864. https://arxiv.org/abs/1905.00161. Retrieved 2026-09-27.
- ↑ 2.0 2.1 2.2 Sanjeev Verma (2015-03-13). "A new way to see and share your world with 360-degree video". YouTube Official Blog. YouTube. https://blog.youtube/news-and-events/a-new-way-to-see-and-share-your-world/. Retrieved 2026-09-27.
- ↑ 3.0 3.1 3.2 3.3 Sachin Deshpande, Miska M. Hannuksela (2021). "Omnidirectional MediA Format (OMAF): Toolbox for Virtual Reality Services". 2021 IEEE Conference on Standards for Communications and Networking (CSCN), pp. 20-25. https://arxiv.org/abs/2203.01183. Retrieved 2026-09-27.
- ↑ 4.0 4.1 John Mannes (2017-04-19). "Facebook will license its new 360 cameras that capture in six degrees of freedom". TechCrunch. https://techcrunch.com/2017/04/19/facebooks-new-360-cameras-that-capture-in-six-degrees-of-freedom-will-be-licensed/. Retrieved 2026-09-27.
- ↑ 5.0 5.1 Maher Saba (2015-09-23). "Introducing 360 Video on Facebook". Meta Newsroom. Meta. https://about.fb.com/news/2015/09/introducing-360-video-on-facebook/. Retrieved 2026-09-27.
- ↑ 6.0 6.1 "Introducing Facebook Surround 360: An open, high-quality 3D-360 video capture system". Engineering at Meta. Meta. 2016-04-12. https://engineering.fb.com/2016/04/12/video-engineering/introducing-facebook-surround-360-an-open-high-quality-3d-360-video-capture-system/. Retrieved 2026-09-27.
- ↑ 7.0 7.1 Robert Anderson, David Gallup, Jonathan T. Barron, Janne Kontkanen, Noah Snavely, Carlos Hernandez Esteban, Sameer Agarwal, Steven M. Seitz (2016). "Jump: Virtual Reality Video". ACM Transactions on Graphics, vol. 35, no. 6 (SIGGRAPH Asia 2016). doi:10.1145/2980179.2980257. https://doi.org/10.1145/2980179.2980257. Retrieved 2026-09-27.
- ↑ 8.0 8.1 8.2 "Spherical Video V2 RFC". google/spatial-media on GitHub. Google. https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md. Retrieved 2026-09-27.
- ↑ 9.0 9.1 Paul James (2015-05-28). "GoPro to Make 16 Camera VR Film Rig Based on Google's 'Jump' Design". Road to VR. https://roadtovr.com/gopro-make-16-camera-vr-rig-google-jump-design/. Retrieved 2026-09-27.
- ↑ 10.0 10.1 10.2 10.3 Chip Brown (2017-03-14). "Bringing pixels front and center in VR video". The Keyword. Google. https://blog.google/products/google-ar-vr/bringing-pixels-front-and-center-vr-video/. Retrieved 2026-09-27.
- ↑ 11.0 11.1 11.2 11.3 David Pio, Evgeny Kuzyakov (2015-10-15). "Under the hood: Building 360 video". Engineering at Meta. Meta. https://engineering.fb.com/2015/10/15/video-engineering/under-the-hood-building-360-video/. Retrieved 2026-09-27.
- ↑ 12.0 12.1 "MPEG-OMAF". Fraunhofer Heinrich Hertz Institute. Fraunhofer HHI. https://www.hhi.fraunhofer.de/en/departments/vca/technologies-and-solutions/mpeg-omaf.html. Retrieved 2026-09-27.
- ↑ Neal Mohan (2016-04-18). "One step closer to reality: introducing 360-degree live streaming and spatial audio on YouTube". YouTube Official Blog. YouTube. https://blog.youtube/news-and-events/one-step-closer-to-reality-introducing/. Retrieved 2026-09-27.
- ↑ 14.0 14.1 Scott Hayden (2017-06-23). "Google to Bring 3D 180 'Point-and-Shoot' Cameras to Vloggers this Winter". Road to VR. https://www.roadtovr.com/google-bring-3d-180-point-shoot-cameras-vloggers-winter/. Retrieved 2026-09-27.
- ↑ Michael Broxton, John Flynn, Ryan Overbeck, Daniel Erickson, Peter Hedman, Matthew DuVall, Jason Dourgarian, Jay Busch, Matt Whalen, Paul Debevec (2020). "Immersive Light Field Video with a Layered Mesh Representation". ACM Transactions on Graphics, vol. 39, no. 4 (SIGGRAPH 2020). https://augmentedperception.github.io/deepviewvideo/. Retrieved 2026-09-27.