Johnny Chung Lee

Johnny Chung Lee, often credited as Johnny Lee, is a human-computer interaction researcher and engineer. He became widely known in late 2007 for a video that used a Nintendo Wii Remote to turn an ordinary display into a head-tracked desktop virtual reality window, and he later worked on the human tracking algorithms for Microsoft's Kinect. At Google he led the team behind Tango (Project Tango), the depth-sensing mobile platform for augmented reality and positional tracking that preceded ARCore. As of September 2026 he is listed as a Fellow at AI Fund, the venture studio.

Reviewed 27 September 2026. Education, thesis, Wii Remote projects, Microsoft/Kinect, Google Tango roles and quotes, and AI Fund status checked against the cited CMU, johnnylee.net, blog, TechCrunch, UploadVR, Road to VR, TED, Microsoft and AI Fund pages. About review dates.

Early life and education

Lee received a B.S. in computer engineering from the University of Virginia in 2001 and a Ph.D. in human-computer interaction from Carnegie Mellon University's Human-Computer Interaction Institute in 2008. His doctoral thesis, "Projector-Based Location Discovery and Tracking", placed light sensors in a projection area and projected a series of Gray-coded structured light patterns so that each sensor could decode its own location from the projector. The thesis abstract states that unifying image projection and location tracking removes many of the calibration and alignment problems of projector-based augmented reality applications, and it describes a hybrid visible and infrared projector that could track input devices invisibly while showing visible content. Parts of this work were published at the ACM Symposium on User Interface Software and Technology (UIST), including "Automatic Projector Calibration using Embedded Light Sensors" (2004), "Moveable Interactive Projected Displays Using Projector Based Tracking" (2005) and "Hybrid Infrared and Visible Light Projection for Location Tracking" (2007). He is a named inventor on a U.S. patent assigned to Mitsubishi Electric Research Laboratories for calibrating projectors to arbitrarily shaped surfaces using optical sensors mounted on those surfaces.

During graduate school Lee also published work on kinetic typography, a haptic feedback stylus for touch screens and, with Desney Tan, the use of a low-cost electroencephalograph for task classification in HCI research (UIST 2006). In 2001 he published a web tutorial for building a camera stabilizer for US$14; his project page says it had been viewed more than 1 million times.

Career

Wii Remote projects

Lee's best-known work used the Nintendo Wii Remote as a low-cost sensor. His project page notes that the controller contains a 1024x768 infrared camera with built-in hardware blob tracking of up to 4 points at 100Hz, and a 3-axis accelerometer that also runs at 100Hz, and that most of his projects needed only the controller and a Bluetooth connection, not the Wii console. His demonstrations included finger tracking with an LED array and reflective tape, and low-cost multipoint interactive whiteboards built with infrared LED pens, and he posted the software for developers to download.

On December 21, 2007 he published "Head Tracking for Desktop VR Displays using the Wiimote". The setup reversed the Wii's usual arrangement: the Wii Remote sat by the screen and watched a head-mounted "sensor bar" of two infrared LEDs worn by the viewer, so the software could track the position of the viewer's head and render view-dependent images. Lee described the effect as turning the display into "a portal to a virtual environment" that reacts to head and body movement "as if it were a real window". The program, a custom C# DirectX application released as sample code, needed only the size of the display and the size of the sensor bar. In a follow-up post he noted that head tracking alone left conflicting stereo depth cues, and that adding shutter or polarized glasses for stereoscopic images would strengthen the illusion; he also gave the Wii Remote camera's field of view as only 45 degrees.

The "Head Tracking for Desktop VR" video was nominated for best instructional video at the YouTube Awards. UploadVR wrote in 2016 that the video had passed 10 million views, and quoted Lee: "Forever I will be known as the Wiimote guy on Youtube." In 2008 he presented "Free or cheap Wii Remote hacks" at TED.

Microsoft and Kinect

Lee joined Microsoft in June 2008 as a researcher in its Applied Sciences Group. Microsoft's profile describes the role as exploring novel input and output devices, advising the direction of existing hardware product lines and building prototypes that might become products; it lists his research areas as projection technology, multitouch input, augmented reality, brain-computer interfaces and haptics. In August 2008 Technology Review named him to its TR35 list of innovators under 35, while he was working at Microsoft.

At Microsoft he spent "more than 2 and a half years working as a core contributor to the human tracking algorithms for Kinect", from early incubation before the project was called "Project Natal" through a launch that shipped 8 million units in its first 60 days.

Google and Project Tango

On January 18, 2011, Lee announced that he had left Microsoft to join a special projects team at Google; TechCrunch reported his title there as "Rapid Evaluator". When Google's Advanced Technology and Projects (ATAP) group announced Project Tango on February 20, 2014, TechCrunch identified Lee as ATAP's technical program lead. The first prototype was a 5-inch Android phone with Kinect-like sensors and a Movidius Myriad 1 vision processor, offered to 200 selected developers. Lee said the project "strives to give mobile devices a human-like understanding of space and motion through advanced sensor fusion and computer vision." AI Fund's biography describes him as the founder of Project Tango.

Lee was a keynote speaker at Augmented World Expo 2015, where the conference billed him as Google's Project Tango lead. In March 2016 UploadVR described him as technical lead on Tango and framed his work as an attempt to solve mobile positional tracking for VR and AR, a problem that Oculus CTO John Carmack was also working on. Lee named power consumption and heat as the main obstacles on phones, and said of solving mobile positional tracking: "I definitely think we can get there." At GDC 2016 Tango added Unreal Engine support alongside existing Unity support, and Lenovo planned to release the first Tango phone later that year.

By January 2017 UploadVR called him head of Google's Tango team. Speaking about the ASUS ZenFone AR, the first phone to support both Tango and Daydream, he said the two platforms ran separately because "VR has a lot of compute requirements. Tango has a lot of compute requirements", which made thermals and heating an issue; he said he was most interested in uses that saved people time or money, such as indoor navigation and shopping. In a June 2017 interview with Kent Bye for Road to VR, which identified him as Tango's engineering director, Lee discussed Tango's tracking, depth sensing, area learning and visual positioning system. Bye noted that Tango was the only phone AR platform built around a depth-sensor camera, and that its visual positioning system for helping customers find products in stores was in early testing with the retailer Lowe's.

On December 15, 2017, Google announced that it would end support for Tango on March 1, 2018 and concentrate on ARCore, which TechCrunch described as its more mass-market AR product. AI Fund's biography states that Tango "eventually evolved into ARCore and the Geospatial API".

Later work

According to AI Fund, Lee's later work at Google included contributing to the development of Google Glass, leading robotics work on learning from human teleoperation, and directing wearable AI projects for AR, including work that influenced Google's Project Astra. He later joined AI Fund as a Fellow, where he works with the firm's founders.

Legacy

Lee's head-tracking demonstration used a consumer game controller and two infrared LEDs to produce view-dependent 3D images on a normal screen, and the video remained the work he was best known for when UploadVR profiled him in 2016. Dan Siewiorek, then director of Carnegie Mellon's Human-Computer Interaction Institute, said in 2008 that Lee had "the uncanny ability to look at problems from new perspectives and create simple, low cost solutions."