Toggle search
Toggle menu
5K
5.9K
229
47.7K
VR & AR Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Google Lens

From VR & AR Wiki
More actions
Google Lens
Information
Type Visual search
Industry Internet search
Developer Google
Operating System Android, iOS
Supported Devices Android phones (standalone app and camera apps of select devices), iPhone and iPad (Google app and Google Photos), Chrome desktop
Release Date 2017
Website https://lens.google/


Google Lens is a camera-based visual search tool developed by Google. A user points a phone camera at an object, text or place, or selects an existing image or screenshot, and Lens identifies what it sees and returns search results, translations or actions. Google CEO Sundar Pichai announced Lens at the Google I/O developer conference on May 17, 2017,[1] and Google first made it available as a preview on its Pixel phones later that year.[2][3]

Lens is mainly a computer vision and search product, but several of its features place digital information over the live camera view, which is a form of phone-based augmented reality. In 2018 Lens began anchoring results to objects in the viewfinder in real time, and later features overlay translated text on the original words and highlight popular dishes directly on a physical menu.[4][5] Google's own I/O 2019 announcement described these Lens and Search features as using "the camera, computer vision and augmented reality (AR)" to overlay information onto physical surroundings.[5]

Usage has grown steadily. Google reported more than a billion questions asked of Lens by May 2019, 8 billion questions a month in September 2022, and nearly 20 billion visual searches a month in October 2024.[5][6][7] In June 2025, Google said more than 1.5 billion people were using Lens each month.[8]

Reviewed 6 October 2026. Re-opened all cited Google blog, help, store and press pages and both plant-identification papers to check dates, figures, feature descriptions, author names and citation titles. About review dates.

History

Announcement and first releases (2017-2018)

At Google I/O 2017, Pichai described Lens as "a set of vision-based computing capabilities that can understand what you're looking at and help you take action based on that information," and said Google would bring it first to Google Assistant and Google Photos.[9] The demonstrations included identifying a flower the user was about to photograph, connecting to a home Wi-Fi network from a photo of the sticker on the router, and showing a business's name, rating and other listing information from a view of its storefront. Google did not give a release date at the event.[1]

At its hardware event on October 4, 2017, Google announced that Lens would launch that year as a preview on the Pixel 2 and Pixel 2 XL, inside Google Assistant and Google Photos.[2] On November 21, 2017, Google said Lens in Assistant would roll out over the following weeks to Pixel and Pixel 2 phones set to English in the United States, United Kingdom, Australia, Canada, India and Singapore.[3]

Lens reached Google Photos on non-Pixel Android phones in early March 2018,[10] and Google Photos for iOS in mid-March 2018.[11] A standalone Lens app appeared on the Google Play Store in June 2018 for phones running Android Marshmallow or later.[12] On December 10, 2018, Lens began rolling out in the Google app for iOS with a live viewfinder.[13]

Lens replaced Google Goggles, an older Google image recognition app. In August 2018 Goggles received its first update since 2014, which replaced the app with a message telling users to install Lens.[14]

Real-time results and camera integration (2018)

At Google I/O 2018, Rajan Patel, then Google's director for Lens, announced three features. Smart text selection let users copy text such as recipes, gift card codes or Wi-Fi passwords from the real world to the phone. Style search showed items similar to a piece of clothing or home decor in view. Real-time results let Lens "proactively surface information instantly" and "anchor it to the things you see," running on both on-device machine learning and Google's cloud TPUs. Google also announced that Lens would be available directly in the camera apps of supported devices from LGE, Motorola, Xiaomi, Sony Mobile, HMD/Nokia, Transsion, TCL, OnePlus, BQ, Asus and Google Pixel.[4][15]

Overlays, reading and translation (2019-2023)

The I/O 2019 update added two overlay features. With the dining feature, Lens highlights popular dishes on a restaurant's physical menu and shows photos and reviews from Google Maps when a dish is tapped. With translation, Lens detects the language of text in view and overlays the translation on top of the original words. Google also showed recipe pages from Bon Appetit that would "come to life" in the camera, and a version of Lens in Google Go, its search app for first-time smartphone users, that reads text aloud.[5] By then, Google said, people had asked Lens more than a billion questions.[5]

At its Search On event in September 2022, Google described a new form of Lens translation that uses generative adversarial networks (GANs) to blend translated text into the background of the image, and announced that multisearch would expand to more than 70 languages.[6] Press coverage of the event called the feature "AR Translate" and reported that it worked in 100 milliseconds on screenshots and in the live Lens camera.[16] In December 2022, Google Translate's camera mode on Android and iOS was replaced with Lens, opened directly to its translate filter.[17] In February 2023, Google said the blended translation was rolling out on Android phones with 6 GB of RAM or more.[18]

Multisearch and desktop (2022-2023)

Multisearch, launched as a beta in English in the United States on April 7, 2022, lets a user add a text query to a Lens image search, for example to find a dress in a different color.[19] On February 8, 2023, Google made multisearch available globally on mobile in all languages and countries where Lens was offered, said people were using Lens more than 10 billion times a month, and announced a "search your screen" feature for Android.[20] By March 2022, Chrome for desktop had replaced the right-click reverse image search option with a Google Lens option.[21]

Generative AI features (2024-2026)

In October 2024, Google added the ability to search with Lens by recording a video and asking questions about moving objects (initially for Search Labs users), voice questions while taking a photo, and a redesigned shopping results page.[7] On September 24, 2025, Google launched Search Live in English in the United States; users already pointing their camera with Lens can select a Live option to hold a spoken, real-time conversation with Search about the camera view.[22] Google expanded Search Live to more than 200 countries and territories on March 26, 2026, again reachable from the Lens camera.[23] In October 2025, Google added a Create mode to Lens in the Google app for Android and iOS that edits photos with its Nano Banana image model, first in English in the United States and India.[24]

The following table lists selected milestones.

Date Milestone Source
May 17, 2017 Announced by Sundar Pichai at Google I/O 2017 [1]
October 4, 2017 Preview announced for Pixel 2 in Google Assistant and Google Photos [2]
November 2017 Lens in Google Assistant rolls out to Pixel phones in six English-speaking countries [3]
March 2018 Google Photos on non-Pixel Android phones, then Google Photos for iOS [10][11]
May 8, 2018 Real-time results, smart text selection, style search; camera-app integration on partner phones [4]
June 2018 Standalone Android app on Google Play [12]
August 2018 Google Goggles update directs users to Lens [14]
December 10, 2018 Lens in the Google app for iOS [13]
May 2019 Dining highlights and translation overlay; Lens in Google Go [5]
April 7, 2022 Multisearch beta (English, United States) [19]
September 28, 2022 GAN-based translation that blends text into the image announced [6]
December 2022 Lens replaces the camera mode in Google Translate [17]
February 8, 2023 Multisearch available globally on mobile [20]
October 3, 2024 Video search and voice questions [7]
September 24, 2025 Search Live reachable from the Lens camera (English, United States) [22]
October 13, 2025 Create mode with the Nano Banana image model [24]
March 26, 2026 Search Live expanded to more than 200 countries and territories [23]

How it works

According to Google, Lens uses computer vision, machine learning and Google's Knowledge Graph to let people turn the things they see in the real world into a visual search query.[25] Rajan Patel, then listed as Google's Director of Augmented Reality, described the reading pipeline built for Lens in Google Go in a 2019 Google Research blog post. The app captures an image through Android's CameraX library, then scales it down and sends it to the Lens server, where the processing is performed. There, optical character recognition uses a region proposal network to find character-level bounding boxes, which are merged into lines with a Hough transform and a "Text Flow" step that can follow curved text. Separable convolutional neural networks with a quantized long short-term memory (LSTM) network handle script detection, direction and recognition, and the Knowledge Graph and nearby words are used to correct OCR errors. Convolutional networks also detect coherent text blocks such as columns, and signals such as alignment, language and the geometry of paragraphs set the reading order. Google Translate's neural machine translation then translates entire sentences rather than single words. To overlay the translation in context, Lens redistributes it into lines of similar length, picks a matching font size and matches the colors of the translation and its background to the original, using a heuristic that assumes the text and background differ in luminosity. Finally, Google Text-to-Speech, based on DeepMind's WaveNet, reads the text aloud while each word is highlighted.[25]

For the I/O 2018 real-time mode, Google said Lens used machine learning with "both on-device intelligence and cloud TPUs" to identify "billions of words, phrases, places, and things in a split second."[4]

Augmented reality features

Lens runs on phones, tablets and desktop browsers rather than headsets. Its camera overlays are a form of Phone-Powered AR: the live camera image is the background, and Lens draws information on top of it. The main overlay features are listed below.

Feature What appears over the camera view Introduced Source
Real-time results Results surfaced instantly and anchored to objects as the viewfinder moves May 2018 [4]
Dining Popular dishes highlighted on the physical menu May 2019 [5]
Translate overlay Translation placed over the original words, more than 100 languages May 2019 [5]
Blended translation ("AR Translate") Original text erased and replaced with translated text that matches the background Announced September 2022; rollout announced February 2023 [6][18]

The Lens name also appears in Google Maps. Search with Live View, announced in September 2022 for London, Los Angeles, New York, San Francisco, Paris and Tokyo, lets a user lift the phone on a street to see nearby places such as ATMs, coffee shops, grocery stores and transit stations in Live View, along with details such as opening hours and how busy a place is.[26] In October 2023 Google renamed it Lens in Maps, describing it as using "AI and augmented reality", and expanded it to more than 50 additional cities including Austin, Las Vegas, Rome, São Paulo and Taipei.[27] In January 2024 the feature was given the full-color Lens icon in the Maps search bar.[28]

Circle to Search, a gesture-based search feature Google announced for Android phones in January 2024, lets users search what is on their screen by circling, highlighting or tapping it, and includes multisearch.[29] Google's help page for Android XR devices describes searching the real world with Circle to Search or the camera icon in the Google app, with passthrough mode turned on; the page does not mention Lens.[30]

Availability

As of October 2026, Google's Lens website lists the Google app on Android and iOS, Google Photos, Chrome on desktop and the camera app of select Android devices as ways to use Lens. Its listed uses include translating text in more than 100 languages, copying text, identifying plants, animals, artwork and landmarks, shopping for products seen in photos, step-by-step homework help, asking spoken questions about a photo and editing photos with text prompts in Create mode.[31] The standalone Android app is published by Google LLC under the package name com.google.ar.lens and lists more than one billion downloads on Google Play.[32]

Research

Lens has been evaluated in academic studies, mostly of its plant identification. A paper by Viktor B. Shapovalov and colleagues at the 2nd International Workshop on Augmented Reality in Education (AREdu 2019) tested Lens on botanical photos for use in Ukrainian school biology teaching. It found that Lens correctly identified plant species in 92.6% of cases, with the best results for trees and plant stems, and that it could not analyze local endemic Ukrainian flora.[33] A 2023 study in PLOS ONE by Neil Campbell, Julie Peacock and Karen L. Bacon compared six apps (Google Lens, iNaturalist, Leaf Snap, Plant Net, Plant Snap and Seek) on 38 herbaceous species photographed with a Samsung Galaxy A50. Plant Net and Leaf Snap outperformed the others; the authors wrote that "Google Lens and iNaturalist both did well" and noted that its wide availability and generally "good" scores make it potentially useful for casual users.[34]

See also

References

  1. ↑ 1.0 1.1 1.2 Sarah Perez (2017-05-17). "Google Lens will let smartphone cameras understand what they see and take action". TechCrunch. https://techcrunch.com/2017/05/17/google-lens-will-let-smartphone-cameras-understand-what-they-see-and-take-action/. Retrieved 2026-10-06.
  2. ↑ 2.0 2.1 2.2 Abner Li (2017-10-04). "Google Lens launching this year as 'preview' on the Pixel 2 in Assistant and Google Photos". 9to5Google. https://9to5google.com/2017/10/04/google-lens-launching-this-year-as-preview-on-the-pixel-2-in-assistant-and-google-photos/. Retrieved 2026-10-06.
  3. ↑ 3.0 3.1 3.2 Abner Li (2017-11-21). "Google Lens rollout in Assistant announced, coming soon to Pixel in six countries". 9to5Google. https://9to5google.com/2017/11/21/google-lens-rollout-official-coming-weeks/. Retrieved 2026-10-06.
  4. ↑ 4.0 4.1 4.2 4.3 4.4 Rajan Patel (2018-05-08). "Google Lens: real-time answers to questions about the world around you". The Keyword (Google blog). Google. https://blog.google/products/google-lens/google-lens-real-time-answers-questions-about-world-around-you/. Retrieved 2026-10-06.
  5. ↑ 5.0 5.1 5.2 5.3 5.4 5.5 5.6 5.7 Aparna Chennapragada (2019-05-07). "Helpful new visual features in Search and Lens". The Keyword (Google blog). Google. https://blog.google/products-and-platforms/products/search/helpful-new-visual-features-search-lens-io/. Retrieved 2026-10-06.
  6. ↑ 6.0 6.1 6.2 6.3 Prabhakar Raghavan (2022-09-28). "Search outside the box: How we're making Search more natural and intuitive". The Keyword (Google blog). Google. https://blog.google/products-and-platforms/products/search/new-search-technology/. Retrieved 2026-10-06.
  7. ↑ 7.0 7.1 7.2 Elizabeth Reid (2024-10-03). "Ask questions in new ways with AI in Search". The Keyword (Google blog). Google. https://blog.google/products-and-platforms/products/search/google-search-lens-october-2024-updates/. Retrieved 2026-10-06.
  8. ↑ Rose La Prairie (2025-06). "Google Lens co-founder on 'searching like a two-year old'". Think with Google. Google. https://business.google.com/uk/think/search-and-video/google-lens-ai-visual-search/. Retrieved 2026-10-06.
  9. ↑ Sundar Pichai (2017-05-17). "Making AI work for everyone". The Keyword (Google blog). Google. https://blog.google/technology/ai/making-ai-work-for-everyone/. Retrieved 2026-10-06.
  10. ↑ 10.0 10.1 Timothy J. Seppala (2018-03-06). "Google Lens is now available for non-Pixel Android phones". Engadget. https://www.engadget.com/2018-03-06-google-lens-android-iphone.html. Retrieved 2026-10-06.
  11. ↑ 11.0 11.1 Tim Hardwick (2018-03-16). "Google Lens Currently Rolling Out to iOS via Google Photos App". MacRumors. https://www.macrumors.com/2018/03/16/google-lens-rolling-out-to-ios/. Retrieved 2026-10-06.
  12. ↑ 12.0 12.1 Ryan Whitwam (2018-06-04). "Google releases standalone Lens app in Play Store". Android Police. https://www.androidpolice.com/2018/06/04/google-releases-standalone-lens-app-play-store/. Retrieved 2026-10-06.
  13. ↑ 13.0 13.1 Abner Li (2018-12-10). "Google Lens visual search now rolling out to Google app for iOS". 9to5Google. https://9to5google.com/2018/12/10/google-lens-ios-rolling-out/. Retrieved 2026-10-06.
  14. ↑ 14.0 14.1 Corbin Davenport (2018-08-17). "Google Goggles is dead, now prompts users to install Lens". Android Police. https://www.androidpolice.com/2018/08/16/google-goggles-dead-now-prompts-users-install-lens/. Retrieved 2026-10-06.
  15. ↑ Abner Li (2018-05-08). "Google Lens integrating w/ camera apps, adding smart text selection & real-time results". 9to5Google. https://9to5google.com/2018/05/08/google-lens-smart-text-real-time/. Retrieved 2026-10-06.
  16. ↑ Abner Li (2022-09-28). "Google Lens getting 'AR Translate' and expanding Multisearch to more languages". 9to5Google. https://9to5google.com/2022/09/28/google-lens-ar-translate/. Retrieved 2026-10-06.
  17. ↑ 17.0 17.1 Abner Li (2022-12-05). "Google Lens replaces Google Translate's camera mode on Android and iOS". 9to5Google. https://9to5google.com/2022/12/05/google-lens-translate-camera/. Retrieved 2026-10-06.
  18. ↑ 18.0 18.1 Ivan Mehta (2023-02-08). "Google announces improved contextual translation features". TechCrunch. https://techcrunch.com/2023/02/08/google-announces-improved-contextual-translation-features/. Retrieved 2026-10-06.
  19. ↑ 19.0 19.1 Aisha Malik (2022-04-07). "Google's new 'multisearch' feature lets you search using text and images at the same time". TechCrunch. https://techcrunch.com/2022/04/07/googles-multisearch-search-using-text-images/. Retrieved 2026-10-06.
  20. ↑ 20.0 20.1 Sarah Perez (2023-02-08). "Google's AI-powered 'multisearch,' which combines text and images in a single query, goes global". TechCrunch. https://techcrunch.com/2023/02/08/googles-a-i-powered-multisearch-which-combines-text-and-images-in-a-single-query-goes-global/. Retrieved 2026-10-06.
  21. ↑ Steve Huff (2022-03-20). "Google Chrome drops right-click reverse image search in favor of Lens". Android Police. https://www.androidpolice.com/google-chrome-drops-right-click-reverse-image-search-in-favor-of-lens/. Retrieved 2026-10-06.
  22. ↑ 22.0 22.1 Liza Ma (2025-09-24). "5 ways to get real-time help by going Live with Search". The Keyword (Google blog). Google. https://blog.google/products-and-platforms/products/search/search-live-tips/. Retrieved 2026-10-06.
  23. ↑ 23.0 23.1 Liza Ma (2026-03-26). "Search Live is expanding globally". The Keyword (Google blog). Google. https://blog.google/products-and-platforms/products/search/search-live-global-expansion/. Retrieved 2026-10-06.
  24. ↑ 24.0 24.1 Lou Wang (2025-10-13). "How to edit images with Nano Banana in Search". The Keyword (Google blog). Google. https://blog.google/products/search/nano-banana-search-lens/. Retrieved 2026-10-06.
  25. ↑ 25.0 25.1 Rajan Patel (2019-09-04). "Giving Lens New Reading Capabilities in Google Go". Google Research Blog. Google. https://research.google/blog/giving-lens-new-reading-capabilities-in-google-go/. Retrieved 2026-10-06.
  26. ↑ Chris Phillips (2022-09-28). "4 new updates that make Maps look and feel more like the real world". The Keyword (Google blog). Google. https://blog.google/products-and-platforms/products/maps/4-new-updates-maps-searchon-2022/. Retrieved 2026-10-06.
  27. ↑ Chris Phillips (2023-10-26). "New Maps updates: Immersive View for routes and other AI features". The Keyword (Google blog). Google. https://blog.google/products-and-platforms/products/maps/google-maps-october-2023-update/. Retrieved 2026-10-06.
  28. ↑ Abner Li (2024-01-19). "Google Lens AR search in Google Maps gets more prominent icon". 9to5Google. https://9to5google.com/2024/01/19/google-maps-lens/. Retrieved 2026-10-06.
  29. ↑ Cathy Edwards (2024-01-17). "Circle (or highlight or scribble) to Search". The Keyword (Google blog). Google. https://blog.google/products-and-platforms/products/search/google-circle-to-search-android/. Retrieved 2026-10-06.
  30. ↑ "How to search with Google on Android XR". Google Search Help. Google. https://support.google.com/websearch/answer/16599167?hl=en. Retrieved 2026-10-06.
  31. ↑ "Lens". Google Search. Google. https://search.google/ways-to-search/lens/. Retrieved 2026-10-06.
  32. ↑ "Google Lens - Apps on Google Play". Google Play. Google. https://play.google.com/store/apps/details?id=com.google.ar.lens. Retrieved 2026-10-06.
  33. ↑ Viktor B. Shapovalov, Yevhenii B. Shapovalov, Zhanna I. Bilyk, Anna P. Megalinska, Ivan O. Muzyka (2020). "The Google Lens analyzing quality: an analysis of the possibility to use in the educational process". Proceedings of the 2nd International Workshop on Augmented Reality in Education (AREdu 2019), CEUR Workshop Proceedings, vol. 2547, pp. 117-129. https://ceur-ws.org/Vol-2547/paper09.pdf. Retrieved 2026-10-06.
  34. ↑ Neil Campbell, Julie Peacock, Karen L. Bacon (2023-04-05). "A repeatable scoring system for assessing Smartphone applications ability to identify herbaceous plants". PLOS ONE, vol. 18, no. 4, e0283386. https://doi.org/10.1371/journal.pone.0283386. Retrieved 2026-10-06.
Retrieved from "https://vrarwiki.com/index.php?title=Google_Lens&oldid=47409"