Google Visual Search Coming for Android Phones

By Clint Boulton  |  Posted 2009-12-04

Google Visual Search Coming for Android Phones

Google is working on a mobile application that lets users take a picture of a location from their Android-powered smartphone and trigger a Google search that pulls up information associated with the image.  

Google, which revealed its plans on CNBC's "Inside the Mind of Google," segment Dec. 3, calls the application Google Visual Search.

In a demonstration for CNBC interviewer Maria Bartiromo, Google Product Manager Hartmut Neven took a picture of Santa Monica pier and Google's positively identified it in search results.

"Imagine you're a tourist and you arrive at this place and you would like to know more about it, all you will have to do is take a shot of the [Santa Monica pier] sign and you see we recognized this as the Santa Monica pier," Neven said.

However, the technology, known internally as Google Goggles, didn't pass muster when Google tested it with a focus group in August. The company's engineers are working out the bugs and building out the immense database required to propel the technology.

If Neven's name sounds familiar for those who follow Google, that's because Google acquired his startup Neven Vision, which made facial and image recognition biometric software, in 2006.

While the technology was initially used to provide image recognition for Google's Picasa photo-sharing application, sleuthing journalists dug up patents filed by Neven that point to the broader implications of the technology. Specifically, there is one for image-based contextual advertisement method and branded barcodes.

The technology allows users to take an image from a camera phone and uses visual recognition engines to recognize objects shown in the image, and return search results based upon that recognition. This is the Visual Mobile Search (VMS) service, and Neven provides more anecdotes of its potential in the patent application:

"Imagine you are on travel in Paris and you visit a museum. If a picture catches your attention you can simply take a photo and send it to the VMS service. Within seconds you will receive an audio-visual narrative explaining the image to you. If you happen to be connected to a 3G network the response time would be below a second. After the museum visit you might step outside and see a coffeehouse."

How Google Visual Search Will Make Google Money

"Just taking another snapshot from within the VMS client application is all you have to do in order to retrieve travel guide information. In this case location information is available through triangulation or inbuilt GPS it can assist the recognition process. Inside the coffeehouse you study the menu but your French happens to be a bit rusty. Your image based search engine supports you in translating words from the menu so that you have at least an idea of what you can order."

Of course, those services can be paired with mobile advertising, as Neven noted in his patent app:

"Visual advertising content may be displayed on a digital billboard or large television screen. A user may take of picture of the billboard and the displayed advertisement to get additional information about the advertised product, enter a contest, etc. The effectiveness of the advertisement can be measured in real time by counting the number of "clicks" the advertisement generates from camera phone users. The content of the advertisement may be adjusted to increase its effectiveness based on the click rate.

The billboard may provide time sensitive advertisements that are target to passing camera phone users such as factory workers arriving leaving work, parents picking up kids from school, or the like. The real-time click rate of the targeted billboard advertisements may confirm or refute assumptions used to generate the targeted advertisement."

Pairing digital ads, possibly from Google's AdMob acquisition, is a natural way to make money from Visual Search. Many people take pictures from their increasingly improving smartphone camera daily, but imagine if those users could leverage the images as search tools instead of simply fun pictures to look at.

Then imagine if a user snapped a snapped a picture of a clothing garment through a storefront window on the street, and imagine if the Visual Search retrieved information about that item without entering the store.

This would save users a considerable amount of time, and if Google then allows users to purchase that clothing item through Google Checkout linked to the retailer, it becomes a powerful e-commerce value proposition.

This technology will eventually be coming to Android phones. Neven didn't tell Bartiromo how soon or whether the app would also be available for Apple's iPhone or other smartphone platforms.

Meanwhile, visual search recognition apps, such as ViPR from Evolution Robotics, are being offered for the iPhone, so Google has some competition when it decides to release Google Visual Search. 

Rocket Fuel