Today, we launched a new version of Google Goggles that enhances the camera on your Android-powered phone. With this new opt-in feature in Goggles, you can simply photograph an image using your phone’s camera, and Goggles will work in the background to analyze your image. If your photo contains items that Goggles can recognize, the app will notify you.
Let’s say that I’m going on vacation, and I decide to use my Android-powered phone as my primary camera. Goggles would identify landmarks, paintings and other interesting objects in my photos. I can share these facts about my vacation with my friends right from my Goggles search history.
(l-r) Goggles has found a search result for this popular landmark; Find out more about this item in your photo within Google Goggles
Photos you take with your phone’s camera will only be seen by Goggles if you enable the Search from Camera feature. Here’s how:
launch Google Goggles
tap Menu > Settings > Search from Camera
Manage your opt-in status from the settings menu
Google Goggles 1.6 is available for Android 2.1 and newer devices. Download by clicking here or by scanning the QR code below:
The Google Goggles team has worked with The J. Paul Getty Museum in Los Angeles to “Goggles-enable” their permanent collection of paintings. You can use the Google Goggles app on your phone to take a photo of a painting from the collection and instantly access information about it from the Getty’s mobile-optimized website and the rest of the web. It's possible to fit only a small amount of information on the wall next to a painting but visitors with Goggles can now enjoy the full story online.
Google Goggles results and the Getty webpage for Portrait of the Sisters Bonaparte
Some of you may already be using the new optical character recognition (OCR) and translation of Russian in Google Goggles that we previewed at last week’s Inside Search event. Starting today, we’re pleased to introduce some additional new features, including a map view of your Search History and the ability to copy contact and text results to the clipboard. We’ve also changed the results interface to make it easier to view and navigate through your results.
Russian optical character recognition (OCR) and translation Since Google Goggles first launched in 2009, it has been able to recognize and translate text in a number of different languages, as long as the language used Latin characters. With the launch of OCR for Russian, Goggles is now able to read Cyrillic characters. Goggles will recognize a picture of Russian text and allow you to translate the text into one of over 40 other languages. Russian OCR is also available for users of Google Goggles on the Google Search app for iOS. Очень полезно!
You can take a picture of Russian text and translate it into over 40 languages.
Map view of your search history If you’ve enabled search history on Goggles, your history contains a list of all the images that you’ve searched for, as well as some information about where you performed the search if you chose to share your location with Google. Sometimes this can be a pretty long list, so we wanted to give you another way to sort and visualize your Goggles results.
We’ve added a map view, which shows your Goggles image search history on a map so you can quickly zoom and pan to find a query from a particular location.
Easily toggle between map view and list view with the button in the upper right.
Copy contact and text results to clipboard Finally, imagine that you wanted to grab a URL or telephone number from a sign and email it to yourself. Now, Goggles gives you the option of copying the recognized text to your phone’s clipboard, allowing you to paste the test into a number of applications.
To try these new features download Google Goggles 1.5 from Android Market, or scan the QR code below with your phone.
Since its launch, we have worked to make Google Goggles faster and more accurate when returning search results for a wide variety of images. Today, we are taking several steps to make Goggles a better and more personal experience. Goggles 1.4 for Android devices introduces an enhanced search history experience, the ability to suggest better results to Google if we are not able to accurately match your image, and improved business card recognition.
Enhanced search history The new search history experience lets you search your Goggles results, make personal notes on specific results and share your results with friends. When you add a personal note to a Goggles result, the note will appear in your search history. I’m trying to learn more about wine, so when I taste something new, it’s easy for me to add a note to help me remember what I liked about the wine. Later, I can search my search history for words in my note to help me find that bottle that went so well with steak. Read more about how to enable search history for Goggles here.
To make a personal note, tap the pencil in the corner when viewing a search result.
Notes are intended to help you better organize your search history, so if you choose to share a result with a friend, your notes will not be shared. However, you can always add a personalized message to your friend when you share your results with them.
Share a result by sending a link to your friends.
Suggest a better result We are constantly working to improve the accuracy of Goggles at recognizing certain categories of items, but visual recognition is still a complicated task. With Goggles 1.4, you are now able to suggest a better result when Goggles cannot find an image match, or the quality of the result is poor.
To send your suggestion to Google, tap “Can you suggest a better result?” on the results page. You can then select the relevant part of the image and submit a tag. Tags will be used to improve recognition in object categories where Goggles already provides some results, like artwork or wine bottles.
When suggesting a better result, you can crop the image and add a description.
Improved business card recognition Business card recognition is one of the most popular uses of Google Goggles, so we're rolling out some new updates to make the experience even quicker and easier. Additionally, instead of simply recognizing the content as text, Goggles now recognizes the information as a contact, making it easier to add to your phone's contact list.
Call or add the person directly as a contact
Google Goggles is available for Android 1.6+ devices. Download it by visiting Android Market or by scanning the QR code below:
Today we’re launching a new version of Google Goggles that’s faster and smarter than ever before. The new Goggles 1.3 client for Android can scan barcodes almost instantly. All versions of Goggles can now recognize print ads in popular magazines and newspapers. Finally, Goggles has also learned a fun new trick for Sudoku fans.
Barcodes
When shopping offline, it’s helpful to be able to learn more about a product by scanning its barcode. With the new Android version of Google Goggles, scanning barcodes is much faster. Open Goggles and hover over the barcode or QR code. Within a second the phone gently vibrates and presents results, without requiring a button press. Simply tap on the result to read product reviews, check in-store availability and compare prices.
Print ads in magazines and newspapers
We’re excited to take another step in our vision of connecting offline media to online media. The next time you're flipping through the pages of your favorite magazine, try taking a picture of an ad with Goggles. Goggles will recognize print ad and return web search results about the product or brand. This new feature of Goggles is enabled for print ads appearing in major U.S. magazines and newspapers from August 2010 onwards.
This feature is different from the marketing experiment that we announced in November. We're now recognizing a much broader range of ads than we initially included in our marketing experiment. And when we recognize a print ad, we return web search results. While in the experiment, we return a specific link to an external website.
Sudoku
Our favorite weekend distraction is a quiet 15 minutes spent solving a Sudoku puzzle. But even that can be an frustrating experience if (like us) you make a mistake and are unable to solve the puzzle. Now, Goggles on Android and iPhone can recognize puzzles and provide answers to help make you faster than a Sudoku champ. So if you ever get stuck, take a clear picture of the entire puzzle with Goggles and we’ll tell you the correct solution. Check out this video to see how it works.
Google Goggles 1.3 with improved barcode scanning is available for download in Android Market. Recognition of print ads and Sudoku solver is now enabled for the Google Goggles app on Android, as well as the Goggles component of the Google Mobile App on iPhone.
Posted by Leon Palm and Jiayong Zhang, Software Engineers
About a year ago, we launched Google Goggles, an app that enables people to search the web by taking pictures with their mobile devices. Since then, people have used Goggles to search for information about lots of different things: famous paintings, landmarks, products and much more.
Today, we’re announcing a Google Goggles marketing experiment. We’re working with five brands—Buick, Disney, Diageo, T-Mobile and Delta Airlines—to extend some of their offline marketing to the mobile web. They have “Goggles-enabled” some of their print ads, movie posters and other media. When users take pictures of these with Google Goggles, they will be recognized by the app, and users will have the option of clicking-through directly to a mobile destination from the brand.
For a closer look at these specific campaigns, take a look at our video:
We developed Google Goggles so that people could more easily explore the world around them with a mobile device. In this experiment, we’re applying the same principles, and the same technology, by “Goggles-enabling” advertisements and other media, and offering to link people to the mobile sites from these brands.
To interact with these experimental campaigns, download Google Goggles from the Android Market or get Google Mobile App for iPhone from the iTunes store, and look for advertising and products from companies in the video above, and others that are coming soon.
Posted by Shailesh Nalawadi, Product Manager, Google Goggles
Searching with Google on your mobile phone can be an eye-opening experience. Now your phone’s camera can even help to find out more about the world around you.
Back in December we introduced Google Goggles on Android: it lets you search by taking pictures. Today, we’re excited to announce that Goggles is now a feature of Google Mobile App for iPhone. Google Mobile App already lets you speak your queries, and it can also use your phone’s location to give you more relevant search results.
In the new version of Google Mobile App just tap on the camera button to search using Goggles. Goggles will analyze the image and highlight the objects it recognizes -- just click on them to find out more. You can see how this works in this short video:
Computer vision is a hard problem and Google Goggles is still a Labs product. It works well for things such as landmarks, logos and the covers of books, DVDs and games. However, it doesn’t yet work for some things you might want to try like animals, plants or food.
Google Mobile App with Google Goggles is now available in the Apple App Store. Just search for “Google Mobile App” and download the latest version - it’s free. It will gradually be appearing in all App Stores around the world starting from today.
Goggles is enabled only for English-speaking users, and since it requires an auto-focusing camera it is supported only on iPhone 3GS and iPhone 4 running iOS 4 or above. Visit our Help Center to learn more or send us your feedback and questions in our Help Forum.
So step away from your computer, get your Goggles on, and search the world around you.
Posted by Milan Broum, Software Engineer, Google Mobile Team
By now, some of you may have noticed our new Google Mobile YouTube channel, with a fresh look that includes a video box that rotates along multiple axes (we love the Rubik’s Cube here at Google). As you click on and discover more videos, we hope you’ll have fun checking out all the ways the box can turn.
In our channel’s “featured” section, we highlight some of our mobile search capabilities -- like search by voice, sight, and location -- that really help you take advantage of your phone’s unique technology. Since we understand that these represent new modes of searching for many of you, we’re going to launch some new videos that help illustrate the variety of practical and fun ways that you can use these features. To start with, check out our “Shoot This” series of videos for Google Goggles. Goggles can actually recognize much more than just bar codes and book covers, and these videos really let you take Goggles for a test drive. You can navigate through these videos by clicking on the embedded annotations to fast-forward or rewind.
When looking at “all apps,” you can not only check out any of our latest videos at a glance, but also quickly sort them by mobile platform or by app. We’ve also made it easier for you to share videos by email or through popular social networking sites.
We hope you enjoy trying out our new Google Mobile YouTube channel. We’ll be adding many more new videos, so please visit youtube.com/googlemobile and subscribe. Also, if you’ve already subscribed to our old channel, note that you need to subscribe to the new one separately.
Posted by Heaven Kim, Product Marketing, Google Mobile
Traveling to another country can be an amazing experience. The opportunity to immerse yourself in a different culture can give you a new perspective. However, it can be hard to fully enjoy the experience if you do not understand the local language. For example, ordering food from a menu you can not read can be an adventure. Today we are introducing a new feature of Google Goggles that will prove useful to travelers and monoglots everywhere: Goggles translation.
Here’s how it works:
Point your phone at a word or phrase. Use the region of interest button to draw a box around specific words
Press the shutter button
If Goggles recognizes the text, it will give you the option to translate
Press the translate button to select the source and destination languages.
Google Goggles in action (click images to see large version)
The first Goggles translation prototype was unveiled earlier this year at the Mobile World Congress in Barcelona and could only recognize German text. Today Goggles can read English, French, Italian, German and Spanish and can translate to many more languages. We are hard at work extending our recognition capabilities to other Latin-based languages. Our goal is to eventually read non-Latin languages (such as Chinese, Hindi and Arabic) as well.
Every new release of Google Goggles contains at least one new feature and a large number of improvements to our existing functionality. In addition to translation, Goggles v1.1 features improved barcode recognition, a larger corpus of artwork, recognition of many more products and logos, an improved user interface, and the ability to initiate visual searches using images in your phone’s photo gallery.
Computer vision is a hard problem. While we are excited about Goggles v1.1, we know that there are many images that we cannot yet recognize. The Google Goggles team is working on solving the technical challenges required to make computers see. We hope you are as excited as we are about the possibilities of visual search.
Google Goggles v1.1 is available on devices running Android 1.6 and higher. To download, please scan the QR code below or go to the Android Market app on your phone and search for “Google Goggles”. See our help center for more information.
Alessandro Bissacco and Avi Flamholz, Software Engineers
Yesterday, at the Mobile World Congress in Barcelona, Eric Schmidt and I demonstrated a prototype version of Google Goggles that showcases the potential of integrating Google's machine translation and image recognition technologies. In the video below, you'll see how we use Goggles to take a picture of a German menu and instantly translate the text into English:
You may wonder what's happening in the background. On the simplest level, this prototype connects the phone's camera to an optical character recognition (OCR) engine, recognizes the image as text and then translates that text into English with Google Translate.
Right now this technology only works for German-to-English translations and it's not yet ready for prime time. However, it shows a lot of promise for what the future might hold. Soon your phone will be able to translate signs, posters and other foreign text instantly into your language. Eventually, we're hoping to build a version of Google Goggles that can translate between all of the 52 languages currently supported by Google Translate — bringing even more information to you on the go.
Editor's note: today Google held a launch event at the Computer History Museum in Mountain View, CA. Fresh off the stage, we've invited Vic to highlight the mobile team's announcements, and the unique set of technologies that make them possible. (All [video] links point to event footage that will be viewable later today.)
A New Era of Computing
Mobile devices straddle the intersection of three significant industry trends: computing (or Moore's Law), connectivity, and the cloud. Simply put:
Phones get more powerful and less expensive all the time
They're connected to the Internet more often, from more places; and
They tap into computational power that's available in datacenters around the world
These "Cs" aren't new: we've discussed them in isolation for over 40 years. But today's smartphones -- for the first time -- combine all three into a personal, handheld experience. We've only begun to appreciate the impact of these converged devices, but we're pretty sure about one thing: we've moved past the PC-only era, into a world where search is forever changed.
Just think: with a sensor-rich phone that's connected to the cloud, users can now search by voice (using the microphone), by location (using GPS and the compass), and by sight (using the camera). And we're excited to share Google's early contributions to this new era of computing.
Search by Voice
We first launched search by voice about a year ago, enabling millions of users to speak to Google. And we're constantly reminded that the combination of a powerful device, an Internet connection, and datacenters in the cloud is what makes it work. After all:
We first stream sound files to Google's datacenters in real-time
We then convert utterances into phonemes, into words, into phrases; and
We then compare phrases against Google's billions of daily queries to assign probability scores to all possible transcriptions; and
We do all of this in the time it takes to speak a few words
Over the past 12 months we've introduced the product on manymoredevices, in more languages, with vastly improved accuracy rates. And today we're announcing that search by voice understands Japanese, joining English and Mandarin.
Looking ahead, we dream of combining voice recognition with our language translation infrastructure to provide in-conversation translation [video]-- a UN interpreter for everyone! And we're just getting started.
Search by Location
Your phone's location is usually your location: it's in your pocket, in your purse, or on your nightstand, and as a result it's more personal than any PC before it. This intimacy is what makes location-based services possible, and for its part, Google continues to invest in things like My Location, real-time traffic, and turn-by-turn navigation. Today we're tackling a question that's simple to ask, but surprisingly difficult to answer: "What's around here, anyway?"
Suppose you're early to pickup your child from school, or your drive to dinner was quicker than expected, or you've just checked into a new hotel. Chances are you've got time to kill, but you don't want to spend it entering addresses, sifting through POI categories, or even typing a search. Instead you just want stuff nearby, whatever that might be. Your location is your query, and we hear you loud and clear.
Today we're announcing "What's Nearby" for Google Maps on Android 1.6+ devices, available as an update from Android Market. To use the feature just long press anywhere on the map, and we'll return a list of the 10 closest places, including restaurants, shops and other points of interest. It's a simple answer to a simple question, finally. (And if you visit google.com from your iPhone or Android device in a few weeks, clicking "Near me now" will deliver the same experience [video].)
Of course our future plans include more than just nearby places. In the new year we'll begin showing local product inventory in search results [video]; and Google Suggest will even include location-specific search terms [video]. All thanks to powerful, Internet-enabled mobile devices.
Search by Sight
When you connect your phone's camera to datacenters in the cloud, it becomes an eye to see and search with. It sees the world like you do, but it simultaneously taps the world's info in ways that you can't. And this makes it a perfect answering machine for your visual questions.
Perhaps you're vacationing in a foreign country, and you want to learn more about the monument in your field of view. Maybe you're visiting a modern art museum, and you want to know who painted the work in front of you. Or maybe you want wine tasting notes for the Cabernet sitting on the dinner table. In every example, the query you care about isn't a text string, or a location -- it's whatever you're looking at. And today we're announcing a Labs product for Android 1.6+ devices that lets users search by sight: Google Goggles.
In a nutshell, Goggles lets users search for objects using images rather than words. Simply take a picture with your phone's camera, and if we recognize the item, Goggles returns relevant search results. Right now Goggles identifies landmarks, works of art, and products (among other things), and in all cases its ability to "see further" is rooted in powerful computing, pervasive connectivity, and the cloud:
We first send the user's image to Google's datacenters
We then create signatures of objects in the image using computer vision algorithms
We then compare signatures against all other known items in our image recognition databases; and
We then figure out how many matches exist; and
We then return one or more search results, based on available meta data and ranking signals; and
We do all of this in just a few seconds
Now, with all this talk of algorithms, image corpora and meta data, you may be wondering, "Why is Goggles in Labs?" The answer -- as you might guess -- lies in both the nascence of the technology, and the scope of our ambitions.
Computer vision, like all of Google's extra-sensory efforts, is still in its infancy. Today Goggles recognizes certain images in certain categories, but our goal is to return high quality results for any image. Today you frame and snap a photo to get results, but one day visual search will be as natural as pointing a finger -- like a mouse for the real world. Either way we've got plenty of work to do, so please download Goggles from Android Market and help us get started.
The Beginning of the Beginning
All of today's mobile announcements -- from Japanese Voice Search to a new version of Maps to Google Goggles -- are just early examples of what's possible when you pair sensor-rich devices with resources in the cloud. After all: we've only recently entered this new era, and we'll have more questions than answers for the foreseeable future. But something has changed. Computing has changed. And the possibilities inspire us.
Posted by Vic Gundotra, Vice President of Engineering