Artificial Intelligence Gives Photos New Life Online

NEWS ANALYSIS: Artificial intelligence used to be all about words. But suddenly this week it's clear that amazing things are possible when machines can understand pictures.

Photo AI 2

Research projects dating back to the 1950s have attempted to apply artificial intelligence to create machines that think—or at least behave as if they can.

The quest to build computers that think like humans has necessarily focused on words. The famous Turing Test, for example, is designed to prove a machine's ability to act intelligently by responding to written questions like a person would.

In reality, the human mind is optimized for visual processing. So much of what makes us both intelligent and human is our ability to recognize patterns, objects and context from what we see.

Until very recently, computers couldn't do much with pictures. Photos, for example, were nothing more than inert, useless files. Unless they were laboriously tagged or otherwise given manually entered notations of some kind, it was impossible for a machine to reveal anything about the content of a picture.

But that's changing rapidly. Just this week, a wide range of major announcements reveal a bold new world of applications that show what kind of magic can happen when you apply artificial intelligence to the job of understanding photographs.

Suddenly, artificial intelligence engines can do all kinds of incredible things with photos.

Here's what's happening.

Google TensorFlow

When Google rolled out its Google Photos in May, the press focused on the power of Google's combination of AI with photo search. Google demonstrated (and users quickly confirmed) that searching for specific people shows photos all the way back to infancy. Dog breeds could be found by searching for the breed name. Types of food could be combined with names, such as "pizza with max" to locate specific pictures.

At the time of the Google Photos launch, the media broadly failed to appreciate how long Google had been working at this. Some of the search features had been available on Google+ for more than a year.

What's new this week is that Google is open-sourcing the main part of its AI capability in the form of a platform Google calls TensorFlow.

Although Google's TensorFlow isn't the first open-source AI platform, Google's is the one most closely associated with Google's impressive photo search A.

The open-sourcing of TensorFlow means other companies, including Startups, can creatively combine AI with photos in ways Google may not have applied. Google isn't sharing key aspects of its many AI technologies, including the ability to run across a large number of servers. Nor is the company sharing the troves of user data that help make it so powerful. But they are enabling unprecedented AI power previously unavailable to small startups.

Expect mind-blowing new applications based on TensorFlow to reach the market next year.

Facebook Photo Magic

Facebook started testing a new feature this week for its Messenger mobile app called Facebook Photo Magic. The opt-in app scans new pictures in your smartphone's camera roll and processes them through the company's facial recognition technology. Photo Magic identifies people in the photos who are also Facebook friends, and suggests that you share the photos with them.

The feature no doubt does double duty for Facebook. First, it encourages more sharing on Messenger. Second, it improves recognition. By merely using this feature, which is presented as a convenience, the user is actually confirming or rejecting Facebook's AI matching of face to name under arbitrary lighting conditions, angles and other variables. The more pictures Facebook has of each person available to its AI, the better the recognition.

Surprisingly, Facebook's so-called "face recognition" can recognize your face even if your face is hidden. The system also looks at hairstyle, posture, clothing and the shape of your body. (Note that it's not clear that Facebook has already implemented this advanced system, but it is clear it's collecting data for it from user photos.)