Artificial Intelligence Gives Photos New Life Online

By Mike Elgan  |  Posted 2015-11-12 Print this article Print
Photo AI

Facebook Photo Magic expands the pool of photo data to collect beyond just Facebook, the social network, to Messenger, the messaging app, which improves the quantity of data. And Photo Magic encourages confirmation or rejection of the matches, which improves the quality of data.

It's clear that Facebook's ultimate goal is to be able to recognize anyone in any situation, even in bad lighting where faces aren't visible. From there, future Facebook AI will no doubt scan and analyze the environment for marketable clues—for example, if certain people often appear in photos at baseball games, advertisers could use that information to target baseball fans even if the words in that person's posts don't reveal that special interest.

They also intend, no doubt, to further build social graphs by seeing who shows up in pictures together.

Microsoft Project Oxford

Microsoft announced updates this week to its Project Oxford, which is a collection of tools that enable developers to make use of Microsoft's artificial intelligence systems via the company's cloud platform, Azure.

The tools enable the application of AI to various things, including spoken language, video and other media types. But the most amazing and powerful of these features is that Project Oxford now enables developers to detect human emotion in pictures of people through the Project Oxford Face API.

So a photo of, say, five people processed through Project Oxford recognizes the faces in the photo and identifies the emotional expression of every single person in the picture—emotions like happiness, anger or disgust.

This capability brings the quality of human-like "understanding" of photos to a new level. When people look at a picture of other people, the most important attribute that viewers note is the emotional state of the person or group.

Pinterest Visual Search

Pinterest unveiled this week a brilliant new photo search feature that helps users find more information and even buy the products they see in pinned photos.

To use it, you select (by drawing a box around) any object in a photograph that's been posted on Pinterest. The search tool then finds similar objects with similar patterns and colors and ideally one that links to a buyable pin, which is a post where you can buy the product.

The feature is based on deep learning AI from Berkeley Vision and Learning Center.

This application of photo AI is the beginning of what you might call a worldwide web of photos, where each object in every picture is linked to identical, similar or related objects.


An image recognition and visual search company called CamFind this year launched a public API called CloudSight.

The API enables developers to leverage CamFind's artificial intelligence to analyze the content of photographs. And many such scans are highly specific, identifying the make and model of cars, for example, or dog breeds and specific types of foods. Once the objects in a photo are analyzed, a developer can use that information to harvest text-based information from the Internet.


Deepomatic developed a software-as-a-service-based smart search engine that the company says can identify all kinds of data from a photograph. Deepomatic specializes in fashion. It not only matches colors, patterns and other data, but also identifies the objects in a photo and matches them against a comprehensive database of fashion products.

Deepomatic's Website claims that its technology imitates how the human brain takes visual information and uses that to understand concepts.

The Big Picture

When I consider this astonishing new world of ubiquitous, extensible, available artificial intelligence that can "understand" what's happening in photographs, I'm struck by the incredible variety of what's possible.

And this is only the beginning. Because most of this technology is being made available in most cases through an API, an open-source process or as a service, we're on the brink of a world where photo AI is as common a feature as Web search. In order to truly mimic human intelligence, computers must get visual. And now they are.



Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel