SAPIR, which stands for Search in Audio-Visual Content Using Peer-to-peer Information Retrieval, churns through pixels in 100 million images and videos to help users find multimedia content they are looking for. IBM and the European Union are working on the project as an alternative to the way traditional search engines such as Google, Yahoo and Microsoft Bing index images and videos. SAPIR also leverages latitude and longitude coordinates for location-based search.
IBM researchers have created a search engine that takes a
different route from Google, Microsoft Bing and other search services to help
users find untagged video, pictures and music content.
which stands for Search in Audio-Visual Content Using Peer-to-peer Information
Retrieval and was funded by the European Union, churns through pixels in 100
million images and videos to help users find multimedia content they are
looking for. IBM and EU launched a SAPIR demo Website here
Sept. 10 for the
public to test.
Traditional search engines Google, Bing and Yahoo rely on
text metadata associated with each image to help users find content. If there
is no metadata associated with the content, those search engines can't help users find
relevant results, Yosi Mass, a scientist at IBM Research-Haifa and project
leader for SAPIR, told eWEEK Sept. 10.
SAPIR indexes the content of each image and clip using descriptors
such as text, color, layout, shapes, or sounds, to help users find comparable
images. For example, SAPIR scans a digitized photograph or the bit streams in an
MP3 sound file, even if they haven't been tagged or indexed with descriptive
information; this is because SAPIR automatically indexes and ranks multimedia
content users upload to the Web for easy retrieval.
In a demo, Mass showed how a search on the keyword
"dolphin" returned dolphin photos of similar colors and shot angles
that users had uploaded to Flickr. After clicking the "similar" link
on top of one of the photos, eWEEK saw photos that resembled the dolphin photo
in color and shape but did not necessarily include dolphins.
A box labeled "use the image in the search"
appeared checked above the SAPIR search, so that when a user types in
and clicks the search button, they will see more images of dolphins.
This search blends dolphin images with similar images that do not
include dolphins. The demo was an exercise in helping users conduct a
combination text and
image search, something that is not possible with current search
Mass also said SAPIR enables users to introduce latitude and
longitude coordinates into searches. Do a search for a topic on SAPIR,
"images" and you'll see boxes for Lat and Long to the left of a button
Maps under the site's search box.
Enter your geographic area of choice and
search for photos on any topic. So, in addition to text and image
search, users can further narrow down their topics with location
services, which also is crucial for travelers accessing SAPIR via
To wit, this YouTube demo
shows how a tourist takes a picture of a statue in Madrid, and
searches for comparable pictures in SAPIR. The
tourist then did a combined search. By adding "Madrid" to the query,
he saw images of the statue he just took a picture of, as well as related pictures
of the surrounding plaza he was in.
Eventually, this service could be put to use by
cities promoting tourism. "Cities that have a large image collection of
monuments in their city can give a very nice service to their tourists,"
pairing images with historical text information about the attractions users can
access from their Web-enabled phones, Mass explained.
IBM has no concrete plans yet to
commercialize SAPIR, but there is a clear value in corralling multimedia
content. If IBM can harness some of this unstructured data with
the SAPIR technology, it might be most useful for the company's enterprise
search portfolio, as well as for consumer service providers looking to provide their customers with better image and video