Microsoft Plans for Voice, Gesture Go Beyond Xbox Kinect

Microsoft's plans for voice, touch and gesture-activated controls extend beyond Xbox Kinect and Windows Phone 7.

Xbox Kinect and Windows Phone 7 are just the beginning of Microsoft's push into natural user interfaces, according to discussions with company executives. Plans include baking touch, gesture and voice-activated technology into products ranging from video games to vehicles.

Ilya Bukshteyn, senior director of marketing for Microsoft Tellme, told eWEEK during a Dec. 1 meeting in New York City that speech represented one of Microsoft's "key initiatives," although it remains "a little more nascent than the cloud."

Microsoft's new Kinect hands-free controller leverages a 3D camera to transcribe the player's body movements onto their on-screen avatar, and it also includes a substantial voice-activated element: Whether shouting commands to a character in an Xbox game, or telling a movie to pause or fast-forward, the spoken word represents a sizable element of the user interface. Meanwhile, some 20 percent of Bing Mobile searches are conducted via voice.

"Speech is this common key ingredient in NUI [natural user interface]," Bukshteyn said. Microsoft's new Windows Phone 7 also heavily leverages speech in its search and features.

He predicted a timeframe of "two to three years" before voice becomes a more ubiquitous factor in both the consumer and enterprise space. For the latter, he suggested that voice, touch and gesture would operate in unison through devices ranging from tablets to whiteboard-style screens. That being said, he also declined to comment on any specific initiatives under development.

"Cloud is an enabler of NUI," Bukshteyn added, suggesting that data from Bing and other venues was helping refine those technologies.

Microsoft first announced its acquisition of Tellme Networks in 2007, in order to buttress its voice-technology offerings for devices and services. "We see voice and speech recognition as ways to improve interaction with productivity software," Jeff Raikes, then president of the Microsoft Business Division, wrote in a statement at the time. "People are on the go. They want the ability to use voice as a way to interface-whether it is to access information or connect with callers."

Previous to the acquisition, Tellme Networks was a privately held company with 320 employees and customers such as UPS, American Airlines and FedEx. Microsoft initially intended its new acquisition's assets to buttress its Unified Communications products, in addition to phones and other devices.

Microsoft recently acquired Canesta, a maker of 3D-image sensor chips and camera modules that can be embedded in a variety of consumer products, including laptops and vehicle dashboards. The recent popularity of Kinect-the controller reportedly sold 2.5 million units within its first 25 days of release-will likely compel Microsoft to leverage that 3D technology toward other devices and applications.

Not content to wait for Microsoft and its manufacturing partners to innovate further on gesture technology, a number of tech pros and tinkerers have modified Kinect's 3D camera for non-gaming functions, including painting 3D images in midair and controlling robots.

After making some initial sounds of disapproval, Microsoft moved to embrace that tinkering. "The first thing to talk about is that Kinect was not actually hacked," Alex Kipman, Microsoft's director of incubation for Xbox, insisted during a Nov. 19 interview with NPR. "Someone wrote an open-source driver for PCs that essentially opens the USB connection, which we didn't protect by design, and reads the inputs from the sensor."

In the meantime, Microsoft seemingly plans for natural user interfaces to play a major part in its future road map.