Highspot Brings Machine Learning to Enterprise Search

By Darryl K. Taft  |  Posted 2014-03-19

Highspot Brings Machine Learning to Enterprise Search

A team of former Microsoft execs and engineers have launched Highspot, a startup with a new cloud knowledge management service that helps organizations capture, share, and cultivate their most valuable working knowledge via machine learning.

Highspot’s chief scientist, Paul Viola, ran the core data science team for Microsoft’s Bing search engine and helped it to pull even with Google in many measurements of search relevance during his tenure. Viola described himself to eWEEK as a "machine learning geek."

In a blog post about the company’s launch, Highspot's CEO Robert Wahbe, who was corporate vice president of product management for the Server and Tools Division at Microsoft, said the inspiration for Highspot came from its founders' collective frustration that their previous organizations had the potential to be much more effective if they "only knew what they knew."

"We spent a ton of time and money producing content, and what came clear to me was that people are not able to find the content they are looking for when they need it," Wahbe told eWEEK. Wahbe cited a Forrester study that says there is a knowledge gap where the failure rate for not finding the information users are looking for is 56 percent, while the process of looking for the information wastes up to 12 percent of users' time.

According to Wahbe, "The information existed to make the organizations smarter, but it simply wasn't available to the people who needed it, when they needed it. Some of this vital working knowledge was formal, such as documents and presentations about a new product launch, while other was more informal such as the fruits of hard-won experience. But regardless of whether formal or informal, most organizations really struggle to capture, share, and cultivate the collective working knowledge of the organization."

He said this is particularly true for frontier functions like engineering, marketing, and sales that operate at the edge of institutional knowledge. "They have an insatiable demand for the latest and greatest information to win the next deal, fend off competitors, captivate customers and inspire innovation," he said. "As often as not, the information they need to win the day exists, it just isn't available when it matters."

Wahbe said Highspot is following the lead of existing Internet solutions like Google, Amazon and Pinterest. "They all use similar pioneer techniques to connect people with information–they use machine learning," he said. "Compare those Internet solutions to existing enterprise search solutions for business. They don't use machine learning, they use tags. However, tags get less and less correct and less and less relevant over time."

The Highspot approach to enterprise search is based on four pillars: to leverage approachable consumer user experiences, to build a comprehensive knowledge graph, to increase relevance through machine learning, and to deliver it all as a modern cloud service.

"To effectively 'know what you know,' Highspot must cast the broadest possible net for information, and gracefully incorporate content wherever it is created, whatever its type, and wherever it is consumed," Wahbe said in his post. "We are committed to integrate with information wherever it resides, including widely used platforms such as Office 365, Google Apps, Box, Dropbox, and Salesforce as well as information types as diverse as documents, presentations, spreadsheets, images, videos, audio, news feeds and news alerts."

Highspot leverages familiar consumer Web experiences and it increases relevance through advanced machine learning. The machine learning platform delivers highly relevant results that get even more relevant over time, Wahbe said.

With machine learning, as people use Highspot the knowledge graph gets smarter. Highspot enables users to organize information in "Spots," which are collections of related content. "Spots are the main ways we organize knowledge," Wahbe said.

The Highspot knowledge graph represents all people and all items in an organization and all the relationships between them. The knowledge graph factors in observed facts such as geography, views, likes and follows, as well as learned facts such as influence, similarity, trending and interest, Viola said.

"I try to put things in front of you that will be of value to you," he noted.

Highspot Brings Machine Learning to Enterprise Search

Highspot automatically brings the latest and most relevant information to users' attention. It enables users to browse in context for the information they need. It also features information genomics to enable users to follow information as it evolves and track documents as they mutate.

"With the rocket growth of our business we were creating content faster than anyone could keep up," said Forest Key, CEO of buuteeq, in a statement. "And with our team more than doubling each year on boarding new people was a huge challenge. Highspot lets us capture our important information and expertise in a way that makes it easy for everyone to find it and leverage it."

Parallels, a provider of hosting and cloud services enablement and cross-platform solutions, also is an early Highspot customer. Parallels is a global business with customers in over 130 countries. "Keeping our leadership and our sales, support, and engineering teams equipped with the latest information was a constant challenge," said John Zanni, chief marketing officer at Parallels, in a statement. "Highspot easily gets the most relevant information into people's hands when they need it and helps our marketing team focus on the content that matters."

Wahbe said enterprise search is estimated to be an $8 billion market. "We believe we are the first to tackle this in the enterprise," he said. The company's focus on frontier foundations such as marketing, sales and engineering represent a sweet spot for the service, he said. "Sales and marketing produce about half of the content coming out of an enterprise. Engineering produces a large amount of content as well, in the form of things like process guidelines, coding guidelines, documentation and more. We feel like this is a very horizontal solution."

Moreover, Wahbe said Highspot has amassed a "world-class" team. In addition to Wahbe, who is a pedigreed entrepreneur and engineer, the rest of the team also has chops. Viola was Distinguished Engineer and general manager of the Relevance and Revenue team for Microsoft Bing. His team controlled and improved all the key algorithms that interpret user queries and create the search results page for Bing. And he holds 35 patents in the areas of advanced machine learning, web search, data mining, and image processing.

Oliver Sharp, the company’s vice president of product, served in a number of roles at Microsoft, including general manager of strategy for the Server and Tools Division, general manager of the Connected Server Team, and a staff role for Bill Gates. Scott Gellock, Highspot’s vice president of engineering, was general manager of engineering for the Identity and Networking Services of the Windows Azure platform. And David Wortendyke, the company’s chief architect, was partner architect for Windows Azure, where he drove the design and architecture of security and messaging services.

Highspot runs on the Amazon Web Services cloud. "We run the service on AWS and it is based on a proven, open-source stack," Wahbe said. Machine learning is very compute intensive, he added.

"Open source is both a business and strategic decision," Wahbe said. "There is so much innovation going on in the open source world."

The service runs on Linux and uses open source components such as MongoDB, Apache Kafka, Solr and Lucene with much of the system written in Clojure, which Viola says is "a good language for data science."

"The notion of bringing machine learning to knowledge management is really quite pioneering; it's frontier," Wahbe said. "Machine learning is going to revolutionize business apps generally."

Hopefully, coming out of the blocks early with a machine learning solution for enterprise search can provide Highspot with an adequate head start.

When Wahbe was at Microsoft, one of the efforts he led was a software modeling project known as "Oslo." Ironically, Wahbe's new role appears to counter a new Microsoft "Oslo" initiative that is working to bring machine learning to the company’s Office platform.

Rocket Fuel