eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Jimmy Wales is taking on search giants such as Google with his open-source/community-based Search Wikia project
In the last few years, many technology markets have felt the effect of open-source and community-based systems. From collaboration to Web services to social networking, the open-source and community-based systems have changed the expectations of users and the business models of many companies.
However, one area that hasn’t been affected too much by these forces is search. Google still rules the roost along with pretty much the same case of competitors that has existed for years.
To a large degree, this is because of the complexity of the algorithms needed to create quality search results and the massive data center requirements necessary to crawl the entire web on a regular basis.
However, Jimmy Wales, the founder of Wikipedia, thinks he may have a solution. About a year ago he started the Search Wikia project with the goal of building an open-source and community-based option to the current search giants. To quote from the Search Wikia website, the four principles of the project are: “Transparency – Openness in how the systems and algorithms operate, both in the form of open -ource licenses and open content + APIs. Collaboration – Everyone is able to contribute in some way (as individuals or entire organizations), strong social and community focus. Quality – Significantly improve the relevancy and accuracy of search results and the searching experience. And privacy – “Pursuing the Holy Grail of Privacy Protection”.
These are certainly worthwhile goals. Many businesses, individuals and governments question the secrecy behind major search company algorithms and the effect they can have on the majority’s view of the Internet. And the last point about privacy is a major sticking point for many people on the Internet, who have seen multiple scares about the amount of information that search companies have on what people do and what they search for.
Still, the Search Wikia project is in a very, very early stage. There currently isn’t much to see outside of developer discussions and idea exchanges.
However, last week Search Wikia did make one major move to address a core requirement for building an open search platform. Wale’s Wikia company acquired the Grub client from LookSmart. Grub works on much the same principles as tools such as the long running Seti@Home project. Users across the globe download the client which runs quietly in the background. Whenever the system is in downtime, the Grub client works in a distributed way to crawl and index Web sites across the world.
With this solution in the fold, Search Wikia has a potential answer to the massive computing resource problem of handling search. The Grub client computers around the world essentially become the datacenter for Search Wikia.
But like the main project, this still has a ways to go.
When I went to the Grub website early this week to download the clients, I wasn’t able to get either the Windows or the Linux client to work properly. And the site does say that neither is really ready for use.
Also, while Search Wikia has said that Grub is now open source, users downloading the clients will still for now need to accept a LookSmart click-through agreement that sounds pretty far from open source.
Still, this is a project well worth watching. Wikipedia has already changed the face of search results (ever notice that one of the top search results for pretty much any topic is always Wikipedia?)
And by building an open- and community-based option to the search giants, Wales and Search Wikia just might change everyone’s expectations about search.