Although commercial data cleansing and standardization software tools have been around for years, until fairly recently they werent suitable for Web applications.
Thats because tools from such vendors as Trillium Software, Vality Technology Inc., Firstlogic Inc. and Innovative Systems Inc. were originally designed for traditional back-office systems and batch processes.
On the Web, however, where consumers and business partners can directly enter, change and access data at any time, batch cleansing wont do. Youve got to be able to assess, validate and fix data on the fly in real time, experts say. If you cant, you risk contaminating core business systems with bad data or, worse, alienating customers or partners.
“In the past, data quality was only seen by a small number of internal employees, so companies were not nearly as exposed,” said Steve Brown, vice president of product strategies at Vality, in Boston. “Now that data is put out there to external clients without anyone there to mediate, buffer or translate. It means it better be good.”
Now, however, upgrades and new product offerings from Vality and other data quality tool vendors are addressing many of those shortcomings—and not a minute too soon. Vality, for example, has beefed up technology used in its traditional Integrity data standardization product to ensure a higher degree of accuracy for real-time data, Brown said. In an evolutionary series of enhancements beginning three years ago, for example, Vality published APIs allowing Web applications to interface to Integrity and optimized the products database access, allowing it to respond quickly to small transactions taking place over the Web. Vality also enhanced Integrity with XML (Extensible Markup Language), Java and other Web-based data sharing capabilities.
The company has also released two products designed to address the unique data quality problems of the Web. Catalist, unveiled in February, categorizes and structures free-form text. That assists in the development of online product catalogs. Beacon, formerly e-Search, is Valitys new search and matching tool optimized for helping online customers or trading partners find what theyre looking for in operational systems.
Other tool vendors are moving in a similar direction. Trillium Software, a Billerica, Mass., division of Harte-Hanks Inc., has added real-time capabilities as well as support for XML, as has Evoke Software Corp., of San Francisco. Firstlogic, of La Crosse, Wis., has taken a step toward real-time capabilities via an alliance with Siebel Systems Inc., of San Mateo, Calif., which is integrating Firstlogics data quality tools into its e-customer relationship management environment.