Close
  • Latest News
  • Artificial Intelligence
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
Read Down
Sign in
Close
Welcome!Log into your account
Forgot your password?
Read Down
Password recovery
Recover your password
Close
Search
Logo
Logo
  • Latest News
  • Artificial Intelligence
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
More
    Home Applications
    • Applications
    • Database

    Data Quality Is First Step Toward Reliable Data Analysis

    By
    Eric Lundquist
    -
    July 22, 2013
    Share
    Facebook
    Twitter
    Linkedin

      You are embarrassed to discover your company is sending mail to a former customer now deceased. You have to hire a swivel-chair data input operator who spends the day re-entering data between incompatible systems by swiveling from screen to screen.

      Government organizations are supposed to share data on suspicious individuals, but a name falls through the cracks and a bombing occurs. Those examples, ranging from embarrassing to fatal, are all examples of data that should have been updated, integrated and shared but wasn’t.

      Individuals outside the data business are perplexed that—in this era of big data, cloud computing and systems that seem to know way too much about you—those “dirty” data problems still exist. Individuals on the inside the data processing industry are surprised the problems are not more prevalent.

      At this year’s MIT chief data officer and information quality symposium held July 17 to 19 at the Massachusetts Institute of Technology’s Cambridge campus, both the success stories and the epic failures were evident. This is the conference’s seventh year, and while past years had a focus on technology development to clean, normalize and make sure the data on which your company depends is accurate, the focus this year was much more on the business strategies that must be installed to make data dependable.

      “The hardest part is not the technology but the people and social issues,” said Deborah Nightingale, the director of the MIT Socio-Technical Systems Research Center, in her remarks that opened the conference. Nightingale has a focus on enterprise technology systems, and she was referring to enterprise systems generally as well as the data that is used to power those systems.

      In an era when big data has moved from little-mentioned to current industry buzzword, the accuracy of the data underlying those systems is often overlooked. The term “big data” was an especially favored term by the approximately 180 conference attendees, but was much on the minds of speakers who noted the industry is in a transition from handling largely structured data to a mix of structured and unstructured data.

      The complexity of working with myriad data types and myriad, often incompatible, systems was underscored by Dat Tran, the deputy assistant secretary for data governance and analysis at the U.S. Department of Veterans Affairs.

      “The VA does not have an integrated data environment; we have myriad systems and databases, and enterprise data standards do not exist. There is no 360-degree view of the customer,” Tran said in a forthright discussion of the obstacles facing a high-profile agency dealing with 11 petabytes of data and 6.3 million patients.

      Tran echoed Nightingale’s remarks in noting that cleaning up the data traffic jam at the VA is more about getting all the business and technology groups agreed on a data quality process and time schedule rather than searching for some quick technology fix.

      “Data quality is a business problem; however, effective data governance requires business users and IT to work together,” said Tran.

      The idea that there is a quick technology fix to data quality was a myth knocked down by numerous speakers and attendees at the conference. The difficult work of mapping out business processes, making sure that those who initially enter the data feel empowered and responsible for data accuracy and creating a data quality infrastructure that also assures privacy and security, existed before the advent of the “big data” buzzword. The existence of large, outside pools of unstructured data inside the enterprise only makes the data quality issue more critical.

      Some technological help may be on the near horizon. Peter Kaomea, CIO for the Sullivan and Cromwell law firm, mentioned internal crowd-sourced data cleansing as an emerging model once security and privacy considerations are in place.

      Mark Temple-Raston, senior vice president of data management at Citigroup said advances in natural language processing applied to business communications could provide business data with mathematical underpinnings currently unavailable.

      The role of the chief data officer in improving data quality was obviously a big topic at the conference, but it was clear that wherever the position resided—whether in business groups or under the CIO auspices—an understanding of the business processes, the ability to communicate the value of data quality and then implementing technology solutions was the prime process for improved data quality overall.

      Eric Lundquist is a technology analyst at Ziff Brothers Investments, a private investment firm. Lundquist, who was editor in chief at eWEEK (previously PC WEEK) from 1996-2008 authored this article for eWEEK to share his thoughts on technology, products and services. No investment advice is offered in this article. All duties are disclaimed. Lundquist works separately for a private investment firm, which may at any time invest in companies whose products are discussed in this article and no disclosure of securities transactions will be made.

      Eric Lundquist
      Since 1996, Eric Lundquist has been Editor in Chief of eWEEK, which includes domestic, international and online editions. As eWEEK's EIC, Lundquist oversees a staff of nearly 40 editors, reporters and Labs analysts covering product, services and companies in the high-technology community. He is a frequent speaker at industry gatherings and user events and sits on numerous advisory boards. Eric writes the popular weekly column, 'Up Front,' and he is a confidant of eWEEK's Spencer F. Katt gossip columnist.
      Get the Free Newsletter!
      Subscribe to Daily Tech Insider for top news, trends & analysis
      This email address is invalid.
      Get the Free Newsletter!
      Subscribe to Daily Tech Insider for top news, trends & analysis
      This email address is invalid.

      MOST POPULAR ARTICLES

      Latest News

      Zeus Kerravala on Networking: Multicloud, 5G, and...

      James Maguire - December 16, 2022 0
      I spoke with Zeus Kerravala, industry analyst at ZK Research, about the rapid changes in enterprise networking, as tech advances and digital transformation prompt...
      Read more
      Applications

      Datadog President Amit Agarwal on Trends in...

      James Maguire - November 11, 2022 0
      I spoke with Amit Agarwal, President of Datadog, about infrastructure observability, from current trends to key challenges to the future of this rapidly growing...
      Read more
      IT Management

      Intuit’s Nhung Ho on AI for the...

      James Maguire - May 13, 2022 0
      I spoke with Nhung Ho, Vice President of AI at Intuit, about adoption of AI in the small and medium-sized business market, and how...
      Read more
      Applications

      Kyndryl’s Nicolas Sekkaki on Handling AI and...

      James Maguire - November 9, 2022 0
      I spoke with Nicolas Sekkaki, Group Practice Leader for Applications, Data and AI at Kyndryl, about how companies can boost both their AI and...
      Read more
      Cloud

      IGEL CEO Jed Ayres on Edge and...

      James Maguire - June 14, 2022 0
      I spoke with Jed Ayres, CEO of IGEL, about the endpoint sector, and an open source OS for the cloud; we also spoke about...
      Read more
      Logo

      eWeek has the latest technology news and analysis, buying guides, and product reviews for IT professionals and technology buyers. The site’s focus is on innovative solutions and covering in-depth technical content. eWeek stays on the cutting edge of technology news and IT trends through interviews and expert analysis. Gain insight from top innovators and thought leaders in the fields of IT, business, enterprise software, startups, and more.

      Facebook
      Linkedin
      RSS
      Twitter
      Youtube

      Advertisers

      Advertise with TechnologyAdvice on eWeek and our other IT-focused platforms.

      Advertise with Us

      Menu

      • About eWeek
      • Subscribe to our Newsletter
      • Latest News

      Our Brands

      • Privacy Policy
      • Terms
      • About
      • Contact
      • Advertise
      • Sitemap
      • California – Do Not Sell My Information

      Property of TechnologyAdvice.
      © 2022 TechnologyAdvice. All Rights Reserved

      Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.

      ×