5 Guidelines for Ensuring Proper Data Preparation

 
 
By Darryl K. Taft  |  Posted 2016-09-12
 
 
 
 
 
 
 
 
 
  • Previous
    1 - 5 Guidelines for Ensuring Proper Data Preparation
    Next

    5 Guidelines for Ensuring Proper Data Preparation

    Organizations looking to analyze big data for deep insights need data that is clean, organized, detailed and easily understood. What's more, data prep is crucial throughout the workflow.
  • Previous
    2 - Data Preparation Needs to Be More Than Just Cleansing
    Next

    Data Preparation Needs to Be More Than Just Cleansing

    Cleaning data is a small part of the battle, and even a clean data set may be not ready for analysis. It takes strategy and experience to start with data in its original forms and reshape or process it to work well for analysis. Typically, the data that leads to insights requires some interpretation and transformation of the original data. Big data analytics involves a different process, data discovery, in which data is pulled from multiple sources, then sifting through it until useable data can be found.
  • Previous
    3 - Preparation Must Be Integral to the Entire Analytic Workflow
    Next

    Preparation Must Be Integral to the Entire Analytic Workflow

    Data prep isn’t just a one-and-done, up-front process. The word “preparation” may imply that the data prep process takes place as an early step. It’s important to understand the role of data preparation in the larger analytic workflow. In big data discovery, data preparation may take place at any point in an iterative, ongoing data discovery process. After analysis, the data discovery may reveal flaws in the data or the need to add new data, revealing new requirements for how the data should be shaped, interpreted, enhanced or cleansed.
  • Previous
    4 - Ensure Flexible Data Organization
    Next

    Ensure Flexible Data Organization

    In practice, data preparation involves several subtasks. The most common ones are data cleansing and data transformation. Yet there are other subtasks, such as looking inside the data for other data of interest. To find deeper diagnostic insights, big data analytics needs to go beyond the standard slicing and dicing of information to find hidden trends. This requires organizing the data during the data preparation phase in different ways than the typical dimensions and metrics. Doing so makes it easy to organize data for more advanced analytics and helps drive general analytical use cases, such as clickstream and time-series analysis, as well as specific ones, such as fraud, preventive maintenance, buying patterns and more.
  • Previous
    5 - Create Rich Data Sets
    Next

    Create Rich Data Sets

    Enriching data by joining it to other data sets, whether from internal corporate systems or external, publicly available data sets, is another way to discover the data hidden in the apparent data. This is common in customer analytics. While transactional history of who bought what and when is a great baseline for analysis, today’s organizations need deeper insights from their transactional data. For example, you might want to ask, what was the impact of weather, seasonality, customer gender, income or education level on sales? Business stakeholders can only find these insights through data enrichment or embellishing.
  • Previous
    6 - Deploy a Highly Collaborative Environment With Full Governance
    Next

    Deploy a Highly Collaborative Environment With Full Governance

    An analysis may need a curated, well-prepared data set. Related analyses may require new datasets that further shape the data. This leads to analysts collaborating, sharing and building on each other’s prepared data sets, helping them save time and increasing their productivity. As such, organizations need a highly collaborative environment for effective data preparation pipelines and deeper analytic insights. Sharing both analytic data sets and content enables analysts to cast wide question nets while ensuring data and results are in line with corporate accuracy standards and compliance requirements.
 

If businesses want to analyze big data for insights that will move the needle, they need data that is clean, organized, detailed and easily understood. The diversity and amount of data available today is extremely large, and organizations must deal with data sets that are far from ideal. Armed with the right data preparation strategy, raw data can be fixed and shaped. There is a laundry list of use cases for data preparation—potentially as large as the candidate data sources multiplied by the ways in which the data will be analyzed. For example, an organization may want to look at purchasing patterns over time, break customers down into demographic groupings and correlate shopping activity at brick-and-mortar locations with weather information. Data prep is not just about de-duplicating and removing corrupted or dirty data; it's about reshaping that data to reveal insights. It’s more than just an initial step companies must take before analyzing data. Data prep may take place at any point in a data discovery process.  Using input from Stefan Groschupf, CEO at big data analytics provider Datameer, eWEEK lists key considerations for taking the data you have and changing it into the data you need.

 
 
 
 
 
 
 
 
 
 
 

Submit a Comment

Loading Comments...
 
Manage your Newsletters: Login   Register My Newsletters























 
 
 
 
 
 
 
 
 
Rocket Fuel