Loading and Administration

 
 
By Jason Brooks  |  Posted 2009-06-04 Email Print this article Print
 
 
 
 
 
 
 


Of course, before I could begin exploring my campaign data, I had to load it into the iLuminate engine, a task for which Illuminate offers two different tool options: the iLuminate Importer and iLook&Load.

With each tool, I identified my data source, the sort of delimiter that marked off my records, the data types of each column, the presence or absence of a header row, and the table into which my data should be loaded. I could also apply rules to modify the data as it was loaded into the repository.

The Importer tool worked more quickly, and iLook&Load tool provided more information about my data as it entered the system, such as the totals and averages of integer-typed columns, as well as the maximum and minimum values of individual columns. iLook&Load also accepts a broader range of input types, including ODBC sources, text and XML files, Access databases, and Excel spreadsheets. The stock importer accepts text files and ODBC sources.

I loaded my largest test table-the 15-million-record, CSV-formatted individual contributions table-with each tool. The Importer took about 1.5 hours to complete the load, and iLook&Load required about 5.5 hours to do the job.

Before I was able to load the data into my repository at all, I had to do some cleanup, such as dealing with nested quotation marks within some of my fields. Contributors who took pains to include their nicknames ("Tex") within their name entries gave me fits, for instance.

As with any other data warehouse project, organizations that select iLuminate will need to use an ETL (extraction, transformation and loading) tool to first clean up their data. For the purposes of my tests, I focused primarily on getting all of my records to load, but actual deployments will demand more attention to data cleansing.

iLuminate runs as a pair of Windows services-one for the core engine and another for its ODBC support. I could start, stop, create and switch among database files using a service monitor tool that would minimize to my system tray. The monitor also provided performance details, as well as information about open queries, active connections and storage characteristics.

Also from the service monitor, I was able to enable remote access to the engine for other iCorrelate clients or ODBC consumers on my network.

Executive Editor Jason Brooks can be reached at jbrooks@eweek.com.



 
 
 
 
As Editor in Chief of eWEEK Labs, Jason Brooks manages the Labs team and is responsible for eWEEK's print edition. Brooks joined eWEEK in 1999, and has covered wireless networking, office productivity suites, mobile devices, Windows, virtualization, and desktops and notebooks. JasonÔÇÖs coverage is currently focused on Linux and Unix operating systems, open-source software and licensing, cloud computing and Software as a Service. Follow Jason on Twitter at jasonbrooks, or reach him by email at jbrooks@eweek.com.
 
 
 
 
 
 
 

Submit a Comment

Loading Comments...
 
Manage your Newsletters: Login   Register My Newsletters























 
 
 
 
 
 
 
 
 
 
 
Thanks for your registration, follow us on our social networks to keep up-to-date
Rocket Fuel