Data warehouses are no longer simply the attics of the business computing world. The use of business intelligence applications is growing exponentially, making such repositories an essential part of daily business life.
Compounding the need is the democratization of BI applications. Employees at all levels-from sales to production to human resources-are using BI applications on a daily basis. Add in the new forms of data that are flooding in, from radio-frequency identification (RFID) readers, Web services applications, mobile devices, cloud-based sources and more, along with the fact that data latency requirements have dropped (below one hour in 60 percent of cases and under one minute in another 35 percent) and it’s clear just how important near real-time data warehouse updates have become.
It’s estimated that up to 90 percent of all Global 2000 companies have now established a link between their data warehouse and at least one mission-critical application used to produce revenues or control costs. That’s a jump from just 25 percent in 2007. Without current and accurate information, it’s impossible to produce actionable insights at the pace of modern business.
Many tools exist to automate the processes used to update data stores but the problem is that most have significant limitations. For example, a DBMS has job-scheduling capabilities (as do even major operating systems including Unix and Windows). But DBMS options typically focus on data maintenance only. And, while operating system tools are convenient, their workflows are limited to tasks occurring within a particular server or operating environment.
Advantages of Job Scheduling Software
Advantages of job scheduling software
Dedicated job scheduling applications have the advantage of accommodating virtually all the quirks and complexities of heterogeneous IT environments and their processes. Tasks can be triggered by the occurrence of an event (say, a sales threshold) rather than simply a date or time of day. One-time or unpredictable situations are not a problem. A range of powerful features-from alerts and failover protection to reports and audit trails-also make it easier for busy IT professionals to manage important processing tasks.
Perhaps one of the greatest advantages of advanced job scheduling applications, in a world where the pressure to increase IT efficiency is at an all-time high, is how they can stretch and optimize resources. Good schedulers can automatically distribute job flows among available computing assets, whether local or across the planet. They can trigger jobs only when suitable assets are available, on the fly if necessary. They can even be used to manage power consumption by automatically placing assets in “suspend” mode when not needed, lowering operating costs and contributing to a greener IT environment.
Robust Job Libraries and Support
Robust job libraries and support
To simplify deployment of data warehouse computing tasks, look for a job scheduling solution with a robust job library that allows work loads to be easily assembled. A good library will reduce or even eliminate the custom scripting that eats up manpower and complicates workflow deployment.
Also, look for suitable support for your particular IT environment. Whether you operate a mix of Linux, Unix or Windows, are heavy into Java, Web Services, Oracle or SQL Server, use OpenPGP to encrypt and decrypt tasks or Secure Shell (SSH) to protect data exchange between devices, your scheduling application should be robust enough to handle it all.
The demands on today’s data warehouses are more stringent and up to the minute than ever before. With a strong job scheduler at work, it’s possible to accommodate these demands while extending human and computing resources-and even improving return on infrastructure. Whether you have a job scheduler in place or are considering adding one, leverage its capabilities wisely and you’ll find your goal of a near real-time data warehouse becoming a reality.
Ben Rosenberg is founder and President at Advanced Systems Concepts. An expert in enterprise-class IT automation and system utilization solutions, Ben has nearly 40 years of experience in engineering and marketing system software products across Windows, Unix, OpenVMS and mainframe platforms. Ben can be reached at brosenberg@advsyscon.com.