Close
  • Latest News
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
Read Down
Sign in
Close
Welcome!Log into your account
Forgot your password?
Read Down
Password recovery
Recover your password
Close
Search
Logo
Logo
  • Latest News
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
More
    Home Applications
    • Applications
    • Database
    • Networking
    • Storage

    How to Achieve a Near Real-Time Data Warehouse

    By
    Ben Rosenberg
    -
    October 5, 2010
    Share
    Facebook
    Twitter
    Linkedin

      Data warehouses are no longer simply the attics of the business computing world. The use of business intelligence applications is growing exponentially, making such repositories an essential part of daily business life.

      Compounding the need is the democratization of BI applications. Employees at all levels-from sales to production to human resources-are using BI applications on a daily basis. Add in the new forms of data that are flooding in, from radio-frequency identification (RFID) readers, Web services applications, mobile devices, cloud-based sources and more, along with the fact that data latency requirements have dropped (below one hour in 60 percent of cases and under one minute in another 35 percent) and it’s clear just how important near real-time data warehouse updates have become.

      It’s estimated that up to 90 percent of all Global 2000 companies have now established a link between their data warehouse and at least one mission-critical application used to produce revenues or control costs. That’s a jump from just 25 percent in 2007. Without current and accurate information, it’s impossible to produce actionable insights at the pace of modern business.

      Many tools exist to automate the processes used to update data stores but the problem is that most have significant limitations. For example, a DBMS has job-scheduling capabilities (as do even major operating systems including Unix and Windows). But DBMS options typically focus on data maintenance only. And, while operating system tools are convenient, their workflows are limited to tasks occurring within a particular server or operating environment.

      Advantages of Job Scheduling Software

      Advantages of job scheduling software

      Dedicated job scheduling applications have the advantage of accommodating virtually all the quirks and complexities of heterogeneous IT environments and their processes. Tasks can be triggered by the occurrence of an event (say, a sales threshold) rather than simply a date or time of day. One-time or unpredictable situations are not a problem. A range of powerful features-from alerts and failover protection to reports and audit trails-also make it easier for busy IT professionals to manage important processing tasks.

      Perhaps one of the greatest advantages of advanced job scheduling applications, in a world where the pressure to increase IT efficiency is at an all-time high, is how they can stretch and optimize resources. Good schedulers can automatically distribute job flows among available computing assets, whether local or across the planet. They can trigger jobs only when suitable assets are available, on the fly if necessary. They can even be used to manage power consumption by automatically placing assets in “suspend” mode when not needed, lowering operating costs and contributing to a greener IT environment.

      Robust Job Libraries and Support

      Robust job libraries and support

      To simplify deployment of data warehouse computing tasks, look for a job scheduling solution with a robust job library that allows work loads to be easily assembled. A good library will reduce or even eliminate the custom scripting that eats up manpower and complicates workflow deployment.

      Also, look for suitable support for your particular IT environment. Whether you operate a mix of Linux, Unix or Windows, are heavy into Java, Web Services, Oracle or SQL Server, use OpenPGP to encrypt and decrypt tasks or Secure Shell (SSH) to protect data exchange between devices, your scheduling application should be robust enough to handle it all.

      The demands on today’s data warehouses are more stringent and up to the minute than ever before. With a strong job scheduler at work, it’s possible to accommodate these demands while extending human and computing resources-and even improving return on infrastructure. Whether you have a job scheduler in place or are considering adding one, leverage its capabilities wisely and you’ll find your goal of a near real-time data warehouse becoming a reality.

      Ben Rosenberg is founder and President at Advanced Systems Concepts. An expert in enterprise-class IT automation and system utilization solutions, Ben has nearly 40 years of experience in engineering and marketing system software products across Windows, Unix, OpenVMS and mainframe platforms. Ben can be reached at brosenberg@advsyscon.com.

      Ben Rosenberg
      Ben Rosenberg is founder and President at Advanced Systems Concepts. An expert in enterprise-class IT automation and system utilization solutions, Ben has nearly 40 years of experience in engineering and marketing system software products across Windows, Unix, OpenVMS and mainframe platforms. Ben can be reached at brosenberg@advsyscon.com.

      MOST POPULAR ARTICLES

      Big Data and Analytics

      Alteryx’s Suresh Vittal on the Democratization of...

      James Maguire - May 31, 2022 0
      I spoke with Suresh Vittal, Chief Product Officer at Alteryx, about the industry mega-shift toward making data analytics tools accessible to a company’s complete...
      Read more
      Cybersecurity

      Visa’s Michael Jabbara on Cybersecurity and Digital...

      James Maguire - May 17, 2022 0
      I spoke with Michael Jabbara, VP and Global Head of Fraud Services at Visa, about the cybersecurity technology used to ensure the safe transfer...
      Read more
      Applications

      Cisco’s Thimaya Subaiya on Customer Experience in...

      James Maguire - May 10, 2022 0
      I spoke with Thimaya Subaiya, SVP and GM of Global Customer Experience at Cisco, about the factors that create good customer experience – and...
      Read more
      Big Data and Analytics

      GoodData CEO Roman Stanek on Business Intelligence...

      James Maguire - May 4, 2022 0
      I spoke with Roman Stanek, CEO of GoodData, about business intelligence, data as a service, and the frustration that many executives have with data...
      Read more
      Cloud

      Yotascale CEO Asim Razzaq on Controlling Multicloud...

      James Maguire - May 5, 2022 0
      Asim Razzaq, CEO of Yotascale, provides guidance on understanding—and containing—the complex cost structure of multicloud computing. Among the topics we covered:  As you survey the...
      Read more
      Logo

      eWeek has the latest technology news and analysis, buying guides, and product reviews for IT professionals and technology buyers. The site’s focus is on innovative solutions and covering in-depth technical content. eWeek stays on the cutting edge of technology news and IT trends through interviews and expert analysis. Gain insight from top innovators and thought leaders in the fields of IT, business, enterprise software, startups, and more.

      Facebook
      Linkedin
      RSS
      Twitter
      Youtube

      Advertisers

      Advertise with TechnologyAdvice on eWeek and our other IT-focused platforms.

      Advertise with Us

      Menu

      • About eWeek
      • Subscribe to our Newsletter
      • Latest News

      Our Brands

      • Privacy Policy
      • Terms
      • About
      • Contact
      • Advertise
      • Sitemap
      • California – Do Not Sell My Information

      Property of TechnologyAdvice.
      © 2022 TechnologyAdvice. All Rights Reserved

      Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.

      ×