eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
With IT budgets contracting under the weight of the recession, the sales pitch for software as a service is sounding more attractive than ever.
SAAS can help organizations quickly roll out new applications, avoid large upfront capital investments, and expend fewer application management and maintenance resources moving forward.
However, the qualities that make SAAS an attractive model for acquiring and consuming applications can introduce a host of new wrinkles for organizations that choose them. A key issue revolves around application reliability, which includes ensuring both the availability and sufficient performance of the applications on which organizations depend.
For one thing, SAAS vendors deliver their wares over the public Internet, which means these applications perform differently from applications that run on local clients or over an organization’s internal network. What’s more, the fact that SAAS applications are hosted and maintained by external parties means that companies have less visibility into the workings of these services than they do with applications hosted on-premises.
Because of the nontraditional delivery method and relatively black box operation of SAAS applications, IT departments must approach their responsibilities around ensuring application reliability a bit differently from what they are probably accustomed to doing.
A successful strategy for ensuring SAAS reliability must include attention to users’ expectations and needs, the health of the network, the architecture and practices of potential SAAS vendors, and the tools and services that can be used to measure and monitor the SAAS applications that your organization consumes.
Inside Your Firewall
Inside Your Firewall
To ensure that your SAAS applications meet your organization’s expectations, you must determine what those expectations are. If you’re looking to replace an existing system with a SAAS-based alternative, identify how long it takes to complete typical transactions and operations, and reserve these numbers as a guide.
If the SAAS application will be pushing significant amounts of data between your users, the Web-based characteristics of SAAS applications may be an issue. For instance, if you’re looking at switching from on-premises to SAAS e-mail, keep in mind that any large files that your users now send from point to point within your campus will have to pass through an external server. Depending on your configuration, you may need to adjust some of your practices to achieve the reliability that your users require.
In addition to clarifying expectations, it’s important to begin your SAAS evaluation by considering the networks through which your users will access these applications. Moving to SAAS means boosting the importance of your company’s connections to the Internet, so it’s important to take stock of your company’s connection to the Internet cloud. You’ll also need to evaluate complementary services, such as DNS. If you’re already experiencing bandwidth and latency issues in any of the locations from which your users will consume the SAAS services, you’ll need to clear these up before the SAAS operation can run reliably.
In addition, you should put monitoring tools in place to stay abreast of issues with the Internet connection so that if problems with the SAAS applications arise, you can find out without too much trouble whether the problems are on your end or with the provider’s service.
Once you’ve defined your SAAS needs and have done a sanity check on your networks, you’re ready to begin evaluating and talking to individual SAAS providers. Among the many issues to discuss, there’s a group of topics that relate directly to reliability.
Questions for the Provider
Questions for the Provider
Since a SAAS provider’s infrastructure is-from the perspective of its customers-largely a black box system, one of the most important reliability-related SAAS characteristics to consider is the provider’s service health dashboard, the system through which a SAAS vendor provides its users with information regarding service health and interruptions. Since most SAAS providers make their status pages available to the public on their Web sites, this is a good place to begin your SAAS provider reliability evaluation.
Make sure that the SAAS vendor you’re evaluating provides information about the uptime and performance state of each of the discrete services it offers, and that the information is accurate, up-to-date and easy to locate. It’s also important that the SAAS provider offers information related not only to current status, but also to historical status. What’s more, the SAAS provider should offer ways to notify you of outages or service degradations, such as through e-mail alerts or RSS feeds that administrators can integrate into their own infrastructure-monitoring dashboards.
When it’s time to discuss reliability with the SAAS provider, SLAs (service-level agreements)-which lay out both the uptime and performance characteristics that customers can expect from their providers, as well as the compensation customers will receive when these commitments are not met-are a natural starting point. However, many SAAS providers, including well-established outfits such as Salesforce.com, do not offer SLAs, due perhaps to challenges of defining SAAS reliability with enough rigor to suit a formal agreement.
If your prospective SAAS vendor will accept an SLA, the agreement can serve as a useful tool for matching up your organization’s expectations with the level of service the vendor is prepared to offer, particularly if your SLA drills down beyond basic uptime to include specific performance metrics.
However, the burden of monitoring uptime and claiming compensation for service outages and degradations typically falls on the customer, and partial credits on the SAAS usage bill won’t make up for significant application downtime. Whether or not the SAAS provider you’re evaluating offers an SLA for its service, the most important thing is that the vendor proves capable of meeting your uptime and performance needs.
You should discuss with the SAAS provider how its architecture deals with surges in load, and how it is prepared to scale to additional customers without affecting your application reliability. Also, you should ask about the geographical distribution of the vendor’s data centers-the closer the data centers are to your users, the better the performance you can expect.
In addition, you should talk with the SAAS provider about customizations it plans to make to its applications and consider the potential performance implications of those customizations. Along similar lines, you should find out about software update schedules and discuss what sort of impact these updates have on performance. If the updates typically occur at a time when you require full uptime and performance from your application, discuss whether you can control the time when the updates occur.
Regarding the SAAS application itself, find out to what extent the vendor’s application takes advantage of client-side execution or RIA (rich Internet application) technologies, which can reduce the impact of Internet connectivity issues-such as poor latency-by reducing the server-to-browser round trips required to execute certain operations, such as item sorting.
Along similar lines, make sure to ask the SAAS vendor whether its application makes any provisions for offline data access. For example, SAAS e-mail services tend to have a built-in offline access story, as both the IMAP and Exchange MAPI protocol work with local mail clients to keep mail available locally as well as in the cloud. With other applications that have fewer standardized data protocols, the offline story might be less attractive. However, technologies such as Google’s Gears, which Google and Zoho employ for some of their SAAS offerings, allow for some offline data access. Even SAAS vendors without a current offline strategy should at least be able to discuss where such capabilities fit into their product road maps.
Testing for Yourself
Testing for Yourself
Once you have defined your needs, have set your expectations and have discussed these matters with the prospective SAAS providers, it’s important to test the SAAS services for yourself, beginning with demo accounts and input from a group of relevant users on how the application performance meets their needs.
Beyond the anecdotal accounts of application performance and availability from the pilot group members, some SAAS services, such as Service-now.com, include page render time counters somewhere on their application screens, which can help quantify the transaction times these applications deliver, both initially during the evaluation of the service and moving forward.
There are also some services, such as TrustSaas.com and Pingdom.com, that keep tabs on SAAS services and offer reports on the uptime of these services, independent of what the vendor has to say about itself.
Simple uptime checks tell part of the story, but to get a more detailed view of SAAS performance, you can turn to products such as Keynote’s Keynote Internet Testing Environment, or KITE, with which you can create interaction scripts that encapsulate the sort of work you do with your SAAS applications, and test the applications’ performance both from within your network and from certain points of presence that Keynote maintains. For more on KITE, see my review here.
Labs Executive Editor Jason Brooks can be reached at jbrooks@eweek.com.