eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
It started with a 5-minute presentation at a long-forgotten conference in 2004. While attending the conference, Lance Shinabarger, who was then working as a network engineer in the corporate IT department of Dominos Pizza, decided he would take a risk on what just a few years ago was still considered a somewhat-exotic form of cutting-edge technology—virtualization.
“It was a very quick overview, and I was kind of intrigued by it,” Shinabarger, now the manager of information security at Dominos, told eWEEK in an extensive interview detailing the companys three-year adoption of virtualization technology within its main data center and remote sites. “On top of that, too, we did have problems. We had an aging server fleet and applications that were still running on [Windows] NT 4.0, and we were looking [for a way to replace] the hardware that was dying and a way to keep everything intact while our developers moved off of that platform.”
It would take a lot of convincing—not only of immediate bosses but other company executives as well—over several months by Shinabarger and the IT department to implement what three years later seems like a fairly simple use of virtualization. Namely, Shinabarger and his staff sought a new way to move and then consolidate mission-critical applications from an aging, out-of-warranty server fleet to newer systems with more RAM and data capacity, better processor power, and enhanced security.
Dominos, of Ann Arbor, Mich., is an expansive company with 145,000 franchise partners and employees worldwide.
Along with the aging server fleet, IT administrators such as Shinabarger were concerned about recovering information if a disaster hit. There were also cost and space considerations as the companys IT infrastructure grew and adding servers became the norm rather than the exception. In addition to its data center, the company uses several remote sites for tasks such as disaster recovery and data backup.
Complicating the companys internal infrastructure is how Dominos is structured. On one hand, the IT department supports and is responsible for about 1,600 pieces of hardware in the companys main data center and remote locations. On the other hand, the IT staff is also responsible for more than 40,000 systems when all the companys franchises—which can plug into the main network—are added into the picture. “Were the smallest big company that you have ever seen,” Shinabarger said.
In an e-mail, Jason McCracken, senior network engineer with Dominos, detailed the companys infrastructure.
“Dominos, like many other franchise infrastructures, has infrastructure to support our corporate operations and infrastructure to support stores [and] franchisees,” McCracken wrote. “Once we take into account all the services, such as network connectivity, that we support for franchisees, the number of devices (and users) rises exponentially. We also have to take into account being able to support our development team(s) who work on our public Web sites (such as Dominos.com) and the support provided for them.”
Adding to an already-complicated picture is that when Dominos first jumped into virtualization in the fall of 2004, there were no road maps to follow. As one IT worker said, if other companies with franchises were setting up virtualization in the data center, they were not going to share their results with a significant rival.
Virtualization, which uses software commonly referred to as a hypervisor, enables physical hardware—such as a server, PC or storage device—to be partitioned, allowing multiple operating systems or applications to run on the hardware independent of one another.
While virtualization can be traced back to IBMs mainframes in the late 1960s, it was VMware—the vendor that Dominos turned to as it began its virtualization of the data center—that brought the technology to the rapidly expanding x86 market. Now owned by EMC, VMware, which is preparing to sell 10 percent of the company in an initial public offering this summer, is the market leader in the field.
VMwares success and the acceptance of virtualization technology as a mainstream IT tool have prompted a number of other companies to enter the field, including SWsoft; Parallels; Virtual Iron; and XenSource, whose founders helped develop the open-source Xen hypervisor as an alternative to proprietary technology.
“I think there has been a big change just in the last three years with what companies like VMware have done with bringing virtualization technology to common platforms like x86 servers,” said Charles King, an analyst with Pund-IT. “Prior to this, virtualization was very much an enterprise-only product. What VMware and, to a lesser extent, the Xen open-source initiative did was bring the technology to a much-larger audience and allow the technology to be deployed on multiple platforms. This has all happened in just the past few years, and it has really arrived as a mainstream product.”
In addition to these companies, some major IT vendors have begun touting the benefits of virtualization in their products. Microsoft is expected to expand its virtualization capabilities in the companys much-anticipated “Longhorn” server operating system when its released next year, while Sun Microsystems executives are talking up the virtualization capabilities in Solaris. Intel and Advanced Micro Devices have begun building virtualization into their processors.
These developments have led analysts to predict that the virtualization market will grow rapidly in the coming years. A May 8 report by research company Gartner found that, while there were about 540,000 virtual machines worldwide in 2006, there could be as many as 4 million in 2009. Much of that growth will involve desktop virtualization, which analysts said could grow faster than what has already occurred in the data center, thanks in large part to the sheer numbers of PCs worldwide.
Next Page: Baby steps for Dominos.
Page 2
All those lofty predictions meant little in 2004, when Dominos IT team took its first tentative steps toward a virtualized data center—specifically, when administrators first tried to take critical applications off older Compaq servers. In the past 10 years, said McCracken, who was present during the IT departments first wave of virtualization, Dominos has undergone a massive reordering of a once- nearly-homogeneous data center that relied primarily on Compaq servers.
“In our old computer room, we had a lot of old Compaq stuff, and when we moved out of there, we changed to Dell for a lot of our infrastructure,” McCracken said. “There was just a lot of phasing out of the older Compaq equipment.”
Today, the companys main data center in Ann Arbor has more of a heterogeneous look, with a combination of IBM and Dell servers—typically using either dual- or quad-core processors and supporting between 8GB and 16GB of RAM—running a variety of operating systems including AIX, Windows 2000 and 2003, and Novells SUSE Linux.
Computing at the companys headquarters is standardized on Dell PCs.
Part of the obstacle to bringing virtualization into the data center, Shinabarger said, was that there were no other companies of comparable size that his team could point to as an example of how the technology could work on high-density, x86 systems. “There was a concern that you had many servers on one physical box, and if that were to go down, what were we going to do,” he said. “At that point, we didnt have a SAN [storage area network], so it was just all those things. It was a lot of fear of the unknown.”
When the Dominos IT department began its first virtualization tests in October 2004, there were fewer virtualization vendors than there are now. However, VMware did offer to help, and developers at Dominos were already using VMware workstation software. “We had seen it. It seemed like it would work, and there were really not a lot of vendors in this space,” Shinabarger said.
Bogomil Balkansky, director of product marketing for VMware, in Palo Alto, Calif., said the situation at Dominos reflected what many other companies were looking for with virtualization—namely, consolidation. From there, other opportunities, such as cost savings and disaster recovery, presented themselves.
“Its really interesting in this case because this was a customer that had consolidation in mind, and they found a tremendous opportunity that virtualization technology can open up,” Balkansky said, adding that virtualization now is about more than simply server consolidation. “Theres a lot more focus on the speed of provisioning that virtualization can provide as well as proving high availability.”
The problem that Dominos and other companies ran into just a few years ago was convincing other executives not only that this technology could work but also that it could provide additional benefits, Balkansky said.
“You dont know whats yours, and you cant put your finger on what is in your box,” said Balkansky, adding that many executives want to be able to see and touch what the server does, something thats impossible with virtual environments. “Once you overcome this, there is a lot less objection, and I think that is part of the reason why virtualization has gone mainstream in just the last few years. People are much more comfortable with this, and they now know what they are getting out of a box.”
The first project Shinabarger received approval for was an attempt to move one mission-critical application—a universal accounts receivable application—from a failing Compaq server to a newer Dell PowerEdge server with about 8GB of RAM, coupled with VMware software.
“We did a [physical-to-virtual migration] as a test, and it was successful, and it was right from the frying pan into the fire,” Shinabarger said. “I dont want to say it was flawless. It took some work to get it going. It was all pretty new at the time, but we did get it going in the end. It did get us out of the fire, and at that point, you did have some mid- and upper-level management buy-in.”
Next Page: Getting VMware to work with SAN.
Page 3
The next step in Dominos virtualization adoption was allowing VMwares technology to work with the companys SAN—also a relatively new enterprise-class technology in late 2004 and early 2005. This was an important step for Dominos because it would mean linking its first virtualized servers together.
The problem at the time, McCracken said, was that VMware had not certified the SAN that Dominos built. All the benefits that come with VMwares technology—such as high availability; VMotion, which allows all working processes to continue throughout a migration to a new host; and backup features—require SAN access.
While the IT department was able to connect the few VMware systems to the SAN, the main roadblock, said Jeff Lang, another of Dominos senior network engineers, was that without certification from VMware, there was no one to call for support if something went amiss. However, that did not stop Lang and McCracken from trying to do as much as they could with what was available. At one point, the IT department was running Windows Compute Cluster Server software with help from a Microsoft patch.
Pund-ITs King said the problems with VMware and SANs were common issues that have now been addressed to an extent.
“There were some issues with initial x86 virtualization solutions and SANs, particularly as to the SAN being able to recognize individual virtual servers hosted within physical servers,” King said. “That issue has been resolved via highly granular management applications, which can also be used to measure usage [and] performance of specific applications and processes within virtual servers, thus providing the means to create accurate SLAs [service-level agreements] and chargeback mechanisms.”
By mid-2005, Dominos executives had seen enough tests and had grown comfortable enough with the technology that the IT department had a freer hand in preinstalling VMware software when new servers were delivered. That initial deployment of replacing the one aging Compaq server grew to four systems that were plugged into the companys nascent SAN and, eventually, to the current 17 systems with preinstalled virtualization software.
“Unless you have a really good reason, were not going to buy a physical machine just for the box unless its a large database or something that requires a lot of horsepower,” McCracken said. “Were just going to virtualize it.”
By early 2007, Dominos had 11 servers in its main data center running virtualization technology and six others in remote data centers that used VMwares ESX Server software. All told, these 17 machines contain approximately 80 virtual environments, a number that could double or triple when the company installs a new SAN, developed by EMC and certified by VMware.
Now the IT department has started to expand the reach of virtualization beyond the main data center in Ann Arbor and is seeking to maximize the technologys use in remote sites used for backup and disaster recovery. Dominos is using ESX Server to remotely control and monitor the virtual environments at two locations.
Starting late last year, Dominos invested in the latest version of ESX Server—3.0—and began moving its internal e-mail application onto systems with virtual environments.
McCracken has started to use virtualization for making adjustments to applications. Instead of hoping that the changes will take, he said he uses VMwares built-in rollback function to test changes before they are deployed on the network.
“Its like you took a ghost image of the machine, and you know its exactly like it was before, and … you know its going to work,” McCracken said. “Its really hard to express how nice of an advantage that is because the last thing that I did it for was a RADIUS server. When you have thousands of stores connected [to the system], and if you cant get it set up, then you cant get your data, and the whole management team is going to be looking at you when they cant get that data because you were the one that upgraded this machine.”
The companys developers have also begun using the technology to develop and test applications in a virtual environment and then return the virtual image to the IT department, where its checked for security and other flaws before its deployed on the network.
Check out eWEEK.coms for the latest news, views and analysis on servers, switches and networking protocols for the enterprise and small businesses.