eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Officials with Amazon Web Services have begun patching a significant number of its Elastic Compute Cloud (EC2) servers to deal with a bug in the Xen hypervisor, a process that will force the giant cloud services provider to reboot up to 10 percent of the systems.
AWS began sending letters out to customers on Sept. 24, letting them know that some of their services will be briefly interrupted during the time the patches are applied and the host system is rebooted. Officials also wanted to assure customers that the issue has nothing to do with the so-called Bash Bug that is generating a lot of headlines and threatens systems running Linux and Unix.
Instead, the problem is a vulnerability in the Xen hypervisor, which Citrix Systems will publicize on Oct. 1 and at the same time issue a patch. Citrix is delaying the release of the Xen Security Announcement to give AWS and other Xen customers time to patch the defect before it becomes public, according to AWS Chief Evangelist Jeff Barr.
AWS engineers began the patching process at 10 p.m. ET Sept. 25, and will complete it by Oct. 1, Barr wrote in a Sept. 25 post on the AWS blog.
“While most software updates are applied without a reboot, certain limited types of updates require a restart,” he wrote. “Instances requiring a reboot will be staggered so that no two regions or availability zones are impacted at the same time and they will restart with all saved data and all automated configuration intact. Most customers should experience no significant issues with the reboots. We understand that for a small subset of customers the reboot will be more inconvenient; we wouldn’t inconvenience our customers if it wasn’t important and time-critical to apply this update.”
The massive cloud services provider has outlined the steps customers should take to reboot their instances. According to the company, it should take only a few minutes to reboot an instance. When there is a reboot, the instance will stay on the same physical host, which means the instance keeps its public DNS name, private IP address, and whatever data is on its instance store volumes.
According to cloud management company RightScale, the patching and rebooting will impact a wide range of instance types, though those not affected will beT1, T2, M2, R3 and HS1. In a Sept. 25 post on the company blog, Shivan Bindal, senior product manager at RightScale, wrote that the notice AWS sent out to customers said that not all instances within the impacted instance types will need to be rebooted, and that customers will need to check their maintenance notices to find the specific IDs for the impacted instances.
“As an example, in our own RightScale accounts, we received notices for roughly 10-20 percent of all our instances,” Bindal wrote. “This obviously represents a higher percentage of instances within the impacted instance types. Among our customers (who tend to be larger consumers of AWS), we are finding the vast majority have instances that will be impacted. However, accounts that use only the smaller T1 and T2 instance types (which is what you get in the AWS Free Tier) may not be impacted at all. Again, in the spirit of excess caution, we would suggest closely monitoring all instances over the coming days.”
He also suggested that customers could relaunch instances before the maintenance in the chance that they could get a host that has already been patched. Given the scale of the patching, there may not be enough capacity on patched hosts to guarantee that preemptive relaunching will work, Bindal said, but it’s worth a try.
“Despite this, we do suggest that you attempt to relaunch impacted instances and then wait a while and check the AWS console to see if the newly launched instances generated a new maintenance window,” he wrote. “Per an AWS employee on the EC2 support forum, as of 9:57 AM Pacific Time on September 25, AWS is saying that the AWS console will be updated hourly with scheduled maintenance events, which should include your newly launched instances.”