The Ubuntu community had to yank five of the eight Ubuntu-hosted community servers sponsored by Canonical offline Aug. 6 after discovering that the servers had been hijacked and were attacking other machines.
It was suggested during an IRC (Internet relay chat) meeting of the Ubuntu colocation team Aug. 14 that the source of the troubles might have been a Chinese IP address trying to log onto the servers by brute force “for a long time now it seems,” said a participant.
On Aug. 14, the community began to bring the machines back up in a safe state so that they could recover data from them. Unfortunately, according to Ubuntu Community Manager Jono Bacon, the servers were all found to be out of date, stuffed with Web software, and missing security patches—at least in the instances where it was easy to determine what version theyre running.
“An attacker could have gotten a shell through almost any of these sites,” Bono wrote in a posting, regarding a change to location server policy that resulted from the incident.
Click here to view an eWEEK Labs walkthrough of Ubuntu 7.04.
“FTP (not sftp, without SSL) was being used to access the machines, so an attacker (in the right place) could also have gotten access by sniffing the clear-text passwords,” he said. Also, “the servers have not been upgraded past breezy due to problems with the network card and later kernels. This probably allowed the attacker to gain root.”
Bringing the servers back up has taken longer than the managers would have liked, Bono said. Given that theyve been relying on help from members spread over the globe, there are “arbitrary limits imposed by those remote hands” and theres a “(relative) lack of bandwidth” available with which data can be copied from the machines, he wrote.
During the Aug. 14 IRC meeting, location teams were given a choice to migrate to the Canonical data center or stay on the hosted/outsourced servers. Canonical, based in the U.K., is a provider of services to individual and corporate open-source software users.
The pluses of moving to the Canonical data center, Bono said, include better hardware and bandwidth, full-time support from Canonicals systems administration team—including software maintenance—and integration into Ubuntus existing backup infrastructure.
Some of the minuses the Ubuntu community will have to deal with in a move to Canonical—the company behind the Ubuntu distribution—include having less software supported—with the wiki engine MoinMoin, the blog platform WordPress and the Ubuntu community forum Planet on the short list of still-supported applications.
The migration was still in swing as of Aug. 14, and the collocation team leaders were looking for help. “Id be very happy if I got one index.html file to ubuntu-fi.org today as a start :) MoinMoin would be very nice too,” one said during the IRC meeting. “One thing I would ask for is patience. I understand that a service outage like this makes many people anxious,” he said, requesting that those anxious about restoration of services go to the #canonical-sysadmin channel and ask publicly so that the first available systems administrator can answer the request.
To read more about software development the Ubuntu way, click here.
In the meantime, data isnt lost, although applications must be deep-sixed since executable code simply cant be trusted following the intrusion.
“Due to the nature of the intrusion, we must assume that any and all executable code of any sort on the old sites is dangerous,” said the meeting leader, “Spads.” “…We have data, but executable code (python, PHP, Perl, any CGI, etc.) will need to be replaced.”
Check out eWEEK.coms Security Center for the latest security news, reviews and analysis. And for insights on security coverage around the Web, take a look at eWEEKs Security Watch blog.
