eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
It was bad enough that federal officials essentially deleted the ca.gov domain in the process of shutting down a hackers redirect to porn pages on Oct. 2. Whats worse: There are still fully hacked ca.gov sites up and serving redirects to drug purveyors.
California state officials were caught off-guard on Tuesday when the federal GSA (General Services Administration), which manages all “.gov” domains, moved to shut down the states access to the Internet.
The GSA took the drastic action after a hacker managed to insert redirects to porn pages onto the Web site of a transportation agency for the county of Marin.
Jim Hanacek, acting deputy director for the state Department of Technology Services Policy and Planning Division, told eWEEK that he wasnt sure why the GSA decided on the widespread shutdown, but that his department wasnt notified in advance—nor, was the right person notified.
“They sent an e-mail to one of our staff thats kind of the normal contact but not necessarily for a major event like this,” Hanacek said. The GSA sent the notice via e-mail around 11 a.m. PDT, but the recipient didnt see it until around noon on Tuesday.
Scrambling ensued. The Informational Technology Department activated its Emergency Operations Center, calling in high-level managers and deputies to get a consortium of expertise in to resolve the situation. It took hours and multiple phone calls from top California officials to resolve the problem, given that nobody quite knew who to call. In addition, it was after normal working hours on the East Coast, Hanacek said.
Read more here about why some foreign and domestic government sites are defenseless to hackers.
The GSA allocates sub domains to the states, and hence allocates the ca.gov domain to California. When the agency became aware that a sub domain allocated to the County of Marin was redirecting to pornographic sites, it basically removed that subdomain—an overreaction that Hanacek likened to using a shotgun to kill a flea.
The effect didnt knock the entire state government offline—at least, not right away. Rather, it was a snowballing effect that cascaded through various networks, with intermittent and random outages for Web pages and e-mail systems.
After the states Informational Technology Department began getting complaints, it moved to use a technique called force propagation to reverse the damage. That technique instructs domains to immediately update their addresses, as opposed to doing it at their normally scheduled time.
By 5:30 p.m. PDT Tuesday, the problem had been mostly cleaned up, and by 7:30 p.m. the Informational Technology Department was mopping up, monitoring to ensure that everything stayed on track, Hanacek said.
Essentially, California averted what could have been “kind of a big one,” he said—an error that could have brought all Web and e-mail access down in government agencies. As it turns out, no data was compromised, and the states IT officials have been conferring on an SOA (system outage analysis) to determine what went wrong and how to avoid a repeat of the incident.
The GSA has since apologized, explaining that it was looking to protect children from lewd material. “We apologize for any inconvenience to the citizens of California. … The potential exposure of pornographic material to the citizens—and tens of thousands of children—in California was a primary motivator for GSA to request immediate corrective action,” the agency said in a statement.
As for the knee-jerk way it elbowed California off the Internet, the agency said that theres not much choice nowadays, given the risk from attackers. “In these days of heightened security concerns from hackers, it is important to quickly stop potentially harmful damage to federal, state and local Web sites from those who have no love for our country,” the GSA said in the statement.
Still, to ensure it doesnt shoot any more fleas with its shotgun, the agency has also tweaked its policies. “GSA recognizes there must be a balance between protecting citizens while not, at the same time, adversely affecting governments ability to serve citizens via the Internet,” the statement continued. “We have therefore revised our policies to now include more internal checks and balances before a site is shut down and to find better ways to more precisely eliminate offending government sites without having to shut down the primary site.”
Page 2: Feds Suspend California Government Web Sites
Feds Suspend California Government
Web Sites”>
Sunbelt Software President Alex Eckelberry, whos been raising the red flag on hacked government and .edu domains for months now, was one of the people who warned the TAM (Transportation Authority of Marin) last month that it had a porn problem. According to a posting in his blog on Oct. 4, the TAM site is still compromised.
“Despite all the hullabaloo, the now-infamous Marin County TAM website, responsible for a federal shutdown of ca.gov sites, is still not completely clean. While its not redirecting to malware or porn anymore, it still has some dirt underneath the fingernails,” he wrote.
Marin isnt the only one with lingering problems. The site for the Superior Court of the County of Madera, Madera.courts.ca.gov, has also “experienced some pwnage,” Eckleberry noted, with a variety of links showing up that direct visitors to online Viagra sellers. Those links were active on the afternoon of Oct. 4, when eWEEK called the Superior Court, only to find out that the court lacks a full-time technology support staff.
eWEEK Labs inspected the Madera County site on Oct. 4, finding a subdirectory named “/papper” with files that support active links to drug selling sites.
Tulare County Superior Court also appeared to have been infiltrated with illegitimate redirects, as could be seen in cached pages of the site on Oct. 4, where links to hundreds of drug, insurance and pornography sites were listed. Current court pages do not contain the links.
Sixty three percent of malware comes from U.S. sites. To read more, click here.
These infiltrations of government and other major sites are, in fact, commonplace. On Oct. 4, Eckelberry pointed out that the Bank of Ghana was serving porn. Earlier that same day, he noted that
“Were a pretty good data center. We have almost 800 employees with a lot of technically competent staff. You get down to some of these very small government entities at the local level, they have a lot of part-time employees. For them to maintain the level of sophistication we can at a big data center here, would be cost-prohibitive,” he said.
eWEEK Labs Director Cameron Sturdevant contributed to this story.
Editors Note: This story was updated to include information from the GSAs statement.
Check out eWEEK.coms Security Center for the latest security news, reviews and analysis. And for insights on security coverage around the Web, take a look at eWEEKs Security Watch blog.