Google Project Zero security researcher Tavis Ormandy reported critical flaws on the Cloudflare cloud security service, that could have potentially enabled the leakage of user passwords and data. The flaws were initially reported by Ormandy on Feb. 18, though public disclosure of the issue was only made on Feb. 23.
“It was clearly a serious issue and we formed a team immediately to work on it,” John Graham-Cumming, CTO of Cloudlfare told eWEEK.
Cloudflare has provided a detailed timeline of the issue, showing just how fast the company was able to respond to the issue and move to protect customers.
According to the timeline, Cloudflare was first alerted to the issue on Feb. 18 and assembled a cross functional team within 8 minutes of receiving details of the bug from Google. Cloudflare didn’t just gather its teams to talk either, rather the company moved quickly to mitigate the risk. According to the timeline, Cloudflare shutdown one of the capabilities that was at the root cause of the data leakage risk, within 90 minutes of getting the vulnerability details from Google.
“Cloudflare have been leaking customer HTTPS sessions for months. Uber, 1Password, FitBit, OKCupid, etc.,” Ormandy wrote in a twitter message on Feb 23.
In his disclosure, Ormandy explained that he was working on a ‘corpus distillation’ project, which is a process used by Google to optimize the results of fuzzing. After digging through some unexpected data, he discovered that if an HTML page hosted behind Cloudflare had a specific combination of unbalanced tags, unitialized data could sometimes be leaked.
“Kinda like heartbleed, but Cloudflare specific and worse for reasons I’ll explain later,” Ormandy wrote in his disclosure. “I didn’t realize how much of the internet was sitting behind a Cloudflare CDN until this incident.”
Heartbleed was discovered by Google researchers in April 2014 and was a vulnerability in the open-source OpenSSL cryptographic library that could have enabled attackers to leak information from encrypted data connections.
In its technical disclosure, Cloudflare explained that the issues were in three minor Cloudflare features that were using a certain HTML parser chain that was un-intentionally leaking data. The three services were shutdown by Cloudflare after getting the Google report and have now been fixed and re-enabled.
Simply patching its own systems however wasn’t enough, since Cloudflare discovered that Google as well as other search engines, had actually cached some of the leaked memory by way of regular crawling.
“The infosec team worked to identify URIs in search engine caches that had leaked memory and get them purged,” Graham-Cumming wrote in a blog post. “With the help of Google, Yahoo, Bing and others, we found 770 unique URIs that had been cached and which contained leaked memory.”
Cloudflare also emphasized that to the best of its knowledge, there were no known or reported exploits as a result of the data leakage. Agilebits, which operates the popular 1Password password service that relies on Cloudflare advised its’ users that none of them were at risk from the Cloudflare HTTPS leakage issue.
Adding further to its due diligence, Cloudflare has also been very busy auditing its other systems to avoid any other potential leakage risks.
“We have already gone back through older software using static analysis and memory tools to ensure there are no further problems,” Graham-Cumming told eWEEK.
Sean Michael Kerner is a senior editor at eWEEK and InternetNews.com. Follow him on Twitter @TechJournalist.