eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Google is reviewing the manner in which it responds to potential fraudulent activity involving cloud customer accounts following criticism over a recent incident in which the company briefly shut down a customer’s critical production application without warning.
In its Medium blog July 2, Google engineering support regional lead Brian Bender apologized for suspending services on the customer account and acknowledged it could have handled the situation better.
“We are now conducting a detailed review of our abuse prevention processes,” Bender said. He outlined several measures Google plans on taking to ensure the same thing doesn’t happen again. “We sincerely apologize for this issue and are working quickly to make things better, not just for this customer but for all [Google Cloud Platform] customers.”
Google’s mea culpa stems from a Medium blog post on June 29 in which a cloud administrator for an unidentified—but apparently large company—slammed Google for turning off a critical service in a “no-warnings-given, abrupt way”.
According to the administrator, the company has a project running on Google Cloud for monitoring hundreds of wind turbines and dozens of solar plants scattered over 8 countries.
Asset managers at the company use the system to monitor the health of wind turbines and associated equipment and to initiate immediate remedial action if anything goes awry. Development and forecasting teams also use the system for use cases that translate directly to revenue for the company, the administrator noted.
On June 28, the company received an automated alert about the entire project being down. “I receive a barrage of emails from Google saying there is some ‘potential suspicious activity’ and all my systems have been turned off,” the administrator wrote on the Medium blog. “EVERYTHING IS OFF. THE MACHINE HAS PULLED THE PLUG WITH NO WARNING,” the poster said referring to Google.
Google’s action resulted in the site being down and all application and database services becoming inaccessible as well. According to the administrator, Google’s warnings about suspicious account activity provided no phone number to call and the customer chat service was turned off.
The only apparent recourse was to fill in a form and to upload a picture of the credit card that was used to open the account and a photo of government issued ID of the cardholder. Google informed the company that failure to provide the information within three days would result in the company’s entire project potentially being deleted. In this particular case, the CFO happened to be the cardholder and the administrator was able to provide the requested information and have Google restore services in about 20 minutes.
“What if the cardholder is on leave and is unreachable for three days? We would have lost everything — years of work — millions of dollars in lost revenue,” the administrator said.
The post went on to slam Google for turning things off and then asking for an explanation instead of the other way around. Other enterprise cloud providers warn about suspicious activity and provide customers an opportunity to sort things out before pulling the plug.
“I understand Google’s need to monitor and prevent suspicious activity,” the administrator noted. But the company needs to have a human element reviewing suspicious activity and not rely purely on automation and AI.
Several commenters on Reddit and Hacker News forums echoed similar criticism over the quality of Google Cloud’s customer service. Some faulted the company for its heavy reliance on automated systems and what they perceived as Google’s failure to invest enough in human support staff.
However, a few wondered why the Google cloud customer had apparently not simply used any of Google’s paid enterprise cloud support options when hosting such a critical production app with the company.
In his blog, Bender said Google would soon implement additional measures for evaluating suspect accounts and provide customers with more notice and opportunities to redress situations. Google will also revaluate the data sources it uses to evaluate potential fraudulent activity. “We will endeavor to [meaningfully] improve the effectiveness of how we communicate account warnings,” Bender said.