Close
  • Latest News
  • Artificial Intelligence
  • Video
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
Read Down
Sign in
Close
Welcome!Log into your account
Forgot your password?
Read Down
Password recovery
Recover your password
Close
Search
Logo
Subscribe
Logo
  • Latest News
  • Artificial Intelligence
  • Video
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
More
    Subscribe
    Home Applications
    • Applications
    • Cloud
    • Storage

    Latest Google Gmail Outage Caused by Insufficient Router Capacity

    Written by

    Clint Boulton
    Published September 2, 2009
    Share
    Facebook
    Twitter
    Linkedin

      eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

      Google’s Gmail application was knocked out for the majority of users for 100 minutes on Sept. 1 due to human error, the company said last night after the Gmail engineering team fixed the issue.

      Google took a small fraction of Gmail’s servers offline to perform routine upgrades. Google does this regularly, sending traffic to other locations when one is offline. That’s when things got hairy, as Ben Treynor, vice president of engineering and site reliability czar for Google, explained:

      ““We had slightly underestimated the load which some recent changes (ironically, some designed to improve service availability) placed on the request routers-servers which direct web queries to the appropriate Gmail server for response. At about 12:30 pm Pacific a few of the request routers became overloaded and in effect told the rest of the system “stop sending us traffic, we’re too slow!” This transferred the load onto the remaining request routers, causing a few more of them to also become overloaded, and within minutes nearly all of the request routers were overloaded. As a result, people couldn’t access Gmail via the Web interface because their requests couldn’t be routed to a Gmail server.”“

      Through internal monitors, Treynor said the Gmail engineering team was alerted to the failures within seconds and added several request routers online to make up for the dearth in capacity and distributed the traffic across the request routers. Gmail came back online around 2:30 p.m. PDT.

      To ensure this lack of server capacity-which is ironic considering that Google allegedly powers the world’s most popular search engine with more than 1 million servers-doesn’t happen again, Google boosted request router capacity well beyond peak demand for extra juice when the application needs it.

      Treynor also said Google is improving the failure isolation in the routers, so a problem in one data center won’t affect servers in another facility. Moreover, he said that Google is taking steps to make sure that when the request routers are overloaded simultaneously, they all should just get slower instead of refusing to accept traffic and shifting their load to another data center.

      It’s also worth noting that when Gmail did go down, Google urged users to access it via the IMAP and POP mail protocols; mail processing continued to work normally because these requests don’t use the same routers at Google.

      “We know how many people rely on Gmail for personal and professional communications, and we take it very seriously when there’s a problem with the service,” Treynor added. “Thus, right up front, I’d like to apologize to all of you-today’s outage was a Big Deal, and we’re treating it as such.”

      So are the Gmail users who use Gmail for their businesses. Donald told Google Watch: “I use G-Mail to run my CPA practice. This is a serious (huge) problem.”

      Sergei added: “This is a huge problem and an outrage. I demand immediate Gmail access. What is with those people?”

      Indeed, more than 1.75 million businesses use Google Apps and some of them pay Google $50 per user, per year for the Google Apps collaboration suite, which boasts Gmail as its backbone. Users have little patience for a service that conks out on them, particularly when they are paying for the extra reliability and security. Read more about this on TechMeme here.

      The latest issue follows a big outage in February, when Gmail went down for two and a half hours due to “unexpected side effects of some new code.” But these last two issues were nothing compared with the August 2008 outage that took Gmail down for nearly 15 hours.

      Clint Boulton
      Clint Boulton

      Get the Free Newsletter!

      Subscribe to Daily Tech Insider for top news, trends & analysis

      Get the Free Newsletter!

      Subscribe to Daily Tech Insider for top news, trends & analysis

      MOST POPULAR ARTICLES

      Artificial Intelligence

      9 Best AI 3D Generators You Need...

      Sam Rinko - June 25, 2024 0
      AI 3D Generators are powerful tools for many different industries. Discover the best AI 3D Generators, and learn which is best for your specific use case.
      Read more
      Cloud

      RingCentral Expands Its Collaboration Platform

      Zeus Kerravala - November 22, 2023 0
      RingCentral adds AI-enabled contact center and hybrid event products to its suite of collaboration services.
      Read more
      Artificial Intelligence

      8 Best AI Data Analytics Software &...

      Aminu Abdullahi - January 18, 2024 0
      Learn the top AI data analytics software to use. Compare AI data analytics solutions & features to make the best choice for your business.
      Read more
      Latest News

      Zeus Kerravala on Networking: Multicloud, 5G, and...

      James Maguire - December 16, 2022 0
      I spoke with Zeus Kerravala, industry analyst at ZK Research, about the rapid changes in enterprise networking, as tech advances and digital transformation prompt...
      Read more
      Video

      Datadog President Amit Agarwal on Trends in...

      James Maguire - November 11, 2022 0
      I spoke with Amit Agarwal, President of Datadog, about infrastructure observability, from current trends to key challenges to the future of this rapidly growing...
      Read more
      Logo

      eWeek has the latest technology news and analysis, buying guides, and product reviews for IT professionals and technology buyers. The site’s focus is on innovative solutions and covering in-depth technical content. eWeek stays on the cutting edge of technology news and IT trends through interviews and expert analysis. Gain insight from top innovators and thought leaders in the fields of IT, business, enterprise software, startups, and more.

      Facebook
      Linkedin
      RSS
      Twitter
      Youtube

      Advertisers

      Advertise with TechnologyAdvice on eWeek and our other IT-focused platforms.

      Advertise with Us

      Menu

      • About eWeek
      • Subscribe to our Newsletter
      • Latest News

      Our Brands

      • Privacy Policy
      • Terms
      • About
      • Contact
      • Advertise
      • Sitemap
      • California – Do Not Sell My Information

      Property of TechnologyAdvice.
      © 2024 TechnologyAdvice. All Rights Reserved

      Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.