Google Cloud Explains Service Disruption Caused by Update Error

On Friday, Google released an incident report apologizing for a significant cloud outage that affected over 70 of its cloud services, including Gmail, Google Drive, and Google Meet. The outage occurred on Thursday and also impacted several third-party services such as Cloudflare, OpenAI, and Shopify. Google attributed the disruption to multiple flawed updates and the premature deployment of a new feature for quota policy checks, which had not been adequately tested. The feature introduced blank entries into Google’s system, which were then distributed to all Google Cloud regions, causing widespread failure.

Although engineers identified the root issue within 10 minutes, the full resolution took over seven hours. Google acknowledged the absence of feature flags—commonly used in the industry to safely roll out updates—which could have prevented the failure from spreading. CEO of Google Cloud, Thomas Kurian, publicly expressed regret over the disruption. As a corrective measure, Google plans to revise its system architecture to ensure resilience in the face of partial failures and will enhance both human and automated communication channels to better inform users during future incidents. The company also committed to a full audit of its systems to avoid similar occurrences.