{"slug": "microsofts-365-outage-traced-to-undetected-software-bug-gateway-failover-and", "title": "Microsoft’s 365 outage traced to undetected software bug, gateway failover, and traffic surges", "summary": "A latent software defect introduced in a July update, combined with a network gateway failover and a traffic surge, caused the November 13 global Microsoft 365 outage that took down Outlook, Teams and services used by customers including OpenAI and Anthropic, Microsoft EVP of Copilot, agents and platforms Charles Lamanna disclosed in a post on X. The bug went undetected because load testing used older, weaker hardware and only surfaced on newer systems with more cores; fifteen hours before the outage, Teams failed over from the alternative gateway to the primary gateway also supporting Exchange Online, and the added load hit the defect during the U.S. peak traffic window. Microsoft's automated recovery processes were not designed to restore service from this broad unresponsive state, so restoration took longer than usual and services were largely back about 24 hours later; Lamanna said, \"We are deeply sorry for the impact to all our customers.", "body_md": "The global [Microsoft outage](https://www.sdxcentral.com/news/microsoft-hit-by-major-global-outage-impacts-outlook-teams-openai-and-more/) that took down 365 services was caused by a trio of factors that, when combined, impacted the network gateway fronting services like Outlook and Teams.\n\nOn November 13, services worldwide were hit, including those of Microsoft customers such as OpenAI and Anthropic. It occurred the same day as the [Federal Aviation Administration outage](https://www.sdxcentral.com/news/faa-ground-stop-saga-lumen-highlights-legacy-network-rut-as-hpe-bats-away-juniper-blame/), though the two are unrelated.\n\nIn a lengthy [post on X](https://x.com/clamanna/status/2122209697226948729), Microsoft’s Charles Lamanna revealed what was behind the incident, blaming a “latent defect” from a software update in July. While no initial issues were uncovered, the bug gradually saw servers “become overwhelmed and unresponsive under certain load conditions.”\n\nThe EVP of Copilot, agents, and platforms said the bug was not picked up during normal load testing that used older and weaker hardware, with the issue only exacerbating when run on newer systems with more cores.\n\nCompounding the undiscovered bug was “a surge in load” on the networking gateway and “an unrelated issue from the previous day.”\n\nLamanna explained that Microsoft uses two independent network gateway systems for 365, allowing for failover in the event of disruption. However, some fifteen hours before the outage, Teams failed over from the alternative networking gateway to the ‘primary’ gateway that also supports Exchange Online, which led to increased congestion.\n\n“Taken together, the increased load and the reduced server capacity caused the network gateway to encounter the latent bug from July. This caused requests to build up until servers ultimately stopped responding. This occurred during the peak window of traffic in the U.S.,” Lamanna wrote.\n\nMicrosoft’s attempts to restore service took “longer than usual,” with most users unable to access any 365 services during the outage. The EVP explained that Microsoft’s normal automated recovery processes were “not designed to restore service from this broad unresponsive state.” And since the bug went undetected as far back as July, Microsoft engineers had a far tougher time trying to roll back updates.\n\nServices were largely restored some 24 hours later.\n\n“We are deeply sorry for the impact to all our customers and recognize the impact that this has on their normal business and operations,” Lamanna said.", "url": "https://wpnews.pro/news/microsofts-365-outage-traced-to-undetected-software-bug-gateway-failover-and", "canonical_source": "https://www.sdxcentral.com/news/microsofts-365-outage-traced-to-undetected-software-bug-gateway-failover-and-traffic-surges/", "published_at": "2026-11-17 10:53:32+00:00", "updated_at": "2026-11-17 20:27:37.223460+00:00", "lang": "en", "topics": ["ai-infrastructure"], "entities": ["Microsoft", "Microsoft 365", "Charles Lamanna", "Outlook", "Teams", "Exchange Online", "OpenAI", "Anthropic"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/microsofts-365-outage-traced-to-undetected-software-bug-gateway-failover-and", "markdown": "https://wpnews.pro/news/microsofts-365-outage-traced-to-undetected-software-bug-gateway-failover-and.md", "text": "https://wpnews.pro/news/microsofts-365-outage-traced-to-undetected-software-bug-gateway-failover-and.txt", "jsonld": "https://wpnews.pro/news/microsofts-365-outage-traced-to-undetected-software-bug-gateway-failover-and.jsonld"}}