[Urgent] Cloudflare problems hit websites around the world

Problems with Cloudflare’s content delivery network followed hot on the heels of an incident with its third-party support portal provider.

Problems at content delivery network Cloudflare are causing problems for online services and websites around the world, leaving them with just the kind of outage that Cloudflare’s services are intended to protect from.

“Cloudflare is experiencing an internal service degradation. Some services may be intermittently impacted. We are focused on restoring service. We will update as we are able to remediate,” the company announced on its status page at 11.48 a.m. UTC.

About 90 minutes later, it reported that users in London would be unable to access WARP, a DNS proxy for enterprises enabling Zero Trust policies, as it had disabled the service there while attempting to remediate the problem. Minutes later, it reported that it had identified the issue, re-enabled WARP access in London, and made changes to that service and to Cloudflare Access that enabled them to recover.

Despite this, numerous websites remained inaccessible, instead displaying errors such as “Internal server error Error code 500 Visit cloudflare.com for more information.”

“We are continuing working on restoring service for application services customers,” Cloudflare wrote almost three hours after the incident began.

During the incident, Downdetector.com reported a spike in reports of problems with web services around the world coinciding with the Cloudflare outage, including at ChatGPT provider OpenAI, online design tool Canva, and AWS. with Bloomberg reporting that problems at OpenAI’s ChatGPT service and social media site X (formerly Twitter) were due to the Cloudflare outage.

Ominously, 31 minutes before the company acknowledged the problems with its global network, it had also reported problems with its support portal.

“Our support portal provider is currently experiencing issues, and as such customers might encounter errors viewing or responding to support cases. Responses on customer inquiries are not affected, and customers can still reach us via live chat (Business and Enterprise) through the Cloudflare Dashboard, or via the emergency telephone line (Enterprise). We are working alongside our 3rd party provider to understand the full impact and mitigate this problem,” it reported at 11.17 a.m. UTC.

While there’s no evidence to suggest that Cloudflare’s support portal problems were linked to the subsequent content delivery service outage, support portals have proven a weak link in several attacks on IT vendors in recent months. Discord saw customer data leaked after a breach at a third-party customer service provider. Salesforce has denied responsibility for security problems that saw a number of its customers compromised — and in several instances the finger is pointing to Salesloft Drift, a third-party AI chat tool service often integrated with it.

And in September Verizon reported that 71% of CISOs had been hit by a third-party security incident in the past year. Third-party risk management has become a key concern for enterprises, according to IDC.

Author | Network World