Cloudflare Explains DNS Resolver Failure 1.1.1.1 Incident

On June 27, 2024, users worldwide experienced issues accessing or experiencing a degradation in performance of the DNS resolver 1.1.1.1 from Cloudflare, originally launched in 2018. The problem was attributed to a combination of route captures via the BGP protocol and route leaks. Cloudflare actively utilizes the RPKI infrastructure to validate the origin of routes. This system enables IP address owners to securely store and share ownership information, while allowing other operators to verify route announcements against ROA.

By utilizing ROV and signed prefixes like 1.1.1.0/24, the impact of route captures is minimized. However, despite the implementation of RPKI, the route 1.1.1.1/32, used by Electronet S.A. and adopted by multiple networks, including a Tier 1 provider, resulted in the DNS resolver becoming inaccessible for over 300 networks in 70 countries.

The situation was exacerbated by a route leak of 1.1.1.1.0/24 from Nova Rede de Telecomunicaçõ, propagated through the Peer-1 Global Internet Exchange, affecting additional users. Cloudflare apologized for the inconvenience and assured users of their efforts to enhance detection and rapid response methods for similar incidents. The company also actively advocates for the use of RPKI mechanisms to prevent route captures and leaks.

Since its launch, the public DNS resolver 1.1.1.1 has become highly popular but has also faced several issues, including route captures through BGP. Route capture occurs when a network incorrectly announces prefixes, potentially directing traffic away from Cloudflare. Route leaks, on the other hand, happen when networks unlawfully announce routes they shouldn’t, leading to network congestion and delays.

The incident on June 27, 2024, was initiated when Eletronet began announcing 1.1.1.1/32 to clients and partners, while Nova Rede de Telecomunicaçõ distributed 1.1.1.1.0/24, compounding the leakage impact. Cloudflare promptly disconnected several points of connection with Eletronet to mitigate the incident’s effects.

Cloudflare continues to enhance route leak detection and promotes RPKI implementation for route validation. The company aims to ensure the validation of both route origins and paths in the future. Despite the challenges posed by route captures and leaks, Cloudflare remains committed to improving detection methods and supporting the

/Reports, release notes, official announcements.