Facebook’s DNS service appeared to be fully restored by approximately 21:30 UTC and is now reachable for most users.
2
13
29
Replies
ThousandEyes tests can confirm that at 15:40 UTC on October 4, the Facebook application became unreachable due to DNS failure. Facebook’s authoritative DNS nameservers became unreachable at that time. The issue is still ongoing as of 17:02 UTC.
22
338
605
Facebook made BGP withdrawals near the time of the incident, however, 2 prefixes covering two of their 4 DNS nameservers (a and b) are still being advertised across the Internet. They are reachable on the Internet but traffic is dropping at Facebook’s network edge.
4
73
127
The 2 DNS nameservers (a and b) are reachable because covering prefix 129.134.0.0/17 is still being advertised, but this advertisement may not have been designed to support the nameserver service.
2
13
56
The 3 specific prefixes covering a and b nameservers before the incident were 129.134.30.0/23, 129.134.30.0/24, 129.134.31.0/24. The specific routes covering all 4 nameservers (a-d) were withdrawn from the Internet at approximately 15:39 UTC.
4
17
56
We've compiled all of these updates into a blog (including some additional graphics), which we'll be updating with our full analysis as soon as it's available. Check back here:
1
23
55
Update from Facebook Engineering: "[...] We want to make clear at this time we believe the root cause of this outage was a faulty configuration change. We also have no evidence that user data was compromised as a result of this downtime."
1
3
13
[New] In-depth outage analysis by
@bitprints
is now live on the blog, answering some of the most common questions we've been getting:
#facebookdown
0
5
17