Doug Madory (also on Bluesky)
@DougMadory
Followers
9K
Following
5K
Media
962
Statuses
6K
Director of Internet Analysis at @Kentikinc. Alum of @dynresearch @renesys @uva @dartmouth @usairforce
Joined August 2013
Island nation of #Tonga is completely offline following a #tsunami triggered by a massive volcanic eruption in the Pacific Ocean. According to @kentikinc data, traffic volumes began to drop around 4:30 UTC (5:30pm local) before finally going to zero at 5:40 UTC (6:40pm local).
48
6K
5K
It’s been >3 weeks since the #UkraineRussiaWar began. Despite many outages, Ukraine's Internet is still online due in part to the heroic efforts by local techs fixing disruptions at great risk to themselves. A 🧵 of 30-day snapshots of Ukrainian internet connectivity. .
17
295
1K
Today's outages in Russia appears to have been caused by a bad substring match by @roscomnadzor. Intending to block Twitter's link shortener t[.]co, Russia blocked all domains containing t[.]co, for example .Microsoft[.]com and Reddit[.]com. (H/T @GregoryKhodyrev).
Russia's attempt to slow internet access to @twitter backfired today, knocking out mobile internet for many Russians. #KeepitOn . @kentikinc netflow data shows two periods when traffic dropped by as much as 24% to RU state telecom Rostelecom starting 07:00UTC (10:00am local).
26
396
776
Facebook outage now 3hrs+ in duration.
Facebook, Whatsapp, and Instagram suffered a global outage today beginning at 15:39 UTC (11:39am ET), according to @kentikinc data. Early indications point to a DNS problem.
38
208
574
Countrywide internet outages in Cuba following widespread anti-government protests. #KeepItOn. According to @kentikinc data, internet traffic to/from Cuba dropped to zero at 20:05 UTC (4:05pm local).
12
340
546
We can confirm that Russian satellite operator Dozor Teleport (AS41942) left the global routing table at about 02:00 UTC earlier today. It is now unreachable, reportedly due to a cyber attack. 4 of 5 routes previously announced by AS41942 were withdrawn, one is now announced by
PMC Wagner has announced that they have taken down the satellite provider Dozor-Teleport and damaged user terminals. Their rivalry with the RU MoD is manifesting in an unusual way. This is the second major satellite provider breach after Viasat
3
169
388
"BGP experts" on Twitter during a major internet outage 😄.
10
37
320
Signs of life for Facebook?. Global outage has lasted over 5.5hrs so far.
Facebook, Whatsapp, and Instagram suffered a global outage today beginning at 15:39 UTC (11:39am ET), according to @kentikinc data. Early indications point to a DNS problem.
35
108
302
Facebook, Whatsapp, and Instagram suffered a global outage today beginning at 15:39 UTC (11:39am ET), according to @kentikinc data. Early indications point to a DNS problem.
14
124
243
UPDATE: Southern Cross Cable Network says it has been able to confirm contact with a communications centre in #Tonga, which is working to restore internet and phone links to the island nation.
Island nation of #Tonga is completely offline following a #tsunami triggered by a massive volcanic eruption in the Pacific Ocean. According to @kentikinc data, traffic volumes began to drop around 4:30 UTC (5:30pm local) before finally going to zero at 5:40 UTC (6:40pm local).
3
227
206
From @Kentikinc’s view, overall peak traffic levels into Ukraine have declined by about 20%. This reflects a drop in internet usage that can be attributed to damaged infrastructure and fewer Ukrainian users due to millions of Ukrainians fleeing, among other factors.
2
21
150
Here's a view of the eruption from space:.
1.14.2021: Large volcanic eruption near Tonga (Hunga Tonga-Hunga Ha'apai volcano) today as seen from outer space. Shown on visible imagery using the Himawari satellite. #hiwx #tsunami #earthquake
1
100
148
Russia's attempt to slow internet access to @twitter backfired today, knocking out mobile internet for many Russians. #KeepitOn . @kentikinc netflow data shows two periods when traffic dropped by as much as 24% to RU state telecom Rostelecom starting 07:00UTC (10:00am local).
3
63
143
BREAKING NEWS: As of early this morning, the South Atlantic island of St Helena is now sending traffic over their new subsea cable branch to @Google's Equiano cable. @kentikinc synthetics saw a dramatic drop in latency from AWS in London: 657ms → 131ms!.
5
45
145
Airstrike in #Yemen knocks out nearly all of country's internet as a comm facility was hit in Al Hudaydah on the Red Sea. According to @kentikinc data, YemenNet went offline at 21:54 UTC (12:54am local). AdenNet is still connected but only serves region around Aden.
5
99
122
From 12:05-12:50 UTC, RU telecom RTComm (AS8342) hijacked a prefix (104.244.42.0/24) belonging to Twitter. The hijack didn't propagate far due to a RPKI ROA which asserted AS13414 was the rightful origin. This is the same prefix hijacked during the coup in Myanmar last year.
4
67
132
Overall, these independent data sources tell a similar story: The overall trend has been a partial decline in traffic and connectivity, but the Ukrainian internet has remained connected. (if I have missed a publicly accessible tool, please reply and I will add it to the list).
4
17
118
2
97
121
We're tracking a drop-off in traffic to Facebook and Instagram beginning at 15:20 UTC. #instagramdown
16
35
125
@norges14 @Luz_Cuba @MarioJPenton Según los datos de @kentikinc, hubo un nuevo apagón en Cuba a las 2:20pm local (18:20 UTC). #KeepItOn #SOSCuba
4
73
125
Internet service returning to #Cuba this morning after a country-wide outage last night. According to @kentikinc data, the blackout began at 00:30 UTC (8:30pm local) and began returning at 07:10 UTC (3:10am local), lasting over 6 hours.
4
56
116
The nation of Cuba experienced another internet disruption today - but not a complete shutdown. According to @kentikinc data, traffic to/from Cuba dipped by 47% between 13:55 UTC (9:55am local) to 17:10 UTC (1:10 pm local). #KeepItOn #SOSCuba .
3
60
102
Internet curfew in #Iran continues for a seventh day. #IranProtests2022 #KeepItOn. According to @kentikinc data, traffic volumes to Irancell (AS44244), MCI (197207) and Rightel (AS57218) fell off around 12:30 UTC (4pm local). Recent outages have lasted 8hrs.
4
65
105
UPDATE: According to @kentikinc data, internet traffic to/from Cuba continues to be erratic since two brief total blackouts beginning at 20:05 UTC (4:05pm local). #SoSCuba #KeepitOn
5
68
94
Alternatively, @gatech_ioda focuses more on infrastructure and less user traffic levels. It observes a visible decline in BGP routes, hosts responding to pings (active probing), as well as background traffic.
1
15
96
Here's a look back at today's simultaneous DDoS attacks against the websites of Ukraine's parliament, foreign ministry, and executive cabinet. #UkraineRussiaCrisis .
3
62
93
Large BGP routing leak out of India this morning. AS55410 mistakenly announced over 30,000 BGP prefixes causing a 13x spike in inbound traffic to their network according to @kentikinc netflow data. (cc: @anurag_bhatia, @aftabsiddiqui, @jaredmauch)
7
53
99
Internet in Myanmar has gone down a 2nd time in 6 days. This time far more severe than last. Previous outage was focused on blocking traffic from the big four (MPT, Telenor, Ooredoo, Mytel). Now the blackout is total. #KeepItOn
2
55
93
Our friends over at @Cloudflare_radar report a decline in user traffic to their network following the invasion but a gradual return to previous traffic levels.
3
16
85
Australia telecom Optus is experiencing a nationwide outage now in its 5th hour. Internet and mobile phone services currently down for millions of Australian customers and businesses. The number of BGP routes announced by AS4804 dropped from 271 to 122 as the outage began.
6
35
95
Moscow-based Infotel (AS8299), the primary provider of the Central Bank of Russia (along with other financial institutions), went down at 10:55 UTC today due to a reported cyberattack. Outage visible in @IODA_live:.
3
43
87
RPKI in action!. On Friday, AS209 created ROAs for 64.193.176.0/20 and 64.193.192.0/20 to regain IP space hijacked by AS134121. Networks that drop RPKI invalids (2914,1299,6939) dropped the routes, although they are still in circulation via other networks.
6
35
90
Rostelecom stopped announcing the BGP routes of Russia's e-government platform (AS196747) outside the country at 16:45 UTC (7:45pm local) 26-Feb. It hosts a purported target of the #Anonymous hacking group. #UkraineRussiaWar.
1
54
86
Large internet outage in Myanmar amid possible military coup d'état. Outage began at 20:40 UTC (5:10am local). Data from @Internetintel and @kentikinc shown below:
3
87
74
Internet connectivity for Iran's primary mobile operators were cut in recent hours. #IranProtests2022 . Traffic volumes for MCI (AS197207) from @kentikinc:
2
36
77
In the past couple of hours, the submarine cable serving Tonga was repaired and island's high speed connection to the internet has been restored. Latency drop from geostationary satellite to submarine cable (ex: 670ms -> 139ms) is huge and visible in @kentikinc Synthetics.
2
32
78
Iranian mobile carriers down again. #IranProtests . Traffic volumes to MCI (AS197207) and IranCell (AS44244) have dropped in the last hour. #KeepItOn
4
43
77
The @Google transparency report shows a decline in traffic web search following the invasion similar to @cloudflare_radar and @kentikinc.
1
9
70
Traffic to Kenya returning to normal levels following national internet outage on day of violent protests. At this moment, we're still trying to verify the claim that the outage was in fact caused by a submarine cable issue.
Severe Internet connectivity problems in #Kenya today beginning at 13:25 UTC (4:25pm local) following violent protests over a controversial new law. Dominant provider Safaricom is blaming the problems on "an outage on two of our under sea cables." .
4
80
76
In Jan 2019, the subsea cable serving Tonga suffered a fault and the island nation had to revert to satellite to maintain connectivity. This hasn't happened today - either due to loss of power or technical failure of their comm systems.
3
26
66
Mobile internet services down in Iran for a third time in as many days. #IranProtests2022 #KeepItOn. According to @kentikinc aggregate NetFlow, traffic volumes to Irancell (AS44244), MCI (197207) and Rightel (AS57218) fell off within minutes of each other just over an hour ago.
2
29
64
Chilling.
It is literally dark in parts of Tonga and people are rushing to safety following the eruption. 🇹🇴
5
16
67
A Brief History of BGP Incidents - a comprehensive post covering the most notable #BGP incidents in the history of internet routing. My latest post for @kentikinc .If I somehow missed your favorite BGP incident, pls reply below.
2
27
66
African Internet is hurting as submarine cable cuts/faults are piling up!. MainOne cable down at around 07:30 UTC earlier today.
Africa Submarine Cable Situation Report - 14/3/24.Situation is Very bad. East Coast of Africa to Europe .Seacom/TGN - Cut.AAE1 - Cut.EIG - Cut.All cut in Red Sea.West Coast of Africa to Europe.WACS - Fault.Mainone - Down.ACE - Fault.SAT3-Down.Faults near Abidjan.
1
48
69
Partial de-peering between Cogent (AS174) and Tata (AS6453) yesterday, first reported by @bgptools. Here's @kentikinc's view of the drop in traffic over 174_6453 beginning around 18:00 UTC on May 16.
(The previous TATA-Cogent Full depeering post was wrong). It appears that TATA-Cogent have partially depeered in APAC. Cogent can't see APAC (notably India) routes anymore. It seems based on statements Cogent has initiated this depeering (Some EU/US routes seem to still be find)
7
29
71
The fact that today’s AT&T outage affected some devices and not others (sometimes within a single household) suggests this isn’t BGP (or solar flares, cyberwar, etc), but some sort of faulty maintenance push by AT&T. #ATTdown.
6
13
66
Everybody can relax on this one (and stop sending it to me😀). It is address space registered to Netgroup of UA announced by Netgroup of UA. I think we're ok. The "hijack" AS path goes through 6849 (Ukrtelecom) and not Russia.
BGP,HJ,hijacked prefix AS212463 31.148.149.0/24, NGROUP, UA,-,By AS35004 NETGRUP, RU,
4
25
65
In recent hours, Ukraine has been the target of multiple DDoS attacks. According to @kentikinc data, targets include Mirohost (AS28907) which hosts websites of the Ukraine Army, as well as Privatbank (AS15742), one of UA's largest banks.
4
37
66
Russian internet infrastructure continues to be the target of DDoS attacks. #UkraineRussiaWar. This includes RIPN (AS42385) which handles the [.]ru TLD which has been receiving DDoS traffic continuously since 02:57 UTC 2-Mar.
4
33
63
Kazakhstan is completely offline today after its govt resigned in response to ongoing #KazakhstanProtests triggered by rising fuel prices. According to @kentikinc, traffic to #Kazakhstan stopped at 10:45 UTC (4:45pm local) and has yet to be restored.
3
28
62
As of 00:19 UTC on 6-Jul, AS3356 started announcing 2000::/12, the largest IPv6 (or IPv4 for that matter) prefix in the global routing table.
Erm . #Level3, R U okay?. "This is an automated alert email about BGP announcements with your certified address space for […] in the [RPKI] service. There are BGP announcements that overlap with your certified address space […]. AS Number Prefix.AS3356 2000::/12".
3
16
56
I spotted the activation of Tonga's cable at 5:56 UTC on 5 Aug 2013. As the graphic shows, the improvement in latency (from satellite to subsea cable) was huge.
1
24
56
Mobile Internet service in Iran down today from 15:00 to 18:30 UTC. The outage followed Iran's loss to England in the #FIFAWorldCup. Iranian national team players choose not to sing national anthem as an act of protest against the Iranian govt.
2
20
58
Here's the view from @kentikinc of the @EpicGames traffic surge due to the #FortniteChapter5Season2 launch last night. Fastly, Akamai, and AWS carrying the load.
The #Fortnite update is making the #CDNs in charge of distribution work hard. At @namex_ixp (Rome #IXP), half a tera of #Internet traffic has just been exceeded.
2
9
60
This morning AWS experienced another outage - this time impacting regions in the western US. #amazondown. From 10:20 to 10:55am ET, us-west-1 and us-west-2 experienced network problems. The outage clearly picked up by @kentikinc's Cloud Performance Dashboard.
1
27
58
In the early hours of Apr 8, LEO satellite service Starlink suffered a global outage. According to @kentikinc aggregate NetFlow, traffic to Starlink went to zero and most (but not all) BGP routes were withdrawn for a more than 20min.
4
16
57
Internet in #Uganda still down despite a winner being declared in this week's national presidential election. Uganda is still disconnected after 72hrs+. #KeepItOn #UgandaDecide2021
1
18
56
The @SpaceXStarlink satellite Internet service experienced a global outage beginning at 20:56 UTC today. According to @kentikinc NetFlow data, traffic to AS14593 began to return after more than an hour of downtime, but service has not yet fully recovered.
7
27
55
This includes all of the Renesys, Dyn Research and Oracle Internet Intelligence blog posts. 😿. Thank god for the @internetarchive!.
Some 38% of webpages that existed in 2013 are not available today, and 8% of pages that existed in 2023 have disappeared. How fleeting is online content?
2
16
54
Earlier today a network from Bangladesh attempted to prepend 9x, but accidentally prepended "9" to their AS path. #typostream. The typo (a common one) caused the route to become RPKI-invalid and its propagation dipped. I covered the reduced propagation of RPKI-invalid routes in
4
16
55
Today is the first day since Sep-21, mobile internet in #Iran did not go down as part of an internet curfew. #IranProtests2022 . Previous mobile internet outages typically began around 12:30 UTC (4pm local) and ended around 20:30 UTC (midnight local).
2
25
51
Hang on. is BGP the three-napkin protocol, or the two-napkin protocol?. Exactly how many napkins were needed to propose BGP in 1989?? Maybe it is three sides of two napkins.
3
14
57
Rare IPv6-only BGP routing leak this morning. AS61573 leaked about 35k prefixes to AS37468 at 13:31 UTC today lasting about 10 minutes.
1
27
44
@gatech_ioda also shows a reduction in successful pings to Cuba (blue line) and complete drop-off in Google traffic (purple line). BGP routes unaffected. #Cuba .
2
23
53
Internet has been cut in the Republic of Congo hours before polling stations were set to open. Congo is holding its presidential election today. According to @kentikinc data, traffic stopped flowing into Congo at 23:04 UTC or just after midnight local. #KeepItOn
2
35
52
@anurag_bhatia USCYBERCOM: Mr President, we're getting BGP hijacked. PRESIDENT: By whom? The Chinese? Russians?. USCYBERCOM: Omaha Steaks, sir. PRESIDENT: Nuke'em. (Actual AS path: . 721 27064 367 575 306 346 22222)
3
19
48
UPDATE: The mysterious #BGP hijacks uncovered in my analysis of the recent Azure outage disappeared from the global routing table on Feb 4th. Still no explanation who was creating these routes with "4826 8075" appended to the end of their AS Paths.
1
15
51
Internet service returned in Cuba at 7:54 UTC (3:54am local) earlier today following a second internet blackout in as many days. This 2nd internet shutdown lasted almost 8 hours. #KeepitOn #CubaPaLaCalle
2
21
49
In addition to the DDoS attacks against Russian govt websites today, there were attempted DDoS attacks against the websites of major Russian banks as well. Sberbank and Alfabank were targeted in consecutive DDoS attacks earlier today. #UkraineRussiaConflict
2
23
49
So the outage in Cali was due to a fiber cut impacting ~7000 users of Movistar Colombia. No evidence of a government-directed internet shutdown as has been reported elsewhere. We observed a very slight decline in traffic volume to AS3816 in Cali, but overall it looks normal.
El Espectador explica lo que pasó en Cali, ojalá entre muchos podamos mantener bajo la lupa estos días la infraestructura, hay riesgos y tensiones, pero es importante recordar que siempre es normal que haya fallas.
4
33
52
I was curious of the origin of the assertion in this piece that Russia cut a subsea cable when it annexed Crimea in 2014. This biggest issue is that there was no subsea cable connecting Crimea at the time. Follow me as I try to find the origin of this story….
Russian naval exercises near Ireland are causing alarm. @jshermcyber explores what could lie beneath.
4
18
43
Myanmar using BGP to blackhole traffic to @Twitter, but routes are leaking out of country interfering with Twitter traffic in the region. Similar to the famous Youtube/Pakistan incident of Feb 2008.
2
48
50
Internet shutdowns, an undersea volcano, BGP hijacks on cryptocurrency, subsea cable outages, and RPKI made 2022 another eventful year on the Internet. Here's my end-of-year post reviewing the analysis @kentikinc published in the past 12 months. Enjoy!.
1
16
47
Let's talk about the "Houthis cut subsea cables near Yemen" story that is making the rounds. At least one submarine cable appears to have suffered a fault this weekend: SEACOM. As far as Houthi involvement, no reliable sources have emerged to confirm this part of the story.
2
16
48
Here is a look in @kentikinc at the BGP hijack of Amazon IP space recently mentioned on the @nanog list:. Attacker appended AS14618 (Amazon) to the AS path while announcing 44.235.216.0/24 (a more-specific of AS16509's 44.224.0.0/11).
1
19
44
Facebook and Instagram are beginning to come back following a 1-hour outage. #instagramdown #facebookdown.Cause of the outage is unknown at this time.
We're tracking a drop-off in traffic to Facebook and Instagram beginning at 15:20 UTC. #instagramdown
3
16
47
Based on @kentikinc performance measurements to Cuba, there was increased latency and packet loss for three distinct periods in late July - possible evidence of throttling of internet connectivity. #SOSCuba . cc: @norges14 @larrypress @arcamachoX
3
29
48
While the internet in Cuba has always been censored and inaccessible, shutdowns and outages are a relatively new phenomenon. #SOSCuba #KeepitOn . Here are some other recent outages:.
Internet outages in Cuba this evening. There appear to be three brief outages based on @kentikinc data. #KeepitOn . Disruptions also visible in @caida_ioda:.
2
26
47
AAE-1 submarine cable outage impacting Internet connectivity in numerous countries in East Africa, Middle East and South Asia. Pakistan, Somalia, Djibouti and Saudi Arabia are among the affected countries according @kentikinc data.
6
32
43
Internet service in Russian-occupied Kherson, Ukraine was disabled at 16:12 UTC (6:12pm local) on Saturday, 30 April. #UkraineRussiaWar . Khersontelecom service was restored ~24hrs later via Russian transit from nearby Crimea.
3
35
43
Large outages today in #Ukraine️. Ukrtelecom (AS6849) down nationally at 9:35 UTC (11:35am local) for ~40min. Triolan (AS13188) has been down nationally for over 12hrs due to reported cyber attack. Still almost entirely offline. Vizes from @gatech_ioda and @kentikinc:
4
35
44
Now seeing DDoS attacks against Russian govt websites in possible retaliation to DDoS attacks against Ukrainian websites. #UkraineRussiaConflict . Among the RU targets, which is hosted by AS8291 (Russia State Internet Network).
0
14
41
In today's post for @kentikinc, I team up with @JobSnijders of @fastly to review the latest RPKI ROV adoption stats in light of a major milestone:. For the first time in RPKI history, >50% of IPv4 routes in the global routing table are covered by ROAs.
1
17
44
.@TeleGeography reviews the Ukrainian fiber backbone in their latest blog post. "There’s no doubt that @SpaceXStarlink terminals could be useful, but satellite comms are a drop in the bucket compared to fiber networks".
1
22
44
In my latest post for @kentikinc, I retraced the steps of yesterday’s (mis)use of RPKI which triggered an hours-long outage of Spain’s second largest mobile operator. This may be the first incident where RPKI was used to inflict a denial of service.
4
24
45
Too soon to know how much this is contributing to the #OptusOutage but AS4804 normally announces many RPKI invalid routes such as 220.239.240.0/20 which were unaffected but are being filtered. Covering prefixes (ex 220.236.0.0/14) were withdrawn leaving IP space unreachable in.
Australia telecom Optus is experiencing a nationwide outage now in its 5th hour. Internet and mobile phone services currently down for millions of Australian customers and businesses. The number of BGP routes announced by AS4804 dropped from 271 to 122 as the outage began.
2
7
45
What’s next for the internet in Afghanistan? - my new blog post for @kentikinc examining the past decade of growth and future risks to the nation's internet now that the Taliban are in charge.
1
26
43
Submarine cables between Europe and Asia cross through the TransEgypt overland fiber crossing. Outages in the mesh network are rare but when there is one, many major cables can be affected degrading service for millions of people in dozens of countries.
In my latest blog post for @kentikinc, I look into last week's outage in Egypt that disrupted Europe-Asia connectivity. I discuss some history of the chokepoint and note some surprising impacts on the major cloud providers. #NetworkObservability.
3
23
40
Thanks for the public outcry. The blog posts have been restored:.
@tmu @anvilwalrusden @jimcowie @OracleCloud Update: @Oracle deleted all of the Renesys, Dyn Research, and Oracle Internet Intel blog posts. 😢. is all we have to use now.
5
7
43
Second internet blackout in #Cuba in as many days. #CubaPaLaCalle. According to @kentikinc data, traffic volume began to drop at 00:06 UTC (8:06pm local).
4
24
43
Interesting detail I found: UDP vs TCP during the Facebook outage yesterday. Normally UDP is delivering video, but during the outage it was carrying millions of DNS retries to unreachable Facebook domains.
New blog! See Kentik’s view of the #Facebook outage using: BGP monitoring, NetFlow analysis, synthetic monitoring, OTT service tracking and more. .#NetworkObservability.
3
21
43
A couple of hours ago, SpaceX/Starlink geoip was updated to include an IPv6 address range for Antarctica! 🛰️. 2406:2d40:4500::/42,AQ,AQ-AQ,Antarctica.
2
10
43