Tokyo to London migration solved Linode connectivity issue
Precise symptoms of the problem: * These two users were able to access all websites except mine.
They were unable to ping my domain name or my Linode's IP address.
They were unable to ping or traceroute tokyo10.linode.com. But they were able to ping or traceroute london356.linode.com.
My domain name was correctly resolving to my Linode's IP address on their systems. Confirmed this with nslookup, ping, etc.
Both these users were in the US.
Both these users were unable to connect to or access my Linode only from a particular network. For example, one was able to access my website from her home network but not from her university network using the same laptop. The other was able to access my website from his office network but not from his home network using the same laptop.
All other users except these two users were able to access my website.
Trace routes were giving up with lots of time-outs after some hops. Trace routes for one friend showed *.verizon-gni.net and *.qwest.net routers. Trace routes for another friend showed nothing but university routers.
I installed Debian on my Linode. It had the default configuration. I didn't setup iptables or anything like it which could have caused the issue. I don't exactly know what caused the problem. It would have been nice to have them run WinMTR to know exactly which router was causing the issue during trace route but I wanted to resolve the issue as soon as possible without burdening them with troubleshooting work.
The only reason that seemed likely to me was that some router in between was transmitting packets to a network segment that were larger than the segment's MTU. I couldn't confirm this hypothesis.
However, assuming this hypothesis to be true, I decided to migrate my Linode from Tokyo facility to London facility and this fixed the issue as soon as the migration was complete.
Update: Had a chat about it on IRC. Linode staff informed me that they have seen this issue earlier with Tokyo IP ranges. They said that Qwest/Verizon are doing some kind of source IP filtering that causes the packets coming from Tokyo to Qwest/Verizon to be dropped.
3 Replies
Tracing route to tokyo10.linode.com [106.187.33.21]
over a maximum of 50 hops:
1 2 ms 2 ms 1 ms 10.140.112.1
2 4 ms 3 ms 3 ms 172.19.252.41
3 3 ms 3 ms 3 ms 172.19.252.30
4 17 ms 3 ms 3 ms wireless-gw2.qwest.asu.edu [172.19.252.50]
5 6 ms 6 ms 4 ms 172.19.252.45
6 7 ms 5 ms 5 ms sparky-jr-core2.dco.asu.edu [172.30.253.2]
7 * * * Request timed out.
…
49 No resources.
Trace complete.
````
Tracing route to london356.linode.com [178.79.151.180]
over a maximum of 50 hops:
1 1 ms 1 ms 1 ms 10.140.112.1
2 5 ms 3 ms 3 ms 172.19.252.41
3 5 ms 3 ms 4 ms 172.19.252.30
4 5 ms 3 ms 3 ms wireless-gw2.qwest.asu.edu [172.19.252.50]
5 5 ms 4 ms 4 ms 172.19.252.45
6 7 ms 5 ms 6 ms sparky-jr-core2.dco.asu.edu [172.30.253.2]
7 7 ms 6 ms 5 ms 172.30.101.2
8 8 ms 6 ms 6 ms 206.206.223.58
9 18 ms 18 ms 16 ms vlan143.car1.Phoenix1.Level3.net [4.53.104.105]
10 18 ms 17 ms 16 ms ae-2-5.bar1.Phoenix1.Level3.net [4.69.148.118]
11 39 ms 41 ms 39 ms ae-8-8.ebr1.Dallas1.Level3.net [4.69.133.30]
12 41 ms 39 ms 39 ms ae-61-61.csw1.Dallas1.Level3.net [4.69.151.125]
13 42 ms 40 ms 40 ms ae-63-63.ebr3.Dallas1.Level3.net [4.69.151.134]
14 61 ms 59 ms 59 ms ae-7-7.ebr3.Atlanta2.Level3.net [4.69.134.22]
15 60 ms 58 ms 58 ms ae-63-63.ebr1.Atlanta2.Level3.net [4.69.148.242]
16 74 ms 72 ms 72 ms ae-6-6.ebr1.Washington12.Level3.net [4.69.148.106]
17 75 ms 72 ms 72 ms ae-1-100.ebr2.Washington12.Level3.net [4.69.143.214]
18 81 ms 81 ms 81 ms 4.69.148.49
19 191 ms 208 ms 208 ms ae-42-42.ebr2.London1.Level3.net [4.69.137.69]
20 151 ms 151 ms 152 ms ae-59-224.csw2.London1.Level3.net [4.69.153.142]
21 * 219 ms 155 ms ae-2-52.edge3.London1.Level3.net [4.69.139.105]
22 245 ms 313 ms 207 ms Telecity.edge3.lon1.l3.net [195.50.113.2]
23 245 ms 207 ms 149 ms te4-1-dist65-01.lon10.telecity.net [217.20.44.218]
24 169 ms 181 ms 208 ms 212.111.33.234
25 234 ms 150 ms 161 ms london356.linode.com [178.79.151.180]
Trace complete.
This is a fairly common problem. The good news is that there are no more unallocated IPv4 /8 blocks, so it should become less of an issue over the next few years.