putty ssh timed out
I've met a problem I cannot ssh to my linode server any longer, but don't know why.
I have two linode VPS (both on Tokyo site), A and B. I was able to ssh to them with putty with no problems.
Today, I found that I cannot ssh to A any longer. But I can still ssh to B with no problem. And I can visit http ports on A with no problem; I can ssh to A from other terminals (such as my mobile phone, or ssh A through ssh B) without any problem.
As far as I know, when I ssh to A from my computer, it stuck on SYN_RECV on the VPS.
Since I can ssh to A from other terminals, I think the problem should be on my computer (Windows 7 x64). But I can ssh to VPS B from my computer with no problem.
I tried to start a VMWare on my computer, and use putty inside it, it could ssh to B as well, but failed on A, too.
I guess maybe there is something wrong with my firewall, but don't know the details.
Do you have any ideas where should I check?
PS: My computer recently experienced some blue screen due to a broken memory stick. And the disk check fixed some errors after that… So I guess anything could be possible if the system is damaged due to the crash…
18 Replies
@Guspaz:
Try using LISH?
http://library.linode.com/troubleshooti … node-shell">http://library.linode.com/troubleshooting/using-lish-the-linode-shell
Hi Guspaz,
I tried to connect with LISH, and successed. Though after I connected with the "LISH password", it connected and displayed " nf_conntrack: automatic helper assignment is deprecated and it will be removed soon. Use the iptables CT target to attach helpers instead."
And I input the root account & password again. It connects.
It is weird, my computer could ssh to other servers, and my server could accept ssh from other clients, but when I ssh my server with my computer, it fails. I tried other terminal software, and get the same result. While connecting to other ports on my server is OK.
As I remember, one day before, I could still connect to my server with ssh. And I didn't do anything afterwards (even didn't shut down my computer nor reboot, and no crash).
Do you have any idea why? Thanks!
@chesty:
do you have fail2ban installed on the servers?
No… I believe not…
I didn't do that, and there is no fail2ban entry in man nor in /etc directory.
on linode run tcpdump -nvi eth0 port 22 (run that from lish with no ssh sessions connected, otherwise you'll get a storm of junk and nothing useful)
then try and ssh in and post the output.
secondly, turn on debugging on putty, and on linode (using lish) stop sshd and manually run it with sshd -ddd
then try and ssh in and post the output.
@chesty:
well, there're two things you can try that I can think of.
on linode run tcpdump -nvi eth0 port 22 (run that from lish with no ssh sessions connected, otherwise you'll get a storm of junk and nothing useful)
then try and ssh in and post the output.
secondly, turn on debugging on putty, and on linode (using lish) stop sshd and manually run it with sshd -ddd
then try and ssh in and post the output.
Hello chesty,
Here is my output:
device eth0 entered promiscuous mode
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
08:36:32.306416 IP (tos 0x0, ttl 116, id 30348, offset 0, flags [DF], proto TCP (6), length 52)
[Client IP].62578 > [Server IP].22: Flags ~~, cksum 0x1d29 (correct), seq 3760837357, win 8192, options [mss 1360,nop,wscale 2,nop,nop,sackOK], length 0
08:36:32.306495 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
[Server IP].22 > [Client IP].62578: Flags [S.], cksum 0x8ad7 (incorrect -> 0x18b7), seq 2733000714, ack 3760837358, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
08:36:33.309001 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
[Server IP].22 > [Client IP].62578: Flags [S.], cksum 0x8ad7 (incorrect -> 0x18b7), seq 2733000714, ack 3760837358, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
08:36:35.309050 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
[Server IP].22 > [Client IP].62578: Flags [S.], cksum 0x8ad7 (incorrect -> 0x18b7), seq 2733000714, ack 3760837358, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
08:36:35.310083 IP (tos 0x0, ttl 116, id 31098, offset 0, flags [DF], proto TCP (6), length 52)
[Client IP].62578 > [Server IP].22: Flags ~~, cksum 0x1d29 (correct), seq 3760837357, win 8192, options [mss 1360,nop,wscale 2,nop,nop,sackOK], length 0
08:36:35.310111 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
[Server IP].22 > [Client IP].62578: Flags [S.], cksum 0x8ad7 (incorrect -> 0x18b7), seq 2733000714, ack 3760837358, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
08:36:39.308973 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
[Server IP].22 > [Client IP].62578: Flags [S.], cksum 0x8ad7 (incorrect -> 0x18b7), seq 2733000714, ack 3760837358, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
08:36:41.306117 IP (tos 0x0, ttl 116, id 32517, offset 0, flags [DF], proto TCP (6), length 48)
[Client IP].62578 > [Server IP].22: Flags ~~, cksum 0x3132 (correct), seq 3760837357, win 8192, options [mss 1360,nop,nop,sackOK], length 0
08:36:41.306172 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
[Server IP].22 > [Client IP].62578: Flags [S.], cksum 0x8ad7 (incorrect -> 0x18b7), seq 2733000714, ack 3760837358, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
08:36:47.309012 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
[Server IP].22 > [Client IP].62578: Flags [S.], cksum 0x8ad7 (incorrect -> 0x18b7), seq 2733000714, ack 3760837358, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
08:37:03.508975 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
[Server IP].22 > [Client IP].62578: Flags [S.], cksum 0x8ad7 (incorrect -> 0x18b7), seq 2733000714, ack 3760837358, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
I cannot understand the meaning, could you help me to check it? Thank you very much!~~
what I'm interpreting from the trace is
client syn
server syn ack
server syn ack
server syn ack
client syn
server syn ack
server syn ack
client syn
server syn ack
server syn ack
server syn ack
the client isn't seeing the servers syn ack, so isn't acking the final leg of the three way hand shake.
paste the output from iptables-save
also do you have a firewall on the desktop? check that, maybe turn it off for a test ssh
there's no quick way to debug this
@chesty:
it looks like the path from the server to the client is broken.
what I'm interpreting from the trace is
client syn
server syn ack
server syn ack
server syn ack
client syn
server syn ack
server syn ack
client syn
server syn ack
server syn ack
server syn ack
the client isn't seeing the servers syn ack, so isn't acking the final leg of the three way hand shake.
paste the output from iptables-save
also do you have a firewall on the desktop? check that, maybe turn it off for a test ssh
there's no quick way to debug this
Yeah, the path from the client to the server seems broken… When I run netstat -an on the server while ssh to it, I can see the the link between the client and the server stuck on SYN_RECV.
I have the default Windows 7 firewall on, but didn't see any config that could block a specific IP. (As I mentioned in the OP, I can ssh to other servers or connect to other ports on this server, all without any problems, from the same computer).
And I tried to turn the firewall off, the problem persists.
Here is the output of iptables-save:
# Generated by iptables-save v1.4.8 on Sat Mar 2 10:01:55 2013
*security
:INPUT ACCEPT [86466:9198024]
:FORWARD ACCEPT [3252:1471232]
:OUTPUT ACCEPT [97237:16378856]
COMMIT
# Completed on Sat Mar 2 10:01:55 2013
# Generated by iptables-save v1.4.8 on Sat Mar 2 10:01:55 2013
*raw
:PREROUTING ACCEPT [89727:10672056]
:OUTPUT ACCEPT [97237:16378856]
COMMIT
# Completed on Sat Mar 2 10:01:55 2013
# Generated by iptables-save v1.4.8 on Sat Mar 2 10:01:55 2013
*nat
:PREROUTING ACCEPT [5859:349105]
:INPUT ACCEPT [5775:343964]
:OUTPUT ACCEPT [12750:1054079]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
# Completed on Sat Mar 2 10:01:55 2013
# Generated by iptables-save v1.4.8 on Sat Mar 2 10:01:55 2013
*mangle
:PREROUTING ACCEPT [89727:10672056]
:INPUT ACCEPT [86466:9198024]
:FORWARD ACCEPT [3252:1471232]
:OUTPUT ACCEPT [97237:16378856]
:POSTROUTING ACCEPT [100489:17850088]
COMMIT
# Completed on Sat Mar 2 10:01:55 2013
# Generated by iptables-save v1.4.8 on Sat Mar 2 10:01:55 2013
*filter
:INPUT ACCEPT [86466:9198024]
:FORWARD ACCEPT [3252:1471232]
:OUTPUT ACCEPT [97237:16378856]
COMMIT
# Completed on Sat Mar 2 10:01:55 2013
I have pptpd installed on the server, and have run "iptables –table nat --append POSTROUTING --out-interface eth0 --jump MASQUERADE"
paste
ip ad
ip ro
maybe try as a test
iptables –table nat --delete POSTROUTING --out-interface eth0 --jump MASQUERADE
two things, pptp isn't very secure, if you can, use openvpn or ipsec.
you should be more specific with your rule, specify a source ip range, and if you can, an in interface (I can't remember if you can on the postrouting nat chain)
Windows is a hacked together pile of mess on the inside and does stupid things like this every so often.
Since I cannot connect the server with 2 computers (both Windows 7), I think my original thought that the problem is on the client side should be wrong. These 2 computers could ssh to the server very well several days ago.
The interesting thing is I can still ssh to the server with my mobile phone (Android, via ConnnectBot app); and I can still visit the other ports of the server (I tried ftp, http, vpn) from all the clients mentioned above.
something else to try, add a line to /etc/ssh/sshd_config
Port 222
and restart sshd and see if you can connect to 222
You're allowed to have multiple Port lines, so you can leave the original in place
@chesty:
are the two computers on the same LAN or behind the same firewall?
something else to try, add a line to /etc/ssh/sshd_config
Port 222
and restart sshd and see if you can connect to 222
You're allowed to have multiple Port lines, so you can leave the original in place
Yeah, changing the port helps.
But I'm still curious about why connecting to the default port fails.
It seems not the problem with my client, nor the problem with the server. Can I conclude that the problem is with my ISP?
@chesty:
You didn't answer the question about whether the two PCs are on the same LAN behind the same firewall, I would guess yes, and I would guess it's the firewalls problem. rebooting it might help.
Hello chesty,
No, the two PCs are not in the same LAN, and are from different ISPs. (One is my PC at home, one is my working computer in the office)
I remember I could use my mobile phone to ssh to the server within the WIFI at home. But as I tried today in the office, the mobile phone could not connect within the office's WIFI, either. While the mobile phone could still ssh to the server with mobile data connection (A different ISP with the office's, also different with my home's)
While listing these out, I think it is more likely the problem is on the ISP side…(Especially with the behaviour of my mobile phone)
I'm a resident in China, I guess there may be some "upgrading" of the China's GFW…
and now this forum has the word china in it, it will likely be blocked soon too.
@chesty:
oh, you never mentioned you were in china, china blocks ssh and vpns all the time, it's likely the new port will stop working soon.
and now this forum has the word china in it, it will likely be blocked soon too.
Well, I hate to say this, but you may be right…
Thank you very much, chesty, to help me tracing this…
I think I need to find some alternative solutions to keep myself unblocked…