View All Products
Compute
Storage
Networking
Databases
Services
Developer Tools
Industries
Pricing
Community
Engage With Us
Community ▶ Questions ▶ [Newbie, Security] Need some help on dealing with attack
Log in to Ask a Question

[Newbie, Security] Need some help on dealing with attack

general

Hi, everybody!

UPDATE:

Azathoth suggested that the attack maybe Slowloris.

After running a Slowloris attack test, I found that the characteristics were not match. So, these attack remains a mystery for me.

Waiting for more suggestions, everybody. Plz help!

BTW, the attacks were stopped about 20 hours ago.

–------------------ original post --------------------------

My linode is attacked now, and I need some help on this. Any advise is appreciate! Thanks in advance!

After doing some investigation and google staff, I can give some surface descriptions on the attack. The attack began two days ago. It's not stop right now.

1. Basic information

My linode is running CentOS 6 and LAMP server, hosts a wordpress blog and UseBB forum with very low normal traffic.

ps aux information is at the end of this post.

Now, iptables allows only port 80 and ssh connection( not default 22 port). Related part reads like this:

-A INPUT -i lo -j ACCEPT 
# bad ip
-A INPUT -s 65.30.63.120/32 -j DROP 
-A INPUT -s 176.9.84.46/32 -j DROP 
-A INPUT -s 61.147.110.15/32 -j DROP 
-A INPUT -s 117.25.148.110/32 -j DROP 
-A INPUT -s 222.186.36.63/32 -j DROP 
-A INPUT -s 222.214.216.194/32 -j DROP 
-A INPUT -s 218.61.18.253/32 -j DROP 
-A INPUT -s 77.75.77.17/32 -j DROP 
-A INPUT -s 119.84.74.8/32 -j DROP 
-A INPUT -s 60.169.75.161/32 -j DROP 
-A INPUT -s 31.222.129.165/32 -j DROP 
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT 
-A INPUT -p tcp -m tcp --dport 80 --tcp-flags FIN,SYN,RST,ACK SYN -m connlimit --connlimit-above 20 --connlimit-mask 32 -j LOG --log-prefix "connlimit blocked: " --log-level 6 
-A INPUT -p tcp -m tcp --dport 80 --tcp-flags FIN,SYN,RST,ACK SYN -m connlimit --connlimit-above 20 --connlimit-mask 32 -j REJECT --reject-with tcp-reset 
-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -m limit --limit 50/min --limit-burst 200 -j ACCEPT 
-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j LOG --log-prefix "HTTP limit blocked: " --log-level 6 
-A INPUT -j DROP

2. Attack is in 2 types:

a. Some IPs keep sending request to my port 80.

If not block it with iptables, the netstat -anp shows like ( and they just hang there for hours)

tcp        0      0 106.187.50.90:80            174.122.6.252:62841         SYN_RECV    -
tcp        0      0 106.187.50.90:80            174.122.6.252:56394         SYN_RECV    -

and tcpdump -n shows like( but there are so many packages like this and could continue for hours)

`23:28:33.931729 IP 174.122.6.252.62841 > 106.187.50.90.http: Flags [s], seq 0, win 8192, length 0`

This type of attack could cause amount of incoming and outgoing traffic. After banning it in iptables, only incoming traffic remains.

**~~[b]~~b. Another type of attack is on port 443 (https)<e>[/b]</e>**

Since port 443 is not open in iptables, `~~[code]~~netstat -anpt<e>[/code]</e>` shows nothing about this type of attack. But `~~[code]~~tcpdump -n<e>[/code]</e>` reads like( but there are so many packages like this and could continue for hours):
`~~[code]~~23:21:18.552302 IP 221.120.194.182.acr-nema > 106.187.50.90.https: Flags [s], seq 0, win 8192, length 0
23:21:18.556223 IP 221.120.194.182.mit-dov > 106.187.50.90.https: Flags [s], seq 0, win 8192, length 0
23:21:18.556239 IP 221.120.194.182.mit-dov > 106.187.50.90.https: Flags [s], seq 0, win 8192, length 0
23:21:18.556247 IP 221.120.194.182.mit-dov > 106.187.50.90.https: Flags [s], seq 0, win 8192, length 0
23:21:18.559609 IP 221.120.194.182.sixxsconfig > 106.187.50.90.https: Flags [s], seq 0, win 8192, length 0
23:21:18.559627 IP 221.120.194.182.sixxsconfig > 106.187.50.90.https: Flags [s], seq 0, win 8192, length 0` 

This type of attack cause amount of incoming traffic but almost no outgoing traffic. And if ping is allowed in iptables and the ip is not banned, it could cause the website on vps responses very slow.

I've banned several IPs doing the attack. After the ip is banned, another ip came.

Here is a traffic graph related:

<url url="http://minus.com/lHaSBa3iY5Zph">~~[url]~~![](http://i.minus.com/jHaSBa3iY5Zph.png)~~[img]~~http://i.minus.com/jHaSBa3iY5Zph.png<e>[/img]</e><e>[/url]</e></url>

ps aux:
`~~[code]~~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.2   2928  1448 ?        Ss   May22   0:00 /sbin/init
root         2  0.0  0.0      0     0 ?        S    May22   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S    May22   0:01 [ksoftirqd/0]
root         4  0.0  0.0      0     0 ?        S    May22   0:11 [kworker/0:0]
root         5  0.0  0.0      0     0 ?        S    May22   0:00 [kworker/u:0]
root         6  0.0  0.0      0     0 ?        S    May22   0:00 [migration/0]
root         7  0.0  0.0      0     0 ?        S    May22   0:00 [migration/1]
root         8  0.0  0.0      0     0 ?        S    May22   0:00 [kworker/1:0]
root         9  0.0  0.0      0     0 ?        S    May22   0:00 [ksoftirqd/1]
root        10  0.0  0.0      0     0 ?        S    May22   0:00 [migration/2]
root        11  0.0  0.0      0     0 ?        S    May22   0:00 [kworker/2:0]
root        12  0.0  0.0      0     0 ?        S    May22   0:00 [ksoftirqd/2]
root        13  0.0  0.0      0     0 ?        S    May22   0:00 [migration/3]
root        14  0.0  0.0      0     0 ?        S    May22   0:00 [kworker/3:0]
root        15  0.0  0.0      0     0 ?        S    May22   0:00 [ksoftirqd/3]
root        16  0.0  0.0      0     0 ?        S<   May22   0:00 [cpuset]
root        17  0.0  0.0      0     0 ?        S<   May22   0:00 [khelper]
root        18  0.0  0.0      0     0 ?        S    May22   0:00 [kworker/u:1]
root        22  0.0  0.0      0     0 ?        S    May22   0:00 [xenwatch]
root        23  0.0  0.0      0     0 ?        S    May22   0:00 [xenbus]
root       149  0.0  0.0      0     0 ?        S    May22   0:00 [sync_supers]
root       151  0.0  0.0      0     0 ?        S    May22   0:00 [bdi-default]
root       153  0.0  0.0      0     0 ?        S<   May22   0:00 [kblockd]
root       163  0.0  0.0      0     0 ?        S<   May22   0:00 [md]
root       247  0.0  0.0      0     0 ?        S<   May22   0:00 [rpciod]
root       249  0.0  0.0      0     0 ?        S    May22   0:02 [kworker/0:1]
root       280  0.0  0.0      0     0 ?        S    May22   0:00 [kswapd0]
root       281  0.0  0.0      0     0 ?        SN   May22   0:00 [ksmd]
root       282  0.0  0.0      0     0 ?        S    May22   0:00 [fsnotify_mark]
root       286  0.0  0.0      0     0 ?        S    May22   0:00 [ecryptfs-kthrea]
root       288  0.0  0.0      0     0 ?        S<   May22   0:00 [nfsiod]
root       291  0.0  0.0      0     0 ?        S    May22   0:00 [jfsIO]
root       292  0.0  0.0      0     0 ?        S    May22   0:00 [jfsCommit]
root       293  0.0  0.0      0     0 ?        S    May22   0:00 [jfsCommit]
root       294  0.0  0.0      0     0 ?        S    May22   0:00 [jfsCommit]
root       295  0.0  0.0      0     0 ?        S    May22   0:00 [jfsCommit]
root       296  0.0  0.0      0     0 ?        S    May22   0:00 [jfsSync]
root       297  0.0  0.0      0     0 ?        S<   May22   0:00 [xfs_mru_cache]
root       298  0.0  0.0      0     0 ?        S<   May22   0:00 [xfslogd]
root       299  0.0  0.0      0     0 ?        S<   May22   0:00 [xfsdatad]
root       300  0.0  0.0      0     0 ?        S<   May22   0:00 [xfsconvertd]
root       301  0.0  0.0      0     0 ?        S<   May22   0:00 [glock_workqueue]
root       302  0.0  0.0      0     0 ?        S<   May22   0:00 [delete_workqueu]
root       303  0.0  0.0      0     0 ?        S<   May22   0:00 [gfs_recovery]
root       304  0.0  0.0      0     0 ?        S<   May22   0:00 [crypto]
root       866  0.0  0.0      0     0 ?        S    May22   0:00 [khvcd]
root       980  0.0  0.0      0     0 ?        S<   May22   0:00 [kpsmoused]
root       981  0.0  0.0      0     0 ?        S    May22   0:05 [kworker/1:1]
root       984  0.0  0.0      0     0 ?        S    May22   0:05 [kworker/2:1]
root      1009  0.0  0.0      0     0 ?        S    May22   0:01 [kjournald]
root      1034  0.0  0.0      0     0 ?        S    May22   0:04 [kworker/3:1]
root      1042  0.0  0.0      0     0 ?        S    May22   0:00 [kauditd]
root      1082  0.0  0.1   2660   700 ?        S~~[/code]~~`~~[/s][/s][/s][/s][/s][/s][/code][/s]~~

8 Replies

From what I can see from your output, your server does not appear to be under attack. Your charts seem perfectly fine, transferring 200 kb/sec is nothing large. What exactly would suggest that this is an attack as opposed to just standard traffic?

Thanks for your concern, iWizardPro!

I consider those attacks because the connections are not for browsing the websites on this vps(there are no httpd log entries related with those IPs) and they are lasting for several hours. There are not too much contents provided by the websites on this vps demand such a long time or traffic to transfer. At short, the traffic caused by these IPs are not normal.

And usually, visits on these websites should NOT generate too much incoming traffic.

Sounds like Slowloris attack to me. Then again I never experienced it myself because I never had Apache in front, always behind nginx which is not susceptible to this attack.

Edit: This is why monitoring like with Munin would come very handy, you could corelate various metrics to see better what's going on, like if my assumption is correct, all Apache processes being busy with decrease of bandwidth used.

@Azathoth:

Sounds like Slowloris attack to me. Then again I never experienced it myself because I never had Apache in front, always behind nginx which is not susceptible to this attack.

Edit: This is why monitoring like with Munin would come very handy, you could corelate various metrics to see better what's going on, like if my assumption is correct, all Apache processes being busy with decrease of bandwidth used.

Hi, Azathoth, thank you very much. Your suggestions mean a lot for me. It takes me a long time to understand.

I thought Slowloris may be it, but I need run some tests to confirm, since I can find any request log about those IPs in my Apache logs. I'll let you know right after finishing the tests.

After that, I'll give Nginx a try.

Thanks a lot!

Also worth noting that modern web browsers (particularly Chrome) tend to predictively open connections. In other words, if Chrome thinks I'm going to click on a particular link, it will open a connection just in case. While that may not be the case here, it does happen.

@hoopycat:

Also worth noting that modern web browsers (particularly Chrome) tend to predictively open connections. In other words, if Chrome thinks I'm going to click on a particular link, it will open a connection just in case. While that may not be the case here, it does happen.

Yes, they do. But I think it's kind of good thing, because it reduces our waiting time and does no or less harm to servers.

Obviously it's not the reason of my case. In my case, the connection seems not stop and could continue for hours.

I believe that setting up CloudFlare would also mitigate this problem, since CloudFlare sits between your web server and the internet. This is, after all, the original intended purpose of CloudFlare, before they realized it also sped things up ;)

That might let you block the attack without having to change web servers.

@Guspaz:

I believe that setting up CloudFlare would also mitigate this problem, since CloudFlare sits between your web server and the internet. This is, after all, the original intended purpose of CloudFlare, before they realized it also sped things up ;)

That might let you block the attack without having to change web servers.

Thank you, Guspaz!

CloudFlare is a great CDN service, but is not always working in my country.

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Log in to Reply
Community Code of Conduct