Port Exhaustion
we have a use case of high burst of traffic on some day when an event happens, There are about 80k users using multiple apis. ( lets say they connect 1 API every second).
When we tested, the problem is that all the ports are getting occupied by our server and once disconnect happens from client side, the ports are in TIME_WAIT status.
Anyone came across this problem?
Option 1 - Can we have multiple Ethernet ports?
Option 2 - I can use reuse sockets by kernel settings, but its not recommended by many experts and if traffic increases then it may be bad for other connections, or incomplete replies etc.
Note that we may not need horizontal scaling because our app is very light weight, it usually replies in less than 100-200ms, so I am not a fan of using multiple servers. I am only being restricted by the ports being exhausted. What is recommended in this case?
2 Replies
"the ports are in TIME_WAIT status."
conntrack in the Linux kernel typically only remembers a connection for around 130 maybe 150 seconds after traffic stops for it. If the port is not able to be used after that then it's likely the API software still holding on to the port. If it's not that I would check to see if you or someone else managing the server have changed any kernel: conntrack, tcp, or socket settings.
This may be useful:
https://www.iptables.org/projects/conntrack-tools/index.html
I can't speak to it's quality or efficacy personally…I just ran across it during a search for something else.
-- sw