lots of connections on TIME_WAIT state

[Igor Sysoev]

On Wed, Apr 08, 2009 at 10:47:16AM +0300, Artis Caune wrote:

> 2009/4/7 Deepan Chakravarthy <codeshepherd at gmail.com>:
> > Hi,
> >   I am using nginx with fast-cgi .  When I run
> > $netstat -np | grep 127.0.0.1:9000
> > I find lot of connections in TIME_WAIT state. Is this because of high
> > keepalive_timeout value ?   When lot of people use (5 requests per second)
> >  nginx takes more  time to respond. System load goes more than 10 during
> > peak hours.
> 
> This is because of how TCP works.
> 
> 
> > debian:~# netstat -np | grep 127.0.0.1:9000
> > tcp        0      0 127.0.0.1:9000          127.0.0.1:45603
> > TIME_WAIT  -
> > tcp        0      0 127.0.0.1:9000          127.0.0.1:45601
> > TIME_WAIT  -
> 
> 
> If you were on FreeBSD, you could disable TIME_WAIT on loopback
> completely by setting:
> 
>     sysctl net.inet.tcp.nolocaltimewait=1

Due to the incorrect implementation this remedy is worse than the disease.
The net.inet.tcp.nolocaltimewait relys on unlimited RST delivery, therefore
if there are too many RSTs, they will be limited by net.inet.icmp.icmplim
and you will have a lot of sockets in the LAST_ACK state on server side
instead of lot of sockets in the TIME_WAIT on client side.


-- 
Igor Sysoev
http://sysoev.ru/en/