Does fasthttp.Client take care of half-open/half-closed connections?

[moredure]

No description provided.

[erikdubbelboer]

What do you mean with a half-closed connection?

[moredure]

lsof show me many sock files with can't identify protocol status sometimes near 200k entries with requests per seconds rate near 4k to many different hosts (web crawling)

[moredure]

Probably relates with #630

[erikdubbelboer]

Sounds like something in the TCP layer unrelated to fasthttp. Could it be that these are sockets in the TIME_WAIT state?

[moredure]

I am using SO_REUSEPORT, thereis only near 30k connections in time wait, 1600+ ESTABLISHED

root@crawler:~# lsof -n | grep "can't" | wc -l
175825
root@crawler:~# netstat -nt  | wc -l
31598

[moredure]

Probably I am trapped into golang/go#18588

[moredure]

I found the root cause. Fasthttp is nice lib. My fault.

[erikdubbelboer]

@mikefaraponov would you mind sharing what the cause was?

[moredure]

The reason in spawning unknown can't identify protocol messages in lsof -n was SO_REUSEPORT behavior for client side connections, it's absolutely normal. The root cause of degraded performance was another dnscache library, even not a library itself but my missing of knowledge how it works internally