netstack: make tcpip.Address hold a []byte

[copybara-service]

netstack: make tcpip.Address hold a []byte

tcp_benchmark throughput increase 2-3%, but allocations go down (25% in the
download benchmark, only 2% in the upload path).

[kdrag0n]

@kevinGC I noticed that #8868 was closed in favor of this. Did you already test netip.Addr and choose []byte instead for performance, or is this just the first pass?

[kdrag0n]

Sorry, didn't notice that it was merged.

[kevinGC]

@kdrag0n netip.Addr would've been nice, but unfortunately its AsSlice() function always allocates. That wasn't acceptable for performance, so we decided to stick with []byte.

Anecdotally, I don't think most users care that it allocates. We're in a different use bucket because (a) we are extremely performance sensitive in our hot path, and (b) we need access to the underlying bytes a lot, whereas most users just pass addresses around and use them to net.Dial and such.

[kdrag0n]

@kevinGC Thanks for the response. I actually tried migrating the implementation to netip.Addr after the opaque type changes were merged and came to a similar conclusion.

Since quite a few tcpip.Address.AsSlice() usages are of the form copy(dst, addr.AsSlice()), I was thinking that an allocation-free idiom could look like this:

func (a Address) CopyTo(dst []byte) {
    if a.Len() == 4 {
         copy(dst, a.As4()[:])
    } else {
         copy(dst, a.As16()[:])
    }
}

But that's probably not worth it, and I haven't tested how it performs.

[kevinGC]

Just curious -- was there something in particular that motivated looking into this? I'm always curious how people are using netstack outside of gVisor.

[kdrag0n]

I'm using gVisor netstack for high-performance userspace VM networking in https://orbstack.dev. Currently pushing up to 48 Gbps (receive side) and 30 Gbps (sender side), which is great, but I'd like to reduce CPU usage and latency (if possible) for some potential use cases in the future.