stateful session persistence

[wbpcode]

Http session persistence is achieved through hash-based (consistent hash algorithm) load balancing. When the state of the backend servers change (new server is added, existing server is deleted, server state is updated, etc.) it breaks the session persistence result.

Is it possible to consider adding a cookie-based stateful session persistence? When this feature is turned on, the host id or something is added to the cookie. When new requests arrive, the corresponding host is resolved from the cookie and passed to the LoadBalancer via Upstream::LoadBalancerContext. The LoadBalancer can return this host first, if possible.

[phlax]

cc @alyssawilk

[wbpcode]

• The Upstream::LoadBalancerContext may needs to extend an interface similar to the one shown below:

struct PrimaryHost {
   std::string host_id_or_hash_or_address_or_something; // something can uniquely identify a host 
   size_t expected_status; // expected health status of host
};

PrimaryHost primaryHostShouldSelected()

• The Http Router need add some new API to enable or disable this function and some code to handle cookie.

• The LoadBlancer can optionally select the host based on the above interface. Of course, when this feature is enabled, we need an additional hash table, which will bring some additional memory overhead.

In general, it is not complicated and does not require much new code.

I am not sure but this may be can close #7218.

[wbpcode]

Add a doc to describe more detail and some possible solutions.
https://docs.google.com/document/d/1IU4b76AgOXijNa4sew1gfBfSiOMbZNiEt5Dhis8QpYg/edit?usp=sharing

[wbpcode]

@alyssawilk @mattklein123 Is this plan feasible for #7218

[alyssawilk]

for cookies, can you comment where this doesn't meet your needs?

https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#envoy-v3-api-msg-config-route-v3-routeaction-hashpolicy-cookie

[wbpcode]

@alyssawilk The most important reason is that no matter what hash policy is used, the existing session persistence is stateless. In the current solution, the cookie generated by Envoy does not directly save the upstream host information. A suitable host is obtained according to the algorithm based on the value calculated by the hash.
When the state of the back-end servers changes (New servers are added, some servers’ state is updated, etc.), the hash ring will be rebuilt, the session persistence is broken.

The requirement in #7218 is actually a good example. When host x is marked as draining or degraded or unhealthy, for the existing session requests, we hope that they will still be routed to host x. If we have the ability to maintain stateful session persistence, and for each session, save the relevant host information, then we can do this.

[alyssawilk]

so at some point if a host is draining it's going to go away. It sounds like what you really want here is a period where no new "sticky" sessions are assigned, but sessions already sticky to an upstream will be routed there?

You say on the doc that maglev might meet your needs if only you could route while draining, but I don't think either of your design options include routing to a host while it's draining, which I don't think your cookie solution would necessarily do either.

That said, encoding ip:port of backend in cookies and using them for direct routing is a pretty common method of LB, and if you want to implement that I think it would be a useful addition to Envoy.

[wbpcode]

so at some point if a host is draining it's going to go away. It sounds like what you really want here is a period where no new "sticky" sessions are assigned, but sessions already sticky to an upstream will be routed there?

Yes.

You say on the doc that maglev might meet your needs if only you could route while draining, but I don't think either of your design options include routing to a host while it's draining, which I don't think your cookie solution would necessarily do either.

The draining host is special. Because it has been removed from the hosts set of the cluster, we may need to think more about it. But unhealthy host and degraded host can use the cookie scheme to maintain the existing sticky session.

That said, encoding ip:port of backend in cookies and using them for direct routing is a pretty common method of LB, and if you want to implement that I think it would be a useful addition to Envoy.

I am happy to implement it.