Support muxing gRPC broker connections over a single net.Conn #288
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fairclothjm
approved these changes
Oct 23, 2023
Contributor
fairclothjm
left a comment
fairclothjm
left a comment
There was a problem hiding this comment.
LGTM! I pulled this down and tested with a non-containerized plugin
swenson
approved these changes
Oct 24, 2023
Contributor
swenson
left a comment
swenson
left a comment
There was a problem hiding this comment.
LGTM, a few minor suggestions
| @@ -0,0 +1,48 @@ | |||
| package grpcmux | |||
Contributor
There was a problem hiding this comment.
Amusingly enough, this package doesn't seem to reference or use GRPC. :)
Contributor
Author
|
Thanks! |
Merged
wata727
added a commit
to terraform-linters/tflint-plugin-sdk
that referenced
this pull request
Feb 24, 2024
wata727
added a commit
to terraform-linters/tflint-plugin-sdk
that referenced
this pull request
Feb 24, 2024
* Bump github.com/hashicorp/go-plugin from 1.4.10 to 1.6.0 Bumps [github.com/hashicorp/go-plugin](https://github.com/hashicorp/go-plugin) from 1.4.10 to 1.6.0. - [Release notes](https://github.com/hashicorp/go-plugin/releases) - [Changelog](https://github.com/hashicorp/go-plugin/blob/main/CHANGELOG.md) - [Commits](hashicorp/go-plugin@v1.4.10...v1.6.0) --- updated-dependencies: - dependency-name: github.com/hashicorp/go-plugin dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Follow up of hashicorp/go-plugin#288 --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kazuma Watanabe <[email protected]>
hanzei
pushed a commit
to hanzei/go-plugin
that referenced
this pull request
Oct 17, 2024
…orp#288) * Support muxing gRPC broker connections over a single net.Conn, via ClientConfig.GRPCBrokerMultiplex * upgrade yamux, fix yamux config, and go mod tidy -compat=1.17 * Check for multiplexing support in protocol negotiation if enabled
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implements multiplexing for the gRPC broker so that additional gRPC connections are multiplexed onto the plugin's initial listener instead of creating a new Unix or TCP listener for each gRPC stream ID. The primary motivation is the simplification this gives containerised plugins which will only have to negotiate creating a single Unix socket that can be used by both sides, instead of one on each side - in particular this will greatly ease supporting rootless container engines running containers with a non-root user.
This will require both sides of the connection to update their go-plugin version, and as such is strictly opt-in via client config.
The implementation uses the same yamux.Session-based multiplexing that netrpc uses. However, given that we don't have control over when the gRPC library might redial a new connection, we introduce the concept of knocking (essentially a handshake, somewhat similar to TCP's SYN-ACK handshake) to synchronise the client and server on establishing a connection for a pre-agreed stream ID.
I briefly explored implementing gRPC's
grpc.ClientConnInterfaceinstead to give us full control over when a new connection is established, which would get rid of the need for knocking/pre-agreeing an ID, but I got put off by having to implementgrpc.CallOptions, and also having to change the library's API, becauseGRPCBroker.Dialcurrently returns a concrete*grpc.ClientConnstruct. However, I'm still open to exploring that further if people think those problems are preferable to the ones this PR currently solves.