Skip to content

feat: migrate to Connect-RPC; new wire path /jobs.v2.JobsService/#146

Closed
rustatian wants to merge 4 commits into
masterfrom
migrate-connect-rpc
Closed

feat: migrate to Connect-RPC; new wire path /jobs.v2.JobsService/#146
rustatian wants to merge 4 commits into
masterfrom
migrate-connect-rpc

Conversation

@rustatian

@rustatian rustatian commented May 10, 2026

Copy link
Copy Markdown
Member

Summary

Switches the jobs plugin's RPC layer from net/rpc + goridge codec to Connect-RPC. Plugin.RPC() now returns the generated jobsV2connect.NewJobsServiceHandler, which the rpc plugin (≥ v6.0.0-beta.4) mounts at /jobs.v2.JobsService/ on its HTTP/2 mux.

The 8 wire methods are reshaped from net/rpc's (req, *resp) error to the Connect handler interface ((ctx, *connect.Request[T]) (*connect.Response[U], error)):

Method Notes
Push now takes PushRequest { Job job = 1; } — the runtime "exactly one job" guard inside PushBatchRequest is gone (proto enforces)
PushBatch unchanged shape
Pause, Resume take Pipelines, return JobsHandlerResponse{}
List takes google.protobuf.Empty, returns Pipelines
Declare takes DeclareRequest
Destroy takes Pipelines, returns Pipelines (destroyed names)
StatGetStats renamed to avoid same-package collision with the Stat message; takes Empty, returns Stats

OTEL spans + per-method mutex preserved. Errors wrap via connect.NewError(connect.CodeInternal, ...) for proper gRPC status codes. Push-side trace-context extraction from job headers (rpcContextFromJob) preserved for cross-process propagation from PHP workers.

Test helpers swapped to an h2c Connect client (jobsV2connect.JobsServiceClient) talking to the rpc plugin's HTTP/2 cleartext listener at 127.0.0.1:6001. The new exported helpers.NewJobsClient(t, address) is available for sibling test packages.

Deps:

  • api-go v6.0.0-beta.4 → v6.0.0-beta.5 (new JobsService bindings)
  • goridge v4.0.0-beta.1 → v4.0.0-beta.2 (pkg/rpc removed)
  • rpc/v6 v6.0.0-beta.3 → v6.0.0-beta.4 (Connect-based RPCer interface)
  • + connectrpc.com/connect, + golang.org/x/net (h2c transport for tests)

Breaking changes

  • Wire protocol changes from goridge codec (raw TCP frames at 127.0.0.1:6001) to Connect/gRPC over HTTP/2 at /jobs.v2.JobsService/<Method>. Downstream Go consumers must use jobsV2connect.NewJobsServiceClient (or any gRPC client). The PHP goridge client migrates separately on its own timeline.
  • Push request type changed from PushBatchRequest (1-element) to PushRequest { Job }.
  • Stat RPC method renamed to GetStats.

Summary by CodeRabbit

  • Chores

    • Updated dependencies to include ConnectRPC framework
    • Modernized internal job service RPC communication layer from legacy protocol to industry-standard HTTP/2-based RPC with improved context propagation support
  • Refactor

    • Migrated job operations (Push, PushBatch, Pause, Resume, List, Declare, Destroy, GetStats) to new RPC handler architecture
    • Updated test suite and helper utilities to work with modernized RPC client

Review Change Stack

Switches the jobs plugin from net/rpc + goridge codec to Connect-RPC.
Plugin.RPC() now returns the generated jobsV2connect.NewJobsServiceHandler
mounted at /jobs.v2.JobsService/ on the rpc plugin's HTTP/2 mux.

The 8 wire methods are reshaped from (req, *resp) error to the Connect
handler interface:
  Push, PushBatch, Pause, Resume, List, Declare, Destroy, GetStats

Notable shape changes:
- Push now takes PushRequest { Job job = 1; }; the legacy runtime guard
  for "exactly one job" inside PushBatchRequest is gone (proto enforces).
- Stat -> GetStats (proto rename to avoid collision with the Stat message).
- List, GetStats take google.protobuf.Empty.
- Errors wrap via connect.NewError(connect.CodeInternal, ...) for proper
  gRPC status codes; OTEL spans + per-method mutex preserved.

Test helpers swapped to an h2c Connect client (jobsV2connect.JobsServiceClient)
talking to the rpc plugin's HTTP/2 cleartext listener at 127.0.0.1:6001.

Deps:
- api-go v6.0.0-beta.4 -> v6.0.0-beta.5 (new JobsService bindings)
- goridge v4.0.0-beta.1 -> v4.0.0-beta.2 (pkg/rpc removed)
- rpc/v6 v6.0.0-beta.3 -> v6.0.0-beta.4 (Connect-based RPCer interface)
- + connectrpc.com/connect, + golang.org/x/net (h2c client in tests)

Breaking changes:
- Wire protocol changes from goridge codec to Connect/gRPC. Downstream
  consumers must use jobsV2connect.NewJobsServiceClient (or any gRPC
  client) against the new path. The PHP goridge client migrates
  separately on its own timeline.
Copilot AI review requested due to automatic review settings May 10, 2026 12:08
@coderabbitai

coderabbitai Bot commented May 10, 2026

Copy link
Copy Markdown
Contributor

Warning

Rate limit exceeded

@rustatian has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 12 minutes and 50 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1ef9de61-dc7f-4ba9-9d97-1280df805c03

📥 Commits

Reviewing files that changed from the base of the PR and between e6c3be3 and 878a7e3.

📒 Files selected for processing (4)
  • plugin.go
  • rpc.go
  • rpc_test.go
  • tests/helpers/helpers.go
📝 Walkthrough

Walkthrough

This PR migrates the jobs service's RPC surface from goridge/net-rpc to Connect-RPC over HTTP/2. Dependencies are bumped, the RPC handler signature is updated to return an HTTP handler, all eight handler methods are converted to Connect signatures with context support, test helpers are rewritten to use a new Connect client factory, and all tests are updated to invoke the new client methods.

Changes

Connect-RPC Migration

Layer / File(s) Summary
Dependencies Update
go.mod, tests/go.mod
api-go/v6 bumped to beta.5, goridge/v4 to beta.2; connectrpc.com/connect added; golang.org/x/net and google.golang.org/protobuf promoted to direct test dependencies.
Handler Registration
plugin.go
RPC() method signature changed from returning any to (string, http.Handler); imports updated to include net/http and jobsV2connect for Connect handler construction.
Connect Handler Methods
rpc.go
All eight RPC methods (Push, PushBatch, Pause, Resume, List, Declare, Destroy, GetStats/Stat) converted from net-rpc signatures to Connect handlers accepting context.Context and *connect.Request[...], returning *connect.Response[...] or errors; imports updated for connectrpc.com/connect and emptypb.
Test Client Factory
tests/helpers/helpers.go
New NewJobsClient function creates an HTTP/2 cleartext Connect client with custom transport and dialer configuration.
Test Helper Functions
tests/helpers/helpers.go
ResumePipes, PausePipelines, PushToPipe variants, DestroyPipelines, and Stats rewritten to invoke corresponding Connect-RPC methods via the new client factory.
Test Integration
tests/jobs_general_test.go
Imports updated to remove net/rpc and goridge dependencies; declareMemoryPipe, consumeMemoryPipe, and TestTracePropagation refactored to use helpers.NewJobsClient and Connect-RPC calls.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • roadrunner-server/jobs#145: Parallel jobs RPC surface migration to v6 API with Connect-style handlers and updated response types.

Suggested labels

enhancement

Suggested reviewers

  • wolfy-j

Poem

🐰 The goridge paths once well-trodden and worn,
Now yield to Connect's bright HTTP dawn.
With handlers remade and handlers rebound,
The jobs service soars without earthly sound.
From net/rpc chains, at last we are free! 🚀

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 18.75% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ❓ Inconclusive The PR description provides a comprehensive summary of changes, detailed method signatures, breaking changes, and dependency updates. However, the template requires a 'Reason for This PR' section with an issue number or explanation, which is not explicitly provided, and the PR Checklist is not addressed. Add the 'Reason for This PR' section with an issue reference and complete the PR Checklist items to confirm all requirements have been met.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: migrating from goridge RPC to Connect-RPC with the new wire path /jobs.v2.JobsService/, which is the primary objective of this PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch migrate-connect-rpc

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates the jobs plugin RPC layer from legacy net/rpc+goridge to Connect-RPC by exposing the generated jobs.v2.JobsService handler (mounted at /jobs.v2.JobsService/) and updating tests/helpers to use an h2c Connect client against the rpc plugin’s HTTP/2 cleartext listener.

Changes:

  • Reworked the 8 wire methods into Connect handler signatures and updated error returns to connect.Error where applicable (including StatGetStats).
  • Updated the plugin’s exported RPC entrypoint to return (path, http.Handler) via the generated Connect service handler.
  • Migrated tests and test helpers from goridge/net/rpc to a Connect h2c client and adjusted request/response shapes (notably PushRequest).

Reviewed changes

Copilot reviewed 7 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
rpc.go Converts Jobs RPC methods to Connect handlers; updates request/response types and error mapping.
plugin.go Updates Plugin.RPC() to return the Connect service handler + mount path.
tests/helpers/helpers.go Introduces NewJobsClient (h2c Connect client) and migrates helper RPC calls to Connect.
tests/jobs_general_test.go Migrates integration tests to use the Connect client and updated proto method shapes.
go.mod / go.sum Bumps api-go and goridge versions to match the new Connect surface.
tests/go.mod / tests/go.sum Adds Connect + h2c deps and updates api-go/rpc versions for tests.
go.work.sum Updates workspace sums for newly introduced/updated modules.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread rpc.go Outdated
Comment thread rpc.go Outdated
Comment thread rpc.go
Comment thread tests/helpers/helpers.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
go.mod (1)

7-22: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Add missing direct dependency connectrpc.com/connect to root go.mod.

rpc.go imports "connectrpc.com/connect" directly (line 9), but the main module's go.mod does not include it in the require block. With Go 1.17+, all directly imported modules must be listed as direct dependencies. Running go build ./... or go mod tidy will fail with a missing module error. The version v1.19.2 is already correctly specified in tests/go.mod and should be added to the root go.mod.

🔧 Proposed fix
 require (
 	github.com/prometheus/client_golang v1.23.2
+	connectrpc.com/connect v1.19.2
 	github.com/roadrunner-server/api-go/v6 v6.0.0-beta.5
 	github.com/roadrunner-server/api-plugins/v6 v6.0.0-beta.2

(Run go mod tidy to place it in the conventional position and refresh go.sum.)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@go.mod` around lines 7 - 22, The root go.mod is missing the direct dependency
connectrpc.com/connect which rpc.go imports; add a require entry for
connectrpc.com/connect v1.19.2 to the root go.mod's require block (then run go
mod tidy to update go.sum and position the entry correctly) so builds no longer
fail when rpc.go imports "connectrpc.com/connect".
🧹 Nitpick comments (1)
rpc.go (1)

60-82: ⚡ Quick win

PushBatch lacks the input validation that Push performs.

Push rejects nil job and empty id with CodeInvalidArgument. PushBatch accepts whatever req.Msg.GetJobs() returns: an empty slice silently succeeds, and a nil element or empty-ID element will be passed straight into from(in[i]) and r.p.PushBatch. That's inconsistent and gives clients confusing internal errors instead of clear InvalidArguments.

🛡️ Proposed validation
 	in := req.Msg.GetJobs()
+	if len(in) == 0 {
+		return nil, connect.NewError(connect.CodeInvalidArgument, errors.E(op, errors.Str("jobs batch is empty")))
+	}
+	for i, j := range in {
+		if j == nil {
+			return nil, connect.NewError(connect.CodeInvalidArgument, errors.E(op, errors.Errorf("job at index %d is nil", i)))
+		}
+		if j.GetId() == "" {
+			return nil, connect.NewError(connect.CodeInvalidArgument, errors.E(op, errors.Errorf("empty ID at index %d", i)))
+		}
+	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rpc.go` around lines 60 - 82, PushBatch currently skips the input validation
that Push performs; update rpc.PushBatch to validate req.Msg.GetJobs()
similarly: if the slice is empty return a
connect.NewError(connect.CodeInvalidArgument, errors.E(op, "empty jobs")), and
for each element ensure it is non-nil and its GetId() is non-empty (return
CodeInvalidArgument with errors.E(op, ...) on the first invalid item); only
after validation call from(in[i]) and r.p.PushBatch(spanCtx, batch) as before so
clients get clear InvalidArgument errors instead of internal errors from from or
r.p.PushBatch.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rpc.go`:
- Around line 33-82: Push and PushBatch drop the incoming context (they use
rpcContextFromJob/rpcContextFromJobs which re-root tracing onto background), so
change Push and PushBatch to pass the inbound ctx into
rpcContextFromJob/rpcContextFromJobs (so the call to r.p.Push and r.p.PushBatch
inherit cancellation/deadline), update the helper signatures rpcContextFromJob,
rpcContextFromJobs, and rpcContextFromHeaders to accept a context parameter, and
update the five test calls in rpc_test.go to pass context.Background() or the
test context to those helpers so tests compile.

---

Outside diff comments:
In `@go.mod`:
- Around line 7-22: The root go.mod is missing the direct dependency
connectrpc.com/connect which rpc.go imports; add a require entry for
connectrpc.com/connect v1.19.2 to the root go.mod's require block (then run go
mod tidy to update go.sum and position the entry correctly) so builds no longer
fail when rpc.go imports "connectrpc.com/connect".

---

Nitpick comments:
In `@rpc.go`:
- Around line 60-82: PushBatch currently skips the input validation that Push
performs; update rpc.PushBatch to validate req.Msg.GetJobs() similarly: if the
slice is empty return a connect.NewError(connect.CodeInvalidArgument,
errors.E(op, "empty jobs")), and for each element ensure it is non-nil and its
GetId() is non-empty (return CodeInvalidArgument with errors.E(op, ...) on the
first invalid item); only after validation call from(in[i]) and
r.p.PushBatch(spanCtx, batch) as before so clients get clear InvalidArgument
errors instead of internal errors from from or r.p.PushBatch.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: de060013-3fc0-44a4-8fc0-3432291e4a5b

📥 Commits

Reviewing files that changed from the base of the PR and between 6e0e046 and e6c3be3.

⛔ Files ignored due to path filters (3)
  • go.sum is excluded by !**/*.sum
  • go.work.sum is excluded by !**/*.sum
  • tests/go.sum is excluded by !**/*.sum
📒 Files selected for processing (6)
  • go.mod
  • plugin.go
  • rpc.go
  • tests/go.mod
  • tests/helpers/helpers.go
  • tests/jobs_general_test.go

Comment thread rpc.go Outdated
rustatian added 2 commits May 10, 2026 14:24
- Push/PushBatch: thread inbound handler ctx through rpcContextFromJob/
  rpcContextFromJobs/rpcContextFromHeaders so client cancellation and
  deadlines propagate. The job-header trace context still overrides the
  span parent (PHP-side propagation preserved); only the propagator's
  parent argument changes.
- Pause/Resume: wrap errors with errors.E(op, err) for consistency
  with other methods (rpc_pause, rpc_resume ops).
- Tests (helpers): t.Cleanup(httpc.CloseIdleConnections) on each
  NewJobsClient to avoid leaking idle HTTP/2 connections across tests.
- rpc_test.go: pass t.Context() to the 5 helper call sites for the new
  signature.
- Destroy: wrap errg.Wait() error with errors.E(op, err) for parity
  with the other 7 methods.
- Push: add unit tests covering nil-job and empty-ID branches (the
  proto-shape replacement for the legacy len(req.Batch) != 1 guard).
- Trim WHAT-restating docstrings on Plugin.RPC, rpc.from, rpc.Push,
  rpc.Declare, helpers.NewJobsClient. Tighten the rpcContextFromJobs
  doc to the single non-obvious WHY (parent-ctx layering).
@rustatian

Copy link
Copy Markdown
Member Author

Superseded by #147 — content merged into the priority-fix branch which is now the consolidated migration PR.

@rustatian rustatian closed this May 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants