Skip to content

feat: GRPC Callback API migration for Non Inference #8062

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

indrajit96
Copy link
Contributor

@indrajit96 indrajit96 commented Mar 11, 2025

What does the PR do?

This PR migrates NON-Inference RPCs from Completion Queue based to Callback based

Checklist

  • PR title reflects the change and is of format <commit_type>: <Title>
  • Changes are described in the pull request.
  • Related issues are referenced.
  • Populated github labels field
  • Added test plan and verified test passes.
  • Verified that the PR passes existing CI.
  • Verified copyright is correct on all changed files.
  • Added succinct git squash message before merging ref.
  • All template sections are filled out.
  • Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

  • build
  • ci
  • docs
  • feat
  • fix
  • perf
  • refactor
  • revert
  • style
  • test

Related PRs:

triton-inference-server/common#132

Where should the reviewer start?

Implementation of virtual inference::GRPCInferenceServiceCallback::CallbackService*

Flow of the changes

Step 1 : Created New Service Classes
Step 2 : Modified HandlerBase Interface
Step 3 : Updated CommonHandler Class
Step 4 : Modified Server Class
Step 5 : Changed Service Registration
Step 6 : Implemented Callback Methods

Test plan:

New test added test_grpc_callbacks.sh.
Moving forward will test with existing GRPC tests

  • CI Pipeline ID:

Caveats:

PR cannot merge directly, as we CANNOT change the service name (will break backward compatibilty for all existing grpc clients)

Background

TEP here
https://docs.google.com/document/d/1LyxivxW7fCL0g8ZG2qT8DU2AV4l0XqYZYDkxRzYH9hs/edit?tab=t.0

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

@indrajit96 indrajit96 marked this pull request as ready for review March 11, 2025 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant