refactor(backends): standardize status reporting across all backends

[doringeman]

Prettify docker model status.

⚠️ This new CLI will show all statuses from older backends as "Error".

MODEL_RUNNER_PORT=8080 make run

E.g.,

• macOS

$ MODEL_RUNNER_HOST=http://localhost:8080 docker model status
Docker Model Runner is running

BACKEND    STATUS         DETAILS
llama.cpp  Running        llama.cpp latest-metal (sha256:13574106bd745302e63cf7b1978483eb47f32c128c40b1271364c937d850fdcb) c55bce4
vllm       Running        vllm-metal v0.1.0-20260126-121650
diffusers  Not Installed  only supported on Linux
mlx        Not Installed  package not installed
sglang     Not Installed  only supported on Linux

• Linux

$ docker model status
Docker Model Runner is running

BACKEND    STATUS         DETAILS
llama.cpp  Running        llama.cpp c55bce4
vllm       Running        vllm 0.12.0
diffusers  Not Installed  package not installed
mlx        Not Installed  only supported on Apple Silicon
sglang     Not Installed  package not installed

[gemini-code-assist]

Summary of Changes

Hello @doringeman, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the user experience of the docker model status command by standardizing how inference backends report their operational state. By centralizing status definitions and providing a structured, sorted tabular output, users can now quickly and clearly understand the status of each backend and any associated details.

Highlights

• Standardized Status Reporting: Introduced new constants and helper functions in pkg/inference/backend.go to ensure consistent status messages (e.g., Running, Error, Not Installed, Installing) and detailed descriptions across all inference backends.
• Enhanced CLI Output for docker model status: The cmd/cli/commands/status.go file was updated to display backend statuses in a new, prettified table format. This table sorts backends first by their status type (Running, Error, Not Installed, Installing) and then alphabetically by name, improving readability.
• Backend Integration of New Status System: All individual backend implementations (llama.cpp, vLLM, Diffusers, MLX, SGLang) were refactored to utilize the newly defined standardized status formatting functions and constants, ensuring uniform reporting.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• cmd/cli/commands/status.go
  • Imported bytes, sort, and inference packages to support new table formatting and status parsing.
  • Modified textStatus function to delegate status display to a new backendStatusTable function.
  • Added backendStatusTable function to generate a formatted, sorted table of backend statuses, categorizing by status type and then by name.
• pkg/inference/backend.go
  • Added StatusRunning, StatusError, StatusNotInstalled, and StatusInstalling constants for standardized status types.
  • Introduced common detail messages such as DetailBinaryNotFound, DetailOnlyLinux, DetailPythonNotFound, etc.
  • Implemented FormatStatus to create consistent status strings from a type and details.
  • Added ParseStatus to extract status type and details from a formatted status string.
  • Provided convenience functions FormatRunning, FormatError, FormatNotInstalled, and FormatInstalling for easier status string creation.
• pkg/inference/backends/diffusers/diffusers.go
  • Updated initial status to inference.FormatNotInstalled("").
  • Modified Install method to use inference.FormatNotInstalled(inference.DetailOnlyLinux) when platform is not supported.
  • Changed Python not found error status to inference.FormatError(inference.DetailPythonNotFound).
  • Updated package not installed status to inference.FormatNotInstalled(inference.DetailPackageNotInstalled).
  • Adjusted running status to inference.FormatRunning(inference.DetailVersionUnknown) or inference.FormatRunning(fmt.Sprintf("diffusers %s", ...)).
• pkg/inference/backends/llamacpp/download.go
  • Imported the inference package.
  • Updated running status messages to use inference.FormatRunning with version details.
  • Changed downloading status to inference.FormatInstalling with inference.DetailDownloading.
• pkg/inference/backends/llamacpp/download_linux.go
  • Imported the inference package.
  • Updated running status to use inference.FormatRunning with version details.
• pkg/inference/backends/llamacpp/download_windows.go
  • Imported the inference package.
  • Updated error statuses for CUDA/OpenCL checks to inference.FormatError.
  • Changed update checking status to inference.FormatInstalling with inference.DetailCheckingForUpdates.
• pkg/inference/backends/llamacpp/llamacpp.go
  • Removed direct l.status = "installing" assignment.
  • Updated installation failure status to inference.FormatError.
• pkg/inference/backends/mlx/mlx.go
  • Updated initial status to inference.FormatNotInstalled("").
  • Modified Install method to use inference.FormatNotInstalled(inference.DetailOnlyAppleSilicon) when platform is not supported.
  • Changed Python not found error status to inference.FormatError(inference.DetailPythonNotFound).
  • Updated package not installed status to inference.FormatNotInstalled(inference.DetailPackageNotInstalled).
  • Adjusted running status to inference.FormatRunning(inference.DetailVersionUnknown) or inference.FormatRunning(fmt.Sprintf("MLX %s", ...)).
• pkg/inference/backends/sglang/sglang.go
  • Updated initial status to inference.FormatNotInstalled("").
  • Modified Install method to use inference.FormatNotInstalled(inference.DetailOnlyLinux) when platform is not supported.
  • Changed Python not found error status to inference.FormatError(inference.DetailPythonNotFound).
  • Updated package not installed status to inference.FormatNotInstalled(inference.DetailPackageNotInstalled).
  • Adjusted running status to inference.FormatRunning(inference.DetailVersionUnknown) or inference.FormatRunning(fmt.Sprintf("sglang %s", ...)).
• pkg/inference/backends/vllm/vllm.go
  • Updated initial status to inference.FormatNotInstalled("").
  • Modified Install method to use inference.FormatNotInstalled(inference.DetailOnlyLinux) when platform is not supported.
  • Changed binary not found status to inference.FormatNotInstalled(inference.DetailBinaryNotFound).
  • Adjusted running status to inference.FormatRunning(inference.DetailVersionUnknown) or inference.FormatRunning(fmt.Sprintf("vllm %s", ...)).
• pkg/inference/backends/vllm/vllm_metal.go
  • Updated initial status to inference.FormatNotInstalled("").
  • Modified Install method to use inference.FormatNotInstalled(inference.DetailOnlyAppleSilicon) when platform is not supported.
  • Changed installing status to inference.FormatInstalling with inference.DetailDownloading.
  • Updated import failed status to inference.FormatError(inference.DetailImportFailed).
  • Adjusted running status to inference.FormatRunning(inference.DetailVersionUnknown) or inference.FormatRunning(fmt.Sprintf("vllm-metal %s", ...)).

Activity

• The pull request author, doringeman, has implemented changes to standardize status reporting across all backends and prettify the docker model status command output.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• In backendStatusTable, legacy values like "not running" will now be parsed as StatusError and shown as errors in the table; consider explicitly filtering or mapping these known legacy strings instead of treating them as generic errors.
• The status parsing in ParseStatus is tightly coupled to the four known prefixes and the exact "<prefix>: " format; if any backend accidentally deviates (e.g., extra spaces or different capitalization), it will be classified as StatusError, so it may be worth either normalizing input or adding a more defensive fallback for partially matching known statuses.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- In `backendStatusTable`, legacy values like `"not running"` will now be parsed as `StatusError` and shown as errors in the table; consider explicitly filtering or mapping these known legacy strings instead of treating them as generic errors.
- The status parsing in `ParseStatus` is tightly coupled to the four known prefixes and the exact `"<prefix>: "` format; if any backend accidentally deviates (e.g., extra spaces or different capitalization), it will be classified as `StatusError`, so it may be worth either normalizing input or adding a more defensive fallback for partially matching known statuses.

## Individual Comments

### Comment 1
<location> `cmd/cli/commands/status.go:84-85` </location>
<code_context>
+	}
+
+	backends := make([]backendInfo, 0, len(backendStatus))
+	for name, statusText := range backendStatus {
+		statusType, details := inference.ParseStatus(statusText)
+
+		// Assign sort order: Running < Error < Not Installed < Installing
</code_context>

<issue_to_address>
**issue:** Parsing status text assumes all backends now emit the new standardized format; legacy or custom values will be surfaced as generic errors.

With `ParseStatus`, any backend still returning legacy or custom status strings (e.g., "not running") will now be treated as `StatusError`, with the raw string shown in DETAILS. If that reclassification would be misleading for existing callers, consider either special-casing common legacy values before calling `ParseStatus`, or extending `ParseStatus` to recognize historical formats to keep behavior consistent during the migration.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

issue: Parsing status text assumes all backends now emit the new standardized format; legacy or custom values will be surfaced as generic errors.

With ParseStatus, any backend still returning legacy or custom status strings (e.g., "not running") will now be treated as StatusError, with the raw string shown in DETAILS. If that reclassification would be misleading for existing callers, consider either special-casing common legacy values before calling ParseStatus, or extending ParseStatus to recognize historical formats to keep behavior consistent during the migration.

[gemini-code-assist]

Code Review

This pull request is a great refactoring that standardizes status reporting across all backends and improves the output of docker model status with a formatted table. The introduction of status constants and helper functions in pkg/inference/backend.go is a solid approach to ensure consistency. The changes in each backend to adopt this new standard are well-executed. I have a few minor suggestions to further improve code clarity and maintainability.