[shiweijiezero]

Goal

Let Chat Completions, Claude Code, and Codex clients use the same Foretoken frontend and model service through their native APIs.

Design

• Group OpenAI Completions, Anthropic Messages, and OpenAI Responses under one API layer. Share request timing, chat dispatch, decoding, idle timeouts, and SSE heartbeats; keep each protocol's request and response formats separate.
• Reuse the existing generation service for model preparation, routing, and execution. One deadline covers the request through completion, and disconnects release backend work and routing reservations.
• Adapt the Messages and Responses support from vLLM's Rust frontend work, retaining upstream provenance and license notices. No protocol-conversion service is added.
• Support JSON and streaming output, client-executed tools, reasoning, and Messages token counting. Preserve output-budget exhaustion and incomplete tool-call semantics.
• Keep vLLM patches together and synchronize their build consumers. The frontend guides provide minimal requests and the settings clients need.

Interface

API	Endpoint
Chat Completions	POST /v1/chat/completions
Messages	POST /v1/messages
Messages token counting	POST /v1/messages/count_tokens
Responses	POST /v1/responses

All generation endpoints accept stream: true. Responses clients set store: false and send conversation history on each turn. Clients execute tools and return their results in the next request. Messages uses max_tokens as the total output budget, including reasoning.

Impact

Existing frontend deployments gain these routes without a protocol-selection setting. The request timeout covers admission, preparation, and generation. Server-side conversation storage, background responses, and hosted tools are outside these APIs' supported mode.