Describe the bug
Sending an image from the built-in web UI (http://localhost:1234/ui) fails with HTTP 500:
invalid image source, expected http https or data url: relative url without a base
Sending the same image directly through the /v1/chat/completions API works fine.
Root cause
The UI uploads the picked image to /api/upload_image, which saves it to the cache dir and returns a relative URL (mistralrs-cli/src/ui/handlers/api.rs):
let url = format!("uploads/{filename}");
(StatusCode::OK, Json(json!({ "path": path, "url": url }))).into_response()
The UI stores that relative string in the message and sends it back as image_url.url to /v1/chat/completions (mistralrs-cli/webui/src/lib/stores/chat.svelte.ts -> image_url: { url }). That endpoint resolves image sources under the ServerRequest media policy, which only accepts http / https / data URLs (mistralrs-server-core/src/media_source.rs):
anyhow::bail!("Invalid {kind} source: expected http, https, or data URL: {err}");
Url::parse("uploads/<uuid>.png") fails with RelativeUrlWithoutBase ("relative url without a base"), producing the 500. The v1 API works because real callers pass a data:/http(s) URL. (The relative URL only works for display, via the /uploads static route.)
The upload flow was added in #1370 / #1364.
Steps to reproduce
mistralrs serve -m <any vision model> (UI is on by default at /ui)
- Open
http://localhost:1234/ui, attach an image, send.
- -> HTTP 500
invalid image source ... relative url without a base.
- The same image via
POST /v1/chat/completions with a data:/http URL succeeds.
Other information
- OS: Linux (WSL2); reproduces regardless of backend (it's a request-layer/URL bug).
Latest commit or version
v0.8.23 / current master.
I have a fix ready (encode UI images as data: URLs client-side, matching what the working v1 path sends) and will open a PR referencing this issue. Note videos share the identical relative-URL path and are fixed the same way.
Describe the bug
Sending an image from the built-in web UI (
http://localhost:1234/ui) fails with HTTP 500:Sending the same image directly through the
/v1/chat/completionsAPI works fine.Root cause
The UI uploads the picked image to
/api/upload_image, which saves it to the cache dir and returns a relative URL (mistralrs-cli/src/ui/handlers/api.rs):The UI stores that relative string in the message and sends it back as
image_url.urlto/v1/chat/completions(mistralrs-cli/webui/src/lib/stores/chat.svelte.ts->image_url: { url }). That endpoint resolves image sources under theServerRequestmedia policy, which only acceptshttp/https/dataURLs (mistralrs-server-core/src/media_source.rs):Url::parse("uploads/<uuid>.png")fails withRelativeUrlWithoutBase("relative url without a base"), producing the 500. The v1 API works because real callers pass adata:/http(s)URL. (The relative URL only works for display, via the/uploadsstatic route.)The upload flow was added in #1370 / #1364.
Steps to reproduce
mistralrs serve -m <any vision model>(UI is on by default at/ui)http://localhost:1234/ui, attach an image, send.invalid image source ... relative url without a base.POST /v1/chat/completionswith adata:/httpURL succeeds.Other information
Latest commit or version
v0.8.23 / current
master.I have a fix ready (encode UI images as
data:URLs client-side, matching what the working v1 path sends) and will open a PR referencing this issue. Note videos share the identical relative-URL path and are fixed the same way.