Skip to content

[BUG] image_model config field is not wired into agent loop — images always route to primary model #1578

@fernandostahelin

Description

@fernandostahelin

Quick Summary

The image_model field in agents.defaults is accepted in config.json but has no effect. Image messages are always sent to the primary model_name, even when image_model points to a vision-capable model.

Environment & Tools

  • PicoClaw Version: v0.2.3-nightly.20260314 (git: c68b4f3)
  • Go Version: go 1.25.7
  • AI Model & Provider: Cerebras (qwen-3-235b-a22b-instruct-2507) as primary, Gemini 2.5 Flash Lite as image_model
  • Operating System: Linux arm64 (Docker on macOS)
  • Channels: Telegram

📸 Steps to Reproduce

  1. Configure config.json with a text-only primary model and a vision-capable image model:
    "agents": {
      "defaults": {
        "model_name": "cerebras-baristo",
        "image_model": "gemini-images",
        "fallbacks": ["claude-fallback"]
      }
    }
  2. Add both models to model_list:
    "model_list": [
      {"model_name": "cerebras-baristo", "model": "cerebras/qwen-3-235b-a22b-instruct-2507", ...},
      {"model_name": "gemini-images", "model": "gemini/gemini-2.5-flash-lite", ...}
    ]
  3. Send an image via Telegram

❌ Actual Behavior

Image messages are sent to the primary model (Cerebras/Qwen), which returns:

Status: 422
Body: {"message":"Content type 'image_url' is not supported by selected model. Only 'text' content type can be used."}

✅ Expected Behavior

Image messages should be routed to the image_model (Gemini) since it supports vision/multimodal input and the primary model does not.

💬 Additional Context

  • ImageModel and ImageModelFallbacks are defined in pkg/config/config.go
  • pkg/routing/features.go detects images via hasAttachments() and sets HasAttachments = true
  • pkg/providers/fallback.go has an ExecuteImage() function for image-specific fallback chains
  • However, selectCandidates() in pkg/agent/loop.go does not reference ImageModel — it only routes between primary and light model based on score
  • The field appears partially implemented but not connected in the agent loop

Related: #1489 (lists image_model as undocumented)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions