feat: responsive image delivery with multi-size PNG + WebP (#5192)

## Summary

- Replace single `plot_thumb.png` with responsive multi-size PNG + WebP
variants (400/800/1200px) for all plot images
- Frontend `<picture>` elements with `srcSet` + `sizes` let the browser
pick optimal size based on viewport and DPR
- Remove `preview_thumb` field from API, database sync, metadata
templates, and all documentation
- Graceful fallback to `plot.png` during migration (before backfill
runs)

### Image savings (scatter-basic/matplotlib example)

| Variant | Size | vs original |
|---------|------|-------------|
| plot.png (original) | 477K | baseline |
| plot_800.webp (typical mobile) | 18K | **-96%** |
| plot_400.webp (small mobile) | 7.2K | **-98%** |

### Changes (33 files)

- **Backend**: `create_responsive_variants()` in `core/images.py`
generates all 7 variants
- **Frontend**: `<picture>` + `srcSet` in ImageCard, SpecDetailView,
SpecOverview, CatalogPage
- **Workflows**: `impl-generate.yml` and `impl-repair.yml` generate +
upload all variants
- **API**: Removed `preview_thumb`/`thumb` from schemas, routers, MCP
server
- **Scripts**: `backfill_responsive_images.py` for historical plots,
`remove_preview_thumb_from_yaml.py` for YAML cleanup

### Post-merge steps

1. Run `python scripts/backfill_responsive_images.py` to generate
variants for ~2,668 existing images (~2h)
2. Run `python scripts/remove_preview_thumb_from_yaml.py` → commit YAML
cleanup separately
3. Later: Alembic migration to drop `preview_thumb` DB column

Closes #5191

## Test plan

- [x] 1061 unit tests pass
- [x] Frontend builds successfully
- [x] Ruff lint + format clean
- [x] Local test: responsive variants generated with correct sizes
- [ ] Deploy and verify fallback to plot.png for historical images
- [ ] Run backfill script on production GCS
- [ ] Trigger impl-generate for one spec to verify new workflow

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: MarkusNeusinger

.github/workflows/impl-generate.yml

@@ -391,7 +391,6 @@ jobs:
               'python_version': py_ver,
               'library_version': lib_ver,
               'preview_url': f'https://storage.googleapis.com/pyplots-images/plots/{spec}/{lib}/plot.png',
-              'preview_thumb': f'https://storage.googleapis.com/pyplots-images/plots/{spec}/{lib}/plot_thumb.png',
               'preview_html': preview_html if preview_html != 'null' else None,
               'quality_score': None,
               'review': {'strengths': [], 'weaknesses': []}
@@ -448,13 +447,15 @@ jobs:
 
           source .venv/bin/activate
 
-          # Process PNG: optimize and create thumbnail
+          # Optimize PNG and generate responsive variants (400/800/1200 x png/webp + full webp)
           python -m core.images process \
             "$IMPL_DIR/plot.png" \
+            "$IMPL_DIR/plot.png"
+          python -m core.images responsive \
             "$IMPL_DIR/plot.png" \
-            "$IMPL_DIR/plot_thumb.png"
+            "$IMPL_DIR/"
 
-          echo "::notice::Processed images: optimized + thumbnail created"
+          echo "::notice::Processed images: optimized + responsive variants created"
           ls -la "$IMPL_DIR/"
 
       # ========================================================================
@@ -532,25 +533,18 @@ jobs:
           echo "$GCS_CREDENTIALS" > /tmp/gcs-key.json
           gcloud auth activate-service-account --key-file=/tmp/gcs-key.json
 
-          # Upload PNG (with watermark)
+          # Upload all plot images (original + responsive variants)
           if [ -f "$IMPL_DIR/plot.png" ]; then
-            gsutil -h "Cache-Control:public, max-age=604800" cp "$IMPL_DIR/plot.png" "${STAGING_PATH}/plot.png"
-            gsutil acl ch -u AllUsers:R "${STAGING_PATH}/plot.png" 2>/dev/null || true
+            gsutil -m -h "Cache-Control:public, max-age=604800" cp "$IMPL_DIR"/plot*.png "$IMPL_DIR"/plot*.webp "${STAGING_PATH}/"
+            gsutil -m acl ch -u AllUsers:R "${STAGING_PATH}/plot*" 2>/dev/null || true
             echo "png_url=${PUBLIC_URL}/plot.png" >> $GITHUB_OUTPUT
             echo "uploaded=true" >> $GITHUB_OUTPUT
+            echo "::notice::Uploaded plot.png + responsive variants (PNG + WebP)"
           else
             echo "::error::No plot.png found - cannot continue without image"
             exit 1
           fi
 
-          # Upload thumbnail
-          if [ -f "$IMPL_DIR/plot_thumb.png" ]; then
-            gsutil -h "Cache-Control:public, max-age=604800" cp "$IMPL_DIR/plot_thumb.png" "${STAGING_PATH}/plot_thumb.png"
-            gsutil acl ch -u AllUsers:R "${STAGING_PATH}/plot_thumb.png" 2>/dev/null || true
-            echo "thumb_url=${PUBLIC_URL}/plot_thumb.png" >> $GITHUB_OUTPUT
-            echo "::notice::Uploaded thumbnail"
-          fi
-
           # Upload HTML (interactive libraries)
           if [ -f "$IMPL_DIR/plot.html" ]; then
             gsutil -h "Cache-Control:public, max-age=604800" cp "$IMPL_DIR/plot.html" "${STAGING_PATH}/plot.html"

.github/workflows/impl-repair.yml

@@ -159,13 +159,15 @@ jobs:
 
           source .venv/bin/activate
 
-          # Process PNG: optimize and create thumbnail
+          # Optimize PNG and generate responsive variants
           python -m core.images process \
             "$IMPL_DIR/plot.png" \
+            "$IMPL_DIR/plot.png"
+          python -m core.images responsive \
             "$IMPL_DIR/plot.png" \
-            "$IMPL_DIR/plot_thumb.png"
+            "$IMPL_DIR/"
 
-          echo "::notice::Processed images: optimized + thumbnail created"
+          echo "::notice::Processed images: optimized + responsive variants created"
           ls -la "$IMPL_DIR/"
 
       - name: Upload repaired plot to GCS staging
@@ -186,14 +188,10 @@ jobs:
           echo "$GCS_CREDENTIALS" > /tmp/gcs-key.json
           gcloud auth activate-service-account --key-file=/tmp/gcs-key.json
 
+          # Upload all plot images (original + responsive variants)
           if [ -f "$IMPL_DIR/plot.png" ]; then
-            gsutil -h "Cache-Control:public, max-age=604800" cp "$IMPL_DIR/plot.png" "${STAGING_PATH}/plot.png"
-            gsutil acl ch -u AllUsers:R "${STAGING_PATH}/plot.png" 2>/dev/null || true
-          fi
-
-          if [ -f "$IMPL_DIR/plot_thumb.png" ]; then
-            gsutil -h "Cache-Control:public, max-age=604800" cp "$IMPL_DIR/plot_thumb.png" "${STAGING_PATH}/plot_thumb.png"
-            gsutil acl ch -u AllUsers:R "${STAGING_PATH}/plot_thumb.png" 2>/dev/null || true
+            gsutil -m -h "Cache-Control:public, max-age=604800" cp "$IMPL_DIR"/plot*.png "$IMPL_DIR"/plot*.webp "${STAGING_PATH}/"
+            gsutil -m acl ch -u AllUsers:R "${STAGING_PATH}/plot*" 2>/dev/null || true
           fi
 
           if [ -f "$IMPL_DIR/plot.html" ]; then

agentic/commands/update.md

@@ -268,11 +268,10 @@ For each library, copy the preview images to the implementations directory for G
 
 ```bash
 cp plots/{spec_id}/implementations/.update-preview/{library}/plot.png plots/{spec_id}/implementations/plot.png
-# Process images (thumbnail + optimization)
+# Process images (optimization)
 uv run python -m core.images process \
   plots/{spec_id}/implementations/plot.png \
-  plots/{spec_id}/implementations/plot.png \
-  plots/{spec_id}/implementations/plot_thumb.png
+  plots/{spec_id}/implementations/plot.png
 ```
 
 Note: Since we process one library at a time for GCS upload, handle sequentially.
@@ -288,23 +287,18 @@ STAGING_PATH="gs://pyplots-images/staging/{spec_id}/{library}"
 gsutil cp plots/{spec_id}/implementations/plot.png "${STAGING_PATH}/plot.png"
 gsutil acl ch -u AllUsers:R "${STAGING_PATH}/plot.png" 2>/dev/null || true
 
-# Upload thumbnail
-gsutil cp plots/{spec_id}/implementations/plot_thumb.png "${STAGING_PATH}/plot_thumb.png"
-gsutil acl ch -u AllUsers:R "${STAGING_PATH}/plot_thumb.png" 2>/dev/null || true
-
 # Upload HTML if it exists (interactive libraries: plotly, bokeh, altair, highcharts, pygal, letsplot)
 if [ -f "plots/{spec_id}/implementations/.update-preview/{library}/plot.html" ]; then
   gsutil cp "plots/{spec_id}/implementations/.update-preview/{library}/plot.html" "${STAGING_PATH}/plot.html"
   gsutil acl ch -u AllUsers:R "${STAGING_PATH}/plot.html" 2>/dev/null || true
 fi
 ```
 
-Update `preview_url` and `preview_thumb` in the metadata YAML to point to the **production** URLs
+Update `preview_url` in the metadata YAML to point to the **production** URL
 (matching `impl-generate.yml` — production URLs are set from the start, `impl-merge.yml` promotes
 GCS files from staging to production on merge):
 
 - `preview_url`: `https://storage.googleapis.com/pyplots-images/plots/{spec_id}/{library}/plot.png`
-- `preview_thumb`: `https://storage.googleapis.com/pyplots-images/plots/{spec_id}/{library}/plot_thumb.png`
 
 #### 6f. Clean Up Preview Directory
 
@@ -637,8 +631,7 @@ Generate thumbnail and optimize:
 ```bash
 uv run python -m core.images process \
   plots/{SPEC_ID}/implementations/.update-preview/{LIBRARY}/plot.png \
-  plots/{SPEC_ID}/implementations/.update-preview/{LIBRARY}/plot.png \
-  plots/{SPEC_ID}/implementations/.update-preview/{LIBRARY}/plot_thumb.png
+  plots/{SPEC_ID}/implementations/.update-preview/{LIBRARY}/plot.png
 ```
 
 ### Step 7: Quality Evaluation & Local Repair Loop

agentic/docs/project-guide.md

@@ -219,7 +219,6 @@ library_version: "3.10.0"
 
 # Previews
 preview_url: https://storage.googleapis.com/pyplots-images/plots/scatter-basic/matplotlib/plot.png
-preview_thumb: https://storage.googleapis.com/pyplots-images/plots/scatter-basic/matplotlib/plot_thumb.png
 preview_html: null
 
 # Quality

api/mcp/server.py

@@ -274,7 +274,6 @@ async def get_spec_detail(spec_id: str) -> dict[str, Any]:
                 library_id=impl.library.id,
                 library_name=impl.library.name,
                 preview_url=impl.preview_url,
-                preview_thumb=impl.preview_thumb,
                 preview_html=impl.preview_html,
                 quality_score=impl.quality_score,
                 code=impl.code,
@@ -366,7 +365,6 @@ async def get_implementation(spec_id: str, library: str) -> dict[str, Any]:
             library_id=impl.library.id,
             library_name=impl.library.name,
             preview_url=impl.preview_url,
-            preview_thumb=impl.preview_thumb,
             preview_html=impl.preview_html,
             quality_score=impl.quality_score,
             code=impl.code,

api/routers/libraries.py

@@ -75,7 +75,7 @@ async def get_library_images(library_id: str, db: AsyncSession = Depends(require
         library_id: The library ID (e.g., 'matplotlib', 'seaborn')
 
     Returns:
-        List of images with spec_id, preview_url, thumb, and html
+        List of images with spec_id, preview_url, and html
     """
 
     # Validate library_id
@@ -99,7 +99,6 @@ async def get_library_images(library_id: str, db: AsyncSession = Depends(require
                         "spec_id": spec.id,
                         "library": impl.library_id,
                         "url": impl.preview_url,
-                        "thumb": impl.preview_thumb,
                         "html": impl.preview_html,
                         "code": strip_noqa_comments(impl.code),
                     }

api/routers/og_images.py

@@ -68,15 +68,24 @@ def _get_http_client() -> httpx.AsyncClient:
     global _http_client
     if _http_client is None or _http_client.is_closed:
         _http_client = httpx.AsyncClient(
-            timeout=30.0,
-            limits=httpx.Limits(max_connections=10, max_keepalive_connections=5),
+            timeout=30.0, limits=httpx.Limits(max_connections=10, max_keepalive_connections=5)
         )
     return _http_client
 
 
 async def _fetch_image(url: str) -> bytes:
-    """Fetch an image from a URL using the shared HTTP client."""
-    response = await _get_http_client().get(url)
+    """Fetch an image from a URL, trying the 800px variant first for efficiency."""
+    client = _get_http_client()
+    # Prefer smaller responsive variant for OG collage (each slot is ~400px wide)
+    if url and url.endswith("/plot.png"):
+        small_url = url.replace("/plot.png", "/plot_800.png")
+        try:
+            response = await client.get(small_url)
+            response.raise_for_status()
+            return response.content
+        except Exception:
+            pass  # Fall back to original
+    response = await client.get(url)
     response.raise_for_status()
     return response.content
 
@@ -167,10 +176,8 @@ async def get_spec_collage_image(
     selected_impls = sorted_impls[:6]
 
     try:
-        # Fetch all images in parallel — prefer thumbnails (smaller, faster)
-        images = list(
-            await asyncio.gather(*[_fetch_image(impl.preview_thumb or impl.preview_url) for impl in selected_impls])
-        )
+        # Fetch all images in parallel
+        images = list(await asyncio.gather(*[_fetch_image(impl.preview_url) for impl in selected_impls]))
         labels = [f"{spec_id} · {impl.library_id}" for impl in selected_impls]
 
         # Create collage

api/routers/plots.py

@@ -328,7 +328,7 @@ def _collect_all_images(all_specs: list) -> list[dict]:
         all_specs: List of Spec objects
 
     Returns:
-        List of image dicts with spec_id, library, quality, url, thumb, html, and title
+        List of image dicts with spec_id, library, quality, url, html, and title
     """
     all_images: list[dict] = []
     for spec_obj in all_specs:
@@ -342,7 +342,6 @@ def _collect_all_images(all_specs: list) -> list[dict]:
                         "library": impl.library_id,
                         "quality": impl.quality_score,
                         "url": impl.preview_url,
-                        "thumb": impl.preview_thumb,
                         "html": impl.preview_html,
                         "title": spec_obj.title,
                     }

api/routers/specs.py

@@ -94,7 +94,6 @@ async def get_spec(spec_id: str, db: AsyncSession = Depends(require_db)):
             library_id=impl.library_id,
             library_name=impl.library.name if impl.library else impl.library_id,
             preview_url=impl.preview_url,
-            preview_thumb=impl.preview_thumb,
             preview_html=impl.preview_html,
             quality_score=impl.quality_score,
             code=strip_noqa_comments(impl.code),
@@ -135,7 +134,7 @@ async def get_spec_images(spec_id: str, db: AsyncSession = Depends(require_db)):
     """
     Get plot images for a specification across all libraries.
 
-    Returns preview_url, preview_thumb, and preview_html from database.
+    Returns preview_url and preview_html from database.
     """
 
     key = cache_key("spec_images", spec_id)
@@ -153,7 +152,7 @@ async def get_spec_images(spec_id: str, db: AsyncSession = Depends(require_db)):
         raise_not_found("Spec with implementations", spec_id)
 
     images = [
-        {"library": impl.library_id, "url": impl.preview_url, "thumb": impl.preview_thumb, "html": impl.preview_html}
+        {"library": impl.library_id, "url": impl.preview_url, "html": impl.preview_html}
         for impl in spec.impls
         if impl.preview_url  # Only include if there's a preview
     ]

api/schemas.py

@@ -15,7 +15,6 @@ class ImplementationResponse(BaseModel):
     library_id: str
     library_name: str
     preview_url: str | None = None
-    preview_thumb: str | None = None
     preview_html: str | None = None
     quality_score: float | None = None
     code: str | None = None
@@ -66,7 +65,6 @@ class ImageResponse(BaseModel):
     spec_id: str
     library: str
     url: str | None = None
-    thumb: str | None = None
     html: str | None = None
     code: str | None = None
 
@@ -93,7 +91,7 @@ class FilteredPlotsResponse(BaseModel):
     """Response for filtered plots endpoint."""
 
     total: int
-    images: list[dict[str, Any]]  # Image dicts with spec_id, library, url, thumb, etc.
+    images: list[dict[str, Any]]  # Image dicts with spec_id, library, url, etc.
     counts: dict[str, dict[str, int]]  # Category -> value -> count
     globalCounts: dict[str, dict[str, int]]  # Same structure for global counts
     orCounts: list[dict[str, int]]  # Per-group OR counts