Large images exceed provider size limits — need built-in resize or crop parameter for {{image:...}} variable

[deanleigh]

Which component is this issue related to?

Umbraco.AI (Core)

Which Umbraco AI version are you using? (Please write the exact version, example: 10.1.0)

17.2.2

Bug summary

When using {{image:umbracoFile}} in a prompt template, the original image file is sent to the AI provider with no resizing or compression. For any reasonably sized upload from a modern camera/phone, this exceeds provider limits.

Example error from Claude (Anthropic provider):

Status Code: BadRequest
{"type":"error","error":{"type":"invalid_request_error",
"message":"messages.0.content.1.image.source.base64: image exceeds 5 MB maximum: 8393108 bytes > 5242880 bytes"}}

The media in question had multiple image crops defined (thumbnail, content variants, social) — all well under 5MB — but the original 8MB file was sent regardless.

Specifics

No response

Steps to reproduce

Testing undocumented crop syntax in the template variable:

{{image:umbracoFile:content3Col}} — template resolves to empty, no image sent
{{image:umbracoFile.content3Col}} — same
{{image:umbracoFile,content3Col}} — same
Confirmed via source: ImageTemplateVariableProcessor passes everything after image: as a single property alias to entity.GetValue(), and AIUmbracoMediaResolver reads raw bytes from the filesystem with no transformation. There's no hook for crop selection or resize.

Expected result / actual result

Crop parameter in the variable syntax — e.g. {{image:umbracoFile:content6Col}} resolves to the named crop URL and fetches that
Config-level max dimensions / file size — provider-aware default (Claude 5MB, OpenAI different) with automatic downscale via ImageSharp before sending
Both — explicit crop when editor wants control, auto-resize as a safety net
Happy to contribute a PR if you'd like — let me know the approach you'd prefer.

Dependencies

Umbraco 17.2.2
Umbraco.AI 1.7.0, Umbraco.AI.Prompt 1.6.0, Umbraco.AI.Anthropic 1.2.2

Note 1: I should also add the of course there is no need to upload such large images in the first place but this would reduce the overall size images going to AI for simple things such as alternative text.

Note 2: this also to go one step towards creating alternative text for images where there are crops as it could describe the different crops and provide alternative text for each where only a part of the image is now showing For example the original image may be a dog and a cat but each of the crops only show only the dog or only the cat and therefore be described as such

[mattbrailsford]

Hi @deanleigh — thanks for the detailed report and for spotting both the missing crop parameter and the oversized-original problem. Both are fixed in #123, which I'd love your help testing before it merges.

Two things land together:

1. Auto-downscale safety net. Oversized images are now automatically resized and re-encoded before being sent to the provider. Defaults are 4 MB / 2048px longest edge / JPEG quality 85, which keeps payloads under every current provider limit (including Anthropic's 5 MB). Configurable under Umbraco:AI:Media — see the PR description for the shape. Animated images (GIF/WebP) pass through unchanged since re-encoding to JPEG would silently drop animation.

2. Named crops in {{image:...}} variables. The syntax is {{image:umbracoFile#cropAlias}} — e.g. {{image:umbracoFile#content3Col}}. I went with # rather than : or . because : is already the prefix separator and I wanted to preserve . for future nested property access (block lists etc). Unknown crop aliases fall back to the uncropped original with a warning, so a typo won't kill the prompt.

Worth flagging for your Note 2 use case: the crop alias is now included in the AI-facing reference name, so {{image:umbracoFile#wide}} {{image:umbracoFile#square}} on the same media produces [Image: My Photo (wide)] and [Image: My Photo (square)] — the AI can distinguish and describe each crop individually for per-crop alt text.

If you get a chance to pull the branch (feature/image-crop-and-downscale) and try it against your 8 MB image + the content3Col/content6Col crops from the original report, I'd appreciate the sanity check. Any feedback on the separator choice or the config defaults especially welcome.

[deanleigh]

Hey @mattbrailsford

1. I know I could limit clients uploading large images, but not everyone will — I think this is a really good quick solution.

2. This one is actually fantastic. I tagged you in a conversation on Discord today about adding alt text per crop, and this immediately opens up a lot of opportunities to address a long-standing issue.

Are you releasing a preview package? If so I'd happily test it quickly on a feature branch of the live project.

[mattbrailsford]

@deanleigh we haven't got a realease process for PR branches atm so you'd need to pull the source code and build / run or publish to packages locally if you want to test it. I'll take it for a spin myself tomorrow either way.

[deanleigh]

You are a star @mattbrailsford !

[mattbrailsford]

No worries @deanleigh I tested it this morning and it's working nicely so I'll get it out in the next release.