Support in Prompt Experiment to automatically resolve input media references

[zsoltvadaszi-grabtaxi]

Describe the feature or potential improvement

Media (image in our case) input experiments can be run only programatically as Langfuse does not automatically resolve media reference tokens (such as @@@langfuseMedia:type=image/png|id=...@@@) to base64 data or raw image content in the input passed to the LLM.

Our use cases are all image input based. We built a Jupyter notebook UI similar to the New Experiment one, however it would be great to have the experimentation in the Langfuse as well.

Additional information

No response

[hassiebp]

Thanks for raising this! We have improved multimodal support across our product on our near-term roadmap, including the idea you raised here

[scullinan]

HI @hassiebp , this is super cool to hear however we are 4 months on from this comment, can you provide some more concrete timelines on this please?

[david-AH]

We also very much need this capability

[sesprit]

I also really missing this capability to evaluate prompts that extract info from or classify images.

[scullinan]

Honestly I find this very surprising and frustrating!
This is a really big blocker for us. Can you please provide some clarity on when this will be implemented on the roadmap?