Have LLM as a judge Evaluator in folder structure ( using / )

[ivishalanand]

Describe the feature or potential improvement

Having so many LLM as a judge evaluators makes things difficult to organise as we dont have any organisation for evaluators.
It will be great to have a directory design organisation as we have for datastet and prompt to organise the evaluators for different purposes.

Goal stretch would be to also provide way of organizing the running evaluators as well! Thanks

Additional information

No response

[marliessophie]

This makes sense, thanks for sharing feedback! No immediate improvement planned but we will want to invest in evals UX, so keeping this open for upvotes and more feedback makes a tone of sense.

Will inform you should we plan work in this direction.