Add the CIDEr metric?

[zuujhyt]

Hi,
I find the api in https://huggingface.co/metrics quite useful.
I am playing around with video/image captioning task, where CIDEr is a popular metric.
Do you plan to add this into the HF datasets library?
Thanks.

[Kamino666]

same

[lvwerra]

For reference: https://github.com/vrama91/cider

cc @sashavor

[Guitaricet]

Official CIDER implementation uses PTBTokenizer from Stanford CoreNLP. Unfortunately, it requires java and a .jar file with the library. What do you think could be an acceptable way to add CIDER to Evaluate?

[dinhanhx]

Official CIDER implementation uses PTBTokenizer from Stanford CoreNLP. Unfortunately, it requires java and a .jar file with the library. What do you think could be an acceptable way to add CIDER to Evaluate?

We can do like NLTK library, have the users provide path to jar file.

[gabrielmotablima]

Relevant topic. I'm working on vision-language models for low-resource languages, and it would be really helpful to have metrics like CIDEr-D, CIDEr-R, SPICE, and SPIDEr available in Evaluate 🐧🐧🐧🐧🐧🐧

[CurryYuan]

Any progress?

[Kamichanw]

I've opened a pr.