BROKEN examples/demo-site

[sroussey]

System Info

A lot of basic stuff is not working in v3 that worked in v2. My own summarization code broke on v3 so I went to see sample code and found the same issues.

A great example of this is the basic test example demo-site.

I made the following change in examples/demo-site/package.json:

"@xenova/transformers": "^2.0.0-alpha.3" ==> "@huggingface/transformers": "^3.2.4",

And then used "@huggingface/transformers" in examples/demo-site/src/worker.js

Broken:

• Summarization w/ distilbart-cnn-6-6 -- results are blank (errors in console)
• Translation w/ t5-small -- results are blank (errors in console)
• Text generation w/ distilgpt2
• Question answering w/ distilbert-base-uncased-distilled-squad
• Code completion w/ Salesforce/codegen-350M-mono
• Speech to text w/ whisper-tiny.en
• Image to text w/ vit-gpt2-image-captioning
• Zero-shot image classification w/ openai/clip-vit-base-patch16

Works:

• Masked language modelling w/ bert-base-cased
• Text classification w/ bert-base-multilingual-uncased-sentiment (warnings in console, but still works)
• Zero-shot classification w/ typeform/distilbert-base-uncased-mnli (warnings in console)
• Image classification w/ google/vit-base-patch16-224
• Object detection w/ facebook/detr-resnet-50 (but only after multiple tries)

Who can help?

@xenova

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

See above.

Expected behavior

To work like v2, or just to work at all.

[xenova]

Hi there 👋 Indeed, the demo site is relatively out-of-date. For up-to-date Transformers.js v3 demos, check out https://github.com/huggingface/transformers.js-examples.

The main reason for these demos not working is that we removed the non-standard callback_function from v2. It has since been replaced by the TextStreamer class (also present in the python transformers library).

[sroussey]

Ah! A "breaking changes" in the announcement blog post or at least the docs would be a great addition.

I'll move the callback inside the streamer. Thanks.

Is it worth updating this example?

[sroussey]

Oh, also the examples in the docs are wrong then.

For example: https://huggingface.co/docs/transformers.js/en/tutorials/react

    callback_function: x => {
          self.postMessage({
              status: 'update',
              output: translator.tokenizer.decode(x[0].output_token_ids, { skip_special_tokens: true })
          });
      }

Also, the new callback function seems to get text not beams, so no decode step is required, though we need to keep a buffer of outputs outselves.

[sroussey]

Also, Xenova/distilgpt2 does not have d8 quant. How to just get the one name with quantized?

[sroussey]

See #1128

[sroussey]

See #1129