Add Model loading from subfolders similar to transformers#443
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
| task = cls.export_feature | ||
| else: | ||
| task = HfApi().model_info(model_id, revision=revision).pipeline_tag | ||
| task = HfApi().model_info(model_id, revision=revision).pipeline_tag # FIXME: load from subfolder? |
There was a problem hiding this comment.
Out of curiosity, the FIXME comment means that we are not sure if this can be done with HfApi?
There was a problem hiding this comment.
One of the issue is that the ONNX export in transformers relies on preprocessors to generate dummy inputs, hence we use preprocessor = get_preprocessor(model_id) that has no option to specify a subfolder. We would need to do a PR in transformers for that. Therefore, the tokenizer must be at the top level of the repo, as well as the config.json that may be required in AutoTokenizer to find the right class.
I changed a bit the code so that we don't need to expect a config.json in the subfolder, we fall back to using the one at the top level if there is not in the subfolder. Does it sound good to you?
There was a problem hiding this comment.
Yes sounds good! In that case, maybe we could log a warning saying that the config was not found in the given subfolder and that we are looking for it in the parent folder.
|
Actually this could be useful if we want to save ONNX models in subfolders on the Hub? cc @JingyaHuang |
3 participants
What does this PR do?
Allow to pass the argument
subfoldertoORTModel.from_pretrained()in a similar fashion to transformersPreTrainedModel. It allows to host several models (e.g. optimized) in the same model repository.The preprocessor is expected to be loaded from the parent directory.
Fixes #308
Before submitting