Skip to content

Conversation

mattt
Copy link
Collaborator

@mattt mattt commented Sep 19, 2025

The Hub module contains a FallbackConfigs directory containing fallback tokenizer configurations whose contents are bundled as package resources. Whereas ModelsTests and TokenizersTests each contain a Resources directory. This PR renames FallbackConfigs to Resources for consistency. It also moves the Vocabs/bert-vocab.txt file in TokenizerTests to that module's Resources directory.

@FL33TW00D
Copy link
Collaborator

Solid clean up!

@FL33TW00D FL33TW00D merged commit a814cc5 into main Sep 19, 2025
2 checks passed
@pcuenca
Copy link
Member

pcuenca commented Sep 19, 2025

I named it differently to better express intention and because these would ideally be removed at some point in the future. We shouldn't need them, as opposed to resources used elsewhere. It's fine to rename, no need to revert. In fact, I'll check if the reasons still stand or if we can remove.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants