Skip to content

Return file paths from gguf and safetensors parsers#2014

Merged
mishig25 merged 8 commits intomainfrom
gguf-and-safetensors-return-filepaths
Mar 11, 2026
Merged

Return file paths from gguf and safetensors parsers#2014
mishig25 merged 8 commits intomainfrom
gguf-and-safetensors-return-filepaths

Conversation

@julien-c
Copy link
Member

@julien-c julien-c commented Mar 1, 2026

Summary

  • [gguf] ggufAllShards now returns urls (the resolved shard URLs) alongside shards and parameterCount
  • [safetensors] parseSafetensorsMetadata now returns filepaths (the resolved file paths, including index + shard files for sharded models)
  • Minor formatting cleanup (trailing commas, whitespace, indentation)

Test plan

  • Verify ggufAllShards return type includes urls for both single and sharded GGUF files
  • Verify parseSafetensorsMetadata return type includes filepaths for both single and sharded safetensors
  • Run existing tests to ensure no regressions

Note

Medium Risk
Mostly additive API surface, but it changes TypeScript return types for ggufAllShards and parseSafetensorsMetadata, which could require downstream updates for consumers relying on the previous shapes.

Overview
Adds resolved file location outputs to weight parsers.

ggufAllShards now returns urls alongside shards and parameterCount (for both sharded and single-file GGUF), and parseSafetensorsMetadata now returns filepaths listing the weights involved (single file, or index + shard files for sharded models). Tests are updated to assert the new fields.

Written by Cursor Bugbot for commit 8099f34. This will update automatically on new commits. Configure here.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Prepend the path prefix (directory portion of the index file path) to each
shard filename in the returned filepaths array, so that all entries are
consistent repo-relative paths rather than a mix of full paths and bare
filenames.

Co-authored-by: Julien Chaumond <julien@huggingface.co>
@julien-c
Copy link
Member Author

@mishig25 and @ngxson can i please get a review on this? 🙏

@julien-c julien-c requested a review from gary149 March 10, 2026 12:42
@julien-c
Copy link
Member Author

CI failure seems unrelated

Copy link
Collaborator

@mishig25 mishig25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! I pushed test cases b365d99

@mishig25
Copy link
Collaborator

this red CI is not relevant to this PR.

this CI passes green on rerun.

the likely scenario why the test is/was flaky is: transient network failure/timeout in CI from hub when downloading the header of one of the 64 shard files (kimi-k2.5). Re-running the CI job should made it pass

@mishig25 mishig25 merged commit 78ec5bc into main Mar 11, 2026
7 of 10 checks passed
@mishig25 mishig25 deleted the gguf-and-safetensors-return-filepaths branch March 11, 2026 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants