Skip to content

Handle dataset loading if schema check fails - #6

Merged
fabianlim merged 2 commits into
garrett361:padding-freefrom
fabianlim:swanand/docs
Jun 26, 2025
Merged

fabianlim merged 2 commits into
garrett361:padding-freefrom
fabianlim:swanand/docs

Conversation

@fabianlim

Copy link
Copy Markdown
Collaborator

changes from @swanand

load_dataset has strict schema checks, that may fail when tools and documents are added. This adds a workaround to load the dataset in pandas, and then create the dataset object

@fabianlim
fabianlim requested a review from dangxuanhong June 21, 2025 20:24
Comment thread open_instruct/finetune.py Resolved
@fabianlim
fabianlim changed the base branch from padding-free-xh to padding-free June 26, 2025 18:39
@fabianlim
fabianlim merged commit f6f5e73 into garrett361:padding-free Jun 26, 2025
@fabianlim
fabianlim deleted the swanand/docs branch June 26, 2025 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants