Skip to content

Conversation

@faaany
Copy link
Contributor

@faaany faaany commented May 12, 2025

What does this PR do?

When running the run_wav2vec2_pretraining_no_trainer.py example, I got fsspec.exceptions.FSTimeoutError.

The reason is that the default timeout (5 * 60s) for the aiohttp library used internally by fsspec is too short for large files, e.g. 5.9G.

I also found an issue discussed in the dataset library, which resolved the issue by the same approach: huggingface/datasets#7164.

@github-actions
Copy link
Contributor

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the Ready for review button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@github-actions github-actions bot marked this pull request as draft May 12, 2025 02:06
@faaany faaany marked this pull request as ready for review May 12, 2025 02:07
@faaany
Copy link
Contributor Author

faaany commented May 12, 2025

cc @Rocketknight1

@Rocketknight1
Copy link
Member

This looks good, but the timeout should probably be even longer! Some connections may still struggle to fetch 6GB in 10 minutes. Maybe make it 60 * 60 to be safe?

@faaany
Copy link
Contributor Author

faaany commented May 13, 2025

This looks good, but the timeout should probably be even longer! Some connections may still struggle to fetch 6GB in 10 minutes. Maybe make it 60 * 60 to be safe?

yes, good suggestion! code updated.

Copy link
Member

@Rocketknight1 Rocketknight1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now, thank you for the PR!

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Rocketknight1 Rocketknight1 merged commit 8fb60bf into huggingface:main May 13, 2025
11 checks passed
zucchini-nlp pushed a commit to zucchini-nlp/transformers that referenced this pull request May 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants