Read (and write) from HF Storage Buckets: load raw data, process and save to Dataset Repos

[lhoestq]

TODO:

• load_dataset("csv", data_files=["hf://buckets/..."])
• load_dataset("buckets/...")
• Dataset.to_csv("hf://buckets/..."), to_json, to_parquet, etc.
• DatasetDict.push_to_hub("buckets/...")
• Dataset.push_to_hub("buckets/...")
• DatasetDict.push_to_hub("buckets/...")
• IterableDataset.push_to_hub("buckets/...")
• IterableDatasetDict.push_to_hub("buckets/...")
• tests
• docs

e.g.

raw data in bucket to dataset repo

from datasets import load_dataset

ds = load_dataset("json", data_files=["hf://buckets/..."])
ds = ds.map(...).filter(...)
ds.push_to_hub("username/dataset_name")

(also taking the opportunity to factorize the code a bit)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.