Skip to content

[data][bug] Dataset.context not being sealed after creation #41573

@raulchen

Description

@raulchen

Ideally, datasets should capture global DataContext when the dataset is created for the first time.
However, a lot of data code is using DataContext.get_current(), instead of the captured DataContext.
This prevents using multiple datasets with different DataContexts.

#41569 is the first attempt to mitigate this issue. But it only fixes the issue for training jobs, where multiple datasets will be propagated to different SplitCoordinator actors for execution.

If multiple Datasets are running in the same process. This bug still exists.

Metadata

Metadata

Assignees

Labels

P1Issue that should be fixed within a few weeksbugSomething that is supposed to be working; but isn'tdataRay Data-related issuesstability

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions