Skip to content

ASO/Rucio is not safe against multiple simultaneous tasks #9111

@belforte

Description

@belforte

Current code assumes that this instance of ASO/Rucio is the only one writing in a given Rucio publish container at any given time, i.e. that it has full control.
But it is possible (as I encountered in #9094 ) that two tasks run with same Pulication tag (i.e. same targed DBS-datsaset/Rucio-Publish-container) in which case we end up with files from different tasks in same DBS-block/Rucio-dataset.

This confuses other code (Publisher e.g.), developers and users since when using ASO/FTS we are consistent with the rule
"multiple tasks can publish in same DBS dataset, each tasks with a separate (set of) block(s)"

Some redesigning and refactoring of ASO/Rucio code is needed, and of its bookkeeping.

Which is also a good time to write down more description of how it works since https://cmscrab.docs.cern.ch/crab-design/crab-rucio-aso.html#rucio-dataset was not detailed/clear enough to catch this.

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions