Skip to content

Copying large files to remote nodes in treemode is slow #557

@luxiaoyong

Description

@luxiaoyong

I use the treemode to copy a 136MB file to 142 remote nodes with the following command, which took 36 minutes:
clush -o-q -w "node1 node2" --copy test.tar --dest /tmp/
However, when I turn off the treemode, the same copy only took 20 seconds. Sometimes, copying large files may freeze (occasionally), but there are no error messages during the copying process. Could you please explain what protocol treemode uses for large file copying? Is it possible to optimize this feature?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions