Skip to content

Only update topk filter when updated filter is more selective  #16432

Open
@Dandandan

Description

@Dandandan

Hm @adriangb another thing I wondered is update_filter does seem to take only the heap of the current partition into account, as in TopK (currently at least) each partition has it's own heap (of k items).

Perhaps we can compare against the current filter and only update the expression if it is greater / more selective?

Originally posted by @Dandandan in #15770 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceMake DataFusion faster

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions