Skip to content

Zero-copy select and return for pyarrow tables #95

Open
@danthegoodman1

Description

@danthegoodman1

Being able to select pyarrow tables without copying, as well as accessing results as pyarrow tables without copying would be massively beneficial to building low-latency ETL pipelines and other data processing pipelines.

Specifically having streaming support would be massive too: https://duckdb.org/2021/12/03/duck-arrow.html#streaming-data-fromto-arrow as this would greatly reduce the required memory usage for queries and post-processing of data

Metadata

Metadata

Assignees

No one assigned

    Labels

    ArrowApache Arrow supportenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions