Skip to content

Better UX for non-schema-matching geoparquet batches #181

@gadomski

Description

@gadomski

From an internal chat:

i'm curious how we can make a good UX around non-schema-matching batches. if you're doing a large workflow, it'd be really nice to "peel off" all the non-compliant items, put them into that initial schema-inference batch, and then re-run

This should be doable:

  • Down in Rust, the writer can accumulate all failed batches ... this could lead to a big memory blow-up, so maybe instead we should just return the failed batch right away?
  • The user could them use add failed batch to the initial "schema inference" batch and re-run

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions