Skip to content

Swap JDBC for Arrow ADBC #1756

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jaidisido opened this issue Nov 9, 2022 · 3 comments
Closed

Swap JDBC for Arrow ADBC #1756

jaidisido opened this issue Nov 9, 2022 · 3 comments
Labels
closing-soon feature

Comments

@jaidisido
Copy link
Contributor

jaidisido commented Nov 9, 2022 •
edited
Loading

Leveraging the Arrow ADBC Driver has a number of advantages:

  • A single, universal driver would replace all existing, proprietary drivers (mssql, postgres...)
  • Better performance due to not processing each row separately and tapping into pyarrow to avoid serialization/deserialization

Additional context
https://arrow.apache.org/docs/dev/format/ADBC.html

P.S. Please do not attach files as it's considered a security risk. Add code snippets directly in the message body as much as possible.
@jaidisido jaidisido added this to the 3.0.0 milestone Nov 9, 2022
@jaidisido jaidisido changed the title Replace existing database connections with JDBC Driver for Arrow Flight SQL Swap JDBC for Arrow ADBC Jan 26, 2023
@jaidisido jaidisido modified the milestones: 3.0.0, 3.1.0 May 3, 2023
@LeonLuttenberger LeonLuttenberger modified the milestones: 3.1.0, 3.2.0 May 5, 2023
@kukushking kukushking removed this from the 3.2.0 milestone Jun 6, 2023
@jaidisido jaidisido added this to the 3.3.0 milestone Jun 6, 2023
@jaidisido jaidisido modified the milestones: 3.3.0, 3.4.0 Aug 1, 2023
@LeonLuttenberger LeonLuttenberger modified the milestones: 3.4.0, 3.5.0 Sep 20, 2023
@LeonLuttenberger LeonLuttenberger removed this from the 3.5.0 milestone Feb 20, 2024
@LeonLuttenberger
Copy link
Contributor

The implementation for this is in progress for Postgresql (see development branch). However, we are currently blocked by this bug in Pandas: pandas-dev/pandas#57539

@LeonLuttenberger LeonLuttenberger mentioned this issue Apr 22, 2024
Closed
@LeonLuttenberger LeonLuttenberger linked a pull request Apr 22, 2024 that will close this issue
feat: add ArrowDBC module #2786
Closed
@github-actions GitHub Actions
Copy link

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 7 days it will automatically be closed.

@github-actions GitHub Actions
Copy link

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 7 days it will automatically be closed.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jul 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
closing-soon feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: add ArrowDBC module aws/aws-sdk-pandas
3 participants
@kukushking @LeonLuttenberger @jaidisido