Skip to content

feat: Linux ARM Release CICD for glibc 2.35 #781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Apr 3, 2025

Conversation

pangjunrong
Copy link
Collaborator

This change re-introduces the Linux ARM Build process to the release workflow and seeks to address the long-standing issue of missing connector-x wheels for Linux ARM in #240, #386, #506, #535, #673, #724 and more recently, #773.

With the introduction of GitHub ARM-based runners earlier this year in public preview, it is now possible to build our distribution natively in an ARM-based Ubuntu 22.04 environment, which should allow compatibility with OSes using glibc 2.35 and above.

A test run can be found in my fork here:
https://github.com/pangjunrong/connector-x/actions/runs/13912310291

@ghost ghost mentioned this pull request Mar 31, 2025
3 tasks
@pangjunrong
Copy link
Collaborator Author

Hey @wangxiaoying whenever you're free, could you take a look at this PR to re-enable Linux ARM releases for newer distributions?

@vc1492a
Copy link
Collaborator

vc1492a commented Apr 1, 2025

Sorely needed by our team, this would be a great addition! Thanks @pangjunrong!

KRB5_INCLUDE_DIR: /usr/include
KRB5_LIB_DIR: /usr/lib/aarch64-linux-gnu

- name: Copy J4RS Dependencies
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

j4rs should no longer be a dependency by default. We removed it from the release script previously for all the platforms.

We can remove Copy J4RS Dependencies and Rebuild Wheel (Native) here as well.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wangxiaoying, I've removed both steps and did a rerun on my fork for a sanity check and it looks good to go!

The new test run with J4RS removed can be found in my fork here: https://github.com/pangjunrong/connector-x/actions/runs/14243748048

Copy link
Collaborator

@vc1492a vc1492a Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is huge @pangjunrong! 🚀

@wangxiaoying
Copy link
Contributor

Hi @pangjunrong , thank you so much for the PR!

I added a minor comment related to the j4rs dependency. After fixing it we can merge it and have an alpha release for this!

@wangxiaoying wangxiaoying merged commit fd12fc9 into sfu-db:main Apr 3, 2025
2 checks passed
@wangxiaoying
Copy link
Contributor

I released a new alpha version, which can be installed by pip install connectorx==0.4.3a1. I don't have a linux arm instance. Can anyone try whether it works?

@pangjunrong
Copy link
Collaborator Author

pangjunrong commented Apr 4, 2025

I released a new alpha version, which can be installed by pip install connectorx==0.4.3a1. I don't have a linux arm instance. Can anyone try whether it works?

Hey @wangxiaoying, thanks for the quick turnaround! I've tested it in a Docker ARM64 Ubuntu 22.04 setup together with PostgresDB and it works (i.e., I am able to retrieve test data from the DB into a dataframe). The run and its logs can be found here:
https://github.com/pangjunrong/connector-x/actions/runs/14260363710

Separately and as expected, the library is not found for Ubuntu 20.04 and cannot be installed as it runs an older version of glibc (i.e., 2.31 instead of the minimum 2.35). Perhaps it will be prudent to update the documentation on this nuance, to provide more clarity on which Linux ARM64 distributions can run connector-x now. I work on the documentation update and submit a PR accordingly 😄

@vc1492a
Copy link
Collaborator

vc1492a commented Apr 4, 2025

FWIW, I built the my company's platform locally on macOS with this pre-release and it works, too! ❗ 🚀

@wangxiaoying
Copy link
Contributor

Thank you @pangjunrong @vc1492a for testing this!

@pangjunrong Yes, we can definitely add a clarification in the documentation about the glibc version. I have invited you as a collaborator of this repo. Hopefully it will make it easier for you to contribute!

I will release a stable version next week!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants