-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-18036: [Packaging] Build Python wheel for musllinux #45470
base: main
Are you sure you want to change the base?
Conversation
Thanks for opening a pull request! If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project. Then could you also rename the pull request title in the following format?
or
See also: |
@github-actions crossbow submit wheel-musllinux-* |
Revision: 732fc35 Submitted crossbow builds: ursacomputing/crossbow @ actions-d80189b582 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR
This one should be investigated, as it ends with a crash in uWSGI, even though the test is meant to check that uWSGI doesn't crash.
This one looks like the test is too strict (it assumes that Linux implies glibc), we should probably relax it on musllinux. |
7c40829
to
8ce0f68
Compare
This looks related with #44071. It's leaking S3 structures on purpose but it's failing. I was able to check that it failed when calling EDIT: I was able to make this successful by not leaking
Yes, I checked the glibc malloc source code and they have the malloc_stats method but musl implementation does not expose the same method. Because of that I added a new condition to only check the stderr if musl is not detected on linux. |
I get the following error when trying to run the new Alpine Python container: $ archery docker run alpine-linux-python bash
Traceback (most recent call last):
...
ValueError: Found errors with docker-compose:
- Service `alpine-linux-cpp - alpine-linux-python` is defined in `x-hierarchy` bot not in `services`
- Service `alpine-linux-cpp` is defined in `services` but not in `x-hierarchy`
- Service `alpine-linux-python` is defined in `services` but not in `x-hierarchy` |
It might, but it would be nice to get a gdb backtrace of the uWSGI crash before going any further. It is weird that creating a piece of leaked memory would trigger a crash. |
Here is the stacktrace from gdb: (gdb) bt full
#0 0x0000f1ed0376e7e0 in ?? () from /usr/local/bin/../lib/libpython3.9.so.1.0
No symbol table info available.
#1 0x0000f1ed0376f630 in ?? () from /usr/local/bin/../lib/libpython3.9.so.1.0
No symbol table info available.
#2 0x0000f1ed037cc734 in _Py_VaBuildStack () from /usr/local/bin/../lib/libpython3.9.so.1.0
No symbol table info available.
#3 0x0000f1ed03717998 in ?? () from /usr/local/bin/../lib/libpython3.9.so.1.0
No symbol table info available.
#4 0x0000f1ed0378d9a8 in PyObject_CallFunction () from /usr/local/bin/../lib/libpython3.9.so.1.0
No symbol table info available.
#5 0x0000f1ed037c7468 in PyImport_Import () from /usr/local/bin/../lib/libpython3.9.so.1.0
No symbol table info available.
#6 0x0000f1ed037c737c in PyImport_ImportModule () from /usr/local/bin/../lib/libpython3.9.so.1.0
No symbol table info available.
#7 0x0000f1ed0321aa60 in get_uwsgi_pydict () from /usr/local/lib/python3.9/site-packages/pyuwsgi.cpython-39-aarch64-linux-gnu.so
No symbol table info available.
#8 0x0000f1ed03217c5c in uwsgi_python_atexit () from /usr/local/lib/python3.9/site-packages/pyuwsgi.cpython-39-aarch64-linux-gnu.so
No symbol table info available.
#9 0x0000f1ed03204928 in uwsgi_plugins_atexit () from /usr/local/lib/python3.9/site-packages/pyuwsgi.cpython-39-aarch64-linux-gnu.so
No symbol table info available.
#10 0x0000f1ed0396deb8 in ?? () from /lib/ld-musl-aarch64.so.1
No symbol table info available.
#11 0x0000f1ed032b99f0 in ?? () from /usr/local/lib/python3.9/site-packages/pyuwsgi.cpython-39-aarch64-linux-gnu.so
No symbol table info available. |
This is weird, as it doesn't seem to call into Arrow at all? The entire stacktrace is in the Python interpreter. |
I think we can simply decide to skip the uwsgi-based test on musllinux, it's not a very important one anyway. |
@nveloso Thanks for this great first contribution to Arrow! Sidenote: the wheel builds seem to be pretty inefficient, are we building libarrow 12 times instead of once for amd64 and once for arm64? Of course not something for this PR :) |
Sure! Thanks for all the reviews so far!
That's weird... I don't see a reason to why that's happening. How did you check it's building libarrow 12 times for amd64? |
@github-actions crossbow submit wheel-musllinux-* |
This comment was marked as outdated.
This comment was marked as outdated.
@nveloso That's just because we build libarrow for each python version (i.e. each line in this table is a job and a build of libarrow), which is not necessary, we could build it once per arch and reuse in the wheel builds. But the current way the jobs are structured doesn't allow for that. But again, that's not on you or something for this PR :) |
…ofile.d. Use login shell to force the execution of those scripts
Only use NuGet on musl with amd64
6894b10
to
f61a9e0
Compare
@github-actions crossbow submit wheel-musllinux-* |
Revision: f61a9e0 Submitted crossbow builds: ursacomputing/crossbow @ actions-8e3be683cb |
It failed to build wheels for python3.13 because there is no alpine3.18 image with python3.13. Do you see any issues with upgrading alpine from 3.18 to 3.21? The free-threaded builds also failed. I have to add new dockerfiles for it. I'll also have to build python3.13 free-threaded because alpine does not have any image with that python version. |
It seems that "musllinux_1_2" is based on Alpine Linux 3.21: https://github.com/pypa/manylinux?tab=readme-ov-file#musllinux_1_2-alpine-linux-321-based-313-compatible Why do we use Alpine Linux 3.18 for building wheels? We use the official Docker image for "musllinux_1_2": quay.io/pypa/musllinux_1_2_x86_64 It must use Alpine Linux 3.21. |
we can override the default |
Yes we use the official Docker image for "musllinux_1_2": quay.io/pypa/musllinux_1_2_x86_64 for building the wheel. |
OK. Can we use something like the following? diff --git a/dev/tasks/python-wheels/github.linux.yml b/dev/tasks/python-wheels/github.linux.yml
index 0aa8bf9b23..6a7d86c321 100644
--- a/dev/tasks/python-wheels/github.linux.yml
+++ b/dev/tasks/python-wheels/github.linux.yml
@@ -31,6 +31,9 @@ jobs:
runs-on: ubuntu-24.04-arm
{% endif %}
env:
+ {% if linux_wheel_name == "musl" and linux_wheel_version == "1_2" %}
+ ALPINE_LINUX: "3.21"
+ {% endif %}
# archery uses these environment variables
{% if arch == "amd64" %}
ARCH: amd64 |
… binary does not work on that architecture Add free-threaded docker services to test the musl wheel
4e8c71a
to
870a758
Compare
@kou Can you please re-trigger the bot to see if the new changes work? |
@github-actions crossbow submit wheel-musllinux-* |
Revision: 870a758 Submitted crossbow builds: ursacomputing/crossbow @ actions-582a8974a4 |
@github-actions crossbow submit wheel-musllinux-* |
Revision: c10ed15 Submitted crossbow builds: ursacomputing/crossbow @ actions-e8aca3b5a3 |
ci/docker/python-free-threaded-wheel-musllinux-test-imports.dockerfile
Outdated
Show resolved
Hide resolved
Add packaging dependency to requirements-wheel-test.txt
ci/docker/python-free-threaded-wheel-musllinux-test-imports.dockerfile
Outdated
Show resolved
Hide resolved
ci/docker/python-free-threaded-wheel-musllinux-test-imports.dockerfile
Outdated
Show resolved
Hide resolved
ci/docker/python-free-threaded-wheel-musllinux-test-imports.dockerfile
Outdated
Show resolved
Hide resolved
Remove symlinks for python and related
Sort parameters of github action in alphabetical order
Rationale for this change
Please check #18036.
What changes are included in this PR?
Almost everything needed for building and testing python wheels for musllinux.
The service
python-wheel-musllinux-test-unittests
is currently broken (see next section) and I need to test running thealpine-linux-verify-rc
docker image.Are these changes tested?
I was able to successfully generate a musllinux wheel by running the following:
I was also able to run
python-wheel-musllinux-test-imports
with no errors.I'm not able to run
python-wheel-musllinux-test-unittests
because there are 2 tests failing and I don't think they are related with my changes. Can you please confirm?The failing tests are:
I believe the root cause is the same which is related to this:
/arrow/cpp/src/arrow/filesystem/s3fs.cc:3461: arrow::fs::FinalizeS3 was not called even though S3 was initialized. This could lead to a segmentation fault at exit !!! uWSGI process 3487 got Segmentation Fault !!!
Do you have any idea of what it might be?
Here are some logs of the failed tests:
There is also a lot of skipped tests (603) and I'm not sure if this is ok. Here is the final report:
============================================= 2 failed, 7200 passed, 603 skipped, 12 xfailed, 2 xpassed, 5 warnings in 80.21s (0:01:20) ==============================================
Are there any user-facing changes?
I don't think so.