feat: add manylinux armv7l (#2052)

mayeut · web-flow · commit df6f8863900b · 2024-11-15T19:07:24.000-05:00
diff --git a/README.md b/README.md
@@ -24,16 +24,16 @@ What does it do?
 
 While cibuildwheel itself requires a recent Python version to run (we support the last three releases), it can target the following versions to build wheels:
 
-|                | macOS Intel | macOS Apple Silicon | Windows 64bit | Windows 32bit | Windows Arm64 | manylinux<br/>musllinux x86_64 | manylinux<br/>musllinux i686 | manylinux<br/>musllinux aarch64 | manylinux<br/>musllinux ppc64le | manylinux<br/>musllinux s390x | musllinux armv7l | Pyodide |
+|                | macOS Intel | macOS Apple Silicon | Windows 64bit | Windows 32bit | Windows Arm64 | manylinux<br/>musllinux x86_64 | manylinux<br/>musllinux i686 | manylinux<br/>musllinux aarch64 | manylinux<br/>musllinux ppc64le | manylinux<br/>musllinux s390x | manylinux<br/>musllinux armv7l | Pyodide |
 |----------------|----|-----|-----|-----|-----|----|-----|----|-----|-----|---|-----|
-| CPython 3.6    | ✅ | N/A | ✅  | ✅  | N/A | ✅  | ✅  | ✅ | ✅  | ✅  | ✅ | N/A |
-| CPython 3.7    | ✅ | N/A | ✅  | ✅  | N/A | ✅ | ✅  | ✅ | ✅  | ✅  | ✅ | N/A |
-| CPython 3.8    | ✅ | ✅  | ✅  | ✅  | N/A | ✅ | ✅  | ✅ | ✅  | ✅  | ✅ | N/A |
-| CPython 3.9    | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅ | ✅ | ✅  | ✅  | ✅ | N/A |
-| CPython 3.10   | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅  | ✅ | ✅  | ✅  | ✅ | N/A |
-| CPython 3.11   | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅  | ✅ | ✅  | ✅  | ✅ | N/A |
-| CPython 3.12   | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅  | ✅ | ✅  | ✅  | ✅  | ✅⁴ |
-| CPython 3.13³  | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅  | ✅ | ✅  | ✅  | ✅  | N/A |
+| CPython 3.6    | ✅ | N/A | ✅  | ✅  | N/A | ✅  | ✅  | ✅ | ✅  | ✅  | ✅⁵ | N/A |
+| CPython 3.7    | ✅ | N/A | ✅  | ✅  | N/A | ✅ | ✅  | ✅ | ✅  | ✅  | ✅⁵ | N/A |
+| CPython 3.8    | ✅ | ✅  | ✅  | ✅  | N/A | ✅ | ✅  | ✅ | ✅  | ✅  | ✅⁵ | N/A |
+| CPython 3.9    | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅ | ✅ | ✅  | ✅  | ✅⁵ | N/A |
+| CPython 3.10   | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅  | ✅ | ✅  | ✅  | ✅⁵ | N/A |
+| CPython 3.11   | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅  | ✅ | ✅  | ✅  | ✅⁵ | N/A |
+| CPython 3.12   | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅  | ✅ | ✅  | ✅  | ✅⁵  | ✅⁴ |
+| CPython 3.13³  | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅  | ✅ | ✅  | ✅  | ✅⁵  | N/A |
 | PyPy 3.7 v7.3  | ✅ | N/A | ✅  | N/A | N/A | ✅¹ | ✅¹  | ✅¹ | N/A | N/A | N/A | N/A |
 | PyPy 3.8 v7.3  | ✅ | ✅  | ✅  | N/A | N/A | ✅¹ | ✅¹  | ✅¹ | N/A | N/A | N/A | N/A |
 | PyPy 3.9 v7.3  | ✅ | ✅  | ✅  | N/A | N/A | ✅¹ | ✅¹  | ✅¹ | N/A | N/A | N/A | N/A |
@@ -43,6 +43,7 @@ While cibuildwheel itself requires a recent Python version to run (we support th
 <sup>² Windows arm64 support is experimental.</sup><br>
 <sup>³ Free-threaded mode requires opt-in using [`CIBW_FREE_THREADED_SUPPORT`](https://cibuildwheel.pypa.io/en/stable/options/#free-threaded-support).</sup><br>
 <sup>⁴ Experimental, not yet supported on PyPI, but can be used directly in web deployment. Use `--platform pyodide` to build.</sup><br>
+<sup>⁵ manylinux armv7l support is experimental. As there are no RHEL based image for this architecture, it's using an Ubuntu based image instead.</sup><br>
 
 - Builds manylinux, musllinux, macOS 10.9+ (10.13+ for Python 3.12+), and Windows wheels for CPython and PyPy
 - Works on GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, GitLab CI, and Cirrus CI
diff --git a/bin/generate_schema.py b/bin/generate_schema.py
@@ -127,6 +127,9 @@
   manylinux-aarch64-image:
     type: string
     description: Specify alternative manylinux / musllinux container images
+  manylinux-armv7l-image:
+    type: string
+    description: Specify alternative manylinux / musllinux container images
   manylinux-i686-image:
     type: string
     description: Specify alternative manylinux / musllinux container images
diff --git a/bin/update_docker.py b/bin/update_docker.py
@@ -6,6 +6,7 @@
 from pathlib import Path
 
 import requests
+from packaging.version import Version
 
 DIR = Path(__file__).parent.resolve()
 RESOURCES = DIR.parent / "cibuildwheel/resources"
@@ -53,6 +54,8 @@ class Image:
     Image("manylinux_2_28", "s390x", "quay.io/pypa/manylinux_2_28_s390x", None),
     Image("manylinux_2_28", "pypy_x86_64", "quay.io/pypa/manylinux_2_28_x86_64", None),
     Image("manylinux_2_28", "pypy_aarch64", "quay.io/pypa/manylinux_2_28_aarch64", None),
+    # manylinux_2_31 images
+    Image("manylinux_2_31", "armv7l", "ghcr.io/mayeut/manylinux_2_31", None),
     # musllinux_1_1 images
     Image("musllinux_1_1", "x86_64", "quay.io/pypa/musllinux_1_1_x86_64", None),
     Image("musllinux_1_1", "i686", "quay.io/pypa/musllinux_1_1_i686", None),
@@ -91,6 +94,21 @@ class Image:
             for (name, info) in tags_dict.items()
             if info["manifest_digest"] == latest_tag["manifest_digest"]
         )
+    elif image.image_name.startswith("ghcr.io/"):
+        repository = image.image_name[8:]
+        response = requests.get(
+            "https://ghcr.io/token", params={"scope": f"repository:{repository}:pull"}
+        )
+        response.raise_for_status()
+        token = response.json()["token"]
+        response = requests.get(
+            f"https://ghcr.io/v2/{repository}/tags/list",
+            headers={"Authorization": f"Bearer {token}"},
+        )
+        response.raise_for_status()
+        ghcr_tags = [(Version(tag), tag) for tag in response.json()["tags"] if tag != "latest"]
+        ghcr_tags.sort(reverse=True)
+        tag_name = ghcr_tags[0][1]
     else:
         response = requests.get(f"https://hub.docker.com/v2/repositories/{image.image_name}/tags")
         response.raise_for_status()
diff --git a/cibuildwheel/logger.py b/cibuildwheel/logger.py
@@ -23,6 +23,7 @@
     "manylinux_aarch64": "manylinux aarch64",
     "manylinux_ppc64le": "manylinux ppc64le",
     "manylinux_s390x": "manylinux s390x",
+    "manylinux_armv7l": "manylinux armv7l",
     "musllinux_x86_64": "musllinux x86_64",
     "musllinux_i686": "musllinux i686",
     "musllinux_aarch64": "musllinux aarch64",
diff --git a/cibuildwheel/resources/build-platforms.toml b/cibuildwheel/resources/build-platforms.toml
@@ -49,6 +49,15 @@ python_configurations = [
   { identifier = "cp312-manylinux_s390x", version = "3.12", path_str = "/opt/python/cp312-cp312" },
   { identifier = "cp313-manylinux_s390x", version = "3.13", path_str = "/opt/python/cp313-cp313" },
   { identifier = "cp313t-manylinux_s390x", version = "3.13", path_str = "/opt/python/cp313-cp313t" },
+  { identifier = "cp36-manylinux_armv7l", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
+  { identifier = "cp37-manylinux_armv7l", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
+  { identifier = "cp38-manylinux_armv7l", version = "3.8", path_str = "/opt/python/cp38-cp38" },
+  { identifier = "cp39-manylinux_armv7l", version = "3.9", path_str = "/opt/python/cp39-cp39" },
+  { identifier = "cp310-manylinux_armv7l", version = "3.10", path_str = "/opt/python/cp310-cp310" },
+  { identifier = "cp311-manylinux_armv7l", version = "3.11", path_str = "/opt/python/cp311-cp311" },
+  { identifier = "cp312-manylinux_armv7l", version = "3.12", path_str = "/opt/python/cp312-cp312" },
+  { identifier = "cp313-manylinux_armv7l", version = "3.13", path_str = "/opt/python/cp313-cp313" },
+  { identifier = "cp313t-manylinux_armv7l", version = "3.13", path_str = "/opt/python/cp313-cp313t" },
   { identifier = "pp37-manylinux_aarch64", version = "3.7", path_str = "/opt/python/pp37-pypy37_pp73" },
   { identifier = "pp38-manylinux_aarch64", version = "3.8", path_str = "/opt/python/pp38-pypy38_pp73" },
   { identifier = "pp39-manylinux_aarch64", version = "3.9", path_str = "/opt/python/pp39-pypy39_pp73" },
diff --git a/cibuildwheel/resources/cibuildwheel.schema.json b/cibuildwheel/resources/cibuildwheel.schema.json
@@ -296,6 +296,11 @@
       "description": "Specify alternative manylinux / musllinux container images",
       "title": "CIBW_MANYLINUX_AARCH64_IMAGE"
     },
+    "manylinux-armv7l-image": {
+      "type": "string",
+      "description": "Specify alternative manylinux / musllinux container images",
+      "title": "CIBW_MANYLINUX_ARMV7L_IMAGE"
+    },
     "manylinux-i686-image": {
       "type": "string",
       "description": "Specify alternative manylinux / musllinux container images",
@@ -562,6 +567,9 @@
           "manylinux-aarch64-image": {
             "$ref": "#/properties/manylinux-aarch64-image"
           },
+          "manylinux-armv7l-image": {
+            "$ref": "#/properties/manylinux-armv7l-image"
+          },
           "manylinux-i686-image": {
             "$ref": "#/properties/manylinux-i686-image"
           },
@@ -656,6 +664,9 @@
         "manylinux-aarch64-image": {
           "$ref": "#/properties/manylinux-aarch64-image"
         },
+        "manylinux-armv7l-image": {
+          "$ref": "#/properties/manylinux-armv7l-image"
+        },
         "manylinux-i686-image": {
           "$ref": "#/properties/manylinux-i686-image"
         },
diff --git a/cibuildwheel/resources/defaults.toml b/cibuildwheel/resources/defaults.toml
@@ -30,6 +30,7 @@ manylinux-i686-image = "manylinux2014"
 manylinux-aarch64-image = "manylinux2014"
 manylinux-ppc64le-image = "manylinux2014"
 manylinux-s390x-image = "manylinux2014"
+manylinux-armv7l-image = "manylinux_2_31"
 manylinux-pypy_x86_64-image = "manylinux2014"
 manylinux-pypy_i686-image = "manylinux2014"
 manylinux-pypy_aarch64-image = "manylinux2014"
diff --git a/cibuildwheel/resources/pinned_docker_images.cfg b/cibuildwheel/resources/pinned_docker_images.cfg
@@ -53,5 +53,6 @@ manylinux_2_24 = quay.io/pypa/manylinux_2_24_aarch64:2022-12-26-0d38463
 manylinux_2_28 = quay.io/pypa/manylinux_2_28_aarch64:2024.11.10-1
 
 [armv7l]
+manylinux_2_31 = ghcr.io/mayeut/manylinux_2_31:2024.11.10-1
 musllinux_1_2 = quay.io/pypa/musllinux_1_2_armv7l:2024.11.10-1
 
diff --git a/cibuildwheel/util.py b/cibuildwheel/util.py
@@ -86,6 +86,7 @@ class EnableGroups(enum.Enum):
     "aarch64",
     "ppc64le",
     "s390x",
+    "armv7l",
     "pypy_aarch64",
     "pypy_i686",
 )
diff --git a/docs/faq.md b/docs/faq.md
@@ -10,7 +10,7 @@ title: Tips and tricks
 
 Linux wheels are built in [`manylinux`/`musllinux` containers](https://github.com/pypa/manylinux) to provide binary compatible wheels on Linux, according to [PEP 600](https://www.python.org/dev/peps/pep-0600/) / [PEP 656](https://www.python.org/dev/peps/pep-0656/). Because of this, when building with `cibuildwheel` on Linux, a few things should be taken into account:
 
--   Programs and libraries are not installed on the CI runner host, but rather should be installed inside the container - using `yum` for `manylinux2010` or `manylinux2014`, `apt-get` for `manylinux_2_24`, `dnf` for `manylinux_2_28` and `apk` for `musllinux_1_1` or `musllinux_1_2`, or manually. The same goes for environment variables that are potentially needed to customize the wheel building.
+-   Programs and libraries are not installed on the CI runner host, but rather should be installed inside the container - using `yum` for `manylinux2010` or `manylinux2014`, `apt-get` for `manylinux_2_24`/`manylinux_2_31`, `dnf` for `manylinux_2_28` and `apk` for `musllinux_1_1` or `musllinux_1_2`, or manually. The same goes for environment variables that are potentially needed to customize the wheel building.
 
     `cibuildwheel` supports this by providing the [`CIBW_ENVIRONMENT`](options.md#environment) and [`CIBW_BEFORE_ALL`](options.md#before-all) options to setup the build environment inside the running container.
 
diff --git a/docs/options.md b/docs/options.md
diff --git a/test/utils.py b/test/utils.py

Original file line number	Diff line number	Diff line change
`@@ -86,6 +86,7 @@ class EnableGroups(enum.Enum):`
`86`	`86`	`"aarch64",`
`87`	`87`	`"ppc64le",`
`88`	`88`	`"s390x",`
	`89`	`+ "armv7l",`
`89`	`90`	`"pypy_aarch64",`
`90`	`91`	`"pypy_i686",`
`91`	`92`	`)`