Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove use of platform-site for other-arch macOS wheel installs. #2163

Merged
merged 6 commits into from
Feb 18, 2025

Conversation

freakboy3742
Copy link
Member

A follow up on #2155.

macOS was previously using a platform-site to allow for the installation of 'other-arch' binary wheel installations (i.e., installing x86_64 wheels when running on arm64 when a universal2 wheel isn't available). With the platform-site being replaced with platform-config, this gives us an opportunity to remove the use of the platform-site hack on macOS, using native pip --platform handling instead.

This forces the use of binary wheels on macOS - but that's broadly advisable anyway, and avoids having to handle pip's error messages when a local wheel compilation fails.

Also includes a tweak to the Apple input filters to catch a change in log preamble output that iOS started generating with (AFIACT) Xcode 16.

PR Checklist:

  • All new features have been tested
  • All new features have been documented
  • I have read the CONTRIBUTING.md file
  • I will abide by the code of conduct

@freakboy3742
Copy link
Member Author

freakboy3742 commented Feb 18, 2025

@mhsmith This has some overlap with #2162, in that this PR adds a new platform quirk, and that PR moves the location of platform quirks to the common macOS folder. As a stop-gap measure, I've added the platform quirk for this PR directly to the base macOS file, so it's an easier merge.

This change is a blocker on landing the updated Python 3.13 support package, which is needed to submit the cibuildwheel iOS patch.

@rmartin16
Copy link
Member

fwiw, a list of packages on PyPI serving a lot of sdists (published feb 2024): https://discuss.python.org/t/358-most-popular-python-packages-have-wheels/43558/11

The list isn't perfect for this because it includes packages with macOS wheels but still have high sdists downloads. However, pysftp, as an example, I don't think would be usable after this change.

Note: please ignore if I'm interpreting "forces the use of binary wheels on macOS" incorrectly.

@freakboy3742
Copy link
Member Author

fwiw, a list of packages on PyPI serving a lot of sdists (published feb 2024): https://discuss.python.org/t/358-most-popular-python-packages-have-wheels/43558/11

The list isn't perfect for this because it includes packages with macOS wheels but still have high sdists downloads. However, pysftp, as an example, I don't think would be usable after this change.

That's correct - it wouldn't be. However, pip wheel pysftp --no-deps produces a py3-none-any wheel. There is no reason the maintainers shouldn't be publishing a wheel, and the fix for Briefcase users is fairly straightforward in the meantime.

I haven't checked all the packages mentioned on that thread, but many of them (future, pyyaml) now have wheels (many of them py3-none-any wheels).

I only found two that were potentially concerning.

psycopg2 only provides wheels for Windows. However, the install message also explicitly calls out psycopg2-binary as the package you probably want, so the problem doesn't exist in practice.

mysqlclient does the same. The maintainer is on the thread you've linked explaining why there's no wheels (essentially Maria/MySQL compatibility)... but saying they'd consider adding wheels. (With my opinionated DB hat on - I almost consider anything that discourages MySQL usage to be a net good in the world 😝 )

The remaining ones that don't have wheels (pyspark, docopt, ...) produces a py3-none-any wheel from pip wheel, same as pystftp. I can't think of any good reason that they don't publish a wheel, other than not being aware that they should.

I only found one (pycrypto) that doesn't produce a py3-none-any wheel - but it produces a universal wheel from pip wheel pycrypto --no-deps. However, the package hasn't been updated in over 12 years - if you're using that package for cryptography at this point, you've got much bigger problems. I suspect it's an example of a package that it high on the download list because of the name, not because it's actually used anywhere.

I'm sure there are others - but at the end of the day, I'm comfortable to use what limited influence BeeWare/Briefcase has as a project to encourage laggards to add add wheels to their release procedures. There's very little argument for not producing wheels, and plenty of arguments why they should be actively encouraged.

@freakboy3742 freakboy3742 requested a review from mhsmith February 18, 2025 04:04

Briefcase *cannot* install packages published as source tarballs into a macOS app, even
if the package is a pure Python package that would produce a ``py3-none-any`` wheel.
This is an inherent limitation in the use of source tarballs as a distribution format.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not really true; it's a limitation in Briefcase.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll stand by this statement. It's not possible to tell purely from a tarball whether a package will produce a binary or non-binary wheel, or to establish anything about the build environments that are supported by the project. That's an inherent limitation in source tarballs as a distribution format for Python artefacts.

@mhsmith
Copy link
Member

mhsmith commented Feb 18, 2025

This forces the use of binary wheels on macOS

I don't follow why this is necessary. This PR changes the way in which the "other" architecture is installed, but that was already using "--only-binary", ":all:". Why do we need to use it on the "current" architecture as well?

@freakboy3742
Copy link
Member Author

This forces the use of binary wheels on macOS

I don't follow why this is necessary. This PR changes the way in which the "other" architecture is installed, but that was already using "--only-binary", ":all:". Why do we need to use it on the "current" architecture as well?

For consistency. If you have one user building on x86_64, and one on ARM64, you can get different results depending on which is the host platform and exactly which binary wheels are available, because only the "off" platform is enforcing a "binary only" limitation.

@freakboy3742 freakboy3742 requested a review from mhsmith February 18, 2025 22:19
@freakboy3742
Copy link
Member Author

@mhsmith As discussed in person, I've added an explanatory comment for why we're using --only-binary even though it doesn't seem strictly necessary.

@mhsmith mhsmith merged commit 228f7c7 into beeware:main Feb 18, 2025
57 checks passed
freakboy3742 added a commit to freakboy3742/briefcase that referenced this pull request Feb 19, 2025
@freakboy3742 freakboy3742 deleted the macos-platform-site branch February 19, 2025 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants