`//src/test/py/bazel:py_test` is broken with Python 3.11 #20660

meteorcloudy · 2023-12-22T14:59:15Z

https://buildkite.com/bazel/bazel-bazel/builds/26029#018c915e-33cc-4272-9169-2722a7a94f6d

We recently update our mac machines on CI and python was upgraded to 3.11. This test started to fail with

File "/private/var/tmp/_bazel_buildkite/42e36ea1c1dcd25f4ab5af8d98bf3465/sandbox/darwin-sandbox/8220/execroot/_main/_tmp/9d5b13726e352fe4fa73b514c6621472/_bazel_buildkite/da87a3ce985221b3cb431edb4e3cae38/execroot/_main/bazel-out/darwin_x86_64-fastbuild/bin/a/a.runfiles/_main/a/a.py", line 1, in <module>
import b
ModuleNotFoundError: No module named 'b'

To reproduce on macOS

brew install python3
bazel test //src/test/py/bazel:py_test --test_filter=PyTest.testSmoke

The text was updated successfully, but these errors were encountered:

meteorcloudy · 2023-12-22T15:01:49Z

I debugged a bit with the sys.path printed

with python 3.9 as default python on PATH

ci@bk-imacpro-11 tmpdp3x5ecv % ./bazel-bin/a/a
['/private/var/tmp/_bazel_ci/ba891f1787a15980e5a85b9711097c39/sandbox/darwin-sandbox/3558/execroot/_main/_tmp/9d5b13726e352fe4fa73b514c6621472/tests_root/tmpdp3x5ecv/a', '/private/var/tmp/_bazel_ci/ba891f1787a15980e5a85b9711097c39/sandbox/darwin-sandbox/3558/execroot/_main/_tmp/9d5b13726e352fe4fa73b514c6621472/tests_root/tmpdp3x5ecv/bazel-bin/a/a.runfiles', '/private/var/tmp/_bazel_ci/ba891f1787a15980e5a85b9711097c39/sandbox/darwin-sandbox/3558/execroot/_main/_tmp/9d5b13726e352fe4fa73b514c6621472/tests_root/tmpdp3x5ecv/bazel-bin/a/a.runfiles/_main', '/private/var/tmp/_bazel_ci/ba891f1787a15980e5a85b9711097c39/sandbox/darwin-sandbox/3558/execroot/_main/_tmp/9d5b13726e352fe4fa73b514c6621472/tests_root/tmpdp3x5ecv/bazel-bin/a/a.runfiles/bazel_tools', '/Applications/Xcode14.2.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python39.zip', '/Applications/Xcode14.2.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9', '/Applications/Xcode14.2.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/lib-dynload', '/Applications/Xcode14.2.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/site-packages']
Hello, World

with python 3.11 as default python on PATH

ci@bk-imacpro-11 tmpdp3x5ecv % ./bazel-bin/a/a
['/private/var/tmp/_bazel_ci/ba891f1787a15980e5a85b9711097c39/sandbox/darwin-sandbox/3558/execroot/_main/_tmp/9d5b13726e352fe4fa73b514c6621472/tests_root/tmpdp3x5ecv/bazel-bin/a/a.runfiles', '/private/var/tmp/_bazel_ci/ba891f1787a15980e5a85b9711097c39/sandbox/darwin-sandbox/3558/execroot/_main/_tmp/9d5b13726e352fe4fa73b514c6621472/tests_root/tmpdp3x5ecv/bazel-bin/a/a.runfiles/_main', '/private/var/tmp/_bazel_ci/ba891f1787a15980e5a85b9711097c39/sandbox/darwin-sandbox/3558/execroot/_main/_tmp/9d5b13726e352fe4fa73b514c6621472/tests_root/tmpdp3x5ecv/bazel-bin/a/a.runfiles/bazel_tools', '/usr/local/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python311.zip', '/usr/local/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11', '/usr/local/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/lib-dynload', '/usr/local/lib/python3.11/site-packages']
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_ci/ba891f1787a15980e5a85b9711097c39/sandbox/darwin-sandbox/3558/execroot/_main/_tmp/9d5b13726e352fe4fa73b514c6621472/tests_root/tmpdp3x5ecv/./bazel-bin/a/a.runfiles/_main/a/a.py", line 3, in <module>
    import b
ModuleNotFoundError: No module named 'b'

/private/var/tmp/_bazel_ci/ba891f1787a15980e5a85b9711097c39/sandbox/darwin-sandbox/3558/execroot/_main/_tmp/9d5b13726e352fe4fa73b514c6621472/tests_root/tmpdp3x5ecv/a is missing with python 3.11.

meteorcloudy · 2023-12-22T15:18:19Z

I'm disabling this test case on macOS to make presubmit green

Due to #20660 RELNOTES: None PiperOrigin-RevId: 593115090 Change-Id: Ifc2a282dbd6dd8a3abfec987398388e5844af91c

rickeylev · 2023-12-22T21:37:11Z

This is probably due to PYTHONSAFEPATH being respected in python 3.11 (it isn't in earlier python versions). This is a good thing and is just exposing a non-hermetic behavior that was being relied upon.

Should be a fairly simple fix. Either the import line needs to change ("from something import x") or an imports = something setting needs to be added a py_* target somewhere

Due to bazelbuild#20660 RELNOTES: None PiperOrigin-RevId: 593115090 Change-Id: Ifc2a282dbd6dd8a3abfec987398388e5844af91c

Due to #20660 RELNOTES: None PiperOrigin-RevId: 593115090 Change-Id: Ifc2a282dbd6dd8a3abfec987398388e5844af91c

The test was failing because the `b.py` file couldn't be imported because, starting with Python 3.11, the `PYTHONSAFEPATH` environment variable (set by the bootstrap startup) is respected. This setting inhibits the default Python behavior of adding the main script's directory to sys.path. To fix, use `py_library.imports` to explicitly add the necessary directory to `sys.path`. Fixes bazelbuild#20660

rickeylev · 2024-01-03T21:22:15Z

Looks like a simple fix. I have #20738 going through CI.

*** Reason for rollback *** Rollforward after fixing the underlying problem in 9b027c8 *** Original change description *** Disable PyTest.testSmoke on macOS Due to #20660 RELNOTES: None PiperOrigin-RevId: 595998556 Change-Id: Idddf6149a918de4408a3fda016d605380a4ab6af

Due to #20660 RELNOTES: None Commit d9dc4fd PiperOrigin-RevId: 593115090 Change-Id: Ifc2a282dbd6dd8a3abfec987398388e5844af91c --------- Co-authored-by: Googler <pcloudy@google.com> Co-authored-by: Ian (Hee) Cha <heec@google.com>

The test was failing because the `b.py` file couldn't be imported because, starting with Python 3.11, the `PYTHONSAFEPATH` environment variable (set by the bootstrap startup) is respected. This setting inhibits the default Python behavior of adding the main script's directory to sys.path. To fix, use `py_library.imports` to explicitly add the necessary directory to `sys.path`. Fixes bazelbuild#20660 Closes bazelbuild#20738. PiperOrigin-RevId: 595498775 Change-Id: I0c9521b210fe9e2c40692727fc87d41995e0968a

python: fix bazel py_test testSmoke for Python 3.11+ The test was failing because the `b.py` file couldn't be imported because, starting with Python 3.11, the `PYTHONSAFEPATH` environment variable (set by the bootstrap startup) is respected. This setting inhibits the default Python behavior of adding the main script's directory to sys.path. To fix, use `py_library.imports` to explicitly add the necessary directory to `sys.path`. Fixes #20660 Closes #20738. PiperOrigin-RevId: 595498775 Change-Id: I0c9521b210fe9e2c40692727fc87d41995e0968a Commit 9b027c8 No public description PiperOrigin-RevId: 694487395 Change-Id: I3f3e154ebfa932721d917661d61a878477a8e574 Commit be2186f --------- Co-authored-by: Richard Levasseur <rlevasseur@google.com> Co-authored-by: Googler <pcloudy@google.com>

iancha1992 · 2025-01-23T00:03:30Z

A fix for this issue has been included in Bazel 7.5.0 RC2. Please test out the release candidate and report any issues as soon as possible.
If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=7.5.0rc2. Thanks!

meteorcloudy added type: bug untriaged team-Rules-Python Native rules for Python labels Dec 22, 2023

meteorcloudy changed the title ~~//src/test/py/bazel:py_test~~ //src/test/py/bazel:py_test is broken with Python 3.11 Dec 22, 2023

meteorcloudy assigned rickeylev Dec 22, 2023

copybara-service bot pushed a commit that referenced this issue Dec 22, 2023

Disable PyTest.testSmoke on macOS

d9dc4fd

Due to #20660 RELNOTES: None PiperOrigin-RevId: 593115090 Change-Id: Ifc2a282dbd6dd8a3abfec987398388e5844af91c

bazel-io pushed a commit to bazel-io/bazel that referenced this issue Jan 3, 2024

Disable PyTest.testSmoke on macOS

69b16c9

Due to bazelbuild#20660 RELNOTES: None PiperOrigin-RevId: 593115090 Change-Id: Ifc2a282dbd6dd8a3abfec987398388e5844af91c

bazel-io mentioned this issue Jan 3, 2024

[6.5.0] Disable PyTest.testSmoke on macOS #20729

Merged

bazel-io pushed a commit to bazel-io/bazel that referenced this issue Jan 3, 2024

Disable PyTest.testSmoke on macOS

2e6d384

Due to bazelbuild#20660 RELNOTES: None PiperOrigin-RevId: 593115090 Change-Id: Ifc2a282dbd6dd8a3abfec987398388e5844af91c

bazel-io mentioned this issue Jan 3, 2024

[7.0.1] Disable PyTest.testSmoke on macOS #20730

Closed

bazel-io pushed a commit to bazel-io/bazel that referenced this issue Jan 3, 2024

Disable PyTest.testSmoke on macOS

7963568

Due to bazelbuild#20660 RELNOTES: None PiperOrigin-RevId: 593115090 Change-Id: Ifc2a282dbd6dd8a3abfec987398388e5844af91c

bazel-io mentioned this issue Jan 3, 2024

[7.1.0] Disable PyTest.testSmoke on macOS #20731

Closed

Wyverald pushed a commit that referenced this issue Jan 3, 2024

Disable PyTest.testSmoke on macOS

3d66bc7

Due to #20660 RELNOTES: None PiperOrigin-RevId: 593115090 Change-Id: Ifc2a282dbd6dd8a3abfec987398388e5844af91c

rickeylev mentioned this issue Jan 3, 2024

python: fix bazel py_test testSmoke for Python 3.11+ #20738

Closed

rickeylev removed the untriaged label Jan 3, 2024

copybara-service bot closed this as completed in 9b027c8 Jan 3, 2024

iancha1992 mentioned this issue Dec 13, 2024

[7.5.0] Backport 9b027c8 and be2186f to fix windows CI failures #24691

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`//src/test/py/bazel:py_test` is broken with Python 3.11 #20660

`//src/test/py/bazel:py_test` is broken with Python 3.11 #20660

meteorcloudy commented Dec 22, 2023

meteorcloudy commented Dec 22, 2023 •

edited

Loading

meteorcloudy commented Dec 22, 2023

rickeylev commented Dec 22, 2023

rickeylev commented Jan 3, 2024

iancha1992 commented Jan 23, 2025

//src/test/py/bazel:py_test is broken with Python 3.11 #20660

//src/test/py/bazel:py_test is broken with Python 3.11 #20660

Comments

meteorcloudy commented Dec 22, 2023

meteorcloudy commented Dec 22, 2023 • edited Loading

meteorcloudy commented Dec 22, 2023

rickeylev commented Dec 22, 2023

rickeylev commented Jan 3, 2024

iancha1992 commented Jan 23, 2025

`//src/test/py/bazel:py_test` is broken with Python 3.11 #20660

`//src/test/py/bazel:py_test` is broken with Python 3.11 #20660

meteorcloudy commented Dec 22, 2023 •

edited

Loading