Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.11.0 #577

Merged
merged 16 commits into from
Oct 25, 2022
Merged

3.11.0 #577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
5b69f0b
Rebase patches
isuruf Aug 23, 2022
2268432
Update to 3.11.0rc1
isuruf Aug 23, 2022
3898fd0
remove unnecessary env vars
isuruf Aug 23, 2022
3df5991
Fix patch numbers
isuruf Aug 23, 2022
186d143
update 0012-Add-CondaEcosystemModifyDllSearchPath.patch
isuruf Aug 23, 2022
52ee12c
fix python jinja var
isuruf Aug 23, 2022
512b216
MNT: Re-rendered with conda-build 3.22.0, conda-smithy 3.21.1, and co…
Aug 23, 2022
4f4c64f
Set --with-build-python
isuruf Aug 23, 2022
40a581d
3.10 -> 3.11
isuruf Aug 24, 2022
8900866
Move Conda DLL patch to pylifecycle.c
isuruf Aug 24, 2022
ca0181e
3.10 -> 3.11
isuruf Aug 24, 2022
21f4471
Fix typo
isuruf Aug 24, 2022
071818b
pynche was removed
isuruf Aug 24, 2022
aad24dd
Update to 3.11.0rc2
chrisburr Sep 28, 2022
0805b1a
Disable downstreams tests due to automatic skiping not working
chrisburr Sep 28, 2022
8d4b02e
update to 3.11.0
isuruf Oct 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .ci_support/linux_64_openssl1.1.1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
readline:
- '8'
sqlite:
Expand Down
2 changes: 1 addition & 1 deletion .ci_support/linux_64_openssl3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
readline:
- '8'
sqlite:
Expand Down
2 changes: 1 addition & 1 deletion .ci_support/linux_aarch64_openssl1.1.1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
readline:
- '8'
sqlite:
Expand Down
2 changes: 1 addition & 1 deletion .ci_support/linux_aarch64_openssl3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
readline:
- '8'
sqlite:
Expand Down
2 changes: 1 addition & 1 deletion .ci_support/linux_ppc64le_openssl1.1.1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
readline:
- '8'
sqlite:
Expand Down
2 changes: 1 addition & 1 deletion .ci_support/linux_ppc64le_openssl3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
readline:
- '8'
sqlite:
Expand Down
6 changes: 3 additions & 3 deletions .ci_support/osx_64_openssl1.1.1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,15 @@ bzip2:
c_compiler:
- clang
c_compiler_version:
- '13'
- '14'
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cxx_compiler:
- clangxx
cxx_compiler_version:
- '13'
- '14'
libffi:
- '3.4'
macos_machine:
Expand Down Expand Up @@ -43,7 +43,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
readline:
- '8'
sqlite:
Expand Down
6 changes: 3 additions & 3 deletions .ci_support/osx_64_openssl3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,15 @@ bzip2:
c_compiler:
- clang
c_compiler_version:
- '13'
- '14'
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cxx_compiler:
- clangxx
cxx_compiler_version:
- '13'
- '14'
libffi:
- '3.4'
macos_machine:
Expand Down Expand Up @@ -43,7 +43,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
readline:
- '8'
sqlite:
Expand Down
6 changes: 3 additions & 3 deletions .ci_support/osx_arm64_openssl1.1.1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@ bzip2:
c_compiler:
- clang
c_compiler_version:
- '13'
- '14'
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cxx_compiler:
- clangxx
cxx_compiler_version:
- '13'
- '14'
libffi:
- '3.4'
macos_machine:
Expand Down Expand Up @@ -41,7 +41,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
readline:
- '8'
sqlite:
Expand Down
6 changes: 3 additions & 3 deletions .ci_support/osx_arm64_openssl3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@ bzip2:
c_compiler:
- clang
c_compiler_version:
- '13'
- '14'
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cxx_compiler:
- clangxx
cxx_compiler_version:
- '13'
- '14'
libffi:
- '3.4'
macos_machine:
Expand Down Expand Up @@ -41,7 +41,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
readline:
- '8'
sqlite:
Expand Down
2 changes: 1 addition & 1 deletion .ci_support/win_64_openssl1.1.1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
sqlite:
- '3'
target_platform:
Expand Down
2 changes: 1 addition & 1 deletion .ci_support/win_64_openssl3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ pin_run_as_build:
zlib:
max_pin: x.x
python:
- '3.10'
- '3.11'
sqlite:
- '3'
target_platform:
Expand Down
8 changes: 0 additions & 8 deletions recipe/build_base.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,6 @@ for /F "tokens=1,2 delims=." %%i in ("%PKG_VERSION%") do (
if NOT "%PY_VER%"=="%%i.%%j" exit 1
)

set "OPENSSL_DIR=%LIBRARY_PREFIX%"
set "SQLITE3_DIR=%LIBRARY_PREFIX%"
for /f "usebackq delims=" %%i in (`conda list -p %PREFIX% sqlite --no-show-channel-urls --json ^| findstr "version"`) do set SQLITE3_VERSION_LINE=%%i
for /f "tokens=2 delims==/ " %%i IN ('echo %SQLITE3_VERSION_LINE%') do (set SQLITE3_VERSION=%%~i)
echo SQLITE3_VERSION detected as %SQLITE3_VERSION%
Expand Down Expand Up @@ -95,17 +93,11 @@ xcopy /s /y /i %SRC_DIR%\Tools\demo %PREFIX%\Tools\demo
if errorlevel 1 exit 1
xcopy /s /y /i %SRC_DIR%\Tools\i18n %PREFIX%\Tools\i18n
if errorlevel 1 exit 1
xcopy /s /y /i %SRC_DIR%\Tools\pynche %PREFIX%\Tools\pynche
if errorlevel 1 exit 1
xcopy /s /y /i %SRC_DIR%\Tools\scripts %PREFIX%\Tools\scripts
if errorlevel 1 exit 1

del %PREFIX%\Tools\demo\README
if errorlevel 1 exit 1
del %PREFIX%\Tools\pynche\README
if errorlevel 1 exit 1
del %PREFIX%\Tools\pynche\pynche
if errorlevel 1 exit 1
del %PREFIX%\Tools\scripts\README
if errorlevel 1 exit 1
del %PREFIX%\Tools\scripts\dutree.doc
Expand Down
8 changes: 5 additions & 3 deletions recipe/build_base.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,8 @@ fi

export CPPFLAGS CFLAGS CXXFLAGS LDFLAGS

declare -a _common_configure_args

if [[ ${target_platform} == osx-* ]]; then
sed -i -e "s/@OSX_ARCH@/$ARCH/g" Lib/distutils/unixccompiler.py
fi
Expand Down Expand Up @@ -174,6 +176,7 @@ if [[ "${CONDA_BUILD_CROSS_COMPILATION}" == "1" ]]; then
export CONFIG_SITE=${PWD}/config.site
# This is needed for libffi:
export PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig
_common_configure_args+=(--with-build-python=${BUILD_PYTHON_PREFIX}/bin/python)
fi

# This causes setup.py to query the sysroot directories from the compiler, something which
Expand Down Expand Up @@ -235,7 +238,6 @@ if [[ ${CC} =~ .*-arm.* ]]; then
TEST_EXCLUDES+=(test_compiler)
fi

declare -a _common_configure_args
_common_configure_args+=(--prefix=${PREFIX})
_common_configure_args+=(--build=${BUILD})
_common_configure_args+=(--host=${HOST})
Expand Down Expand Up @@ -391,7 +393,7 @@ fi
ln -s ${PREFIX}/bin/python${VER} ${PREFIX}/bin/python
ln -s ${PREFIX}/bin/pydoc${VER} ${PREFIX}/bin/pydoc
# Workaround for https://github.com/conda/conda/issues/10969
ln -s ${PREFIX}/bin/python3.10 ${PREFIX}/bin/python3.1
ln -s ${PREFIX}/bin/python3.11 ${PREFIX}/bin/python3.1

# Remove test data to save space
# Though keep `support` as some things use that.
Expand Down Expand Up @@ -513,4 +515,4 @@ fi

# Workaround for old conda versions which fail to install noarch packages for Python 3.10+
# https://github.com/conda/conda/issues/10969
ln -s "${PREFIX}/lib/python3.10" "${PREFIX}/lib/python3.1"
ln -s "${PREFIX}/lib/python3.11" "${PREFIX}/lib/python3.1"
8 changes: 1 addition & 7 deletions recipe/conda_build_config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,8 @@
python:
- 3.10
- 3.11
python_impl:
- cpython
numpy:
- 1.16
c_compiler: # [win]
- vs2017 # [win]
cxx_compiler: # [win]
- vs2017 # [win]
vc: # [win]
- 14.1 # [win]
MACOSX_SDK_VERSION: # [osx and x86_64]
- 11.0 # [osx and x86_64]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intended? We can built it pretty nicely with 10.14 SDK on osx-64

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's intended. Building with 10.14 will give you a python interpreter that's broken on 11.0. For eg: #575

Copy link

@danpetry danpetry Sep 15, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, using 11.0 doesn't build for intel 64 macos here. It was ok when building python 3.10.4 but not python 3.11. We're having a look at why, if you have any insights that would be great.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, using 11.0 doesn't build for intel 64 macos here.

I don't understand what you are saying here. Details please.

Copy link

@danpetry danpetry Sep 16, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use macOS SDK version 11.0, we can't build python 3.11 for the osx-64 target. (which is 64-bit intel CPU Mac). However, for previous versions of Python (we're on 3.10.4 at the moment), we successfully used SDK version 11.0 to build for all targets (both ARM and Intel mac).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the error when you use SDK 11.0?

Copy link

@danpetry danpetry Sep 19, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_2092500kee/croots/recipe/python-split_1663247373378/work/Objects/longobject.c:5693:5: error: implicit declaration of function 'static_assert' is invalid in C99 [-Werror,-Wimplicit-function-declaration]

The only change between a package that builds successfully, and one which fails with this message, is changing the SDK version from 10.14 to 11.0. This is when building Python v3.11 rc 1, for osx-64. When building the same feedstock for osx-arm64, using SDK version 11.0, the build completes successfully.

The static_assert call in question was added somewhere between Python v3.10.4 and v3.11.

Copy link
Member

@h-vetinari h-vetinari Oct 25, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implicit declaration of function 'static_assert' is invalid in C99 [-Werror,-Wimplicit-function-declaration]

That's a consequence of upstream python switching to C11, which introduces a static_assert macro (xref upstream conversion PR).

Have you verified that you're compiling with -std=c11?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That topic is also discussed quite extensively in python/cpython#91731

56 changes: 27 additions & 29 deletions recipe/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{% set version = "3.10.6" %}
{% set version = "3.11.0" %}
{% set dev = "" %}
{% set dev_ = "" %}
{% set ver2 = '.'.join(version.split('.')[0:2]) %}
Expand Down Expand Up @@ -47,41 +47,40 @@ source:
{% else %}
- url: https://www.python.org/ftp/python/{{ version }}/Python-{{ version }}{{ dev }}.tar.xz
# md5 from: https://www.python.org/downloads/release/python-{{ ver3nd }}/
md5: afc7e14f7118d10d1ba95ae8e2134bf0
md5: fe92acfa0db9b9f5044958edb451d463
{% endif %}
patches:
- patches/0001-Win32-Change-FD_SETSIZE-from-512-to-2048.patch
- patches/0002-Win32-distutils-Add-support-to-cygwinccompiler-for-V.patch
- patches/0003-Support-cross-compiling-byte-code.patch
# https://github.com/python/cpython/pull/28501
- patches/0004-bpo-45258-search-for-isysroot-in-addition-to-sysroot.patch
- patches/0003-bpo-45258-search-for-isysroot-in-addition-to-sysroot.patch
# https://github.com/pypa/distutils/pull/54
- patches/0005-runtime_library_dir_option-Use-1st-word-of-CC-as-com.patch
- patches/0006-Win32-Do-not-download-externals.patch
- patches/0007-Fix-find_library-so-that-it-looks-in-sys.prefix-lib-.patch
- patches/0004-runtime_library_dir_option-Use-1st-word-of-CC-as-com.patch
- patches/0005-Win32-Do-not-download-externals.patch
- patches/0006-Fix-find_library-so-that-it-looks-in-sys.prefix-lib-.patch
# https://github.com/python/cpython/pull/28397
- patches/0008-bpo-22699-Allow-compiling-on-debian-ubuntu-with-a-di.patch
- patches/0009-Disable-registry-lookup-unless-CONDA_PY_ALLOW_REG_PA.patch
- patches/0010-Unvendor-openssl.patch
- patches/0011-Unvendor-sqlite3.patch
- patches/0007-bpo-22699-Allow-compiling-on-debian-ubuntu-with-a-di.patch
- patches/0008-Disable-registry-lookup-unless-CONDA_PY_ALLOW_REG_PA.patch
- patches/0009-Unvendor-openssl.patch
- patches/0010-Unvendor-sqlite3.patch
# https://github.com/pypa/distutils/pull/53
- patches/0012-Use-ranlib-from-env-if-env-variable-is-set.patch
- patches/0013-Add-CondaEcosystemModifyDllSearchPath.patch
- patches/0014-Add-d1trimfile-SRC_DIR-to-make-pdbs-more-relocatable.patch
- patches/0015-Doing-d1trimfile.patch
- patches/0011-Use-ranlib-from-env-if-env-variable-is-set.patch
- patches/0012-Add-CondaEcosystemModifyDllSearchPath.patch
- patches/0013-Add-d1trimfile-SRC_DIR-to-make-pdbs-more-relocatable.patch
- patches/0014-Doing-d1trimfile.patch
# https://github.com/python/cpython/pull/23523
- patches/0016-cross-compile-darwin.patch
- patches/0017-Fix-TZPATH-on-windows.patch
- patches/0015-cross-compile-darwin.patch
- patches/0016-Fix-TZPATH-on-windows.patch
# https://github.com/python/cpython/pull/24324
- patches/0018-Make-dyld-search-work-with-SYSTEM_VERSION_COMPAT-1.patch
- patches/0017-Make-dyld-search-work-with-SYSTEM_VERSION_COMPAT-1.patch
# https://github.com/pypa/distutils/pull/53
- patches/0019-Fix-LDSHARED-when-CC-is-overriden-on-Linux-too.patch
- patches/0020-Unvendor-bzip2.patch
- patches/0021-Unvendor-libffi.patch
- patches/0022-Unvendor-tcltk.patch
- patches/0023-unvendor-xz.patch
- patches/0024-unvendor-zlib.patch
- patches/0025-Do-not-pass-g-to-GCC-when-not-Py_DEBUG.patch
- patches/0018-Fix-LDSHARED-when-CC-is-overriden-on-Linux-too.patch
- patches/0019-Unvendor-bzip2.patch
- patches/0020-Unvendor-libffi.patch
- patches/0021-Unvendor-tcltk.patch
- patches/0022-unvendor-xz.patch
- patches/0023-unvendor-zlib.patch
- patches/0024-Do-not-pass-g-to-GCC-when-not-Py_DEBUG.patch

build:
number: {{ build_number }}
Expand Down Expand Up @@ -140,8 +139,6 @@ outputs:
# Putting these here means they get emitted to build_env_setup.{sh,bat} meaning we can launch IDEs
# after sourcing or calling that script without examine the contents of conda_build.{sh,bat} for
# important env. vars.
- OPENSSL_DIR=%PREFIX%\Library # [win]
- SQLITE3_DIR=%PREFIX%\Library # [win]
{% if 'conda-forge' in channel_targets %}
- CONDA_FORGE=yes
{% else %}
Expand Down Expand Up @@ -190,8 +187,9 @@ outputs:
test:
downstreams:
# Temporarily disable dowmstream tests on aarch64 as the solver causes timeouts on drone
- cython # [not aarch64]
- setuptools # [not aarch64]
# Fully disable while migrating as the automatic skip doesn't work
# - cython # [not aarch64]
# - setuptools # [not aarch64]
requires:
- ripgrep
- cmake
Expand Down
10 changes: 5 additions & 5 deletions recipe/patches/0001-Win32-Change-FD_SETSIZE-from-512-to-2048.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
From a9cd933d936a603aa2326fe57a73d086609603c3 Mon Sep 17 00:00:00 2001
From 79ed4c580d008ac554648f570209972a73c157f2 Mon Sep 17 00:00:00 2001
From: Ray Donnelly <mingw.android@gmail.com>
Date: Wed, 16 Aug 2017 11:53:55 +0100
Subject: [PATCH 01/25] Win32: Change FD_SETSIZE from 512 to 2048
Subject: [PATCH 01/24] Win32: Change FD_SETSIZE from 512 to 2048

https://github.com/ContinuumIO/anaconda-issues/issues/1241
---
Modules/selectmodule.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c
index 3afcb0e2a0..1ee953a6ae 100644
index 4eea928a26..e3c5d82722 100644
--- a/Modules/selectmodule.c
+++ b/Modules/selectmodule.c
@@ -33,7 +33,7 @@
@@ -38,7 +38,7 @@
FD_SETSIZE higher before this; e.g., via compiler /D switch.
*/
#if defined(MS_WINDOWS) && !defined(FD_SETSIZE)
Expand All @@ -22,5 +22,5 @@ index 3afcb0e2a0..1ee953a6ae 100644

#if defined(HAVE_POLL_H)
--
2.32.1 (Apple Git-133)
2.30.2

Loading