pydebug.h not found on Ubuntu 22.04 with Python 3.10

[he-hesce]

When trying to build PythonQt on Ubuntu 22.04 LTS with distribution-shipped Python 3.10, the header pydebug.h is not found automatically. This is because it has been moved to a subdirectory cpython either by Ubuntu or by Python 3.10. This new include path is not output by python3-config --includes used in build/python.prf so the expectation might be that one is supposed to use #include <cpython/pydebug.h> since Python 3.10. There might also be other header files which have moved but this is the first reported by the compiler. With Python 3.9 (RHEL 9) and earlier (v3.8 on Ubuntu 20.04 LTS, v3.6 on RHEL8, v2.7 on RHEL 7, v2.6 on RHEL 6) this specific file is not located in the cpython subdirectory and hence compilation works.

Compiler error:

PythonQt.cpp:61:10: fatal error: pydebug.h: No such file or directory
   61 | #include <pydebug.h>
      |          ^~~~~~~~~~~
compilation terminated.

System information:

$ cat /etc/os-release | grep PRETTY_NAME
PRETTY_NAME="Ubuntu 22.04.1 LTS"

$ gcc --version
gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ python3 --version
Python 3.10.6

$ qmake --version
QMake version 3.1
Using Qt version 5.15.3 in /usr/lib/x86_64-linux-gnu

$ python3-config --includes
-I/usr/include/python3.10 -I/usr/include/python3.10

$ find /usr -name pydebug.h
/usr/include/python3.10/cpython/pydebug.h

[he-hesce]

Until this can be resolved properly in a correct way, a workaround (hack) for anyone running into this is on Ubuntu 22.04 (and nowhere else) is add to the end of build/python.prf:

unix {
  OS_RELEASE_ID = $$system(eval "$(env echo -e $(cat /etc/os-release))" printenv ID)
  OS_RELEASE_ID_LIKE = $$system(eval "$(env echo -e $(cat /etc/os-release))" printenv ID_LIKE)
  OS_RELEASE_VERSION_ID = $$system(eval "$(env echo -e $(cat /etc/os-release))" printenv VERSION_ID)

  equals(OS_RELEASE_ID, "ubuntu") {
    equals(OS_RELEASE_VERSION_ID, "22.04") {
      unix:QMAKE_CXXFLAGS += "-I/usr/include/python3.10/cpython"
    }
  }
}

Above if of course an ugly hack, non-portable, et cetera, and only an emergency workaround.

[mrbean-bremen]

This is a change in Python 3.10, introduced with this commit, so we probably have to adapt the build script depending on the Python version.

[he-hesce]

Ah... the Python people breaking backward compatibility and forcing everyone to update their code. Who could have expected that. (sarcasm off)

How about including the Python.h file instead? As discussed in the same ticket.

[mrbean-bremen]

Yes, this would be the correct fix, I think.
BTW, I applied your workaround in the CI build, and the build still fails because it now can't find qmetaobjectbuilder_p.h - which indeed seems not to be present, besides qtbase5-private-dev is installed. I guess you got it running - do you know what I'm missing?

[mrbean-bremen]

Actually, according to this PR pydebug.h is not used and can just be removed. If I do this, I still get the unrelated error mentioned above, but that seems not to be related.

[he-hesce]

Great. As long as we don't include any of the other moved files.

[he-hesce]

Hmm, critical is to do my hack-workaround last in the file so that the cpython directory is last in search. Otherwise I got some other error as header files were in both directories or something and only the one in the "top" directory worked. So it was critical that it found the top level header files first before the ones in cpython subdirectory.

However, for your issue with qmetaobjectbuilder_p.h it indeed appears to you need to install qtbase5-private-dev:

$ find /usr -name qmetaobjectbuilder_p.h
/usr/include/x86_64-linux-gnu/qt5/QtCore/5.15.3/QtCore/private/qmetaobjectbuilder_p.h

$ dpkg-query -S /usr/include/x86_64-linux-gnu/qt5/QtCore/5.15.3/QtCore/private/qmetaobjectbuilder_p.h
qtbase5-private-dev:amd64: /usr/include/x86_64-linux-gnu/qt5/QtCore/5.15.3/QtCore/private/qmetaobjectbuilder_p.h

[he-hesce]

I would suggest install all Qt packages (libqt5-*-dev, libqt5*5, qt*5-dev, q*-qt5) on Ubuntu to also ensure the generated interface is complete. Same on RHEL (and derivates) with qt5-*-devel.

I wish Ubuntu people could use a uniform package naming scheme (as now it is also over the place) like the RHEL people.

[he-hesce]

This is what I install on Ubuntu 22.04:

libqt5core5a
libqt5gui5
qhelpgenerator-qt5
libqt5svg5-dev
libqt5xmlpatterns5-dev
qtmultimedia5-dev
qttools5-dev
qtbase5-private-dev
libqt5webkit5-dev
qtwebengine5-dev
libqt5webchannel5-dev
python3-dev

and on RHEL/Rocky 9:

epel-release
qt5-qtbase-devel
qt5-qtbase-gui
qt5-qtbase-private-devel
qt5-qtsvg-devel
qt5-qtxmlpatterns-devel
qt5-qtmultimedia-devel
qt5-qttools-devel
qt5-qttools-static
qt5-qtwebkit-devel
qt5-qtwebengine-devel
python3-devel

Though not all of the above may be needed to build PythonQt alone. The packages listed are needed to build our application which builds PythonQt as part of its libsrc tree.

[mrbean-bremen]

However, for your issue with qmetaobjectbuilder_p.h it indeed appears to you need to install qtbase5-private-dev

I actually do this in the docker image and thought that I did it locally, but obviously I was wrong

I would suggest install all Qt packages (libqt5--dev, libqt55, qt5-dev, q-qt5) on Ubuntu

I tried this, but this creates some incompatibility issues between packages, which I have to sort out first. I'll probably merge the current state(I also removed the unneeded include, which will fix #81), and will check some time later how to optimize this. After all, this is currently only to check that the code generation works in principle, the generated files are not provided - this is something that would certainly be helpful in the future.

[mrbean-bremen]

Actually, this comment belongs more to the PR than to this issue, but as the PR will hopefully fix this, I'll leave it.