compatibility with pillow wheels

[MrBitBucket]

I'm trying to extend the reportlab builds with latest cibuildwheel (2.2.0a1 and get cp310 as well) I have a failure in something I thought should work the build phase worked and reports in

https://github.com/MrBitBucket/reportlab-mirror/runs/3766263538?check_suite_focus=true

Fixed-up wheel written to /tmp/cibuildwheel/repaired_wheel/reportlab-3.6.2-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl

However, this fails in the testing phase because reportlab requires Pillow. It seems that the test failed to use any of the wheels in pypi at https://pypi.org/project/Pillow/#files and then tried to build from source. Resulting in a failure because of a lack in the environment of a libjpeg.

Just above the 3.10 build I see that the "Building cp39-manylinux_x86_64 wheel" build and test worked fine and there a pillow wheel was downloaded. The wheel produced in 3.9 is
reportlab-3.6.2-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl

and the pillow wheel downloaded was
Pillow-8.3.2-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl

In the cp310 case I suppose Pillow-8.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl is the nearest, but obviously there's a compatibilty issue and it wants to build from source; that will cause issues for other users. Is there a way out?

[Czaki]

they provide manylinux2014 wheels and you probably try to build your wheels on manylinux2010 docker image. I suggest to build 3.10 wheels on manylinux2014.

[henryiii]

We could implement this as the default - currently, if none, then we set manylinux2010 for the normal archs, and 2014 for the special arches. I don't think it would be too hard to make this default to 2014 in the cp10 case (after #854, anyway) if we wanted to.

[MrBitBucket]

I tried to follow the distutils sig discussions on manylinux 1 / 2010 / 2014, but did not really understand the inclusion relation. Python seems to have built its own version of dll hell with the many binary versioning issues.

[henryiii]

This issue (pypa/manylinux#994) and the table in the readme https://github.com/pypa/manylinux are useful. Basically, you have to have a minimum supported GLIBC version, and those came from the compiling environment (docker image) - manylinux1 -> CentOS 5, manylinux2010 -> CentOS 6, manylinux2014 -> CentOS 7. You can't run with an older GLIBC, but you can run with a newer one. Sadly, it also depended on Pip, so there's a min pip version too. Nowadays, the naming is manylinux_2_<minor>, which is much clearer (directly from the GLIBC version), but requires a very new pip, so everything gets tagged with both names. From now on, though, current pip can install future versions of manylinux. By Python 3.10, all supported pips support 2014, and 2014 is already old enough for basically all users able to run 3.10.

Anyway, you can't load a manylinux2014 wheel in CentOS 6 (manylinux2010) or CentOS 5 (manylinux1), so you can't build / test if a dependency is only available in a newer format - I'd always use the highest dependency version.

[MrBitBucket]

Thanks for the reply. I tried switching to manylinux2014, but immediately get errors I didn't see before. Ina rather pedantic attempt to see what's going on I attempted to handomatically test things in the failing docker which I believe is quay.io/pypa/manylinux2014_x86_64:2021-09-19-a5ef179.

I run the following before build yum install cairo-devel fribidi -y, but I immediately see differences in the results. In the cibuild/action output I see Package cairo-devel.x86_64 0:1.8.8-6.el6_6 will be installed whereas in my attempted handomatic docker I see Package cairo-devel.x86_64 0:1.15.12-4.el7 will be installed so I assume the image is modified in some way. I checked the digests and the local image appears to match what the action reports.

So is there a way to get cibuild to create a properly setup docker perhaps given the desired python/os etc etc?

The docker command I run looks like this

docker run -v /home/devel/reportlab/tmp/project:/project -it 'quay.io/pypa/manylinux2014_x86_64:2021-09-19-a5ef179' /bin/bash

but obviously my docker version is different as is the host OS. FWIW I am attempting to build cp36-manylinux_x86_64 wheel (where the python is supposed to be /opt/python/cp36-cp36m/bin/python.

I am able to duplicate what I think are all the steps in the failing action, but in my case I see no errors. The handomatic script looks llike

cd /project
[ ! -h /lib64/libcairo.so ] && yum install cairo-devel fribidi -y
rm -rf /tmp/cibuildwheel/built_wheel
mkdir -p /tmp/cibuildwheel/built_wheel
ls -alrt /tmp/cibuildwheel/built_wheel
/opt/python/cp36-cp36m/bin/python3 -m pip wheel /project --wheel-dir=/tmp/cibuildwheel/built_wheel --no-deps -vvv
rm -rf /tmp/cibuildwheel/repaired_wheel
mkdir -p /tmp/cibuildwheel/repaired_wheel
sh -c 'auditwheel repair -w /tmp/cibuildwheel/repaired_wheel /tmp/cibuildwheel/built_wheel/reportlab-3.6.2-cp36-cp36m-linux_x86_64.whl'
rm -rf /tproject
#/opt/python/cp36-cp36m/bin/python3 -mvenv /tproject
/opt/python/cp36-cp36m/bin/python3 -mpip install virtualenv -c ./constraints-python36.txt
/opt/python/cp36-cp36m/bin/python3 -mvirtualenv --no-download /tproject
source /tproject/bin/activate
/tproject/bin/python -mpip install -U pip
/tproject/bin/python -mpip install /tmp/cibuildwheel/repaired_wheel/reportlab-3.6.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
source secrets.txt
sh -c '/tproject/bin/python3 -u -c "import sys,os;r=os.system('"'"' '"'"'.join((sys.executable,'"'"'-u'"'"',os.path.join(r'"'"'/project'"'"','"'"'setup.py'"'"'),'"'"'tests-postinstall'"'"','"'"'--verbose'"'"','"'"'--pip-install=pyphen'"'"','"'"'--pip-install=rlPyCairo'"'"','"'"'--rl-index-url=https://'$CITOOLS_USER:$CITOOLS_PASSWORD'@www.reportlab.com/pypi'"'"','"'"'--rl-pip-install=pyfribidi'"'"','"'"'--rl-pip-freeze'"'"','"'"'2>&1'"'"')));sys.exit(('"'"'!!!!! test command --> %s !!!!!'"'"'%r) if r else r)" 2>&1'

but as noted above it is already wrong at line 3 after the yum.

[MrBitBucket]

I found that we can set up the docker using cibuildwheel a a command.