In the dockerfile includes are removed:
rm -rf /usr/local/include/*
That removal does not allow to install extensions that need to be recompiled like pg_partman
My suggestion is not to remove the include files.
After commenting that line, pg_partman can be installed