We should make sure the wheel build is not used for any non pure-Python packages since those should instead be built with cibuildwheel. I'm not sure what the best way is to do this but we could simply check for e.g. 'any' in the output wheel filename.