Package binary as a Python wheel

[ejohb]

See #226

[ingydotnet]

The tricky part here is where I build and publish the wheels from.

I need to explain the release process. It is partially automated and a bit involved.

YAMLScript's ys and libyamlscript.{so,dylib} native binaries ship to 4 platforms: the matrix of linux/macos and intel/arm as seen here: https://github.com/yaml/yamlscript/releases

I do all the releases by hand currently. I have a linux/intel laptop and I have 3 other machines for the other 3 platforms that I can ssh into.

I run a command that looks like:

YS_GH_USER=ingydotnet YS_GH_TOKEN=<github-api-token> make release-yamlscript  o=0.1.96 n=0.1.97

This runs util/release-yamlscript a script written in yamlscript. It has 11 steps, and is pretty well commented. You should give it a read.

The first 9 steps build and release the native artifacts to a new https://github.com/yaml/yamlscript/releases version section (for linux/intel).
The script then pauses after printing the new github release id number.

At this point I ssh into the other 3 machines and run:

YS_GH_USER=ingydotnet YS_GH_TOKEN=<github-api-token> make release-yamlscript  RELEASE_ID=213416675

When the RELEASE_ID var is set, the script only runs step 9: make release-assets

Back on my linux/intel machine I press enter and the script continues with:

Step 10 does the release of each language binding (including to pypi).

Step 11 publishes the website.

Then it is complete.

---

For this we need to publish wheels somewhere in here. I suppose that could be a new step 9b that runs on all 4 machines.

Hopefully we can soon fully automate this.

---

Note I have a lot going on until June 1st so I might not be so active on this until then.

---

If you are interested, another big thing YS needs is package manager installs, like brew and apt and yum and on and on.

If you have any interest/time/expertise helping there would be awesome.

[ejohb]

Many thanks for this. Most helpful.

My current approach in this PR is to add the wheel building part in a platform/arch-agnostic way. That means that when run under your current flow, it will additionally release a binary wheel, albeit for linux/intel only.

We would then need to adapt your flow as you suggest (to run step 10 under the other platforms) and it should build/release bindings for those platforms too without any additional changes.

[ejohb]

OK, I'm now at the first stage I mentioned:

• Binary wheels are built for whichever of the supported platforms make dist is run under.
• I've tested this under linux/amd64 and linux/arm64. All looks good. See the release assets below.
• It should also work under macOS, but I don't have easy access to test.
• I've added the Dockerfile I used to test to the repo.
• I added a show_info console script for my own convenience when testing.

It would be cool if I could get the following feedback:

• Generally do a review let me know what you think (I would have requested one, but don't think I have permissions).
• When it comes to modifying your flow so wheels are released for all platforms, let me know if you want to do that in a separate PR, or in this one.

yamlscript-0.1.96-cp313-cp313-linux_x86_64.whl.gz
yamlscript-0.1.96-cp313-cp313-linux_aarch64.whl.gz
bdist_wheel.info.txt
sdist.info.txt
yamlscript-0.1.96.tar.gz

[ingydotnet]

Hi @ejohb

Sorry for not noticing your progress on this.

I'll try playing with it to publish a Linux/Intel wheel.

I'm not sure what the purpose of the Dockerfile is.
It's not used by the Makefile or the setup.py afaict.

We are going to need to build and ship 4 different wheels...
The matrix of Linux/Mac x Intel/ARM like in https://github.com/yaml/yamlscript/releases

I'll see if I can at least publish the Linux/Intel one to PyPI and we can discuss from there.

[ejohb]

Yep no worries. Just to clarify that the current code should work with the full matrix now, albeit if you run make dist manually. It would be good if you could do so actually (whether during a dry-run or not) just so we can have some confirmation the wheel built correctly under macOS.