Add GH workflow to push releases to PYPi and introduce CalVer

[betatim]

This adds a new GH Action workflow that will build and upload a new
package to pypi.org. The release instructions are updated to use
calendar versioning. Outdated instructions related to the change log
have been removed.

The proposed calver tag format is 2021.01.0 or YYYY.MM.N. N can be incremented if we need to make a second (or N+1th release in the same month). The pip tool uses YY.MM.N and dask distributed uses YYYY.MM.N. This made me pick the proposed format. However i have no strong opinion over using YY or YYYY, nor could I explain why you should pick one over the other.

We could automate the releasing via a cron job but I decided that this would be out of scope for this PR. Before we automate it I think we should confirm that the mechanics work and see what unintended consequences there are of making a release every month.

This PR also does away with maintaining a change log. The thinking is that if we want to automatically create releases we won't be able to have a change log. While we could automatically generate one, it would probably be based on a list of the PRs since the last release. So instead of having to maintain tooling to generate the change log we can insert a link that lists PRs made before a certain tag in reverse chronological order into the documentation. It has been nearly a year since we tagged our last release and since then not a single change has made its way into the change log. I think this means that the perceived value of doing this extra step is not high enough and would use it to officially stop maintenance of a change log.

[betatim]

I added a pypi token in PYPI_API_TOKEN to the secrets of this repository. I think once we merge this we can delete the PYPI_PASSWORD secret. I am not sure what it is used for but I assume it is related to the travis job somehow? I can't find it referenced anywhere in this repositories GH workflows.

[jtpio]

Thanks!

This should fix #980

[choldgraf]

This look great and I'm +1 on the workflow of monthly rather than "when we decide it" releases. The logic for GHA releases also looks correct to me.

One quick thought, perhaps we could add a "helper script" that uses github-activity for whenever people do want to update the Changelog? Something like:

---

While updating the changelog is not a pre-requisite for creating a release, we should keep it updated semi-regularly. To quickly auto-generate a list of pull-requests that have been merged since the last release, you can use the github-activity command-line tool. Follow these steps:

1. Install github-activity if you haven't already

   pip install github-activity
   

2. Generate markdown for merged PRs since the last release:

   github-activity jupyterhub/repo2docker -s <latest-release-tag>
   

   for example:

   github-activity jupyterhub/repo2docker -s 0.11.0
   

3. Copy/paste the relevant PRs (you may remove some if they are minor or not noteworthy) into the changelog, and make a PR for the updates.

---

(note - I'd also be +1 on just automating this if we set up a CRON for the releases

[betatim]

I don't know what the right thing to do is regarding a change log. As a user of projects I hate it when there is no change log or as a positive: "I love being a user of the Stripe API because of how good the 'stuff has changed' experience is" (you get a personalised change log).

As a user the value of the changelog is that it is a pre-digested/editorialised version of what has changed. Not an exhaustive list of all the things that have changed. Because if I had the time and skill to make a judgement of "what will happen/break/get fixed if I upgrade" based on a unedited list of all changes I wouldn't need a change log. I'd just read the list of PRs :D

github-activity does some sorting of PRs into categories but because we have tried but not succeeded in consistently tagging PRs and issues it is not much better than a list of PRs. It does an amazing job but as always with data "rubbish in means rubbish out".

So at the moment my feeling is "let's make it official that there won't be a changelog going forward instead of doing a not-very-good job of it". A bit of a feeling of despair :-/

[betatim]

For (self) reference:

https://github.com/jupyterhub/repo2docker/pulls?q=+sort%3Aupdated-desc+closed%3A2020-02-05..2021-01-21+ gives you all PRs merged between the release of 0.10.0 and today.

https://github.com/jupyterhub/repo2docker/issues?q=is%3Aissue+sort%3Aupdated-desc+closed%3A2020-02-05..2021-01-21+ gives you the list of issues that were closed in the same date range

[manics]

Given the choice between releases with no changelog, and the current situation of no releases but telling people to install from master, I think we should go for the former 😄.

We can always go back and add a changelog later. One thing that's been mentioned elsewhere is a blog post, so what if we were to aim for e.g. one or two posts a year summarising the major changes?

[choldgraf]

+1 from me if we want to just say "there is no changelog", I don't think a changelog should block a release (well...usually I do, but not if there are no changelogs at all!)

[betatim]

Thinking a bit more about change logs: I think as a user I like it when there is prose. "This release contains mostly bug fixes and we added a new feature to reticulate spines. We don't expect any backwards incompatibilities or changes in behaviour wrt the previous version." or "This release contains two changes which alter existing behaviour (more details link-here and link-here). We also added a new feature to reticulate spines. With the exception of the two mentioned changes we don't expect any backwards incompatibilities or changes in behaviour."

Maybe another useful thing is a direct link to the list of all PRs of the release for the curious.

One thing I realised I like as a contributor (as a user I don't care so much) is seeing my name mentioned as someone who contributed to a release. In particular if you put "contribute to X" on your CV or website or such. This is something that github-activity does (even in a more inclusive way than just "got a PR merged"). So maybe we can use it to generate that list for us.

Overall: should we merge this PR and make a new one to get a release made?

[betatim]

Any smart ideas why npm has vanished from inside the containers r2d builds? (Why does the world not stop changing around us? :D )

[manics]

I'm in favour of getting this in and dealing with changelogs in a follow up PR.

If we want to minimise maintainer work we could have the workflow automatically create a GitHub release containing the output of github-activity?
https://github.com/marketplace/actions/create-a-release

It means the changes aren't in a changelog file in the repo, but it avoids requiring a maintainer to explicitly commit a changelog- all you need to do to create the tag as in this PR. We could optimally make it a draft release so someone can add a friendly summary before it's published?

[betatim]

I'd wait with automating the release procedure until we have used it a few times and ironed out kinks. I thought about making this workflow run on a cronjob every 3rd of the month but then decided that that might go horribly wrong/be hard to debug.

But yeah, automating the steps involved would be nice. Right now the steps would be: (1) run github-activity (2) create a tag locally and git push it (3) create a release on GH (4) paste parts of github-activity into the GH release message.

[manics]

Any smart ideas why npm has vanished from inside the containers r2d builds? (Why does the world not stop changing around us? :D )

It's very weird, I reran the tests on #1003 to check everything was still passing there and they all passed. I'll restart the tests for this PR, I've copied the failing logs for future reference:

logs.txt

[betatim]

Looks like we are ready to merge this and then give it a go by making a release.

[choldgraf]

let's see what happens :-)

[betatim]

How did you make the release Chris? Did you do something like the second half of #1004 (comment) describes? Just created the release on GH? A third way?

Curious to know because I wasn't 100% sure that "it would just work", but it looks like it did. Now I want the recipe :D

[choldgraf]

I just made a release on github, and that’s it 🙂 I was thinking about updating the docs to suggest this actually. If you create a release on github it automatically creates a git tag in the repo, so the action is triggered and the manual “create a tag” step isn’t needed. What do you think?

(This is the same workflow we use in executable books, and it is really nice and low maintenance)

[betatim]

There is a reason (that I don't remember) for the "make the tag locally". @minrk discovered it, maybe he remembers. I thought it was related to the docker image appearing on docker hub or some such but there is an image for the tag so maybe that isn't it.