Github Action Unit Testing

[BrianPugh]

I recently created the github action install-micropython, making it much simpler to obtain a micropython executable in Github Actions. I can volunteer to add a workflow to run unit tests to this repo, but would like to discuss it first. There are a few issues that make the ideal implementation non-obvious.

1. Would we simply iterate over all folders in python-stdlib (and other) and run micropython -m unittest? This has a few issues described below. An alternative is to setup an explicit job/step for each package. Advantage: much greater clarity in the github action dashboard. Disadvantage: a lot more manual configuration.

2. Some modules are not intended to be standalone, for example (python-stdlib/os-path)

3. Inconsistent folder structure: some packages have tests in a tests/ folder, others simply have a test_xyz.py file next to the primary library file.

4. As a provisioning step, should unittest and unittest-discover be copied into their own directory first, then that path added to MICROPYTHONPATH?

[jonnor]

Hi @BrianPugh - this is a great initiative. Having basic test coverage of modules would hopefully make it easier to accept contributions, because of the reduced risk of breaking things in an undetected way.

As you mention, there are a lot of differences between packages at the moment. I also think that it would be highly beneficial to have each package show up as its own job in the Github Actions dashboard - so that it is easy to identify which package failed.

I believe that just starting by adding a Github Actions runner configuration for some modules that already have some useful tests. Then one can identify commonalities, and harmonize the approaches from there on.

Rome was not built in a day, and well-tested, quality software is the same :D

[jonnor]

I see now that there is a Actions job that runs tests.
https://github.com/micropython/micropython-lib/actions/workflows/package_tests.yml
It calls the script ci_package_tests_run
https://github.com/micropython/micropython-lib/blob/master/tools/ci.sh#L48
which has a list of .py files which are executed as a test

So there is something basic in place now that can be extended. So I think we can close this issue, and maybe open new issues or merge requests to focus on particular improvements.

[BrianPugh]

yup, looks good!