Update external docs

garrett · web-flow · commit 8a3ff2f2da50 · 2023-11-23T02:05:08.000Z
diff --git a/external/bots/README.md b/external/bots/README.md
@@ -90,6 +90,10 @@ When generating a new personal access token, the scopes should contain
 `repo:status` and `read:org`. Note in particular, that `repo` and
 `public_repo` scopes each grant full push access, and should not be used.
 
+You need at least "Write" access to the project for triggering statuses, either
+individually per repo (e.g. [cockpit](https://github.com/cockpit-project/cockpit/settings/access)
+or for [all cockpit-project repos](https://github.com/orgs/cockpit-project/teams/committers).
+
 If you'd like to download Red Hat-only internal images from S3, you'll
 need to create a key file in `~/.config/cockpit-dev/s3-keys/[domain]`.
 The `[domain]` can be any non-toplevel domain which contains the S3 URL
@@ -124,7 +128,7 @@ For describing tests which we want to run we use __contexts__. A context has the
     image[/scenario][@bots#bots_pr][@owner/project/ref]
 
 where items have the following meaning:
-- image: Name of the image on which tests should run (e.g. 'fedora-testing').
+- image: Name of the image on which tests should run (e.g. 'fedora-coreos').
 - scenario: Name of a specific test. This is specific for each separate project and
   is passed verbatim to 'test/run' in `$TEST_SCENARIO`.
 - bots_pr: Number of pull request that exists in bots repository. When specified,
@@ -134,28 +138,28 @@ where items have the following meaning:
 - ref: Reference in the project (usually branch) (e.g. 'rhel-8.2'). Default is
   the project's primary branch.
 
-For example, context for scenario 'firefox' on 'fedora-testing' is:
+For example, context for scenario 'firefox' on 'fedora-coreos' is:
 
-    fedora-testing/firefox
+    fedora-coreos/firefox
 
 If we want to trigger it on 'cockpit-project/cockpit':
 
-    fedora-testing/firefox@cockpit-project/cockpit
+    fedora-coreos/firefox@cockpit-project/cockpit
 
 If we want to also not run it on the primary branch, but on 'rhel-8-0' branch:
 
-    fedora-testing/firefox@cockpit-project/cockpit/rhel-8-0
+    fedora-coreos/firefox@cockpit-project/cockpit/rhel-8-0
 
-If we want to run tests on 'fedora-testing' but with bots from pull request '169':
+If we want to run tests on 'fedora-coreos' but with bots from pull request '169':
 
-    fedora-testing@bots#169
+    fedora-coreos@bots#169
 
 ### Retrying a failed test
 
-If you want to run the "fedora-testing" testsuite again for pull
+If you want to run the "fedora-coreos" testsuite again for pull
 request #1234 of cockpit-project/cockpit, run tests-trigger like so:
 
-    ./tests-trigger --repo cockpit-project/cockpit 1234 fedora-testing
+    ./tests-trigger --repo cockpit-project/cockpit 1234 fedora-coreos
 
 You can also invoke bots/tests/trigger from any project checkout, in which case
 you don't need the explicit `--repo` -- it will default to the GitHub origin of
@@ -165,7 +169,7 @@ the current directory's project.
 
 If you want to run all tests on pull request #1234 that has been opened by
 someone who does not have push access to the repository nor isn't in the
-[Contributors team](https://github.com/orgs/cockpit-project/teams/contributors/members),
+[allowlist](https://github.com/cockpit-project/bots/blob/main/lib/allowlist.py)
 run tests-trigger with `--allow`:
 
     ./tests-trigger --allow [...]
@@ -186,21 +190,21 @@ tests-trigger reads the repo. This has to be set per cockpit project.
 Test images are refreshed automatically once per week, and even if the
 last refresh has failed, the machines wait one week before trying again.
 
-If you want the machines to refresh the fedora-testing image immediately,
+If you want the machines to refresh the fedora-coreos image immediately,
 run image-trigger like so:
 
-    ./image-trigger fedora-testing
+    ./image-trigger fedora-coreos
 
 ### Creating new images for a pull request
 
 If as part of some new feature you need to change the content of some
 or all images, you can ask the machines to create those images.
 
-If you want to have a new fedora-testing image for pull request #1234, add
+If you want to have a new fedora-coreos image for pull request #1234, add
 a bullet point to that pull request's description like so, and add the
 "bot" label to the pull request.
 
-    * [ ] image-refresh fedora-testing
+    * [ ] image-refresh fedora-coreos
 
 The machines will post comments to the pull request about their
 progress and at the end there will be links to commits with the new
@@ -215,21 +219,6 @@ used as default OS for test VMs.
 1. If this is a new image, add `_manual` test contexts for the new image to `lib/testmap.py`, and land that into `main`.
 2. Create a PR that updates `TEST_OS_DEFAULT` in `lib/constants.py`, and trigger all tests for that image there.
 
-#### Fedora testing image
-
-The `fedora-testing` image is a Fedora image with updates-testing enabled,
-the version of the image is determined by what the
-`fedora-testing.bootstrap` symlink points too.
-
-To update the Fedora version used:
-
-1. Update the `fedora-testing.bootstrap` symlink to the latest Fedora
-   release.
-2. Update the naughty symlink `naughty/fedora-testing` to the latest
-   Fedora release.
-3. Create a new PR and refresh the image.
-
-
 #### Fedora CoreOS
 
 The Fedora CoreOS image is updated to a new Fedora release out of our
diff --git a/external/source/HACKING.md b/external/source/HACKING.md
@@ -185,13 +185,42 @@ git submodules:
 Refer to the [testing README](test/README.md) for details on running the Cockpit
 integration tests locally.
 
-## Testing the Python bridge
+## Python bridge
 
-Cockpit currently has an experimental replacement for `cockpit-bridge`
-written in Python.  It resides in `src/cockpit` with most of its rules in
-`src/Makefile.am`.  This directory was chosen because it matches the standard
-so-called "src layout" convention for Python packages, where each package
-(`cockpit`) is a subdirectory of the `src` directory.
+Most distro releases now ship a replacement for the C bridge written in Python.
+It resides in `src/cockpit` with most of its rules in `src/Makefile.am`.  This
+directory was chosen because it matches the standard so-called "src layout"
+convention for Python packages, where each package (`cockpit`) is a
+subdirectory of the `src` directory.
+
+### Running the bridge
+
+The Python bridge can be used interactively on a local machine:
+
+    PYTHONPATH=src python3 -m cockpit.bridge
+
+To make it easy to test out channels without having to write out messages
+manually, `cockpit.misc.print` can be used:
+
+    PYTHONPATH=src python3 -m cockpit.misc.print open fslist1 path=/etc watch=False | PYTHONPATH=src python3 -m cockpit.bridge
+
+These shell aliases might be useful when experimenting with the protocol:
+
+    alias cpy='PYTHONPATH=src python3 -m cockpit.bridge'
+    alias cpf='PYTHONPATH=src python3 -m cockpit.misc.print'
+
+When working with the Python bridge on test images, note that `RHEL/CentOS 8`,
+`debian-stable`, and `ubuntu-2204` still use the C bridge. So if you want to
+explicitly have the Python bridge on those images use:
+
+    ./test/image-prepare --python
+
+To enable debug logging in journal on a test image, you can pass `--debug` to
+`image-prepare`. This will set `COCKPIT_DEBUG=all` to `/etc/environment`, if
+you are only interested channel debug messages change `all` to
+`cockpit.channel`.
+
+### Testing the Python bridge
 
 There are a growing number of Python `unittest` tests being written to test
 parts of the new bridge code.  You can run these with `make pytest` or
diff --git a/external/source/test/README.md b/external/source/test/README.md
@@ -3,8 +3,9 @@ title: Integration Tests of Cockpit
 source: https://github.com/cockpit-project/cockpit/blob/master/test/README.md
 ---
 
-This directory contains automated integration tests for Cockpit, and the support
-files for them.
+This directory contains automated integration tests for Cockpit, and the
+support files for them. The architecture of the automated integration tests is
+described in [ARCHITECTURE](./ARCHITECTURE.md)
 
 To run the tests on Fedora, refer to the [HACKING](../HACKING.md) guide for
 installation of all of the necessary build and test dependencies. There's
@@ -87,10 +88,70 @@ ssh and web.  See the "Helpful tips" section below.
 
 ## Pixel tests
 
-The verify test suite contains ["pixel tests"]({{ site.baseurl }}/blog/pixel-testing.html).
-Make sure to create the test/reference submodule before running tests which contain pixel tests.
+Pixel tests in Cockpit ensure that updates of our dependencies or code changes
+don't break the UI: for example slight changes of layout, padding, color and
+everything which isn't easily spotted by a human. They also give us confidence
+that an update of our UI Framework doesn't introduce changes in how Cockpit
+looks.
 
- * test/common/pixel-tests pull
+Pixel tests make a screenshot of a selector and compare it to a known good
+reference image. if there is a difference, the test fails and a pixel
+difference is shown.
+
+This works as our tests run in the [cockpit/tasks container](https://quay.io/repository/cockpit/tasks)
+which pins the browser and font rendering so repeated runs provide the same
+pixels. To generate new pixels, this tasks container must be used; your own
+browser and font rendering software might generate different results. For more
+information read the ["introduction blog post"]({{ site.baseurl }}/blog/pixel-testing.html).
+
+The test images are stored in a git submodule in the `test/reference` directory
+and be fetched with:
+
+```sh
+./test/common/pixel-tests update
+```
+
+As Cockpit tests under multiple distributions and it is not worth the effort to
+run pixel tests on every supported distribution we only run them for the
+image configured in `test/reference-image`.
+
+Our tests call `Browser.assert_pixels` at interesting and strategic places.
+This assertion method requires at least a CSS selector and an image title.
+Pixel tests are generated in five layouts by default: desktop, medium, mobile,
+dark and rtl.
+
+Take a screenshot of the content in `#detail-content`:
+```python
+browser.assert_pixels("#detail-content", "filesystem")
+```
+
+Take a screenshot of the content in `#detail-content` and ignore all elements
+with a class `disk-stats` as they change per test run:
+```python
+browser.assert_pixels("#detail-content", "filesystem", ignore=[".disks-stats"])
+```
+
+Take a screenshot of the content in `#detail-content` and skip it for a
+specific layout as it generates unstable pixels:
+```python
+browser.assert_pixels("#detail-content", "filesystem", skip_layouts=["rtl"])
+```
+
+To update pixel tests, locally run the test in the current tasks container, or
+create a draft PR and let the tests run for `test/reference-image` and
+afterwards fetch the new pixels:
+
+```
+./test/common/pixel-tests fetch "https://cockpit-logs.us-east-1.linodeobjects.com/<snip>/log.html"
+```
+
+Finally, upload the new pixel tests and commit the newly generated submodule commit:
+```
+./test/common/pixel-tests push
+```
+
+**Note** that you have to a part of the [Contributors group](https://github.com/orgs/cockpit-project/teams/contributors)
+to push pixel tests.
 
 ## Test Configuration
 
@@ -100,17 +161,17 @@ You can set these environment variables to configure the test suite:
                   "centos-8-stream"
                   "debian-stable"
                   "debian-testing"
-                  "fedora-37"
                   "fedora-38"
+                  "fedora-39"
                   "fedora-coreos"
                   "fedora-testing"
-                  "rhel-8-9"
-                  "rhel-8-9-distropkg"
+                  "rhel-8-10"
+                  "rhel-8-10-distropkg"
                   "rhel-9-3"
                   "rhel4edge",
                   "ubuntu-2204"
                   "ubuntu-stable"
-               "fedora-38" is the default (TEST_OS_DEFAULT in bots/lib/constants.py)
+               "fedora-39" is the default (TEST_OS_DEFAULT in bots/lib/constants.py)
 
     TEST_JOBS  How many tests to run in parallel.  The default is 1.
 
diff --git a/external/wiki/Contributing.md b/external/wiki/Contributing.md
@@ -14,7 +14,8 @@ https://cockpit-project.org/running.html
  * Keep in mind the [project ideals]({{ site.baseurl }}/ideals.html)
  * If you are looking for someplace to start developing, issues are marked [with a good-first-issue label](https://github.com/cockpit-project/cockpit/issues?q=is%3Aopen+is%3Aissue+label%3Agood-first-issue) when they are a good introduction to Cockpit and of limited scope.
  * [Documentation]({{ site.baseurl }}/guide/latest/) (be aware: possible version differences)
- * [Set up integration tests](https://github.com/cockpit-project/cockpit/blob/master/test/README.md). Automated tests are an important part of Cockpit and every new feature should be accompanied by its tests. 
+ * [Set up integration tests](https://github.com/cockpit-project/cockpit/blob/master/test/README.md). Automated tests are an important part of Cockpit and every new feature should be accompanied by its tests.
+ * Join the project's [GitHub Discussions page](https://github.com/cockpit-project/cockpit/discussions) to connect with the community, ask questions, and participate in discussions about the project's development and features.
 
 ### Issue tracker
 
