Skip to content

Improve scripts and tool configurations #1693

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
Oct 4, 2023
Merged

Improve scripts and tool configurations #1693

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
8c4df3c
Add pre-commit hook to run shellcheck
EliahKagan Sep 27, 2023
f3be76f
Force color when running shellcheck in pre-commit
EliahKagan Sep 27, 2023
7dd8add
Suppress SC2086 where word splitting is intended
EliahKagan Sep 27, 2023
21875b5
Don't split and glob the interpreter name
EliahKagan Sep 27, 2023
0920371
Extract suggest_venv out of the else block
EliahKagan Sep 27, 2023
e973f52
Use some handy bash-isms in version check script
EliahKagan Sep 27, 2023
be53823
Have init script treat master unambiguously as a branch
EliahKagan Sep 27, 2023
e604b46
Use 4-space indentation in all shell scripts
EliahKagan Sep 27, 2023
19dfbd8
Make the init script a portable POSIX shell script
EliahKagan Sep 27, 2023
7110bf8
Move extra tag-fetching step into init script
EliahKagan Sep 27, 2023
c7cdaf4
Reduce code duplication in version check script
EliahKagan Sep 27, 2023
f6dbba2
A couple more script tweaks for clarity
EliahKagan Sep 27, 2023
5060c9d
Explain what each step in the init script achieves
EliahKagan Sep 27, 2023
d5479b2
Use set -u in init script
EliahKagan Sep 28, 2023
52f9a68
Make the "all" Makefile target more robust
EliahKagan Sep 28, 2023
b88d07e
Use a single awk instead of two greps and a cut
EliahKagan Sep 28, 2023
d36818c
Add a black check to pre-commit
EliahKagan Sep 28, 2023
4ba5ad1
Fix typo in comment
EliahKagan Sep 28, 2023
5d8ddd9
Use two hooks for black: to check, and format
EliahKagan Sep 28, 2023
a872d9c
Pass --all-files explicitly so it is retained
EliahKagan Oct 3, 2023
9b9de11
Fix the formatting
EliahKagan Oct 3, 2023
5d15063
Add "make lint" to lint without auto-formatting
EliahKagan Sep 28, 2023
6de86a8
Update readme about most of the test/lint tools
EliahKagan Sep 28, 2023
f094909
Add BUILDDIR var to doc/Makefile; have tox use it
EliahKagan Sep 28, 2023
fc96980
Have init script check for GitHub Actions
EliahKagan Sep 28, 2023
b98f15e
Get tags for tests from original repo as fallback
EliahKagan Sep 29, 2023
7cca7d2
Don't print the exact same warning twice
EliahKagan Sep 29, 2023
e4e009d
Reword comment to fix ambiguity
EliahKagan Sep 29, 2023
e16e4c0
Format all YAML files in the same style
EliahKagan Sep 29, 2023
62c024e
Let tox run lint, mypy, and html envs without 3.9
EliahKagan Sep 29, 2023
9e245d0
Update readme: CI jobs not just for "main" branch
EliahKagan Oct 1, 2023
c2472e9
Note that the init script can be run from Git Bash
EliahKagan Oct 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
A couple more script tweaks for clarity 
- Because the substitition string after the hyphen is empty,
  "${VIRTUAL_ENV:-}" and "${VIRTUAL_ENV-}" have the same effect.
  However, the latter, which this changes it to, expresses the
  correct idea that the special case being handled is when the
  variable is unset: in this case, we expand an empty field rather
  than triggering an error due to set -u. When the variable is set
  but empty, it already expands to the substitution value, and
  including that in the special case with ":" is thus misleading.

- Continuing in the vein of d18d90a (and 1e0b3f9), this removes
  another explicit newline by adding another echo command to print
  the leading blank line before the table.
  • Loading branch information
@EliahKagan
EliahKagan committed Oct 3, 2023
commit f6dbba2ae4ad9eb3c470b30acce280a4fb6d0445
2 changes: 1 addition & 1 deletion build-release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ function suggest_venv() {
printf "HELP: To avoid this error, use a virtual-env with '%s' instead.\n" "$venv_cmd"
}

if test -n "${VIRTUAL_ENV:-}"; then
if test -n "${VIRTUAL_ENV-}"; then
deps=(build twine) # Install twine along with build, as we need it later.
echo "Virtual environment detected. Adding packages: ${deps[*]}"
pip install --quiet --upgrade "${deps[@]}"
Expand Down
3 changes: 2 additions & 1 deletion check-version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@ head_sha="$(git rev-parse HEAD)"
latest_tag_sha="$(git rev-parse "${latest_tag}^{commit}")"

# Display a table of all the current version, tag, and HEAD commit information.
echo $'\nThe VERSION must be the same in all locations, and so must the HEAD and tag SHA'
echo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This conveys spacing much better thanks to the separate echo line, even though I have a feeling this was done for portability as $'magic' might not be allowed everywhere.

Copy link
Member Author

@EliahKagan EliahKagan Oct 4, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did it for spacing, and to to take fuller advantage of echo to have fewer \n sequences. Although it's true that $' ' is not POSIX and shouldn't be used in a #!/bin/sh script, this script is a bash script and $' ' is a available in bash on all systems. (ksh, zsh, and probably some other shells also have $' ', but not all POSIX-compatible shells have it.)

Regarding spacing, the table is currently formatted using printf and this has the advantage that its own output spacing can be adjusted by, for example, changing 14 to 15. But I originally used printf for it because I was preferring printf to echo, because originally I was doing this inline in Makefile, and a Makefile runs commands using /bin/sh (unless SHELL is assigned in the Makefile itself). This was to follow the practice of avoiding echo in portable sh scripts, at least when displaying data read from files.

But that is not necessary in a #!/bin/bash script, where we know how echo works and where echo doesn't expand escape sequences by default. So it would actually be okay, at this point, to go back to displaying the table this way:

echo
echo 'The VERSION must be the same in all locations, and so must the HEAD and tag SHA'
echo "VERSION file   = $version_version"
echo "changes.rst    = $changes_version"
echo "Latest tag     = $latest_tag"
echo "HEAD SHA       = $head_sha"
echo "Latest tag SHA = $latest_tag_sha"

This is arguably more readable, and arguably conveys spacing the best. I'd be happy to change it to this if you like.

(Another option could be to go in the opposite direction and make this check-version.sh script, and maybe the build-release.sh script as well, a #!/bin/sh script, which is feasible.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is arguably more readable, and arguably conveys spacing the best. I'd be happy to change it to this if you like.
(Another option could be to go in the opposite direction and make this check-version.sh script, and maybe the build-release.sh script as well, a #!/bin/sh script, which is feasible.)

Since trap is supported in POSIX shells and thus sh, making these scripts more portable seems valuable, even though I think that as the project currently is setup there isn't any need as it's only me using them.
Alternatives involve running these on CI which also supports bash everywhere (at least so it seems).

If you think there is benefits to portability assuming that one day CI can perform trusted publishes, I think it's fair to adjust for portability next time you touch them. Otherwise, it would be just as fair to change printf with echo as shown above. It's perfectly optional as well.

Copy link
Member Author

@EliahKagan EliahKagan Oct 9, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The desired interaction of trap and set -e may be achievable in sh. Even if not, we don't have to use trap; it can be replaced even without making the control-flow logic more complicated or error prone. For example, the script could accept -q to not append the failure message, and otherwise invoke itself with -q, check the result, print the message if it failed, and exit with the original failing exit code. (There is also an argument to be made that set -e shouldn't be relied on at all.)

There are not many systems where bash doesn't exist and can't be made available. However, due to the problem described in 729372f--which only happens when MinGW make is being used on a native Windows system that has also WSL installed, and does not happen outside make nor on other systems--the scripts' bash hashbangs are written as #!/bin/bash rather than #!/usr/bin/env bash, and thus assume bash is in /bin.

Making them #!/bin/sh scripts would also fix that. It is even safer to assume sh is in bin than to assume env is in /usr/bin, so #!/usr/bin/env sh is rarely used. (Anyway, there is no corresponding WSL-related sh.exe in System32.) So, when combined with the other minor reasons for using #!/bin/sh, that is probably worth doing.

As you've suggested, next time I work on those scripts I'll look into making them POSIX sh scripts. However, this would be unrelated to facilitating trusted publishing. CI does support bash everywhere, at least on runners hosted by GitHub Actions.

echo 'The VERSION must be the same in all locations, and so must the HEAD and tag SHA'
printf '%-14s = %s\n' 'VERSION file' "$version_version" \
'changes.rst' "$changes_version" \
'Latest tag' "$latest_tag" \
Expand Down