Crashing python check does not change exit code

[lacop11]

When a custom Python check crashes the tool does not exit with non-zero exit code, which means CI integrations could be silently ignoring checks due to bugs.

Reproduction:

git clone https://github.com/lacop11/checkov-crash-repro.git
cd checkov-crash-repro
uvx [email protected] -d . --run-all-external-checks --external-checks-dir checks --quiet
echo $?

Output:

$ uvx ...
[ERROR]  Failed to run check CKV_BUG_1 on /main.tf:null_resource.example
Traceback (most recent call last):
...

$ echo $?
0

Expected output: Should exit with non-zero code (probably 2).

Notes

I feel like I'm missing something obvious but I can't find nothing relevant in docs nor github issues. There is the flag --no-fail-on-crash which I'm not setting and it defaults to false, and it's existence implies a crash should result in exit code 2.

Is there some other configuration change I need to make? It's surprising that default behavior would not fail on crashing check.