Skip to content

Conversation

bolinfest
Copy link
Collaborator

@bolinfest bolinfest commented Sep 8, 2025

I started looking at https://nexte.st/ because I was interested in a test harness that lets a test dynamically declare itself "skipped," which would be a nice alternative to this pattern:

if std::env::var(CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR).is_ok() {
println!(
"Skipping test because it cannot execute when network is disabled in a Codex sandbox."
);
return;
}

ChatGPT pointed me at https://nexte.st/, which also claims to be "up to 3x as fast as cargo test." Locally, in codex-rs, I see

  • cargo nextest run finishes in 19s
  • cargo test finishes in 37s

Though looking at CI, the wins are quite as big, presumably because my laptop has more cores than our GitHub runners (which is a separate issue...). Comparing the CI jobs from this PR with that of a recent open PR:

cargo test cargo nextest
macos-14 - aarch64-apple-darwin 2m16s 1m51s
macos-14 - aarch64-apple-darwin 5m04s 3m44s
ubuntu-24.04 - x86_64-unknown-linux-musl 2m02s 1m56s
ubuntu-24.04-arm - aarch64-unknown-linux-musl 2m01s 1m35s
windows-latest - x86_64-pc-windows-msvc 3m07s 2m53s
windows-11-arm - aarch64-pc-windows-msvc 3m10s 2m45s

I thought that, to start, we would only make this change in CI before declaring it the "official" way for the team to run the test suite.

Though unfortunately, I do not believe that cargo nextest actually supports a dynamic skip feature, so I guess I'll have to keep looking? Some related discussions:

@bolinfest bolinfest force-pushed the pr3323 branch 2 times, most recently from 3eed899 to ad620d1 Compare September 8, 2025 19:07
@bolinfest bolinfest changed the title chore: try switching to cargo nextest chore: try switching to cargo nextest to speed up CI builds Sep 8, 2025
@bolinfest bolinfest requested a review from jif-oai September 8, 2025 20:39
- name: cargo test
- uses: taiki-e/install-action@v2
with:
tool: nextest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
tool: nextest
tool: nextest
version: 0.9.103

Let's pin the version to prevent any malicious injections and prevent any risks if ok for you?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pinned version for nextest and taiki-e/install-action

@bolinfest
Copy link
Collaborator Author

bolinfest commented Sep 9, 2025 via email

@jif-oai
Copy link
Contributor

jif-oai commented Sep 9, 2025

I trust Github way more than a third-party maintained library but yes actually

@bolinfest bolinfest merged commit 16309d6 into main Sep 9, 2025
38 checks passed
@bolinfest bolinfest deleted the pr3323 branch September 9, 2025 04:39
@github-actions github-actions bot locked and limited conversation to collaborators Sep 9, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants