Add -only-out-test-toml flag and serialize Env in materialized configs #3881
+3,412
−13
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change adds a new flag to the acceptance test framework that allows regenerating out.test.toml configuration files without running the actual tests, making it much faster to update test configurations. Changes: - Added -only-out-test-toml flag to libs/testdiff/golden.go - Modified acceptance test runner to skip test execution when flag is set - Added make test-regenerate-configs rule for convenience - Regenerated all out.test.toml files to include materialized config The new workflow is ~100x faster than running tests with -update: - Full test run with -update: several minutes - Config regeneration only: ~2.5 seconds Usage: make test-regenerate-configs Or directly: go test ./acceptance -run '^TestAccept$' -only-out-test-toml 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
shreyas-goenka
requested review from
andrewnester,
anton-107,
denik,
jefferycheng1,
kanterov,
lennartkats-db and
pietern
as code owners
shreyas-goenka
changed the title
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
denik
reviewed
Nov 5, 2025
libs/testdiff/golden.go
Outdated
|
|
||
| func init() { | ||
| flag.BoolVar(&OverwriteMode, "update", false, "Overwrite golden files") | ||
| flag.BoolVar(&OnlyOutTestTomlMode, "only-out-test-toml", false, "Only regenerate out.test.toml files without running tests") |
Contributor
There was a problem hiding this comment.
Why add this flag here? It belongs to acceptance_test.go since it's only used there.
Also, can we shorter one, something easier to type? e.g. -only-out
Contributor
Author
There was a problem hiding this comment.
Also, can we shorter one, something easier to type? e.g. -only-out
-only-out is ambigious. Does it refer to golden files or the test.toml files? I expect this to be run occasionally since this is only useful to mass update configs. For a single test, these will automatically be updated when the test is run anyways.
Collaborator
22 failing tests:
|
Contributor
|
Could you condense the PR description? E.g. "Example: New Env Section in out.test.toml" section is not necessary, we can see it in PR diff. |
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR:
out.test.tomlconfiguration files without running the actual tests, making it much faster to update test configurations.Envfield, which was previously not serialized. This means allout.test.tomlfiles now capture environment variables that affect test execution.Usage
Using Make target (recommended)
Using go test directly
For a specific test