Fix: Auto-increment seed across batch_run iterations #2841
+8
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
When using
batch_run()
with a single seed value and multiple iterations, all iterations use the same seed, producing identical results instead of independent replications.See #2835.
Solution
Modify
_model_run_func
to automatically increment the seed for each iteration: 42, 43, 44, etc.Behavior changes
seed=42, iterations=3
: currently all use 42, now uses 42, 43, 44seed=[42, 43, 44], iterations=1
: unchangedCode that passes a single seed with multiple iterations will get different results. The current behavior seems like a bug (why run multiple identical iterations?), but this technically breaks existing code.
Review
I'm in doubt about this. What if users change have other random elements in their model? Do we do good obscuring this?
Secondly, is this a bugfix or a breaking change? Should we treat it as a fix and merge, or wait for a major version?
Might close #2835. @dylan-munson curious what you think.