Open
Description
Executing a test suite that has several benchmarks, the time of the benchmarks is dependent on the time of the previous benchmarks. For instance we can see that the time of benchmark-concatenate-for
is 32 seconds:
$ _build/bin/issue-652-test-suite
Suite issue-652-test-suite:
Benchmark benchmark-deque: PASSED in 14.929141s and 14MiB
Benchmark benchmark-list: PASSED in 19.617091s and 15MiB
Benchmark benchmark-current: PASSED in 24.570700s and 11MiB
Benchmark benchmark-join-map: PASSED in 28.224557s and 12MiB
Benchmark benchmark-concatenate-for: PASSED in 32.803978s and 13MiB
Ran 5 assertions
Ran 5 benchmarks
PASSED in 117.145467 seconds
But repeating the test, only for this benchmark:
$ _build/bin/issue-652-test-suite --run benchmark-concatenate-for
Suite issue-652-test-suite:
Benchmark benchmark-concatenate-for: PASSED in 15.142817s and 13MiB
Ran 1 assertion
Ran 5 benchmarks: 4 skipped
PASSED in 15.142817 seconds
we can see that the time is the half.