Skip to content

Benchmarks timing is not independent from previous benchmarks #197

Open
@fraya

Description

@fraya

Executing a test suite that has several benchmarks, the time of the benchmarks is dependent on the time of the previous benchmarks. For instance we can see that the time of benchmark-concatenate-for is 32 seconds:

$ _build/bin/issue-652-test-suite
Suite issue-652-test-suite:
  Benchmark benchmark-deque: PASSED in 14.929141s and 14MiB
  Benchmark benchmark-list: PASSED in 19.617091s and 15MiB
  Benchmark benchmark-current: PASSED in 24.570700s and 11MiB
  Benchmark benchmark-join-map: PASSED in 28.224557s and 12MiB
  Benchmark benchmark-concatenate-for: PASSED in 32.803978s and 13MiB
Ran 5 assertions
Ran 5 benchmarks
PASSED in 117.145467 seconds

But repeating the test, only for this benchmark:

$ _build/bin/issue-652-test-suite --run benchmark-concatenate-for
Suite issue-652-test-suite:
  Benchmark benchmark-concatenate-for: PASSED in 15.142817s and 13MiB
Ran 1 assertion
Ran 5 benchmarks: 4 skipped
PASSED in 15.142817 seconds

we can see that the time is the half.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions