Parametrize test_benchmarks.py test by criterion benchmarks

[roypat]

Currently, test_benchmarks.py contains a single test that runs all criterion benchmarks. With increasing number of benchmarks, we thus increase the duration of this test, and need to adjust its timeout. Instead, we want to parameterize the test by the list of criterion benchmarks we have in the repository. This means introducing a fixture that yields the benchmarks listed by cargo bench --all -- --list individually. Then the test itself would only run the benchmark it is provided. This should also make it easier to see which benchmarks are failing.

Originally posted by @pb8o in #4830 (comment)

[MacOS]

If someone has not done it yet, I would take it over.

[bchalios]

Hey @MacOS, please start working on it if you want.

[cm-iwata]

@MacOS
CC: @bchalios

Are you still working on this?
If you are busy, may I handle it instead you?

[MacOS]

I would submit something next week, however, I am not mad if you do it.

[gjkeller]

@MacOS @cm-iwata @bchalios

I wanted to check and see if this issue was still being worked on. If not, would I be able to work on it?

[cm-iwata]

@gjkeller
I'm waiting for a pull request review...

[gjkeller]

@cm-iwata
No worries! I was unsure of if it was complete due to the failing tests, but seems like the failure was only due to timeouts and network errors. Good luck with getting your PR merged!

[roypat]

@gjkeller I'm waiting for a pull request review...

Hi, sorry, I thought the latest update on the PR was that you were looking into why the CI was failing. Did you need help with anything? :o

[cm-iwata]

@roypat
I have made some adjustments, so could you please re-run the CI and review it?