PYTHON-5345 Streamline the standard tasks #2312
Conversation
…cleanup_task_combos
|
evergreen retry |
1 similar comment
|
evergreen retry |
| @@ -461,7 +462,7 @@ def create_doctests_variants(): | |||
| expansions = dict(TEST_NAME="doctest") | |||
| return [ | |||
| create_variant( | |||
| [".standard-linux .standalone-noauth-nossl"], | |||
| [".test-named .standalone-noauth-nossl"], | |||
There was a problem hiding this comment.
What does "named" mean here?
There was a problem hiding this comment.
I couldn't think of a good name for "tests that use a specific marker or test name"
There was a problem hiding this comment.
Updated to test-standard, test-non-standard, and test-no-orchestration.
|
Can you schedule a patch build with everything? |
| list(product(ALL_VERSIONS, TOPOLOGIES)), ALL_PYTHONS | ||
| ): | ||
| task_combos = [] | ||
| # For each version and topology, rotate through the CPythons. |
There was a problem hiding this comment.
Do these tests not have sync/async versions? Or do we always run both for all of them?
There was a problem hiding this comment.
We run both sync and async, because it is a limited number of tests due to the pytest marker.
|
Full patch build, 1117 tasks across 87 build variants: https://spruce.mongodb.com/version/68123deadf02d30007c0da2c/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC |
|
So far the failures have been due to PyPy, PyOpenSSL, free-threading on macos, or this error which pre-existed: https://jira.mongodb.org/browse/PYTHON-5365. Nothing unexpected. |
|
Most of the jobs have completed, I don't see any concerns. |
| @@ -593,31 +601,32 @@ def create_server_version_tasks(): | |||
| if topology == "sharded_cluster" and auth == "auth" and ssl == "ssl": | |||
| continue | |||
| task_types.append((python, topology, auth, ssl)) | |||
| for python, topology, auth, ssl in task_types: | |||
| for (python, topology, auth, ssl), sync in zip_cycle(task_types, SYNCS): | |||
There was a problem hiding this comment.
Making sure I understand this change: now async and sync are cycled between over our standard tasks? So every other test will be sync, and every other test will be async?
There was a problem hiding this comment.
Can we ensure we still always run sync and async on every combo of version, topology, auth, and ssl?
There was a problem hiding this comment.
Patch build with just the new server version tests, 306 tasks across 9 build variants: https://spruce.mongodb.com/version/68135c314806e0000738a155/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC
There was a problem hiding this comment.
That brings the total up to 1270 tasks across 87 build variants, from 1117 tasks.
There was a problem hiding this comment.
I think that's worth it to ensure 100% coverage of combinations.
See the ticket for more background on the reasoning.
Patch build: https://spruce.mongodb.com/version/680b9b831d659e0007140dda/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC
Doctests:
Load Balancer:
No C Ext (partial list shown):
MongoDB 4.4: