[Tests] Fix mps reproducibility issue when running with pytest-xdist (huggingface#976)

* [WIP] Debugging mps DDIM tests

* revert num_steps

* check warmup with a generator

* more warmup!

* remove xdist

* just use a single process

Author: anton-l

.github/workflows/pr_tests.yml

@@ -89,7 +89,7 @@ jobs:
     - name: Run all fast tests on MPS
       shell: arch -arch arm64 bash {0}
       run: |
-        ${CONDA_RUN} python -m pytest -n 4 --max-worker-restart=0 --dist=loadfile -s -v --make-reports=tests_torch_mps tests/
+        ${CONDA_RUN} python -m pytest -n 1 -s -v --make-reports=tests_torch_mps tests/
 
     - name: Failure short reports
       if: ${{ failure() }}