Skip to content

--fix hangs with parallel rustc #140093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
matthiaskrgr opened this issue Apr 20, 2025 · 2 comments · Fixed by #140145
Closed

--fix hangs with parallel rustc #140093

matthiaskrgr opened this issue Apr 20, 2025 · 2 comments · Fixed by #140145
Labels
A-parallel-compiler Area: parallel compiler C-bug Category: This is a bug.

Comments

@matthiaskrgr
Copy link
Member

repro:

git clone https://github.com/rust-lang/rust-analyzer
cd rust-analyzer
git checkout 723121e5958cf282db3fdb06970776724a7326d5
RUSTFLAGS="-Zthreads=2" cargo clippy --fix    -- -Aclippy::all  -Wclippy::clone_on_copy

At some point there will be no more progress during the builds, it looks like we get stuck on build.rs scripts?
Works fine without the -Zthreads=2 and also works fine without the --fix

What is weird that we don't seem to hang with active cpu load, but I can still see some rustc processes running

rustc 1.88.0-nightly (077cedc2a 2025-04-19)
binary: rustc
commit-hash: 077cedc2afa8ac0b727b7a6cbe012940ba228deb
commit-date: 2025-04-19
host: x86_64-unknown-linux-gnu
release: 1.88.0-nightly
LLVM version: 20.1.2
@matthiaskrgr matthiaskrgr added the C-bug Category: This is a bug. label Apr 20, 2025
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Apr 20, 2025
@matthiaskrgr matthiaskrgr added A-parallel-compiler Area: parallel compiler and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Apr 20, 2025
@matthiaskrgr
Copy link
Member Author

also reproducible on stable rustc 1.86.0

@matthiaskrgr matthiaskrgr changed the title clippy --fix hangs with parallel rustc --fix hangs with parallel rustc Apr 20, 2025
@matthiaskrgr
Copy link
Member Author

also happens with just RUSTFLAGS="-Zthreads=2" cargo fix 🤔

@Zoxc Zoxc mentioned this issue Apr 22, 2025
Merged
bors added a commit to rust-lang-ci/rust that referenced this issue Apr 22, 2025
@bors
Auto merge of rust-lang#140145 - Zoxc:job-server-proxy, r=<try>
c4cd906 
Add a jobserver proxy to ensure at least one token is always held

This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.

The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.

Fixes rust-lang#133873 and rust-lang#140093.
@bors bors closed this as completed in 6e23095 May 1, 2025
lnicola pushed a commit to lnicola/rust-analyzer that referenced this issue May 1, 2025
@bors
Auto merge of #140145 - Zoxc:job-server-proxy, r=SparrowLii
a2052e9 
Add a jobserver proxy to ensure at least one token is always held

This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.

The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.

Fixes rust-lang/rust#67385.
Fixes rust-lang/rust#133873.
Fixes rust-lang/rust#140093.
github-actions bot pushed a commit to rust-lang/miri that referenced this issue May 3, 2025
@bors
Auto merge of #140145 - Zoxc:job-server-proxy, r=SparrowLii
16fc6b8 
Add a jobserver proxy to ensure at least one token is always held

This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.

The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.

Fixes rust-lang/rust#67385.
Fixes rust-lang/rust#133873.
Fixes rust-lang/rust#140093.
github-actions bot pushed a commit to rust-lang/rustc-dev-guide that referenced this issue May 5, 2025
@bors
Auto merge of #140145 - Zoxc:job-server-proxy, r=SparrowLii
8ca711c 
Add a jobserver proxy to ensure at least one token is always held

This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.

The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.

Fixes rust-lang/rust#67385.
Fixes rust-lang/rust#133873.
Fixes rust-lang/rust#140093.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-parallel-compiler Area: parallel compiler C-bug Category: This is a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add a jobserver proxy to ensure at least one token is always held Zoxc/rust
2 participants
@matthiaskrgr @rustbot