Skip to content

-Zthreads causes rustfix/jobserver to deadlock #133873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
kornelski opened this issue Dec 4, 2024 · 0 comments · Fixed by #140145
Closed

-Zthreads causes rustfix/jobserver to deadlock #133873

kornelski opened this issue Dec 4, 2024 · 0 comments · Fixed by #140145
Labels
A-parallel-compiler Area: parallel compiler C-bug Category: This is a bug. I-hang Issue: The compiler never terminates, due to infinite loops, deadlock, livelock, etc. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@kornelski
Copy link
Contributor 

git clone --recursive -b rustfix_deadlock_repro https://gitlab.com/lib.rs/main.git/
cd main
cargo +nightly fix --edition-idioms --all --allow-dirty
cargo +nightly fix --edition-idioms --all --allow-dirty

The config in .cargo/config.toml has -Zthreads=10. I'm running it on 16-core Apple M3 Max. I can also reproduce the deadlock with -Zthreads=2. It doesn't deadlock without -Zthreads.

The deadlock happens reliably when running cargo fix. It doesn't happen when running cargo test or cargo check.

Cargo is waiting in rustfix_crate on a TCP read, waiting for rustc to give signs of life. It can get stuck with even a single rustc process running.

Rustc process is stuck here:

* thread #1, name = 'main', queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x000000018abdfab0 libsystem_kernel.dylib`read + 8
    frame #1: 0x000000010d9929c0 librustc_driver-ad3c18566d12557c.dylib`<jobserver::imp::Client>::acquire_allow_interrupts + 60
    frame #2: 0x000000010e296584 librustc_driver-ad3c18566d12557c.dylib`rustc_data_structures::jobserver::acquire_thread + 60
    frame #3: 0x000000010da4dc6c librustc_driver-ad3c18566d12557c.dylib`<rayon_core::thread_pool::ThreadPool>::wait_until_stopped + 272
    frame #4: 0x000000010e2e9958 librustc_driver-ad3c18566d12557c.dylib`rustc_interface::util::run_in_thread_pool_with_globals::<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>> + 7224
    frame #5: 0x000000010e329b2c librustc_driver-ad3c18566d12557c.dylib`rustc_driver_impl::run_compiler + 6600
    frame #6: 0x000000010e335e20 librustc_driver-ad3c18566d12557c.dylib`rustc_driver_impl::main + 824
    frame #7: 0x00000001043f2cd0 rustc`rustc_main::main + 12
    frame #8: 0x00000001043f2c84 rustc`std::sys::backtrace::__rust_begin_short_backtrace::<fn(), ()> + 12
    frame #9: 0x00000001043f2c9c rustc`std::rt::lang_start::<()>::{closure#0} + 16
    frame #10: 0x000000010ff27dd4 librustc_driver-ad3c18566d12557c.dylib`std::rt::lang_start_internal::he7368dee48875b7d + 1092
    frame #11: 0x00000001043f2d04 rustc`main + 52
    frame #12: 0x000000018a8a0274 dyld`start + 2840
  thread #2, name = 'ctrl-c'
    frame #0: 0x000000018abdfab0 libsystem_kernel.dylib`read + 8
    frame #1: 0x000000010e2eb480 librustc_driver-ad3c18566d12557c.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<ctrlc::set_handler_inner<rustc_driver_impl::install_ctrlc_handler::{closure#0}>::{closure#0}, ()> + 48
    frame #2: 0x000000010e2fb4b0 librustc_driver-ad3c18566d12557c.dylib`<<std::thread::Builder>::spawn_unchecked_<ctrlc::set_handler_inner<rustc_driver_impl::install_ctrlc_handler::{closure#0}>::{closure#0}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 488
    frame #3: 0x000000010ff4ecc8 librustc_driver-ad3c18566d12557c.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::hf11e21675a82323b + 52
    frame #4: 0x000000018ac202e4 libsystem_pthread.dylib`_pthread_start + 136

Backtrace from another run when there was only a single rustc process spawned:

bt all
* thread #1, name = 'main', queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x000000018abe26cc libsystem_kernel.dylib`__psynch_cvwait + 8
    frame #1: 0x000000018ac20894 libsystem_pthread.dylib`_pthread_cond_wait + 1204
    frame #2: 0x000000010c272390 librustc_driver-ad3c18566d12557c.dylib`<rayon_core::latch::LockLatch>::wait + 136
    frame #3: 0x000000010c275c4c librustc_driver-ad3c18566d12557c.dylib`<rayon_core::thread_pool::ThreadPool>::wait_until_stopped + 240
    frame #4: 0x000000010cb11958 librustc_driver-ad3c18566d12557c.dylib`rustc_interface::util::run_in_thread_pool_with_globals::<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>> + 7224
    frame #5: 0x000000010cb51b2c librustc_driver-ad3c18566d12557c.dylib`rustc_driver_impl::run_compiler + 6600
    frame #6: 0x000000010cb5de20 librustc_driver-ad3c18566d12557c.dylib`rustc_driver_impl::main + 824
    frame #7: 0x0000000102cfecd0 rustc`rustc_main::main + 12
    frame #8: 0x0000000102cfec84 rustc`std::sys::backtrace::__rust_begin_short_backtrace::<fn(), ()> + 12
    frame #9: 0x0000000102cfec9c rustc`std::rt::lang_start::<()>::{closure#0} + 16
    frame #10: 0x000000010e74fdd4 librustc_driver-ad3c18566d12557c.dylib`std::rt::lang_start_internal::he7368dee48875b7d + 1092
    frame #11: 0x0000000102cfed04 rustc`main + 52
    frame #12: 0x000000018a8a0274 dyld`start + 2840
  thread #2, name = 'ctrl-c'
    frame #0: 0x000000018abdfab0 libsystem_kernel.dylib`read + 8
    frame #1: 0x000000010cb13480 librustc_driver-ad3c18566d12557c.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<ctrlc::set_handler_inner<rustc_driver_impl::install_ctrlc_handler::{closure#0}>::{closure#0}, ()> + 48
    frame #2: 0x000000010cb234b0 librustc_driver-ad3c18566d12557c.dylib`<<std::thread::Builder>::spawn_unchecked_<ctrlc::set_handler_inner<rustc_driver_impl::install_ctrlc_handler::{closure#0}>::{closure#0}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 488
    frame #3: 0x000000010e776cc8 librustc_driver-ad3c18566d12557c.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::hf11e21675a82323b + 52
    frame #4: 0x000000018ac202e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #3, name = 'rustc'
    frame #0: 0x000000018abdfab0 libsystem_kernel.dylib`read + 8
    frame #1: 0x000000010c1ba9c0 librustc_driver-ad3c18566d12557c.dylib`<jobserver::imp::Client>::acquire_allow_interrupts + 60
    frame #2: 0x000000010cabe584 librustc_driver-ad3c18566d12557c.dylib`rustc_data_structures::jobserver::acquire_thread + 60
    frame #3: 0x0000000110e54098 librustc_driver-ad3c18566d12557c.dylib`<rayon_core::sleep::Sleep>::sleep + 744
    frame #4: 0x0000000110e53d5c librustc_driver-ad3c18566d12557c.dylib`<rayon_core::registry::WorkerThread>::wait_until_cold + 324
    frame #5: 0x000000010c272764 librustc_driver-ad3c18566d12557c.dylib`<rayon_core::registry::ThreadBuilder>::run + 344
    frame #6: 0x000000010cb24070 librustc_driver-ad3c18566d12557c.dylib`<<crossbeam_utils::thread::ScopedThreadBuilder>::spawn<<rayon_core::ThreadPoolBuilder>::build_scoped<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#0}, rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}::{closure#0}::{closure#0}, ()>::{closure#0} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 192
    frame #7: 0x000000010cb13414 librustc_driver-ad3c18566d12557c.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()> + 32
    frame #8: 0x000000010cb230c8 librustc_driver-ad3c18566d12557c.dylib`<<std::thread::Builder>::spawn_unchecked_<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 456
    frame #9: 0x000000010e776cc8 librustc_driver-ad3c18566d12557c.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::hf11e21675a82323b + 52
    frame #10: 0x000000018ac202e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #4, name = 'rustc'
    frame #0: 0x000000018abdfab0 libsystem_kernel.dylib`read + 8
    frame #1: 0x000000010c1ba9c0 librustc_driver-ad3c18566d12557c.dylib`<jobserver::imp::Client>::acquire_allow_interrupts + 60
    frame #2: 0x000000010cabe584 librustc_driver-ad3c18566d12557c.dylib`rustc_data_structures::jobserver::acquire_thread + 60
    frame #3: 0x000000010c2726bc librustc_driver-ad3c18566d12557c.dylib`<rayon_core::registry::ThreadBuilder>::run + 176
    frame #4: 0x000000010cb24070 librustc_driver-ad3c18566d12557c.dylib`<<crossbeam_utils::thread::ScopedThreadBuilder>::spawn<<rayon_core::ThreadPoolBuilder>::build_scoped<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#0}, rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}::{closure#0}::{closure#0}, ()>::{closure#0} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 192
    frame #5: 0x000000010cb13414 librustc_driver-ad3c18566d12557c.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()> + 32
    frame #6: 0x000000010cb230c8 librustc_driver-ad3c18566d12557c.dylib`<<std::thread::Builder>::spawn_unchecked_<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 456
    frame #7: 0x000000010e776cc8 librustc_driver-ad3c18566d12557c.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::hf11e21675a82323b + 52
    frame #8: 0x000000018ac202e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #5, name = 'rustc'
    frame #0: 0x000000018abdfab0 libsystem_kernel.dylib`read + 8
    frame #1: 0x000000010c1ba9c0 librustc_driver-ad3c18566d12557c.dylib`<jobserver::imp::Client>::acquire_allow_interrupts + 60
    frame #2: 0x000000010cabe584 librustc_driver-ad3c18566d12557c.dylib`rustc_data_structures::jobserver::acquire_thread + 60
    frame #3: 0x000000010c2726bc librustc_driver-ad3c18566d12557c.dylib`<rayon_core::registry::ThreadBuilder>::run + 176
    frame #4: 0x000000010cb24070 librustc_driver-ad3c18566d12557c.dylib`<<crossbeam_utils::thread::ScopedThreadBuilder>::spawn<<rayon_core::ThreadPoolBuilder>::build_scoped<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#0}, rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}::{closure#0}::{closure#0}, ()>::{closure#0} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 192
    frame #5: 0x000000010cb13414 librustc_driver-ad3c18566d12557c.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()> + 32
    frame #6: 0x000000010cb230c8 librustc_driver-ad3c18566d12557c.dylib`<<std::thread::Builder>::spawn_unchecked_<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 456
    frame #7: 0x000000010e776cc8 librustc_driver-ad3c18566d12557c.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::hf11e21675a82323b + 52
    frame #8: 0x000000018ac202e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #6, name = 'rustc'
    frame #0: 0x000000018abdfab0 libsystem_kernel.dylib`read + 8
    frame #1: 0x000000010c1ba9c0 librustc_driver-ad3c18566d12557c.dylib`<jobserver::imp::Client>::acquire_allow_interrupts + 60
    frame #2: 0x000000010cabe584 librustc_driver-ad3c18566d12557c.dylib`rustc_data_structures::jobserver::acquire_thread + 60
    frame #3: 0x000000010c2726bc librustc_driver-ad3c18566d12557c.dylib`<rayon_core::registry::ThreadBuilder>::run + 176
    frame #4: 0x000000010cb24070 librustc_driver-ad3c18566d12557c.dylib`<<crossbeam_utils::thread::ScopedThreadBuilder>::spawn<<rayon_core::ThreadPoolBuilder>::build_scoped<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#0}, rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}::{closure#0}::{closure#0}, ()>::{closure#0} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 192
    frame #5: 0x000000010cb13414 librustc_driver-ad3c18566d12557c.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()> + 32
    frame #6: 0x000000010cb230c8 librustc_driver-ad3c18566d12557c.dylib`<<std::thread::Builder>::spawn_unchecked_<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 456
    frame #7: 0x000000010e776cc8 librustc_driver-ad3c18566d12557c.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::hf11e21675a82323b + 52
    frame #8: 0x000000018ac202e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #7, name = 'rustc'
    frame #0: 0x000000018abdfab0 libsystem_kernel.dylib`read + 8
    frame #1: 0x000000010c1ba9c0 librustc_driver-ad3c18566d12557c.dylib`<jobserver::imp::Client>::acquire_allow_interrupts + 60
    frame #2: 0x000000010cabe584 librustc_driver-ad3c18566d12557c.dylib`rustc_data_structures::jobserver::acquire_thread + 60
    frame #3: 0x000000010c2726bc librustc_driver-ad3c18566d12557c.dylib`<rayon_core::registry::ThreadBuilder>::run + 176
    frame #4: 0x000000010cb24070 librustc_driver-ad3c18566d12557c.dylib`<<crossbeam_utils::thread::ScopedThreadBuilder>::spawn<<rayon_core::ThreadPoolBuilder>::build_scoped<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#0}, rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}::{closure#0}::{closure#0}, ()>::{closure#0} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 192
    frame #5: 0x000000010cb13414 librustc_driver-ad3c18566d12557c.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()> + 32
    frame #6: 0x000000010cb230c8 librustc_driver-ad3c18566d12557c.dylib`<<std::thread::Builder>::spawn_unchecked_<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 456
    frame #7: 0x000000010e776cc8 librustc_driver-ad3c18566d12557c.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::hf11e21675a82323b + 52
    frame #8: 0x000000018ac202e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #8, name = 'rustc'
    frame #0: 0x000000018abdfab0 libsystem_kernel.dylib`read + 8
    frame #1: 0x000000010c1ba9c0 librustc_driver-ad3c18566d12557c.dylib`<jobserver::imp::Client>::acquire_allow_interrupts + 60
    frame #2: 0x000000010cabe584 librustc_driver-ad3c18566d12557c.dylib`rustc_data_structures::jobserver::acquire_thread + 60
    frame #3: 0x000000010c2726bc librustc_driver-ad3c18566d12557c.dylib`<rayon_core::registry::ThreadBuilder>::run + 176
    frame #4: 0x000000010cb24070 librustc_driver-ad3c18566d12557c.dylib`<<crossbeam_utils::thread::ScopedThreadBuilder>::spawn<<rayon_core::ThreadPoolBuilder>::build_scoped<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#0}, rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}::{closure#0}::{closure#0}, ()>::{closure#0} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 192
    frame #5: 0x000000010cb13414 librustc_driver-ad3c18566d12557c.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()> + 32
    frame #6: 0x000000010cb230c8 librustc_driver-ad3c18566d12557c.dylib`<<std::thread::Builder>::spawn_unchecked_<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 456
    frame #7: 0x000000010e776cc8 librustc_driver-ad3c18566d12557c.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::hf11e21675a82323b + 52
    frame #8: 0x000000018ac202e4 libsystem_pthread.dylib`_pthread_start + 136
  thread #9, name = 'rustc'
    frame #0: 0x000000018abdfab0 libsystem_kernel.dylib`read + 8
    frame #1: 0x000000010c1ba9c0 librustc_driver-ad3c18566d12557c.dylib`<jobserver::imp::Client>::acquire_allow_interrupts + 60
    frame #2: 0x000000010cabe584 librustc_driver-ad3c18566d12557c.dylib`rustc_data_structures::jobserver::acquire_thread + 60
    frame #3: 0x000000010c2726bc librustc_driver-ad3c18566d12557c.dylib`<rayon_core::registry::ThreadBuilder>::run + 176
    frame #4: 0x000000010cb24070 librustc_driver-ad3c18566d12557c.dylib`<<crossbeam_utils::thread::ScopedThreadBuilder>::spawn<<rayon_core::ThreadPoolBuilder>::build_scoped<rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#0}, rustc_interface::util::run_in_thread_pool_with_globals<rustc_interface::interface::run_compiler<core::result::Result<(), rustc_span::ErrorGuaranteed>, rustc_driver_impl::run_compiler::{closure#0}>::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#3}::{closure#0}::{closure#1}, core::result::Result<(), rustc_span::ErrorGuaranteed>>::{closure#0}::{closure#0}::{closure#0}, ()>::{closure#0} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 192
    frame #5: 0x000000010cb13414 librustc_driver-ad3c18566d12557c.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()> + 32
    frame #6: 0x000000010cb230c8 librustc_driver-ad3c18566d12557c.dylib`<<std::thread::Builder>::spawn_unchecked_<alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send>, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} + 456
    frame #7: 0x000000010e776cc8 librustc_driver-ad3c18566d12557c.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::hf11e21675a82323b + 52
    frame #8: 0x000000018ac202e4 libsystem_pthread.dylib`_pthread_start + 136

rustc 1.85.0-nightly (c44b3d50f 2024-12-03)
binary: rustc
commit-hash: c44b3d50fea96a3e0417e8264c16ea21a0a3fca2
commit-date: 2024-12-03
host: aarch64-apple-darwin
release: 1.85.0-nightly
LLVM version: 19.1.4
@kornelski kornelski added the C-bug Category: This is a bug. label Dec 4, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Dec 4, 2024
@fmease fmease added I-hang Issue: The compiler never terminates, due to infinite loops, deadlock, livelock, etc. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. A-parallel-compiler Area: parallel compiler labels Dec 4, 2024
@jieyouxu jieyouxu removed the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Dec 5, 2024
@ehuss ehuss marked this as a duplicate of rust-lang/cargo#15175 Feb 12, 2025
@ehuss ehuss mentioned this issue Feb 12, 2025
Closed
@Zoxc Zoxc mentioned this issue Apr 22, 2025
Merged
bors added a commit to rust-lang-ci/rust that referenced this issue Apr 22, 2025
@bors
Auto merge of rust-lang#140145 - Zoxc:job-server-proxy, r=<try>
c4cd906 
Add a jobserver proxy to ensure at least one token is always held

This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.

The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.

Fixes rust-lang#133873 and rust-lang#140093.
@bors bors closed this as completed in 6e23095 May 1, 2025
lnicola pushed a commit to lnicola/rust-analyzer that referenced this issue May 1, 2025
@bors
Auto merge of #140145 - Zoxc:job-server-proxy, r=SparrowLii
a2052e9 
Add a jobserver proxy to ensure at least one token is always held

This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.

The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.

Fixes rust-lang/rust#67385.
Fixes rust-lang/rust#133873.
Fixes rust-lang/rust#140093.
github-actions bot pushed a commit to rust-lang/miri that referenced this issue May 3, 2025
@bors
Auto merge of #140145 - Zoxc:job-server-proxy, r=SparrowLii
16fc6b8 
Add a jobserver proxy to ensure at least one token is always held

This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.

The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.

Fixes rust-lang/rust#67385.
Fixes rust-lang/rust#133873.
Fixes rust-lang/rust#140093.
github-actions bot pushed a commit to rust-lang/rustc-dev-guide that referenced this issue May 5, 2025
@bors
Auto merge of #140145 - Zoxc:job-server-proxy, r=SparrowLii
8ca711c 
Add a jobserver proxy to ensure at least one token is always held

This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.

The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.

Fixes rust-lang/rust#67385.
Fixes rust-lang/rust#133873.
Fixes rust-lang/rust#140093.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-parallel-compiler Area: parallel compiler C-bug Category: This is a bug. I-hang Issue: The compiler never terminates, due to infinite loops, deadlock, livelock, etc. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add a jobserver proxy to ensure at least one token is always held Zoxc/rust
4 participants
@kornelski @fmease @jieyouxu @rustbot