Skip to content

take into account the num of processes by ulimit #143614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

take into account the num of processes by ulimit #143614

wants to merge 1 commit into from

Conversation

klim-iv
Copy link

@klim-iv klim-iv commented Jul 7, 2025

No description provided.

@rustbot
Copy link
Collaborator

rustbot commented Jul 7, 2025

r? @tgross35

rustbot has assigned @tgross35.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added O-unix Operating system: Unix-like S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jul 7, 2025
@klim-iv
Copy link
Author

klim-iv commented Jul 7, 2025 •
edited
Loading 

due to ulimit restrictions is not used in std::thread
apears crashes like this one:
thread 'main' panicked at .../.cargo/registry/.../rayon-core-1.12.1/src/registry.rs:168:10:
The global thread pool has not been initialized.: ThreadPoolBuildError
 { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }

tgross35
tgross35 reviewed Jul 7, 2025
View reviewed changes
library/std/src/sys/pal/unix/thread.rs
@@ -440,13 +440,24 @@ pub fn available_parallelism() -> io::Result<NonZero<usize>> {
}
}
}

let mut ulimit = u64::MAX;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type is rlim_t, this won't work on x86-32

library/std/src/sys/pal/unix/thread.rs
Comment on lines +445 to +446
#[cfg(any(target_os = "android", target_os = "linux"))]
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The above block has the same cfg, this can be combined

@tgross35
Copy link
Contributor

tgross35 commented Jul 7, 2025

Could you please put a more detailed description of how the problem is reproduced in the PR description? Or open a separate issue for it.

The change here also needs comments, and the docs at https://doc.rust-lang.org/beta/std/thread/fn.available_parallelism.html should be updated if this winds up being accepted.

Since this is a user-visible change this will need libs-api signoff, but first cc @cuviper since it looks like the issue shows up in Rayon (possibly rayon-rs/rayon#694?)

@rust-log-analyzer
Copy link
Collaborator

The job pr-check-1 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

@cuviper
Copy link
Member

cuviper commented Jul 7, 2025

I doubt that this will be useful -- e.g. starting a number of threads equal to the current RLIMIT_NPROC is already doomed by the existence of the current thread, nevermind any other threads or even other processes.

cuviper
cuviper reviewed Jul 7, 2025
View reviewed changes
library/std/src/sys/pal/unix/thread.rs
let mut r: libc::rlimit = unsafe { mem::zeroed() };
unsafe {
if libc::getrlimit(libc::RLIMIT_NPROC, &mut r) == 0 {
ulimit = r.rlim_max
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, the "soft" rlim_cur is the value that is currently enforced, while the "hard" rlim_max is a limit on how high an unprivileged process can adjust its own rlim_cur.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cuviper cuviper cuviper left review comments

@tgross35 tgross35 tgross35 left review comments

Assignees

@tgross35 tgross35

Labels
O-unix Operating system: Unix-like S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@klim-iv @rustbot @tgross35 @rust-log-analyzer @cuviper