Support -Clink-self-contained=+linker for ld.lld linker flavor

[lqd]

When using an ld.lld linker flavor, rustc will invoke lld directly. This PR adds support to choose rust-lld instead of the system lld when the self-contained linker is enabled (on the CLI or by the target).

There's some slight wrinkle/cycle here: inferring whether self-contained linking is enabled needs to know the linker (on mingw), but to choose the linker (rust-lld instead of lld) we need to know if the self-contained linker is enabled... So I only check for explicit self-contained linking components in the target (without the inference implied by the LinkSelfContainedDefault infra), as well as the CLI. That seems fine?

The linker command and binary is usually hidden by rustc so while I'm not enamored with the test looking for the linker name from a forced linking error, it feels cleaner/easier/more sensible/more stable than parsing rustc's debug logs looking for the linker that is invoked in the command we log.

Since this is somewhat of a stabilization concern for x64 linux, I focused on the Gnu(Cc::No, Lld::Yes) flavor, where we know it works, and not the other target-specific Lld::Yes flavors. We already support extending the $PATH with the needed directories when the linker is rust-lld so this makes use of that, and it should work on more targets (that we're not stabilizing).

r? @petrochenkov

try-job: x86_64-gnu
try-job: x86_64-gnu-stable
try-job: x86_64-gnu-aux
try-job: x86_64-gnu-debug

[lqd]

@bors try

[bors]

⌛ Trying commit d74189c with merge f7bcc22...

[bors]

☀️ Try build successful - checks-actions
Build commit: f7bcc22 (f7bcc220e390dcb70e71ee0db3fa4b742fe552dc)

[rustbot]

Some changes occurred in compiler/rustc_codegen_ssa

cc @WaffleLapkin

This PR modifies run-make tests.

cc @jieyouxu

[petrochenkov]

I don't think changing the linker name to rust-lld is the right thing in general, although I'm not sure the difference is observable in practice.

I would expect -Clink-self-contained=+linker only to add the <sysroot>/<target>/bin/self-contained/linker to PATH with high priority when invoking the linker (ld.lld in case of the corresponding linker flavor), which is the same thing as -B does in case of cc+lld flavors.

I'd also want to migrate all the bare metal targets from using linker="rust-lld" in target specs to using self-contained=+linker with the default lld name there.
Ideally the path where rust-lld lives shouldn't even be in PATH!

[petrochenkov]

With mingw in particular this logic + logic in detect_self_contained_mingw mean that self-contained=+linker always turns into self-contained=yes :D

[petrochenkov]

Updating PATH will likely be of comparable complexity with this PR.
@rustbot author

[lqd]

I would expect -Clink-self-contained=+linker only to add the //bin/self-contained/linker to PATH with high priority when invoking the linker (ld.lld in case of the corresponding linker flavor), which is the same thing as -B does in case of cc+lld flavors.

I don't see how that would work since rustc calls the lld binary for the ld.lld flavor? We don't distribute any such executable, only a rust-lld and the various ld.lld wrappers: no binaries named lld would take priority in the PATH.

[petrochenkov]

I would expect -Clink-self-contained=+linker only to add the //bin/self-contained/linker to PATH with high priority when invoking the linker (ld.lld in case of the corresponding linker flavor), which is the same thing as -B does in case of cc+lld flavors.

I don't see how that would work since rustc calls the lld binary for the ld.lld flavor? We don't distribute any such executable, only a rust-lld and the various ld.lld wrappers: no binaries named lld would take priority in the PATH.

Ah, right we are using lld -flavor gnu instead of ld.lld.
It's not recommended though.
So it would require some more changes.

UPD: Calling ld.lld -flavor gnu is also fine, which may be a smaller change.

(There's also always an option to keep the non-default option combinations unstable to avoid changes before stabilization.)

[lqd]

Yeah also changing the normal linker invocation last minute feels a bit rushed. We can take more time for analysis and tests to do that. Here, it felt acceptable because it starts from an unstable situation and most things still internally still rely on rust-lld (to the detriment of possibly breaking mingw as you pointed out above).

There's also always an option to keep the non-default option combinations unstable to avoid changes before stabilization

That's interesting. Something like only stabilizing -Clink-self-contained=+/-linker and -Clinker-features=+/-lld for the Gnu(Cc::Yes, _) flavor.

(I guess there's also the option of only stabilizing the opt-outs, to go back to the pre-stabilization state, but not the opt-ins. That's a bit less appealing than the previous idea.)