[EXPERIMENT][perf] Replace LLD wrapper with Wild

[mati865]

On my machine, this is a small but measurable improvement using rustc-perf, but the noise makes me question my numbers.
Can we see how this works with Rust's CI?

r? @ghost

try-job: dist-x86_64-linux

[lqd]

@bors try

[bors]

⌛ Trying commit 7149db7 with merge aee6da7...

[mati865]

I didn't even have a chance to ask for the run on Zulip ;)
Thanks.

[rust-log-analyzer]

The job dist-x86_64-linux failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

file:.git/config remote.origin.url=https://github.com/rust-lang-ci/rust
file:.git/config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
file:.git/config gc.auto=0
file:.git/config http.https://github.com/.extraheader=AUTHORIZATION: basic ***
file:.git/config branch.try.remote=origin
file:.git/config branch.try.merge=refs/heads/try
file:.git/config submodule.library/backtrace.active=true
file:.git/config submodule.library/backtrace.url=https://github.com/rust-lang/backtrace-rs.git
file:.git/config submodule.library/stdarch.active=true
file:.git/config submodule.library/stdarch.url=https://github.com/rust-lang/stdarch.git
file:.git/config submodule.src/doc/book.active=true
---
[RUSTC-TIMING] crc32fast test:false 0.209
[RUSTC-TIMING] pkg_config test:false 0.364
[RUSTC-TIMING] foldhash test:false 0.210
[RUSTC-TIMING] sharded_offset_map test:false 0.113
   Compiling bumpalo-herd v0.1.2
[RUSTC-TIMING] arrayvec test:false 0.233
[RUSTC-TIMING] build_script_build test:false 0.108
[RUSTC-TIMING] smallvec test:false 0.259
   Compiling itertools v0.14.0
[RUSTC-TIMING] either test:false 0.224
---
[RUSTC-TIMING] itertools test:false 1.711
[RUSTC-TIMING] build_script_build test:false 0.159
[RUSTC-TIMING] build_script_build test:false 0.164
   Compiling matchers v0.1.0
   Compiling linker-utils v0.4.0 (https://github.com/davidlattimore/wild.git?rev=e046ec6#e046ec6d)
   Compiling libz-rs-sys v0.5.0
[RUSTC-TIMING] matchers test:false 0.096
[RUSTC-TIMING] object test:false 1.930
   Compiling flate2 v1.1.1
[RUSTC-TIMING] libz_rs_sys test:false 0.116
---
   Compiling zstd v0.13.3
[RUSTC-TIMING] zstd_safe test:false 0.139
[RUSTC-TIMING] zstd test:false 0.281
[RUSTC-TIMING] serde test:false 2.275
   Compiling postcard v1.1.1
[RUSTC-TIMING] heapless test:false 0.478
   Compiling linker-layout v0.4.0 (https://github.com/davidlattimore/wild.git?rev=e046ec6#e046ec6d)
   Compiling linker-trace v0.4.0 (https://github.com/davidlattimore/wild.git?rev=e046ec6#e046ec6d)
[RUSTC-TIMING] postcard test:false 0.249
   Compiling libwild v0.4.0 (https://github.com/davidlattimore/wild.git?rev=e046ec6#e046ec6d)
[RUSTC-TIMING] tracing_subscriber test:false 4.256
[RUSTC-TIMING] linker_trace test:false 0.430
[RUSTC-TIMING] linker_layout test:false 0.594
[RUSTC-TIMING] libwild test:false 26.209
   Compiling lld-wrapper v0.1.0 (/checkout/src/tools/lld-wrapper)
---
[RUSTC-TIMING] rustc_borrowck test:false 112.066
   Compiling rustc_driver v0.0.0 (/checkout/compiler/rustc_driver)
error: linking with `clang` failed: exit status: 1
  |
  = note:  "clang" "-Wl,--version-script=/tmp/rustcrksWZR/list" "-Wl,--no-undefined-version" "-m64" "/tmp/rustcrksWZR/symbols.o" "<248 object files omitted>" "<sysroot>-rustc/x86_64-unknown-linux-gnu/release/deps/rustc_driver-52d072462bbd24ad.519i5z6ojre6pacl3mcxxpzp1.rcgu.rmeta" "-Wl,--as-needed" "-Wl,-Bstatic" "/tmp/rustcrksWZR/{librustc_codegen_llvm-6a14a3ed70dffd38.rlib,librustc_llvm-37880870d350b18d.rlib,libblake3-a496edbf5b81f2e1.rlib,libpsm-de23c8734973f806.rlib,libprofiler_builtins-65199d31c78cc52e.rlib}.rlib" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib/{libcompiler_builtins-*}.rlib" "-Wl,-Bdynamic" "-lLLVM-20-rust-1.88.0-nightly" "-ldl" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-L" "/tmp/rustcrksWZR/raw-dylibs" "-B<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,-znostart-stop-gc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "<sysroot>-rustc/x86_64-unknown-linux-gnu/release/build/psm-272fa1000abb8f7d/out" "-L" "<sysroot>-rustc/x86_64-unknown-linux-gnu/release/build/blake3-27aaa8c4245b0918/out" "-L" "<sysroot>-rustc/x86_64-unknown-linux-gnu/release/build/blake3-27aaa8c4245b0918/out" "-L" "<sysroot>-rustc/x86_64-unknown-linux-gnu/release/build/rustc_llvm-5790ef985ade0178/out" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/lib" "-L" "/rustroot/bin/../lib/gcc/x86_64-pc-linux-gnu/9.5.0/../../../../lib64" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "<sysroot>-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_driver-52d072462bbd24ad.so" "-shared" "-Wl,-soname=librustc_driver-52d072462bbd24ad.so" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-u" "__llvm_profile_runtime" "-Wl,-z,origin" "-Wl,-rpath,$ORIGIN/../lib" "-Wl,--icf=all"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: Error: Failed to parse object file `/tmp/rustcrksWZR/librustc_codegen_llvm-6a14a3ed70dffd38.rlib @ ef10e86dc40538c1-SymbolWrapper.o`
          
          Caused by:
              Unsupported ELF header
          
          Stack backtrace:
---
            21: <unknown>
            22: <unknown>
            23: <unknown>
            24: start_thread
            25: __clone
          clang: error: linker command failed with exit code 255 (use -v to see invocation)
          WARNING: wild: --icf=all is not yet supported
          

[RUSTC-TIMING] rustc_driver test:false 98.632
error: could not compile `rustc_driver` (lib) due to 1 previous error
Build completed unsuccessfully in 0:14:24
---
Caused by:
    Command RUST_BACKTRACE=full python3 /checkout/x.py build --target x86_64-unknown-linux-gnu --host x86_64-unknown-linux-gnu --stage 2 library/std --rust-profile-generate /tmp/tmp-multistage/opt-artifacts/rustc-pgo --set llvm.thin-lto=false --set llvm.link-shared=true [at /checkout/obj] has failed with exit code Some(1)

Stack backtrace:
   0: <anyhow::Error>::msg::<alloc::string::String>
             at /rust/deps/anyhow-1.0.97/src/backtrace.rs:27:14
   1: <opt_dist::exec::CmdBuilder>::run
             at /rustc/aee6da7125ba2b2b21c7f67fe4bf9592cec0f2e7/src/tools/opt-dist/src/exec.rs:80:17
   2: <opt_dist::exec::Bootstrap>::run
             at /rustc/aee6da7125ba2b2b21c7f67fe4bf9592cec0f2e7/src/tools/opt-dist/src/exec.rs:181:9
   3: opt_dist::execute_pipeline::{closure#1}::{closure#0}
             at /rustc/aee6da7125ba2b2b21c7f67fe4bf9592cec0f2e7/src/tools/opt-dist/src/main.rs:230:13
   4: <opt_dist::timer::TimerSection>::section::<opt_dist::execute_pipeline::{closure#1}::{closure#0}, ()>
             at /rustc/aee6da7125ba2b2b21c7f67fe4bf9592cec0f2e7/src/tools/opt-dist/src/timer.rs:111:22
   5: opt_dist::execute_pipeline::{closure#1}
             at /rustc/aee6da7125ba2b2b21c7f67fe4bf9592cec0f2e7/src/tools/opt-dist/src/main.rs:219:9
   6: <opt_dist::timer::TimerSection>::section::<opt_dist::execute_pipeline::{closure#1}, opt_dist::training::RustcPGOProfile>
             at /rustc/aee6da7125ba2b2b21c7f67fe4bf9592cec0f2e7/src/tools/opt-dist/src/timer.rs:111:22
   7: opt_dist::execute_pipeline
             at /rustc/aee6da7125ba2b2b21c7f67fe4bf9592cec0f2e7/src/tools/opt-dist/src/main.rs:216:29
   8: opt_dist::main
             at /rustc/aee6da7125ba2b2b21c7f67fe4bf9592cec0f2e7/src/tools/opt-dist/src/main.rs:416:18
   9: <fn() -> core::result::Result<(), anyhow::Error> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/45165c82a4c5315ff52c391ad138f41ff40b52d8/library/core/src/ops/function.rs:250:5
  10: std::sys::backtrace::__rust_begin_short_backtrace::<fn() -> core::result::Result<(), anyhow::Error>, core::result::Result<(), anyhow::Error>>
             at /rustc/45165c82a4c5315ff52c391ad138f41ff40b52d8/library/std/src/sys/backtrace.rs:152:18
  11: std::rt::lang_start::<core::result::Result<(), anyhow::Error>>::{closure#0}
             at /rustc/45165c82a4c5315ff52c391ad138f41ff40b52d8/library/std/src/rt.rs:199:18
  12: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/45165c82a4c5315ff52c391ad138f41ff40b52d8/library/core/src/ops/function.rs:284:13
  13: std::panicking::try::do_call
             at /rustc/45165c82a4c5315ff52c391ad138f41ff40b52d8/library/std/src/panicking.rs:589:40
  14: std::panicking::try
             at /rustc/45165c82a4c5315ff52c391ad138f41ff40b52d8/library/std/src/panicking.rs:552:19

[bors]

💔 Test failed - checks-actions

[mati865]

Yeah...
I was afraid PGOing and BOLTing the compiler might discover new issues. Smaller projects worked, though.

Does Rust use linker-plugin-lto?

EDIT: Why do I see myself there? Is it trying to tell me something?

Build PGO instrumented rustc and LLVM ended: FAIL (865.00s)

[jieyouxu]

@bors delegate+ (for try jobs)

[bors]

✌️ @mati865, you can now approve this pull request!

If @jieyouxu told you to "r=me" after making some further change, please make that change, then do @bors r=@jieyouxu

[jieyouxu]

Does Rust use linker-plugin-lto?

I don't immediately see a use of that flag looking through bootstrap, but I might have missed sth

[Noratrieb]

rustc uses normal rustc thin LTO

[mati865]

I'm more interested in LLVM which IIRC also is optimised as part of PGO.