Linking C runtime without the C compiler

[davidlattimore]

This is a continuation of a discussion on #112 with @madsmtm and @mati865. I figured I'd move the discussion here, since it's not clear yet what if anything would need to change in Wild. We might well file a new issue once we have some idea.

It'd be nice if it were possible to integrate Wild into compilers and do linking directly from the compiler. The main impediment to this is that the C compiler knows what object files and flags are needed in order to link the C runtime, whereas compilers for high level languages generally don't.

Running rustc for example and telling it to invoke the linker directly is pretty tricky:

❯ rustc -Clinker=ld main.rs
  = note: ld: cannot find -lgcc_s: No such file or directory

To get a hello-world binary that runs, we likely need something like this:

❯ rustc -Clinker=ld -Clink-args='/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o -L/lib/x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/11 --dynamic-linker=/lib64/ld-linux-x86-64.so.2 /usr/lib/gcc/x86_64-linux-gnu/11/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crtn.o' main.rs

This does run, however suffers from two problems. It's a pain to figure out what arguments to pass. It's also not entirely correct, since objects like Scrt1.o are supposed to be at the start before any other objects, but rustc doesn't seem to give us a way to add linker arguments at the start. This would mostly show up as a problem if some user-supplied object added to the .init. or .fini sections, which these days is pretty unlikely.

It'd be nice if there was a standard way to find all the arguments that need to go at the start and end of the link command on a particular platform. These arguments also depend on the link mode. e.g. static linking will add different objects.

Another option would be for the compiler to decide not to use those bits of the C runtime and provide its own runtime for the target platform. That runtime potentially doesn't need to do very much. e.g. call __libc_start_main, passing a pointer to main. IMO, a library like libc should be usable without the need for a C compiler. i.e. it should be documented and stable as to what symbols need to be provided and what initialisation functions need to be called and when.

[mati865]

My initial idea is to replicate what Clang does, hardcode a list of known locations, and iterate over them to find the right ones: https://github.com/llvm/llvm-project/blob/3f648992bf317a3496c4d137374d2c1532423d1c/clang/lib/Driver/ToolChains/Gnu.cpp#L2199-L2222

So, a binary called cc-wild or whatever would be like any other linker proxy, except it would call wild as a library to avoid spawning additional processes. That shouldn't be hard to achieve.

Your idea sounds cool but probably needs more effort to implement correctly. Perhaps there could be two approaches within the same binary, so if one fails, the user can try the other (or even provide automatic fallback).

[mati865]

I planned to do it later, but CMake annoyed me enough to reorder my priorities: https://github.com/mati865/wild-proxy

Currently, only the fallback (calling gcc/clang and parsing its arguments) is implemented, but my goal (albeit not the highest priority) remains to implement this in a way to avoid calling compiler binaries.

I wanted to see how much slower is fallback compared to using compilers directly and:

❯ PATH=~/Projects/wild-proxy:$PATH powerprofilesctl launch -p performance hyperfine --warmup 5 -N "g++ hello.cpp" "g++ hello.cpp -B ~/Projects/wild" "wild-g++ hello.cpp" "clang++ hello.cpp" "clang++ hello.cpp -B ~/Projects/wild" "wild-clang++ hello.cpp"
Benchmark 1: g++ hello.cpp
  Time (mean ± σ):     159.8 ms ±   2.0 ms    [User: 142.1 ms, System: 17.3 ms]
  Range (min … max):   156.3 ms … 164.3 ms    18 runs

Benchmark 2: g++ hello.cpp -B ~/Projects/wild
  Time (mean ± σ):     159.4 ms ±   1.6 ms    [User: 141.3 ms, System: 17.6 ms]
  Range (min … max):   156.0 ms … 163.7 ms    18 runs

Benchmark 3: wild-g++ hello.cpp
  Time (mean ± σ):     143.6 ms ±   2.0 ms    [User: 153.7 ms, System: 83.3 ms]
  Range (min … max):   141.1 ms … 147.9 ms    20 runs

Benchmark 4: clang++ hello.cpp
  Time (mean ± σ):     173.3 ms ±   1.7 ms    [User: 146.8 ms, System: 25.8 ms]
  Range (min … max):   170.8 ms … 176.4 ms    17 runs

Benchmark 5: clang++ hello.cpp -B ~/Projects/wild
  Time (mean ± σ):     173.2 ms ±   1.4 ms    [User: 144.9 ms, System: 27.7 ms]
  Range (min … max):   170.6 ms … 175.6 ms    17 runs

Benchmark 6: wild-clang++ hello.cpp
  Time (mean ± σ):     159.9 ms ±   2.7 ms    [User: 161.9 ms, System: 95.4 ms]
  Range (min … max):   156.9 ms … 166.4 ms    18 runs

Summary
  wild-g++ hello.cpp ran
    1.11 ± 0.02 times faster than g++ hello.cpp -B ~/Projects/wild
    1.11 ± 0.02 times faster than g++ hello.cpp
    1.11 ± 0.02 times faster than wild-clang++ hello.cpp
    1.21 ± 0.02 times faster than clang++ hello.cpp -B ~/Projects/wild
    1.21 ± 0.02 times faster than clang++ hello.cpp

This is fully reproducible. Clearly, something is broken/missing.

[mati865]

Some fixes later, it can successfully be used as a drop-in replacement for Rust and C/C++:

Wild:

❯ RUSTFLAGS="-C linker=$HOME/Projects/wild-proxy/wild-cc" cargo b
...
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 9.36s

❯ readelf -p .comment target/debug/wild

String dump of section '.comment':
  [     0]  GCC: (GNU) 14.2.1 20250207
  [    1b]  rustc version 1.86.0 (05f9846f8 2025-03-31)
  [    48]  Linker: Wild version 0.4.0

LLVM (this used to work but now fails even with regular Wild):

❯ PATH=~/Projects/wild-proxy:$PATH CC=wild-gcc CXX=wild-g++ cmake -DLLVM_ENABLE_PROJECTS=clang -DCMAKE_BUILD_TYPE=Release -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ../llvm

❯ ninja
...

I'll try to debug it tomorrow:

(lldb) r
Process 208592 launched: '/home/mateusz/Projects/llvm-project/build/bin/llvm-tblgen' (x86_64)
Process 208592 stopped
* thread #1, name = 'llvm-tblgen', stop reason = signal SIGSEGV: invalid permissions for mapped object (fault address: 0x555555554000)
    frame #0: 0x0000555555554000 llvm-tblgen
llvm-tblgen`__ehdr_start:
->  0x555555554000 <+0>: jg     0x555555554047 ; llvm-tblgen.PT_LOAD[0] + 71

[davidlattimore]

I just noticed that this post by Maskray goes into a bit of detail about rules for how the compiler decides what arguments to pass to the linker. Might be a useful reference.