Skip to content

Multiple artifacts and/or metadata files are declared at the same output location unpack.sh #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
steveklabnik opened this issue May 5, 2025 · 5 comments
Closed

Multiple artifacts and/or metadata files are declared at the same output location unpack.sh #1

steveklabnik opened this issue May 5, 2025 · 5 comments

Comments

@steveklabnik
Copy link

steveklabnik commented May 5, 2025 •
edited
Loading

First of all, very cool!

Second, I decided to give this a build on my machine. I'm using WSL, and I decided to try out dotslash:

#!/usr/bin/env dotslash

{
  "name": "buck2",
  "platforms": {
    "linux-x86_64": {
      "size": 31679931,
      "hash": "blake3",
      "digest": "18568d5b0d78748a9088dfce328c69ee81a4fb7e8017d3e679db65c285994ce1",
      "format": "zst",
      "path": "buck2",
      "providers": [
        {
          "type": "github-release",
          "repo": "facebook/buck2",
          "tag": "2025-05-01",
          "name": "buck2-x86_64-unknown-linux-gnu.zst"
        }
      ]
    }
  }
}

When I try to ./buck2 run stage2:rustc -- --version --verbose I get this:

Build ID: ddbeda42-644a-479d-9235-5245d2706270
Command: run.
Time elapsed: 0.0s
BUILD FAILED
Error running analysis for `rust//stage2:rustc (cfg:<empty>#397dda5f85c5448a)`

Caused by:
    0: Error running analysis for `rust//stage2:rustc (rust//platforms/stage2:compiler#7979eef5c530a256)`
    1: Error running analysis for `rust//:rustc-main-rustc-main (rust//platforms/stage2:compiler#7979eef5c530a256)`
    2: Error running analysis for `rust//:rustc-main-0.0.0-rustc-main (rust//platforms/stage2:compiler#7979eef5c530a256)`
    3: Error running analysis for `toolchains//:rust (rust//platforms/stage2:compiler#7979eef5c530a256) (rust//platforms/stage2:compiler-build-script#11300d7bb94cd8d6)`
    4: Error running analysis for `rust//stage1:rustc (rust//platforms/stage2:compiler-build-script#11300d7bb94cd8d6)`
    5: Error running analysis for `rust//stage1:rustc (rust//platforms/stage1:compiler#9b3dac76c891780a)`
    6: Error running analysis for `rust//:rustc-main-rustc-main (rust//platforms/stage1:compiler#9b3dac76c891780a)`
    7: Error running analysis for `rust//:rustc-main-0.0.0-rustc-main (rust//platforms/stage1:compiler#9b3dac76c891780a)`
    8: Error running analysis for `toolchains//:rust (rust//platforms/stage1:compiler#9b3dac76c891780a) (rust//platforms/stage1:compiler-build-script#4fa65a1586118b89)`
    9: Error running analysis for `rust//stage0:rustdoc (rust//platforms/stage1:compiler-build-script#4fa65a1586118b89)`
   10: Error running analysis for `rust//stage0:rustdoc (stage0#e75f176f4461b34e)`
   11: Error running analysis for `rust//stage0:extract (stage0#e75f176f4461b34e)`
   12: Traceback (most recent call last):
         File <builtin>, in <module>
         * stage0/defs.bzl:132, in _stage0_extract_impl
             contents, _ = unarchive(
         * prelude/http_archive/unarchive.bzl:169, in unarchive
             script, _ = ctx.actions.write(

       error: Multiple artifacts and/or metadata files are declared at the same output location `unpack.sh` declared at `prelude/http_archive/unarchive.bzl:169:17-178:6`.
          --> prelude/http_archive/unarchive.bzl:169:17
           |
       169 |       script, _ = ctx.actions.write(
           |  _________________^
       170 | |         "unpack.{}".format(ext),
       171 | |         [
       172 | |             cmd_args(script_output, format = mkdir),
       173 | |             cmd_args(script_output, format = "cd {}"),
       174 | |             cmd_args([unarchive_cmd] + exclude_flags, delimiter = " ", relative_to = script_output),
       175 | |         ],
       176 | |         is_executable = True,
       177 | |         allow_args = True,
       178 | |     )
           | |_____^
           |

I'm still a newbie at starlark, so I'm not sure if this is something on my end, or maybe something where this doesn't yet handle WSL correctly?
@dtolnay
Copy link
Owner

dtolnay commented May 5, 2025

You need a newer version of buck that includes facebook/buck2@d56d60b, or you need to turn off prelude = bundled in .buckconfig and add a prelude submodule checked out to a newer commit.

@steveklabnik
Copy link
Author

Great, thanks! After using the latest nightly buck2, and doing some heroic fighting to get some sort of modern clang going on ubuntu:

❯ time buck2 run stage2:rustc -- --version --verbose
File changed: rust//llvm-snapshot.gpg.key
File changed: rust//llvm.sh
File changed: rust//.jj/working_copy/working_copy.lock
169 additional file change events
Build ID: 37ea4eca-9c9e-4397-bfa0-97f52c08b382
Network: Up: 0B  Down: 96MiB
Loading targets.   Remaining     0/1                                                         9 dirs read, 780 targets declared
Analyzing targets. Remaining     0/1888
Executing actions. Remaining     0/6523                                                      35:11.8s exec time total
Command: run.      Finished 1290 local
Time elapsed: 4:27.8s
BUILD SUCCEEDED - starting your binary
rustc 1.86.0 (buckified stage2)
binary: rustc
commit-hash: unknown
commit-date: unknown
host: x86_64-unknown-linux-gnu
release: 1.86.0
LLVM version: 19.1.7

real    4m27.822s
user    0m1.423s
sys     0m1.126s

I thought this was pretty fast compared to upstream, where I remember it taking > 20 minutes to compile, but it's been a long time since I did so. Today:

❯ time ./x build
Building bootstrap
    Finished `dev` profile [unoptimized] target(s) in 0.02s
Updating submodule library/backtrace
Submodule 'library/backtrace' (https://github.com/rust-lang/backtrace-rs.git) registered for path 'library/backtrace'
Cloning into '/home/steveklabnik/src/rust/library/backtrace'...
remote: Enumerating objects: 152, done.
remote: Counting objects: 100% (152/152), done.
remote: Compressing objects: 100% (117/117), done.
remote: Total 152 (delta 6), reused 99 (delta 4), pack-reused 0 (from 0)
Receiving objects: 100% (152/152), 122.21 KiB | 2.84 MiB/s, done.
Resolving deltas: 100% (6/6), done.
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
remote: Enumerating objects: 21, done.
remote: Counting objects: 100% (21/21), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 11 (delta 6), reused 5 (delta 0), pack-reused 0 (from 0)
Unpacking objects: 100% (11/11), 1.83 KiB | 937.00 KiB/s, done.
From https://github.com/rust-lang/backtrace-rs
 * branch            9d2c34e7e63afe1e71c333b247065e3b7ba4d883 -> FETCH_HEAD
Submodule path 'library/backtrace': checked out '9d2c34e7e63afe1e71c333b247065e3b7ba4d883'
Updating submodule library/stdarch
Submodule 'library/stdarch' (https://github.com/rust-lang/stdarch.git) registered for path 'library/stdarch'
Cloning into '/home/steveklabnik/src/rust/library/stdarch'...
remote: Enumerating objects: 362, done.
remote: Counting objects: 100% (362/362), done.
remote: Compressing objects: 100% (309/309), done.
remote: Total 362 (delta 57), reused 181 (delta 22), pack-reused 0 (from 0)
Receiving objects: 100% (362/362), 1.63 MiB | 4.66 MiB/s, done.
Resolving deltas: 100% (57/57), done.
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
remote: Enumerating objects: 87, done.
remote: Counting objects: 100% (87/87), done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 44 (delta 29), reused 14 (delta 1), pack-reused 0 (from 0)
Unpacking objects: 100% (44/44), 65.47 KiB | 1.39 MiB/s, done.
From https://github.com/rust-lang/stdarch
 * branch            f1c1839c0deb985a9f98cbd6b38a6d43f2df6157 -> FETCH_HEAD
Submodule path 'library/stdarch': checked out 'f1c1839c0deb985a9f98cbd6b38a6d43f2df6157'
downloading https://ci-artifacts.rust-lang.org/rustc-builds/d6a325d93a84077580c677f89affbe45682606c3/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
########################################################################################################################################################### 100.0%
extracting /home/steveklabnik/src/rust/build/cache/llvm-d6a325d93a84077580c677f89affbe45682606c3-false/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz to /home/steveklabnik/src/rust/build/x86_64-unknown-linux-gnu/ci-llvm
Building stage0 library artifacts (x86_64-unknown-linux-gnu)
    Updating crates.io index
  Downloaded rustc-literal-escaper v0.0.2
  Downloaded hashbrown v0.15.3
  Downloaded miniz_oxide v0.8.8
  Downloaded compiler_builtins v0.1.157
  Downloaded libc v0.2.172
  Downloaded 5 crates (1.2MiB) in 0.21s
   Compiling compiler_builtins v0.1.157
   Compiling core v0.0.0 (/home/steveklabnik/src/rust/library/core)
   Compiling libc v0.2.172
   Compiling object v0.36.7
   Compiling std v0.0.0 (/home/steveklabnik/src/rust/library/std)
   Compiling rustc-std-workspace-core v1.99.0 (/home/steveklabnik/src/rust/library/rustc-std-workspace-core)
   Compiling alloc v0.0.0 (/home/steveklabnik/src/rust/library/alloc)
   Compiling cfg-if v1.0.0
   Compiling adler2 v2.0.0
   Compiling memchr v2.7.4
   Compiling rustc-demangle v0.1.24
   Compiling unwind v0.0.0 (/home/steveklabnik/src/rust/library/unwind)
   Compiling rustc-std-workspace-alloc v1.99.0 (/home/steveklabnik/src/rust/library/rustc-std-workspace-alloc)
   Compiling panic_abort v0.0.0 (/home/steveklabnik/src/rust/library/panic_abort)
   Compiling panic_unwind v0.0.0 (/home/steveklabnik/src/rust/library/panic_unwind)
   Compiling gimli v0.31.1
   Compiling hashbrown v0.15.3
   Compiling std_detect v0.1.5 (/home/steveklabnik/src/rust/library/stdarch/crates/std_detect)
   Compiling miniz_oxide v0.8.8
   Compiling addr2line v0.24.2
   Compiling rustc-std-workspace-std v1.99.0 (/home/steveklabnik/src/rust/library/rustc-std-workspace-std)
   Compiling unicode-width v0.1.14
   Compiling rustc-literal-escaper v0.0.2
   Compiling proc_macro v0.0.0 (/home/steveklabnik/src/rust/library/proc_macro)
   Compiling getopts v0.2.21
   Compiling test v0.0.0 (/home/steveklabnik/src/rust/library/test)
   Compiling sysroot v0.0.0 (/home/steveklabnik/src/rust/library/sysroot)
    Finished `release` profile [optimized + debuginfo] target(s) in 16.09s
Building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
    Updating crates.io index
  Downloaded derive_setters v0.1.6
  Downloaded darling_macro v0.20.11
  Downloaded punycode v0.4.1
  Downloaded leb128 v0.2.5
  Downloaded zerofrom v0.1.6
  Downloaded ctrlc v3.4.6
  Downloaded unic-langid-macros-impl v0.9.5
  Downloaded foldhash v0.1.5
  Downloaded datafrog v2.0.1
  Downloaded rustc-stable-hash v0.1.2
  Downloaded rand_xoshiro v0.7.0
  Downloaded zerofrom-derive v0.1.6
  Downloaded unic-langid v0.9.5
  Downloaded psm v0.1.26
  Downloaded tracing-tree v0.3.1
  Downloaded tinyvec v1.9.0
  Downloaded unicode-script v0.5.7
  Downloaded wasm-encoder v0.219.2
  Downloaded unicode-security v0.1.2
  Downloaded unic-langid-impl v0.9.5
  Downloaded tracing v0.1.37
  Downloaded ruzstd v0.7.3
  Downloaded rand v0.9.1
  Downloaded cc v1.2.16
  Downloaded pulldown-cmark v0.11.3
  Downloaded rustc_apfloat v0.2.2+llvm-462a31f5a5ab
  Downloaded tracing-subscriber v0.3.18
  Downloaded darling_core v0.20.11
  Downloaded wasmparser v0.219.2
  Downloaded zerocopy v0.8.25
  Downloaded wasmparser v0.222.1
  Downloaded blake3 v1.8.2
  Downloaded tracing-core v0.1.30
  Downloaded regex-automata v0.2.0
  Downloaded syn v2.0.101
  Downloaded bstr v1.12.0
  Downloaded serde_derive v1.0.219
  Downloaded rustix v1.0.7
  Downloaded serde v1.0.219
  Downloaded ryu v1.0.20
  Downloaded rustc-rayon-core v0.5.1
  Downloaded proc-macro2 v1.0.95
  Downloaded unicode-ident v1.0.18
  Downloaded rustc-hash v2.1.1
  Downloaded measureme v12.0.1
  Downloaded icu_locid_transform_data v1.5.1
  Downloaded ar_archive_writer v0.4.2
  Downloaded smallvec v1.15.0
  Downloaded rustc-hash v1.1.0
  Downloaded perf-event-open-sys v3.0.0
  Downloaded log v0.4.27
  Downloaded getrandom v0.3.2
  Downloaded jiff v0.2.12
  Downloaded fluent-bundle v0.15.3
  Downloaded bitflags v2.9.0
  Downloaded thin-vec v0.2.14
  Downloaded sha2 v0.10.9
  Downloaded nu-ansi-term v0.50.1
  Downloaded fluent-syntax v0.11.1
  Downloaded derive-where v1.4.0
  Downloaded odht v0.3.1
  Downloaded litemap v0.7.5
  Downloaded libloading v0.8.6
  Downloaded jobserver v0.1.33
  Downloaded icu_provider_adapters v1.5.0
  Downloaded getrandom v0.2.16
  Downloaded thorin-dwp v0.9.0
  Downloaded termize v0.1.1
  Downloaded stacker v0.1.21
  Downloaded self_cell v1.2.0
  Downloaded darling v0.20.11
  Downloaded annotate-snippets v0.11.5
  Downloaded unicode-xid v0.2.6
  Downloaded twox-hash v1.6.3
  Downloaded synstructure v0.13.2
  Downloaded intl-memoizer v0.5.2
  Downloaded fluent-langneg v0.13.0
  Downloaded elsa v1.11.2
  Downloaded intl_pluralrules v7.0.2
  Downloaded icu_list_data v1.5.1
  Downloaded gsgdt v0.1.2
  Downloaded rand_core v0.9.3
  Downloaded proc-macro-hack v0.5.20+deprecated
  Downloaded ppv-lite86 v0.2.21
  Downloaded polonius-engine v0.13.0
  Downloaded memmap2 v0.2.3
  Downloaded icu_list v1.5.0
  Downloaded arrayref v0.3.9
  Downloaded unic-langid-macros v0.9.5
  Downloaded type-map v0.5.0
  Downloaded self_cell v0.10.3
  Downloaded rand_chacha v0.9.0
  Downloaded itoa v1.0.15
  Downloaded errno v0.3.11
  Downloaded ena v0.14.3
  Downloaded constant_time_eq v0.3.1
  Downloaded linux-raw-sys v0.9.4
  Downloaded 97 crates (8.0MiB) in 0.37s (largest was `linux-raw-sys` at 2.2MiB)
   Compiling proc-macro2 v1.0.95
   Compiling unicode-ident v1.0.18
   Compiling cfg-if v1.0.0
   Compiling libc v0.2.172
   Compiling stable_deref_trait v1.2.0
   Compiling shlex v1.3.0
   Compiling bitflags v2.9.0
   Compiling memchr v2.7.4
   Compiling smallvec v1.15.0
   Compiling once_cell v1.21.3
   Compiling equivalent v1.0.2
   Compiling autocfg v1.4.0
   Compiling foldhash v0.1.5
   Compiling allocator-api2 v0.2.21
   Compiling getrandom v0.3.2
   Compiling typenum v1.18.0
   Compiling version_check v0.9.5
   Compiling parking_lot_core v0.9.10
   Compiling scopeguard v1.2.0
   Compiling log v0.4.27
   Compiling crossbeam-utils v0.8.21
   Compiling pin-project-lite v0.2.16
   Compiling rustc-hash v1.1.0
   Compiling rustix v1.0.7
   Compiling proc-macro-hack v0.5.20+deprecated
   Compiling linux-raw-sys v0.9.4
   Compiling rustc-stable-hash v0.1.2
   Compiling zerocopy v0.8.25
   Compiling rustc_macros v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_macros)
   Compiling fastrand v2.3.0
   Compiling thin-vec v0.2.14
   Compiling either v1.15.0
   Compiling arrayvec v0.7.6
   Compiling elsa v1.11.2
   Compiling cpufeatures v0.2.17
   Compiling rustc_graphviz v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_graphviz)
   Compiling rustc-hash v2.1.1
   Compiling cc v1.2.16
   Compiling tracing-core v0.1.30
   Compiling arrayref v0.3.9
   Compiling constant_time_eq v0.3.1
   Compiling itoa v1.0.15
   Compiling unicode-width v0.2.0
   Compiling scoped-tls v1.0.1
   Compiling rustc_arena v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_arena)
   Compiling ena v0.14.3
   Compiling thiserror v1.0.69
   Compiling adler2 v2.0.0
   Compiling wasmparser v0.222.1
   Compiling rustc_hashes v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_hashes)
   Compiling serde v1.0.219
   Compiling static_assertions v1.1.0
   Compiling miniz_oxide v0.8.8
   Compiling crc32fast v1.4.2
   Compiling icu_locid_transform_data v1.5.1
   Compiling litemap v0.7.5
   Compiling hashbrown v0.15.3
   Compiling writeable v0.5.5
   Compiling object v0.36.7
   Compiling unicode-xid v0.2.6
   Compiling rustc-literal-escaper v0.0.2
   Compiling unicode-properties v0.1.3
   Compiling serde_json v1.0.140
   Compiling ryu v1.0.20
   Compiling icu_list_data v1.5.1
   Compiling lock_api v0.4.12
   Compiling generic-array v0.14.7
   Compiling itertools v0.12.1
   Compiling type-map v0.5.0
   Compiling self_cell v1.2.0
   Compiling rustc_lexer v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_lexer)
   Compiling regex-automata v0.2.0
   Compiling fnv v1.0.7
   Compiling ident_case v1.0.1
   Compiling strsim v0.11.1
   Compiling self_cell v0.10.3
   Compiling odht v0.3.1
   Compiling anstyle v1.0.10
   Compiling unicode-width v0.1.14
   Compiling rustc_error_codes v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_error_codes)
   Compiling termcolor v1.4.1
   Compiling rustc_apfloat v0.2.2+llvm-462a31f5a5ab
   Compiling datafrog v2.0.1
   Compiling rustc_parse_format v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_parse_format)
   Compiling tinyvec_macros v0.1.1
   Compiling aho-corasick v1.1.3
   Compiling tinyvec v1.9.0
   Compiling quote v1.0.40
   Compiling annotate-snippets v0.11.5
   Compiling getopts v0.2.21
   Compiling flate2 v1.1.1
   Compiling regex-syntax v0.8.5
   Compiling unicode-script v0.5.7
   Compiling libloading v0.8.6
   Compiling fallible-iterator v0.3.0
   Compiling crossbeam-epoch v0.9.18
   Compiling rustc-demangle v0.1.24
   Compiling syn v2.0.101
   Compiling polonius-engine v0.13.0
   Compiling punycode v0.4.1
   Compiling regex-syntax v0.6.29
   Compiling indexmap v2.9.0
   Compiling pulldown-cmark v0.11.3
   Compiling overload v0.1.1
   Compiling crossbeam-deque v0.8.6
   Compiling lazy_static v1.5.0
   Compiling cfg_aliases v0.2.1
   Compiling leb128 v0.2.5
   Compiling nix v0.29.0
   Compiling sharded-slab v0.1.7
   Compiling nu-ansi-term v0.46.0
   Compiling thread_local v1.1.8
   Compiling wasm-encoder v0.219.2
   Compiling rustc-rayon-core v0.5.1
   Compiling unicase v2.8.1
   Compiling pathdiff v0.2.3
   Compiling pulldown-cmark-escape v0.11.0
   Compiling tracing-log v0.2.0
   Compiling nu-ansi-term v0.50.1
   Compiling jiff v0.2.12
   Compiling rustc-main v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc)
   Compiling crypto-common v0.1.6
   Compiling block-buffer v0.10.4
   Compiling unicode-normalization v0.1.24
   Compiling psm v0.1.26
   Compiling stacker v0.1.21
   Compiling blake3 v1.8.2
   Compiling digest v0.10.7
   Compiling rustc_llvm v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_llvm)
   Compiling rustc_serialize v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_serialize)
   Compiling perf-event-open-sys v3.0.0
   Compiling memmap2 v0.2.3
   Compiling jobserver v0.1.33
   Compiling sha2 v0.10.9
   Compiling sha1 v0.10.6
   Compiling md-5 v0.10.6
   Compiling parking_lot v0.12.3
   Compiling tempfile v3.19.1
   Compiling getrandom v0.2.16
   Compiling rand_core v0.9.3
   Compiling termize v0.1.1
   Compiling unicode-security v0.1.2
   Compiling gimli v0.31.1
   Compiling rand_core v0.6.4
   Compiling ppv-lite86 v0.2.21
   Compiling measureme v12.0.1
   Compiling rand_xoshiro v0.7.0
   Compiling rustc_fs_util v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_fs_util)
   Compiling rand_chacha v0.3.1
   Compiling rand_chacha v0.9.0
   Compiling regex-automata v0.4.9
   Compiling regex-automata v0.1.10
   Compiling rand v0.9.1
   Compiling rand v0.8.5
   Compiling ctrlc v3.4.6
   Compiling twox-hash v1.6.3
   Compiling matchers v0.1.0
   Compiling ruzstd v0.7.3
   Compiling synstructure v0.13.2
   Compiling darling_core v0.20.11
   Compiling displaydoc v0.2.5
   Compiling tracing-attributes v0.1.28
   Compiling zerofrom-derive v0.1.6
   Compiling yoke-derive v0.7.5
   Compiling zerovec-derive v0.10.3
   Compiling rustc_index_macros v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_index_macros)
   Compiling derive-where v1.4.0
   Compiling thiserror-impl v1.0.69
   Compiling serde_derive v1.0.219
   Compiling icu_provider_macros v1.5.0
   Compiling rustc_type_ir_macros v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_type_ir_macros)
   Compiling tinystr v0.7.6
   Compiling regex v1.11.1
   Compiling bstr v1.12.0
   Compiling unic-langid-impl v0.9.5
   Compiling darling_macro v0.20.11
   Compiling unic-langid-macros-impl v0.9.5
   Compiling zerofrom v0.1.6
   Compiling yoke v0.7.5
   Compiling tracing v0.1.37
   Compiling fluent-syntax v0.11.1
   Compiling zerovec v0.10.4
   Compiling tracing-subscriber v0.3.18
   Compiling darling v0.20.11
   Compiling derive_setters v0.1.6
   Compiling unic-langid-macros v0.9.5
   Compiling unic-langid v0.9.5
   Compiling fluent-langneg v0.13.0
   Compiling intl_pluralrules v7.0.2
   Compiling intl-memoizer v0.5.2
   Compiling fluent-bundle v0.15.3
   Compiling icu_locid v1.5.0
   Compiling rustc_fluent_macro v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_fluent_macro)
   Compiling rustc_index v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_index)
   Compiling tracing-tree v0.3.1
   Compiling icu_provider v1.5.0
   Compiling rustc_data_structures v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_data_structures)
   Compiling rustc_log v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_log)
   Compiling icu_locid_transform v1.5.0
   Compiling ar_archive_writer v0.4.2
   Compiling thorin-dwp v0.9.0
   Compiling icu_list v1.5.0
   Compiling icu_provider_adapters v1.5.0
   Compiling rustc_baked_icu_data v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_baked_icu_data)
   Compiling rustc_span v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_span)
   Compiling rustc_ast_ir v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_ast_ir)
   Compiling gsgdt v0.1.2
   Compiling rustc_abi v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_abi)
   Compiling rustc_ast v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_ast)
   Compiling rustc_type_ir v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_type_ir)
   Compiling rustc_error_messages v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_error_messages)
   Compiling rustc_feature v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_feature)
   Compiling rustc_target v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_target)
   Compiling rustc_ast_pretty v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_ast_pretty)
   Compiling rustc_next_trait_solver v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_next_trait_solver)
   Compiling rustc_attr_data_structures v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_attr_data_structures)
   Compiling rustc_hir v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_hir)
   Compiling rustc_lint_defs v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_lint_defs)
   Compiling rustc_hir_pretty v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_hir_pretty)
   Compiling rustc_errors v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_errors)
   Compiling rustc_session v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_session)
   Compiling rustc_query_system v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_query_system)
   Compiling rustc_attr_parsing v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_attr_parsing)
   Compiling rustc_parse v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_parse)
   Compiling rustc_middle v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_middle)
   Compiling rustc_ast_passes v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_ast_passes)
   Compiling rustc_expand v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_expand)
   Compiling rustc_builtin_macros v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_builtin_macros)
   Compiling rustc_infer v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_infer)
   Compiling rustc_transmute v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_transmute)
   Compiling rustc_mir_dataflow v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_mir_dataflow)
   Compiling rustc_symbol_mangling v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_symbol_mangling)
   Compiling rustc_metadata v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_metadata)
   Compiling rustc_pattern_analysis v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_pattern_analysis)
   Compiling rustc_incremental v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_incremental)
   Compiling rustc_ast_lowering v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_ast_lowering)
   Compiling rustc_query_impl v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_query_impl)
   Compiling rustc_smir v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_smir)
   Compiling rustc_monomorphize v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_monomorphize)
   Compiling stable_mir v0.1.0-preview (/home/steveklabnik/src/rust/compiler/stable_mir)
   Compiling rustc_trait_selection v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_trait_selection)
   Compiling rustc_resolve v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_resolve)
   Compiling rustc_lint v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_lint)
   Compiling rustc_ty_utils v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_ty_utils)
   Compiling rustc_const_eval v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_const_eval)
   Compiling rustc_sanitizers v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_sanitizers)
   Compiling rustc_traits v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_traits)
   Compiling rustc_hir_analysis v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_hir_analysis)
   Compiling rustc_codegen_ssa v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_codegen_ssa)
   Compiling rustc_borrowck v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_borrowck)
   Compiling rustc_privacy v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_privacy)
   Compiling rustc_passes v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_passes)
   Compiling rustc_codegen_llvm v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_codegen_llvm)
   Compiling rustc_mir_build v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_mir_build)
   Compiling rustc_hir_typeck v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_hir_typeck)
   Compiling rustc_mir_transform v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_mir_transform)
   Compiling rustc_interface v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_interface)
   Compiling rustc_driver_impl v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_driver_impl)
   Compiling rustc_driver v0.0.0 (/home/steveklabnik/src/rust/compiler/rustc_driver)
    Finished `release` profile [optimized + debuginfo] target(s) in 1m 47s
Creating a sysroot for stage1 compiler (use `rustup toolchain link 'name' build/host/stage1`)
Building stage0 tool lld-wrapper (x86_64-unknown-linux-gnu)
   Compiling lld-wrapper v0.1.0 (/home/steveklabnik/src/rust/src/tools/lld-wrapper)
    Finished `release` profile [optimized + debuginfo] target(s) in 0.27s
Building stage1 library artifacts {alloc, core, panic_abort, panic_unwind, proc_macro, std, sysroot, test, unwind} (x86_64-unknown-linux-gnu)
   Compiling compiler_builtins v0.1.157
   Compiling core v0.0.0 (/home/steveklabnik/src/rust/library/core)
   Compiling libc v0.2.172
   Compiling object v0.36.7
   Compiling std v0.0.0 (/home/steveklabnik/src/rust/library/std)
   Compiling rustc-std-workspace-core v1.99.0 (/home/steveklabnik/src/rust/library/rustc-std-workspace-core)
   Compiling alloc v0.0.0 (/home/steveklabnik/src/rust/library/alloc)
   Compiling cfg-if v1.0.0
   Compiling memchr v2.7.4
   Compiling adler2 v2.0.0
   Compiling rustc-demangle v0.1.24
   Compiling unwind v0.0.0 (/home/steveklabnik/src/rust/library/unwind)
   Compiling rustc-std-workspace-alloc v1.99.0 (/home/steveklabnik/src/rust/library/rustc-std-workspace-alloc)
   Compiling panic_abort v0.0.0 (/home/steveklabnik/src/rust/library/panic_abort)
   Compiling panic_unwind v0.0.0 (/home/steveklabnik/src/rust/library/panic_unwind)
   Compiling gimli v0.31.1
   Compiling miniz_oxide v0.8.8
   Compiling hashbrown v0.15.3
   Compiling std_detect v0.1.5 (/home/steveklabnik/src/rust/library/stdarch/crates/std_detect)
   Compiling addr2line v0.24.2
   Compiling rustc-std-workspace-std v1.99.0 (/home/steveklabnik/src/rust/library/rustc-std-workspace-std)
   Compiling unicode-width v0.1.14
   Compiling rustc-literal-escaper v0.0.2
   Compiling proc_macro v0.0.0 (/home/steveklabnik/src/rust/library/proc_macro)
   Compiling getopts v0.2.21
   Compiling test v0.0.0 (/home/steveklabnik/src/rust/library/test)
   Compiling sysroot v0.0.0 (/home/steveklabnik/src/rust/library/sysroot)
    Finished `release` profile [optimized + debuginfo] target(s) in 42.22s
Building tool rustdoc_tool_binary (stage0 -> stage1, x86_64-unknown-linux-gnu)
  Downloaded threadpool v1.8.1
  Downloaded basic-toml v0.1.10
  Downloaded winnow v0.7.9
  Downloaded minifier v0.3.5
  Downloaded pulldown-cmark v0.9.6
  Downloaded askama_parser v0.13.0
  Downloaded askama_derive v0.13.1
  Downloaded askama v0.13.1
  Downloaded 8 crates (576.4KiB) in 0.14s
   Compiling proc-macro2 v1.0.95
   Compiling unicode-ident v1.0.18
   Compiling libc v0.2.172
   Compiling version_check v0.9.5
   Compiling typenum v1.18.0
   Compiling cfg-if v1.0.0
   Compiling memchr v2.7.4
   Compiling once_cell v1.21.3
   Compiling autocfg v1.4.0
   Compiling serde v1.0.219
   Compiling parking_lot_core v0.9.10
   Compiling smallvec v1.15.0
   Compiling scopeguard v1.2.0
   Compiling regex-syntax v0.8.5
   Compiling regex-syntax v0.6.29
   Compiling lazy_static v1.5.0
   Compiling rustix v1.0.7
   Compiling winnow v0.7.9
   Compiling overload v0.1.1
   Compiling pin-project-lite v0.2.16
   Compiling cpufeatures v0.2.17
   Compiling bitflags v2.9.0
   Compiling getrandom v0.3.2
   Compiling log v0.4.27
   Compiling pulldown-cmark v0.9.6
   Compiling rustc-hash v2.1.1
   Compiling itoa v1.0.15
   Compiling serde_json v1.0.140
   Compiling minifier v0.3.5
   Compiling linux-raw-sys v0.9.4
   Compiling fastrand v2.3.0
   Compiling equivalent v1.0.2
   Compiling sharded-slab v0.1.7
   Compiling hashbrown v0.15.3
   Compiling unicase v2.8.1
   Compiling nu-ansi-term v0.46.0
   Compiling ryu v1.0.20
   Compiling nu-ansi-term v0.50.1
   Compiling either v1.15.0
   Compiling tracing-core v0.1.30
   Compiling thread_local v1.1.8
   Compiling generic-array v0.14.7
   Compiling unicode-segmentation v1.12.0
   Compiling pulldown-cmark-escape v0.11.0
   Compiling base64 v0.21.7
   Compiling arrayvec v0.7.6
   Compiling itertools v0.12.1
   Compiling lock_api v0.4.12
   Compiling aho-corasick v1.1.3
   Compiling tracing-log v0.2.0
   Compiling quote v1.0.40
   Compiling indexmap v2.9.0
   Compiling syn v2.0.101
   Compiling num_cpus v1.16.0
   Compiling parking_lot v0.12.3
   Compiling threadpool v1.8.1
   Compiling block-buffer v0.10.4
   Compiling crypto-common v0.1.6
   Compiling digest v0.10.7
   Compiling sha2 v0.10.9
   Compiling regex-automata v0.1.10
   Compiling rustdoc v0.0.0 (/home/steveklabnik/src/rust/src/librustdoc)
   Compiling regex-automata v0.4.9
   Compiling tempfile v3.19.1
   Compiling matchers v0.1.0
   Compiling basic-toml v0.1.10
   Compiling serde_derive v1.0.219
   Compiling tracing-attributes v0.1.28
   Compiling regex v1.11.1
   Compiling tracing v0.1.37
   Compiling tracing-subscriber v0.3.18
   Compiling askama_parser v0.13.0
   Compiling tracing-tree v0.3.1
   Compiling askama_derive v0.13.1
   Compiling rustdoc-json-types v0.1.0 (/home/steveklabnik/src/rust/src/rustdoc-json-types)
   Compiling askama v0.13.1
   Compiling rustdoc-tool v0.0.0 (/home/steveklabnik/src/rust/src/tools/rustdoc)
    Finished `release` profile [optimized + debuginfo] target(s) in 34.09s
Build completed successfully in 0:03:33

real    3m33.689s
user    36m25.528s
sys     1m49.120s

Not that I'm guessing "being faster" is a main goal of this project, just figured I'd toss in a comparison while I'm here.

@dtolnay
Copy link
Owner

dtolnay commented May 5, 2025

Your x.py only built stage1 not stage2.

@dtolnay
Copy link
Owner

dtolnay commented May 6, 2025

From my machine, here is a comparison of stage2 rustc build time, with download time split out because Cargo keeps crate downloads in a separate location that is not cleaned by x.py:

  • buck2-rustc-bootstrap: 6m25s for buck2 clean && time buck2 bxl scripts/download.bxl:main && time buck2 build stage2:rustc
  • rust-lang/rust: 8m02s for ./x.py clean && time ./x.py build compiler --stage 2

It's also interesting if you are building not just rustc but other tools, since Buck will happily build an arbitrary dependency graph concurrently while x.py is limited to building each different tool serially.

  • 7m11s for time buck2 build stage2:rustc stage2:rustdoc stage2:clippy-driver
    • This is +46s over just building rustc because clippy is the long pole and takes exactly that much longer to build than rustc_driver (which it depends on).
  • 10m35s for time ./x.py build compiler src/tools/rustdoc src/tools/clippy --stage 2
    • This is +2m33s minutes longer than just building rustc, because rustdoc takes +1m09s and clippy takes +1m24s, and all three tools build serially (and also rebuild many of the same dependencies).

Some other sources of the Buck build being faster:

  • x.py builds multiple copies of many crates — see Allow linking rustc and rustdoc against the same single tracing crate rust-lang/rust#140627. tracing and its dependency graph are built separately for the rustc sysroot and for rustdoc's dependency on tracing. Whereas the Buck build only uses one build of tracing that can be used by any tool.

  • In x.py, C++ dependencies (like the llvm-wrapper built by rustc_llvm's build.rs) build pretty late in the build process, after a stage1 standard library is finished compiling, after rustc_llvm's build-dependencies and build script are finished compiling. In Buck the llvm-wrapper build is one of the first things to build because it does not depend on any Rust code or anything else other than the extraction of downloaded LLVM headers.

@steveklabnik
Copy link
Author

Ah, oops. Killing build and running with --stage 2 takes

Build completed successfully in 0:07:50

real    7m50.850s
user    68m58.023s
sys     2m46.065s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@steveklabnik @dtolnay