Skip to content

Unable to use local clone of rust-gpu #55

@tmvkrpxl0

Description

@tmvkrpxl0

Below is Cargo.toml file of my shader crate:

[package]
name = "shader-path"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[lib]
crate-type = ["dylib"]

[dependencies]
spirv-std = { path = "/home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std" }
beam-types-shared = { workspace = true }

[package.metadata.rust-gpu.build]
capability = ["RuntimeDescriptorArray", "Int8", "InputAttachment", "SampledImageArrayNonUniformIndexing"]
extension = ["SPV_EXT_descriptor_indexing"]
debug = true
spirv-metadata = "Full"

This fails to build with below error:

$ cargo build    
🦀 Cloning `rust-gpu` repo...
[2025-03-13T05:32:41Z ERROR cargo_gpu] couldn't clone `rust-gpu` /home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std to /home/tmvkrpxl0/.cache/rust-gpu/rust-gpu-repo/_home_tmvkrpxl0_CLionProjects_rust-gpu_crates_spirv-std+v0_9_0
    fatal: repository '/home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std' does not exist
    
Error: couldn't clone `rust-gpu` /home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std to /home/tmvkrpxl0/.cache/rust-gpu/rust-gpu-repo/_home_tmvkrpxl0_CLionProjects_rust-gpu_crates_spirv-std+v0_9_0
fatal: repository '/home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std' does not exist

The path '/home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std' indeed exist. It's just not root of git repository. I did use git to clone that repo but the .git directory is located at '/home/tmvkrpxl0/CLionProjects/rust-gpu/.git'

Activity

tombh

tombh commented on Mar 14, 2025

@tombh
Collaborator

Oh! Sorry about that, we did actually at least write some code to support paths, but we've never tested it, so it's fallen by the wayside. We'll fix it soon!

Firestar99

Firestar99 commented on Mar 15, 2025

@Firestar99
Member

Apparently cargo gpu build has this parameter, maybe it also accepts file paths? That way you can point the spirv-std dependency to the /crates/spirv-std subfolder and make cargo gpu point to the repository root.

        --spirv-builder-source <SPIRV_BUILDER_SOURCE>
            Source of `spirv-builder` dependency Eg: "https://github.com/Rust-GPU/rust-gpu"
tmvkrpxl0

tmvkrpxl0 commented on Mar 17, 2025

@tmvkrpxl0
ContributorAuthor

Apparently cargo gpu build has this parameter, maybe it also accepts file paths? That way you can point the spirv-std dependency to the /crates/spirv-std subfolder and make cargo gpu point to the repository root.

        --spirv-builder-source <SPIRV_BUILDER_SOURCE>
            Source of `spirv-builder` dependency Eg: "https://github.com/Rust-GPU/rust-gpu"

That has same error, it tries to clone /home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std as if it were git repository

schell

schell commented on Mar 17, 2025

@schell
Collaborator

This is most definitely a use case I though we supported, so good catch on this bug @tmvkrpxl0 . We'll have to add this to our CI tests as a requirement of closing this ticket.

added theissue type on Mar 17, 2025
tmvkrpxl0

tmvkrpxl0 commented on Apr 17, 2025

@tmvkrpxl0
ContributorAuthor

I'm working this this issue along with #66
Is there reason why we clone spirv-builder-cli as well, not just spirv-builder?
On surface level, just supplying target json file would be enough

schell

schell commented on Apr 17, 2025

@schell
Collaborator

@tmvkrpxl0 I'm not sure I'm following, can you point me the code you're asking about?

The only instance where cargo-gpu is git cloneing (that I can find ATM) is here, in cargo_gpu::spirv_source::SpirvSource::checkout, but that's of the rust-gpu repo, not spirv-builder-cli?

Or maybe you're talking about .clone()?

tmvkrpxl0

tmvkrpxl0 commented on Apr 25, 2025

@tmvkrpxl0
ContributorAuthor

line 209 in crates/cargo-gpu/src/install.rs
where it copies spirv-builder-cli from cargo-gpu itself into cache and compiles project-specific one. Why is it necessary?

schell

schell commented on Apr 27, 2025

@schell
Collaborator

Ah, ok, I think there may just be a misunderstanding here. The binding name is checkout, but it's not being cloned with git.

So, onward - if the built spirv-builder-cli binary does not exist, we copy the files from SPIRV_BUILDER_FILES into the cache dir and then build it, using the checkout of spirv-builder.

The reason we do this is because spirv-builder is pinned to a specific spirv-std and subsequently rustc. This means that any program that includes spirv-builder must also be pinned to a specific rustc. This is a bit of a bummer because we want to be able to develop CPU code that is not pinned to an older rustc, and we also would like cargo-gpu not to be pinned to rustc so we need to provide a "compilation gap" between spirv-builder-cli and cargo-gpu. We also don't want people's shader crates to depend on spirv-builder...

We have talked about merging spirv-builder-cli with spirv-builder but haven't taken on that work yet.

Another complication is that folks use different spirv-std versions in their projects, so we have to be able to support building spirv-builder-cli for each version they may use, which can come from a bunch of different places (local file system being one of them).

Arguably these complications are why cargo-gpu exists.

Firestar99

Firestar99 commented on Apr 27, 2025

@Firestar99
Member

We have talked about merging spirv-builder-cli with spirv-builder but haven't taken on that work yet.

I started working on that, as in fact spirv-builder does not care about the rustc version at all, it just depends on rustc_codegen_spirv which does need a specific rustc version. So to simplify things, I'm making the rustc_codegen_spirv dependency a feature, so that we may use SpirvBuilder from stable, and using cargo gpu to better manage building that rustc_codegen_spirv dylib.

The cli should not change at all, but makes the internal workings a lot simpler, and may significantly simplify the path to build scripts with cargo gpu as a lib.

3 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority

    Type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Unable to use local clone of rust-gpu · Issue #55 · Rust-GPU/cargo-gpu