Skip to content

feat: upgrading for feynman #1690

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Jul 4, 2025
Merged

feat: upgrading for feynman #1690

merged 18 commits into from
Jul 4, 2025

Conversation

noel2004
Copy link
Member

@noel2004 noel2004 commented Jul 4, 2025

This PR would trace the upgrades in the progress of feynman integration

Summary by CodeRabbit

  • Chores
    • Updated dependency revisions for several workspace components.
    • Added a new configuration file to override dependencies with custom forks for GPU builds.
    • Removed obsolete configuration and patch files from various directories.
    • Updated .gitignore to exclude additional files.
    • Removed multiple scripts and configuration files related to repository cloning and patch overrides.
    • Updated version tag to v4.5.27.
  • Build
    • Revised Makefile to separate GPU and CPU build targets and streamline the build process.
  • Scripts
    • Enhanced the release download script with version mapping and support for downloading additional configuration files.
  • Docker
    • Adjusted Dockerfile COPY commands to better organize crate directory structure within containers.
    • Simplified Docker workflow by removing SSH setup and repository cloning steps.

Copy link

coderabbitai bot commented Jul 4, 2025

Walkthrough

This update revises dependency versions for several zkVM prover workspace crates, restructures GPU and CPU build flows in the Makefile, introduces a new GPU-specific Cargo patch configuration, removes multiple configuration files and legacy scripts related to GPU repo cloning and patching, and updates the release download script to handle new versioning and download logic for required files.

Changes

Files / Paths Change Summary
Cargo.toml Updated commit hash for three scroll-zkvm-prover dependencies.
crates/gpu_override/.cargo/config.toml Added new Cargo patch config overriding dependencies with custom GPU-related forks and features.
crates/prover-bin/.cargo/config.toml Deleted previous patch config for dependency overrides.
zkvm-prover/.work/.gitignore Added openvm.toml and root-verifier* to ignored files.
zkvm-prover/.work/batch/openvm.toml,
zkvm-prover/.work/bundle/openvm.toml,
zkvm-prover/.work/chunk/openvm.toml
Deleted configuration files specifying FRI, VM, and cryptography parameters.
zkvm-prover/Makefile Refactored build targets, added explicit GPU/CPU build separation, changed build directory for GPU builds.
zkvm-prover/download-release.sh Updated to use version mapping, improved file download logic, and now downloads openvm.toml files as well.
build/dockerfiles/coordinator-api/checkout_all.sh Deleted script that checked out specific commits for GPU-related repositories.
build/dockerfiles/coordinator-api/config.toml Removed local path patch dependency declarations for OpenVM, Stark backend, Plonky3, and GPU crates.
common/version/version.go Updated version tag from "v4.5.26" to "v4.5.27".
build/dockerfiles/coordinator-api.Dockerfile Changed base images to non-CUDA versions, adjusted COPY commands, and removed GPU-related scripts and configs.
.github/workflows/docker.yml Removed SSH setup and repository cloning steps for GPU repos in coordinator-api job.
build/dockerfiles/coordinator-api/clone_*.sh scripts Deleted scripts for cloning plonky3-gpu, openvm-stark-gpu, and openvm-gpu repositories.
build/dockerfiles/coordinator-api/gitconfig Deleted Git config file that rewrote SSH GitHub URLs to HTTPS.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Makefile
    participant crates/gpu_override
    participant crates/prover-bin

    User->>Makefile: make prover
    Makefile->>Makefile: Set ZK_VERSION (CPU version)
    Makefile->>Makefile: Run prover_gpu target
    Makefile->>crates/gpu_override: Build prover with GPU patches
    Note right of Makefile: CPU build uses prover_cpu target (optional)
Loading
sequenceDiagram
    participant User
    participant download-release.sh
    participant GitHub Releases

    User->>download-release.sh: ./download-release.sh [version]
    download-release.sh->>download-release.sh: Map version name to release version
    download-release.sh->>download-release.sh: Create .work/chunk, .work/batch, .work/bundle
    download-release.sh->>GitHub Releases: Download app.vmexe and openvm.toml for each circuit type
Loading

Possibly related PRs

Suggested reviewers

  • colinlyguo

Poem

In the warren, code hops anew,
GPU and CPU now split in two.
Configs swept from their burrows deep,
While Makefile bunnies vigil keep.
With hashes fresh and scripts that gleam,
The prover builds—swift as a dream!
🐇✨


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c8712d1 and 6fa1625.

📒 Files selected for processing (1)
  • .github/workflows/docker.yml (9 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/docker.yml
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@noel2004 noel2004 marked this pull request as ready for review July 4, 2025 06:20
georgehao
georgehao previously approved these changes Jul 4, 2025
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (6)
crates/gpu_override/.cargo/config.toml (2)

2-14: Commented-out patch block clutters the config.

If these overrides are no longer needed, delete them; if they are needed, ship them active and pin to a commit for reproducibility.


20-45: Many crates point to the same commit – consider workspace-level patch & a constant.

Duplicating the hash 18 times is error-prone. Move the override to the workspace root or use a Cargo workspace.dependencies table to reference a single variable.

zkvm-prover/Makefile (1)

35-42: Dead/disabled GPU-selection logic – remove or revive.

The old ifeq (${PLONKY3_GPU_VERSION},) block is now commented. Keeping stale code hampers readability.

zkvm-prover/download-release.sh (1)

23-35: Three almost-identical wget blocks – loop it.

for part in chunk batch bundle; do
  dir=.work/$part
  mkdir -p "$dir"
  for file in app.vmexe openvm.toml; do
    wget -q "https://circuit-release.s3.us-west-2.amazonaws.com/scroll-zkvm/releases/$SCROLL_ZKVM_VERSION/$part/$file" \
         -O "$dir/$file"
  done
done

Cuts duplication and makes future circuit additions trivial.

Cargo.toml (2)

6-7: Workspace still lists crates/prover-bin but Makefile switched to gpu_override.

If prover-bin is obsolete, drop it and add crates/gpu_override to avoid unused-crate bloat and ensure cargo check covers the new code.


19-23: Revision bump LGTM, but pin by tag if possible.

Using raw commit hashes is fine short-term; long-term, tagged releases convey intent and survive force-pushes.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a0ee508 and 0e1d3e4.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (9)
  • Cargo.toml (1 hunks)
  • crates/gpu_override/.cargo/config.toml (1 hunks)
  • crates/prover-bin/.cargo/config.toml (0 hunks)
  • zkvm-prover/.work/.gitignore (1 hunks)
  • zkvm-prover/.work/batch/openvm.toml (0 hunks)
  • zkvm-prover/.work/bundle/openvm.toml (0 hunks)
  • zkvm-prover/.work/chunk/openvm.toml (0 hunks)
  • zkvm-prover/Makefile (2 hunks)
  • zkvm-prover/download-release.sh (1 hunks)
💤 Files with no reviewable changes (4)
  • zkvm-prover/.work/bundle/openvm.toml
  • zkvm-prover/.work/chunk/openvm.toml
  • zkvm-prover/.work/batch/openvm.toml
  • crates/prover-bin/.cargo/config.toml
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: tests
🔇 Additional comments (1)
zkvm-prover/.work/.gitignore (1)

2-2: Good call on ignoring openvm.toml.

Keeps transient config out of source control.

@colinlyguo colinlyguo changed the title Upgrading for feynman feat: upgrading for feynman Jul 4, 2025
colinlyguo
colinlyguo previously approved these changes Jul 4, 2025
@colinlyguo
Copy link
Member

bump version?

@noel2004 noel2004 dismissed stale reviews from colinlyguo and georgehao via 654cd42 July 4, 2025 06:55
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
build/dockerfiles/coordinator-api/config.toml (2)

4-15: Absolute paths tie the build to a specific container layout

All patched crates reference /openvm-gpu/... (leading slash).
This:

  1. Breaks cargo check outside the Docker image;
  2. Makes future container re-organisation painful.

Prefer relative paths (e.g. path = "../../openvm-gpu/…" from the config file location) or anchor them via an env var ({ path = "${OPENVM_GPU_SRC}/crates/…" }) set in the Dockerfile.


58-58: Aliasing p3-monty-31 without features may disable SIMD/GPU paths

The upstream crate exposes optional feature flags (gpu, simd, etc.).
By patching it with no default-features stanza you inherit whatever the crate’s defaults are—which could differ between upstream tags and break determinism.
Explicitly include the expected feature set, e.g.:

-p3-monty-31 = { path = "/plonky3-gpu/monty-31" }
+p3-monty-31 = { path = "/plonky3-gpu/monty-31", default-features = false, features = ["gpu"] }

Also applies to: 87-87

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0e1d3e4 and 654cd42.

📒 Files selected for processing (3)
  • build/dockerfiles/coordinator-api/checkout_all.sh (1 hunks)
  • build/dockerfiles/coordinator-api/config.toml (5 hunks)
  • crates/gpu_override/.cargo/config.toml (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • build/dockerfiles/coordinator-api/checkout_all.sh
🚧 Files skipped from review as they are similar to previous changes (1)
  • crates/gpu_override/.cargo/config.toml
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: tests
🔇 Additional comments (1)
build/dockerfiles/coordinator-api/config.toml (1)

44-46: nightly-features enabled – ensure the toolchain is pinned

The p3-baby-bear patch now opts into "nightly-features".
Compilation will fail on stable Rust. Double-check that:

  1. A rust-toolchain.toml (or rust-toolchain) at workspace root pins an appropriate nightly;
  2. CI images/install scripts install the same nightly.

If this is accidental, drop the feature flag.

Also applies to: 73-75

@georgehao georgehao added the bump-version Bump the version tag for deployment label Jul 4, 2025
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
build/dockerfiles/coordinator-api.Dockerfile (1)

42-47: Slim-down the final image & keep security hygiene
Installing vim and leaving APT caches bloats the runtime image by hundreds of MB. Use --no-install-recommends and clean the cache.

-RUN apt update && apt install vim netcat-openbsd net-tools curl jq -y
+RUN apt-get update \
+ && DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
+      netcat-openbsd net-tools curl jq \
+ && rm -rf /var/lib/apt/lists/*
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5667d0b and 8b62b67.

📒 Files selected for processing (8)
  • .github/workflows/docker.yml (0 hunks)
  • build/dockerfiles/coordinator-api.Dockerfile (2 hunks)
  • build/dockerfiles/coordinator-api/checkout_all.sh (0 hunks)
  • build/dockerfiles/coordinator-api/clone_openvm_gpu.sh (0 hunks)
  • build/dockerfiles/coordinator-api/clone_openvm_stark_gpu.sh (0 hunks)
  • build/dockerfiles/coordinator-api/clone_plonky3_gpu.sh (0 hunks)
  • build/dockerfiles/coordinator-api/config.toml (0 hunks)
  • build/dockerfiles/coordinator-api/gitconfig (0 hunks)
💤 Files with no reviewable changes (7)
  • build/dockerfiles/coordinator-api/gitconfig
  • build/dockerfiles/coordinator-api/clone_openvm_stark_gpu.sh
  • build/dockerfiles/coordinator-api/checkout_all.sh
  • build/dockerfiles/coordinator-api/clone_plonky3_gpu.sh
  • build/dockerfiles/coordinator-api/clone_openvm_gpu.sh
  • .github/workflows/docker.yml
  • build/dockerfiles/coordinator-api/config.toml
🧰 Additional context used
🪛 Hadolint (2.12.0)
build/dockerfiles/coordinator-api.Dockerfile

[error] 3-3: Use absolute WORKDIR

(DL3000)

⏰ Context from checks skipped due to timeout of 90000ms (10)
  • GitHub Check: bridgehistoryapi-db-cli
  • GitHub Check: coordinator-cron
  • GitHub Check: coordinator-api
  • GitHub Check: bridgehistoryapi-api
  • GitHub Check: gas_oracle
  • GitHub Check: bridgehistoryapi-fetcher
  • GitHub Check: rollup_relayer
  • GitHub Check: rollup-db-cli
  • GitHub Check: blob_uploader
  • GitHub Check: tests
🔇 Additional comments (1)
build/dockerfiles/coordinator-api.Dockerfile (1)

42-50: Double-check glibc compatibility after switching to Ubuntu 20.04
libzkp.so is compiled in the builder (Ubuntu 22.04 base image). Running it on older glibc (20.04 ships 2.31) can cause symbol-version errors. Please verify at runtime or align both stages to the same distro level.

@georgehao georgehao merged commit 12e8920 into develop Jul 4, 2025
3 checks passed
@georgehao georgehao deleted the upgrade/feynman branch July 4, 2025 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bump-version Bump the version tag for deployment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants