Skip to content

[perf] Cache nixpkgs resolution #2576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 9, 2025
Merged

[perf] Cache nixpkgs resolution #2576

merged 2 commits into from
Apr 9, 2025

Conversation

mikeland73
Copy link
Contributor

@mikeland73 mikeland73 commented Apr 8, 2025

Summary

This caches resolutions for unlocked nixpkgs (the default) for 90 days on the user's machine. Otherwise, everytime devbox tries to resolve github:NixOS/nixpkgs/nixpkgs-unstable it will download a new version of nixpkgs. This is slow (40+ seconds) and usually nothing changes.

If a user wants to update nixpkgs there are two ways of doing it:

  • devbox update without any arguments will update nixpkgs. This is existing functionality.
  • devbox update nixpkgs [...pkgs] will also update nixpkgs.

I'm not convinced updating nixpkgs when using update without arguments is best, but I didn't want to break past functionality. It adds 40+ seconds to update operations when there is no package to update.

An alternative command syntax considered was devbox update --nixpkgs. The benefit of doing this would be to avoid conflicts in the future.

I did not use stdenv because it already exists and it is a different package.

Bug fixes:

  • Updating a specific package (e.g. devbox update go) no longer updates nixpkgs

Alternative approaches:

  • We could try to search the local nix database to try to find existing nixpkgs, but this seemed more trouble than it was worth.

How was it tested?

  • Created a new project and logged the cache being used.
  • Replaced the cache file data with an old version, created a new project and observed the old version used.
  • Tried several permutations of devbox update with and without packages

Community Contribution License

All community contributions in this pull request are licensed to the project
maintainers under the terms of the
Apache 2 License.

By creating this pull request, I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 License as stated in
the
Community Contribution License.

@mikeland73 mikeland73 requested review from gcurtis and Lagoja April 8, 2025 22:44
@Allsimon
Copy link

Allsimon commented Apr 9, 2025

This issue is slightly related: #2560

Is it possible to pin nixpkgs to a specific version ?

Copy link
Collaborator

@gcurtis gcurtis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we shorten the cache time to something less? Maybe about a week?

There are times where an outdated stdenv breaks packages. This recently happened on macOS where the stdenv was overhauled, leading to weird runtime errors about git or frameworks not being found.

@gcurtis
Copy link
Collaborator

gcurtis commented Apr 9, 2025

This issue is slightly related: #2560

Is it possible to pin nixpkgs to a specific version ?

@Allsimon you can try setting the top-level { "nixpkgs": { "commit": "<hash>" } } in devbox.json or manually editing devbox.lock. I know this is a bit hacky though. Devbox generally uses a search index to map package versions (e.g., [email protected]) to nixpkgs commit hashes, so pinning a specific hash is a little tricky.

@mikeland73 mikeland73 merged commit 55011d2 into main Apr 9, 2025
29 checks passed
@mikeland73 mikeland73 deleted the landau/cache-nixpkgs branch April 9, 2025 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants