fix: set backend path environment variables for llama.cpp #6937
+451
−121
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Ensure that the backend executable’s directory is added to the appropriate environment variable (`PATH`, `LD_LIBRARY_PATH`, or `DYLD_LIBRARY_PATH`) before invoking `llama_load` and `get_devices`. This change fixes load failures on Windows, Linux, and macOS where the dynamic loader cannot locate the required libraries without the proper search paths, and cleans up unused imports.
Contributor
Barecheck - Code coverage reportTotal: 29.99%Your code coverage diff: -0.02% ▾ ✅ All code changes are covered |
Move the library‑path configuration logic out of the TypeScript code into the Rust `setup_library_path` helper. The TypeScript files no longer set the `PATH`, `LD_LIBRARY_PATH`, or `DYLD_LIBRARY_PATH` environment variables directly; instead they defer to the Rust side, which now accepts a `Path` and performs platform‑specific normalization (including UNC‑prefix trimming on Windows). This removes duplicated code, keeps environment configuration consistent across the plugin, and simplifies maintenance. The import order in `device.rs` was corrected and small formatting fixes were applied. No functional changes to the public API occur.
Add utilities to detect CUDA installations on Windows and Linux, automatically inject CUDA paths into the process environment, and warn when the llama.cpp binary requires CUDA but the runtime is not found. The library‑path setup has been refactored to prepend new paths and normalise UNC prefixes for Windows. This ensures the backend can load CUDA libraries correctly and provides diagnostic information when CUDA is missing.
This update unifies backend type handling across the llamacpp extension. Previously, the stored backend preference, the version string, and the auto‑update logic used inconsistent identifiers (raw backend names versus their effective mapped forms). The patch: * Maps legacy backend names to their new “effective” type before any comparison or storage. * Stores the full `version/effectiveType` string instead of just the type, ensuring the configuration and localStorage stay in sync. * Updates all logging and warning messages to reference the effective backend type. * Simplifies the update check logic by comparing the effective type and version together, preventing unnecessary migrations. These changes eliminate bugs that occurred when the backend type changed after an update and make the internal state more coherent.
Enhance `_isCudaInstalled` to accept the backend directory and CUDA version, checking both the new and legacy installation paths. If a library is found in the old location, move it to the new `build/bin` directory and create any missing folders. Update `mapOldBackendToNew` formatting and remove duplicated comments. Minor consistency and readability fixes were also applied throughout the backend module.
This update expands the regular expression used to parse llama‑cpp extension archives. The new pattern now supports: - Optional prefixes and the `-main` segment - Version strings that include a hash suffix - An optional `-cudart-llama` part - A wide range of backend detail strings These changes ensure `installBackend` can correctly handle the latest naming conventions (e.g., `k_llama-main-b4314-09c61e1-bin-win-cuda-12.8-x64-avx2.zip`) while preserving backward compatibility with older formats.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe Your Changes
Ensure that the backend executable’s directory is added to the appropriate environment variable (
PATH,LD_LIBRARY_PATH, orDYLD_LIBRARY_PATH) before invokingllama_loadandget_devices.This change fixes load failures on Windows, Linux, and macOS where the dynamic loader cannot locate the required libraries without the proper search paths, and cleans up unused imports.
Fixes Issues
Self Checklist