distributed load #14477

Unanswered

thunderFireSword asked this question in Q&A

thunderFireSword
Jul 1, 2025

Can llama.cpp load models in a distributed manner across multiple CPU nodes? That is, each node retains a portion of the model's computation graph.

Replies: 1 comment

0cc4m
Jul 1, 2025
Collaborator

https://github.com/ggml-org/llama.cpp/tree/master/tools/rpc

0 replies

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment