Skip to content

Backend Clarification on set_tensor and get_tensor #14519

Answered by slaren
taronaeo asked this question in Q&A
Discussion options

You must be logged in to vote

get_tensor and set_tensor copy data between a user buffer and a backend buffer. If the buffer is stored in the CPU and does not apply any transformations to the tensor data layout, the implementation is simply a memcpy:

static void ggml_backend_cpu_buffer_set_tensor(ggml_backend_buffer_t buffer, struct ggml_tensor * tensor, const void * data, size_t offset, size_t size) {
memcpy((char *)tensor->data + offset, data, size);
GGML_UNUSED(buffer);
}
static void ggml_backend_cpu_buffer_get_tensor(ggml_backend_buffer_t buffer, const struct ggml_tensor * tensor, void * data, size_t offset, s…

Replies: 1 comment 8 replies

Comment options

You must be logged in to vote
8 replies
@slaren
Comment options

@taronaeo
Comment options

@slaren
Comment options

@taronaeo
Comment options

@slaren
Comment options

Answer selected by taronaeo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants