Skip to content

Fix race condition bug in non-inplace ggml_compute_forward_diag_mask_f32 #1454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 14, 2023
Merged

Fix race condition bug in non-inplace ggml_compute_forward_diag_mask_f32 #1454

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1e6b5bf
fix race condition bug in non-inplace ggml_compute_forward_diag_mask_f32
xaedes May 14, 2023
b81a09a
remove trailing whitespace
xaedes May 14, 2023
279c372
Merge branch 'master' into fix-race-condition-diag-mask
ggerganov May 14, 2023
d03a85a
Update ggml.c
ggerganov May 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 14 additions & 20 deletions ggml.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10501,34 +10501,28 @@ static void ggml_compute_forward_diag_mask_f32(
assert(src1->type == GGML_TYPE_I32);
assert(ggml_nelements(src1) == 2);

const int ith = params->ith;
const int nth = params->nth;

const int n_past = ((int32_t *) src1->data)[0];
const bool inplace = (bool)((int32_t *) src1->data)[1];
assert(n_past >= 0);

if (params->type == GGML_TASK_INIT) {
// TODO: this hack is not good, need a better way to handle this
if (!inplace) {
// use the init task to copy src -> dst
struct ggml_compute_params params_cpy = *params;

params_cpy.ith = 0;
params_cpy.nth = 1;
params_cpy.type = GGML_TASK_COMPUTE;

ggml_compute_forward_dup_same_cont(&params_cpy, src0, dst);
}

return;
if (!inplace && (params->type == GGML_TASK_INIT)) {
// memcpy needs to be synchronized across threads to avoid race conditions.
// => do it in INIT phase
GGML_ASSERT(ggml_nelements(dst) == ggml_nelements(src0));
GGML_ASSERT(ggml_is_contiguous(dst) && ggml_is_contiguous(src0));
memcpy(
((char *) dst->data),
((char *) src0->data),
ggml_nbytes(dst));
}

if (params->type == GGML_TASK_FINALIZE) {
if (params->type == GGML_TASK_INIT || params->type == GGML_TASK_FINALIZE) {
return;
}

const int ith = params->ith;
const int nth = params->nth;

assert(n_past >= 0);

// TODO: handle transposed/permuted matrices

const int n = ggml_nrows(src0);
Expand Down