Skip to content

Simplify the SIMD code #324

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 24, 2022
Merged

Simplify the SIMD code #324

merged 2 commits into from
Dec 24, 2022

Conversation

ggerganov
Copy link
Member

@ggerganov ggerganov commented Dec 23, 2022

This is an attempt to simplify and unify the SIMD implementation for various architectures.
The idea is similar to #95 (comment)

We define a common subset of C macros which map to specific intrinsics based on the current architecture:

#define GGML_F32_VEC        ...
#define GGML_F32_VEC_ZERO   ...
#define GGML_F32_VEC_SET1   ...
#define GGML_F32_VEC_LOAD   ...
#define GGML_F32_VEC_STORE  ...
#define GGML_F32_VEC_FMA    ...
#define GGML_F32_VEC_ADD    ...
#define GGML_F32_VEC_MUL    ...
#define GGML_F32_VEC_REDUCE ...

#define GGML_F16_VEC        ...
#define GGML_F16_VEC_ZERO   ...
#define GGML_F16_VEC_SET1   ...
#define GGML_F16_VEC_LOAD   ...
#define GGML_F16_VEC_STORE  ...
#define GGML_F16_VEC_FMA    ...
#define GGML_F16_VEC_ADD    ...
#define GGML_F16_VEC_MUL    ...
#define GGML_F16_VEC_REDUCE ...

We then implement the SIMD functions using only these macros.
Refactored functions:

Supported and tested architectures:

  • Arm64
  • ARMv7
  • x86
  • WASM
  • ppc64le

Still not sure if this would be easier to maintain, but I will give it a try

@ggerganov ggerganov force-pushed the simd branch 3 times, most recently from 9e726a3 to bad040f Compare December 23, 2022 16:47
@ggerganov ggerganov marked this pull request as ready for review December 23, 2022 16:52
@ggerganov ggerganov merged commit ac521a5 into master Dec 24, 2022
@ggerganov ggerganov deleted the simd branch December 24, 2022 08:22
rock3125 pushed a commit to rock3125/whisper.cpp that referenced this pull request Feb 21, 2023
* ggml : simplify the SIMD code

* ggml : generic reduce for all register sizes + comments
anandijain pushed a commit to anandijain/whisper.cpp that referenced this pull request Apr 28, 2023
* ggml : simplify the SIMD code

* ggml : generic reduce for all register sizes + comments
jacobwu-b pushed a commit to jacobwu-b/Transcriptify-by-whisper.cpp that referenced this pull request Oct 24, 2023
* ggml : simplify the SIMD code

* ggml : generic reduce for all register sizes + comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant