Failed to run on Mac mini M4 with Metal

[ShiSheng233]

Hi there,

I built whisper.cpp by following the Quick start guide on my Mac mini M4 and tested it with the tiny.en model.
The building process was successful, but when executing the whisper-cli program, it throws thousands of lines of errors related to ggml_metal_init and kernel.

I've attached the entire log of whisper-cli for this issue. Please check it out.

run.stderr.log
run.stdout.log

---

CMake log:

> cmake -B build -DWHISPER_SDL2=ON

CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.


-- The C compiler identification is AppleClang 16.0.0.16000026
-- The CXX compiler identification is AppleClang 16.0.0.16000026
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /opt/homebrew/bin/git (found version "2.48.1")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: arm64
-- Including CPU backend
-- Accelerate framework found
-- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES) 
-- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES) 
-- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND) 
CMake Warning at ggml/src/ggml-cpu/CMakeLists.txt:53 (message):
  OpenMP not found
Call Stack (most recent call first):
  ggml/src/CMakeLists.txt:318 (ggml_add_cpu_backend_variant_impl)


-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- ARM -mcpu not found, -mcpu=native will be used
-- Performing Test GGML_MACHINE_SUPPORTS_dotprod
-- Performing Test GGML_MACHINE_SUPPORTS_dotprod - Success
-- Performing Test GGML_MACHINE_SUPPORTS_i8mm
-- Performing Test GGML_MACHINE_SUPPORTS_i8mm - Success
-- Performing Test GGML_MACHINE_SUPPORTS_sve
-- Performing Test GGML_MACHINE_SUPPORTS_sve - Failed
-- ARM feature DOTPROD enabled
-- ARM feature MATMUL_INT8 enabled
-- ARM feature FMA enabled
-- ARM feature FP16_VECTOR_ARITHMETIC enabled
-- Adding CPU backend variant ggml-cpu: -mcpu=native+dotprod+i8mm+nosve 
-- Looking for dgemm_
-- Looking for dgemm_ - found
-- Found BLAS: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk/System/Library/Frameworks/Accelerate.framework
-- BLAS found, Libraries: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk/System/Library/Frameworks/Accelerate.framework
-- BLAS found, Includes: 
-- Including BLAS backend
-- Metal framework found
-- The ASM compiler identification is AppleClang
-- Found assembler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Including METAL backend
-- SDL2_INCLUDE_DIRS = /opt/homebrew/Cellar/sdl2/2.30.12/include;/opt/homebrew/Cellar/sdl2/2.30.12/include/SDL2
-- SDL2_LIBRARIES    = SDL2::SDL2
-- Configuring done (3.3s)
-- Generating done (0.1s)
-- Build files have been written to: /Volumes/SSD 2T/Development/whispers/whisper.cpp/build

> cmake --build build --config Release

[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[  2%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[  3%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[  4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[  6%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[  7%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[  9%] Linking CXX shared library libggml-base.dylib
[  9%] Built target ggml-base
[ 11%] Generate assembly for embedded Metal library
Embedding Metal library
[ 12%] Building C object ggml/src/ggml-metal/CMakeFiles/ggml-metal.dir/ggml-metal.m.o
[ 13%] Building ASM object ggml/src/ggml-metal/CMakeFiles/ggml-metal.dir/__/__/__/autogenerated/ggml-metal-embed.s.o
[ 14%] Linking C shared library libggml-metal.dylib
[ 14%] Built target ggml-metal
[ 16%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[ 17%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[ 18%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o
[ 19%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o
[ 20%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o
[ 22%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o
[ 23%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[ 24%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
[ 25%] Linking CXX shared library libggml-cpu.dylib
[ 25%] Built target ggml-cpu
[ 27%] Building CXX object ggml/src/ggml-blas/CMakeFiles/ggml-blas.dir/ggml-blas.cpp.o
[ 28%] Linking CXX shared library libggml-blas.dylib
[ 28%] Built target ggml-blas
[ 29%] Building CXX object ggml/src/CMakeFiles/ggml.dir/ggml-backend-reg.cpp.o
[ 30%] Linking CXX shared library libggml.dylib
[ 30%] Built target ggml
[ 32%] Building CXX object src/CMakeFiles/whisper.dir/whisper.cpp.o
[ 33%] Linking CXX shared library libwhisper.dylib
[ 33%] Built target whisper
[ 34%] Building CXX object examples/CMakeFiles/common.dir/common.cpp.o
[ 35%] Building CXX object examples/CMakeFiles/common.dir/common-ggml.cpp.o
[ 37%] Building CXX object examples/CMakeFiles/common.dir/grammar-parser.cpp.o
[ 38%] Linking CXX static library libcommon.a
[ 38%] Built target common
[ 39%] Building CXX object examples/CMakeFiles/common-sdl.dir/common-sdl.cpp.o
[ 40%] Linking CXX static library libcommon-sdl.a
[ 40%] Built target common-sdl
[ 41%] Building CXX object examples/cli/CMakeFiles/whisper-cli.dir/cli.cpp.o
[ 43%] Linking CXX executable ../../bin/whisper-cli
[ 43%] Built target whisper-cli
[ 44%] Building CXX object examples/bench/CMakeFiles/whisper-bench.dir/bench.cpp.o
[ 45%] Linking CXX executable ../../bin/whisper-bench
[ 45%] Built target whisper-bench
[ 46%] Building CXX object examples/server/CMakeFiles/whisper-server.dir/server.cpp.o
[ 48%] Linking CXX executable ../../bin/whisper-server
[ 48%] Built target whisper-server
[ 49%] Building CXX object examples/quantize/CMakeFiles/quantize.dir/quantize.cpp.o
[ 50%] Linking CXX executable ../../bin/quantize
[ 50%] Built target quantize
[ 51%] Building CXX object examples/stream/CMakeFiles/whisper-stream.dir/stream.cpp.o
[ 53%] Linking CXX executable ../../bin/whisper-stream
[ 53%] Built target whisper-stream
[ 54%] Building CXX object examples/command/CMakeFiles/whisper-command.dir/command.cpp.o
[ 55%] Linking CXX executable ../../bin/whisper-command
[ 55%] Built target whisper-command
[ 56%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/talk-llama.cpp.o
[ 58%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama.cpp.o
[ 59%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-adapter.cpp.o
[ 60%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-arch.cpp.o
[ 61%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-batch.cpp.o
[ 62%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-chat.cpp.o
[ 64%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-context.cpp.o
[ 65%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-cparams.cpp.o
[ 66%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-grammar.cpp.o
[ 67%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-hparams.cpp.o
[ 69%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-impl.cpp.o
[ 70%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-kv-cache.cpp.o
[ 71%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-mmap.cpp.o
[ 72%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-model-loader.cpp.o
[ 74%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-model.cpp.o
[ 75%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-quant.cpp.o
[ 76%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-sampling.cpp.o
[ 77%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-vocab.cpp.o
[ 79%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/unicode.cpp.o
[ 80%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/unicode-data.cpp.o
[ 81%] Linking CXX executable ../../bin/whisper-talk-llama
[ 81%] Built target whisper-talk-llama
[ 82%] Building CXX object examples/lsp/CMakeFiles/lsp.dir/lsp.cpp.o
[ 83%] Linking CXX executable ../../bin/lsp
[ 83%] Built target lsp
[ 85%] Building CXX object examples/deprecation-warning/CMakeFiles/main.dir/deprecation-warning.cpp.o
[ 86%] Linking CXX executable ../../bin/main
[ 86%] Built target main
[ 87%] Building CXX object examples/deprecation-warning/CMakeFiles/bench.dir/deprecation-warning.cpp.o
[ 88%] Linking CXX executable ../../bin/bench
[ 88%] Built target bench
[ 90%] Building CXX object examples/deprecation-warning/CMakeFiles/stream.dir/deprecation-warning.cpp.o
[ 91%] Linking CXX executable ../../bin/stream
[ 91%] Built target stream
[ 92%] Building CXX object examples/deprecation-warning/CMakeFiles/command.dir/deprecation-warning.cpp.o
[ 93%] Linking CXX executable ../../bin/command
[ 93%] Built target command
[ 95%] Building CXX object examples/wchess/libwchess/CMakeFiles/wchess-core.dir/WChess.cpp.o
[ 96%] Building CXX object examples/wchess/libwchess/CMakeFiles/wchess-core.dir/Chessboard.cpp.o
[ 97%] Linking CXX static library libwchess-core.a
[ 97%] Built target wchess-core
[ 98%] Building CXX object examples/wchess/wchess.cmd/CMakeFiles/wchess.dir/wchess.cmd.cpp.o
[100%] Linking CXX executable ../../../bin/wchess
[100%] Built target wchess

Part of whisper-cli log:

whisper_backend_init_gpu: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M4
ggml_metal_init: picking default device: Apple M4
ggml_metal_init: using embedded metal library
ggml_metal_init: error: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:61:35: error: unknown type name 'block_q4_0'
void dequantize_q4_0(device const block_q4_0 * xb, short il, thread type4x4 & reg) {
                                  ^
<lines of error>

ggml_backend_metal_device_init: error: failed to allocate context
whisper_backend_init_gpu: failed to initialize Metal backend
whisper_backend_init: using BLAS backend

[nabeards]

Seeing the same issue here on an M1 Pro.

[nabeards]

Turned out for me I had the regular base model installed but was trying to use the medium CoreML model I built. They need to match (obviously)!

[ShiSheng233]

I found that building and running on the macOS system drive does not appear kernel errors and works fine, but I hope it can also run on an external drive.

[astrowonk]

I found that building and running on the macOS system drive does not appear kernel errors and works fine, but I hope it can also run on an external drive.

Yes, moving from an external to the boot drive on my Mac solved the issue for me too.

Building on my startup disk, not an external SSD, also fixed an issue I had with compiling with CoreML. The CoreML version would compile but the Metal backend would fail to load at runtime and while the CoreML appeared load and run, it was using CPU only and rather slow.

Both CoreML and Metal are using my GPU now that I have compiled with the source on my startup disk.