Mac os native build not working

[glebnaz]

LocalAI version:
v2.4.1

Environment, CPU architecture, OS, and Version:
MBP 14 M1 PRO

Describe the bug
Not working make build and make BUILD_TYPE=metal build

To Reproduce
run in local make BUILD_TYPE=metal build (like in instruction)

Expected behavior
Error

Logs

CGO_LDFLAGS=" -lcblas -framework Accelerate -framework Foundation -framework Metal -framework MetalKit -framework MetalPerformanceShaders" C_INCLUDE_PATH=/Users/glebnaz/Documents/#main/workspace/LocalAI/sources/go-ggml-transformers LIBRARY_PATH=/Users/glebnaz/Documents/#main/workspace/LocalAI/sources/go-ggml-transformers \
	go build -ldflags " -X "github.com/go-skynet/LocalAI/internal.Version=v2.4.1-2-g62a02cd" -X "github.com/go-skynet/LocalAI/internal.Commit=62a02cd1feb7cf8a75bcbe253fb95f204f022c1f"" -tags "" -o backend-assets/grpc/falcon-ggml ./backend/go/llm/falcon-ggml/
backend/go/llm/transformers/dolly.go:11:2: falcon.go: malformed #cgo argument: -I/Users/glebnaz/Documents/#main/workspace/LocalAI/sources/go-ggml-transformers/ggml.cpp/include/
make: *** [backend-assets/grpc/falcon-ggml] Error 1

Additional context

[themeaningofmeaning]

Interesting, I'm getting the same error as well with multiple build attempts.

LocalAI version:
v2.5.1

Environment, CPU architecture, OS, and Version:
MBP 16 M1 PRO

I've tried using Docker to build this after cloning the repo and it shows "unable to locate package conda" even though the build instructions do specify conda as a required dependency. I also ran a variety of make prepare build attempts and it throws an error about golang not existing even though the mentioned gpt4all-bindings/golang does in fact exist and contains the necessary dependencies. Strange.

1. Docker log:

1.254 E: Unable to locate package conda
------
Dockerfile:60
--------------------
  59 |     
  60 | >>> RUN curl https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg --dearmor > conda.gpg && \
  61 | >>>     install -o root -g root -m 644 conda.gpg /usr/share/keyrings/conda-archive-keyring.gpg && \
  62 | >>>     gpg --keyring /usr/share/keyrings/conda-archive-keyring.gpg --no-default-keyring --fingerprint 34161F5BF5EB1D4BFBBB8F0A8AEB4F8B29D82806 && \
  63 | >>>     echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" > /etc/apt/sources.list.d/conda.list && \
  64 | >>>     echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" | tee -a /etc/apt/sources.list.d/conda.list && \
  65 | >>>     apt-get update && \
  66 | >>>     apt-get install -y conda && apt-get clean
  67 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c curl https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg --dearmor > conda.gpg &&     install -o root -g root -m 644 conda.gpg /usr/share/keyrings/conda-archive-keyring.gpg &&     gpg --keyring /usr/share/keyrings/conda-archive-keyring.gpg --no-default-keyring --fingerprint 34161F5BF5EB1D4BFBBB8F0A8AEB4F8B29D82806 &&     echo \"deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main\" > /etc/apt/sources.list.d/conda.list &&     echo \"deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main\" | tee -a /etc/apt/sources.list.d/conda.list &&     apt-get update &&     apt-get install -y conda && apt-get clean" did not complete successfully: exit code: 100

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/zasirx96yr7cszlyhxyn4ddi4

2. make prepare log:

Switched to a new branch 'build'
touch get-sources
go mod edit -replace github.com/nomic-ai/gpt4all/gpt4all-bindings/golang=/Users/meaning/Desktop/uploading d1/LocalAI/sources/gpt4all/gpt4all-bindings/golang
go: open d1/LocalAI/sources/gpt4all/gpt4all-bindings/golang: no such file or directory
make: *** [replace] Error 1

[muellest]

@themeaningofmeaning: Unfortunately I get the same error =(. Any news on this?

LocalAI version:
v.2.3.1
v.2.4.0
v2.5.1

Environment, CPU architecture, OS, and Version:
MBP 14 M1 PRO

Logs

failed to solve: executor failed running [/bin/sh -c curl https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg --dearmor > conda.gpg && install -o root -g root -m 644 conda.gpg /usr/share/keyrings/conda-archive-keyring.gpg && gpg --keyring /usr/share/keyrings/conda-archive-keyring.gpg --no-default-keyring --fingerprint 34161F5BF5EB1D4BFBBB8F0A8AEB4F8B29D82806 && echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" > /etc/apt/sources.list.d/conda.list && echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" | tee -a /etc/apt/sources.list.d/conda.list && apt-get update && apt-get install -y conda]: exit code: 100

[glebnaz]

@themeaningofmeaning i have the same error with docker. Should we wait help from author?

[themeaningofmeaning]

@glebnaz @muellest - I haven't been able to figure out a solution as it appears to be an issue with the release and Apple Silicon. I haven't tried installing conda or Miniconda before doing a Docker build....that would be one thing to test, but I think we need to wait to hear from the author. I've tried every build in the book and it's not working on the M1

[muellest]

Perhaps this is an alternative to using Docker: https://localai.io/basics/build/ (not tested yet).

[gaoyifan]

The following patch allows me to compile on the Apple Silicon platform. However, the error messages I encountered during compilation are different from @glebnaz , so please take them as reference only.

diff --git a/Makefile b/Makefile
index 6afc644..4414eaa 100644
--- a/Makefile
+++ b/Makefile
@@ -112,7 +112,7 @@ ifeq ($(BUILD_TYPE),hipblas)
 endif

 ifeq ($(BUILD_TYPE),metal)
-	CGO_LDFLAGS+=-framework Foundation -framework Metal -framework MetalKit -framework MetalPerformanceShaders
+	CGO_LDFLAGS+=-framework Foundation -framework Metal -framework MetalKit -framework MetalPerformanceShaders -framework CoreML
 	export LLAMA_METAL=1
 	export WHISPER_METAL=1
 endif

[glebnaz]

@gaoyifan For me your patch doesn't help.

go mod edit -replace github.com/nomic-ai/gpt4all/gpt4all-bindings/golang=/Users/glebnaz/Documents/#main/workspace/localai/sources/gpt4all/gpt4all-bindings/golang
go mod edit -replace github.com/go-skynet/go-ggml-transformers.cpp=/Users/glebnaz/Documents/#main/workspace/localai/sources/go-ggml-transformers
go mod edit -replace github.com/donomii/go-rwkv.cpp=/Users/glebnaz/Documents/#main/workspace/localai/sources/go-rwkv
go mod edit -replace github.com/ggerganov/whisper.cpp=/Users/glebnaz/Documents/#main/workspace/localai/sources/whisper.cpp
go mod edit -replace github.com/ggerganov/whisper.cpp/bindings/go=/Users/glebnaz/Documents/#main/workspace/localai/sources/whisper.cpp/bindings/go
go mod edit -replace github.com/go-skynet/go-bert.cpp=/Users/glebnaz/Documents/#main/workspace/localai/sources/go-bert
go mod edit -replace github.com/mudler/go-stable-diffusion=/Users/glebnaz/Documents/#main/workspace/localai/sources/go-stable-diffusion
go mod edit -replace github.com/M0Rf30/go-tiny-dream=/Users/glebnaz/Documents/#main/workspace/localai/sources/go-tiny-dream
go mod edit -replace github.com/mudler/go-piper=/Users/glebnaz/Documents/#main/workspace/localai/sources/go-piper
go mod download
touch prepare-sources
touch prepare
CGO_LDFLAGS=" -lcblas -framework Accelerate -framework Foundation -framework Metal -framework MetalKit -framework MetalPerformanceShaders -framework CoreML" C_INCLUDE_PATH=/Users/glebnaz/Documents/#main/workspace/localai/sources/go-ggml-transformers LIBRARY_PATH=/Users/glebnaz/Documents/#main/workspace/localai/sources/go-ggml-transformers \
	go build -ldflags " -X "github.com/go-skynet/LocalAI/internal.Version=v2.4.1-2-g62a02cd" -X "github.com/go-skynet/LocalAI/internal.Commit=62a02cd1feb7cf8a75bcbe253fb95f204f022c1f"" -tags "" -o backend-assets/grpc/falcon-ggml ./backend/go/llm/falcon-ggml/
backend/go/llm/transformers/dolly.go:11:2: falcon.go: malformed #cgo argument: -I/Users/glebnaz/Documents/#main/workspace/localai/sources/go-ggml-transformers/ggml.cpp/include/
make: *** [backend-assets/grpc/falcon-ggml] Error 1

but thank you for your comment

[muellest]

Anything new on this topic? - Are there any identified workarounds or solutions?

@mudler , could you please confirm whether this can be recognized as a confirmed bug?

[glebnaz]

@mudler do you have time to participate to solution?

[mudler]

I don't have a Mac (yet) to double check, but the CI doesn't show the same symptoms. Actually the falcon-ggml backend is a candidate for deprecation (see: #1126 for more context).

Maybe @dave-gray101 can chime in here, I think you fixed MacOS builds lately right?

However, in any case I think you could workaround that but selectively choose what backend to build during build time, for instance to build only llama.cpp, you can run:

make GRPC_BACKENDS=backend-assets/grpc/llama-cpp build

see also: https://localai.io/basics/build/#build-only-a-single-backend