Cactus is a lightweight, high-performance framework for running AI models on mobile devices, with simple and consistent APIs across Flutter and React-Native. Cactus currently leverages GGML backends to support any GGUF model already compatible with Llama.cpp.

• Text completion and chat completion
• Vision Language Models
• Streaming token generation
• Embedding generation
• Text-to-speech model support (early stages)
• JSON mode with schema validation
• Chat templates with Jinja2 support
• Low memory footprint
• Battery-efficient inference
• Background processing

• Agentic workflows (cross-app interactions etc.)
• Phone tool use (gallery search, read email, DM...)
• Thinking mode (planning, evals...)
• Higher-level APIs (sentiments, OCR, TTS...)

┌─────────────────────────────────────────┐
│                   Apps                  │
└─────────────┬─────────────┬─────────────┘
              │             │              
┌─────────────▼─────────────▼─────────────┐
│   ┌───-───┐   ┌────-────┐   ┌───-───┐   │
│   │ React │   │ Flutter │   │ Native│   │
│   └───────┘   └─────────┘   └───────┘   │
│                 Bindings                │
└─────────────┬─────────────┬─────────────┘
              │             │              
┌─────────────▼─────────────▼─────────────┐
│              Cactus Core C++            │
└─────────────┬─────────────┬─────────────┘
              │             │       
       ┌─────-▼────┐  ┌─────▼─────┐  
       │ Llama.cpp │  │ GGML/GGUF │  
       └───────────┘  └───────────┘  

1. Update pubspec.yaml: Add cactus to your project's dependencies. Ensure you have flutter: sdk: flutter (usually present by default).

   dependencies:
     flutter:
       sdk: flutter
     cactus: ^0.0.3

2. Install dependencies: Execute the following command in your project terminal:

   flutter pub get

1. Install the cactus-react-native package: Using npm:

   npm install cactus-react-native

   Or using yarn:

   yarn add cactus-react-native

2. Install iOS Pods (if not using Expo): For native iOS projects, ensure you link the native dependencies. Navigate to your ios directory and run:

   npx pod-install

Cactus backend is written in C/C++, layered on top of GGML/GGUF to support models in the GGUF format. Developers and contributors in this niche can easily get started with examples for:

N/B: Should have CMake installed, or install with brew install cmake (on macOS) or standard package managers on Linux.

• Language Models:

  1. Navigate to the example directory: cd example/cpp-llm
  2. Make the build script executable (only needs to be done once): chmod +x build.sh
  3. Run the example: ./build.sh (This will download the Qwen 3 model)
  4. Play with models and prompts in example/cpp-llm/main.cpp.

• Vision-Language Models:

  1. Navigate to the example directory: cd example/cpp-vlm
  2. Make the build script executable (only needs to be done once): chmod +x build.sh
  3. Run the example: ./build.sh (This will download the SmolVLM model)
  4. Play with models and prompts in example/cpp-vlm/main.cpp.

• Text-to-Speech:

  1. Navigate to the example directory: cd example/cpp-tts
  2. Make the build script executable (only needs to be done once): chmod +x build.sh
  3. Run the example: ./build.sh (This will download the OuteTTS model)
  4. Play with models and prompts in example/cpp-tts/main.cpp.

We host our docs on Deep Wiki, so you can additionally ask Devin any question about Cactus! It does not index frequently enough to keep up with our update speed though, so we have manually written docs for the APIs

• Deep Wiki
• C++ Docs
• Flutter Docs
• React-Native Docs

We have ready-to-run-and-deploy example apps:

1. Flutter Chat
2. Flutter Notes
3. React Chat
4. React Productivity
5. React Diary
6. C++ Language Model (LLM)
7. C++ Vision-Language Model (VLM)
8. C++ Text-to-Speech (TTS)

We welcome contributions! Here's how you can help:

1. Clone the Repository: For simplicity at this stage, clone the repository to your local machine.
2. Create a Branch: Create a new branch for your contribution.
3. Implement Changes: Make your desired changes or additions.
4. Run Tests (for C/C++ contributors):
   • Ensure all tests pass by running the script: scripts/test-cactus.sh
5. Flutter & React-Native Testing: (Testing procedures for these platforms will be updated soon.)
6. Submit a Pull Request (PR): Once you're ready, submit a PR with your changes!
7. Contribution Ideas Example apps, polishing the examples, features, submitting benchmarks, etc.

We created a demo chat app we use for benchmarking: