This application serves as a proxy that implements the Ollama API interface but forwards requests to different LLM providers like Anthropic's Claude and Perplexity AI. This allows IDE plugins that support Ollama to work with these alternative LLM providers.

Looking at you, JetBrains.

• Implements Ollama's API endpoints:
  • /api/chat - For chat completions
  • /api/tags - Lists available models
  • /api/show - Shows model details
  • / - Health check endpoint
• Supports multiple LLM providers:
  • Perplexity AI (Llama models)
  • Anthropic (Claude models)
• Configurable server settings
• Easy provider switching via configuration

1. Clone the repository
2. Install Rust if you haven't already (https://rustup.rs/)
3. Create a Config.toml file in the project root

Create a Config.toml file with the following structure:

# Provider configuration
provider_type = "perplexity"  # or "anthropic"
perplexity_api_key = "your-perplexity-key"
anthropic_api_key = "your-anthropic-key"

# Server configuration
[server]
host = "127.0.0.1"
port = 11434

• llama-3.1-sonar-small-128k-online (8B parameters)
• llama-3.1-sonar-large-128k-online (70B parameters)
• llama-3.1-sonar-huge-128k-online (405B parameters)

• claude-3-5-sonnet-20241022
• claude-3-5-haiku-20241022
• claude-3-opus-20240229

1. Start the server:

   cargo run

2. Configure your IDE's Ollama plugin to use the proxy URL:

   http://localhost:11434  # or your configured host:port
   

Returns "Ollama is running" to indicate the server is up.

Lists all available models for the configured provider.

Handles chat completions. Example request:

{
  "model": "llama2",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "Hello!"
    }
  ],
  "options": {
    "temperature": 0.7,
    "top_p": 0.9
  }
}

Shows details about a specific model. Example request:

{
  "name": "llama2"
}

The application is built with:

• Rust
• Axum web framework
• Tokio async runtime
• Reqwest for HTTP clients

The architecture follows a trait-based approach for provider implementations, making it easy to add new LLM providers.

The application includes comprehensive error handling for:

• API communication errors
• Request parsing errors
• Configuration errors
• Invalid model selections

1. Fork the repository
2. Create a feature branch
3. Commit your changes
4. Push to the branch
5. Create a Pull Request

This project is licensed under the MIT License - see the LICENSE file for details.