Inference Provider SDK for Flutter/Dart #

A comprehensive Flutter/Dart SDK for the Inference Provider API. Build powerful AI agents with RAG (Retrieval-Augmented Generation), tool calling, and MCP (Model Context Protocol) integration.

Features #

✅ Agent Management - Complete CRUD operations for AI agents ✅ Inference API - Run text and vision-based inference ✅ Conversation History - Maintain context across multiple turns ✅ RAG Support - Retrieval-Augmented Generation with vector search ✅ Tool Calling - Integrate REST APIs, JavaScript functions, and MCP servers ✅ Provider Management - Manage AI providers and models ✅ Type Safety - Full Dart type annotations with JSON serialization ✅ Error Handling - Comprehensive exception hierarchy ✅ HTTP Client - Built on package:http with automatic retries

Installation #

Add this to your package's pubspec.yaml file:

dependencies:
  inference_provider_sdk: ^1.0.0

Then run:

flutter pub get

Quick Start #

1. Initialize the Client #

import 'package:inference_provider_sdk/inference_provider_sdk.dart';

final client = InferenceProviderClient(
  apiKey: 'ip_your_api_key',
  apiSecret: 'ips_your_api_secret',
  baseUrl: 'https://your-api-domain.com',
);

2. Create an Agent #

final agent = await client.createAgent(
  CreateAgentRequest(
    name: 'My AI Assistant',
    systemPrompt: 'You are a helpful AI assistant.',
    aiProviderId: 'provider-id',
    aiModelId: 'model-id',
    maxTokens: 1000,
    temperature: 0.7,
  ),
);

print('Created agent: ${agent.id}');

3. Run Inference #

final response = await client.runInference(
  InferenceRequest(
    agentId: agent.id,
    prompt: 'Hello! How can you help me today?',
  ),
);

print('Response: ${response.text}');
print('Session ID: ${response.sessionId}');
print('Tokens used: ${response.usage?.totalTokens}');

4. Continue Conversation #

final followUp = await client.runInference(
  InferenceRequest(
    agentId: agent.id,
    prompt: 'Tell me more about that.',
    sessionId: response.sessionId, // Maintain context
  ),
);

print('Follow-up response: ${followUp.text}');

Usage Examples #

Agent Management #

List All Agents

final agents = await client.listAgents();
for (final agent in agents) {
  print('${agent.name} (${agent.id})');
}

Get Agent Details

final agent = await client.getAgent('agent-id');
print('Agent: ${agent.name}');
print('System Prompt: ${agent.systemPrompt}');

Update Agent

final updated = await client.updateAgent(
  'agent-id',
  UpdateAgentRequest(
    name: 'Updated Name',
    temperature: 0.9,
  ),
);

Delete Agent

await client.deleteAgent('agent-id');

Vision Inference #

final response = await client.runVisionInference(
  agentId: 'agent-id',
  prompt: 'What do you see in this image?',
  imageUrl: 'https://example.com/image.jpg',
);

print('Vision response: ${response.text}');

Conversation History #

final history = await client.getConversationHistory('session-id');
for (final message in history) {
  print('${message.role}: ${message.content}');
}

AI Provider Management #

Create Provider

final provider = await client.createProvider(
  CreateProviderRequest(
    name: 'My OpenAI Provider',
    providerType: 'openai',
    apiKey: 'sk-...',
  ),
);

List Providers

final providers = await client.listProviders();
for (final provider in providers) {
  print('${provider.name} (${provider.providerType})');
}

List Models

final models = await client.listModels('provider-id');
for (final model in models) {
  print('${model.name}: ${model.modelId}');
}

Tool Management #

Create Tool

final tool = await client.createTool(
  CreateToolRequest(
    toolName: 'get_weather',
    displayName: 'Get Weather',
    description: 'Get current weather for a location',
    toolType: 'rest_api',
    config: {
      'url': 'https://api.weather.com/current',
      'method': 'GET',
      'headers': {
        'API-Key': 'your-key',
      },
    },
  ),
);

List Tools

final tools = await client.listTools();
for (final tool in tools) {
  print('${tool.displayName}: ${tool.toolType}');
}

Attach Tool to Agent

await client.attachToolToAgent('agent-id', 'tool-id', 1);

List Agent Tools

final agentTools = await client.listAgentTools('agent-id');
for (final agentTool in agentTools) {
  print('Tool ID: ${agentTool.toolId}, Priority: ${agentTool.priority}');
}

MCP Server Management #

Create MCP Server

final mcpServer = await client.createMCPServer(
  CreateMCPServerRequest(
    name: 'File System MCP',
    description: 'MCP server for file system operations',
    command: 'npx',
    args: ['-y', '@modelcontextprotocol/server-filesystem', '/path/to/files'],
  ),
);

List MCP Servers

final servers = await client.listMCPServers();
for (final server in servers) {
  print('${server.name}: ${server.command}');
}

Attach MCP Server to Agent

await client.attachMCPServerToAgent('agent-id', 'server-id', 1);

List Agent MCP Servers

final agentServers = await client.listAgentMCPServers('agent-id');
for (final agentServer in agentServers) {
  print('Server ID: ${agentServer.mcpServerId}');
}

RAG / Vector Search #

Upload File for RAG

final ragFile = await client.uploadRAGFile(
  UploadRAGFileRequest(
    sessionId: 'session-id',
    variableName: 'documentation',
    fileName: 'docs.txt',
    fileContent: 'Your documentation content here...',
    chunkSize: 1000,
    chunkOverlap: 200,
  ),
);

print('Uploaded: ${ragFile.fileName}, Chunks: ${ragFile.chunkCount}');

Search RAG Vectors

final results = await client.searchRAG(
  RAGSearchRequest(
    sessionId: 'session-id',
    variableName: 'documentation',
    query: 'How do I install the SDK?',
    topK: 5,
    similarityThreshold: 0.7,
  ),
);

for (final result in results) {
  print('Score: ${result.score}, Content: ${result.content}');
}

List RAG Files

final ragFiles = await client.listRAGFiles('session-id');
for (final file in ragFiles) {
  print('${file.fileName}: ${file.chunkCount} chunks');
}

Using Variables #

final agent = await client.createAgent(
  CreateAgentRequest(
    name: 'Support Agent',
    systemPrompt: 'You are a support agent for {{company_name}}.',
    variables: {
      'company_name': 'Acme Corp',
    },
  ),
);

// Variables will be automatically substituted in prompts
final response = await client.runInference(
  InferenceRequest(
    agentId: agent.id,
    prompt: 'What services do you offer?',
  ),
);

Error Handling #

try {
  final response = await client.runInference(
    InferenceRequest(
      agentId: 'invalid-id',
      prompt: 'Hello',
    ),
  );
} on AuthenticationException catch (e) {
  print('Authentication failed: ${e.message}');
} on NotFoundException catch (e) {
  print('Agent not found: ${e.message}');
} on RateLimitException catch (e) {
  print('Rate limited. Retry after: ${e.retryAfter} seconds');
} on ValidationException catch (e) {
  print('Validation error: ${e.message}');
} on ServerException catch (e) {
  print('Server error: ${e.message}');
} on NetworkException catch (e) {
  print('Network error: ${e.message}');
} on InferenceProviderException catch (e) {
  print('Error: ${e.message}');
}

Clean Up #

Remember to dispose of the client when done:

client.dispose();

Advanced Features #

Custom Base URL #

final client = InferenceProviderClient(
  apiKey: 'your-key',
  apiSecret: 'your-secret',
  baseUrl: 'https://custom-domain.com',
);

Custom HTTP Client #

import 'package:http/http.dart' as http;

final customClient = http.Client();
final client = InferenceProviderClient(
  apiKey: 'your-key',
  apiSecret: 'your-secret',
  httpClient: customClient,
);

API Reference #

InferenceProviderClient #

Main client class for interacting with the API.

Agent Methods:

createAgent(CreateAgentRequest) → Future<Agent>
getAgent(String) → Future<Agent>
listAgents() → Future<List<Agent>>
updateAgent(String, UpdateAgentRequest) → Future<Agent>
deleteAgent(String) → Future<void>

Inference Methods:

runInference(InferenceRequest) → Future<InferenceResponse>
runVisionInference(...) → Future<InferenceResponse>
getConversationHistory(String) → Future<List<ConversationMessage>>

Provider Methods:

createProvider(CreateProviderRequest) → Future<AIProvider>
getProvider(String) → Future<AIProvider>
listProviders() → Future<List<AIProvider>>
listModels(String) → Future<List<AIModel>>

Tool Methods:

createTool(CreateToolRequest) → Future<ToolDefinition>
getTool(String) → Future<ToolDefinition>
listTools() → Future<List<ToolDefinition>>
updateTool(String, UpdateToolRequest) → Future<ToolDefinition>
deleteTool(String) → Future<void>
attachToolToAgent(String, String, int?) → Future<AgentTool>
listAgentTools(String) → Future<List<AgentTool>>
detachToolFromAgent(String, String) → Future<void>

MCP Server Methods:

createMCPServer(CreateMCPServerRequest) → Future<MCPServer>
getMCPServer(String) → Future<MCPServer>
listMCPServers() → Future<List<MCPServer>>
updateMCPServer(String, UpdateMCPServerRequest) → Future<MCPServer>
deleteMCPServer(String) → Future<void>
attachMCPServerToAgent(String, String, int?) → Future<AgentMCPServer>
listAgentMCPServers(String) → Future<List<AgentMCPServer>>
detachMCPServerFromAgent(String, String) → Future<void>

RAG / Vector Methods:

uploadRAGFile(UploadRAGFileRequest) → Future<RAGVariableFile>
searchRAG(RAGSearchRequest) → Future<List<RAGSearchResult>>
listRAGFiles(String) → Future<List<RAGVariableFile>>
deleteRAGFile(String) → Future<void>

Utility:

dispose() → void

Exception Hierarchy #

InferenceProviderException - Base exception
- AuthenticationException - 401 errors
- ValidationException - 400 errors
- NotFoundException - 404 errors
- RateLimitException - 429 errors
- ServerException - 500+ errors
- NetworkException - Network failures

Examples #

See the example directory for a complete Flutter app demonstrating all features.

Requirements #

Dart SDK: >=3.0.0 <4.0.0
Flutter: >=3.0.0

Dependencies #

http: ^1.1.0 - HTTP client
json_annotation: ^4.8.1 - JSON serialization

Development #

Running Tests #

flutter test

Generating JSON Serialization #

flutter pub run build_runner build

Contributing #

Contributions are welcome! Please feel free to submit a Pull Request.

License #

This project is licensed under the MIT License - see the LICENSE file for details.

Support #

Documentation: GitHub Repository
Issues: GitHub Issues
API Documentation: Available in your web interface

TypeScript/JavaScript: npm install inference-provider-sdk
Python: pip install inference-provider-sdk

Made with ❤️ by DevMayur

inference_provider_sdk 1.0.2 inference_provider_sdk: ^1.0.2 copied to clipboard

Metadata