Build better LLM apps — faster, smarter, production-ready.
A curated, list of 100+ libraries and frameworks for AI engineers building with Large Language Models. This toolkit includes battle-tested tools, frameworks, templates, and reference implementations for developing, deploying, and optimizing LLM-powered systems.
| Tool | Description | Language | License |
|---|---|---|---|
| Pinecone | Managed vector database for production AI applications | API/SDK | Commercial |
| Weaviate | Open-source vector database with GraphQL API | Go | BSD-3 |
| Qdrant | Vector similarity search engine with extended filtering | Rust | Apache-2.0 |
| Chroma | Open-source embedding database for LLM apps | Python | Apache-2.0 |
| Milvus | Cloud-native vector database for scalable similarity search | Go/C++ | Apache-2.0 |
| FAISS | Library for efficient similarity search and clustering | C++/Python | MIT |
| Tool | Description | Language | License |
|---|---|---|---|
| LangChain | Framework for developing LLM applications | Python/JS | MIT |
| LlamaIndex | Data framework for LLM applications | Python | MIT |
| Haystack | End-to-end NLP framework for production | Python | Apache-2.0 |
| DSPy | Framework for algorithmically optimizing LM prompts | Python | MIT |
| Semantic Kernel | SDK for integrating AI into conventional programming languages | C#/Python/Java | MIT |
| Langflow | Visual no-code platform for building and deploying LLM workflows | Python/TypeScript | MIT |
| Flowise | Drag-and-drop UI for creating LLM chains and agents | TypeScript | MIT |
| Tool | Description | Language | License |
|---|---|---|---|
| Docling | AI-powered toolkit converting PDF, DOCX, PPTX, HTML, images into structured JSON/Markdown with layout, OCR, table, and code recognition | Python | MIT |
| pdfplumber | Drill through PDFs at a character level, extract text & tables, and visually debug extraction | Python | MIT |
| PyMuPDF (fitz) | Lightweight, high-performance PDF parser for text/image extraction and manipulation | Python / C | AGPL-3.0 |
| PDF.js | Browser-based PDF renderer with text extraction capabilities | JavaScript | Apache-2.0 |
| Camelot | Extracts structured tabular data from PDFs into DataFrames and CSVs | Python | MIT |
| Tool | Description | Language | License |
|---|---|---|---|
| RAGFlow | Open-source RAG engine based on deep document understanding | Python | Apache-2.0 |
| Verba | Retrieval Augmented Generation (RAG) chatbot | Python | BSD-3 |
| PrivateGPT | Interact with documents using local LLMs | Python | Apache-2.0 |
| AnythingLLM | All-in-one AI application for any LLM | JavaScript | MIT |
| Quivr | Your GenAI second brain | Python/TypeScript | Apache-2.0 |
| Jina | Cloud-native neural search framework for multimodal RAG | Python | Apache-2.0 |
| txtai | All-in-one embeddings database for semantic search and workflows | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Ragas | Evaluation framework for RAG pipelines | Python | Apache-2.0 |
| LangSmith | Platform for debugging, testing, and monitoring LLM applications | API/SDK | Commercial |
| Phoenix | ML observability for LLM, vision, language, and tabular models | Python | Apache-2.0 |
| DeepEval | LLM evaluation framework for unit testing LLM outputs | Python | Apache-2.0 |
| TruLens | Evaluation and tracking for LLM experiments | Python | MIT |
| Inspect | Framework for large language model evaluations | Python | Apache-2.0 |
| UpTrain | Open-source tool to evaluate and improve LLM applications | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Hugging Face Hub | Client library for Hugging Face Hub | Python | Apache-2.0 |
| MLflow | Platform for ML lifecycle management | Python | Apache-2.0 |
| Weights & Biases | Developer tools for ML | Python | MIT |
| DVC | Data version control for ML projects | Python | Apache-2.0 |
| Comet ML | Experiment tracking and visualization for ML/LLM workflows | Python | MIT |
| ClearML | End-to-end MLOps platform with LLM support | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Firecrawl | AI-powered web crawler that extracts and structures content for LLM pipelines | TypeScript | MIT |
| Scrapy | Fast, high-level web crawling & scraping framework | Python | BSD-3 |
| Playwright | Web automation & scraping with headless browsers | TypeScript/Python/Java/.NET | Apache-2.0 |
| BeautifulSoup | Easy HTML/XML parsing for quick scraping tasks | Python | MIT |
| Selenium | Browser automation framework (supports scraping) | Multiple | Apache-2.0 |
| Apify SDK | Web scraping & automation platform SDK | Python/JavaScript | Apache-2.0 |
| Newspaper3k | News & article extraction library | Python | MIT |
| Framework | Description | Language | License |
|---|---|---|---|
| AutoGen | Multi-agent conversation framework | Python | CC-BY-4.0 |
| CrewAI | Framework for orchestrating role-playing autonomous AI agents | Python | MIT |
| LangGraph | Build resilient language agents as graphs | Python | MIT |
| AgentOps | Python SDK for AI agent monitoring, LLM cost tracking, benchmarking | Python | MIT |
| Swarm | Educational framework for exploring ergonomic, lightweight multi-agent orchestration | Python | MIT |
| Agency Swarm | An open-source agent framework designed to automate your workflows | Python | MIT |
| Multi-Agent Systems | Research into multi-agent systems and applications | Python | MIT |
| Auto-GPT | Autonomous AI agent for task execution using GPT models | Python | MIT |
| BabyAGI | Task-driven autonomous agent inspired by AGI | Python | MIT |
| SuperAGI | Infrastructure for building and managing autonomous agents | Python | MIT |
| Phidata | Build AI agents with memory, tools, and knowledge | Python | MIT |
| MemGPT | Self-improving agents with infinite context via memory management | Python | MIT |
| Tool | Description | Language | License |
|---|---|---|---|
| PyTorch Lightning | High-level PyTorch interface for LLMs | Python | Apache-2.0 |
| unsloth | Fine-tune LLMs faster with less memory | Python | Apache-2.0 |
| Axolotl | Post-training pipeline for AI models | Python | Apache-2.0 |
| LLaMA-Factory | Easy & efficient LLM fine-tuning | Python | Apache-2.0 |
| PEFT | Parameter-Efficient Fine-Tuning library | Python | Apache-2.0 |
| DeepSpeed | Distributed training & inference optimization | Python | MIT |
| TRL | Train transformer LMs with reinforcement learning | Python | Apache-2.0 |
| Transformers | Pretrained models for text, vision, and audio tasks | Python | Apache-2.0 |
| LLMBox | Unified training pipeline & model evaluation | Python | MIT |
| LitGPT | Train and fine-tune LLMs lightning fast | Python | Apache-2.0 |
| Mergoo | Merge multiple LLM experts efficiently | Python | Apache-2.0 |
| Ludwig | Low-code framework for custom LLMs | Python | Apache-2.0 |
| txtinstruct | Framework for training instruction-tuned models | Python | Apache-2.0 |
| xTuring | Fast fine-tuning of open-source LLMs | Python | Apache-2.0 |
| RL4LMs | RL library to fine-tune LMs to human preferences | Python | Apache-2.0 |
| torchtune | PyTorch-native library for fine-tuning LLMs | Python | BSD-3 |
| Accelerate | Library to easily train on multiple GPUs/TPUs with mixed precision | Python | Apache-2.0 |
| BitsandBytes | 8-bit optimizers and quantization for efficient LLM training | Python | MIT |
| Tool | Description | Language | License |
|---|---|---|---|
| LLM Compressor | Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment | Python | Apache-2.0 |
| LightLLM | Lightweight Python-based LLM inference and serving framework with easy scalability and high performance | Python | Apache-2.0 |
| vLLM | High-throughput and memory-efficient inference and serving engine for LLMs | Python | Apache-2.0 |
| torchchat | Run PyTorch LLMs locally on servers, desktop, and mobile | Python | MIT |
| TensorRT-LLM | NVIDIA library for optimizing LLM inference with TensorRT | C++/Python | Apache-2.0 |
| WebLLM | High-performance in-browser LLM inference engine | TypeScript/Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| JailbreakEval | Automated evaluators for assessing jailbreak attempts | Python | MIT |
| EasyJailbreak | Easy-to-use Python framework to generate adversarial jailbreak prompts | Python | Apache-2.0 |
| Guardrails | Add guardrails to large language models | Python | MIT |
| LLM Guard | Security toolkit for LLM interactions | Python | Apache-2.0 |
| AuditNLG | Reduce risks in generative AI systems for language | Python | MIT |
| NeMo Guardrails | Toolkit for adding programmable guardrails to LLM conversational systems | Python | Apache-2.0 |
| Garak | LLM vulnerability scanner | Python | MIT |
| DeepTeam | LLM red teaming framework | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Reflex | Build full-stack web apps powered by LLMs with Python-only workflows and reactive UIs. | Python | Apache-2.0 |
| Gradio | Create quick, interactive UIs for LLM demos and prototypes. | Python | Apache-2.0 |
| Streamlit | Build and share AI/ML apps fast with Python scripts and interactive widgets. | Python | Apache-2.0 |
| Taipy | End-to-end Python framework for building production-ready AI apps with dashboards and pipelines. | Python | Apache-2.0 |
| Tool | Description | Language | License |
|---|---|---|---|
| Ollama | Get up and running with large language models locally | Go | MIT |
| LM Studio | Desktop app for running local LLMs | - | Commercial |
| GPT4All | Open-source chatbot ecosystem | C++ | MIT |
| LocalAI | Self-hosted OpenAI-compatible API | Go | MIT |
| Platform | Description | Pricing | Features |
|---|---|---|---|
| Clarifai | Lightning-fast compute for AI models & agents | Free tier + Pay-as-you-go | Pre-trained models, Deploy your own models on Dedicated compute, Model training, Workflow automation |
| Modal | Serverless platform for AI/ML workloads | Pay-per-use | Serverless GPU, Auto-scaling |
| Replicate | Run open-source models with a cloud API | Pay-per-use | Pre-built models, Custom training |
| Together AI | Cloud platform for open-source models | Various | Open models, Fine-tuning |
| Anyscale | Ray-based platform for AI applications | Enterprise | Distributed training, Serving |
We welcome contributions! This toolkit grows stronger with community input.
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-tool) - Add your contribution (new tool, template, or tutorial)
- Submit a pull request
- Quality over quantity - Focus on tools and resources that provide real value
- Production-ready - Include tools that work in real-world scenarios
- Well-documented - Provide clear descriptions and usage examples
- Up-to-date - Ensure tools are actively maintained
Get weekly AI engineering insights, tool reviews, and exclusive demos and AI Projects delivered to your inbox:
📧 Subscribe to AI Engineering Newsletter →
Join 100,000+ engineers building better LLM applications
Built with ❤️ for the AI Engineering community
Star ⭐ this repo if you find it helpful!