🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN

Embedding Atlas is a tool that provides interactive visualizations for large embeddings. It allows you to visualize, cross-filter, and search embeddings and metadata.

RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal device.

This repository contains expert evaluation interface and data evaluation script for the OpenScholar project.

Everything you need to know to build your own RAG application

Efficient Triton Kernels for LLM Training

A curated list of papers and resources based on "Large Language Models on Graphs: A Comprehensive Survey" (TKDE)

Fast and memory-efficient exact attention

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality

Implementation for MatMul-free LM.

Evaluate your LLM's response with Prometheus and GPT4 💯

[SIGIR 2024] The official repo for paper "Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding"

언어모델을 학습하기 위한 공개 한국어 instruction dataset들을 모아두었습니다.

Perplexica is an AI-powered answering engine. It is an Open source alternative to Perplexity AI

A list of multi-vector retrieval resources

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Longformer: The Long-Document Transformer

[WWW 2024] The official repo for paper "Scalable and Effective Generative Information Retrieval".

An open science effort to benchmark legal reasoning in foundation models

LexGLUE: A Benchmark Dataset for Legal Language Understanding in English

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

A series of large language models trained from scratch by developers @01-ai

A curated list of awesome LLM agents frameworks.

LLM finetuned for medical question answering

Reading list of hallucination in LLMs. Check out our new survey paper: "Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models"

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.