This project is still in progress
- Python 3.12 installed
ollamainstalled (Installation Guide)pipinstalled (comes with Python).msgemail files available for processing- LangSmith api key
- Huggingface api key
Run the following command in your project's root directory:
python3 -m venv myvenv or
conda create -n myvenv python=3.12 -ypython -m venv myvenv or
conda create -n myvenv python=3.12 -ysource myvenv/bin/activate or
conda activate myvenvmyvenv\Scripts\activate or
conda activate myvenvpip install -r requirements.txtBefore running the program, download the required model (or any model you wish to use):
ollama pull llama3.1
# Synchronous model inference (serial execution)
python email_preprocessing_sync.py /path/to/your/data/directory # Asynchronous model inference
python email_preprocessing_async.py /path/to/your/data/directory # Distributed model inference using vllm
python email_preprocessing_vllm_unordered.py /path/to/your/data/directory # Distributed model inference using accelerate with distinct prompts
# (will be transformed into an agent later)
python email_preprocessing_agent /path/to/your/data/directory# Create a vector database from a deduplicated email list
python create_embeddings.py
# Run Retrieval-Augmented Generation (RAG) on the vector database
python rag_embedDB.py# Create Neo4j knowledge graph from emails
python create_kg.py
# Run the bot with an agent implementing RAG on the knowledge graph
python bot.py /path/to/your/data/directory should be replaced with the actual path where your .msg files are stored.