The goal of the project is to enhance the performance of text-to-SQL conversion using an agentic approach. Each agent in the system will handle a specialized task in the SQL generation process. The project aims to evaluate the performance of the existing MAC-SQL approach, implemented on an open-source model (Llama2), and to achieve better results. This will be accomplished through prompt engineering, the addition of specialized agents, or fine-tuning the model to create more effective agents tailored for specific tasks in the text-to-SQL pipeline.

1. Config your local environment.

conda create -n macsql python=3.9 -y
conda activate macsql
pip install -r requirements.txt
python -c "import nltk; nltk.download('punkt')"

2. Install ollama, download llama3.1 8B model and start the server.

curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.1
./ollama serve 

This will help us to give the model an input thriugh rest API.

Download the bird and spider datasets from Google Drive. After downloading the data.zip file, you should delete the existing data folder in the project directory and replace it with the unzipped data folder from data.zip.

The run script (run.sh) contains the commands to run the pipeline on spider and bird datasets. All the logs and predictions are stored in output folder.

After running the run.sh script, run the evaluation scripts evaluation_spider_ex.sh for spider and evaluation_bird_ex_ves.sh for bird dataset.

Run streamlit run demo.py

Here’s the table updated with the additional column for Execution Accuracy (%) (With Review Agent Modified Prompt):