[feature] configurable llm

[AlbertDeFusco]

The LLM must be a subclass of langchain BaseChatModel. You will need to separately install the required langchain integration packages, only langchain-core is a required dependency.

To change the LLM use the ~/.anaconda/config.toml file

[plugin.assistant_conda.llm]
driver = "langchain_ollama.chat_models:ChatOllama"
params = { model = "llama2", temperature = 0.1 }
combine_messages = true

For the above llama2 model you will need to 1) have Ollama installed and ollama serve running and 2) install langchain-ollama

Some models do not work well with separate system and user messages. The default for combine_messages above is false. If you encounter issues with the response try setting it to true