|
19 | 19 | "- Chroma DB for storing cached query results and performing RAG\n",
|
20 | 20 | "- LangGraph for creating an agent\n",
|
21 | 21 | "\n",
|
22 |
| - "Best of all, because both Glean and NVIDIA NIMs can be deployed in your private cloud, it is possible to create this type of enterprise chatbot without any data leaving your control.\n", |
| 22 | + "Best of all, because both Glean and NVIDIA NIMs can be deployed in your private environment, it is possible to create this type of enterprise chatbot without any data leaving your control.\n", |
23 | 23 | "\n",
|
24 | 24 | "To get started with this notebook, set the following environment variables. You will need a Glean deployment, a Glean API key, and a [NVIDA API Key](https://build.nvidia.com)."
|
25 | 25 | ]
|
|
41 | 41 | "id": "d0e2689f-335d-4203-8121-530641257de9",
|
42 | 42 | "metadata": {},
|
43 | 43 | "source": [
|
44 |
| - "We start by instantiating the LLM and embedding model. You can update this code to use different foundational LLMs, or add the `base_url` parameter if you are using on-premise NVIDIA NIMs." |
| 44 | + "We start by instantiating the LLM and embedding model. You can update this code to use different foundational LLMs, or add the `base_url` parameter if you are using private NVIDIA NIM microservices." |
45 | 45 | ]
|
46 | 46 | },
|
47 | 47 | {
|
|
88 | 88 | "id": "3beda74c-7885-4885-babd-8166a049fd38",
|
89 | 89 | "metadata": {},
|
90 | 90 | "source": [
|
91 |
| - "While the model is able to interpret our question and formulate a response, it does not have access to any information about company-specific policies. To add this type of information we will follow a two multi-step process: \n", |
| 91 | + "While the model is able to interpret our question and formulate a response, it does not have access to any information about company-specific policies. To add this type of information we will follow a multi-step process: \n", |
92 | 92 | "\n",
|
93 |
| - "1. Have the LLM translate the user's question into a query for the Glean knowledge base.\n", |
94 |
| - "2. Query the Glean knowledge base using the Glean search API to get the most relevant supporting documents.\n", |
| 93 | + "1. Have the LLM interpret the user's question and add any relevant context. Most free form questions can be passed directly to the Glean Search API.\n", |
| 94 | + "2. Add relevant context about the user and then query the Glean knowledge base using the Glean search API to get the most relevant supporting documents. \n", |
95 | 95 | "3. Embed those supporting documents into a local vector DB.\n",
|
96 | 96 | "4. Use a retriever model to fetch the most relevant supporting document based on the user's original question.\n",
|
97 |
| - "5. Take the most relevant supporting document and add it to the LLM by adding it to the model's prompt (RAG).\n", |
98 |
| - "6. Ask the model to answer the user's question with this new relevant context.\n", |
| 97 | + "5. Take the most relevant supporting document and add it to the LLM by adding it to the LLM's prompt (RAG).\n", |
| 98 | + "6. Ask the LLM to summarize the results and answer the user's question with this new relevant context.\n", |
99 | 99 | "\n",
|
100 | 100 | "To help organize these steps we use a LangGraph agent. The full implementation of the agent is available in the file `glean_example/src/agent.py`. The following code samples explain some core concepts of that code.\n",
|
101 | 101 | "\n",
|
102 | 102 | "```python\n",
|
103 | 103 | "class InfoBotState(BaseModel):\n",
|
104 | 104 | " messages: List[Tuple[str, str]] = None\n",
|
105 |
| - " glean_query: Optional[str] = None\n", |
| 105 | + " glean_query_required: Optional[bool] = None\n", |
106 | 106 | " glean_results: Optional[List[str]] = None\n",
|
107 | 107 | " db: Optional[Any] = None\n",
|
108 | 108 | " answer_candidate: Optional[str] = None\n",
|
109 | 109 | "\n",
|
110 | 110 | "graph = StateGraph(InfoBotState)\n",
|
111 |
| - "graph.add_node(\"call_bot\", call_bot)\n", |
| 111 | + "graph.add_node(\"determine_user_intent\", determine_user_intent)\n", |
112 | 112 | "graph.add_node(\"call_glean\", call_glean)\n",
|
113 |
| - "graph.add_node(\"answer_candidates\", answer_candidates)\n", |
114 |
| - "graph.add_node(\"create_glean_query\", create_glean_query)\n", |
115 | 113 | "graph.add_node(\"add_embeddings\", add_embeddings)\n",
|
116 |
| - "\n", |
117 |
| - "graph.add_edge(START, \"create_glean_query\")\n", |
118 |
| - "graph.add_edge(\"create_glean_query\", \"call_glean\")\n", |
| 114 | + "graph.add_node(\"answer_candidates\", answer_candidates)\n", |
| 115 | + "graph.add_node(\"summarize_answer\", summarize_answer)\n", |
| 116 | + "graph.add_edge(START, \"determine_user_intent\")\n", |
| 117 | + "graph.add_conditional_edges(\n", |
| 118 | + " \"determine_user_intent\",\n", |
| 119 | + " route_glean, \n", |
| 120 | + " {\"call_glean\": \"call_glean\", \"summarize_answer\": \"summarize_answer\"}\n", |
| 121 | + ")\n", |
119 | 122 | "graph.add_edge(\"call_glean\", \"add_embeddings\")\n",
|
120 | 123 | "graph.add_edge(\"add_embeddings\", \"answer_candidates\")\n",
|
121 |
| - "graph.add_edge(\"answer_candidates\", \"call_bot\")\n", |
122 |
| - "graph.add_edge(\"call_bot\", END)\n", |
| 124 | + "graph.add_edge(\"answer_candidates\", \"summarize_answer\")\n", |
| 125 | + "graph.add_edge(\"summarize_answer\", END)\n", |
123 | 126 | "agent = graph.compile()\n",
|
| 127 | + "\n", |
124 | 128 | "```\n",
|
125 | 129 | "\n",
|
126 | 130 | "This code is responsible for creating the agent. Each node represents a function responsible for implementing one of the six steps in our process. The `InfoBotState` is a special type of dictionary that will hold all of the information the agent needs through each step of the process. \n",
|
127 | 131 | "\n",
|
128 | 132 | "The source of each function is also available in `glean_example/src/agent.py`. For example, the implementation of `call_bot` is: \n",
|
129 | 133 | "\n",
|
130 | 134 | "```python\n",
|
131 |
| - "def call_bot(state: InfoBotState):\n", |
| 135 | + "def summarize_answer(state: InfoBotState):\n", |
132 | 136 | " \"\"\"the main agent responsible for taking all the context and answering the question\"\"\"\n",
|
133 | 137 | " logger.info(\"Generate final answer\")\n",
|
134 | 138 | "\n",
|
|
137 | 141 | " response = llm.invoke(\n",
|
138 | 142 | " {\n",
|
139 | 143 | " \"messages\": state.messages,\n",
|
140 |
| - " \"glean_query\": state.glean_query,\n", |
141 | 144 | " \"glean_search_result_documents\": state.glean_results,\n",
|
142 | 145 | " \"answer_candidate\": state.answer_candidate,\n",
|
143 | 146 | " }\n",
|
|
146 | 149 | " return state\n",
|
147 | 150 | "```\n",
|
148 | 151 | "\n",
|
149 |
| - "This function takes the NVIDIA NIM foundational LLM model and invokes it with a specific prompt and the information available in the agent state. The prompt tells the agent what to do, injecting the relevant information fronm the agent state. You can see the prompts in the file `glean_example/src/prompts.py`. For example, the `PROMPT_ANSWER` is: \n", |
| 152 | + "This function takes the NVIDIA NIM foundational LLM model and invokes it with a specific prompt and the information available in the agent state. The prompt tells the agent what to do, injecting the relevant information from the agent state. You can see the prompts in the file `glean_example/src/prompts.py`. For example, the `PROMPT_ANSWER` is: \n", |
150 | 153 | "\n",
|
151 | 154 | "```raw\n",
|
152 | 155 | "You are the final part of an agent graph. Your job is to answer the user's question based on the information below. Include a url citation in your answer.\n",
|
153 | 156 | "\n",
|
154 | 157 | "Message History: {messages}\n",
|
155 | 158 | "\n",
|
156 |
| - "Glean Search: {glean_query}\n", |
157 |
| - "\n", |
158 | 159 | "All Supporting Documents from Glean: \n",
|
159 | 160 | "\n",
|
160 | 161 | "{glean_search_result_documents}\n",
|
|
227 | 228 | "source": [
|
228 | 229 | "from glean_example.src.agent import agent\n",
|
229 | 230 | "\n",
|
230 |
| - "msg = \"do I need to take PTO if I am sick\"\n", |
| 231 | + "msg = \"What's the latest on the new API project?\"\n", |
231 | 232 | "history = []\n",
|
232 | 233 | "history.append((\"user\", msg))\n",
|
233 | 234 | "response = agent.invoke({\"messages\": history})\n",
|
|
0 commit comments