-
Notifications
You must be signed in to change notification settings - Fork 697
Open
Description
Hi, firstly, thank you so much for doing this.
this is not an issue, not sure where else i could post this. If there is a better place, pls let me know.
i saw that anyio was used in this manner to store the streaming message objects
https://github.com/run-llama/sec-insights/blob/main/backend/app/api/endpoints/conversation.py#L96
Wondering if there are any advantages using this vs using a python asynchronous queue object?
I'm asking because in my company policy bot's RAG implementation i'm currently storing it in a python queue object.
Does using anyio have any latency benefits?
Thanks!
Metadata
Metadata
Assignees
Labels
No labels