Skip to content

Commit 6776c1e

Browse files
alexrudallalexrudall
committed
Add specs and docs
1 parent 1ded4b7 commit 6776c1e

File tree

4 files changed

+136
-2
lines changed

4 files changed

+136
-2
lines changed

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ puts response.dig("choices", 0, "message", "content")
134134

135135
[Quick guide to streaming ChatGPT with Rails 7 and Hotwire](https://gist.github.com/alexrudall/cb5ee1e109353ef358adb4e66631799d)
136136

137-
You can stream from the API in realtime, which can be much faster and used to create a more engaging user experience. Pass a [Proc](https://ruby-doc.org/core-2.6/Proc.html) to the `stream` parameter to receive the stream of text chunks as they are generated. Each time one or more chunks is received, the Proc will be called once with each chunk, parsed as a Hash. If OpenAI returns an error, `ruby-openai` will pass that to your proc as a Hash.
137+
You can stream from the API in realtime, which can be much faster and used to create a more engaging user experience. Pass a [Proc](https://ruby-doc.org/core-2.6/Proc.html) (or any object with a `#call` method) to the `stream` parameter to receive the stream of text chunks as they are generated. Each time one or more chunks is received, the proc will be called once with each chunk, parsed as a Hash. If OpenAI returns an error, `ruby-openai` will pass that to your proc as a Hash.
138138

139139
```ruby
140140
client.chat(

‎lib/openai/http.rb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ def json_post(path:, parameters:)
1212
req.options.on_data = to_json_stream(user_proc: parameters[:stream])
1313
parameters[:stream] = true # Necessary to tell OpenAI to stream.
1414
elsif parameters[:stream]
15-
raise ArgumentError, "stream must be callable"
15+
raise ArgumentError, "The stream parameter must be a Proc or have a #call method"
1616
end
1717

1818
req.headers = headers

‎spec/fixtures/cassettes/gpt-3_5-turbo_streamed_chat_without_proc.yml

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
---
2+
http_interactions:
3+
- request:
4+
method: post
5+
uri: https://api.openai.com/v1/chat/completions
6+
body:
7+
encoding: UTF-8
8+
string: '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"Hello!"}],"stream":true}'
9+
headers:
10+
Content-Type:
11+
- application/json
12+
Authorization:
13+
- Bearer <OPENAI_ACCESS_TOKEN>
14+
Accept-Encoding:
15+
- gzip;q=1.0,deflate;q=0.6,identity;q=0.3
16+
Accept:
17+
- "*/*"
18+
User-Agent:
19+
- Ruby
20+
response:
21+
status:
22+
code: 200
23+
message: OK
24+
headers:
25+
Date:
26+
- Mon, 15 May 2023 05:08:18 GMT
27+
Content-Type:
28+
- text/event-stream
29+
Transfer-Encoding:
30+
- chunked
31+
Connection:
32+
- keep-alive
33+
Access-Control-Allow-Origin:
34+
- "*"
35+
Cache-Control:
36+
- no-cache, must-revalidate
37+
Openai-Model:
38+
- gpt-3.5-turbo-0301
39+
Openai-Organization:
40+
- user-jxm65ijkzc1qrfhc0ij8moic
41+
Openai-Processing-Ms:
42+
- '485'
43+
Openai-Version:
44+
- '2020-10-01'
45+
Strict-Transport-Security:
46+
- max-age=15724800; includeSubDomains
47+
X-Ratelimit-Limit-Requests:
48+
- '3500'
49+
X-Ratelimit-Limit-Tokens:
50+
- '90000'
51+
X-Ratelimit-Remaining-Requests:
52+
- '3499'
53+
X-Ratelimit-Remaining-Tokens:
54+
- '89980'
55+
X-Ratelimit-Reset-Requests:
56+
- 17ms
57+
X-Ratelimit-Reset-Tokens:
58+
- 12ms
59+
X-Request-Id:
60+
- 80aafe6bf6d6cc66fdeb4bc6fbff1065
61+
Cf-Cache-Status:
62+
- DYNAMIC
63+
Server:
64+
- cloudflare
65+
Cf-Ray:
66+
- 7c78dcbc0d318862-LHR
67+
Alt-Svc:
68+
- h3=":443"; ma=86400, h3-29=":443"; ma=86400
69+
body:
70+
encoding: UTF-8
71+
string: |+
72+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"role":"assistant"},"index":0,"finish_reason":null}]}
73+
74+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"Hello"},"index":0,"finish_reason":null}]}
75+
76+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" there"},"index":0,"finish_reason":null}]}
77+
78+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"!"},"index":0,"finish_reason":null}]}
79+
80+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" How"},"index":0,"finish_reason":null}]}
81+
82+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" can"},"index":0,"finish_reason":null}]}
83+
84+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" I"},"index":0,"finish_reason":null}]}
85+
86+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" assist"},"index":0,"finish_reason":null}]}
87+
88+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" you"},"index":0,"finish_reason":null}]}
89+
90+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" today"},"index":0,"finish_reason":null}]}
91+
92+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"?"},"index":0,"finish_reason":null}]}
93+
94+
data: {"id":"chatcmpl-7GKjSFht9ZKlAR0YpUlpqfOdJbcjI","object":"chat.completion.chunk","created":1684127298,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{},"index":0,"finish_reason":"stop"}]}
95+
96+
data: [DONE]
97+
98+
recorded_at: Mon, 15 May 2023 05:08:19 GMT
99+
recorded_with: VCR 6.1.0
100+
...

‎spec/openai/client/chat_spec.rb

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,40 @@
3838
expect(chunks.dig(0, "choices", 0, "index")).to eq(0)
3939
end
4040
end
41+
42+
context "with an object with a call method" do
43+
let(:cassette) { "#{model} streamed chat without proc".downcase }
44+
let(:stream) do
45+
Class.new do
46+
attr_reader :chunks
47+
48+
def initialize
49+
@chunks = []
50+
end
51+
52+
def call(chunk)
53+
@chunks << chunk
54+
end
55+
end.new
56+
end
57+
58+
it "succeeds" do
59+
VCR.use_cassette(cassette) do
60+
response
61+
expect(stream.chunks.dig(0, "choices", 0, "index")).to eq(0)
62+
end
63+
end
64+
end
65+
66+
context "with an object without a call method" do
67+
let(:stream) { Object.new }
68+
69+
it "raises an error" do
70+
VCR.use_cassette(cassette) do
71+
expect { response }.to raise_error(ArgumentError)
72+
end
73+
end
74+
end
4175
end
4276
end
4377

0 commit comments

Comments
 (0)