Fix streaming data parsing in OpenAI::HTTP's to_json_stream

Problem:
The to_json_stream method failed to handle JSON objects split across multiple data packets from the Azure AI service, leading to JSON parsing errors and incomplete/missing data.

Solution:
Implemented a buffering mechanism within to_json_stream to accumulate data until a complete JSON object is formed. This ensures proper parsing and data integrity, regardless of how the data packets are received.

Author: mrbokx

lib/openai/http.rb

@@ -54,10 +54,16 @@ def to_json(string)
     # @return [Proc] An outer proc that iterates over a raw stream, converting it to JSON.
     def to_json_stream(user_proc:)
       proc do |chunk, _|
-        chunk.scan(/(?:data|error): (\{.*\})/i).flatten.each do |data|
-          user_proc.call(JSON.parse(data))
-        rescue JSON::ParserError
-          # Ignore invalid JSON.
+        @buffer ||= ""
+        @buffer += chunk
+        while (match = @buffer.match(/(?:data|error): (\{.*\})/i))
+          data = match[1]
+          @buffer = @buffer[match.end(0)..-1]  # Remove the processed data from the buffer
+          begin
+            user_proc.call(JSON.parse(data))
+          rescue JSON::ParserError => e
+            # Ignore invalid JSON.
+          end
         end
       end
     end