Skip to content

openai_stt.py _turn_audio_buffer maybe empty #821

Open
@moseshu

Description

@moseshu

Describe the bug

_turn_audio_buffer may be empty

Error in audio forwarder: need at least one array to concatenate
Traceback (most recent call last):
  File "/workspace/LLM-TEST/llm-test/videodigital/openai_realtime_tts.py", line 84, in forward_pipeline_events
    async for event in result_stream:
  File "/usr/local/lib/python3.11/site-packages/agents/voice/result.py", line 287, in stream
    raise self._stored_exception
  File "/usr/local/lib/python3.11/site-packages/agents/voice/pipeline.py", line 145, in process_turns
    raise e
  File "/usr/local/lib/python3.11/site-packages/agents/voice/pipeline.py", line 137, in process_turns
    async for input_text in transcription_session.transcribe_turns():
  File "/usr/local/lib/python3.11/site-packages/agents/voice/models/openai_stt.py", line 356, in transcribe_turns
    raise self._stored_exception
  File "/usr/local/lib/python3.11/site-packages/agents/voice/models/openai_stt.py", line 241, in _handle_events
    raise e
  File "/usr/local/lib/python3.11/site-packages/agents/voice/models/openai_stt.py", line 232, in _handle_events
    self._end_turn(transcript)
  File "/usr/local/lib/python3.11/site-packages/agents/voice/models/openai_stt.py", line 126, in _end_turn
    self._tracing_span.span_data.input = _audio_to_base64(self._turn_audio_buffer)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/agents/voice/models/openai_stt.py", line 43, in _audio_to_base64
    concatenated_audio = np.concatenate(audio_data)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: need at least one array to concatenate

Debug information

  • Agents SDK version: (e.g. v0.0.17)
  • Python version (e.g. Python 3.11)

Repro steps

_turn_audio_buffer may be empty

Expected behavior

def _end_turn(self, _transcript: str) -> None:
    if len(_transcript) < 1:
        return

    if self._tracing_span:
        # Check if there is audio data to process AND tracing is enabled for it
        if self._trace_include_sensitive_audio_data and self._turn_audio_buffer: # <-- ADDED CHECK HERE
            self._tracing_span.span_data.input = _audio_to_base64(self._turn_audio_buffer)

        self._tracing_span.span_data.input_format = "pcm"

        if self._trace_include_sensitive_data:
            self._tracing_span.span_data.output = _transcript

        self._tracing_span.finish()
        self._turn_audio_buffer = []  # Clear the buffer for the next turn
        self._tracing_span = None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions