Initial message is not sent

[ilirosmanaj]

Even when uncommenting the line for sending the initial message, it's not said in the call and it starts based on what the assistant figures from the system prompt. Any ideas?

[ilirosmanaj]

Tagging @pkamp3 just in case

[ilirosmanaj]

@pkamp3 any ideas?

[pkamp3]

Unfortunately, I can't reproduce this - I tried with both Python and Node and the assistant speaks first for me.

What have you tried to debug so far? I'd hesitate to guess here, but you could try adding a delay before https://github.com/twilio-samples/speech-assistant-openai-realtime-api-python/blob/main/main.py#L205-L206 to see if it changes the behavior.

Here is what OpenAI says you should receive after sending the response.create.

[ilirosmanaj]

It seems the AI kicks in too fast and this is not respected. Even with a delay of 4s before and after doesn't seem to make a difference for me.

This is how the function looks, so not sure how to debug this for now 🤔

async def send_initial_conversation_item(openai_ws):
    """Send initial conversation item if AI talks first."""
    initial_conversation_item = {
        "type": "conversation.item.create",
        "item": {
            "type": "message",
            "role": "user",
            "content": [
                {
                    "type": "input_text",
                    "text": "Hallo ich bin die KI Anrufannahme der Ilir. Wie kann ich Ihnen heute helfen?"
                }
            ]
        }
    }
    time.sleep(4)
    await openai_ws.send(json.dumps(initial_conversation_item))
    await openai_ws.send(json.dumps({"type": "response.create"}))
    time.sleep(4)