From 2abb5f02bcbb9738fdeabeadb935a4c708795a6f Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Mon, 28 Apr 2025 22:38:19 +0100 Subject: [PATCH 1/5] websockets new path --- .../docs/ai-gateway/{configuration => }/websockets-api/index.mdx | 1 + .../{configuration => }/websockets-api/non-realtime-api.mdx | 0 .../{configuration => }/websockets-api/realtime-api.mdx | 0 3 files changed, 1 insertion(+) rename src/content/docs/ai-gateway/{configuration => }/websockets-api/index.mdx (99%) rename src/content/docs/ai-gateway/{configuration => }/websockets-api/non-realtime-api.mdx (100%) rename src/content/docs/ai-gateway/{configuration => }/websockets-api/realtime-api.mdx (100%) diff --git a/src/content/docs/ai-gateway/configuration/websockets-api/index.mdx b/src/content/docs/ai-gateway/websockets-api/index.mdx similarity index 99% rename from src/content/docs/ai-gateway/configuration/websockets-api/index.mdx rename to src/content/docs/ai-gateway/websockets-api/index.mdx index 731efe17563d829..3e1a24e107fc7a2 100644 --- a/src/content/docs/ai-gateway/configuration/websockets-api/index.mdx +++ b/src/content/docs/ai-gateway/websockets-api/index.mdx @@ -4,6 +4,7 @@ pcx_content_type: configuration sidebar: group: badge: Beta + order: 5 --- The AI Gateway WebSockets API provides a persistent connection for AI interactions, eliminating repeated handshakes and reducing latency. This API is divided into two categories: diff --git a/src/content/docs/ai-gateway/configuration/websockets-api/non-realtime-api.mdx b/src/content/docs/ai-gateway/websockets-api/non-realtime-api.mdx similarity index 100% rename from src/content/docs/ai-gateway/configuration/websockets-api/non-realtime-api.mdx rename to src/content/docs/ai-gateway/websockets-api/non-realtime-api.mdx diff --git a/src/content/docs/ai-gateway/configuration/websockets-api/realtime-api.mdx b/src/content/docs/ai-gateway/websockets-api/realtime-api.mdx similarity index 100% rename from src/content/docs/ai-gateway/configuration/websockets-api/realtime-api.mdx rename to src/content/docs/ai-gateway/websockets-api/realtime-api.mdx From 1fdc725289cb6e386d7f786c1db5361eabca3a11 Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Mon, 28 Apr 2025 23:00:51 +0100 Subject: [PATCH 2/5] invalid links --- public/__redirects | 4 +++ .../ai-gateway/2025-03-20-websockets.mdx | 30 ++++++++++--------- src/content/docs/ai-gateway/get-started.mdx | 4 +-- .../docs/ai-gateway/providers/universal.mdx | 2 +- .../docs/ai-gateway/websockets-api/index.mdx | 14 ++++----- 5 files changed, 30 insertions(+), 24 deletions(-) diff --git a/public/__redirects b/public/__redirects index 8cbe8b2402d7fac..8ab2efe98949143 100644 --- a/public/__redirects +++ b/public/__redirects @@ -145,6 +145,10 @@ /ai-gateway/integration/aig-workers-ai-binding/ /ai-gateway/integrations/aig-workers-ai-binding/ 301 /ai-gateway/integration/ /ai-gateway/integrations/ 301 /ai-gateway/providers/open-router/ /ai-gateway/providers/openrouter/ 301 +/ai-gateway/configuration/websockets-api/ /ai-gateway/websockets-api/ 301 +/ai-gateway/configuration/websockets-api/non-realtime-api/ /ai-gateway/websockets-api/non-realtime-api/ 301 +/ai-gateway/configuration/websockets-api/realtime-api/ /ai-gateway/websockets-api/realtime-api/ 301 + # agents /agents/capabilities/mcp-server/ /agents/model-context-protocol/ 301 diff --git a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx index 5c9a63ac90fc26e..09d079f5196baf7 100644 --- a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx +++ b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx @@ -1,37 +1,39 @@ --- -title: AI Gateway launches Realtime WebSockets API -description: AI Gateway now supports end-to-end, client-to-provider WebSockets +title: AI Gateway launches Realtime WebSockets API +description: AI Gateway now supports end-to-end, client-to-provider WebSockets date: 2025-03-21T09:00:00Z --- -We are excited to announce that [AI Gateway](/ai-gateway/) now supports real-time AI interactions with the new [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/). + +We are excited to announce that [AI Gateway](/ai-gateway/) now supports real-time AI interactions with the new [Realtime WebSockets API](/ai-gateway/websockets-api/realtime-api/). This new capability allows developers to establish persistent, low-latency connections between their applications and AI models, enabling natural, real-time conversational AI experiences, including speech-to-speech interactions. The Realtime WebSockets API works with the [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime#connect-with-websockets), [Google Gemini Live API](https://ai.google.dev/gemini-api/docs/multimodal-live), and supports real-time text and speech interactions with models from [Cartesia](https://docs.cartesia.ai/api-reference/tts/tts), and [ElevenLabs](https://elevenlabs.io/docs/conversational-ai/api-reference/conversational-ai/websocket). Here's how you can connect AI Gateway to [OpenAI's Realtime API](https://platform.openai.com/docs/guides/realtime#connect-with-websockets) using WebSockets: + ```javascript title="OpenAI Realtime API example" import WebSocket from "ws"; const url = - "wss://gateway.ai.cloudflare.com/v1///openai?model=gpt-4o-realtime-preview-2024-12-17"; + "wss://gateway.ai.cloudflare.com/v1///openai?model=gpt-4o-realtime-preview-2024-12-17"; const ws = new WebSocket(url, { - headers: { - "cf-aig-authorization": process.env.CLOUDFLARE_API_KEY, - Authorization: "Bearer " + process.env.OPENAI_API_KEY, - "OpenAI-Beta": "realtime=v1", - }, + headers: { + "cf-aig-authorization": process.env.CLOUDFLARE_API_KEY, + Authorization: "Bearer " + process.env.OPENAI_API_KEY, + "OpenAI-Beta": "realtime=v1", + }, }); ws.on("open", () => console.log("Connected to server.")); ws.on("message", (message) => console.log(JSON.parse(message.toString()))); ws.send( - JSON.stringify({ - type: "response.create", - response: { modalities: ["text"], instructions: "Tell me a joke" }, - }), + JSON.stringify({ + type: "response.create", + response: { modalities: ["text"], instructions: "Tell me a joke" }, + }), ); ``` -Get started by checking out the [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/) documentation. +Get started by checking out the [Realtime WebSockets API](/ai-gateway/websockets-api/realtime-api/) documentation. diff --git a/src/content/docs/ai-gateway/get-started.mdx b/src/content/docs/ai-gateway/get-started.mdx index 3aa222748a7ac0f..dae7b7daacf2bb9 100644 --- a/src/content/docs/ai-gateway/get-started.mdx +++ b/src/content/docs/ai-gateway/get-started.mdx @@ -32,9 +32,9 @@ When setting up a new gateway, you can choose between an authenticated and unaut Next, connect your AI provider to your gateway. -AI Gateway offers multiple endpoints for each Gateway you create - one endpoint per provider, and one Universal Endpoint. To use AI Gateway, you will need to create your own account with each provider and provide your API key. AI Gateway acts as a proxy for these requests, enabling observability, caching, and more. +AI Gateway offers multiple endpoints for each Gateway you create - one endpoint per provider, and one Universal Endpoint. To use AI Gateway, you will need to create your own account with each provider and provide your API key. AI Gateway acts as a proxy for these requests, enabling observability, caching, and more. -Additionally, AI Gateway has a [WebSockets API](/ai-gateway/configuration/websockets-api/) which provides a single persistent connection, enabling continuous communication. This API supports all AI providers connected to AI Gateway, including those that do not natively support WebSockets. +Additionally, AI Gateway has a [WebSockets API](/ai-gateway/websockets-api/) which provides a single persistent connection, enabling continuous communication. This API supports all AI providers connected to AI Gateway, including those that do not natively support WebSockets. Below is a list of our supported model providers: diff --git a/src/content/docs/ai-gateway/providers/universal.mdx b/src/content/docs/ai-gateway/providers/universal.mdx index 09fe85e847a76c6..ea3e391552da673 100644 --- a/src/content/docs/ai-gateway/providers/universal.mdx +++ b/src/content/docs/ai-gateway/providers/universal.mdx @@ -31,7 +31,7 @@ The above will send a request to Workers AI Inference API, if it fails it will p ## WebSockets API -The Universal Endpoint can also be accessed via a [WebSockets API](/ai-gateway/configuration/websockets-api/) which provides a single persistent connection, enabling continuous communication. This API supports all AI providers connected to AI Gateway, including those that do not natively support WebSockets. +The Universal Endpoint can also be accessed via a [WebSockets API](/ai-gateway/websockets-api/) which provides a single persistent connection, enabling continuous communication. This API supports all AI providers connected to AI Gateway, including those that do not natively support WebSockets. ## WebSockets example diff --git a/src/content/docs/ai-gateway/websockets-api/index.mdx b/src/content/docs/ai-gateway/websockets-api/index.mdx index 3e1a24e107fc7a2..63f111e3506b331 100644 --- a/src/content/docs/ai-gateway/websockets-api/index.mdx +++ b/src/content/docs/ai-gateway/websockets-api/index.mdx @@ -23,14 +23,14 @@ WebSockets are long-lived TCP connections that enable bi-directional, real-time ## **Key differences** -| Feature | Realtime APIs | Non-Realtime APIs | -| :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------ | :----------------------------------------------------------------------------------------------- | -| **Purpose** | Enables real-time, multimodal AI interactions for providers that offer dedicated WebSocket endpoints. | Supports WebSocket-based AI interactions with providers that do not natively support WebSockets. | -| **Use Case** | Streaming responses for voice, video, and live interactions. | Text-based queries and responses, such as LLM requests. | -| **AI Provider Support** | [Limited to providers offering real-time WebSocket APIs.](/ai-gateway/configuration/websockets-api/realtime-api/#supported-providers) | [All AI providers in AI Gateway.](/ai-gateway/providers) | -| **Streaming Support** | Providers natively support real-time data streaming. | AI Gateway handles streaming via WebSockets. | +| Feature | Realtime APIs | Non-Realtime APIs | +| :---------------------- | :---------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------- | +| **Purpose** | Enables real-time, multimodal AI interactions for providers that offer dedicated WebSocket endpoints. | Supports WebSocket-based AI interactions with providers that do not natively support WebSockets. | +| **Use Case** | Streaming responses for voice, video, and live interactions. | Text-based queries and responses, such as LLM requests. | +| **AI Provider Support** | [Limited to providers offering real-time WebSocket APIs.](/ai-gateway/websockets-api/realtime-api/#supported-providers) | [All AI providers in AI Gateway.](/ai-gateway/providers) | +| **Streaming Support** | Providers natively support real-time data streaming. | AI Gateway handles streaming via WebSockets. | For details on implementation, refer to the next sections: - [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/) -- [Non-Realtime WebSockets API](/ai-gateway/configuration/websockets-api/non-realtime-api/) +- [Non-Realtime WebSockets API](/ai-gateway/websockets-api/non-realtime-api/) From ba6f1667b20e59d682b409a7cae6e3b668df1f6a Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Mon, 28 Apr 2025 23:21:14 +0100 Subject: [PATCH 3/5] invalid links --- src/content/docs/ai-gateway/websockets-api/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/websockets-api/index.mdx b/src/content/docs/ai-gateway/websockets-api/index.mdx index 63f111e3506b331..6d2dbceb38bf5c6 100644 --- a/src/content/docs/ai-gateway/websockets-api/index.mdx +++ b/src/content/docs/ai-gateway/websockets-api/index.mdx @@ -32,5 +32,5 @@ WebSockets are long-lived TCP connections that enable bi-directional, real-time For details on implementation, refer to the next sections: -- [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/) +- [Realtime WebSockets API](/ai-gateway/websockets-api/realtime-api/) - [Non-Realtime WebSockets API](/ai-gateway/websockets-api/non-realtime-api/) From 6ae01ad6dfe4c8c789cedd207833e8b01086bf18 Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Tue, 29 Apr 2025 17:40:24 +0100 Subject: [PATCH 4/5] Update index.mdx --- src/content/docs/ai-gateway/websockets-api/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/websockets-api/index.mdx b/src/content/docs/ai-gateway/websockets-api/index.mdx index 6d2dbceb38bf5c6..899a2dd6cacc6d4 100644 --- a/src/content/docs/ai-gateway/websockets-api/index.mdx +++ b/src/content/docs/ai-gateway/websockets-api/index.mdx @@ -21,7 +21,7 @@ WebSockets are long-lived TCP connections that enable bi-directional, real-time - **Reduced overhead**: Avoid overhead of repeated handshakes and TLS negotiations by maintaining a single, persistent connection. - **Provider compatibility**: Works with all AI providers in AI Gateway. Even if your chosen provider does not support WebSockets, Cloudflare handles it for you, managing the requests to your preferred AI provider. -## **Key differences** +## Key differences | Feature | Realtime APIs | Non-Realtime APIs | | :---------------------- | :---------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------- | From 2445abf03a5a3c67e81bb4578baa30038d379421 Mon Sep 17 00:00:00 2001 From: daisyfaithauma Date: Wed, 7 May 2025 15:35:09 +0100 Subject: [PATCH 5/5] Update src/content/docs/ai-gateway/websockets-api/index.mdx Co-authored-by: Pedro Sousa <680496+pedrosousa@users.noreply.github.com> --- src/content/docs/ai-gateway/websockets-api/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/ai-gateway/websockets-api/index.mdx b/src/content/docs/ai-gateway/websockets-api/index.mdx index 356ba2de98168b7..f43487e47e35ab0 100644 --- a/src/content/docs/ai-gateway/websockets-api/index.mdx +++ b/src/content/docs/ai-gateway/websockets-api/index.mdx @@ -27,7 +27,7 @@ WebSockets are long-lived TCP connections that enable bi-directional, real-time | :---------------------- | :---------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------- | | **Purpose** | Enables real-time, multimodal AI interactions for providers that offer dedicated WebSocket endpoints. | Supports WebSocket-based AI interactions with providers that do not natively support WebSockets. | | **Use Case** | Streaming responses for voice, video, and live interactions. | Text-based queries and responses, such as LLM requests. | -| **AI Provider Support** | [Limited to providers offering real-time WebSocket APIs.](/ai-gateway/websockets-api/realtime-api/#supported-providers) | [All AI providers in AI Gateway.](/ai-gateway/providers) | +| **AI Provider Support** | [Limited to providers offering real-time WebSocket APIs.](/ai-gateway/websockets-api/realtime-api/#supported-providers) | [All AI providers in AI Gateway.](/ai-gateway/providers/) | | **Streaming Support** | Providers natively support real-time data streaming. | AI Gateway handles streaming via WebSockets. | For details on implementation, refer to the next sections: