1
- < img src = " arduino_ai_speech_assets/elato-alien.png " alt = " Elato Logo " width = " 100% " >
1
+ ![ Elato Logo ] ( https://raw.githubusercontent.com/openai/openai-cookbook/refs/heads/main/examples/voice_solutions/ arduino_ai_speech_assets/elato-alien.png)
2
2
3
- # 👾 ElatoAI: Running OpenAI Realtime API Speech on ESP32 on Arduino with Deno Edge Functions
3
+ ## 👾 ElatoAI: Running OpenAI Realtime API Speech on ESP32 on Arduino with Deno Edge Functions
4
4
5
5
This guide shows how to build a AI voice agent device with Realtime AI Speech powered by OpenAI Realtime API, ESP32, Secure WebSockets, and Deno Edge Functions for >10-minute uninterrupted global conversations.
6
6
7
7
An active version of this README is available at [ ElatoAI] ( https://github.com/akdeb/ElatoAI ) .
8
8
9
9
<div align =" center " >
10
-
11
- [ ![ Discord Follow] ( https://dcbadge.vercel.app/api/server/KJWxDPBRUj?style=flat )] ( https://discord.gg/KJWxDPBRUj )
12
- [ ![ License: MIT] ( https://img.shields.io/badge/license-MIT-blue )] ( https://www.gnu.org/licenses/gpl-3.0.en.html ) &ensp ;&ensp ;&ensp ;
13
- ![ Node.js] ( https://img.shields.io/badge/Node.js-22.13.0-yellow.svg )
14
- ![ Next.js] ( https://img.shields.io/badge/Next.js-14.2.7-brightgreen.svg )
15
- ![ React] ( https://img.shields.io/badge/React-18.2.0-blue.svg )
16
-
17
- </div >
18
-
19
- ## Demo Video
20
-
21
- https://github.com/user-attachments/assets/aa60e54c-5847-4a68-80b5-5d6b1a5b9328
22
-
23
- <div align =" center " >
24
- <a href =" https://www.youtube.com/watch?v=o1eIAwVll5I " >
25
- <img src =" https://img.shields.io/badge/Watch%20Demo-YouTube-red?style=for-the-badge&logo=youtube " alt =" Watch Demo on YouTube " >
26
- </a >
10
+ <a href="https://www.youtube.com/watch?v=o1eIAwVll5I" target="_blank">
11
+ <img src="https://raw.githubusercontent.com/akdeb/ElatoAI/refs/heads/main/assets/thumbnail.png" alt="Elato AI Demo Video" width="100%" style="border-radius:10px" />
12
+ </a >
27
13
</div >
28
14
29
- ## Hardware Design
15
+ ## ⚡️ DIY Hardware Design
30
16
31
17
The reference implementation uses an ESP32-S3 microcontroller with minimal additional components:
32
18
33
- <img src =" arduino_ai_speech_assets/pcb-design.png " alt =" Hardware Setup " width =" 100% " >
19
+ <img src =" https://raw.githubusercontent.com/openai/openai-cookbook/refs/heads/main/examples/voice_solutions/ arduino_ai_speech_assets/pcb-design.png" alt =" Hardware Setup " width =" 100% " >
34
20
35
21
** Required Components:**
36
22
- ESP32-S3 development board
@@ -40,13 +26,23 @@ The reference implementation uses an ESP32-S3 microcontroller with minimal addit
40
26
- RGB LED for visual feedback
41
27
- Optional: touch sensor for alternative control
42
28
43
- ** Optional hardware :**
29
+ ** Hardware options :**
44
30
A fully assembled PCB and device is available in the [ ElatoAI store] ( https://www.elatoai.com/products ) .
45
31
46
- ## 🚀 Quick Start Guide
32
+ ## 📱 App Design
33
+
34
+ Control your ESP32 AI device from your phone with your own webapp.
35
+
36
+ <img src =" https://raw.githubusercontent.com/openai/openai-cookbook/refs/heads/main/examples/voice_solutions/arduino_ai_speech_assets/mockups.png " alt =" App Screenshots " width =" 100% " >
37
+
38
+ | Select from a list of AI characters | Talk to your AI with real-time responses | Create personalized AI characters |
39
+ | :--:| :--:| :--:|
40
+
41
+
42
+ ## ✨ Quick Start Tutorial
47
43
48
44
<a href =" https://www.youtube.com/watch?v=bXrNRpGOJWw " >
49
- <img src =" https://img.shields.io/badge/Quickstart %20Tutorial-YouTube-yellow?style=for-the-badge&logo=youtube " alt =" Watch Demo on YouTube " >
45
+ <img src =" https://img.shields.io/badge/Quick%20start %20Tutorial-YouTube-yellow?style=for-the-badge&logo=youtube " alt =" Watch Demo on YouTube " >
50
46
</a >
51
47
52
48
1 . ** Clone the repository**
@@ -192,35 +188,13 @@ We have a [Usecases.md](https://github.com/akdeb/ElatoAI/tree/main/Usecases.md)
192
188
193
189
## 🗺️ High-Level Flow
194
190
195
- ``` mermaid
196
- flowchart TD
197
- User[User Speech] --> ESP32
198
- ESP32[ESP32 Device] -->|WebSocket| Edge[Deno Edge Function]
199
- Edge -->|OpenAI API| OpenAI[OpenAI Realtime API]
200
- OpenAI --> Edge
201
- Edge -->|WebSocket| ESP32
202
- ESP32 --> User[AI Generated Speech]
203
- ```
191
+ <img src =" https://raw.githubusercontent.com/openai/openai-cookbook/refs/heads/main/examples/voice_solutions/arduino_ai_speech_assets/flowchart.png " alt =" App Screenshots " width =" 100% " >
204
192
205
193
## Project Structure
206
194
207
- ``` mermaid
208
- graph TD
209
- repo[ElatoAI]
210
- repo --> frontend[Frontend Vercel NextJS]
211
- repo --> deno[Deno Edge Function]
212
- repo --> esp32[ESP32 Arduino Client]
213
- deno --> supabase[Supabase DB]
214
-
215
- frontend --> supabase
216
- esp32 --> websockets[Secure WebSockets]
217
- esp32 --> opus[Opus Codec]
218
- esp32 --> audio_tools[arduino-audio-tools]
219
- esp32 --> libopus[arduino-libopus]
220
- esp32 --> ESPAsyncWebServer[ESPAsyncWebServer]
221
- ```
195
+ <img src =" https://raw.githubusercontent.com/openai/openai-cookbook/refs/heads/main/examples/voice_solutions/arduino_ai_speech_assets/structure.png " alt =" App Screenshots " width =" 100% " >
222
196
223
- ## ⚙️ PlatformIO Configuration
197
+ ## ⚙️ PlatformIO Config
224
198
225
199
``` ini
226
200
[env:esp32-s3-devkitc-1]
@@ -264,4 +238,4 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
264
238
265
239
---
266
240
267
- ** If you find this project interesting or useful, drop a GitHub ⭐️ at [ ElatoAI] ( https://github.com/akdeb/ElatoAI ) . It helps a lot !**
241
+ ** This example is part of the [ OpenAI Cookbook ] ( https://github.com/openai/openai-cookbook ) . For the full project and latest updates, check out [ ElatoAI] ( https://github.com/akdeb/ElatoAI ) and consider giving it a ⭐️ if you find it useful !**
0 commit comments