Skip to content

Commit bb9a112

Browse files
committed
Update readme
1 parent 0595b3b commit bb9a112

File tree

1 file changed

+47
-44
lines changed

1 file changed

+47
-44
lines changed

README.md

Lines changed: 47 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Speech2Touch 🗣️👆
22

3-
[![Build Status](https://img.shields.io/badge/build-passing-brightgreen)](https://github.com/edholmes2232/Speech2Touch/actions)
3+
[![Build Firmware](https://github.com/edholmes2232/Speech2Touch/actions/workflows/build-firmware.yml/badge.svg)](https://github.com/edholmes2232/Speech2Touch/actions/workflows/build-firmware.yml)
44
[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE.md)
55
[![Test Coverage](https://img.shields.io/badge/tests-pending-lightgrey)](https://github.com/edholmes2232/Speech2Touch/actions)
66

@@ -12,12 +12,17 @@ Based on STM32WB55, it leverages [Picovoice](https://picovoice.ai/) to process s
1212

1313
The INMP441 MEMS microphone is used for voice input.
1414

15+
**Pre-built firmware:** [Download latest](https://github.com/edholmes2232/Speech2Touch/actions/workflows/build-firmware.yml) (select latest successful run → Artifacts)
16+
17+
**Coverage:** This project has been written about on [Hackaday](https://hackaday.com/2025/09/24/coffee-by-command-the-speech2touch-voice-hack/) and [Hackster.io](https://www.hackster.io/news/espresso-yourself-with-your-voice-b83a2757f170).
18+
1519
---
1620

1721
## ☕️ Demo
1822

1923
https://github.com/user-attachments/assets/7025197e-daeb-4745-9b6d-d1ec124fa88a
2024

25+
---
2126

2227
## 🤖 Prototype Hardware
2328

@@ -27,6 +32,7 @@ https://github.com/user-attachments/assets/7025197e-daeb-4745-9b6d-d1ec124fa88a
2732

2833
The shape and orientation of the protoboard were dictated by the position of the USB ports of the Franke A600.
2934

35+
---
3036

3137
## 🏭 Hardware-In-Loop (HIL) Test
3238

@@ -40,47 +46,57 @@ https://github.com/user-attachments/assets/d8d1ce1c-74fb-45fa-a442-dd6b9ee583c0
4046
- STM32WB55 USB Dongle dev board
4147
- INMP441 microphone
4248
- Franke A600 (or compatible) touchscreen device
43-
- QT (for HIL testing)
44-
- VSCode (for debugging)
45-
- [STM32Cube for Visual Studio Code](https://www.st.com/content/st_com/en/stm32-mcu-developer-zone/software-development-tools/stm32cubevscode.html) extension
49+
- Qt (for HIL testing)
50+
- VSCode with [STM32Cube extension](https://www.st.com/content/st_com/en/stm32-mcu-developer-zone/software-development-tools/stm32cubevscode.html)
51+
- See `Dockerfile` for toolchain and package requirements
4652

4753
### 🚀 Container Build & Flash (Recommended)
48-
Coming soon...
54+
55+
1. Open in VSCode and reopen in dev container (`F1` → "Dev Containers: Reopen in Container")
56+
2. Configure and build:
57+
```bash
58+
cmake -DCMAKE_BUILD_TYPE=Release \
59+
-DCMAKE_TOOLCHAIN_FILE=/workspaces/Speech2Touch/cmake/gcc-arm-none-eabi.cmake \
60+
-S /workspaces/Speech2Touch -B /workspaces/Speech2Touch/build/Release -G Ninja
61+
cmake --build /workspaces/Speech2Touch/build/Release --target all
62+
```
63+
3. Flash `build/Release/Speech2Touch.bin` to your STM32WB55
4964

5065
### 🛠️ Manual Build & Flash
5166

52-
1. Clone this repository.
53-
2. Set up the project in VSCode using the STM32Cube extension.
54-
3. Build and flash the firmware from VSCode menus.
55-
4. Connect the device to the coffee machine via USB.
67+
1. Clone this repository
68+
2. Set up the project in VSCode using the STM32Cube extension
69+
3. Build and flash the firmware from VSCode
70+
4. Connect the device to the coffee machine via USB
5671

5772
### 🧪 HIL Testing
5873

59-
The Hardware-In-Loop test creates a QT GUI window which emulates the position of touch targets to match the Franke A600. It utilizes Linux text-to-speech utilities to trigger the device, and tests that the correct corresponding touch target is triggered.
74+
The Hardware-In-Loop test suite uses a Qt GUI to emulate the Franke A600 touchscreen layout. It leverages Linux text-to-speech utilities to trigger the device and validates that commands activate the correct touch targets.
6075

61-
1. Build the QT test suite:
62-
```
63-
$ cmake -DCMAKE_BUILD_TYPE=Test -S Speech2Touch -B Speech2Touch/build/Test -G Ninja
64-
$ cmake --build /home/ed/Projects/Speech2Touch/build/Test --target all --
65-
```
66-
2. Connect the embedded device with the latest firmware to the host PC USB port.
67-
3. Use `dmesg` to find the `/dev/input/eventX` USB input device path.
68-
4. Run automated test:
76+
1. Build the Qt test suite:
77+
```bash
78+
cmake -DCMAKE_BUILD_TYPE=Test -S Speech2Touch -B Speech2Touch/build/Test -G Ninja
79+
cmake --build Speech2Touch/build/Test --target all
6980
```
70-
$ ./build/Test/Test/hil/runner/test_full_loop --input /dev/input/event10
81+
2. Connect the embedded device with the latest firmware to the host PC USB port
82+
3. Use `dmesg` to identify the `/dev/input/eventX` USB input device path
83+
4. Run the automated test:
84+
```bash
85+
./build/Test/Test/hil/runner/test_full_loop --input /dev/input/event10
7186
```
7287

7388
---
7489

7590
## 🏗️ Architecture Overview
7691

7792
```
78-
[INMP441 microphone] → [Picovoice Speech Recognition] → [STM32WB55 MCU] → [Custom USB HID] → [Touchscreen Device]
93+
[INMP441 Microphone] → [Picovoice Speech Recognition] → [STM32WB55 MCU] → [Custom USB HID] → [Touchscreen Device]
7994
```
80-
- **Input:** Microphone captures user speech.
81-
- **Processing:** Picovoice library processes audio and extracts commands.
82-
- **Translation:** Commands are mapped to touchscreen coordinates.
83-
- **Output:** Custom USB HID packets simulate touch events on the target device.
95+
96+
- **Input:** INMP441 microphone captures audio
97+
- **Processing:** Picovoice library performs speech recognition and command extraction
98+
- **Translation:** Commands are mapped to touchscreen coordinates
99+
- **Output:** Custom USB HID packets simulate touch events
84100

85101
### 🧵 Threading
86102

@@ -100,26 +116,14 @@ sequenceDiagram
100116
end
101117
102118
activate Speech
103-
Speech->>Speech: Speech Regognition
104-
Speech->>Speech: Convert to target co-ords
119+
Speech->>Speech: Speech Recognition
120+
Speech->>Speech: Convert to Target Coords
105121
deactivate Speech
106122
107123
Speech->>Touch: Touch Coordinates
108124
Touch->>USB: USB HID Report
109-
110125
```
111126

112-
113-
114-
115-
---
116-
117-
## 🔬 HIL Testing Suite
118-
119-
- **QT-based GUI** replicates the Franke A600 touchscreen.
120-
- **Automated tests** ensure voice commands map to correct on-screen buttons.
121-
- **Continuous integration** ready.
122-
123127
---
124128

125129
## 🔮 Extending
@@ -139,14 +143,13 @@ The Picovoice precompiled binary at `Core/Lib/picovoice/libpicovoice.a` is pulle
139143
The configuration files in `Core/Lib/picovoice/include` are specifically set up for a Franke A600, including using "Franke" as the wake-word. New configuration files can be generated from the [Picovoice Console](https://console.picovoice.ai/).
140144

141145
---
142-
## ☑️ To Do
143-
- Create .devcontainer for firmware builds within a container.
144-
- Replace Dev Board + Protoboard with a PCB.
146+
147+
## ☑️ Roadmap
148+
149+
- Replace Dev Board + Protoboard with a PCB
145150
- Unit testing.
146-
- CI/CD with GitHub Actions for generating firmware, running unit tests.
147151
- Extract audio over RTT for tuning.
148-
- Decouple Franke A600 specific functionality for easier adapting of Speech2Touch to different applications.
149-
152+
- Decouple Franke A600-specific functionality for easier adapting of Speech2Touch to different applications.
150153

151154
---
152155

0 commit comments

Comments
 (0)