213 lines
13 KiB
Markdown
213 lines
13 KiB
Markdown
# Technology Stack
|
|
|
|
**Project:** LED Sync Studio
|
|
**Researched:** 2026-04-03
|
|
**Confidence:** MEDIUM (hardware-specific constraints limit definitive verification for some choices)
|
|
|
|
---
|
|
|
|
## Recommended Stack
|
|
|
|
### TUI Layer (Raspberry Pi Host App)
|
|
|
|
| Technology | Version | Purpose | Why |
|
|
|------------|---------|---------|-----|
|
|
| Python | 3.11+ | Runtime | Ships on Raspberry Pi OS bookworm; 3.11 asyncio improvements matter for this use case |
|
|
| Textual | 8.2.1 | TUI framework | Latest stable (March 2026). CSS-based theming with custom color variables maps cleanly to cyberpunk/neon aesthetics. asyncio-native — no threading hacks needed alongside audio callbacks. Runs cleanly over SSH. |
|
|
| Rich | (Textual dependency) | Terminal rendering | Pulled in by Textual; use directly for log panels and progress bars |
|
|
|
|
**Why Textual and not alternatives:**
|
|
- `curses` / `urwid`: No CSS, no async, painful for custom widgets like a timeline ruler
|
|
- `blessed`: Imperative, no widget system, would require building everything from scratch
|
|
- `prompt_toolkit`: Good for REPL-style apps, not layout-based dashboards
|
|
|
|
### Audio Layer
|
|
|
|
| Technology | Version | Purpose | Why |
|
|
|------------|---------|---------|-----|
|
|
| sounddevice | 0.5.x | Real-time audio capture from PipeWire/PulseAudio | Uses PortAudio under the hood; supports asyncio generators for block-by-block audio callbacks; works with PipeWire's PulseAudio compatibility layer without extra config |
|
|
| aubio | 0.4.9 | Beat detection + onset detection | Written in C, Python bindings via NumPy arrays, designed explicitly for causal real-time processing. Configurable hop size controls latency (512 samples at 44.1kHz = ~11ms per frame). |
|
|
| numpy | 1.26+ | Audio buffer manipulation, FFT for spectrum display | Required by aubio; use `numpy.fft.rfft` directly for real-time frequency band analysis without librosa overhead |
|
|
| miniaudio | 1.61 | MP3/FLAC/WAV file decoding and playback with position tracking | Pure C backend, no SDL dependency (unlike pygame), exposes playback position in samples, supports seeking. `just_playback` wraps it but miniaudio directly gives more control. |
|
|
|
|
**Why aubio over librosa for real-time:**
|
|
- librosa is offline-first — designed for batch analysis of whole files, not streaming callbacks
|
|
- aubio's C core processes audio frames in-place with ~11ms latency at 44.1kHz/512-sample hop
|
|
- The `aubio.tempo` object maintains internal beat-tracking state across frames
|
|
|
|
**Why sounddevice over pyaudio:**
|
|
- pyaudio requires ALSA headers to compile and has no asyncio support
|
|
- sounddevice provides `sd.InputStream` with a Python callback that receives NumPy arrays directly
|
|
- PipeWire exposes a PulseAudio-compatible socket on Raspberry Pi OS bookworm — sounddevice connects without special config
|
|
|
|
**Why miniaudio over pygame for playback:**
|
|
- pygame's audio system conflicts with sounddevice when both need ALSA/PulseAudio simultaneously
|
|
- miniaudio exposes exact sample position, enabling precise choreography timeline sync
|
|
- No SDL dependency = no display server requirement (SSH-only environment)
|
|
|
|
### Communication Layer (Pi → ESP32)
|
|
|
|
| Technology | Version | Purpose | Why |
|
|
|------------|---------|---------|-----|
|
|
| asyncio (stdlib) | Python 3.11+ | UDP datagram sending | No external dependency; `loop.create_datagram_endpoint()` gives fire-and-forget UDP with ~1-5ms overhead |
|
|
| UDP (not TCP) | — | Transport protocol | TCP Nagle algorithm adds 40-200ms latency on small packets without tuning; UDP is connectionless and sub-5ms RTT on local WiFi. Commands are idempotent (animation start/stop) so lost packets are acceptable. |
|
|
| JSON (stdlib `json`) | — | Command serialization | Already specified in requirements; human-readable for debugging; ArduinoJson v7 on ESP side handles deserialization efficiently |
|
|
|
|
**Protocol design note:** Keep payload small. `{"cmd":"start","anim":"pulse","zone":0,"color":"#FF00FF","speed":1.5}` fits in a single UDP datagram (MTU safe under 512 bytes). No fragmentation, no connection overhead.
|
|
|
|
**Why not WebSocket/HTTP:**
|
|
- HTTP adds 100-300ms per request (connection setup + headers)
|
|
- WebSocket persistent connection is viable but adds complexity on ESP side and has known ESP32-C3 crash issues with me-no-dev library
|
|
- UDP fire-and-forget matches the use case: beat hits → send command → ESP acts immediately
|
|
|
|
### ESP32-C3 Firmware
|
|
|
|
| Technology | Version | Purpose | Why |
|
|
|------------|---------|---------|-----|
|
|
| PlatformIO | latest | Build system | Far superior to Arduino IDE for this project: proper dependency management, `platformio.ini` reproducible builds, VS Code integration, OTA flash support |
|
|
| Arduino framework for ESP32 | 3.x (espressif32) | Hardware abstraction | Mature, large library ecosystem, good ESP32-C3 support in core 3.x |
|
|
| NeoPixelBus by Makuna | 2.8.x | SK6812 single-wire control via RMT | Uses ESP32-C3 RMT peripheral (channels 0-1 available), hardware-timed — no CPU spin waiting. Handles RGBW SK6812 correctly including white channel. |
|
|
| Adafruit DotStar / raw SPI | — | WS2801 SPI control | WS2801 is a clock+data SPI protocol, not single-wire. Use Arduino `SPI.h` directly or Adafruit DotStar library which supports WS2801 via hardware SPI. NeoPixelBus also supports SPI method for WS2801. |
|
|
| ArduinoJson | 7.4.x | JSON deserialization of commands | Version 7 uses heap-allocated elastic documents — no need to pre-size buffers. Benchmarked at <1ms for typical command payloads on ESP32. |
|
|
| AsyncUDP (ESP32 Arduino stdlib) | — | UDP packet reception | Built into arduino-esp32 core, no external dependency, runs in background task. More stable on C3 than ESPAsyncWebServer HTTP. |
|
|
|
|
**LED library strategy — two strips, two protocols:**
|
|
|
|
The ESP32-C3 SuperMini has 2 RMT channels and hardware SPI. Use them independently:
|
|
- **SK6812 (Wand, 300 LEDs):** NeoPixelBus with `NeoEsp32RmtMethodBase` on GPIO4 (RMT channel 0)
|
|
- **WS2801 (Schrank, 160 LEDs):** Hardware SPI via `SPI.h` on GPIO6 (MOSI) + GPIO4 (SCK) — but check GPIO conflicts with SK6812 RMT pin assignment. Use GPIO pin mapping carefully: avoid GPIO2, GPIO8, GPIO9 (boot-critical), GPIO12-17 (internal flash).
|
|
|
|
**Concrete pin assignment to validate on hardware:**
|
|
- SK6812 data: GPIO3 (RMT channel 0)
|
|
- WS2801 MOSI: GPIO7, WS2801 CLK: GPIO6 (hardware SPI bus 1)
|
|
- Leave GPIO8 (onboard LED/WS2812) unused to avoid conflicts
|
|
|
|
**Why not WLED:**
|
|
- WLED is a complete opinionated firmware — cannot receive custom JSON animation commands
|
|
- Animation logic must run on ESP32 with own state machine; WLED's architecture doesn't expose low-level animation API
|
|
- Building custom firmware is the right call here
|
|
|
|
**Why not me-no-dev ESPAsyncWebServer:**
|
|
- Archived January 2025, read-only
|
|
- Known crash bugs on ESP32-C3 single-core with WebSocket connections
|
|
- Successor: ESP32Async/ESPAsyncWebServer — but unnecessary complexity for this project. Plain AsyncUDP is simpler and lower latency.
|
|
|
|
### Data Storage (Choreography Files)
|
|
|
|
| Technology | Version | Purpose | Why |
|
|
|------------|---------|---------|-----|
|
|
| JSON (stdlib) | — | Choreography file format | Human-readable, easily version-controlled, Python `json` module needs zero deps. YAML adds a dependency and parsing overhead with no benefit for machine-generated files. |
|
|
| Pydantic | 2.x | Schema validation for choreography files | V2 is Rust-backed, fast. Validates animation parameter ranges, timeline consistency. Generates JSON schemas for free. Worth the dependency for data integrity. |
|
|
|
|
### Development Tooling
|
|
|
|
| Technology | Purpose |
|
|
|------------|---------|
|
|
| uv | Python package manager — significantly faster than pip, proper lockfiles, works on Raspberry Pi ARM |
|
|
| PlatformIO | ESP32 firmware build, dependency resolution, OTA upload |
|
|
| pytest + pytest-asyncio | Test suite for choreography engine and protocol logic |
|
|
|
|
---
|
|
|
|
## Alternatives Considered
|
|
|
|
| Category | Recommended | Alternative | Why Not |
|
|
|----------|-------------|-------------|---------|
|
|
| TUI | Textual 8.2.1 | curses, urwid, blessed | No CSS theming, no asyncio integration, verbose widget construction |
|
|
| Beat detection | aubio | librosa | librosa is offline/batch, not real-time streaming; adds heavy scipy/sklearn deps |
|
|
| Beat detection | aubio | essentia | Essentia is ML-heavy, not pip-installable cleanly on Raspberry Pi ARM without build hassles |
|
|
| Audio capture | sounddevice | pyaudio | pyaudio needs ALSA headers, no asyncio, harder to pip install |
|
|
| Audio playback | miniaudio | pygame | pygame conflicts with sounddevice for audio device access; no playback position API |
|
|
| Audio playback | miniaudio | gstreamer | gstreamer Python bindings (gi.repository) are fragile, heavyweight |
|
|
| Transport | UDP | HTTP/REST | HTTP adds 100-300ms latency per command, unacceptable for beat-sync |
|
|
| Transport | UDP | WebSocket | ESP32-C3 WebSocket stability issues; more complex firmware; no benefit for one-directional command stream |
|
|
| LED (SK6812) | NeoPixelBus (RMT) | Adafruit NeoPixel | NeoPixelBus RMT is hardware-timed and interrupt-free; Adafruit NeoPixel disables interrupts during transfer, causing WiFi packet drops |
|
|
| Firmware build | PlatformIO | Arduino IDE | Arduino IDE has no proper dependency management, no CLI, no lockfiles |
|
|
| Choreography format | JSON | YAML | Zero extra dependency for JSON; no user-facing config editing needed |
|
|
| Config validation | Pydantic v2 | dataclasses | Pydantic gives JSON schema, field validators, and serialization; dataclasses don't |
|
|
|
|
---
|
|
|
|
## Installation
|
|
|
|
### Raspberry Pi Host App
|
|
|
|
```bash
|
|
# Install uv (fast Python package manager)
|
|
curl -LsSf https://astral.sh/uv/install.sh | sh
|
|
|
|
# Create project environment
|
|
uv init led-sync-studio
|
|
cd led-sync-studio
|
|
uv add textual==8.2.1
|
|
uv add sounddevice miniaudio aubio numpy pydantic
|
|
|
|
# System dependency for sounddevice (PortAudio)
|
|
sudo apt install -y libportaudio2
|
|
```
|
|
|
|
### ESP32-C3 Firmware (PlatformIO)
|
|
|
|
```ini
|
|
; platformio.ini
|
|
[env:esp32c3_supermini]
|
|
platform = espressif32
|
|
board = esp32-c3-devkitm-1
|
|
board_build.mcu = esp32c3
|
|
framework = arduino
|
|
build_flags =
|
|
-DARDUINO_USB_CDC_ON_BOOT=1
|
|
-DARDUINO_USB_MODE=1
|
|
monitor_speed = 115200
|
|
|
|
lib_deps =
|
|
makuna/NeoPixelBus @ ^2.8.0
|
|
bblanchon/ArduinoJson @ ^7.4.0
|
|
```
|
|
|
|
WS2801 via hardware SPI uses only Arduino core `SPI.h` — no external library needed.
|
|
|
|
---
|
|
|
|
## Confidence Assessment
|
|
|
|
| Component | Confidence | Notes |
|
|
|-----------|------------|-------|
|
|
| Textual 8.2.1 | HIGH | Verified via GitHub releases page |
|
|
| aubio for real-time | HIGH | Official docs confirm causal/streaming design |
|
|
| sounddevice + PipeWire | MEDIUM | PipeWire PulseAudio compat layer works in practice; not officially documented for sounddevice |
|
|
| miniaudio for playback | MEDIUM | Position tracking confirmed in docs; Raspberry Pi ARM support confirmed via PyPI |
|
|
| UDP over TCP | HIGH | Latency characteristics well-documented; ESP32 TCP Nagle issue confirmed in arduino-esp32 issues |
|
|
| NeoPixelBus RMT for SK6812 | HIGH | RMT method confirmed for ESP32-C3; RGBW SK6812 support documented |
|
|
| WS2801 via SPI | MEDIUM | SPI method works in NeoPixelBus or raw SPI; GPIO pin assignments need hardware verification |
|
|
| ArduinoJson 7.4.x | HIGH | Latest version confirmed via ESP Component Registry |
|
|
| ESP32-C3 GPIO constraints | MEDIUM | SuperMini pinout documented; specific GPIO conflicts need hardware validation |
|
|
| PlatformIO build | HIGH | Well-documented, community-confirmed for ESP32-C3 |
|
|
|
|
---
|
|
|
|
## Critical Constraints to Validate Early
|
|
|
|
1. **GPIO conflict: SK6812 RMT + WS2801 SPI on ESP32-C3 SuperMini** — The SuperMini has limited broken-out pins. Verify that chosen RMT GPIO and SPI GPIO for simultaneous operation don't conflict. NeoPixelBus RMT does not block SPI, so hardware parallelism is possible.
|
|
|
|
2. **sounddevice + miniaudio simultaneous device access** — Both will request the default PulseAudio/PipeWire sink. On Raspberry Pi OS bookworm (PipeWire default), multiple readers on a loopback monitor source should work, but test early.
|
|
|
|
3. **aubio version on Raspberry Pi ARM** — aubio 0.4.9 is the last stable release (2019). The pip wheel for ARM may not exist; may need to compile from source. Test `pip install aubio` on target Pi before committing.
|
|
|
|
4. **WiFi latency floor** — UDP on local WiFi to ESP32 has measured RTT of 20-60ms depending on router/environment. Commands are fire-and-forget so one-way latency is 10-30ms. Musical sync at 120BPM has 500ms between beats — 30ms latency is well within tolerance for beat-level sync, but not sub-beat.
|
|
|
|
---
|
|
|
|
## Sources
|
|
|
|
- Textual releases: https://github.com/Textualize/textual/releases
|
|
- aubio real-time design: https://aubio.org/doc/ — https://github.com/zak-45/WLEDAudioSyncRTBeat
|
|
- sounddevice docs: https://python-sounddevice.readthedocs.io/
|
|
- miniaudio Python: https://github.com/irmen/pyminiaudio
|
|
- NeoPixelBus ESP32-C3 RMT: https://github.com/Makuna/NeoPixelBus/issues/477
|
|
- ArduinoJson 7.4.x: https://components.espressif.com/components/bblanchon/arduinojson/versions/7.4.2
|
|
- ESPAsyncWebServer archived + successor: https://github.com/mathieucarbou/ESPAsyncWebServer
|
|
- ESP32-C3 SuperMini pinout: https://www.espboards.dev/esp32/esp32-c3-super-mini/
|
|
- PlatformIO ESP32-C3: https://docs.platformio.org/en/latest/boards/espressif32/esp32-c3-devkitm-1.html
|
|
- UDP latency on ESP32: https://github.com/espressif/arduino-esp32/issues/1283
|