Gitea-Admin/led-sync-studio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: research complete

Browse Source
This commit is contained in:
Claude
2026-04-03 12:14:03 +02:00
parent 136a6c877d
commit c3680306ee
5 changed files with 1296 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

212
.planning/research/STACK.md Normal file
View File

@@ -0,0 +1,212 @@
# Technology Stack
**Project:** LED Sync Studio
**Researched:** 2026-04-03
**Confidence:** MEDIUM (hardware-specific constraints limit definitive verification for some choices)
---
## Recommended Stack
### TUI Layer (Raspberry Pi Host App)
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| Python | 3.11+ | Runtime | Ships on Raspberry Pi OS bookworm; 3.11 asyncio improvements matter for this use case |
| Textual | 8.2.1 | TUI framework | Latest stable (March 2026). CSS-based theming with custom color variables maps cleanly to cyberpunk/neon aesthetics. asyncio-native — no threading hacks needed alongside audio callbacks. Runs cleanly over SSH. |
| Rich | (Textual dependency) | Terminal rendering | Pulled in by Textual; use directly for log panels and progress bars |
**Why Textual and not alternatives:**
- `curses` / `urwid`: No CSS, no async, painful for custom widgets like a timeline ruler
- `blessed`: Imperative, no widget system, would require building everything from scratch
- `prompt_toolkit`: Good for REPL-style apps, not layout-based dashboards
### Audio Layer
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| sounddevice | 0.5.x | Real-time audio capture from PipeWire/PulseAudio | Uses PortAudio under the hood; supports asyncio generators for block-by-block audio callbacks; works with PipeWire's PulseAudio compatibility layer without extra config |
| aubio | 0.4.9 | Beat detection + onset detection | Written in C, Python bindings via NumPy arrays, designed explicitly for causal real-time processing. Configurable hop size controls latency (512 samples at 44.1kHz = ~11ms per frame). |
| numpy | 1.26+ | Audio buffer manipulation, FFT for spectrum display | Required by aubio; use `numpy.fft.rfft` directly for real-time frequency band analysis without librosa overhead |
| miniaudio | 1.61 | MP3/FLAC/WAV file decoding and playback with position tracking | Pure C backend, no SDL dependency (unlike pygame), exposes playback position in samples, supports seeking. `just_playback` wraps it but miniaudio directly gives more control. |
**Why aubio over librosa for real-time:**
- librosa is offline-first — designed for batch analysis of whole files, not streaming callbacks
- aubio's C core processes audio frames in-place with ~11ms latency at 44.1kHz/512-sample hop
- The `aubio.tempo` object maintains internal beat-tracking state across frames
**Why sounddevice over pyaudio:**
- pyaudio requires ALSA headers to compile and has no asyncio support
- sounddevice provides `sd.InputStream` with a Python callback that receives NumPy arrays directly
- PipeWire exposes a PulseAudio-compatible socket on Raspberry Pi OS bookworm — sounddevice connects without special config
**Why miniaudio over pygame for playback:**
- pygame's audio system conflicts with sounddevice when both need ALSA/PulseAudio simultaneously
- miniaudio exposes exact sample position, enabling precise choreography timeline sync
- No SDL dependency = no display server requirement (SSH-only environment)
### Communication Layer (Pi → ESP32)
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| asyncio (stdlib) | Python 3.11+ | UDP datagram sending | No external dependency; `loop.create_datagram_endpoint()` gives fire-and-forget UDP with ~1-5ms overhead |
| UDP (not TCP) | — | Transport protocol | TCP Nagle algorithm adds 40-200ms latency on small packets without tuning; UDP is connectionless and sub-5ms RTT on local WiFi. Commands are idempotent (animation start/stop) so lost packets are acceptable. |
| JSON (stdlib `json`) | — | Command serialization | Already specified in requirements; human-readable for debugging; ArduinoJson v7 on ESP side handles deserialization efficiently |
**Protocol design note:** Keep payload small. `{"cmd":"start","anim":"pulse","zone":0,"color":"#FF00FF","speed":1.5}` fits in a single UDP datagram (MTU safe under 512 bytes). No fragmentation, no connection overhead.
**Why not WebSocket/HTTP:**
- HTTP adds 100-300ms per request (connection setup + headers)
- WebSocket persistent connection is viable but adds complexity on ESP side and has known ESP32-C3 crash issues with me-no-dev library
- UDP fire-and-forget matches the use case: beat hits → send command → ESP acts immediately
### ESP32-C3 Firmware
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| PlatformIO | latest | Build system | Far superior to Arduino IDE for this project: proper dependency management, `platformio.ini` reproducible builds, VS Code integration, OTA flash support |
| Arduino framework for ESP32 | 3.x (espressif32) | Hardware abstraction | Mature, large library ecosystem, good ESP32-C3 support in core 3.x |
| NeoPixelBus by Makuna | 2.8.x | SK6812 single-wire control via RMT | Uses ESP32-C3 RMT peripheral (channels 0-1 available), hardware-timed — no CPU spin waiting. Handles RGBW SK6812 correctly including white channel. |
| Adafruit DotStar / raw SPI | — | WS2801 SPI control | WS2801 is a clock+data SPI protocol, not single-wire. Use Arduino `SPI.h` directly or Adafruit DotStar library which supports WS2801 via hardware SPI. NeoPixelBus also supports SPI method for WS2801. |
| ArduinoJson | 7.4.x | JSON deserialization of commands | Version 7 uses heap-allocated elastic documents — no need to pre-size buffers. Benchmarked at <1ms for typical command payloads on ESP32. |
| AsyncUDP (ESP32 Arduino stdlib) | — | UDP packet reception | Built into arduino-esp32 core, no external dependency, runs in background task. More stable on C3 than ESPAsyncWebServer HTTP. |
**LED library strategy — two strips, two protocols:**
The ESP32-C3 SuperMini has 2 RMT channels and hardware SPI. Use them independently:
- **SK6812 (Wand, 300 LEDs):** NeoPixelBus with `NeoEsp32RmtMethodBase` on GPIO4 (RMT channel 0)
- **WS2801 (Schrank, 160 LEDs):** Hardware SPI via `SPI.h` on GPIO6 (MOSI) + GPIO4 (SCK) — but check GPIO conflicts with SK6812 RMT pin assignment. Use GPIO pin mapping carefully: avoid GPIO2, GPIO8, GPIO9 (boot-critical), GPIO12-17 (internal flash).
**Concrete pin assignment to validate on hardware:**
- SK6812 data: GPIO3 (RMT channel 0)
- WS2801 MOSI: GPIO7, WS2801 CLK: GPIO6 (hardware SPI bus 1)
- Leave GPIO8 (onboard LED/WS2812) unused to avoid conflicts
**Why not WLED:**
- WLED is a complete opinionated firmware — cannot receive custom JSON animation commands
- Animation logic must run on ESP32 with own state machine; WLED's architecture doesn't expose low-level animation API
- Building custom firmware is the right call here
**Why not me-no-dev ESPAsyncWebServer:**
- Archived January 2025, read-only
- Known crash bugs on ESP32-C3 single-core with WebSocket connections
- Successor: ESP32Async/ESPAsyncWebServer — but unnecessary complexity for this project. Plain AsyncUDP is simpler and lower latency.
### Data Storage (Choreography Files)
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| JSON (stdlib) | — | Choreography file format | Human-readable, easily version-controlled, Python `json` module needs zero deps. YAML adds a dependency and parsing overhead with no benefit for machine-generated files. |
| Pydantic | 2.x | Schema validation for choreography files | V2 is Rust-backed, fast. Validates animation parameter ranges, timeline consistency. Generates JSON schemas for free. Worth the dependency for data integrity. |
### Development Tooling
| Technology | Purpose |
|------------|---------|
| uv | Python package manager — significantly faster than pip, proper lockfiles, works on Raspberry Pi ARM |
| PlatformIO | ESP32 firmware build, dependency resolution, OTA upload |
| pytest + pytest-asyncio | Test suite for choreography engine and protocol logic |
---
## Alternatives Considered
| Category | Recommended | Alternative | Why Not |
|----------|-------------|-------------|---------|
| TUI | Textual 8.2.1 | curses, urwid, blessed | No CSS theming, no asyncio integration, verbose widget construction |
| Beat detection | aubio | librosa | librosa is offline/batch, not real-time streaming; adds heavy scipy/sklearn deps |
| Beat detection | aubio | essentia | Essentia is ML-heavy, not pip-installable cleanly on Raspberry Pi ARM without build hassles |
| Audio capture | sounddevice | pyaudio | pyaudio needs ALSA headers, no asyncio, harder to pip install |
| Audio playback | miniaudio | pygame | pygame conflicts with sounddevice for audio device access; no playback position API |
| Audio playback | miniaudio | gstreamer | gstreamer Python bindings (gi.repository) are fragile, heavyweight |
| Transport | UDP | HTTP/REST | HTTP adds 100-300ms latency per command, unacceptable for beat-sync |
| Transport | UDP | WebSocket | ESP32-C3 WebSocket stability issues; more complex firmware; no benefit for one-directional command stream |
| LED (SK6812) | NeoPixelBus (RMT) | Adafruit NeoPixel | NeoPixelBus RMT is hardware-timed and interrupt-free; Adafruit NeoPixel disables interrupts during transfer, causing WiFi packet drops |
| Firmware build | PlatformIO | Arduino IDE | Arduino IDE has no proper dependency management, no CLI, no lockfiles |
| Choreography format | JSON | YAML | Zero extra dependency for JSON; no user-facing config editing needed |
| Config validation | Pydantic v2 | dataclasses | Pydantic gives JSON schema, field validators, and serialization; dataclasses don't |
---
## Installation
### Raspberry Pi Host App
```bash
# Install uv (fast Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh
# Create project environment
uv init led-sync-studio
cd led-sync-studio
uv add textual==8.2.1
uv add sounddevice miniaudio aubio numpy pydantic
# System dependency for sounddevice (PortAudio)
sudo apt install -y libportaudio2
```
### ESP32-C3 Firmware (PlatformIO)
```ini
; platformio.ini
[env:esp32c3_supermini]
platform = espressif32
board = esp32-c3-devkitm-1
board_build.mcu = esp32c3
framework = arduino
build_flags =
-DARDUINO_USB_CDC_ON_BOOT=1
-DARDUINO_USB_MODE=1
monitor_speed = 115200
lib_deps =
makuna/NeoPixelBus @ ^2.8.0
bblanchon/ArduinoJson @ ^7.4.0
```
WS2801 via hardware SPI uses only Arduino core `SPI.h` — no external library needed.
---
## Confidence Assessment
| Component | Confidence | Notes |
|-----------|------------|-------|
| Textual 8.2.1 | HIGH | Verified via GitHub releases page |
| aubio for real-time | HIGH | Official docs confirm causal/streaming design |
| sounddevice + PipeWire | MEDIUM | PipeWire PulseAudio compat layer works in practice; not officially documented for sounddevice |
| miniaudio for playback | MEDIUM | Position tracking confirmed in docs; Raspberry Pi ARM support confirmed via PyPI |
| UDP over TCP | HIGH | Latency characteristics well-documented; ESP32 TCP Nagle issue confirmed in arduino-esp32 issues |
| NeoPixelBus RMT for SK6812 | HIGH | RMT method confirmed for ESP32-C3; RGBW SK6812 support documented |
| WS2801 via SPI | MEDIUM | SPI method works in NeoPixelBus or raw SPI; GPIO pin assignments need hardware verification |
| ArduinoJson 7.4.x | HIGH | Latest version confirmed via ESP Component Registry |
| ESP32-C3 GPIO constraints | MEDIUM | SuperMini pinout documented; specific GPIO conflicts need hardware validation |
| PlatformIO build | HIGH | Well-documented, community-confirmed for ESP32-C3 |
---
## Critical Constraints to Validate Early
1. **GPIO conflict: SK6812 RMT + WS2801 SPI on ESP32-C3 SuperMini** — The SuperMini has limited broken-out pins. Verify that chosen RMT GPIO and SPI GPIO for simultaneous operation don't conflict. NeoPixelBus RMT does not block SPI, so hardware parallelism is possible.
2. **sounddevice + miniaudio simultaneous device access** — Both will request the default PulseAudio/PipeWire sink. On Raspberry Pi OS bookworm (PipeWire default), multiple readers on a loopback monitor source should work, but test early.
3. **aubio version on Raspberry Pi ARM** — aubio 0.4.9 is the last stable release (2019). The pip wheel for ARM may not exist; may need to compile from source. Test `pip install aubio` on target Pi before committing.
4. **WiFi latency floor** — UDP on local WiFi to ESP32 has measured RTT of 20-60ms depending on router/environment. Commands are fire-and-forget so one-way latency is 10-30ms. Musical sync at 120BPM has 500ms between beats — 30ms latency is well within tolerance for beat-level sync, but not sub-beat.
---
## Sources
- Textual releases: https://github.com/Textualize/textual/releases
- aubio real-time design: https://aubio.org/doc/ — https://github.com/zak-45/WLEDAudioSyncRTBeat
- sounddevice docs: https://python-sounddevice.readthedocs.io/
- miniaudio Python: https://github.com/irmen/pyminiaudio
- NeoPixelBus ESP32-C3 RMT: https://github.com/Makuna/NeoPixelBus/issues/477
- ArduinoJson 7.4.x: https://components.espressif.com/components/bblanchon/arduinojson/versions/7.4.2
- ESPAsyncWebServer archived + successor: https://github.com/mathieucarbou/ESPAsyncWebServer
- ESP32-C3 SuperMini pinout: https://www.espboards.dev/esp32/esp32-c3-super-mini/
- PlatformIO ESP32-C3: https://docs.platformio.org/en/latest/boards/espressif32/esp32-c3-devkitm-1.html
- UDP latency on ESP32: https://github.com/espressif/arduino-esp32/issues/1283