Gitea-Admin/led-sync-studio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
led-sync-studio/.planning/research/STACK.md
Claude c3680306ee docs: research complete
2026-04-03 12:14:03 +02:00

13 KiB
Raw Permalink Blame History

Technology Stack

Project: LED Sync Studio Researched: 2026-04-03 Confidence: MEDIUM (hardware-specific constraints limit definitive verification for some choices)

Recommended Stack

TUI Layer (Raspberry Pi Host App)

Technology Version Purpose Why
Python 3.11+ Runtime Ships on Raspberry Pi OS bookworm; 3.11 asyncio improvements matter for this use case
Textual 8.2.1 TUI framework Latest stable (March 2026). CSS-based theming with custom color variables maps cleanly to cyberpunk/neon aesthetics. asyncio-native — no threading hacks needed alongside audio callbacks. Runs cleanly over SSH.
Rich (Textual dependency) Terminal rendering Pulled in by Textual; use directly for log panels and progress bars

Why Textual and not alternatives:

  • curses / urwid: No CSS, no async, painful for custom widgets like a timeline ruler
  • blessed: Imperative, no widget system, would require building everything from scratch
  • prompt_toolkit: Good for REPL-style apps, not layout-based dashboards

Audio Layer

Technology Version Purpose Why
sounddevice 0.5.x Real-time audio capture from PipeWire/PulseAudio Uses PortAudio under the hood; supports asyncio generators for block-by-block audio callbacks; works with PipeWire's PulseAudio compatibility layer without extra config
aubio 0.4.9 Beat detection + onset detection Written in C, Python bindings via NumPy arrays, designed explicitly for causal real-time processing. Configurable hop size controls latency (512 samples at 44.1kHz = ~11ms per frame).
numpy 1.26+ Audio buffer manipulation, FFT for spectrum display Required by aubio; use numpy.fft.rfft directly for real-time frequency band analysis without librosa overhead
miniaudio 1.61 MP3/FLAC/WAV file decoding and playback with position tracking Pure C backend, no SDL dependency (unlike pygame), exposes playback position in samples, supports seeking. just_playback wraps it but miniaudio directly gives more control.

Why aubio over librosa for real-time:

  • librosa is offline-first — designed for batch analysis of whole files, not streaming callbacks
  • aubio's C core processes audio frames in-place with ~11ms latency at 44.1kHz/512-sample hop
  • The aubio.tempo object maintains internal beat-tracking state across frames

Why sounddevice over pyaudio:

  • pyaudio requires ALSA headers to compile and has no asyncio support
  • sounddevice provides sd.InputStream with a Python callback that receives NumPy arrays directly
  • PipeWire exposes a PulseAudio-compatible socket on Raspberry Pi OS bookworm — sounddevice connects without special config

Why miniaudio over pygame for playback:

  • pygame's audio system conflicts with sounddevice when both need ALSA/PulseAudio simultaneously
  • miniaudio exposes exact sample position, enabling precise choreography timeline sync
  • No SDL dependency = no display server requirement (SSH-only environment)

Communication Layer (Pi → ESP32)

Technology Version Purpose Why
asyncio (stdlib) Python 3.11+ UDP datagram sending No external dependency; loop.create_datagram_endpoint() gives fire-and-forget UDP with ~1-5ms overhead
UDP (not TCP) — Transport protocol TCP Nagle algorithm adds 40-200ms latency on small packets without tuning; UDP is connectionless and sub-5ms RTT on local WiFi. Commands are idempotent (animation start/stop) so lost packets are acceptable.
JSON (stdlib json) — Command serialization Already specified in requirements; human-readable for debugging; ArduinoJson v7 on ESP side handles deserialization efficiently

Protocol design note: Keep payload small. {"cmd":"start","anim":"pulse","zone":0,"color":"#FF00FF","speed":1.5} fits in a single UDP datagram (MTU safe under 512 bytes). No fragmentation, no connection overhead.

Why not WebSocket/HTTP:

  • HTTP adds 100-300ms per request (connection setup + headers)
  • WebSocket persistent connection is viable but adds complexity on ESP side and has known ESP32-C3 crash issues with me-no-dev library
  • UDP fire-and-forget matches the use case: beat hits → send command → ESP acts immediately

ESP32-C3 Firmware

Technology Version Purpose Why
PlatformIO latest Build system Far superior to Arduino IDE for this project: proper dependency management, platformio.ini reproducible builds, VS Code integration, OTA flash support
Arduino framework for ESP32 3.x (espressif32) Hardware abstraction Mature, large library ecosystem, good ESP32-C3 support in core 3.x
NeoPixelBus by Makuna 2.8.x SK6812 single-wire control via RMT Uses ESP32-C3 RMT peripheral (channels 0-1 available), hardware-timed — no CPU spin waiting. Handles RGBW SK6812 correctly including white channel.
Adafruit DotStar / raw SPI — WS2801 SPI control WS2801 is a clock+data SPI protocol, not single-wire. Use Arduino SPI.h directly or Adafruit DotStar library which supports WS2801 via hardware SPI. NeoPixelBus also supports SPI method for WS2801.
ArduinoJson 7.4.x JSON deserialization of commands Version 7 uses heap-allocated elastic documents — no need to pre-size buffers. Benchmarked at <1ms for typical command payloads on ESP32.
AsyncUDP (ESP32 Arduino stdlib) — UDP packet reception Built into arduino-esp32 core, no external dependency, runs in background task. More stable on C3 than ESPAsyncWebServer HTTP.

LED library strategy — two strips, two protocols:

The ESP32-C3 SuperMini has 2 RMT channels and hardware SPI. Use them independently:

  • SK6812 (Wand, 300 LEDs): NeoPixelBus with NeoEsp32RmtMethodBase on GPIO4 (RMT channel 0)
  • WS2801 (Schrank, 160 LEDs): Hardware SPI via SPI.h on GPIO6 (MOSI) + GPIO4 (SCK) — but check GPIO conflicts with SK6812 RMT pin assignment. Use GPIO pin mapping carefully: avoid GPIO2, GPIO8, GPIO9 (boot-critical), GPIO12-17 (internal flash).

Concrete pin assignment to validate on hardware:

  • SK6812 data: GPIO3 (RMT channel 0)
  • WS2801 MOSI: GPIO7, WS2801 CLK: GPIO6 (hardware SPI bus 1)
  • Leave GPIO8 (onboard LED/WS2812) unused to avoid conflicts

Why not WLED:

  • WLED is a complete opinionated firmware — cannot receive custom JSON animation commands
  • Animation logic must run on ESP32 with own state machine; WLED's architecture doesn't expose low-level animation API
  • Building custom firmware is the right call here

Why not me-no-dev ESPAsyncWebServer:

  • Archived January 2025, read-only
  • Known crash bugs on ESP32-C3 single-core with WebSocket connections
  • Successor: ESP32Async/ESPAsyncWebServer — but unnecessary complexity for this project. Plain AsyncUDP is simpler and lower latency.

Data Storage (Choreography Files)

Technology Version Purpose Why
JSON (stdlib) — Choreography file format Human-readable, easily version-controlled, Python json module needs zero deps. YAML adds a dependency and parsing overhead with no benefit for machine-generated files.
Pydantic 2.x Schema validation for choreography files V2 is Rust-backed, fast. Validates animation parameter ranges, timeline consistency. Generates JSON schemas for free. Worth the dependency for data integrity.

Development Tooling

Technology Purpose
uv Python package manager — significantly faster than pip, proper lockfiles, works on Raspberry Pi ARM
PlatformIO ESP32 firmware build, dependency resolution, OTA upload
pytest + pytest-asyncio Test suite for choreography engine and protocol logic

Alternatives Considered

Category Recommended Alternative Why Not
TUI Textual 8.2.1 curses, urwid, blessed No CSS theming, no asyncio integration, verbose widget construction
Beat detection aubio librosa librosa is offline/batch, not real-time streaming; adds heavy scipy/sklearn deps
Beat detection aubio essentia Essentia is ML-heavy, not pip-installable cleanly on Raspberry Pi ARM without build hassles
Audio capture sounddevice pyaudio pyaudio needs ALSA headers, no asyncio, harder to pip install
Audio playback miniaudio pygame pygame conflicts with sounddevice for audio device access; no playback position API
Audio playback miniaudio gstreamer gstreamer Python bindings (gi.repository) are fragile, heavyweight
Transport UDP HTTP/REST HTTP adds 100-300ms latency per command, unacceptable for beat-sync
Transport UDP WebSocket ESP32-C3 WebSocket stability issues; more complex firmware; no benefit for one-directional command stream
LED (SK6812) NeoPixelBus (RMT) Adafruit NeoPixel NeoPixelBus RMT is hardware-timed and interrupt-free; Adafruit NeoPixel disables interrupts during transfer, causing WiFi packet drops
Firmware build PlatformIO Arduino IDE Arduino IDE has no proper dependency management, no CLI, no lockfiles
Choreography format JSON YAML Zero extra dependency for JSON; no user-facing config editing needed
Config validation Pydantic v2 dataclasses Pydantic gives JSON schema, field validators, and serialization; dataclasses don't

Installation

Raspberry Pi Host App

# Install uv (fast Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create project environment
uv init led-sync-studio
cd led-sync-studio
uv add textual==8.2.1
uv add sounddevice miniaudio aubio numpy pydantic

# System dependency for sounddevice (PortAudio)
sudo apt install -y libportaudio2

ESP32-C3 Firmware (PlatformIO)

; platformio.ini
[env:esp32c3_supermini]
platform = espressif32
board = esp32-c3-devkitm-1
board_build.mcu = esp32c3
framework = arduino
build_flags =
    -DARDUINO_USB_CDC_ON_BOOT=1
    -DARDUINO_USB_MODE=1
monitor_speed = 115200

lib_deps =
    makuna/NeoPixelBus @ ^2.8.0
    bblanchon/ArduinoJson @ ^7.4.0

WS2801 via hardware SPI uses only Arduino core SPI.h — no external library needed.

Confidence Assessment

Component Confidence Notes
Textual 8.2.1 HIGH Verified via GitHub releases page
aubio for real-time HIGH Official docs confirm causal/streaming design
sounddevice + PipeWire MEDIUM PipeWire PulseAudio compat layer works in practice; not officially documented for sounddevice
miniaudio for playback MEDIUM Position tracking confirmed in docs; Raspberry Pi ARM support confirmed via PyPI
UDP over TCP HIGH Latency characteristics well-documented; ESP32 TCP Nagle issue confirmed in arduino-esp32 issues
NeoPixelBus RMT for SK6812 HIGH RMT method confirmed for ESP32-C3; RGBW SK6812 support documented
WS2801 via SPI MEDIUM SPI method works in NeoPixelBus or raw SPI; GPIO pin assignments need hardware verification
ArduinoJson 7.4.x HIGH Latest version confirmed via ESP Component Registry
ESP32-C3 GPIO constraints MEDIUM SuperMini pinout documented; specific GPIO conflicts need hardware validation
PlatformIO build HIGH Well-documented, community-confirmed for ESP32-C3

Critical Constraints to Validate Early

  1. GPIO conflict: SK6812 RMT + WS2801 SPI on ESP32-C3 SuperMini — The SuperMini has limited broken-out pins. Verify that chosen RMT GPIO and SPI GPIO for simultaneous operation don't conflict. NeoPixelBus RMT does not block SPI, so hardware parallelism is possible.

  2. sounddevice + miniaudio simultaneous device access — Both will request the default PulseAudio/PipeWire sink. On Raspberry Pi OS bookworm (PipeWire default), multiple readers on a loopback monitor source should work, but test early.

  3. aubio version on Raspberry Pi ARM — aubio 0.4.9 is the last stable release (2019). The pip wheel for ARM may not exist; may need to compile from source. Test pip install aubio on target Pi before committing.

  4. WiFi latency floor — UDP on local WiFi to ESP32 has measured RTT of 20-60ms depending on router/environment. Commands are fire-and-forget so one-way latency is 10-30ms. Musical sync at 120BPM has 500ms between beats — 30ms latency is well within tolerance for beat-level sync, but not sub-beat.

Sources

Reference in New Issue View Git Blame Copy Permalink