led-sync-studio/CLAUDE.md

<!-- GSD:project-start source:PROJECT.md -->
## Project

**LED Sync Studio**

Eine Cyberpunk-Terminal-App (Python/Textual) zur Steuerung von LED-Streifen, die auf einem Raspberry Pi 4B laeuft und per SSH bedient wird. Die App ermoeglicht sowohl Song-Choreografie (Animationen manuell auf Zeitpunkte im Song mappen) als auch Live-Reaktiv-Modus (Beat-Detection aus System-Audio). Die Animationen laufen lokal auf einem ESP32-C3, der per JSON-Kommandos ueber WiFi gesteuert wird.

**Core Value:** Songs mit LED-Animationen choreografieren und abspielen — der Nutzer baut Timeline-basierte Lichtshows zu seiner Musik.

### Constraints

- **Platform**: Raspberry Pi 4B (ARM, Linux) — App muss performant auf Pi laufen
- **Terminal**: Textual TUI Framework (Python) — muss ueber SSH funktionieren
- **ESP**: ESP32-C3 SuperMini — begrenzter Speicher, ein Core, WiFi only
- **LED-Protokoll**: WS2801 (SPI/Clock+Data) und SK6812 (single-wire like NeoPixel) — unterschiedliche Ansteuerung
- **Latenz**: JSON-Kommandos muessen schnell genug sein fuer musikalische Synchronisation
<!-- GSD:project-end -->

<!-- GSD:stack-start source:research/STACK.md -->
## Technology Stack

## Recommended Stack
### TUI Layer (Raspberry Pi Host App)
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| Python | 3.11+ | Runtime | Ships on Raspberry Pi OS bookworm; 3.11 asyncio improvements matter for this use case |
| Textual | 8.2.1 | TUI framework | Latest stable (March 2026). CSS-based theming with custom color variables maps cleanly to cyberpunk/neon aesthetics. asyncio-native — no threading hacks needed alongside audio callbacks. Runs cleanly over SSH. |
| Rich | (Textual dependency) | Terminal rendering | Pulled in by Textual; use directly for log panels and progress bars |
- `curses` / `urwid`: No CSS, no async, painful for custom widgets like a timeline ruler
- `blessed`: Imperative, no widget system, would require building everything from scratch
- `prompt_toolkit`: Good for REPL-style apps, not layout-based dashboards
### Audio Layer
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| sounddevice | 0.5.x | Real-time audio capture from PipeWire/PulseAudio | Uses PortAudio under the hood; supports asyncio generators for block-by-block audio callbacks; works with PipeWire's PulseAudio compatibility layer without extra config |
| aubio | 0.4.9 | Beat detection + onset detection | Written in C, Python bindings via NumPy arrays, designed explicitly for causal real-time processing. Configurable hop size controls latency (512 samples at 44.1kHz = ~11ms per frame). |
| numpy | 1.26+ | Audio buffer manipulation, FFT for spectrum display | Required by aubio; use `numpy.fft.rfft` directly for real-time frequency band analysis without librosa overhead |
| miniaudio | 1.61 | MP3/FLAC/WAV file decoding and playback with position tracking | Pure C backend, no SDL dependency (unlike pygame), exposes playback position in samples, supports seeking. `just_playback` wraps it but miniaudio directly gives more control. |
- librosa is offline-first — designed for batch analysis of whole files, not streaming callbacks
- aubio's C core processes audio frames in-place with ~11ms latency at 44.1kHz/512-sample hop
- The `aubio.tempo` object maintains internal beat-tracking state across frames
- pyaudio requires ALSA headers to compile and has no asyncio support
- sounddevice provides `sd.InputStream` with a Python callback that receives NumPy arrays directly
- PipeWire exposes a PulseAudio-compatible socket on Raspberry Pi OS bookworm — sounddevice connects without special config
- pygame's audio system conflicts with sounddevice when both need ALSA/PulseAudio simultaneously
- miniaudio exposes exact sample position, enabling precise choreography timeline sync
- No SDL dependency = no display server requirement (SSH-only environment)
### Communication Layer (Pi → ESP32)
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| asyncio (stdlib) | Python 3.11+ | UDP datagram sending | No external dependency; `loop.create_datagram_endpoint()` gives fire-and-forget UDP with ~1-5ms overhead |
| UDP (not TCP) | — | Transport protocol | TCP Nagle algorithm adds 40-200ms latency on small packets without tuning; UDP is connectionless and sub-5ms RTT on local WiFi. Commands are idempotent (animation start/stop) so lost packets are acceptable. |
| JSON (stdlib `json`) | — | Command serialization | Already specified in requirements; human-readable for debugging; ArduinoJson v7 on ESP side handles deserialization efficiently |
- HTTP adds 100-300ms per request (connection setup + headers)
- WebSocket persistent connection is viable but adds complexity on ESP side and has known ESP32-C3 crash issues with me-no-dev library
- UDP fire-and-forget matches the use case: beat hits → send command → ESP acts immediately
### ESP32-C3 Firmware
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| PlatformIO | latest | Build system | Far superior to Arduino IDE for this project: proper dependency management, `platformio.ini` reproducible builds, VS Code integration, OTA flash support |
| Arduino framework for ESP32 | 3.x (espressif32) | Hardware abstraction | Mature, large library ecosystem, good ESP32-C3 support in core 3.x |
| NeoPixelBus by Makuna | 2.8.x | SK6812 single-wire control via RMT | Uses ESP32-C3 RMT peripheral (channels 0-1 available), hardware-timed — no CPU spin waiting. Handles RGBW SK6812 correctly including white channel. |
| Adafruit DotStar / raw SPI | — | WS2801 SPI control | WS2801 is a clock+data SPI protocol, not single-wire. Use Arduino `SPI.h` directly or Adafruit DotStar library which supports WS2801 via hardware SPI. NeoPixelBus also supports SPI method for WS2801. |
| ArduinoJson | 7.4.x | JSON deserialization of commands | Version 7 uses heap-allocated elastic documents — no need to pre-size buffers. Benchmarked at <1ms for typical command payloads on ESP32. |
| AsyncUDP (ESP32 Arduino stdlib) | — | UDP packet reception | Built into arduino-esp32 core, no external dependency, runs in background task. More stable on C3 than ESPAsyncWebServer HTTP. |
- **SK6812 (Wand, 300 LEDs):** NeoPixelBus with `NeoEsp32RmtMethodBase` on GPIO4 (RMT channel 0)
- **WS2801 (Schrank, 160 LEDs):** Hardware SPI via `SPI.h` on GPIO6 (MOSI) + GPIO4 (SCK) — but check GPIO conflicts with SK6812 RMT pin assignment. Use GPIO pin mapping carefully: avoid GPIO2, GPIO8, GPIO9 (boot-critical), GPIO12-17 (internal flash).
- SK6812 data: GPIO3 (RMT channel 0)
- WS2801 MOSI: GPIO7, WS2801 CLK: GPIO6 (hardware SPI bus 1)
- Leave GPIO8 (onboard LED/WS2812) unused to avoid conflicts
- WLED is a complete opinionated firmware — cannot receive custom JSON animation commands
- Animation logic must run on ESP32 with own state machine; WLED's architecture doesn't expose low-level animation API
- Building custom firmware is the right call here
- Archived January 2025, read-only
- Known crash bugs on ESP32-C3 single-core with WebSocket connections
- Successor: ESP32Async/ESPAsyncWebServer — but unnecessary complexity for this project. Plain AsyncUDP is simpler and lower latency.
### Data Storage (Choreography Files)
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| JSON (stdlib) | — | Choreography file format | Human-readable, easily version-controlled, Python `json` module needs zero deps. YAML adds a dependency and parsing overhead with no benefit for machine-generated files. |
| Pydantic | 2.x | Schema validation for choreography files | V2 is Rust-backed, fast. Validates animation parameter ranges, timeline consistency. Generates JSON schemas for free. Worth the dependency for data integrity. |
### Development Tooling
| Technology | Purpose |
|------------|---------|
| uv | Python package manager — significantly faster than pip, proper lockfiles, works on Raspberry Pi ARM |
| PlatformIO | ESP32 firmware build, dependency resolution, OTA upload |
| pytest + pytest-asyncio | Test suite for choreography engine and protocol logic |
## Alternatives Considered
| Category | Recommended | Alternative | Why Not |
|----------|-------------|-------------|---------|
| TUI | Textual 8.2.1 | curses, urwid, blessed | No CSS theming, no asyncio integration, verbose widget construction |
| Beat detection | aubio | librosa | librosa is offline/batch, not real-time streaming; adds heavy scipy/sklearn deps |
| Beat detection | aubio | essentia | Essentia is ML-heavy, not pip-installable cleanly on Raspberry Pi ARM without build hassles |
| Audio capture | sounddevice | pyaudio | pyaudio needs ALSA headers, no asyncio, harder to pip install |
| Audio playback | miniaudio | pygame | pygame conflicts with sounddevice for audio device access; no playback position API |
| Audio playback | miniaudio | gstreamer | gstreamer Python bindings (gi.repository) are fragile, heavyweight |
| Transport | UDP | HTTP/REST | HTTP adds 100-300ms latency per command, unacceptable for beat-sync |
| Transport | UDP | WebSocket | ESP32-C3 WebSocket stability issues; more complex firmware; no benefit for one-directional command stream |
| LED (SK6812) | NeoPixelBus (RMT) | Adafruit NeoPixel | NeoPixelBus RMT is hardware-timed and interrupt-free; Adafruit NeoPixel disables interrupts during transfer, causing WiFi packet drops |
| Firmware build | PlatformIO | Arduino IDE | Arduino IDE has no proper dependency management, no CLI, no lockfiles |
| Choreography format | JSON | YAML | Zero extra dependency for JSON; no user-facing config editing needed |
| Config validation | Pydantic v2 | dataclasses | Pydantic gives JSON schema, field validators, and serialization; dataclasses don't |
## Installation
### Raspberry Pi Host App
# Install uv (fast Python package manager)
# Create project environment
# System dependency for sounddevice (PortAudio)
### ESP32-C3 Firmware (PlatformIO)
## Confidence Assessment
| Component | Confidence | Notes |
|-----------|------------|-------|
| Textual 8.2.1 | HIGH | Verified via GitHub releases page |
| aubio for real-time | HIGH | Official docs confirm causal/streaming design |
| sounddevice + PipeWire | MEDIUM | PipeWire PulseAudio compat layer works in practice; not officially documented for sounddevice |
| miniaudio for playback | MEDIUM | Position tracking confirmed in docs; Raspberry Pi ARM support confirmed via PyPI |
| UDP over TCP | HIGH | Latency characteristics well-documented; ESP32 TCP Nagle issue confirmed in arduino-esp32 issues |
| NeoPixelBus RMT for SK6812 | HIGH | RMT method confirmed for ESP32-C3; RGBW SK6812 support documented |
| WS2801 via SPI | MEDIUM | SPI method works in NeoPixelBus or raw SPI; GPIO pin assignments need hardware verification |
| ArduinoJson 7.4.x | HIGH | Latest version confirmed via ESP Component Registry |
| ESP32-C3 GPIO constraints | MEDIUM | SuperMini pinout documented; specific GPIO conflicts need hardware validation |
| PlatformIO build | HIGH | Well-documented, community-confirmed for ESP32-C3 |
## Critical Constraints to Validate Early
## Sources
- Textual releases: https://github.com/Textualize/textual/releases
- aubio real-time design: https://aubio.org/doc/ — https://github.com/zak-45/WLEDAudioSyncRTBeat
- sounddevice docs: https://python-sounddevice.readthedocs.io/
- miniaudio Python: https://github.com/irmen/pyminiaudio
- NeoPixelBus ESP32-C3 RMT: https://github.com/Makuna/NeoPixelBus/issues/477
- ArduinoJson 7.4.x: https://components.espressif.com/components/bblanchon/arduinojson/versions/7.4.2
- ESPAsyncWebServer archived + successor: https://github.com/mathieucarbou/ESPAsyncWebServer
- ESP32-C3 SuperMini pinout: https://www.espboards.dev/esp32/esp32-c3-super-mini/
- PlatformIO ESP32-C3: https://docs.platformio.org/en/latest/boards/espressif32/esp32-c3-devkitm-1.html
- UDP latency on ESP32: https://github.com/espressif/arduino-esp32/issues/1283
<!-- GSD:stack-end -->

<!-- GSD:conventions-start source:CONVENTIONS.md -->
## Conventions

Conventions not yet established. Will populate as patterns emerge during development.
<!-- GSD:conventions-end -->

<!-- GSD:architecture-start source:ARCHITECTURE.md -->
## Architecture

Architecture not yet mapped. Follow existing patterns found in the codebase.
<!-- GSD:architecture-end -->

<!-- GSD:workflow-start source:GSD defaults -->
## GSD Workflow Enforcement

Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.

Use these entry points:
- `/gsd:quick` for small fixes, doc updates, and ad-hoc tasks
- `/gsd:debug` for investigation and bug fixing
- `/gsd:execute-phase` for planned phase work

Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.
<!-- GSD:workflow-end -->



<!-- GSD:profile-start -->
## Developer Profile

> Profile not yet configured. Run `/gsd:profile-user` to generate your developer profile.
> This section is managed by `generate-claude-profile` -- do not edit manually.
<!-- GSD:profile-end -->