led-sync-studio/CLAUDE.md

Project

LED Sync Studio

Eine Cyberpunk-Terminal-App (Python/Textual) zur Steuerung von LED-Streifen, die auf einem Raspberry Pi 4B laeuft und per SSH bedient wird. Die App ermoeglicht sowohl Song-Choreografie (Animationen manuell auf Zeitpunkte im Song mappen) als auch Live-Reaktiv-Modus (Beat-Detection aus System-Audio). Die Animationen laufen lokal auf einem ESP32-C3, der per JSON-Kommandos ueber WiFi gesteuert wird.

Core Value: Songs mit LED-Animationen choreografieren und abspielen — der Nutzer baut Timeline-basierte Lichtshows zu seiner Musik.

Constraints

• Platform: Raspberry Pi 4B (ARM, Linux) — App muss performant auf Pi laufen
• Terminal: Textual TUI Framework (Python) — muss ueber SSH funktionieren
• ESP: ESP32-C3 SuperMini — begrenzter Speicher, ein Core, WiFi only
• LED-Protokoll: WS2801 (SPI/Clock+Data) und SK6812 (single-wire like NeoPixel) — unterschiedliche Ansteuerung
• Latenz: JSON-Kommandos muessen schnell genug sein fuer musikalische Synchronisation

Technology Stack

Recommended Stack

TUI Layer (Raspberry Pi Host App)

Technology	Version	Purpose	Why
Python	3.11+	Runtime	Ships on Raspberry Pi OS bookworm; 3.11 asyncio improvements matter for this use case
Textual	8.2.1	TUI framework	Latest stable (March 2026). CSS-based theming with custom color variables maps cleanly to cyberpunk/neon aesthetics. asyncio-native — no threading hacks needed alongside audio callbacks. Runs cleanly over SSH.
Rich	(Textual dependency)	Terminal rendering	Pulled in by Textual; use directly for log panels and progress bars

• curses / urwid: No CSS, no async, painful for custom widgets like a timeline ruler
• blessed: Imperative, no widget system, would require building everything from scratch
• prompt_toolkit: Good for REPL-style apps, not layout-based dashboards

Audio Layer

Technology	Version	Purpose	Why
sounddevice	0.5.x	Real-time audio capture from PipeWire/PulseAudio	Uses PortAudio under the hood; supports asyncio generators for block-by-block audio callbacks; works with PipeWire's PulseAudio compatibility layer without extra config
aubio	0.4.9	Beat detection + onset detection	Written in C, Python bindings via NumPy arrays, designed explicitly for causal real-time processing. Configurable hop size controls latency (512 samples at 44.1kHz = ~11ms per frame).
numpy	1.26+	Audio buffer manipulation, FFT for spectrum display	Required by aubio; use numpy.fft.rfft directly for real-time frequency band analysis without librosa overhead
miniaudio	1.61	MP3/FLAC/WAV file decoding and playback with position tracking	Pure C backend, no SDL dependency (unlike pygame), exposes playback position in samples, supports seeking. just_playback wraps it but miniaudio directly gives more control.

• librosa is offline-first — designed for batch analysis of whole files, not streaming callbacks
• aubio's C core processes audio frames in-place with ~11ms latency at 44.1kHz/512-sample hop
• The aubio.tempo object maintains internal beat-tracking state across frames
• pyaudio requires ALSA headers to compile and has no asyncio support
• sounddevice provides sd.InputStream with a Python callback that receives NumPy arrays directly
• PipeWire exposes a PulseAudio-compatible socket on Raspberry Pi OS bookworm — sounddevice connects without special config
• pygame's audio system conflicts with sounddevice when both need ALSA/PulseAudio simultaneously
• miniaudio exposes exact sample position, enabling precise choreography timeline sync
• No SDL dependency = no display server requirement (SSH-only environment)

Communication Layer (Pi → ESP32)

Technology	Version	Purpose	Why
asyncio (stdlib)	Python 3.11+	UDP datagram sending	No external dependency; loop.create_datagram_endpoint() gives fire-and-forget UDP with ~1-5ms overhead
UDP (not TCP)	—	Transport protocol	TCP Nagle algorithm adds 40-200ms latency on small packets without tuning; UDP is connectionless and sub-5ms RTT on local WiFi. Commands are idempotent (animation start/stop) so lost packets are acceptable.
JSON (stdlib json)	—	Command serialization	Already specified in requirements; human-readable for debugging; ArduinoJson v7 on ESP side handles deserialization efficiently

• HTTP adds 100-300ms per request (connection setup + headers)
• WebSocket persistent connection is viable but adds complexity on ESP side and has known ESP32-C3 crash issues with me-no-dev library
• UDP fire-and-forget matches the use case: beat hits → send command → ESP acts immediately

ESP32-C3 Firmware

Technology	Version	Purpose	Why
PlatformIO	latest	Build system	Far superior to Arduino IDE for this project: proper dependency management, platformio.ini reproducible builds, VS Code integration, OTA flash support
Arduino framework for ESP32	3.x (espressif32)	Hardware abstraction	Mature, large library ecosystem, good ESP32-C3 support in core 3.x
NeoPixelBus by Makuna	2.8.x	SK6812 single-wire control via RMT	Uses ESP32-C3 RMT peripheral (channels 0-1 available), hardware-timed — no CPU spin waiting. Handles RGBW SK6812 correctly including white channel.
Adafruit DotStar / raw SPI	—	WS2801 SPI control	WS2801 is a clock+data SPI protocol, not single-wire. Use Arduino SPI.h directly or Adafruit DotStar library which supports WS2801 via hardware SPI. NeoPixelBus also supports SPI method for WS2801.
ArduinoJson	7.4.x	JSON deserialization of commands	Version 7 uses heap-allocated elastic documents — no need to pre-size buffers. Benchmarked at <1ms for typical command payloads on ESP32.
AsyncUDP (ESP32 Arduino stdlib)	—	UDP packet reception	Built into arduino-esp32 core, no external dependency, runs in background task. More stable on C3 than ESPAsyncWebServer HTTP.

• SK6812 (Wand, 300 LEDs): NeoPixelBus with NeoEsp32RmtMethodBase on GPIO4 (RMT channel 0)
• WS2801 (Schrank, 160 LEDs): Hardware SPI via SPI.h on GPIO6 (MOSI) + GPIO4 (SCK) — but check GPIO conflicts with SK6812 RMT pin assignment. Use GPIO pin mapping carefully: avoid GPIO2, GPIO8, GPIO9 (boot-critical), GPIO12-17 (internal flash).
• SK6812 data: GPIO3 (RMT channel 0)
• WS2801 MOSI: GPIO7, WS2801 CLK: GPIO6 (hardware SPI bus 1)
• Leave GPIO8 (onboard LED/WS2812) unused to avoid conflicts
• WLED is a complete opinionated firmware — cannot receive custom JSON animation commands
• Animation logic must run on ESP32 with own state machine; WLED's architecture doesn't expose low-level animation API
• Building custom firmware is the right call here
• Archived January 2025, read-only
• Known crash bugs on ESP32-C3 single-core with WebSocket connections
• Successor: ESP32Async/ESPAsyncWebServer — but unnecessary complexity for this project. Plain AsyncUDP is simpler and lower latency.

Data Storage (Choreography Files)

Technology	Version	Purpose	Why
JSON (stdlib)	—	Choreography file format	Human-readable, easily version-controlled, Python json module needs zero deps. YAML adds a dependency and parsing overhead with no benefit for machine-generated files.
Pydantic	2.x	Schema validation for choreography files	V2 is Rust-backed, fast. Validates animation parameter ranges, timeline consistency. Generates JSON schemas for free. Worth the dependency for data integrity.

Development Tooling

Technology	Purpose
uv	Python package manager — significantly faster than pip, proper lockfiles, works on Raspberry Pi ARM
PlatformIO	ESP32 firmware build, dependency resolution, OTA upload
pytest + pytest-asyncio	Test suite for choreography engine and protocol logic

Alternatives Considered

Category	Recommended	Alternative	Why Not
TUI	Textual 8.2.1	curses, urwid, blessed	No CSS theming, no asyncio integration, verbose widget construction
Beat detection	aubio	librosa	librosa is offline/batch, not real-time streaming; adds heavy scipy/sklearn deps
Beat detection	aubio	essentia	Essentia is ML-heavy, not pip-installable cleanly on Raspberry Pi ARM without build hassles
Audio capture	sounddevice	pyaudio	pyaudio needs ALSA headers, no asyncio, harder to pip install
Audio playback	miniaudio	pygame	pygame conflicts with sounddevice for audio device access; no playback position API
Audio playback	miniaudio	gstreamer	gstreamer Python bindings (gi.repository) are fragile, heavyweight
Transport	UDP	HTTP/REST	HTTP adds 100-300ms latency per command, unacceptable for beat-sync
Transport	UDP	WebSocket	ESP32-C3 WebSocket stability issues; more complex firmware; no benefit for one-directional command stream
LED (SK6812)	NeoPixelBus (RMT)	Adafruit NeoPixel	NeoPixelBus RMT is hardware-timed and interrupt-free; Adafruit NeoPixel disables interrupts during transfer, causing WiFi packet drops
Firmware build	PlatformIO	Arduino IDE	Arduino IDE has no proper dependency management, no CLI, no lockfiles
Choreography format	JSON	YAML	Zero extra dependency for JSON; no user-facing config editing needed
Config validation	Pydantic v2	dataclasses	Pydantic gives JSON schema, field validators, and serialization; dataclasses don't

Installation

Raspberry Pi Host App

Install uv (fast Python package manager)

Create project environment

System dependency for sounddevice (PortAudio)

ESP32-C3 Firmware (PlatformIO)

Confidence Assessment

Component	Confidence	Notes
Textual 8.2.1	HIGH	Verified via GitHub releases page
aubio for real-time	HIGH	Official docs confirm causal/streaming design
sounddevice + PipeWire	MEDIUM	PipeWire PulseAudio compat layer works in practice; not officially documented for sounddevice
miniaudio for playback	MEDIUM	Position tracking confirmed in docs; Raspberry Pi ARM support confirmed via PyPI
UDP over TCP	HIGH	Latency characteristics well-documented; ESP32 TCP Nagle issue confirmed in arduino-esp32 issues
NeoPixelBus RMT for SK6812	HIGH	RMT method confirmed for ESP32-C3; RGBW SK6812 support documented
WS2801 via SPI	MEDIUM	SPI method works in NeoPixelBus or raw SPI; GPIO pin assignments need hardware verification
ArduinoJson 7.4.x	HIGH	Latest version confirmed via ESP Component Registry
ESP32-C3 GPIO constraints	MEDIUM	SuperMini pinout documented; specific GPIO conflicts need hardware validation
PlatformIO build	HIGH	Well-documented, community-confirmed for ESP32-C3

Critical Constraints to Validate Early

Sources

• Textual releases: https://github.com/Textualize/textual/releases
• aubio real-time design: https://aubio.org/doc/ — https://github.com/zak-45/WLEDAudioSyncRTBeat
• sounddevice docs: https://python-sounddevice.readthedocs.io/
• miniaudio Python: https://github.com/irmen/pyminiaudio
• NeoPixelBus ESP32-C3 RMT: https://github.com/Makuna/NeoPixelBus/issues/477
• ArduinoJson 7.4.x: https://components.espressif.com/components/bblanchon/arduinojson/versions/7.4.2
• ESPAsyncWebServer archived + successor: https://github.com/mathieucarbou/ESPAsyncWebServer
• ESP32-C3 SuperMini pinout: https://www.espboards.dev/esp32/esp32-c3-super-mini/
• PlatformIO ESP32-C3: https://docs.platformio.org/en/latest/boards/espressif32/esp32-c3-devkitm-1.html
• UDP latency on ESP32: https://github.com/espressif/arduino-esp32/issues/1283

Conventions

Conventions not yet established. Will populate as patterns emerge during development.

Architecture

Architecture not yet mapped. Follow existing patterns found in the codebase.

GSD Workflow Enforcement

Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.

Use these entry points:

• /gsd:quick for small fixes, doc updates, and ad-hoc tasks
• /gsd:debug for investigation and bug fixing
• /gsd:execute-phase for planned phase work

Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.

Developer Profile

Profile not yet configured. Run /gsd:profile-user to generate your developer profile. This section is managed by generate-claude-profile -- do not edit manually.