diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 2abb702..6b36848 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -77,13 +77,35 @@ | Requirement | Phase | Status | |-------------|-------|--------| -| (populated during roadmap creation) | | | +| FW-01 | Phase 1 | Pending | +| FW-02 | Phase 1 | Pending | +| FW-03 | Phase 1 | Pending | +| FW-04 | Phase 1 | Pending | +| FW-05 | Phase 1 | Pending | +| AUD-01 | Phase 2 | Pending | +| AUD-02 | Phase 2 | Pending | +| AUD-03 | Phase 2 | Pending | +| AUD-04 | Phase 2 | Pending | +| CHR-04 | Phase 2 | Pending | +| CHR-05 | Phase 2 | Pending | +| CHR-01 | Phase 3 | Pending | +| CHR-02 | Phase 3 | Pending | +| CHR-03 | Phase 3 | Pending | +| UI-01 | Phase 3 | Pending | +| UI-02 | Phase 3 | Pending | +| UI-03 | Phase 3 | Pending | +| UI-04 | Phase 3 | Pending | +| UI-05 | Phase 3 | Pending | +| UI-06 | Phase 3 | Pending | +| LIVE-01 | Phase 4 | Pending | +| LIVE-02 | Phase 4 | Pending | +| SPT-01 | Phase 4 | Pending | **Coverage:** -- v1 requirements: 22 total -- Mapped to phases: 0 -- Unmapped: 22 +- v1 requirements: 23 total +- Mapped to phases: 23 +- Unmapped: 0 --- *Requirements defined: 2026-04-03* -*Last updated: 2026-04-03 after initial definition* +*Last updated: 2026-04-03 after roadmap creation* diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md new file mode 100644 index 0000000..436365e --- /dev/null +++ b/.planning/ROADMAP.md @@ -0,0 +1,79 @@ +# Roadmap: LED Sync Studio + +## Overview + +Four phases build a music-synchronized LED choreography system from hardware up. Phase 1 validates the ESP32 firmware and retires all hardware risks before any Pi-side code is written. Phase 2 builds the headless Pi core — audio playback, beat detection, choreography engine, and UDP transport — all CLI-testable against real hardware. Phase 3 assembles the full cyberpunk TUI on top of the stable core, delivering the choreography editor end-to-end. Phase 4 adds live-reactive mode and Spotify as a second audio source. + +## Phases + +**Phase Numbering:** +- Integer phases (1, 2, 3): Planned milestone work +- Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED) + +Decimal phases appear between their surrounding integers in numeric order. + +- [ ] **Phase 1: ESP32 Firmware** - Both LED strips driven by a WiFi-connected ESP32 that executes named animations from JSON commands +- [ ] **Phase 2: App Core + Audio** - Headless Pi layer: song playback, beat detection, choreography engine, UDP transport +- [ ] **Phase 3: Choreography TUI** - Full cyberpunk terminal UI with timeline editor, event list, transport controls, and animation panel +- [ ] **Phase 4: Live Reactive + Spotify** - Beat-driven animation mode with sensitivity calibration and Spotify as audio source + +## Phase Details + +### Phase 1: ESP32 Firmware +**Goal**: Both LED strips are driven by one ESP32-C3 that receives JSON commands over WiFi and executes named animations autonomously +**Depends on**: Nothing (first phase) +**Requirements**: FW-01, FW-02, FW-03, FW-04, FW-05 +**Success Criteria** (what must be TRUE): + 1. WS2801 (160 LEDs, SPI) and SK6812 (300 LEDs, RMT/RGBW) both illuminate correctly and simultaneously with no flicker from WiFi interference + 2. Sending a UDP JSON command (e.g. `{"zone":"wall","animation":"chase","speed":0.5}`) from a laptop triggers the named animation on the correct strip within 50ms + 3. All 8 built-in animations (Chase, Pulse, Rainbow, Strobe, Color Wash, Breathe, Sparkle, Gradient Sweep) run on either zone without crashing + 4. Each zone can be commanded independently — different animations run on Schrank and Wand simultaneously + 5. Animation parameters (color/RGBW, speed, intensity) visibly change behavior when included in the JSON command +**Plans**: TBD + +### Phase 2: App Core + Audio +**Goal**: The Pi can play songs, detect beats, schedule choreography events, and send animation commands to the ESP32 — all testable from the command line without any TUI +**Depends on**: Phase 1 +**Requirements**: AUD-01, AUD-02, AUD-03, AUD-04, CHR-04, CHR-05 +**Success Criteria** (what must be TRUE): + 1. A local MP3, FLAC, or WAV file plays with audible output and the current playback position is readable at any moment + 2. Playback can be paused, resumed, and seeked to an arbitrary timestamp via CLI commands + 3. Beat events are detected from the system audio stream in real time (aubio) with visible console output per beat + 4. A choreography JSON file loads and plays back: correct animations fire on the ESP32 at the timestamps specified in the file + 5. Choreography edits (adding an event) save to a JSON file that reloads correctly on next run +**Plans**: TBD + +### Phase 3: Choreography TUI +**Goal**: The full cyberpunk terminal UI is usable over SSH: user can load a song, place animations on the timeline, and play back a complete light show +**Depends on**: Phase 2 +**Requirements**: CHR-01, CHR-02, CHR-03, UI-01, UI-02, UI-03, UI-04, UI-05, UI-06 +**Success Criteria** (what must be TRUE): + 1. The TUI launches over SSH with cyberpunk/neon styling — dark background, glowing colored elements — and all panels are legible in a standard terminal + 2. User can select a position in the song, choose an animation with parameters, and assign it to a zone — it appears as a block on the timeline view + 3. User can tap a key on the beat during playback to stamp timing marks, and those marks are visible on the timeline + 4. User can switch between Timeline view (horizontal blocks) and Event List view (table of timestamps) for the same choreography + 5. Pressing Play runs the full light show — animations fire on the ESP32 in sync with the song — and the playhead advances visibly on the timeline +**Plans**: TBD +**UI hint**: yes + +### Phase 4: Live Reactive + Spotify +**Goal**: Users can switch into live-reactive mode where beats drive LED animations in real time, and can optionally use Spotify as the audio source +**Depends on**: Phase 3 +**Requirements**: LIVE-01, LIVE-02, SPT-01 +**Success Criteria** (what must be TRUE): + 1. User can switch from choreography mode to live-reactive mode without restarting the app, and LEDs respond to beats within 100ms of audio onset + 2. User can adjust beat-detection sensitivity in the UI and see the effect immediately — more sensitive triggers fire on quieter transients, less sensitive ignores them + 3. Spotify audio passes through the system audio pipeline and triggers beat-reactive animations the same way local file playback does +**Plans**: TBD + +## Progress + +**Execution Order:** +Phases execute in numeric order: 1 → 2 → 3 → 4 + +| Phase | Plans Complete | Status | Completed | +|-------|----------------|--------|-----------| +| 1. ESP32 Firmware | 0/? | Not started | - | +| 2. App Core + Audio | 0/? | Not started | - | +| 3. Choreography TUI | 0/? | Not started | - | +| 4. Live Reactive + Spotify | 0/? | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md new file mode 100644 index 0000000..af550c6 --- /dev/null +++ b/.planning/STATE.md @@ -0,0 +1,64 @@ +# Project State + +## Project Reference + +See: .planning/PROJECT.md (updated 2026-04-03) + +**Core value:** Songs mit LED-Animationen choreografieren und abspielen — der Nutzer baut Timeline-basierte Lichtshows zu seiner Musik. +**Current focus:** Phase 1 — ESP32 Firmware + +## Current Position + +Phase: 1 of 4 (ESP32 Firmware) +Plan: 0 of ? in current phase +Status: Ready to plan +Last activity: 2026-04-03 — Roadmap created, all 23 v1 requirements mapped to 4 phases + +Progress: [░░░░░░░░░░] 0% + +## Performance Metrics + +**Velocity:** +- Total plans completed: 0 +- Average duration: - +- Total execution time: 0 hours + +**By Phase:** + +| Phase | Plans | Total | Avg/Plan | +|-------|-------|-------|----------| +| - | - | - | - | + +**Recent Trend:** +- Last 5 plans: - +- Trend: - + +*Updated after each plan completion* + +## Accumulated Context + +### Decisions + +Decisions are logged in PROJECT.md Key Decisions table. +Recent decisions affecting current work: + +- [Roadmap]: Hardware-first build order — ESP32 is the validation target for all Pi-side work; developing against a mock risks protocol mismatches late. +- [Roadmap]: WiFi+RMT coexistence must be tested on day 1 of Phase 1 (documented FastLED #1657, February 2026 — actively problematic on single-core ESP32-C3). +- [Roadmap]: SK6812 is RGBW (4 bytes/LED) — configure explicitly before any animation code; retrofitting requires protocol version bump. + +### Pending Todos + +None yet. + +### Blockers/Concerns + +- [Phase 1]: ESP32-C3 RMT+WiFi coexistence has conflicting documentation — test simultaneously on first flash. +- [Phase 1]: GPIO pin assignment for simultaneous RMT (SK6812) + SPI (WS2801) on SuperMini needs hardware validation. +- [Phase 2]: aubio 0.4.9 ARM pip wheel on Raspberry Pi OS bookworm unverified — may require source build. +- [Phase 2]: sounddevice + miniaudio simultaneous PipeWire access needs early testing. + +## Session Continuity + +Last session: 2026-04-03 +Stopped at: Roadmap written, STATE.md initialized. No plans exist yet. +Resume file: None diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..e640fef --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,168 @@ + +## Project + +**LED Sync Studio** + +Eine Cyberpunk-Terminal-App (Python/Textual) zur Steuerung von LED-Streifen, die auf einem Raspberry Pi 4B laeuft und per SSH bedient wird. Die App ermoeglicht sowohl Song-Choreografie (Animationen manuell auf Zeitpunkte im Song mappen) als auch Live-Reaktiv-Modus (Beat-Detection aus System-Audio). Die Animationen laufen lokal auf einem ESP32-C3, der per JSON-Kommandos ueber WiFi gesteuert wird. + +**Core Value:** Songs mit LED-Animationen choreografieren und abspielen — der Nutzer baut Timeline-basierte Lichtshows zu seiner Musik. + +### Constraints + +- **Platform**: Raspberry Pi 4B (ARM, Linux) — App muss performant auf Pi laufen +- **Terminal**: Textual TUI Framework (Python) — muss ueber SSH funktionieren +- **ESP**: ESP32-C3 SuperMini — begrenzter Speicher, ein Core, WiFi only +- **LED-Protokoll**: WS2801 (SPI/Clock+Data) und SK6812 (single-wire like NeoPixel) — unterschiedliche Ansteuerung +- **Latenz**: JSON-Kommandos muessen schnell genug sein fuer musikalische Synchronisation + + + +## Technology Stack + +## Recommended Stack +### TUI Layer (Raspberry Pi Host App) +| Technology | Version | Purpose | Why | +|------------|---------|---------|-----| +| Python | 3.11+ | Runtime | Ships on Raspberry Pi OS bookworm; 3.11 asyncio improvements matter for this use case | +| Textual | 8.2.1 | TUI framework | Latest stable (March 2026). CSS-based theming with custom color variables maps cleanly to cyberpunk/neon aesthetics. asyncio-native — no threading hacks needed alongside audio callbacks. Runs cleanly over SSH. | +| Rich | (Textual dependency) | Terminal rendering | Pulled in by Textual; use directly for log panels and progress bars | +- `curses` / `urwid`: No CSS, no async, painful for custom widgets like a timeline ruler +- `blessed`: Imperative, no widget system, would require building everything from scratch +- `prompt_toolkit`: Good for REPL-style apps, not layout-based dashboards +### Audio Layer +| Technology | Version | Purpose | Why | +|------------|---------|---------|-----| +| sounddevice | 0.5.x | Real-time audio capture from PipeWire/PulseAudio | Uses PortAudio under the hood; supports asyncio generators for block-by-block audio callbacks; works with PipeWire's PulseAudio compatibility layer without extra config | +| aubio | 0.4.9 | Beat detection + onset detection | Written in C, Python bindings via NumPy arrays, designed explicitly for causal real-time processing. Configurable hop size controls latency (512 samples at 44.1kHz = ~11ms per frame). | +| numpy | 1.26+ | Audio buffer manipulation, FFT for spectrum display | Required by aubio; use `numpy.fft.rfft` directly for real-time frequency band analysis without librosa overhead | +| miniaudio | 1.61 | MP3/FLAC/WAV file decoding and playback with position tracking | Pure C backend, no SDL dependency (unlike pygame), exposes playback position in samples, supports seeking. `just_playback` wraps it but miniaudio directly gives more control. | +- librosa is offline-first — designed for batch analysis of whole files, not streaming callbacks +- aubio's C core processes audio frames in-place with ~11ms latency at 44.1kHz/512-sample hop +- The `aubio.tempo` object maintains internal beat-tracking state across frames +- pyaudio requires ALSA headers to compile and has no asyncio support +- sounddevice provides `sd.InputStream` with a Python callback that receives NumPy arrays directly +- PipeWire exposes a PulseAudio-compatible socket on Raspberry Pi OS bookworm — sounddevice connects without special config +- pygame's audio system conflicts with sounddevice when both need ALSA/PulseAudio simultaneously +- miniaudio exposes exact sample position, enabling precise choreography timeline sync +- No SDL dependency = no display server requirement (SSH-only environment) +### Communication Layer (Pi → ESP32) +| Technology | Version | Purpose | Why | +|------------|---------|---------|-----| +| asyncio (stdlib) | Python 3.11+ | UDP datagram sending | No external dependency; `loop.create_datagram_endpoint()` gives fire-and-forget UDP with ~1-5ms overhead | +| UDP (not TCP) | — | Transport protocol | TCP Nagle algorithm adds 40-200ms latency on small packets without tuning; UDP is connectionless and sub-5ms RTT on local WiFi. Commands are idempotent (animation start/stop) so lost packets are acceptable. | +| JSON (stdlib `json`) | — | Command serialization | Already specified in requirements; human-readable for debugging; ArduinoJson v7 on ESP side handles deserialization efficiently | +- HTTP adds 100-300ms per request (connection setup + headers) +- WebSocket persistent connection is viable but adds complexity on ESP side and has known ESP32-C3 crash issues with me-no-dev library +- UDP fire-and-forget matches the use case: beat hits → send command → ESP acts immediately +### ESP32-C3 Firmware +| Technology | Version | Purpose | Why | +|------------|---------|---------|-----| +| PlatformIO | latest | Build system | Far superior to Arduino IDE for this project: proper dependency management, `platformio.ini` reproducible builds, VS Code integration, OTA flash support | +| Arduino framework for ESP32 | 3.x (espressif32) | Hardware abstraction | Mature, large library ecosystem, good ESP32-C3 support in core 3.x | +| NeoPixelBus by Makuna | 2.8.x | SK6812 single-wire control via RMT | Uses ESP32-C3 RMT peripheral (channels 0-1 available), hardware-timed — no CPU spin waiting. Handles RGBW SK6812 correctly including white channel. | +| Adafruit DotStar / raw SPI | — | WS2801 SPI control | WS2801 is a clock+data SPI protocol, not single-wire. Use Arduino `SPI.h` directly or Adafruit DotStar library which supports WS2801 via hardware SPI. NeoPixelBus also supports SPI method for WS2801. | +| ArduinoJson | 7.4.x | JSON deserialization of commands | Version 7 uses heap-allocated elastic documents — no need to pre-size buffers. Benchmarked at <1ms for typical command payloads on ESP32. | +| AsyncUDP (ESP32 Arduino stdlib) | — | UDP packet reception | Built into arduino-esp32 core, no external dependency, runs in background task. More stable on C3 than ESPAsyncWebServer HTTP. | +- **SK6812 (Wand, 300 LEDs):** NeoPixelBus with `NeoEsp32RmtMethodBase` on GPIO4 (RMT channel 0) +- **WS2801 (Schrank, 160 LEDs):** Hardware SPI via `SPI.h` on GPIO6 (MOSI) + GPIO4 (SCK) — but check GPIO conflicts with SK6812 RMT pin assignment. Use GPIO pin mapping carefully: avoid GPIO2, GPIO8, GPIO9 (boot-critical), GPIO12-17 (internal flash). +- SK6812 data: GPIO3 (RMT channel 0) +- WS2801 MOSI: GPIO7, WS2801 CLK: GPIO6 (hardware SPI bus 1) +- Leave GPIO8 (onboard LED/WS2812) unused to avoid conflicts +- WLED is a complete opinionated firmware — cannot receive custom JSON animation commands +- Animation logic must run on ESP32 with own state machine; WLED's architecture doesn't expose low-level animation API +- Building custom firmware is the right call here +- Archived January 2025, read-only +- Known crash bugs on ESP32-C3 single-core with WebSocket connections +- Successor: ESP32Async/ESPAsyncWebServer — but unnecessary complexity for this project. Plain AsyncUDP is simpler and lower latency. +### Data Storage (Choreography Files) +| Technology | Version | Purpose | Why | +|------------|---------|---------|-----| +| JSON (stdlib) | — | Choreography file format | Human-readable, easily version-controlled, Python `json` module needs zero deps. YAML adds a dependency and parsing overhead with no benefit for machine-generated files. | +| Pydantic | 2.x | Schema validation for choreography files | V2 is Rust-backed, fast. Validates animation parameter ranges, timeline consistency. Generates JSON schemas for free. Worth the dependency for data integrity. | +### Development Tooling +| Technology | Purpose | +|------------|---------| +| uv | Python package manager — significantly faster than pip, proper lockfiles, works on Raspberry Pi ARM | +| PlatformIO | ESP32 firmware build, dependency resolution, OTA upload | +| pytest + pytest-asyncio | Test suite for choreography engine and protocol logic | +## Alternatives Considered +| Category | Recommended | Alternative | Why Not | +|----------|-------------|-------------|---------| +| TUI | Textual 8.2.1 | curses, urwid, blessed | No CSS theming, no asyncio integration, verbose widget construction | +| Beat detection | aubio | librosa | librosa is offline/batch, not real-time streaming; adds heavy scipy/sklearn deps | +| Beat detection | aubio | essentia | Essentia is ML-heavy, not pip-installable cleanly on Raspberry Pi ARM without build hassles | +| Audio capture | sounddevice | pyaudio | pyaudio needs ALSA headers, no asyncio, harder to pip install | +| Audio playback | miniaudio | pygame | pygame conflicts with sounddevice for audio device access; no playback position API | +| Audio playback | miniaudio | gstreamer | gstreamer Python bindings (gi.repository) are fragile, heavyweight | +| Transport | UDP | HTTP/REST | HTTP adds 100-300ms latency per command, unacceptable for beat-sync | +| Transport | UDP | WebSocket | ESP32-C3 WebSocket stability issues; more complex firmware; no benefit for one-directional command stream | +| LED (SK6812) | NeoPixelBus (RMT) | Adafruit NeoPixel | NeoPixelBus RMT is hardware-timed and interrupt-free; Adafruit NeoPixel disables interrupts during transfer, causing WiFi packet drops | +| Firmware build | PlatformIO | Arduino IDE | Arduino IDE has no proper dependency management, no CLI, no lockfiles | +| Choreography format | JSON | YAML | Zero extra dependency for JSON; no user-facing config editing needed | +| Config validation | Pydantic v2 | dataclasses | Pydantic gives JSON schema, field validators, and serialization; dataclasses don't | +## Installation +### Raspberry Pi Host App +# Install uv (fast Python package manager) +# Create project environment +# System dependency for sounddevice (PortAudio) +### ESP32-C3 Firmware (PlatformIO) +## Confidence Assessment +| Component | Confidence | Notes | +|-----------|------------|-------| +| Textual 8.2.1 | HIGH | Verified via GitHub releases page | +| aubio for real-time | HIGH | Official docs confirm causal/streaming design | +| sounddevice + PipeWire | MEDIUM | PipeWire PulseAudio compat layer works in practice; not officially documented for sounddevice | +| miniaudio for playback | MEDIUM | Position tracking confirmed in docs; Raspberry Pi ARM support confirmed via PyPI | +| UDP over TCP | HIGH | Latency characteristics well-documented; ESP32 TCP Nagle issue confirmed in arduino-esp32 issues | +| NeoPixelBus RMT for SK6812 | HIGH | RMT method confirmed for ESP32-C3; RGBW SK6812 support documented | +| WS2801 via SPI | MEDIUM | SPI method works in NeoPixelBus or raw SPI; GPIO pin assignments need hardware verification | +| ArduinoJson 7.4.x | HIGH | Latest version confirmed via ESP Component Registry | +| ESP32-C3 GPIO constraints | MEDIUM | SuperMini pinout documented; specific GPIO conflicts need hardware validation | +| PlatformIO build | HIGH | Well-documented, community-confirmed for ESP32-C3 | +## Critical Constraints to Validate Early +## Sources +- Textual releases: https://github.com/Textualize/textual/releases +- aubio real-time design: https://aubio.org/doc/ — https://github.com/zak-45/WLEDAudioSyncRTBeat +- sounddevice docs: https://python-sounddevice.readthedocs.io/ +- miniaudio Python: https://github.com/irmen/pyminiaudio +- NeoPixelBus ESP32-C3 RMT: https://github.com/Makuna/NeoPixelBus/issues/477 +- ArduinoJson 7.4.x: https://components.espressif.com/components/bblanchon/arduinojson/versions/7.4.2 +- ESPAsyncWebServer archived + successor: https://github.com/mathieucarbou/ESPAsyncWebServer +- ESP32-C3 SuperMini pinout: https://www.espboards.dev/esp32/esp32-c3-super-mini/ +- PlatformIO ESP32-C3: https://docs.platformio.org/en/latest/boards/espressif32/esp32-c3-devkitm-1.html +- UDP latency on ESP32: https://github.com/espressif/arduino-esp32/issues/1283 + + + +## Conventions + +Conventions not yet established. Will populate as patterns emerge during development. + + + +## Architecture + +Architecture not yet mapped. Follow existing patterns found in the codebase. + + + +## GSD Workflow Enforcement + +Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync. + +Use these entry points: +- `/gsd:quick` for small fixes, doc updates, and ad-hoc tasks +- `/gsd:debug` for investigation and bug fixing +- `/gsd:execute-phase` for planned phase work + +Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it. + + + + + +## Developer Profile + +> Profile not yet configured. Run `/gsd:profile-user` to generate your developer profile. +> This section is managed by `generate-claude-profile` -- do not edit manually. +