Gitea-Admin/led-sync-studio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: create roadmap (4 phases)

Browse Source
This commit is contained in:
Claude
2026-04-03 12:39:17 +02:00
parent 1d26fc2dc5
commit 72ed07b0e9
4 changed files with 338 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
.planning/REQUIREMENTS.md
View File

@@ -77,13 +77,35 @@
| Requirement | Phase | Status | | Requirement | Phase | Status |
|-------------|-------|--------| |-------------|-------|--------|
| (populated during roadmap creation) | | | | FW-01 | Phase 1 | Pending |
| FW-02 | Phase 1 | Pending |
| FW-03 | Phase 1 | Pending |
| FW-04 | Phase 1 | Pending |
| FW-05 | Phase 1 | Pending |
| AUD-01 | Phase 2 | Pending |
| AUD-02 | Phase 2 | Pending |
| AUD-03 | Phase 2 | Pending |
| AUD-04 | Phase 2 | Pending |
| CHR-04 | Phase 2 | Pending |
| CHR-05 | Phase 2 | Pending |
| CHR-01 | Phase 3 | Pending |
| CHR-02 | Phase 3 | Pending |
| CHR-03 | Phase 3 | Pending |
| UI-01 | Phase 3 | Pending |
| UI-02 | Phase 3 | Pending |
| UI-03 | Phase 3 | Pending |
| UI-04 | Phase 3 | Pending |
| UI-05 | Phase 3 | Pending |
| UI-06 | Phase 3 | Pending |
| LIVE-01 | Phase 4 | Pending |
| LIVE-02 | Phase 4 | Pending |
| SPT-01 | Phase 4 | Pending |
**Coverage:** **Coverage:**
- v1 requirements: 22 total - v1 requirements: 23 total
- Mapped to phases: 0 - Mapped to phases: 23
- Unmapped: 22 - Unmapped: 0
--- ---
*Requirements defined: 2026-04-03* *Requirements defined: 2026-04-03*
*Last updated: 2026-04-03 after initial definition* *Last updated: 2026-04-03 after roadmap creation*

79
.planning/ROADMAP.md Normal file
View File

@@ -0,0 +1,79 @@
# Roadmap: LED Sync Studio
## Overview
Four phases build a music-synchronized LED choreography system from hardware up. Phase 1 validates the ESP32 firmware and retires all hardware risks before any Pi-side code is written. Phase 2 builds the headless Pi core — audio playback, beat detection, choreography engine, and UDP transport — all CLI-testable against real hardware. Phase 3 assembles the full cyberpunk TUI on top of the stable core, delivering the choreography editor end-to-end. Phase 4 adds live-reactive mode and Spotify as a second audio source.
## Phases
**Phase Numbering:**
- Integer phases (1, 2, 3): Planned milestone work
- Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED)
Decimal phases appear between their surrounding integers in numeric order.
- [ ] **Phase 1: ESP32 Firmware** - Both LED strips driven by a WiFi-connected ESP32 that executes named animations from JSON commands
- [ ] **Phase 2: App Core + Audio** - Headless Pi layer: song playback, beat detection, choreography engine, UDP transport
- [ ] **Phase 3: Choreography TUI** - Full cyberpunk terminal UI with timeline editor, event list, transport controls, and animation panel
- [ ] **Phase 4: Live Reactive + Spotify** - Beat-driven animation mode with sensitivity calibration and Spotify as audio source
## Phase Details
### Phase 1: ESP32 Firmware
**Goal**: Both LED strips are driven by one ESP32-C3 that receives JSON commands over WiFi and executes named animations autonomously
**Depends on**: Nothing (first phase)
**Requirements**: FW-01, FW-02, FW-03, FW-04, FW-05
**Success Criteria** (what must be TRUE):
1. WS2801 (160 LEDs, SPI) and SK6812 (300 LEDs, RMT/RGBW) both illuminate correctly and simultaneously with no flicker from WiFi interference
2. Sending a UDP JSON command (e.g. `{"zone":"wall","animation":"chase","speed":0.5}`) from a laptop triggers the named animation on the correct strip within 50ms
3. All 8 built-in animations (Chase, Pulse, Rainbow, Strobe, Color Wash, Breathe, Sparkle, Gradient Sweep) run on either zone without crashing
4. Each zone can be commanded independently — different animations run on Schrank and Wand simultaneously
5. Animation parameters (color/RGBW, speed, intensity) visibly change behavior when included in the JSON command
**Plans**: TBD
### Phase 2: App Core + Audio
**Goal**: The Pi can play songs, detect beats, schedule choreography events, and send animation commands to the ESP32 — all testable from the command line without any TUI
**Depends on**: Phase 1
**Requirements**: AUD-01, AUD-02, AUD-03, AUD-04, CHR-04, CHR-05
**Success Criteria** (what must be TRUE):
1. A local MP3, FLAC, or WAV file plays with audible output and the current playback position is readable at any moment
2. Playback can be paused, resumed, and seeked to an arbitrary timestamp via CLI commands
3. Beat events are detected from the system audio stream in real time (aubio) with visible console output per beat
4. A choreography JSON file loads and plays back: correct animations fire on the ESP32 at the timestamps specified in the file
5. Choreography edits (adding an event) save to a JSON file that reloads correctly on next run
**Plans**: TBD
### Phase 3: Choreography TUI
**Goal**: The full cyberpunk terminal UI is usable over SSH: user can load a song, place animations on the timeline, and play back a complete light show
**Depends on**: Phase 2
**Requirements**: CHR-01, CHR-02, CHR-03, UI-01, UI-02, UI-03, UI-04, UI-05, UI-06
**Success Criteria** (what must be TRUE):
1. The TUI launches over SSH with cyberpunk/neon styling — dark background, glowing colored elements — and all panels are legible in a standard terminal
2. User can select a position in the song, choose an animation with parameters, and assign it to a zone — it appears as a block on the timeline view
3. User can tap a key on the beat during playback to stamp timing marks, and those marks are visible on the timeline
4. User can switch between Timeline view (horizontal blocks) and Event List view (table of timestamps) for the same choreography
5. Pressing Play runs the full light show — animations fire on the ESP32 in sync with the song — and the playhead advances visibly on the timeline
**Plans**: TBD
**UI hint**: yes
### Phase 4: Live Reactive + Spotify
**Goal**: Users can switch into live-reactive mode where beats drive LED animations in real time, and can optionally use Spotify as the audio source
**Depends on**: Phase 3
**Requirements**: LIVE-01, LIVE-02, SPT-01
**Success Criteria** (what must be TRUE):
1. User can switch from choreography mode to live-reactive mode without restarting the app, and LEDs respond to beats within 100ms of audio onset
2. User can adjust beat-detection sensitivity in the UI and see the effect immediately — more sensitive triggers fire on quieter transients, less sensitive ignores them
3. Spotify audio passes through the system audio pipeline and triggers beat-reactive animations the same way local file playback does
**Plans**: TBD
## Progress
**Execution Order:**
Phases execute in numeric order: 1 → 2 → 3 → 4
| Phase | Plans Complete | Status | Completed |
|-------|----------------|--------|-----------|
| 1. ESP32 Firmware | 0/? | Not started | - |
| 2. App Core + Audio | 0/? | Not started | - |
| 3. Choreography TUI | 0/? | Not started | - |
| 4. Live Reactive + Spotify | 0/? | Not started | - |

64
.planning/STATE.md Normal file
View File

@@ -0,0 +1,64 @@
# Project State
## Project Reference
See: .planning/PROJECT.md (updated 2026-04-03)
**Core value:** Songs mit LED-Animationen choreografieren und abspielen — der Nutzer baut Timeline-basierte Lichtshows zu seiner Musik.
**Current focus:** Phase 1 — ESP32 Firmware
## Current Position
Phase: 1 of 4 (ESP32 Firmware)
Plan: 0 of ? in current phase
Status: Ready to plan
Last activity: 2026-04-03 — Roadmap created, all 23 v1 requirements mapped to 4 phases
Progress: [░░░░░░░░░░] 0%
## Performance Metrics
**Velocity:**
- Total plans completed: 0
- Average duration: -
- Total execution time: 0 hours
**By Phase:**
| Phase | Plans | Total | Avg/Plan |
|-------|-------|-------|----------|
| - | - | - | - |
**Recent Trend:**
- Last 5 plans: -
- Trend: -
*Updated after each plan completion*
## Accumulated Context
### Decisions
Decisions are logged in PROJECT.md Key Decisions table.
Recent decisions affecting current work:
- [Roadmap]: Hardware-first build order — ESP32 is the validation target for all Pi-side work; developing against a mock risks protocol mismatches late.
- [Roadmap]: WiFi+RMT coexistence must be tested on day 1 of Phase 1 (documented FastLED #1657, February 2026 — actively problematic on single-core ESP32-C3).
- [Roadmap]: SK6812 is RGBW (4 bytes/LED) — configure explicitly before any animation code; retrofitting requires protocol version bump.
### Pending Todos
None yet.
### Blockers/Concerns
- [Phase 1]: ESP32-C3 RMT+WiFi coexistence has conflicting documentation — test simultaneously on first flash.
- [Phase 1]: GPIO pin assignment for simultaneous RMT (SK6812) + SPI (WS2801) on SuperMini needs hardware validation.
- [Phase 2]: aubio 0.4.9 ARM pip wheel on Raspberry Pi OS bookworm unverified — may require source build.
- [Phase 2]: sounddevice + miniaudio simultaneous PipeWire access needs early testing.
## Session Continuity
Last session: 2026-04-03
Stopped at: Roadmap written, STATE.md initialized. No plans exist yet.
Resume file: None

168
CLAUDE.md Normal file
View File

@@ -0,0 +1,168 @@
<!-- GSD:project-start source:PROJECT.md -->
## Project
**LED Sync Studio**
Eine Cyberpunk-Terminal-App (Python/Textual) zur Steuerung von LED-Streifen, die auf einem Raspberry Pi 4B laeuft und per SSH bedient wird. Die App ermoeglicht sowohl Song-Choreografie (Animationen manuell auf Zeitpunkte im Song mappen) als auch Live-Reaktiv-Modus (Beat-Detection aus System-Audio). Die Animationen laufen lokal auf einem ESP32-C3, der per JSON-Kommandos ueber WiFi gesteuert wird.
**Core Value:** Songs mit LED-Animationen choreografieren und abspielen — der Nutzer baut Timeline-basierte Lichtshows zu seiner Musik.
### Constraints
- **Platform**: Raspberry Pi 4B (ARM, Linux) — App muss performant auf Pi laufen
- **Terminal**: Textual TUI Framework (Python) — muss ueber SSH funktionieren
- **ESP**: ESP32-C3 SuperMini — begrenzter Speicher, ein Core, WiFi only
- **LED-Protokoll**: WS2801 (SPI/Clock+Data) und SK6812 (single-wire like NeoPixel) — unterschiedliche Ansteuerung
- **Latenz**: JSON-Kommandos muessen schnell genug sein fuer musikalische Synchronisation
<!-- GSD:project-end -->
<!-- GSD:stack-start source:research/STACK.md -->
## Technology Stack
## Recommended Stack
### TUI Layer (Raspberry Pi Host App)
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| Python | 3.11+ | Runtime | Ships on Raspberry Pi OS bookworm; 3.11 asyncio improvements matter for this use case |
| Textual | 8.2.1 | TUI framework | Latest stable (March 2026). CSS-based theming with custom color variables maps cleanly to cyberpunk/neon aesthetics. asyncio-native — no threading hacks needed alongside audio callbacks. Runs cleanly over SSH. |
| Rich | (Textual dependency) | Terminal rendering | Pulled in by Textual; use directly for log panels and progress bars |
- `curses` / `urwid`: No CSS, no async, painful for custom widgets like a timeline ruler
- `blessed`: Imperative, no widget system, would require building everything from scratch
- `prompt_toolkit`: Good for REPL-style apps, not layout-based dashboards
### Audio Layer
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| sounddevice | 0.5.x | Real-time audio capture from PipeWire/PulseAudio | Uses PortAudio under the hood; supports asyncio generators for block-by-block audio callbacks; works with PipeWire's PulseAudio compatibility layer without extra config |
| aubio | 0.4.9 | Beat detection + onset detection | Written in C, Python bindings via NumPy arrays, designed explicitly for causal real-time processing. Configurable hop size controls latency (512 samples at 44.1kHz = ~11ms per frame). |
| numpy | 1.26+ | Audio buffer manipulation, FFT for spectrum display | Required by aubio; use `numpy.fft.rfft` directly for real-time frequency band analysis without librosa overhead |
| miniaudio | 1.61 | MP3/FLAC/WAV file decoding and playback with position tracking | Pure C backend, no SDL dependency (unlike pygame), exposes playback position in samples, supports seeking. `just_playback` wraps it but miniaudio directly gives more control. |
- librosa is offline-first — designed for batch analysis of whole files, not streaming callbacks
- aubio's C core processes audio frames in-place with ~11ms latency at 44.1kHz/512-sample hop
- The `aubio.tempo` object maintains internal beat-tracking state across frames
- pyaudio requires ALSA headers to compile and has no asyncio support
- sounddevice provides `sd.InputStream` with a Python callback that receives NumPy arrays directly
- PipeWire exposes a PulseAudio-compatible socket on Raspberry Pi OS bookworm — sounddevice connects without special config
- pygame's audio system conflicts with sounddevice when both need ALSA/PulseAudio simultaneously
- miniaudio exposes exact sample position, enabling precise choreography timeline sync
- No SDL dependency = no display server requirement (SSH-only environment)
### Communication Layer (Pi → ESP32)
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| asyncio (stdlib) | Python 3.11+ | UDP datagram sending | No external dependency; `loop.create_datagram_endpoint()` gives fire-and-forget UDP with ~1-5ms overhead |
| UDP (not TCP) | — | Transport protocol | TCP Nagle algorithm adds 40-200ms latency on small packets without tuning; UDP is connectionless and sub-5ms RTT on local WiFi. Commands are idempotent (animation start/stop) so lost packets are acceptable. |
| JSON (stdlib `json`) | — | Command serialization | Already specified in requirements; human-readable for debugging; ArduinoJson v7 on ESP side handles deserialization efficiently |
- HTTP adds 100-300ms per request (connection setup + headers)
- WebSocket persistent connection is viable but adds complexity on ESP side and has known ESP32-C3 crash issues with me-no-dev library
- UDP fire-and-forget matches the use case: beat hits → send command → ESP acts immediately
### ESP32-C3 Firmware
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| PlatformIO | latest | Build system | Far superior to Arduino IDE for this project: proper dependency management, `platformio.ini` reproducible builds, VS Code integration, OTA flash support |
| Arduino framework for ESP32 | 3.x (espressif32) | Hardware abstraction | Mature, large library ecosystem, good ESP32-C3 support in core 3.x |
| NeoPixelBus by Makuna | 2.8.x | SK6812 single-wire control via RMT | Uses ESP32-C3 RMT peripheral (channels 0-1 available), hardware-timed — no CPU spin waiting. Handles RGBW SK6812 correctly including white channel. |
| Adafruit DotStar / raw SPI | — | WS2801 SPI control | WS2801 is a clock+data SPI protocol, not single-wire. Use Arduino `SPI.h` directly or Adafruit DotStar library which supports WS2801 via hardware SPI. NeoPixelBus also supports SPI method for WS2801. |
| ArduinoJson | 7.4.x | JSON deserialization of commands | Version 7 uses heap-allocated elastic documents — no need to pre-size buffers. Benchmarked at <1ms for typical command payloads on ESP32. |
| AsyncUDP (ESP32 Arduino stdlib) | — | UDP packet reception | Built into arduino-esp32 core, no external dependency, runs in background task. More stable on C3 than ESPAsyncWebServer HTTP. |
- **SK6812 (Wand, 300 LEDs):** NeoPixelBus with `NeoEsp32RmtMethodBase` on GPIO4 (RMT channel 0)
- **WS2801 (Schrank, 160 LEDs):** Hardware SPI via `SPI.h` on GPIO6 (MOSI) + GPIO4 (SCK) — but check GPIO conflicts with SK6812 RMT pin assignment. Use GPIO pin mapping carefully: avoid GPIO2, GPIO8, GPIO9 (boot-critical), GPIO12-17 (internal flash).
- SK6812 data: GPIO3 (RMT channel 0)
- WS2801 MOSI: GPIO7, WS2801 CLK: GPIO6 (hardware SPI bus 1)
- Leave GPIO8 (onboard LED/WS2812) unused to avoid conflicts
- WLED is a complete opinionated firmware — cannot receive custom JSON animation commands
- Animation logic must run on ESP32 with own state machine; WLED's architecture doesn't expose low-level animation API
- Building custom firmware is the right call here
- Archived January 2025, read-only
- Known crash bugs on ESP32-C3 single-core with WebSocket connections
- Successor: ESP32Async/ESPAsyncWebServer — but unnecessary complexity for this project. Plain AsyncUDP is simpler and lower latency.
### Data Storage (Choreography Files)
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| JSON (stdlib) | — | Choreography file format | Human-readable, easily version-controlled, Python `json` module needs zero deps. YAML adds a dependency and parsing overhead with no benefit for machine-generated files. |
| Pydantic | 2.x | Schema validation for choreography files | V2 is Rust-backed, fast. Validates animation parameter ranges, timeline consistency. Generates JSON schemas for free. Worth the dependency for data integrity. |
### Development Tooling
| Technology | Purpose |
|------------|---------|
| uv | Python package manager — significantly faster than pip, proper lockfiles, works on Raspberry Pi ARM |
| PlatformIO | ESP32 firmware build, dependency resolution, OTA upload |
| pytest + pytest-asyncio | Test suite for choreography engine and protocol logic |
## Alternatives Considered
| Category | Recommended | Alternative | Why Not |
|----------|-------------|-------------|---------|
| TUI | Textual 8.2.1 | curses, urwid, blessed | No CSS theming, no asyncio integration, verbose widget construction |
| Beat detection | aubio | librosa | librosa is offline/batch, not real-time streaming; adds heavy scipy/sklearn deps |
| Beat detection | aubio | essentia | Essentia is ML-heavy, not pip-installable cleanly on Raspberry Pi ARM without build hassles |
| Audio capture | sounddevice | pyaudio | pyaudio needs ALSA headers, no asyncio, harder to pip install |
| Audio playback | miniaudio | pygame | pygame conflicts with sounddevice for audio device access; no playback position API |
| Audio playback | miniaudio | gstreamer | gstreamer Python bindings (gi.repository) are fragile, heavyweight |
| Transport | UDP | HTTP/REST | HTTP adds 100-300ms latency per command, unacceptable for beat-sync |
| Transport | UDP | WebSocket | ESP32-C3 WebSocket stability issues; more complex firmware; no benefit for one-directional command stream |
| LED (SK6812) | NeoPixelBus (RMT) | Adafruit NeoPixel | NeoPixelBus RMT is hardware-timed and interrupt-free; Adafruit NeoPixel disables interrupts during transfer, causing WiFi packet drops |
| Firmware build | PlatformIO | Arduino IDE | Arduino IDE has no proper dependency management, no CLI, no lockfiles |
| Choreography format | JSON | YAML | Zero extra dependency for JSON; no user-facing config editing needed |
| Config validation | Pydantic v2 | dataclasses | Pydantic gives JSON schema, field validators, and serialization; dataclasses don't |
## Installation
### Raspberry Pi Host App
# Install uv (fast Python package manager)
# Create project environment
# System dependency for sounddevice (PortAudio)
### ESP32-C3 Firmware (PlatformIO)
## Confidence Assessment
| Component | Confidence | Notes |
|-----------|------------|-------|
| Textual 8.2.1 | HIGH | Verified via GitHub releases page |
| aubio for real-time | HIGH | Official docs confirm causal/streaming design |
| sounddevice + PipeWire | MEDIUM | PipeWire PulseAudio compat layer works in practice; not officially documented for sounddevice |
| miniaudio for playback | MEDIUM | Position tracking confirmed in docs; Raspberry Pi ARM support confirmed via PyPI |
| UDP over TCP | HIGH | Latency characteristics well-documented; ESP32 TCP Nagle issue confirmed in arduino-esp32 issues |
| NeoPixelBus RMT for SK6812 | HIGH | RMT method confirmed for ESP32-C3; RGBW SK6812 support documented |
| WS2801 via SPI | MEDIUM | SPI method works in NeoPixelBus or raw SPI; GPIO pin assignments need hardware verification |
| ArduinoJson 7.4.x | HIGH | Latest version confirmed via ESP Component Registry |
| ESP32-C3 GPIO constraints | MEDIUM | SuperMini pinout documented; specific GPIO conflicts need hardware validation |
| PlatformIO build | HIGH | Well-documented, community-confirmed for ESP32-C3 |
## Critical Constraints to Validate Early
## Sources
- Textual releases: https://github.com/Textualize/textual/releases
- aubio real-time design: https://aubio.org/doc/ — https://github.com/zak-45/WLEDAudioSyncRTBeat
- sounddevice docs: https://python-sounddevice.readthedocs.io/
- miniaudio Python: https://github.com/irmen/pyminiaudio
- NeoPixelBus ESP32-C3 RMT: https://github.com/Makuna/NeoPixelBus/issues/477
- ArduinoJson 7.4.x: https://components.espressif.com/components/bblanchon/arduinojson/versions/7.4.2
- ESPAsyncWebServer archived + successor: https://github.com/mathieucarbou/ESPAsyncWebServer
- ESP32-C3 SuperMini pinout: https://www.espboards.dev/esp32/esp32-c3-super-mini/
- PlatformIO ESP32-C3: https://docs.platformio.org/en/latest/boards/espressif32/esp32-c3-devkitm-1.html
- UDP latency on ESP32: https://github.com/espressif/arduino-esp32/issues/1283
<!-- GSD:stack-end -->
<!-- GSD:conventions-start source:CONVENTIONS.md -->
## Conventions
Conventions not yet established. Will populate as patterns emerge during development.
<!-- GSD:conventions-end -->
<!-- GSD:architecture-start source:ARCHITECTURE.md -->
## Architecture
Architecture not yet mapped. Follow existing patterns found in the codebase.
<!-- GSD:architecture-end -->
<!-- GSD:workflow-start source:GSD defaults -->
## GSD Workflow Enforcement
Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.
Use these entry points:
- `/gsd:quick` for small fixes, doc updates, and ad-hoc tasks
- `/gsd:debug` for investigation and bug fixing
- `/gsd:execute-phase` for planned phase work
Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.
<!-- GSD:workflow-end -->
<!-- GSD:profile-start -->
## Developer Profile
> Profile not yet configured. Run `/gsd:profile-user` to generate your developer profile.
> This section is managed by `generate-claude-profile` -- do not edit manually.
<!-- GSD:profile-end -->