Gitea-Admin/led-sync-studio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
led-sync-studio/.planning/research/FEATURES.md
Claude c3680306ee docs: research complete
2026-04-03 12:14:03 +02:00

11 KiB
Raw Permalink Blame History

Feature Landscape

Domain: LED Choreography / Music-Synced Light Show Controller Researched: 2026-04-03 Confidence: MEDIUM — based on xLights, WLED, LedFx, Jinx! analysis; ESP32/Pi-specific constraints from community sources

Reference Products Surveyed

Product Model Relevance
xLights Timeline-based sequencer, open source Direct analog — timeline + audio + multi-model
WLED ESP32 firmware + web UI What runs on the ESP side; preset/segment model
LedFx Network audio-reactive engine Live reactive reference implementation
Jinx! LED matrix control, audio/MIDI sync Beat detection + sound-to-light baseline
VenueMagic Professional show control High-end cue-list model

Table Stakes

Features users expect. Missing = product feels incomplete or unusable.

Feature Why Expected Complexity Notes
Audio waveform display on timeline Every LED sequencer shows this; without it, manual timing is blind Medium Display audio amplitude/waveform as scrubbing reference
Play / pause / seek Fundamental transport controls Low Must work while animation preview is live on LEDs
Per-zone independent control Both zones (WS2801 + SK6812) behave differently; single global control frustrates Low Two named zones minimum
Block-based timeline editor xLights, Vixen, and all sequencers use drag-drop blocks on a horizontal track High Blocks represent animation events at timestamps
Animation library (built-in effects) Users expect ready-made animations: chase, pulse, rainbow, strobe, color wash Medium At minimum 8–12 distinct named effects per zone
Per-animation parameter config Speed, color, intensity — parameterization is baseline since Jinx! v1 Medium Each effect has configurable fields exposed in sidebar/panel
Save / load choreography files Without persistence, no session survives a restart Low JSON or YAML; portable between sessions
Live reactive mode (beat detection) Users familiar with WLED/LedFx expect this as a parallel mode High System audio capture via PipeWire/PulseAudio; onset/beat detection
Visual feedback of current position Playhead showing current song position during playback Low Mandatory for sequencing work
Loop / repeat control for blocks Avoids manually placing repeated blocks; xLights, WLED playlists do this Low Repeat N times or until next event
Manual timing mark placement xLights "tapper" workflow: press key on beat to stamp marks Low Core ergonomic feature for choreography

Differentiators

Features that set the product apart. Not universally expected, but create strong value.

Feature Value Proposition Complexity Notes
Cyberpunk TUI (Textual, SSH-native) No existing LED sequencer runs as a terminal UI over SSH; this is genuinely unique High xLights/LedFx all require a desktop or browser on the same machine
Dual-mode in one tool (timeline + live reactive) Most tools are one or the other; switching modes without restarting is rare High Unified session state; reactive mode as a layer on top of timeline
Zone-aware beat-reactive routing Route bass frequencies → Zone A, treble → Zone B Medium LedFx does this with virtuals; no TUI equivalent exists
Auto-beat detection timing marks VAMP-style beat analysis that stamps timing marks automatically, then user refines High xLights uses VAMP plugins; librosa or aubio in Python are viable
Streaming audio source (Spotify) Playing from Spotify while syncing LEDs to it bridges the "my music is streaming" gap Very High Requires Spotify Connect or OS audio loopback — treat as stretch goal
Animation preview in-terminal ASCII/block-character real-time preview of what LEDs will do, without needing to look at physical strip High Textual widget, color simulation
Per-block transition effects Crossfade, cut, dissolve between adjacent animation blocks Medium WLED playlists have transition time; xLights has layer blending
Export / import xLights sequences Import existing xLights .xsq files to reuse community-built shows Very High Complex format; defer unless strong demand
Scripted effect language Jinx!Script-style: define custom animation logic in a mini-DSL Very High Not needed for V1; advanced power-user feature
Undo / redo Timeline editors without undo are painful; all quality DAWs have it Medium Required for comfortable editing; Textual app state management

Anti-Features

Features to deliberately NOT build.

Anti-Feature Why Avoid What to Do Instead
Web UI / browser interface Contradicts the entire terminal-SSH design; adds a full stack (server, frontend) SSH + Textual TUI is the whole point
Frame-by-frame pixel streaming from Pi to ESP WiFi jitter (47ms+ on ESP8266, 12ms on ESP32) makes real-time frame data unreliable; kills bandwidth ESP executes animations locally; Pi sends only named commands + params
Support for more than one ESP32 Multiplies coordination complexity exponentially; no hardware for it Keep single ESP32-C3 as sole controller
Cloud sync / multi-user collaboration No network dependency is a feature for a local-first tool Local files only (JSON/YAML)
DMX / E1.31 / sACN protocol support xLights is already excellent at this; out of hardware scope JSON-over-WiFi to ESP32 is the transport layer
Mobile app Out of scope, contradicts SSH-terminal model Terminal is accessible anywhere SSH works
2D matrix effects Hardware is 1D strips; 2D matrix effects (WLED matrix, xLights matrix) are irrelevant All effects are 1D strip effects
Automatic Spotify token management (OAuth flow) Extremely complex auth flow through terminal; security risk If Spotify is implemented, use device auth flow or OS-level audio capture instead

Feature Dependencies

Audio waveform display ──────────────────> Audio file loading (MP3/FLAC/WAV)
                                            |
Manual timing mark placement ──────────────> Audio waveform display
                                            |
Auto-beat detection timing marks ──────────> Audio waveform display + librosa/aubio

Block-based timeline editor ───────────────> Per-zone independent control
                                            |
Per-block transition effects ──────────────> Block-based timeline editor
                                            |
Loop / repeat control ─────────────────────> Block-based timeline editor

Live reactive mode ────────────────────────> Beat detection (onset/BPM)
                                            |
Zone-aware beat-reactive routing ──────────> Live reactive mode + per-zone control

Animation library ─────────────────────────> (standalone, no deps)
                                            |
Per-animation parameter config ────────────> Animation library
                                            |
Animation preview in-terminal ─────────────> Animation library + Per-animation parameter config

Save / load choreography ─────────────────> Block-based timeline editor (saves timeline state)

Undo / redo ───────────────────────────────> Block-based timeline editor (wraps edit operations)

Dual-mode (timeline + live reactive) ─────> Timeline editor + Live reactive mode (both complete first)

MVP Recommendation

Build in this order — each tier unblocks the next:

Tier 1 — Foundation (without this, nothing works):

  1. Audio file loading and playback (MP3/FLAC/WAV)
  2. Play / pause / seek transport
  3. Per-zone control model (WS2801 + SK6812 as named zones)
  4. Animation library (8–12 built-in effects with parameter config)
  5. Pi → ESP JSON command protocol

Tier 2 — Core Editor (the actual product): 6. Audio waveform display in timeline 7. Block-based timeline editor (place/move/resize/delete animation blocks) 8. Manual timing mark placement (tapper) 9. Loop / repeat per block 10. Save / load choreography (JSON)

Tier 3 — Live Reactive (second major mode): 11. System audio capture (PipeWire/PulseAudio) 12. Beat / onset detection 13. Live reactive mode (beat → trigger animation)

Tier 4 — Polish (quality of life): 14. Undo / redo 15. Zone-aware beat-reactive routing 16. Auto-beat detection timing marks (librosa/aubio) 17. Per-block transition effects

Defer indefinitely:

  • Spotify integration (very high complexity, unclear if system audio loopback is simpler)
  • Animation preview in-terminal (nice to have, not critical when you have physical LEDs)
  • Export/import xLights sequences

Key Observations

What xLights does that this tool must do:

  • Waveform display with zoom and timing marks is non-negotiable for manual sync work
  • Timing tracks with auto-generated beat marks (xLights uses VAMP plugins; Python has librosa.beat.beat_track) dramatically speed up choreography

What WLED does that informs ESP firmware design:

  • Segments = zones; presets = named animation configs; playlists = ordered preset sequences with timing
  • The WLED mental model (preset per zone, triggered by command) closely matches what this project's JSON protocol needs

What LedFx does that informs live reactive mode:

  • Melbank (mel-scale filterbank) over raw FFT for audio features is the right approach — perceptually uniform frequency buckets
  • Latency target for reactive mode: ~16ms audio-to-LED pipeline (LedFx achieves this)
  • Virtual device layering (effect → post-process → device) is a useful architecture even for two zones

Latency reality check (MEDIUM confidence — community sources):

  • WiFi jitter on ESP32: ~12ms variance (acceptable for music sync at ~70ms total budget)
  • System audio monitoring buffer latency: 100–300ms (must be accounted for in beat detection offset)
  • Command-based approach (vs frame streaming) sidesteps most latency problems — the animation runs autonomously on ESP, only start/stop/param-change commands cross WiFi

Sources

Reference in New Issue View Git Blame Copy Permalink