# Feature Research

**Domain:** Music-to-light synchronization system (personal LED show controller)
**Researched:** 2026-04-05
**Confidence:** HIGH (cross-referenced across QLC+, xLights, SoundSwitch, Lightjams, WLED, Resolume, and Python/librosa ecosystem)

---

## Feature Landscape

### Table Stakes (Users Expect These)

Features that any music-synchronized light controller must have to feel complete. Missing
these causes the whole system to feel broken or amateur.

| Feature | Why Expected | Complexity | Notes |
|---------|--------------|------------|-------|
| Audio file loading + playback | You cannot sync to music without playing it | LOW | MPV subprocess handles this; need play/pause/seek/position |
| Waveform display | Every sequencer since the 90s shows the waveform; without it you're editing blind | MEDIUM | Render audio waveform as visual reference in timeline; requires audio decoding to samples |
| Timeline with per-device tracks | The core editing metaphor — every show editor (xLights, VenueMagic, QLC+ Show Manager) uses it | HIGH | Each device gets a horizontal track; effects are blocks placed on the track |
| Audio playback position cursor | Moving vertical line on timeline showing current playback position | LOW | Follows MPV position; drives what's rendered live |
| Effect blocks on timeline | Drag-and-place blocks representing an animation + its params | MEDIUM | Block = animation type + duration + params; resize/move/delete |
| Animation library | Preset animations to apply (chase, pulse, rainbow, strobe, etc.) | MEDIUM | Users pick from a list, configure per-block params (speed, color, direction) |
| Beat detection / timing marks | All major tools (xLights via VAMP, SoundSwitch, Lightjams) treat this as core | MEDIUM | librosa beat_track + onset_detect; results shown as marks on timeline |
| Snap-to-beat | Effects placed/resized should snap to detected beat marks | LOW | Once timing marks exist, snapping is a positioning assist; low complexity |
| Play a show live | The whole point — execute the timeline and send frames/commands to devices | MEDIUM | Follows playback position, dispatches UDP packets at correct timestamps |
| Device registry | Name, type, LED count, IP, port — without this, you can't address anything | LOW | Static config; a JSON file or DB table |
| Show save / load | Persist your work as JSON so you don't lose it between sessions | LOW | Timeline state → JSON; all sequencers support this |
| Color picker per effect block | Choose colors for an animation block directly in the UI | LOW | Standard HTML color input; feeds into animation params |
| Basic undo/redo | Lightkey markets this as a selling point because so many tools lack it | MEDIUM | Command pattern on timeline mutations; minimal viable: last 20 actions |

### Differentiators (Competitive Advantage)

Features that elevate the tool above the baseline. These align with the LightSync core
value: precise control + automated intelligence.

| Feature | Value Proposition | Complexity | Notes |
|---------|-------------------|------------|-------|
| YouTube URL loading via yt-dlp | No other personal tool does this cleanly; commercial tools don't touch it | LOW | Subprocess yt-dlp + MPV pipe; already decided in PROJECT.md |
| Structural segmentation overlay | librosa can detect verse/chorus/bridge boundaries — display as colored regions on waveform | HIGH | librosa.segment + recurrence matrix; surfaces song structure for manual cue placement |
| AI-assisted show generation | Given beat/segment analysis, auto-fill a timeline with matching animations | HIGH | Post-processing step on top of beat detection; requires designing a mapping heuristic or LLM prompt |
| Dual UDP protocol (command vs frames) | Low-bandwidth animation commands for simple effects; raw RGBW frames for complex AI sequences | MEDIUM | Two encoders: one sends (animation_id, params), other sends per-frame pixel arrays |
| Live preview panel | Show a simulated LED strip rendering in the browser as playback runs | MEDIUM | Canvas/WebGL; render animation frames based on current timeline state without needing hardware |
| Parameter animation curves | Let effect params (speed, brightness) vary over the block's duration using envelope curves | HIGH | Think DAW automation lanes; very powerful but complex to implement and edit in UI |
| Frequency band mapping | Map bass / mid / treble energy to different devices or effect params | HIGH | librosa spectral analysis per frame; enables bass-driven floor lights, treble-driven ceiling flicker |
| Effect grouping / scenes | Group multiple device blocks into a named scene, re-use it across the timeline | MEDIUM | Reduces repetition when building multi-device shows |
| Show export for standalone playback | Export a show as a self-contained playback file + firmware-flashable sequence | HIGH | Firmware deferred; but JSON export format should support offline playback eventually |
| Terminal/hacker aesthetic UI | Differentiates visually from every off-the-shelf DMX tool | LOW | CSS/font work; monospace, dark theme, CRT vibes — already decided in PROJECT.md |

### Anti-Features (Commonly Requested, Often Problematic)

Explicitly what NOT to build for a personal single-operator tool.

| Feature | Why Requested | Why Problematic | Alternative |
|---------|---------------|-----------------|-------------|
| Real-time microphone input | Feels cool, "reactive" | Adds latency-sensitive audio pipeline, conflicts with file-based sync model, requires AGC tuning | Stay file/URL based; deterministic playback always wins for show editing |
| Multi-user collaborative editing | "What if a friend wants to help?" | Requires conflict resolution, WebSocket state sync, presence — enormous complexity for zero personal-use value | Single-operator; share the JSON file if collaboration is needed |
| Cloud sync / remote access | "Access from phone" | Adds auth, hosted backend, security surface — out of scope per PROJECT.md | Local network only; Traefik + Authelia provides access from the local network |
| MIDI controller hardware support | DJs use it; feels pro | Hardware dependency, OS driver layer, latency calibration — high complexity for a personal show builder | Beat detection handles what MIDI would be used for in this context |
| Plugin / extension system | "Make it extensible" | Proper plugin APIs are complex to design well; versioning, sandboxing, discovery | Codebase is open; add effects directly in Python. Modular device abstraction covers extensibility |
| 3D fixture positioning / visualizer | Professional VJs use Resolume Arena / L8 for this | Massive scope; requires a 3D renderer, fixture database, spatial math | 2D live preview panel is sufficient for LED strips |
| DMX universe / ArtNet / E1.31 | Professional standard; QLC+ and xLights support it | Adds protocol translation layer; LightSync uses direct UDP to microcontrollers — simpler and lower latency | Custom UDP protocol is more direct and lower overhead for this scale |
| Scheduling / playlist / show runner | Light-O-Rama and VenueMagic have this | Automated playlists make sense for installation art / Christmas shows, not interactive personal shows | Manual launch from the web UI is the right model here |
| MIDI clock sync with DAW | Ableton Link, Lightkey advertise this | No DAW in this workflow; adds dependency on external BPM source | librosa beat detection is fully offline and self-contained |

---

## Feature Dependencies

```
Audio Playback (MPV subprocess)
    └──enables──> Waveform Display
    └──enables──> Beat Detection / Timing Marks
                      └──enables──> Snap-to-Beat
                      └──enables──> AI-Assisted Show Generation
                      └──enables──> Structural Segmentation Overlay

Device Registry
    └──required by──> Live Show Execution (UDP dispatch)
    └──required by──> Live Preview Panel

Timeline Editor (tracks + effect blocks)
    └──requires──> Animation Library (something to place)
    └──requires──> Device Registry (what to assign blocks to)
    └──enables──> Show Save/Load
    └──enables──> AI-Assisted Show Generation (fills the timeline)

Animation Library
    └──enables──> Effect Blocks on Timeline
    └──enhanced by──> Color Picker per Block
    └──enhanced by──> Parameter Animation Curves

Live Show Execution
    └──requires──> Timeline (what to execute)
    └──requires──> Device Registry (where to send)
    └──requires──> Audio Playback (position reference)
    └──enhanced by──> Dual UDP Protocol (command vs frames)

Live Preview Panel
    └──requires──> Animation Library (rendering logic)
    └──requires──> Device Registry (LED counts / layout)
    └──enhanced by──> Live Show Execution (synced preview)

Beat Detection
    └──enhances──> Timeline Editor (snap grid)
    └──enhances──> AI Show Generation (cue placement)

Frequency Band Mapping
    └──requires──> Beat Detection (same audio analysis pass)
    └──conflicts with──> Simple show model (adds per-frame complexity)
```

### Dependency Notes

- **Beat detection requires audio playback to be loaded first** — analysis runs on the audio file at load or on-demand; cannot run without the file.
- **AI show generation requires beat detection** — it uses timing marks as anchor points; generating without them produces incoherent results.
- **Live show execution requires device registry** — without knowing IP/port/LED count, UDP packets can't be formed.
- **Live preview panel requires animation rendering logic** — the same code that renders frames for UDP output should power the preview; they must share a renderer.
- **Dual UDP protocol requires clear protocol spec first** — command mode and frame mode must be designed together before implementing either.
- **Undo/redo must be designed early** — retrofitting undo into a stateful timeline editor is painful; build the command pattern from the first timeline operation.

---

## MVP Definition

### Launch With (v1)

Minimum viable for the system to be useful at all.

- [ ] Audio file loading + MPV playback (play/pause/seek) — without this nothing works
- [ ] Waveform display — required for editing without guessing
- [ ] Beat detection via librosa — auto-generates timing marks on the timeline
- [ ] Device registry (static JSON config) — at least 2 devices (SK6812 + WS2801)
- [ ] Animation library with 8-12 core effects (rainbow, chase, pulse, strobe, color fill, sparkle, fade in/out, wipe) — minimum useful palette
- [ ] Timeline editor: per-device tracks, drag-and-place effect blocks, snap-to-beat
- [ ] Color picker + basic params (speed, brightness, direction) per effect block
- [ ] Show save / load as JSON
- [ ] UDP command sender — dispatch animation + params at the right timestamp
- [ ] Undo/redo for timeline mutations — build from day one or regret it

### Add After Validation (v1.x)

Add once the core editing loop works and you've built a real show with it.

- [ ] YouTube URL loading via yt-dlp — once local-file workflow is proven
- [ ] Live preview panel — valuable once you have a library of shows to tweak
- [ ] Structural segmentation overlay — once beat marks aren't enough
- [ ] UDP frame sender (raw pixel data) — needed for AI-generated sequences
- [ ] AI-assisted show generation — on top of the raw frame pipeline
- [ ] Effect grouping / scenes — once shows are complex enough to need it

### Future Consideration (v2+)

Defer until v1 is regularly used and the pain points are clear.

- [ ] Parameter animation curves — powerful but high editor complexity; wait until basic params feel limiting
- [ ] Frequency band mapping — adds real-time analysis path; revisit when advanced reactive effects are wanted
- [ ] Moving heads / laser support — requires extending device abstraction; build only when hardware exists
- [ ] Fog machine control — simple on/off trigger; easy to add when hardware is present

---

## Feature Prioritization Matrix

| Feature | User Value | Implementation Cost | Priority |
|---------|------------|---------------------|----------|
| Audio playback (MPV) | HIGH | LOW | P1 |
| Waveform display | HIGH | MEDIUM | P1 |
| Beat detection (librosa) | HIGH | MEDIUM | P1 |
| Timeline editor | HIGH | HIGH | P1 |
| Animation library (core 8-12) | HIGH | MEDIUM | P1 |
| Device registry | HIGH | LOW | P1 |
| Show save/load | HIGH | LOW | P1 |
| UDP command sender | HIGH | MEDIUM | P1 |
| Undo/redo | MEDIUM | MEDIUM | P1 |
| Snap-to-beat | HIGH | LOW | P1 |
| Color picker per block | MEDIUM | LOW | P1 |
| YouTube loading (yt-dlp) | HIGH | LOW | P2 |
| Live preview panel | HIGH | MEDIUM | P2 |
| Structural segmentation overlay | MEDIUM | HIGH | P2 |
| UDP frame sender | MEDIUM | MEDIUM | P2 |
| AI show generation | HIGH | HIGH | P2 |
| Effect grouping / scenes | MEDIUM | MEDIUM | P2 |
| Parameter animation curves | MEDIUM | HIGH | P3 |
| Frequency band mapping | MEDIUM | HIGH | P3 |
| Moving head / laser support | LOW | MEDIUM | P3 |

**Priority key:**
- P1: Must have for launch
- P2: Should have, add after core is validated
- P3: Nice to have, future consideration

---

## Competitor Feature Analysis

| Feature | xLights | QLC+ | SoundSwitch | LightSync Approach |
|---------|---------|------|-------------|-------------------|
| Timeline editor | Yes — DAW-style, per-model tracks | Yes — Show Manager | No (real-time only) | Yes — per-device tracks, effect blocks |
| Beat detection | Yes — VAMP plugins | No built-in | Yes — beatgrid sync | Yes — librosa (offline, no plugin install) |
| Audio waveform | Yes — sonographic display | No | N/A | Yes — rendered from audio file |
| Snap to beat | Yes — snaps to timing marks | N/A | N/A | Yes — on placement + resize |
| Animation library | Yes — 100+ effects | Yes — scenes/functions | Yes — autoloops (32 built-in) | Yes — curated set (~12 core, extensible) |
| AI show generation | No | No | Phrase detection (limited) | Yes — planned differentiator |
| Structural segmentation | No | No | No | Yes — librosa segment analysis |
| Live preview | Yes — 2D/3D layout | Yes — fixture map | No | Yes — canvas strip simulation |
| Custom protocol | E1.31 / ArtNet / DDP | DMX / ArtNet | MIDI + DMX | Custom UDP (command + frame modes) |
| YouTube support | No | No | No | Yes — yt-dlp passthrough |
| Undo/redo | Partial (some operations) | Limited | N/A | Yes — command pattern from day 1 |
| Personal tool scale | Overkill (100+ universes) | Overkill (full DMX) | Wrong use case (DJ) | Right scale — 2 strips + future devices |

---

## Sources

- [QLC+ Features page](https://www.qlcplus.org/discover/features) — DMX feature set, Show Manager, RGB Matrix
- [xLights Manual — Timeline and Waveform](https://manual.xlights.org/xlights/chapters/chapter-four-sequencer/timeline-and-waveform) — zoom, playback, waveform display
- [xLights Manual — Timing Tracks](https://manual.xlights.org/xlights/chapters/chapter-four-sequencer/timing-tracks) — VAMP plugins, beat detection, snap behavior
- [SoundSwitch — What's New in 2.9](https://support.soundswitch.com/en/support/solutions/articles/69000858481-soundswitch-what-s-new-in-soundswitch-2-9-) — BPM detection, phrase detection, autoloops
- [WLED Audio Reactive documentation](https://kno.wled.ge/advanced/audio-reactive/) — 30+ sound-reactive effects, audio input options
- [Lightjams — Sync Music and Lights](https://www.lightjams.com/musicToDMX.html) — 120 frequency bands, beat/mid/treble energy
- [Resolume Avenue/Arena](https://resolume.com/software/avenue-arena) — FFT audio analysis, BPM sync, LED mapping
- [librosa documentation](https://librosa.org/doc/0.11.0/tutorial.html) — beat tracking, onset detection, structural segmentation
- [WLED UDP Realtime Control](https://github.com/Aircoookie/WLED/wiki/UDP-Realtime-Control) — protocol patterns, FPS guidance (15-30 recommended)
- [VenueMagic Show Control](https://www.venuemagic.com/show-control-software/) — drag-drop timeline, beat tapper
- [DMXDesktop Effect Types](https://www.dmxdesktop.com/knowledgebase/effect-types) — chase, sparkle, pulse, strobe definitions

---

*Feature research for: music-to-light synchronization system (LightSync)*
*Researched: 2026-04-05*