232 lines
16 KiB
Markdown
232 lines
16 KiB
Markdown
# Feature Research
|
|
|
|
**Domain:** Music-to-light synchronization system (personal LED show controller)
|
|
**Researched:** 2026-04-05
|
|
**Confidence:** HIGH (cross-referenced across QLC+, xLights, SoundSwitch, Lightjams, WLED, Resolume, and Python/librosa ecosystem)
|
|
|
|
---
|
|
|
|
## Feature Landscape
|
|
|
|
### Table Stakes (Users Expect These)
|
|
|
|
Features that any music-synchronized light controller must have to feel complete. Missing
|
|
these causes the whole system to feel broken or amateur.
|
|
|
|
| Feature | Why Expected | Complexity | Notes |
|
|
|---------|--------------|------------|-------|
|
|
| Audio file loading + playback | You cannot sync to music without playing it | LOW | MPV subprocess handles this; need play/pause/seek/position |
|
|
| Waveform display | Every sequencer since the 90s shows the waveform; without it you're editing blind | MEDIUM | Render audio waveform as visual reference in timeline; requires audio decoding to samples |
|
|
| Timeline with per-device tracks | The core editing metaphor — every show editor (xLights, VenueMagic, QLC+ Show Manager) uses it | HIGH | Each device gets a horizontal track; effects are blocks placed on the track |
|
|
| Audio playback position cursor | Moving vertical line on timeline showing current playback position | LOW | Follows MPV position; drives what's rendered live |
|
|
| Effect blocks on timeline | Drag-and-place blocks representing an animation + its params | MEDIUM | Block = animation type + duration + params; resize/move/delete |
|
|
| Animation library | Preset animations to apply (chase, pulse, rainbow, strobe, etc.) | MEDIUM | Users pick from a list, configure per-block params (speed, color, direction) |
|
|
| Beat detection / timing marks | All major tools (xLights via VAMP, SoundSwitch, Lightjams) treat this as core | MEDIUM | librosa beat_track + onset_detect; results shown as marks on timeline |
|
|
| Snap-to-beat | Effects placed/resized should snap to detected beat marks | LOW | Once timing marks exist, snapping is a positioning assist; low complexity |
|
|
| Play a show live | The whole point — execute the timeline and send frames/commands to devices | MEDIUM | Follows playback position, dispatches UDP packets at correct timestamps |
|
|
| Device registry | Name, type, LED count, IP, port — without this, you can't address anything | LOW | Static config; a JSON file or DB table |
|
|
| Show save / load | Persist your work as JSON so you don't lose it between sessions | LOW | Timeline state → JSON; all sequencers support this |
|
|
| Color picker per effect block | Choose colors for an animation block directly in the UI | LOW | Standard HTML color input; feeds into animation params |
|
|
| Basic undo/redo | Lightkey markets this as a selling point because so many tools lack it | MEDIUM | Command pattern on timeline mutations; minimal viable: last 20 actions |
|
|
|
|
### Differentiators (Competitive Advantage)
|
|
|
|
Features that elevate the tool above the baseline. These align with the LightSync core
|
|
value: precise control + automated intelligence.
|
|
|
|
| Feature | Value Proposition | Complexity | Notes |
|
|
|---------|-------------------|------------|-------|
|
|
| YouTube URL loading via yt-dlp | No other personal tool does this cleanly; commercial tools don't touch it | LOW | Subprocess yt-dlp + MPV pipe; already decided in PROJECT.md |
|
|
| Structural segmentation overlay | librosa can detect verse/chorus/bridge boundaries — display as colored regions on waveform | HIGH | librosa.segment + recurrence matrix; surfaces song structure for manual cue placement |
|
|
| AI-assisted show generation | Given beat/segment analysis, auto-fill a timeline with matching animations | HIGH | Post-processing step on top of beat detection; requires designing a mapping heuristic or LLM prompt |
|
|
| Dual UDP protocol (command vs frames) | Low-bandwidth animation commands for simple effects; raw RGBW frames for complex AI sequences | MEDIUM | Two encoders: one sends (animation_id, params), other sends per-frame pixel arrays |
|
|
| Live preview panel | Show a simulated LED strip rendering in the browser as playback runs | MEDIUM | Canvas/WebGL; render animation frames based on current timeline state without needing hardware |
|
|
| Parameter animation curves | Let effect params (speed, brightness) vary over the block's duration using envelope curves | HIGH | Think DAW automation lanes; very powerful but complex to implement and edit in UI |
|
|
| Frequency band mapping | Map bass / mid / treble energy to different devices or effect params | HIGH | librosa spectral analysis per frame; enables bass-driven floor lights, treble-driven ceiling flicker |
|
|
| Effect grouping / scenes | Group multiple device blocks into a named scene, re-use it across the timeline | MEDIUM | Reduces repetition when building multi-device shows |
|
|
| Show export for standalone playback | Export a show as a self-contained playback file + firmware-flashable sequence | HIGH | Firmware deferred; but JSON export format should support offline playback eventually |
|
|
| Terminal/hacker aesthetic UI | Differentiates visually from every off-the-shelf DMX tool | LOW | CSS/font work; monospace, dark theme, CRT vibes — already decided in PROJECT.md |
|
|
|
|
### Anti-Features (Commonly Requested, Often Problematic)
|
|
|
|
Explicitly what NOT to build for a personal single-operator tool.
|
|
|
|
| Feature | Why Requested | Why Problematic | Alternative |
|
|
|---------|---------------|-----------------|-------------|
|
|
| Real-time microphone input | Feels cool, "reactive" | Adds latency-sensitive audio pipeline, conflicts with file-based sync model, requires AGC tuning | Stay file/URL based; deterministic playback always wins for show editing |
|
|
| Multi-user collaborative editing | "What if a friend wants to help?" | Requires conflict resolution, WebSocket state sync, presence — enormous complexity for zero personal-use value | Single-operator; share the JSON file if collaboration is needed |
|
|
| Cloud sync / remote access | "Access from phone" | Adds auth, hosted backend, security surface — out of scope per PROJECT.md | Local network only; Traefik + Authelia provides access from the local network |
|
|
| MIDI controller hardware support | DJs use it; feels pro | Hardware dependency, OS driver layer, latency calibration — high complexity for a personal show builder | Beat detection handles what MIDI would be used for in this context |
|
|
| Plugin / extension system | "Make it extensible" | Proper plugin APIs are complex to design well; versioning, sandboxing, discovery | Codebase is open; add effects directly in Python. Modular device abstraction covers extensibility |
|
|
| 3D fixture positioning / visualizer | Professional VJs use Resolume Arena / L8 for this | Massive scope; requires a 3D renderer, fixture database, spatial math | 2D live preview panel is sufficient for LED strips |
|
|
| DMX universe / ArtNet / E1.31 | Professional standard; QLC+ and xLights support it | Adds protocol translation layer; LightSync uses direct UDP to microcontrollers — simpler and lower latency | Custom UDP protocol is more direct and lower overhead for this scale |
|
|
| Scheduling / playlist / show runner | Light-O-Rama and VenueMagic have this | Automated playlists make sense for installation art / Christmas shows, not interactive personal shows | Manual launch from the web UI is the right model here |
|
|
| MIDI clock sync with DAW | Ableton Link, Lightkey advertise this | No DAW in this workflow; adds dependency on external BPM source | librosa beat detection is fully offline and self-contained |
|
|
|
|
---
|
|
|
|
## Feature Dependencies
|
|
|
|
```
|
|
Audio Playback (MPV subprocess)
|
|
└──enables──> Waveform Display
|
|
└──enables──> Beat Detection / Timing Marks
|
|
└──enables──> Snap-to-Beat
|
|
└──enables──> AI-Assisted Show Generation
|
|
└──enables──> Structural Segmentation Overlay
|
|
|
|
Device Registry
|
|
└──required by──> Live Show Execution (UDP dispatch)
|
|
└──required by──> Live Preview Panel
|
|
|
|
Timeline Editor (tracks + effect blocks)
|
|
└──requires──> Animation Library (something to place)
|
|
└──requires──> Device Registry (what to assign blocks to)
|
|
└──enables──> Show Save/Load
|
|
└──enables──> AI-Assisted Show Generation (fills the timeline)
|
|
|
|
Animation Library
|
|
└──enables──> Effect Blocks on Timeline
|
|
└──enhanced by──> Color Picker per Block
|
|
└──enhanced by──> Parameter Animation Curves
|
|
|
|
Live Show Execution
|
|
└──requires──> Timeline (what to execute)
|
|
└──requires──> Device Registry (where to send)
|
|
└──requires──> Audio Playback (position reference)
|
|
└──enhanced by──> Dual UDP Protocol (command vs frames)
|
|
|
|
Live Preview Panel
|
|
└──requires──> Animation Library (rendering logic)
|
|
└──requires──> Device Registry (LED counts / layout)
|
|
└──enhanced by──> Live Show Execution (synced preview)
|
|
|
|
Beat Detection
|
|
└──enhances──> Timeline Editor (snap grid)
|
|
└──enhances──> AI Show Generation (cue placement)
|
|
|
|
Frequency Band Mapping
|
|
└──requires──> Beat Detection (same audio analysis pass)
|
|
└──conflicts with──> Simple show model (adds per-frame complexity)
|
|
```
|
|
|
|
### Dependency Notes
|
|
|
|
- **Beat detection requires audio playback to be loaded first** — analysis runs on the audio file at load or on-demand; cannot run without the file.
|
|
- **AI show generation requires beat detection** — it uses timing marks as anchor points; generating without them produces incoherent results.
|
|
- **Live show execution requires device registry** — without knowing IP/port/LED count, UDP packets can't be formed.
|
|
- **Live preview panel requires animation rendering logic** — the same code that renders frames for UDP output should power the preview; they must share a renderer.
|
|
- **Dual UDP protocol requires clear protocol spec first** — command mode and frame mode must be designed together before implementing either.
|
|
- **Undo/redo must be designed early** — retrofitting undo into a stateful timeline editor is painful; build the command pattern from the first timeline operation.
|
|
|
|
---
|
|
|
|
## MVP Definition
|
|
|
|
### Launch With (v1)
|
|
|
|
Minimum viable for the system to be useful at all.
|
|
|
|
- [ ] Audio file loading + MPV playback (play/pause/seek) — without this nothing works
|
|
- [ ] Waveform display — required for editing without guessing
|
|
- [ ] Beat detection via librosa — auto-generates timing marks on the timeline
|
|
- [ ] Device registry (static JSON config) — at least 2 devices (SK6812 + WS2801)
|
|
- [ ] Animation library with 8-12 core effects (rainbow, chase, pulse, strobe, color fill, sparkle, fade in/out, wipe) — minimum useful palette
|
|
- [ ] Timeline editor: per-device tracks, drag-and-place effect blocks, snap-to-beat
|
|
- [ ] Color picker + basic params (speed, brightness, direction) per effect block
|
|
- [ ] Show save / load as JSON
|
|
- [ ] UDP command sender — dispatch animation + params at the right timestamp
|
|
- [ ] Undo/redo for timeline mutations — build from day one or regret it
|
|
|
|
### Add After Validation (v1.x)
|
|
|
|
Add once the core editing loop works and you've built a real show with it.
|
|
|
|
- [ ] YouTube URL loading via yt-dlp — once local-file workflow is proven
|
|
- [ ] Live preview panel — valuable once you have a library of shows to tweak
|
|
- [ ] Structural segmentation overlay — once beat marks aren't enough
|
|
- [ ] UDP frame sender (raw pixel data) — needed for AI-generated sequences
|
|
- [ ] AI-assisted show generation — on top of the raw frame pipeline
|
|
- [ ] Effect grouping / scenes — once shows are complex enough to need it
|
|
|
|
### Future Consideration (v2+)
|
|
|
|
Defer until v1 is regularly used and the pain points are clear.
|
|
|
|
- [ ] Parameter animation curves — powerful but high editor complexity; wait until basic params feel limiting
|
|
- [ ] Frequency band mapping — adds real-time analysis path; revisit when advanced reactive effects are wanted
|
|
- [ ] Moving heads / laser support — requires extending device abstraction; build only when hardware exists
|
|
- [ ] Fog machine control — simple on/off trigger; easy to add when hardware is present
|
|
|
|
---
|
|
|
|
## Feature Prioritization Matrix
|
|
|
|
| Feature | User Value | Implementation Cost | Priority |
|
|
|---------|------------|---------------------|----------|
|
|
| Audio playback (MPV) | HIGH | LOW | P1 |
|
|
| Waveform display | HIGH | MEDIUM | P1 |
|
|
| Beat detection (librosa) | HIGH | MEDIUM | P1 |
|
|
| Timeline editor | HIGH | HIGH | P1 |
|
|
| Animation library (core 8-12) | HIGH | MEDIUM | P1 |
|
|
| Device registry | HIGH | LOW | P1 |
|
|
| Show save/load | HIGH | LOW | P1 |
|
|
| UDP command sender | HIGH | MEDIUM | P1 |
|
|
| Undo/redo | MEDIUM | MEDIUM | P1 |
|
|
| Snap-to-beat | HIGH | LOW | P1 |
|
|
| Color picker per block | MEDIUM | LOW | P1 |
|
|
| YouTube loading (yt-dlp) | HIGH | LOW | P2 |
|
|
| Live preview panel | HIGH | MEDIUM | P2 |
|
|
| Structural segmentation overlay | MEDIUM | HIGH | P2 |
|
|
| UDP frame sender | MEDIUM | MEDIUM | P2 |
|
|
| AI show generation | HIGH | HIGH | P2 |
|
|
| Effect grouping / scenes | MEDIUM | MEDIUM | P2 |
|
|
| Parameter animation curves | MEDIUM | HIGH | P3 |
|
|
| Frequency band mapping | MEDIUM | HIGH | P3 |
|
|
| Moving head / laser support | LOW | MEDIUM | P3 |
|
|
|
|
**Priority key:**
|
|
- P1: Must have for launch
|
|
- P2: Should have, add after core is validated
|
|
- P3: Nice to have, future consideration
|
|
|
|
---
|
|
|
|
## Competitor Feature Analysis
|
|
|
|
| Feature | xLights | QLC+ | SoundSwitch | LightSync Approach |
|
|
|---------|---------|------|-------------|-------------------|
|
|
| Timeline editor | Yes — DAW-style, per-model tracks | Yes — Show Manager | No (real-time only) | Yes — per-device tracks, effect blocks |
|
|
| Beat detection | Yes — VAMP plugins | No built-in | Yes — beatgrid sync | Yes — librosa (offline, no plugin install) |
|
|
| Audio waveform | Yes — sonographic display | No | N/A | Yes — rendered from audio file |
|
|
| Snap to beat | Yes — snaps to timing marks | N/A | N/A | Yes — on placement + resize |
|
|
| Animation library | Yes — 100+ effects | Yes — scenes/functions | Yes — autoloops (32 built-in) | Yes — curated set (~12 core, extensible) |
|
|
| AI show generation | No | No | Phrase detection (limited) | Yes — planned differentiator |
|
|
| Structural segmentation | No | No | No | Yes — librosa segment analysis |
|
|
| Live preview | Yes — 2D/3D layout | Yes — fixture map | No | Yes — canvas strip simulation |
|
|
| Custom protocol | E1.31 / ArtNet / DDP | DMX / ArtNet | MIDI + DMX | Custom UDP (command + frame modes) |
|
|
| YouTube support | No | No | No | Yes — yt-dlp passthrough |
|
|
| Undo/redo | Partial (some operations) | Limited | N/A | Yes — command pattern from day 1 |
|
|
| Personal tool scale | Overkill (100+ universes) | Overkill (full DMX) | Wrong use case (DJ) | Right scale — 2 strips + future devices |
|
|
|
|
---
|
|
|
|
## Sources
|
|
|
|
- [QLC+ Features page](https://www.qlcplus.org/discover/features) — DMX feature set, Show Manager, RGB Matrix
|
|
- [xLights Manual — Timeline and Waveform](https://manual.xlights.org/xlights/chapters/chapter-four-sequencer/timeline-and-waveform) — zoom, playback, waveform display
|
|
- [xLights Manual — Timing Tracks](https://manual.xlights.org/xlights/chapters/chapter-four-sequencer/timing-tracks) — VAMP plugins, beat detection, snap behavior
|
|
- [SoundSwitch — What's New in 2.9](https://support.soundswitch.com/en/support/solutions/articles/69000858481-soundswitch-what-s-new-in-soundswitch-2-9-) — BPM detection, phrase detection, autoloops
|
|
- [WLED Audio Reactive documentation](https://kno.wled.ge/advanced/audio-reactive/) — 30+ sound-reactive effects, audio input options
|
|
- [Lightjams — Sync Music and Lights](https://www.lightjams.com/musicToDMX.html) — 120 frequency bands, beat/mid/treble energy
|
|
- [Resolume Avenue/Arena](https://resolume.com/software/avenue-arena) — FFT audio analysis, BPM sync, LED mapping
|
|
- [librosa documentation](https://librosa.org/doc/0.11.0/tutorial.html) — beat tracking, onset detection, structural segmentation
|
|
- [WLED UDP Realtime Control](https://github.com/Aircoookie/WLED/wiki/UDP-Realtime-Control) — protocol patterns, FPS guidance (15-30 recommended)
|
|
- [VenueMagic Show Control](https://www.venuemagic.com/show-control-software/) — drag-drop timeline, beat tapper
|
|
- [DMXDesktop Effect Types](https://www.dmxdesktop.com/knowledgebase/effect-types) — chase, sparkle, pulse, strobe definitions
|
|
|
|
---
|
|
|
|
*Feature research for: music-to-light synchronization system (LightSync)*
|
|
*Researched: 2026-04-05*
|