16 KiB
16 KiB
Feature Research
Domain: Music-to-light synchronization system (personal LED show controller) Researched: 2026-04-05 Confidence: HIGH (cross-referenced across QLC+, xLights, SoundSwitch, Lightjams, WLED, Resolume, and Python/librosa ecosystem)
Feature Landscape
Table Stakes (Users Expect These)
Features that any music-synchronized light controller must have to feel complete. Missing these causes the whole system to feel broken or amateur.
| Feature | Why Expected | Complexity | Notes |
|---|---|---|---|
| Audio file loading + playback | You cannot sync to music without playing it | LOW | MPV subprocess handles this; need play/pause/seek/position |
| Waveform display | Every sequencer since the 90s shows the waveform; without it you're editing blind | MEDIUM | Render audio waveform as visual reference in timeline; requires audio decoding to samples |
| Timeline with per-device tracks | The core editing metaphor — every show editor (xLights, VenueMagic, QLC+ Show Manager) uses it | HIGH | Each device gets a horizontal track; effects are blocks placed on the track |
| Audio playback position cursor | Moving vertical line on timeline showing current playback position | LOW | Follows MPV position; drives what's rendered live |
| Effect blocks on timeline | Drag-and-place blocks representing an animation + its params | MEDIUM | Block = animation type + duration + params; resize/move/delete |
| Animation library | Preset animations to apply (chase, pulse, rainbow, strobe, etc.) | MEDIUM | Users pick from a list, configure per-block params (speed, color, direction) |
| Beat detection / timing marks | All major tools (xLights via VAMP, SoundSwitch, Lightjams) treat this as core | MEDIUM | librosa beat_track + onset_detect; results shown as marks on timeline |
| Snap-to-beat | Effects placed/resized should snap to detected beat marks | LOW | Once timing marks exist, snapping is a positioning assist; low complexity |
| Play a show live | The whole point — execute the timeline and send frames/commands to devices | MEDIUM | Follows playback position, dispatches UDP packets at correct timestamps |
| Device registry | Name, type, LED count, IP, port — without this, you can't address anything | LOW | Static config; a JSON file or DB table |
| Show save / load | Persist your work as JSON so you don't lose it between sessions | LOW | Timeline state → JSON; all sequencers support this |
| Color picker per effect block | Choose colors for an animation block directly in the UI | LOW | Standard HTML color input; feeds into animation params |
| Basic undo/redo | Lightkey markets this as a selling point because so many tools lack it | MEDIUM | Command pattern on timeline mutations; minimal viable: last 20 actions |
Differentiators (Competitive Advantage)
Features that elevate the tool above the baseline. These align with the LightSync core value: precise control + automated intelligence.
| Feature | Value Proposition | Complexity | Notes |
|---|---|---|---|
| YouTube URL loading via yt-dlp | No other personal tool does this cleanly; commercial tools don't touch it | LOW | Subprocess yt-dlp + MPV pipe; already decided in PROJECT.md |
| Structural segmentation overlay | librosa can detect verse/chorus/bridge boundaries — display as colored regions on waveform | HIGH | librosa.segment + recurrence matrix; surfaces song structure for manual cue placement |
| AI-assisted show generation | Given beat/segment analysis, auto-fill a timeline with matching animations | HIGH | Post-processing step on top of beat detection; requires designing a mapping heuristic or LLM prompt |
| Dual UDP protocol (command vs frames) | Low-bandwidth animation commands for simple effects; raw RGBW frames for complex AI sequences | MEDIUM | Two encoders: one sends (animation_id, params), other sends per-frame pixel arrays |
| Live preview panel | Show a simulated LED strip rendering in the browser as playback runs | MEDIUM | Canvas/WebGL; render animation frames based on current timeline state without needing hardware |
| Parameter animation curves | Let effect params (speed, brightness) vary over the block's duration using envelope curves | HIGH | Think DAW automation lanes; very powerful but complex to implement and edit in UI |
| Frequency band mapping | Map bass / mid / treble energy to different devices or effect params | HIGH | librosa spectral analysis per frame; enables bass-driven floor lights, treble-driven ceiling flicker |
| Effect grouping / scenes | Group multiple device blocks into a named scene, re-use it across the timeline | MEDIUM | Reduces repetition when building multi-device shows |
| Show export for standalone playback | Export a show as a self-contained playback file + firmware-flashable sequence | HIGH | Firmware deferred; but JSON export format should support offline playback eventually |
| Terminal/hacker aesthetic UI | Differentiates visually from every off-the-shelf DMX tool | LOW | CSS/font work; monospace, dark theme, CRT vibes — already decided in PROJECT.md |
Anti-Features (Commonly Requested, Often Problematic)
Explicitly what NOT to build for a personal single-operator tool.
| Feature | Why Requested | Why Problematic | Alternative |
|---|---|---|---|
| Real-time microphone input | Feels cool, "reactive" | Adds latency-sensitive audio pipeline, conflicts with file-based sync model, requires AGC tuning | Stay file/URL based; deterministic playback always wins for show editing |
| Multi-user collaborative editing | "What if a friend wants to help?" | Requires conflict resolution, WebSocket state sync, presence — enormous complexity for zero personal-use value | Single-operator; share the JSON file if collaboration is needed |
| Cloud sync / remote access | "Access from phone" | Adds auth, hosted backend, security surface — out of scope per PROJECT.md | Local network only; Traefik + Authelia provides access from the local network |
| MIDI controller hardware support | DJs use it; feels pro | Hardware dependency, OS driver layer, latency calibration — high complexity for a personal show builder | Beat detection handles what MIDI would be used for in this context |
| Plugin / extension system | "Make it extensible" | Proper plugin APIs are complex to design well; versioning, sandboxing, discovery | Codebase is open; add effects directly in Python. Modular device abstraction covers extensibility |
| 3D fixture positioning / visualizer | Professional VJs use Resolume Arena / L8 for this | Massive scope; requires a 3D renderer, fixture database, spatial math | 2D live preview panel is sufficient for LED strips |
| DMX universe / ArtNet / E1.31 | Professional standard; QLC+ and xLights support it | Adds protocol translation layer; LightSync uses direct UDP to microcontrollers — simpler and lower latency | Custom UDP protocol is more direct and lower overhead for this scale |
| Scheduling / playlist / show runner | Light-O-Rama and VenueMagic have this | Automated playlists make sense for installation art / Christmas shows, not interactive personal shows | Manual launch from the web UI is the right model here |
| MIDI clock sync with DAW | Ableton Link, Lightkey advertise this | No DAW in this workflow; adds dependency on external BPM source | librosa beat detection is fully offline and self-contained |
Feature Dependencies
Audio Playback (MPV subprocess)
└──enables──> Waveform Display
└──enables──> Beat Detection / Timing Marks
└──enables──> Snap-to-Beat
└──enables──> AI-Assisted Show Generation
└──enables──> Structural Segmentation Overlay
Device Registry
└──required by──> Live Show Execution (UDP dispatch)
└──required by──> Live Preview Panel
Timeline Editor (tracks + effect blocks)
└──requires──> Animation Library (something to place)
└──requires──> Device Registry (what to assign blocks to)
└──enables──> Show Save/Load
└──enables──> AI-Assisted Show Generation (fills the timeline)
Animation Library
└──enables──> Effect Blocks on Timeline
└──enhanced by──> Color Picker per Block
└──enhanced by──> Parameter Animation Curves
Live Show Execution
└──requires──> Timeline (what to execute)
└──requires──> Device Registry (where to send)
└──requires──> Audio Playback (position reference)
└──enhanced by──> Dual UDP Protocol (command vs frames)
Live Preview Panel
└──requires──> Animation Library (rendering logic)
└──requires──> Device Registry (LED counts / layout)
└──enhanced by──> Live Show Execution (synced preview)
Beat Detection
└──enhances──> Timeline Editor (snap grid)
└──enhances──> AI Show Generation (cue placement)
Frequency Band Mapping
└──requires──> Beat Detection (same audio analysis pass)
└──conflicts with──> Simple show model (adds per-frame complexity)
Dependency Notes
- Beat detection requires audio playback to be loaded first — analysis runs on the audio file at load or on-demand; cannot run without the file.
- AI show generation requires beat detection — it uses timing marks as anchor points; generating without them produces incoherent results.
- Live show execution requires device registry — without knowing IP/port/LED count, UDP packets can't be formed.
- Live preview panel requires animation rendering logic — the same code that renders frames for UDP output should power the preview; they must share a renderer.
- Dual UDP protocol requires clear protocol spec first — command mode and frame mode must be designed together before implementing either.
- Undo/redo must be designed early — retrofitting undo into a stateful timeline editor is painful; build the command pattern from the first timeline operation.
MVP Definition
Launch With (v1)
Minimum viable for the system to be useful at all.
- Audio file loading + MPV playback (play/pause/seek) — without this nothing works
- Waveform display — required for editing without guessing
- Beat detection via librosa — auto-generates timing marks on the timeline
- Device registry (static JSON config) — at least 2 devices (SK6812 + WS2801)
- Animation library with 8-12 core effects (rainbow, chase, pulse, strobe, color fill, sparkle, fade in/out, wipe) — minimum useful palette
- Timeline editor: per-device tracks, drag-and-place effect blocks, snap-to-beat
- Color picker + basic params (speed, brightness, direction) per effect block
- Show save / load as JSON
- UDP command sender — dispatch animation + params at the right timestamp
- Undo/redo for timeline mutations — build from day one or regret it
Add After Validation (v1.x)
Add once the core editing loop works and you've built a real show with it.
- YouTube URL loading via yt-dlp — once local-file workflow is proven
- Live preview panel — valuable once you have a library of shows to tweak
- Structural segmentation overlay — once beat marks aren't enough
- UDP frame sender (raw pixel data) — needed for AI-generated sequences
- AI-assisted show generation — on top of the raw frame pipeline
- Effect grouping / scenes — once shows are complex enough to need it
Future Consideration (v2+)
Defer until v1 is regularly used and the pain points are clear.
- Parameter animation curves — powerful but high editor complexity; wait until basic params feel limiting
- Frequency band mapping — adds real-time analysis path; revisit when advanced reactive effects are wanted
- Moving heads / laser support — requires extending device abstraction; build only when hardware exists
- Fog machine control — simple on/off trigger; easy to add when hardware is present
Feature Prioritization Matrix
| Feature | User Value | Implementation Cost | Priority |
|---|---|---|---|
| Audio playback (MPV) | HIGH | LOW | P1 |
| Waveform display | HIGH | MEDIUM | P1 |
| Beat detection (librosa) | HIGH | MEDIUM | P1 |
| Timeline editor | HIGH | HIGH | P1 |
| Animation library (core 8-12) | HIGH | MEDIUM | P1 |
| Device registry | HIGH | LOW | P1 |
| Show save/load | HIGH | LOW | P1 |
| UDP command sender | HIGH | MEDIUM | P1 |
| Undo/redo | MEDIUM | MEDIUM | P1 |
| Snap-to-beat | HIGH | LOW | P1 |
| Color picker per block | MEDIUM | LOW | P1 |
| YouTube loading (yt-dlp) | HIGH | LOW | P2 |
| Live preview panel | HIGH | MEDIUM | P2 |
| Structural segmentation overlay | MEDIUM | HIGH | P2 |
| UDP frame sender | MEDIUM | MEDIUM | P2 |
| AI show generation | HIGH | HIGH | P2 |
| Effect grouping / scenes | MEDIUM | MEDIUM | P2 |
| Parameter animation curves | MEDIUM | HIGH | P3 |
| Frequency band mapping | MEDIUM | HIGH | P3 |
| Moving head / laser support | LOW | MEDIUM | P3 |
Priority key:
- P1: Must have for launch
- P2: Should have, add after core is validated
- P3: Nice to have, future consideration
Competitor Feature Analysis
| Feature | xLights | QLC+ | SoundSwitch | LightSync Approach |
|---|---|---|---|---|
| Timeline editor | Yes — DAW-style, per-model tracks | Yes — Show Manager | No (real-time only) | Yes — per-device tracks, effect blocks |
| Beat detection | Yes — VAMP plugins | No built-in | Yes — beatgrid sync | Yes — librosa (offline, no plugin install) |
| Audio waveform | Yes — sonographic display | No | N/A | Yes — rendered from audio file |
| Snap to beat | Yes — snaps to timing marks | N/A | N/A | Yes — on placement + resize |
| Animation library | Yes — 100+ effects | Yes — scenes/functions | Yes — autoloops (32 built-in) | Yes — curated set (~12 core, extensible) |
| AI show generation | No | No | Phrase detection (limited) | Yes — planned differentiator |
| Structural segmentation | No | No | No | Yes — librosa segment analysis |
| Live preview | Yes — 2D/3D layout | Yes — fixture map | No | Yes — canvas strip simulation |
| Custom protocol | E1.31 / ArtNet / DDP | DMX / ArtNet | MIDI + DMX | Custom UDP (command + frame modes) |
| YouTube support | No | No | No | Yes — yt-dlp passthrough |
| Undo/redo | Partial (some operations) | Limited | N/A | Yes — command pattern from day 1 |
| Personal tool scale | Overkill (100+ universes) | Overkill (full DMX) | Wrong use case (DJ) | Right scale — 2 strips + future devices |
Sources
- QLC+ Features page — DMX feature set, Show Manager, RGB Matrix
- xLights Manual — Timeline and Waveform — zoom, playback, waveform display
- xLights Manual — Timing Tracks — VAMP plugins, beat detection, snap behavior
- SoundSwitch — What's New in 2.9 — BPM detection, phrase detection, autoloops
- WLED Audio Reactive documentation — 30+ sound-reactive effects, audio input options
- Lightjams — Sync Music and Lights — 120 frequency bands, beat/mid/treble energy
- Resolume Avenue/Arena — FFT audio analysis, BPM sync, LED mapping
- librosa documentation — beat tracking, onset detection, structural segmentation
- WLED UDP Realtime Control — protocol patterns, FPS guidance (15-30 recommended)
- VenueMagic Show Control — drag-drop timeline, beat tapper
- DMXDesktop Effect Types — chase, sparkle, pulse, strobe definitions
Feature research for: music-to-light synchronization system (LightSync) Researched: 2026-04-05