Files
led2/.planning/research/PITFALLS.md

378 lines
30 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Pitfalls Research
**Domain:** Music-to-light synchronization system (Python backend, UDP LED control, web UI, MPV audio)
**Researched:** 2026-04-05
**Confidence:** MEDIUM — Core timing and protocol pitfalls are HIGH confidence from official docs and community reports; genre-specific beat detection accuracy is MEDIUM from GitHub issues; some Windows-specific timing claims are LOW confidence from limited benchmarks.
---
## Critical Pitfalls
### Pitfall 1: Audio Clock vs Wall Clock Drift
**What goes wrong:**
The backend polls MPV's playback position over IPC (named pipe on Windows) and uses that value to schedule UDP packets. The polling loop runs on Python's event loop, which has its own cadence. Over a 4-minute song, even a 10ms per-poll drift compounds into visible desync — lights ahead or behind the music by hundreds of milliseconds.
**Why it happens:**
Two clocks that are never formally synchronized: MPV's internal audio clock (tied to the audio hardware buffer) and Python's event loop clock (tied to OS scheduling). `asyncio.sleep()` on Windows is notoriously imprecise — the Windows OS timer resolution defaults to 15.6ms unless explicitly set to 1ms via `timeBeginPeriod`. Polling at 50Hz with 15ms jitter per tick means position reads that are consistently stale or erratic.
**How to avoid:**
- Use MPV's `playback-time` property via IPC as the authoritative clock — never derive position from wall clock elapsed time.
- Read MPV position on every scheduling tick, not just once at playback start.
- Add a configurable global offset (in ms) to compensate for known LED pipeline latency — expose this in the UI as a sync calibration control.
- Do not use `asyncio.sleep()` for the tight scheduling loop — use a dedicated `threading.Thread` with `time.perf_counter()` for sub-millisecond resolution.
- On Windows, call `timeBeginPeriod(1)` at startup (via `ctypes`) to raise the OS timer resolution from 15.6ms to 1ms.
**Warning signs:**
- Lights consistently lag or lead the beat by a fixed offset (fixed offset = pipeline latency, fixable with calibration offset).
- Offset grows over song duration rather than staying constant (growing drift = clock mismatch, not just latency).
- Position reads from MPV IPC have jitter larger than ~5ms on a lightly loaded system.
**Phase to address:**
Phase where MPV playback loop is first implemented. The calibration offset UI should be part of the same phase — not deferred.
---
### Pitfall 2: MPV IPC Named Pipe Blocking on Windows
**What goes wrong:**
On Windows, MPV IPC uses a named pipe (not a Unix socket). Named pipe reads are blocking by default. If the scheduling loop queries MPV position synchronously on the same thread that sends UDP packets, one slow IPC response stalls the entire send loop. At 30fps, you have 33ms per frame — a single blocked read can consume the entire budget.
**Why it happens:**
Linux MPV IPC uses Unix domain sockets, which are well-supported in Python's `asyncio`. Windows named pipes require either overlapped I/O or a separate thread. The `python-mpv-jsonipc` library handles this but its response latency on Windows is not documented and varies with system load. Developers port code from Linux and assume the same behavior.
**How to avoid:**
- Run the MPV IPC reader in a dedicated daemon thread, storing the last known position in a shared variable protected by a `threading.Lock` (or use `threading.Event`).
- The UDP sender thread reads from this shared variable — it never blocks on IPC.
- Use `python-mpv-jsonipc` rather than raw pipe I/O to avoid Windows-specific overlapped I/O complexity.
- If using `python-mpv` (C binding via libmpv), prefer the event-based callback API over property polling.
**Warning signs:**
- Intermittent UDP packet bursts (multiple frames sent in rapid succession, then silence) — sign that the send loop is getting stalled and catching up.
- IPC response times that vary from 1ms to 50ms+ under load.
**Phase to address:**
Phase where MPV integration is built. Test on Windows explicitly — do not assume Linux behavior transfers.
---
### Pitfall 3: librosa Beat Detection Returns Wrong Timestamps
**What goes wrong:**
`librosa.beat.beat_track()` returns beat timestamps that are consistently 2060ms late relative to the actual musical onset. Additionally, it estimates a single global tempo and may place beats at rhythmically correct but perceptually wrong positions, especially near the start and end of a song. Show files built on these timestamps produce lights that trigger visibly after the beat.
**Why it happens:**
The beat tracker reports beats at onset peaks (the spectral flux maximum), not at the perceptual beat moment (the onset itself). This is a documented design behavior, not a bug. The function also smooths aggressively over the first and last few beats to avoid tempo estimation errors — these boundary beats are often the most visible ones in a show.
**How to avoid:**
- After running `beat_track()`, apply a fixed backward shift of 2040ms to all returned timestamps (tune by ear against the actual track).
- Cross-validate with `librosa.onset.onset_detect()` for the first 4 beats — if they disagree significantly, trust onset detection for downbeat placement.
- Expose beat offsets as editable in the timeline — treat librosa output as a starting suggestion, not ground truth.
- For tempo-varying music (see Pitfall 4), use `librosa.beat.plp()` (predominant local pulse) which handles tempo drift better than `beat_track`.
**Warning signs:**
- Lights fire consistently after the kick drum in EDM tracks.
- Beat positions reported by librosa do not align with any visible transient in a waveform view.
- First and last beat in the array seem off while middle beats seem accurate.
**Phase to address:**
Beat detection analysis phase. The backward-shift calibration should be validated with at least 3 different genres before shipping.
---
### Pitfall 4: Beat Tracking Fails on Jazz, Classical, and Free-Tempo Music
**What goes wrong:**
`librosa.beat.beat_track()` assumes approximately constant tempo. On jazz recordings with swing feel, accelerating classical passages, or tracks with breaks and tempo changes, the tracker either locks to the wrong subdivision (halftime/doubletime error) or produces rhythmically incorrect beats that cannot be fixed by a simple offset.
**Why it happens:**
The algorithm uses dynamic programming that optimizes for tempo consistency over the whole track. A section that runs 5 BPM slower forces the entire track's grid to compromise. The default prior also strongly favors 120 BPM, biasing detection on sparse or ambiguous tracks.
**How to avoid:**
- For known tempo-variable music, use `librosa.beat.beat_track(units='time', start_bpm=<estimated>, tightness=50)` — lower tightness allows more tempo variation.
- Use `librosa.beat.plp()` to get a per-frame pulse probability, then find peaks — this handles tempo variation far better than `beat_track`.
- Offer a "manual beat tap" mode in the UI where the user can define beats at playback time and override the detected grid.
- For classical and jazz content, onset detection (`librosa.onset.onset_detect`) with hand-picked onsets is more useful than beat tracking.
**Warning signs:**
- Detected tempo is exactly half or double what the music sounds like (halftime/doubletime error).
- Beat grid looks correct for the first 30 seconds, then drifts noticeably.
- High-energy tracks with clear drums produce good results, but slow tracks with soft attacks do not.
**Phase to address:**
Beat detection phase. Implement `plp()` alongside `beat_track()` from the start — not as a later upgrade.
---
### Pitfall 5: UDP Packet Loss Causes Show Corruption, Not Graceful Degradation
**What goes wrong:**
UDP is fire-and-forget. If a frame packet is dropped (WiFi interference, microcontroller buffer overflow, OS socket buffer overflow), the LED strip holds the previous frame indefinitely. In a fast animation, a dropped frame is invisible. In a slow fade or color hold, a dropped frame means the strip stays on the wrong color for the entire hold duration — visually obvious corruption.
**Why it happens:**
Developers test over localhost (no drops) or on idle WiFi (no congestion). Production WiFi with other devices, microwave interference, or microcontroller processing lag causes sporadic drops. The microcontroller does not request retransmission — it simply never receives the packet.
**How to avoid:**
- Implement sequence numbers in the packet header. The microcontroller firmware checks for gaps and logs them (visible in the UI during testing).
- For slow animations (fades, color holds), re-send the same frame 23 times with identical sequence numbers. Microcontroller deduplicates by sequence number — no visual effect, but drop resilience increases significantly.
- Add a "heartbeat" packet: if no packet is received within 500ms, microcontroller fades to black (fail-safe) rather than holding the last state.
- Test with intentional packet loss (Linux `tc netem`, or simply run over a congested WiFi network) before marking the communication phase complete.
**Warning signs:**
- Show looks perfect on localhost simulator but has occasional glitches on real hardware.
- LED strip "freezes" on a color mid-animation.
- Packet loss only appears during high-bandwidth phases (many LEDs, high frame rate).
**Phase to address:**
Communication protocol phase. The sequence number and retransmit strategy must be in the protocol spec before firmware is written — retrofitting it later requires firmware updates to all devices.
---
### Pitfall 6: MTU Fragmentation for Large LED Strips
**What goes wrong:**
The SK6812 strip has 300 LEDs at 4 bytes/pixel (RGBW) = 1200 bytes of pixel data per frame. Combined with packet header overhead, this exceeds the WiFi MTU of ~1470 bytes if any framing is added. IP fragmentation silently occurs — the OS splits the packet, both fragments must arrive, and reassembly latency adds jitter. Fragment drops (which are more common than whole-packet drops) cause total frame loss.
**Why it happens:**
Developers calculate 300 × 4 = 1200 bytes and conclude "that fits." They don't account for packet headers (UDP: 8 bytes, IP: 20 bytes, Ethernet: ~18 bytes) plus any custom protocol header. They also don't test the specific WiFi link's effective MTU, which may be lower than 1500 due to WPA encryption overhead (~8 bytes) or VPN tunnels.
**How to avoid:**
- For 300 RGBW LEDs: use two packets with a start-index header (similar to the WLED DNRGB protocol approach). Packet 1 covers LEDs 0169, packet 2 covers LEDs 170299.
- Set packet boundaries to 480 RGB or 360 RGBW pixels maximum to stay comfortably under 1470 bytes.
- In the device registry, store `leds_per_packet` per device — the sender handles multi-packet frames transparently.
- Test with `ping -f -l 1472 <device_ip>` (Windows) to verify the actual path MTU to each microcontroller.
**Warning signs:**
- Full-strip animations work for half the strip only.
- Frame corruption that appears as a sharp vertical line (packets for different frame segments arriving from different frames).
- Wireshark shows IP fragmentation on the UDP stream.
**Phase to address:**
Protocol design phase. The multi-packet frame format must be in the spec before implementing the sender.
---
### Pitfall 7: Windows Firewall Blocks UDP to Microcontrollers
**What goes wrong:**
The Python backend sends UDP to microcontroller IPs on the local network. Windows Defender Firewall prompts on first run; if the user clicks "Private network only" or if the laptop switches to a "Public" network profile (common at different locations), outbound UDP is silently blocked. The app appears to work (no errors), LEDs simply do not respond.
**Why it happens:**
Windows treats outbound UDP differently from inbound — by default outbound is allowed, but "Public" network profiles block more aggressively. Some corporate or home routers apply "AP isolation" (clients cannot talk to each other on WiFi), which looks identical to a firewall block from the app's perspective.
**How to avoid:**
- On startup, the backend performs a UDP reachability test to each registered device and reports status in the UI (green/red per device).
- Document the firewall rule requirement in the app's setup page: allow Python on private networks.
- Include a diagnostic command in the UI that sends a test packet and shows whether a response was received (requires firmware cooperation — the microcontroller sends a UDP ACK back to the sender on a diagnostic port).
- Warn explicitly if the Windows network profile is "Public" (detectable via `subprocess` + `netsh interface show interface`).
**Warning signs:**
- LEDs work when the laptop is on one WiFi network but not another.
- No Python socket errors are thrown — the send succeeds silently.
- `ping` to the microcontroller works but UDP does not (ICMP and UDP have different firewall handling).
**Phase to address:**
Communication layer phase. The startup device health check should be part of the initial integration, not an afterthought.
---
### Pitfall 8: Show File Format Locked to Absolute Timestamps
**What goes wrong:**
Show files stored with absolute second-timestamps become tightly coupled to a specific audio file and its exact tempo map. If the audio file is ever replaced (different encode, trimmed intro, different yt-dlp download) the entire show is invalidated. There is no way to "shift" a show by 2 seconds without editing every single timestamp.
**Why it happens:**
Absolute timestamps are the obvious starting format — they map directly to MPV's playback position. Beat-relative storage (e.g., "beat 32, subdivision 2") requires a tempo map at edit time and at playback time, which feels like extra complexity. Developers defer the tempo-map integration and end up with a format they can't change without a migration.
**How to avoid:**
- Store events in the show file with both absolute time (seconds) and beat position (beat index + subdivision) if a beat grid is available.
- Include a `source_fingerprint` field (SHA256 of first 64KB of audio) and a `generated_at_tempo` field so mismatches can be detected at load time.
- Support a "shift all events by N seconds" operation in the UI — this requires only a single loop over the event list.
- Design the JSON schema before writing any playback code. Include a `schema_version` field from day one — even if v1 and v2 are identical, the field must exist to allow future migration.
**Warning signs:**
- Show works perfectly for one audio file but drifts immediately on a different download of the same song.
- Users cannot easily create variations of an existing show for a remixed track.
- Any tempo change requires rebuilding the show from scratch.
**Phase to address:**
Show file design phase (before any show playback is implemented). Once shows are saved in a format, migration is painful.
---
### Pitfall 9: Python GIL Starves the UDP Send Loop
**What goes wrong:**
The beat detection analysis, JSON processing, or WebSocket broadcast runs CPU-bound Python code on the same interpreter as the UDP send loop. The GIL allows only one thread to hold the interpreter at a time. Under load, the send loop is starved — it wakes up late, misses its frame window, and sends a burst of stale packets.
**Why it happens:**
Python threads feel like concurrency but CPU-bound code (numpy operations, beat analysis) holds the GIL for entire computation chunks. `asyncio` does not help here — it is single-threaded and cooperative. Beat analysis that takes 2 seconds in a background thread still causes GIL contention with anything else running Python bytecode.
**How to avoid:**
- Run beat analysis in a `ProcessPoolExecutor` (separate process, own GIL), not a `ThreadPoolExecutor`.
- Keep the UDP send loop and MPV position polling in a dedicated `threading.Thread` — this is I/O-bound and releases the GIL appropriately.
- Never run numpy-heavy processing in the same process as the real-time send loop (or accept that analysis is only triggered while playback is paused).
- FastAPI's async endpoints are fine for HTTP/WebSocket — but any CPU-bound analysis must be offloaded with `asyncio.run_in_executor(pool, ...)` using a `ProcessPoolExecutor`.
**Warning signs:**
- Send loop timing is smooth when the app is idle but irregular when the frontend is actively communicating.
- Beat analysis triggered during playback causes visible LED stutters.
- `time.perf_counter()` measurements in the send loop show consistent jitter during JSON processing.
**Phase to address:**
Architecture phase. The process boundary between analysis and real-time playback must be established before either is implemented.
---
### Pitfall 10: Frontend Audio Controls Mirror Wrong Clock
**What goes wrong:**
The web UI displays playback position, seek controls, and a timeline cursor. If the frontend derives position from its own JavaScript clock (Date.now() since playback started), it drifts from MPV's actual position. After a seek, the frontend and backend disagree on position until the next WebSocket update. The user drags the timeline cursor to a timestamp, but the show plays from the wrong position.
**Why it happens:**
`Date.now()` interpolation between WebSocket position updates is a natural optimization (avoids relying on slow poll updates for smooth cursor movement). But every seek, pause/resume, or buffer event invalidates the interpolated position. The backend MPV clock and the frontend JavaScript clock are never formally synchronized.
**How to avoid:**
- The backend broadcasts MPV position updates at 10Hz via WebSocket — include a monotonic sequence number and the backend's `time.perf_counter()` timestamp alongside the MPV position.
- The frontend uses MPV position as ground truth on every update, but only interpolates between updates using the known update interval (100ms) — not wall clock elapsed time.
- After every seek command, the frontend enters a "pending" state (cursor frozen, grayed out) until the next position update confirms the seek landed.
- Never let the frontend calculate what time it "should" be — only what time it "was told" it is.
**Warning signs:**
- Seek operations that visually snap to position but playback starts from a different point.
- Timeline cursor running ahead of or behind the waveform after a fast seek.
- Position display that looks smooth but is inconsistent with audible playback position.
**Phase to address:**
Playback UI phase. The state contract between backend clock authority and frontend display must be explicit in the design.
---
## Technical Debt Patterns
| Shortcut | Immediate Benefit | Long-term Cost | When Acceptable |
|----------|-------------------|----------------|-----------------|
| Poll MPV position on every frame instead of event-driven | Simple to implement | IPC overhead at 30fps, latency coupling between IPC and send loop | Never — use a background reader thread from day one |
| Absolute-only timestamps in show files | Simple storage format | Shows break on audio file replacement; no "shift" operation | Only if shows are never expected to be portable or edited |
| Single asyncio event loop for everything (IPC + UDP + WebSocket) | Less code | GIL starvation, IPC blocks stall UDP sends | Never for real-time send path |
| Hard-coded device IPs in code | Faster first demo | Every test environment requires code changes | Development only — device registry must exist before first firmware test |
| Run librosa analysis synchronously in HTTP handler | Simple request/response | Blocks other requests for 210 seconds on a 4-minute song | Never — always offload to ProcessPoolExecutor |
| Use WebSocket polling as the "heartbeat" for LED state | One transport for everything | WebSocket reconnects create brief LED state gaps | Only if willing to implement reconnect recovery explicitly |
---
## Integration Gotchas
| Integration | Common Mistake | Correct Approach |
|-------------|----------------|------------------|
| MPV IPC on Windows | Assume Unix socket behavior; use blocking reads on named pipe | Dedicated reader thread; `python-mpv-jsonipc` for abstraction |
| MPV IPC on Windows | Query `time-pos` (may return null when paused) | Query `playback-time` — it returns the last valid position even when paused |
| librosa.load() | Load at native sample rate (`sr=None`) for consistency | Always resample to 22050 Hz (`sr=22050`) — beat tracking is tuned to this rate; native SR breaks tempo estimation |
| librosa.load() with MP3 | Pass a file object | Pass a file path string — soundfile cannot decode MP3, only audioread (file path) can |
| UDP send on Windows | Assume loopback always works | Loopback UDP works but "Public" network profile may block non-loopback targets — test against a real device IP early |
| yt-dlp + MPV | Buffer yt-dlp output into MPV stdin | Use `--input` with a yt-dlp URL passthrough — MPV handles yt-dlp integration natively via `--script-opts` or URL schemes |
| FastAPI WebSocket | Use `asyncio.sleep()` in background task | Background task must check disconnect events; use `asyncio.wait_for()` with timeout |
---
## Performance Traps
| Trap | Symptoms | Prevention | When It Breaks |
|------|----------|------------|----------------|
| Sending full-strip RGBW frames at 60fps over WiFi | WiFi congestion, irregular frame delivery, ESP32 TX buffer overflow | Cap at 30fps for WiFi; use 40fps only with Ethernet | With 300 LEDs RGBW at 60fps = ~288 KB/s continuous — exceeds reliable WiFi throughput under contention |
| Loading full audio into memory for librosa analysis | 4-minute WAV at 44100Hz = ~40MB; 20+ songs in a session = memory pressure | Stream analysis in chunks, or enforce that analysis is single-file and results are cached to disk as JSON | At ~10 simultaneous cached analyses or >5 songs loaded at once |
| WebSocket broadcasting position updates at 60Hz | CPU overhead on backend; browser rendering stalls from processing too many messages | 10Hz is sufficient for UI display; 30Hz max if sub-frame cursor interpolation is required | At 30+ connected clients (not relevant for single-operator app, but good discipline) |
| Synchronous JSON serialization of large show files | Blocking the event loop during save | Use `asyncio.run_in_executor` for file I/O; or write to a temp file and rename atomically | Show files with >10,000 events and high-resolution per-LED animation data |
---
## Security Mistakes
| Mistake | Risk | Prevention |
|---------|------|------------|
| Accepting yt-dlp URLs without validation | An attacker with UI access could trigger yt-dlp against arbitrary URLs on the internal network (SSRF) | Behind Authelia SSO — single admin user — risk is low; still, validate URL scheme (`https://` only) and domain against an allowlist |
| Storing device IPs in show files | Hardcoded IPs in exported shows break when device moves; no security risk but operational problem | Store device by name in show files; resolve to IP at playback time from the device registry |
| No rate limiting on analysis endpoint | User can queue 20 CPU-intensive analyses simultaneously | Single analysis queue with one concurrent worker; reject if queue is non-empty |
---
## UX Pitfalls
| Pitfall | User Impact | Better Approach |
|---------|-------------|-----------------|
| Beat detection runs synchronously on "Sync" button click | UI freezes for 210 seconds with no feedback | Show a progress indicator; run analysis async and push result via WebSocket when complete |
| No visual diff between "detected beats" and "manually placed events" | User cannot tell which events are reliable vs hand-placed | Color-code events by source (auto-detected vs manual vs AI) in the timeline |
| Seeking during playback updates position but not device state | Devices hold last-sent color — after seek, devices show wrong color until next animation event | On seek, send an "all clear" or "fade to black" packet immediately to all registered devices |
| Animation preview only visible during live playback | User must play the whole show to verify new event placement | Provide a scrub-preview: single-frame UDP send on timeline cursor position change |
| No feedback when UDP send fails silently | Lights don't respond; user doesn't know if it's app, network, or device | Per-device health indicator in the UI; UDP send loop logs drop count accessible in real time |
---
## "Looks Done But Isn't" Checklist
- [ ] **MPV position polling:** Often missing thread safety — verify that position reads and writes use a lock or atomic structure.
- [ ] **Beat detection:** Often returns beats without a backward-shift calibration — verify by overlaying beat grid on a waveform at 10x zoom and checking against kick drum transients.
- [ ] **Multi-packet frames:** Often implemented for the send side only — verify the microcontroller simulator reassembles multi-packet frames correctly before writing firmware.
- [ ] **Show file format:** Often missing `schema_version` field — verify it is in the schema before any shows are saved that would need migration.
- [ ] **Windows timer resolution:** Often not set — verify by measuring `time.perf_counter()` jitter in the send loop; should be <2ms on Windows with `timeBeginPeriod(1)`.
- [ ] **WebSocket disconnect handling:** Often the background send task keeps running after the browser tab closes — verify the task is cancelled on disconnect.
- [ ] **UDP device health check:** Often bypassed during development — verify it actually detects a powered-off device before marking communication phase complete.
- [ ] **librosa MP3 loading:** Works with WAV, often breaks silently with MP3 if soundfile is the only backend — verify with an MP3 file explicitly.
---
## Recovery Strategies
| Pitfall | Recovery Cost | Recovery Steps |
|---------|---------------|----------------|
| Show files in wrong format (no schema_version) | MEDIUM | Write a one-time migration script; add schema_version = 1 to all existing files; add version check on load |
| Audio-LED clock drift discovered after shows are built | LOW | Add calibration offset field to show file; user adjusts offset per-show via UI without rebuilding |
| MTU issue discovered after firmware is written | HIGH | Requires firmware update for all devices to support multi-packet protocol; test MTU before finalizing protocol spec |
| GIL starvation causing send loop jitter | MEDIUM | Refactor analysis to run in ProcessPoolExecutor; no show file changes required |
| Librosa beat timestamps systematically wrong | LOW | Add a per-show or per-analysis backward-shift field; re-analyze existing shows with corrected offset |
| Windows firewall blocking UDP post-deployment | LOW | Document firewall rule; add startup diagnostic; add to setup guide |
---
## Pitfall-to-Phase Mapping
| Pitfall | Prevention Phase | Verification |
|---------|------------------|--------------|
| Audio clock vs wall clock drift | MPV integration phase | Measure drift over a 4-minute song; delta between MPV position and expected position must be <50ms |
| MPV IPC named pipe blocking | MPV integration phase | Run IPC read and UDP send in parallel for 10 minutes; measure max send loop jitter |
| librosa beat timestamps late | Beat detection phase | Visual overlay of beat grid vs waveform transients on 3 genre samples |
| Beat tracking on tempo-variable music | Beat detection phase | Test against a jazz track and a classical track; verify `plp()` fallback exists |
| UDP packet loss show corruption | Protocol design phase | Intentional 5% packet loss test; verify strip recovers gracefully |
| MTU fragmentation | Protocol design phase | Test with 300-LED full-frame packet; verify no IP fragmentation in Wireshark capture |
| Windows firewall blocking UDP | Communication layer phase | Test from a "Public" network profile; startup diagnostic must detect and report blocked state |
| Show file absolute timestamp lock-in | Show file design phase | Verify schema_version, source_fingerprint, and shift-all-events operation exist before first show is saved |
| Python GIL starving send loop | Architecture phase | Beat analysis triggered during playback must not cause >5ms send loop jitter |
| Frontend clock drift | Playback UI phase | Seek 10 times rapidly; verify frontend position matches backend position within 100ms after each seek |
---
## Sources
- librosa beat tracking documentation and known timing bias: https://librosa.org/doc/main/generated/librosa.beat.beat_track.html
- librosa beat timing issue (2060ms late): https://github.com/librosa/librosa/issues/1052
- librosa default resampling rationale: https://librosa.org/blog/2019/07/17/resample-on-load/
- librosa onset vs beat tracking: https://deepwiki.com/librosa/librosa/5.1-beat-tracking
- MPV IPC protocol documentation: https://github.com/mpv-player/mpv/blob/master/DOCS/man/ipc.rst
- MPV Windows named pipe IPC challenges: https://github.com/mpv-player/mpv/issues/10116
- python-mpv-jsonipc library: https://github.com/iwalton3/python-mpv-jsonipc
- ESP32 UDP packet loss and buffer overflow: https://www.esp32.com/viewtopic.php?t=5411
- ESP32 UDP performance issues: https://github.com/espressif/arduino-esp32/issues/1317
- WLED UDP realtime protocol and MTU: https://kno.wled.ge/interfaces/udp-realtime/
- WLED DNRGB multi-packet protocol: https://github.com/Aircoookie/WLED/wiki/UDP-Realtime-Control
- UDP pixel streaming protocol (start index header): https://github.com/EnviralDesign/NodeMCU-Arduino-Pixel-Driver/wiki/UDP-Pixel-Streaming-protocol
- Windows asyncio timing limitations: https://runebook.dev/en/docs/python/library/asyncio-platforms/windows
- Python GIL and real-time threading: https://realpython.com/python-gil/
- FastAPI WebSocket common mistakes: https://hexshift.medium.com/managing-per-user-websocket-state-in-fastapi-9ceaa2b312ac
- Audio sync in browser vs backend: https://medium.com/fender-engineering/near-realtime-animations-with-synchronized-audio-in-javascript-6d845afcf1c5
- Windows firewall UDP blocking: https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/windows-fireware-rule-block-udp-communication
- Timeline absolute vs BPM-relative anchoring: https://resolume.com/forum/viewtopic.php?t=13032
---
*Pitfalls research for: music-to-light synchronization (LightSync)*
*Researched: 2026-04-05*