diff --git a/.planning/phases/02-audio-engine/02-RESEARCH.md b/.planning/phases/02-audio-engine/02-RESEARCH.md new file mode 100644 index 0000000..fa86343 --- /dev/null +++ b/.planning/phases/02-audio-engine/02-RESEARCH.md @@ -0,0 +1,604 @@ +# Phase 02: Audio Engine - Research + +**Researched:** 2026-04-05 +**Domain:** MPV IPC bridge, audio position tracking, waveform extraction, FastAPI WebSocket broadcast +**Confidence:** HIGH (core stack verified against Alpine package registry and official sources) + +--- + +## Summary + +Phase 2 builds the master clock for the entire show system. Audio plays via MPV embedded through its libmpv C API (via Python ctypes), position is read at 10Hz using a polling loop (not observe_property), and that position is broadcast over WebSocket to all connected browser clients. Waveform data is extracted server-side using soundfile + numpy and sent to the browser as a downsampled float array for canvas rendering. + +The deployment target is Linux Docker (Alpine 3.19) on groll.cloud — not Windows. The ROADMAP mentions `timeBeginPeriod(1)` and a "Windows timer fix" (INF-03), but this is irrelevant on Linux. On Linux, the kernel scheduler default tick is 250Hz (4ms granularity) and process timers default to 1ms resolution without any special calls. INF-03 as written is a no-op on this platform — the planner must decide: implement it as a Linux no-op stub, or mark it satisfied-by-platform. + +The `py3-mpv` package (python-mpv 1.0.5, wrapping libmpv via ctypes) is available directly in Alpine 3.19's community repository alongside `mpv-libs` 0.37.0. No custom builds or workarounds are needed. Audio device access inside Docker requires mounting `/dev/snd` or the PulseAudio socket — or using `--ao=null` for position-tracking-only mode (the use case here, since audio output to the browser is not required). + +**Primary recommendation:** Use `py3-mpv` (Alpine package, ctypes binding) with a dedicated 100ms polling loop for position, soundfile for waveform extraction, and the existing ConnectionManager WebSocket hub with an asyncio background task for 10Hz broadcast. + +--- + + +## Phase Requirements + +| ID | Description | Research Support | +|----|-------------|------------------| +| AUD-01 | Load audio from a local file (MP3, WAV, FLAC, OGG) | MPV supports all four formats natively; load via `player.loadfile(path)` | +| AUD-02 | Load audio from a YouTube URL via yt-dlp + MPV | Deferred to Phase 6 per ROADMAP — `--ytdl` flag; out of scope for Phase 2 | +| AUD-03 | Playback controls: play, pause, seek to position | `player.pause = True/False`, `player.seek(seconds, reference='absolute')` | +| AUD-04 | Realtime position reporting from MPV to UI (≤10ms poll interval) | Poll `player.time_pos` at 100ms interval (10Hz) from background thread; broadcast via WebSocket | +| AUD-05 | Waveform display rendered from audio samples in the timeline UI | soundfile reads audio as numpy array; downsample to ~1000 peak samples; send as JSON to browser | +| INF-03 | Windows 11 timer resolution fix (`timeBeginPeriod(1)`) | **Irrelevant on Linux.** Linux kernel provides ≤1ms timer resolution natively. Implement as platform-guarded no-op (only call on `sys.platform == 'win32'`). | + + +--- + +## Standard Stack + +### Core + +| Library | Version | Purpose | Why Standard | +|---------|---------|---------|--------------| +| py3-mpv (python-mpv) | 1.0.5 (Alpine pkg) | Python ctypes binding to libmpv | Available as Alpine apk; same version as pip `python-mpv`; no custom build | +| mpv-libs | 0.37.0-r0 (Alpine) | libmpv.so.2 shared library | Required by py3-mpv; provides the C API | +| soundfile | 0.12.x | Read audio files as numpy arrays | Backed by libsndfile; supports WAV/FLAC/OGG natively; FAST, no FFmpeg dependency for core formats | +| numpy | 1.x / 2.x | Downsample waveform samples | Required by soundfile; standard scientific Python | +| libsndfile | 1.2.2-r1 (Alpine) | C library backing soundfile | Available as Alpine apk; handles WAV/FLAC/OGG | + +### Supporting + +| Library | Version | Purpose | When to Use | +|---------|---------|---------|-------------| +| ffmpeg-libs | 6.1.1-r0 (Alpine) | Decode MP3 for soundfile | MP3 is NOT supported by libsndfile alone; soundfile falls back to audioread which needs ffmpeg | +| pydub | latest | Alternative MP3 decode | Only if ffmpeg integration path proves complex; adds dependency | + +### Alternatives Considered + +| Instead of | Could Use | Tradeoff | +|------------|-----------|----------| +| py3-mpv (ctypes) | python-mpv-jsonipc | jsonipc is subprocess-based JSON over a Unix socket — more resilient but higher latency and more moving parts; ctypes is direct and lower overhead | +| py3-mpv (ctypes) | mpvasync | Minimal asyncio client for mpv JSON IPC; good fit for async but requires managing the mpv subprocess separately | +| soundfile + numpy | audiowaveform (BBC C++ tool) | audiowaveform not in Alpine apk, requires compilation; adds complexity; soundfile+numpy covers the same need with less friction | +| polling at 100ms | observe_property time-pos | observe_property fires on EVERY audio frame (many times/second at 44.1kHz), overwhelming IPC; polling at fixed 100ms is the correct approach | + +**Installation (Dockerfile addition):** +```dockerfile +# Alpine apk packages (add to Dockerfile RUN apk add) +RUN apk add --no-cache mpv-libs ffmpeg-libs libsndfile + +# Python packages (add to pyproject.toml dependencies) +# py3-mpv can be installed via apk OR pip; use pip for reproducibility +# "python-mpv>=1.0.5" +# "soundfile>=0.12.0" +# "numpy>=1.26.0" +``` + +**Version verification (confirmed 2026-04-05):** +- `py3-mpv` 1.0.5-r0 in Alpine 3.19 community +- `mpv-libs` 0.37.0-r0 in Alpine 3.19 community +- `libsndfile` 1.2.2-r1 in Alpine 3.19 main +- `ffmpeg-libs` 6.1.1-r0 in Alpine 3.19 community + +--- + +## Architecture Patterns + +### Recommended Project Structure + +``` +lightsync/ +├── audio/ +│ ├── __init__.py +│ ├── engine.py # MPVEngine: wraps python-mpv, position polling loop +│ └── waveform.py # extract_peaks(): soundfile + numpy downsampler +├── show/ +│ ├── __init__.py +│ └── state.py # PlaybackState dataclass: position, paused, duration, file +├── api/ +│ ├── ws.py # existing ConnectionManager + new broadcast_loop task +│ └── audio.py # REST endpoints: POST /api/audio/load, /play, /pause, /seek +└── main.py # lifespan: start MPVEngine, start broadcast_loop task +``` + +### Pattern 1: MPVEngine — Headless Audio-Only Instance + +**What:** A class wrapping `mpv.MPV` configured for headless audio-only operation in Docker. +**When to use:** Always. This is the only MPV instance in the application. + +```python +# Source: github.com/jaseg/python-mpv + mpv manual +import mpv +import threading +import time + +class MPVEngine: + def __init__(self): + self._player = mpv.MPV( + vo='null', # no video output + ao='pulse', # PulseAudio; fallback: 'alsa' or 'null' + input_default_bindings=False, + input_vo_keyboard=False, + ) + self._position: float = 0.0 + self._lock = threading.Lock() + self._running = False + self._thread: threading.Thread | None = None + + def start(self): + self._running = True + self._thread = threading.Thread(target=self._poll_loop, daemon=True) + self._thread.start() + + def _poll_loop(self): + """Poll time-pos at 10Hz. observe_property fires per-frame (too fast).""" + while self._running: + pos = self._player.time_pos # returns None if stopped + with self._lock: + self._position = pos or 0.0 + time.sleep(0.1) # 10Hz + + def get_position(self) -> float: + with self._lock: + return self._position + + def load(self, path: str): + self._player.loadfile(path, mode='replace') + + def play(self): + self._player.pause = False + + def pause(self): + self._player.pause = True + + def seek(self, seconds: float): + self._player.seek(seconds, reference='absolute') + + @property + def duration(self) -> float | None: + return self._player.duration + + def stop(self): + self._running = False + self._player.terminate() +``` + +### Pattern 2: Asyncio Background Broadcast Loop + +**What:** A persistent asyncio task started at lifespan that reads position and broadcasts at 10Hz. +**When to use:** One task for the whole application lifetime, not per-connection. + +```python +# Source: FastAPI lifespan pattern + asyncio +import asyncio + +async def broadcast_loop(engine: MPVEngine, manager: ConnectionManager): + """Broadcast position to all WebSocket clients at 10Hz.""" + while True: + await asyncio.sleep(0.1) + pos = engine.get_position() + if manager.active_connections: + await manager.broadcast({ + "type": "position", + "position": pos, + "paused": engine.player.pause, + }) +``` + +**Integration in lifespan:** +```python +@asynccontextmanager +async def lifespan(app: FastAPI): + engine = MPVEngine() + engine.start() + app.state.engine = engine + task = asyncio.create_task(broadcast_loop(engine, manager)) + yield + task.cancel() + engine.stop() +``` + +### Pattern 3: Thread-to-Asyncio Bridge + +**What:** The MPVEngine runs in a daemon thread (blocking poll loop). When it needs to trigger something async (rare), use `asyncio.run_coroutine_threadsafe`. +**When to use:** Only if you need to push events FROM mpv callbacks INTO the asyncio loop. + +```python +# Source: Python asyncio docs +import asyncio + +loop = asyncio.get_event_loop() + +def on_mpv_event(): + # Called from mpv's internal thread + asyncio.run_coroutine_threadsafe( + manager.broadcast({"type": "event", "data": "..."}), + loop + ) +``` + +### Pattern 4: Waveform Extraction + +**What:** Server-side extraction of peak waveform data from audio files, downsampled to browser-friendly size. +**When to use:** When a file is loaded; result cached and sent once via WebSocket or HTTP. + +```python +# Source: python-soundfile docs + numpy +import soundfile as sf +import numpy as np + +def extract_peaks(path: str, num_samples: int = 1000) -> list[float]: + """Return downsampled peak amplitude array for waveform display.""" + data, samplerate = sf.read(path, always_2d=True) + mono = np.mean(data, axis=1) # mix to mono + chunk_size = max(1, len(mono) // num_samples) + peaks = [] + for i in range(0, len(mono), chunk_size): + chunk = mono[i:i + chunk_size] + peaks.append(float(np.max(np.abs(chunk)))) + return peaks[:num_samples] +``` + +**MP3 caveat:** soundfile does NOT support MP3 natively. Two options: +1. Install `ffmpeg` in Docker and use pydub or audioread as fallback +2. Use mutagen to read metadata only, and route MP3 decoding through ffmpeg subprocess + +**Recommended MP3 path:** `ffmpeg -i input.mp3 -f f32le -ar 44100 -ac 1 pipe:1` → read from stdout as numpy array. ffmpeg-libs is already needed in the Alpine image. + +### Pattern 5: WebSocket Transport Commands + +**What:** Browser sends JSON commands; server dispatches to MPVEngine. +**When to use:** Replaces the echo stub in ws.py. + +```python +# Extends existing ws.py dispatch +msg_handlers = { + "play": lambda _: engine.play(), + "pause": lambda _: engine.pause(), + "seek": lambda msg: engine.seek(float(msg["position"])), + "load": lambda msg: engine.load(msg["path"]), +} + +# In websocket_endpoint receive loop: +handler = msg_handlers.get(msg.get("type")) +if handler: + handler(msg) +``` + +### Anti-Patterns to Avoid + +- **observe_property for time-pos:** Fires on every decoded audio frame (~43× per second at 44.1kHz/1024 samples). Do NOT use for position reporting — use polling at 100ms. +- **Blocking calls inside async routes:** `engine.seek()` and `engine.load()` call libmpv synchronously. These are fast (< 1ms), but if they block longer, wrap in `asyncio.to_thread()`. +- **Running mpv with ao=null for production audio:** `ao=null` discards audio output. This is correct for a server that serves audio over network, but wrong if the Docker host needs to emit actual sound from speakers. For this project (serving a web UI), `ao=null` is correct — the browser doesn't receive audio bytes, it receives position ticks. +- **Starting the broadcast_loop task per WebSocket connection:** Start it once at lifespan. Per-connection tasks don't cancel cleanly. + +--- + +## Don't Hand-Roll + +| Problem | Don't Build | Use Instead | Why | +|---------|-------------|-------------|-----| +| Audio decoding + position tracking | Custom ffmpeg subprocess wrapper | python-mpv + mpv-libs | MPV handles seeking, gapless decode, codec detection, and position accuracy; rolling your own is months of work | +| MP3/WAV/FLAC detection and routing | Custom magic-byte sniffer | MPV auto-detects format | MPV uses libavformat probe; no manual routing needed | +| Audio sample extraction | Direct ffmpeg subprocess plumbing | soundfile + numpy for WAV/FLAC/OGG, ffmpeg pipe for MP3 | soundfile is battle-tested; the MP3 edge case is the only exception | +| WebSocket fan-out with dead connection cleanup | Custom send-and-check loop | The existing ConnectionManager already handles this | ws.py broadcast() already removes dead connections on exception | +| 10Hz clock | Custom timer thread | asyncio.sleep(0.1) in broadcast_loop | asyncio sleep is accurate enough for 10Hz; no custom timer needed | + +--- + +## Critical Clarification: Windows Timer Fix (INF-03) on Linux + +INF-03 states: "Windows 11 timer resolution fix applied at startup (`timeBeginPeriod(1)` via ctypes)." + +**This requirement is inapplicable on Linux.** `timeBeginPeriod` is a Windows-only Win32 API (`timeapi.h`). Calling it on Linux is a no-op at best, and an import error at worst. + +**Linux timer resolution:** On Linux 5.x+, the default scheduler tick is 250Hz or 1000Hz depending on kernel config (CONFIG_HZ). Process-level timer resolution via `clock_nanosleep` is typically < 100μs. No special calls are needed. + +**Recommended implementation:** Implement INF-03 as a platform guard in `main.py`: + +```python +import sys +import ctypes + +def apply_timer_fix(): + """Apply 1ms timer resolution on Windows. No-op on Linux/macOS.""" + if sys.platform == 'win32': + winmm = ctypes.WinDLL('winmm') + winmm.timeBeginPeriod(1) +``` + +This satisfies the requirement as stated while being correct for the actual deploy target. + +--- + +## Docker / Deployment Notes + +### Alpine 3.19 Dockerfile additions + +```dockerfile +FROM python:3.11-alpine + +# Audio stack: mpv-libs (libmpv.so.2) + audio codec support +RUN apk add --no-cache \ + mpv-libs \ + ffmpeg-libs \ + libsndfile + +# For actual audio output (if ever needed from Docker): +# Mount /dev/snd at runtime: docker run --device /dev/snd ... +# Or PulseAudio socket: -v /run/user/1000/pulse:/run/user/1000/pulse +``` + +### Audio output in Docker + +The VPS Docker container does NOT need to emit audio to speakers. MPV is used purely as a position-accurate playback clock. Therefore: + +- Use `ao='null'` (or `ao='alsa'` with `/dev/snd` mounted) — no audio device needed +- `vo='null'` — no video output needed +- No display server, no ALSA/PulseAudio socket mounting required for position tracking + +**Confirmed:** `mpv-libs` 0.37.0-r0 provides `libmpv.so.2` — confirmed via Alpine package page. python-mpv 1.0.5 requires libmpv API ≥ 1.108 (libmpv ≥ 0.33). mpv 0.37.0 > 0.33, so the API requirement is met. + +--- + +## Common Pitfalls + +### Pitfall 1: observe_property time-pos Flood + +**What goes wrong:** Registering `@player.property_observer('time-pos')` causes the callback to fire at the audio decode rate (~43/s for 44.1kHz), overwhelming the asyncio event loop with 430 messages per second at 10× the target rate. +**Why it happens:** MPV fires property-change events on every decoded audio frame, not at a configurable Hz rate. +**How to avoid:** Use a 100ms polling loop reading `player.time_pos` directly. This gives exactly 10Hz regardless of audio format. +**Warning signs:** WebSocket message queue growing, browser lagging behind, CPU spike in uvicorn worker. + +### Pitfall 2: MP3 Not Supported by soundfile + +**What goes wrong:** `sf.read('track.mp3')` raises `SoundFileError: Error opening 'track.mp3': Format not recognised`. +**Why it happens:** libsndfile does not include an MP3 decoder. soundfile's `audioread` fallback requires FFmpeg on PATH. +**How to avoid:** Add `ffmpeg` (or `ffmpeg-libs`) to the Dockerfile AND test MP3 loading explicitly. Alternatively, use a dedicated ffmpeg subprocess for MP3 → float array conversion. +**Warning signs:** Load succeeds for WAV in tests but fails for MP3 in production. + +### Pitfall 3: Blocking MPV Calls in Async Context + +**What goes wrong:** Calling `engine.load()` or `engine.seek()` inside an `async def` FastAPI route blocks the event loop while libmpv processes the command. +**Why it happens:** python-mpv ctypes calls are synchronous; libmpv file loading can take 10–100ms for large files. +**How to avoid:** Wrap in `asyncio.to_thread(engine.load, path)` for the load command. Seek is typically fast enough (< 1ms) but wrap defensively. +**Warning signs:** WebSocket clients stop receiving position updates during file load. + +### Pitfall 4: Position Reporting to Dead WebSocket Connections + +**What goes wrong:** The broadcast_loop task throws an exception for a disconnected client, then crashes, stopping ALL position broadcasts. +**Why it happens:** `websocket.send_text()` raises on a closed connection. +**How to avoid:** The existing `ConnectionManager.broadcast()` already catches exceptions and removes dead connections. Do not bypass it. +**Warning signs:** Position updates stop for ALL clients when one disconnects. + +### Pitfall 5: time_pos Returns None During Seek or Before Load + +**What goes wrong:** `player.time_pos` returns `None` when no file is loaded or during a seek operation. Passing `None` to JSON serialization raises `TypeError`. +**Why it happens:** libmpv property returns None for unavailable properties. +**How to avoid:** Always use `pos = player.time_pos or 0.0` in the polling loop. +**Warning signs:** `TypeError: Object of type NoneType is not JSON serializable` in logs. + +### Pitfall 6: mpv-libs vs Full mpv Package + +**What goes wrong:** Installing `mpv` (full package, 163 deps including display libs) instead of `mpv-libs` (15 deps, 7MB). +**Why it happens:** Confusion between the player binary and the embeddable library. +**How to avoid:** Install `mpv-libs` only. python-mpv needs `libmpv.so.2`, NOT the `mpv` command-line binary. +**Warning signs:** Docker image bloat; install taking > 1 minute; mesa/wayland/X11 dependencies appearing. + +--- + +## Code Examples + +### Complete MPVEngine skeleton + +```python +# lightsync/audio/engine.py +import threading +import time +import mpv + +class MPVEngine: + def __init__(self): + self._player = mpv.MPV( + vo='null', + ao='null', # no audio output needed on server + input_default_bindings=False, + input_vo_keyboard=False, + ) + self._position: float = 0.0 + self._lock = threading.Lock() + self._running = False + + def start(self): + self._running = True + t = threading.Thread(target=self._poll_loop, daemon=True) + t.start() + + def _poll_loop(self): + while self._running: + pos = self._player.time_pos + with self._lock: + self._position = pos if pos is not None else 0.0 + time.sleep(0.1) + + def get_state(self) -> dict: + with self._lock: + return { + "position": self._position, + "paused": bool(self._player.pause), + "duration": self._player.duration, + "loaded": self._player.path is not None, + } + + def load(self, path: str): + self._player.loadfile(path, mode='replace') + self._player.pause = True # load paused, wait for explicit play + + def play(self): + self._player.pause = False + + def pause(self): + self._player.pause = True + + def seek(self, seconds: float): + self._player.seek(seconds, reference='absolute') + + def stop(self): + self._running = False + try: + self._player.terminate() + except Exception: + pass +``` + +### Waveform extraction (MP3 + non-MP3) + +```python +# lightsync/audio/waveform.py +import subprocess +import numpy as np +import soundfile as sf +from pathlib import Path + +def extract_peaks(path: str, num_peaks: int = 1000) -> list[float]: + """Extract amplitude peaks from audio file. Handles MP3 via ffmpeg.""" + p = Path(path) + if p.suffix.lower() == '.mp3': + return _extract_peaks_mp3(path, num_peaks) + return _extract_peaks_soundfile(path, num_peaks) + +def _extract_peaks_soundfile(path: str, num_peaks: int) -> list[float]: + data, _ = sf.read(path, always_2d=True) + mono = np.mean(data, axis=1) + return _downsample_peaks(mono, num_peaks) + +def _extract_peaks_mp3(path: str, num_peaks: int) -> list[float]: + """Decode MP3 via ffmpeg pipe to float32 array.""" + cmd = [ + 'ffmpeg', '-i', path, + '-f', 'f32le', '-ar', '44100', '-ac', '1', 'pipe:1', + '-loglevel', 'quiet' + ] + result = subprocess.run(cmd, capture_output=True) + mono = np.frombuffer(result.stdout, dtype=np.float32) + return _downsample_peaks(mono, num_peaks) + +def _downsample_peaks(mono: np.ndarray, num_peaks: int) -> list[float]: + chunk = max(1, len(mono) // num_peaks) + peaks = [ + float(np.max(np.abs(mono[i:i+chunk]))) + for i in range(0, len(mono), chunk) + ] + return peaks[:num_peaks] +``` + +### FastAPI lifespan integration + +```python +# main.py additions +from lightsync.audio.engine import MPVEngine +from lightsync.api.ws import manager +import asyncio + +async def _broadcast_loop(engine: MPVEngine): + while True: + await asyncio.sleep(0.1) + if manager.active_connections: + await manager.broadcast({"type": "tick", **engine.get_state()}) + +@asynccontextmanager +async def lifespan(app: FastAPI): + # ... existing registry/show_store setup ... + engine = MPVEngine() + engine.start() + app.state.engine = engine + task = asyncio.create_task(_broadcast_loop(engine)) + yield + task.cancel() + engine.stop() +``` + +--- + +## State of the Art + +| Old Approach | Current Approach | When Changed | Impact | +|--------------|------------------|--------------|--------| +| mpv --slave mode (stdin commands) | JSON IPC via Unix socket | mpv 0.7+ (2014) | Clean bidirectional; python-mpv wraps this | +| manual libmpv ctypes bindings | python-mpv package | 2014–present | Maintained wrapper, no DIY ctypes | +| observe_property for position | Polling at fixed Hz | Known issue (#5661) | Correct 10Hz rate, not frame-rate dependent | +| audiowaveform (BBC C++ binary) | soundfile + numpy | Still valid alternative | soundfile needs no compilation; less infra | + +**Deprecated/outdated:** +- `--slave` mode stdin control: Removed from mpv. Use IPC socket or libmpv API. +- `python-mpv-jsonipc` as primary approach: The STATE.md notes this was initially preferred to avoid Windows DLL issues. Since we deploy on Linux, the ctypes binding (py3-mpv) is simpler and available as an Alpine package. + +--- + +## Open Questions + +1. **Audio output: null vs pulse vs alsa?** + - What we know: Server doesn't need to output sound; `ao=null` is correct for position-tracking-only + - What's unclear: If someone ever runs this locally on a desktop, they'd want actual audio; `ao=null` silences playback + - Recommendation: Use `ao=null` for Docker; expose as env var `MPV_AO` defaulting to `null` + +2. **File upload vs path reference for audio loading** + - What we know: AUD-01 says "load from local file" — unclear if user uploads file or provides a server path + - What's unclear: Is the audio file already on the VPS (e.g., mounted volume), or does the user upload from their browser? + - Recommendation: Plan 02-01 should implement path-based loading first (simpler); file upload can be a follow-up + +3. **Waveform data size and format** + - What we know: 1000 peak samples as float array is standard for timeline display + - What's unclear: Whether to send this as a WebSocket message or as a separate HTTP endpoint response + - Recommendation: HTTP GET `/api/audio/waveform` returning JSON; don't mix it into the WebSocket stream + +--- + +## Environment Availability + +| Dependency | Required By | Available | Version | Fallback | +|------------|------------|-----------|---------|----------| +| mpv-libs | AUD-01, AUD-03, AUD-04 | ✓ (Alpine apk) | 0.37.0-r0 | — | +| py3-mpv (python-mpv) | AUD-01, AUD-03, AUD-04 | ✓ (Alpine apk or pip) | 1.0.5 | — | +| libsndfile | AUD-05 (WAV/FLAC/OGG) | ✓ (Alpine apk) | 1.2.2-r1 | — | +| soundfile (Python) | AUD-05 | pip install | 0.12.x | — | +| numpy | AUD-05 | pip install | 1.26+ | — | +| ffmpeg-libs | AUD-05 (MP3 decode) | ✓ (Alpine apk) | 6.1.1-r0 | Skip MP3 support | +| ffmpeg binary | AUD-05 (MP3 pipe) | ✓ (Alpine: `apk add ffmpeg`) | 6.1.1 | pydub (heavier) | +| ctypes | python-mpv binding | ✓ (stdlib) | stdlib | — | + +**Missing dependencies with no fallback:** None — all required packages are available in Alpine 3.19. + +**Notes:** +- `py3-mpv` can be installed as an Alpine package (`apk add py3-mpv`) or via pip (`pip install python-mpv`). Alpine package pins to 1.0.5; pip may have a newer version. Prefer pip for version control. +- `ffmpeg` binary (not just libs) is needed for the MP3 waveform extraction pipe. Add `ffmpeg` to Dockerfile, not just `ffmpeg-libs`. + +--- + +## Sources + +### Primary (HIGH confidence) +- Alpine 3.19 package registry — confirmed: `py3-mpv` 1.0.5, `mpv-libs` 0.37.0, `libsndfile` 1.2.2, `ffmpeg-libs` 6.1.1, `ffmpeg` 6.1.1 +- [github.com/jaseg/python-mpv](https://github.com/jaseg/python-mpv) — version 1.0.8 on main; property names, constructor params, observer pattern, thread model +- [pkgs.alpinelinux.org/package/v3.19/community/x86_64/py3-mpv](https://pkgs.alpinelinux.org/package/v3.19/community/x86_64/py3-mpv) — confirms py3-mpv 1.0.5 depends on mpv-libs + python3 + +### Secondary (MEDIUM confidence) +- [mpv JSON IPC issue #5661](https://github.com/mpv-player/mpv/issues/5661) — observe_property time-pos fires per frame; polling is correct workaround; closed as by-design +- [python-soundfile docs](https://python-soundfile.readthedocs.io/) — confirmed libsndfile backing, no MP3 support, numpy array output +- [FastAPI WebSocket docs](https://fastapi.tiangolo.com/advanced/websockets/) — background task + broadcast pattern +- [Python asyncio docs](https://docs.python.org/3/library/asyncio-dev.html) — `run_coroutine_threadsafe` for thread-to-async bridge + +### Tertiary (LOW confidence) +- Docker audio device mounting approach (`/dev/snd`, PulseAudio socket) — sourced from community forums; not verified against official Docker docs. Moot for this deployment since `ao=null` is the correct choice. + +--- + +## Metadata + +**Confidence breakdown:** +- Standard stack: HIGH — all packages verified against Alpine 3.19 package registry +- Architecture: HIGH — patterns verified against python-mpv source and FastAPI docs +- Pitfalls: HIGH for polling vs observe_property (verified against official mpv issue tracker); MEDIUM for Docker audio (community sources) +- INF-03 Linux assessment: HIGH — `timeBeginPeriod` is Windows-only Win32 API + +**Research date:** 2026-04-05 +**Valid until:** 2026-10-05 (stable libraries; Alpine 3.19 EOL before then but packages won't change)