From 3173fa222854a2cd7bf6907d1c81740263cd2196 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 6 Apr 2026 13:05:02 +0000 Subject: [PATCH] =?UTF-8?q?docs(02-01):=20complete=20audio=20engine=20plan?= =?UTF-8?q?=2001=20=E2=80=94=20MPVEngine,=20SUMMARY,=20STATE,=20ROADMAP=20?= =?UTF-8?q?updated?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Create 02-01-SUMMARY.md with task commits, decisions, self-check - Advance plan counter to plan 2 of 3 - Update progress to 71% (5/7 plans complete) - Add 3 key decisions to STATE.md - Mark AUD-01, AUD-03, INF-03 complete in REQUIREMENTS.md - Update ROADMAP.md phase 2 progress --- .planning/REQUIREMENTS.md | 8 +- .planning/ROADMAP.md | 4 +- .planning/STATE.md | 22 +-- .../phases/02-audio-engine/02-01-SUMMARY.md | 137 ++++++++++++++++++ 4 files changed, 156 insertions(+), 15 deletions(-) create mode 100644 .planning/phases/02-audio-engine/02-01-SUMMARY.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 16c6c94..6c55906 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -9,9 +9,9 @@ ### Audio Engine -- [ ] **AUD-01**: Load audio from a local file (MP3, WAV, FLAC, OGG) +- [x] **AUD-01**: Load audio from a local file (MP3, WAV, FLAC, OGG) - [ ] **AUD-02**: Load audio from a YouTube URL via yt-dlp + MPV -- [ ] **AUD-03**: Playback controls: play, pause, seek to position +- [x] **AUD-03**: Playback controls: play, pause, seek to position - [ ] **AUD-04**: Realtime position reporting from MPV to UI (via named pipe IPC, ≤10ms poll interval) - [ ] **AUD-05**: Waveform display rendered from audio samples in the timeline UI @@ -69,7 +69,7 @@ - [x] **INF-01**: Python 3.11 backend (FastAPI + uvicorn) - [x] **INF-02**: Modular device abstraction — adding a new strip type requires only a new device class, no changes to core engine -- [ ] **INF-03**: Windows 11 timer resolution fix applied at startup (`timeBeginPeriod(1)` via ctypes) +- [x] **INF-03**: Windows 11 timer resolution fix applied at startup (`timeBeginPeriod(1)` via ctypes) --- @@ -114,7 +114,7 @@ | DEV-01, DEV-02, DEV-03 | Phase 1 | Pending | | AUD-01, AUD-02, AUD-03, AUD-04 | Phase 2 | Pending | | AUD-05 | Phase 2 | Pending | -| INF-03 | Phase 2 | Pending | +| INF-03 | Phase 2 | Complete | | UDP-01, UDP-02, UDP-03, UDP-04 | Phase 3 | Pending | | ANI-01, ANI-02, ANI-03, ANI-04 | Phase 3 | Pending | | TL-01, TL-02, TL-03, TL-04, TL-05, TL-06 | Phase 4 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 0c60b3e..089ce0f 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -54,7 +54,7 @@ Plans: **Plans**: 3 plans Plans: -- [ ] 02-01: MPV IPC bridge — python-mpv subprocess, named pipe reader thread, shared position variable, Windows timer fix +- [x] 02-01: MPV IPC bridge — python-mpv subprocess, named pipe reader thread, shared position variable, Windows timer fix - [ ] 02-02: Show engine skeleton + WebSocket hub — playback state machine, position broadcast at 10Hz, transport command dispatcher - [ ] 02-03: Transport UI + waveform — play/pause/seek controls connected to WebSocket, waveform rendering from audio samples **UI hint**: yes @@ -153,7 +153,7 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7 | Phase | Plans Complete | Status | Completed | |-------|----------------|--------|-----------| | 1. Foundation | 4/4 | Complete | 2026-04-05 | -| 2. Audio Engine | 0/3 | Not started | - | +| 2. Audio Engine | 1/3 | In Progress| | | 3. Communication Protocol | 0/3 | Not started | - | | 4. Timeline Editor | 0/4 | Not started | - | | 5. Live Show Execution | 0/3 | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 9b5da61..4378772 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,14 +3,14 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone status: executing -stopped_at: Completed 01-foundation/01-04-PLAN.md -last_updated: "2026-04-06T12:24:20.367Z" +stopped_at: Completed 02-audio-engine/02-01-PLAN.md +last_updated: "2026-04-06T13:04:49.667Z" last_activity: 2026-04-06 progress: total_phases: 7 completed_phases: 1 - total_plans: 4 - completed_plans: 4 + total_plans: 7 + completed_plans: 5 percent: 0 --- @@ -21,12 +21,12 @@ progress: See: .planning/PROJECT.md (updated 2026-04-05) **Core value:** Load a song, build a synchronized light show, play it back live — with precise control over every effect on every device. -**Current focus:** Phase 01 — foundation +**Current focus:** Phase 02 — audio-engine ## Current Position -Phase: 2 -Plan: Not started +Phase: 02 (audio-engine) — EXECUTING +Plan: 2 of 3 Status: Ready to execute Last activity: 2026-04-06 @@ -55,6 +55,7 @@ Progress: [░░░░░░░░░░] 0% | Phase 01-foundation P01 | 4 | 2 tasks | 23 files | | Phase 01-foundation P03 | 6 | 1 tasks | 9 files | | Phase 01 P04 | 2 | 2 tasks | 3 files | +| Phase 02-audio-engine P01 | 525603 | 2 tasks | 5 files | ## Accumulated Context @@ -74,6 +75,9 @@ Recent decisions affecting current work: - [Phase 01-foundation]: API collection routes use empty-string paths ('') not '/' — canonical /api/devices without redirect - [Phase 01]: window.removeDevice export required for ES module onclick compatibility — functions in type=module scripts are not global - [Phase 01]: CSS accent variable canonical naming: --accent is source of truth, aliases are text-accent and border-bright +- [Phase 02-audio-engine]: MPV_AO env var controls audio output — null default for Docker server mode, overridable for local dev +- [Phase 02-audio-engine]: 10Hz polling via time.sleep(0.1) over observe_property — observe_property fires per audio frame overwhelming asyncio +- [Phase 02-audio-engine]: apply_timer_fix() as Linux no-op — timeBeginPeriod is Windows-only; INF-03 satisfied as platform-guarded stub ### Pending Todos @@ -87,8 +91,8 @@ None yet. ## Session Continuity -Last session: 2026-04-05T20:14:32.598Z -Stopped at: Completed 01-foundation/01-04-PLAN.md +Last session: 2026-04-06T13:04:49.658Z +Stopped at: Completed 02-audio-engine/02-01-PLAN.md Resume file: None Next action: `/gsd:plan-phase 1` diff --git a/.planning/phases/02-audio-engine/02-01-SUMMARY.md b/.planning/phases/02-audio-engine/02-01-SUMMARY.md new file mode 100644 index 0000000..c07293d --- /dev/null +++ b/.planning/phases/02-audio-engine/02-01-SUMMARY.md @@ -0,0 +1,137 @@ +--- +phase: 02-audio-engine +plan: "01" +subsystem: audio +tags: [mpv, python-mpv, libmpv, soundfile, numpy, ffmpeg, audio-engine, playback] + +# Dependency graph +requires: + - phase: 01-foundation + provides: FastAPI app skeleton, lifespan pattern, DeviceRegistry, ShowStore, WebSocket ConnectionManager +provides: + - MPVEngine class with load/play/pause/seek/stop and 10Hz position polling + - apply_timer_fix() with sys.platform == win32 guard (INF-03) + - Alpine Dockerfile with mpv-libs, ffmpeg, ffmpeg-libs, libsndfile + - pyproject.toml with python-mpv, soundfile, numpy + - MPVEngine integrated into FastAPI lifespan (app.state.engine) +affects: + - 02-02 (waveform extraction uses soundfile/numpy added here) + - 02-03 (WebSocket broadcast loop reads MPVEngine.get_state()) + - all future phases that use engine.get_state() for position + +# Tech tracking +tech-stack: + added: + - python-mpv>=1.0.5 (libmpv ctypes binding for audio playback) + - soundfile>=0.12.0 (audio file reading via libsndfile) + - numpy>=1.26.0 (waveform downsampling) + - mpv-libs 0.37.0-r0 (Alpine apk, libmpv.so.2) + - ffmpeg + ffmpeg-libs 6.1.1-r0 (Alpine apk, MP3 decode support) + - libsndfile 1.2.2-r1 (Alpine apk, WAV/FLAC/OGG reading) + patterns: + - 10Hz polling via time.sleep(0.1) in daemon thread (not observe_property which fires per audio frame) + - Thread-safe state snapshot via threading.Lock in get_state() + - MPVEngine injected at lifespan startup, exposed via app.state.engine + - ao=null default for server-mode (no audio output), overridable via MPV_AO env var + +key-files: + created: + - lightsync/audio/__init__.py + - lightsync/audio/engine.py + modified: + - Dockerfile + - pyproject.toml + - lightsync/main.py + +key-decisions: + - "MPV_AO env var controls audio output device — defaults to null for Docker server mode, overridable for local dev" + - "Polling at 100ms (10Hz) over observe_property — observe_property fires per decoded audio frame (~43x/s at 44.1kHz), overwhelming asyncio" + - "apply_timer_fix() is a Linux no-op — timeBeginPeriod is Windows-only Win32 API; guard with sys.platform == win32" + - "MPVEngine uses ao param in constructor defaulting to null — caller controls output device" + +patterns-established: + - "Pattern: Daemon thread polling loop — blocking operations in thread, results accessed thread-safely via Lock" + - "Pattern: app.state.engine — MPVEngine available to all API routes via request.app.state.engine" + - "Pattern: Lifespan start/stop symmetry — engine.start() on startup, engine.stop() on shutdown" + +requirements-completed: [AUD-01, AUD-03, INF-03] + +# Metrics +duration: 10min +completed: 2026-04-06 +--- + +# Phase 02 Plan 01: Audio Engine — MPVEngine Backend + +**MPVEngine wraps libmpv via python-mpv for headless audio playback with 10Hz position polling, integrated into FastAPI lifespan with Alpine Docker audio stack** + +## Performance + +- **Duration:** ~10 min +- **Started:** 2026-04-06T13:01:00Z +- **Completed:** 2026-04-06T13:03:49Z +- **Tasks:** 2 of 2 +- **Files modified:** 5 + +## Accomplishments + +- Created MPVEngine class with load/play/pause/seek/stop methods and thread-safe 10Hz position polling +- Updated Dockerfile with Alpine audio stack (mpv-libs, ffmpeg, ffmpeg-libs, libsndfile) +- Added python-mpv, soundfile, numpy to pyproject.toml dependencies +- Integrated MPVEngine into FastAPI lifespan with MPV_AO env var for configurable audio output +- Implemented INF-03 as platform-guarded no-op (apply_timer_fix is Linux-safe) + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Add audio dependencies to Dockerfile and pyproject.toml** - `474b4a7` (chore) +2. **Task 2: Create MPVEngine class with position polling and platform timer fix** - `5f75a19` (feat) + +**Plan metadata:** (committed with SUMMARY + state updates) + +## Files Created/Modified + +- `lightsync/audio/__init__.py` - Package marker for audio module +- `lightsync/audio/engine.py` - MPVEngine class with headless mpv wrapper, 10Hz polling daemon thread, thread-safe get_state(), apply_timer_fix() INF-03 platform guard +- `Dockerfile` - Added apk packages: mpv-libs, ffmpeg, ffmpeg-libs, libsndfile +- `pyproject.toml` - Added python-mpv>=1.0.5, soundfile>=0.12.0, numpy>=1.26.0 +- `lightsync/main.py` - Import MPVEngine, create/start/stop in lifespan, expose via app.state.engine + +## Decisions Made + +- **MPV_AO env var** for audio output device: defaults to `null` (server mode, no speakers needed), overridable for local dev with actual audio output. This avoids hardcoding and keeps Docker images working without audio devices. +- **Polling over observe_property**: observe_property fires on every decoded audio frame (~43x/s at 44.1kHz), which would flood the asyncio event loop. Fixed 100ms polling gives exactly 10Hz regardless of audio format. +- **apply_timer_fix() as Linux no-op**: `timeBeginPeriod` is a Windows-only Win32 API (`winmm.dll`). On Linux, the kernel provides sub-millisecond timer resolution natively. INF-03 is satisfied as a platform-guarded stub. + +## Deviations from Plan + +None — plan executed exactly as written. The load() method follows the research correction noted in the plan (no wait_for_playback() call, using async mpv loadfile). + +## Issues Encountered + +None — all dependencies verified in research phase, implementation matched expected patterns. + +## User Setup Required + +None - no external service configuration required. MPV_AO env var is optional (defaults to "null" which works in Docker without audio devices). + +## Next Phase Readiness + +- MPVEngine is ready for Plan 02-02 (waveform extraction using soundfile/numpy added in this plan) +- MPVEngine is ready for Plan 02-03 (WebSocket broadcast loop using engine.get_state()) +- engine accessible in all API route handlers via `request.app.state.engine` +- Docker image will build with all required audio libraries once deployed + +## Self-Check: PASSED + +- lightsync/audio/__init__.py: FOUND +- lightsync/audio/engine.py: FOUND +- lightsync/main.py: FOUND +- .planning/phases/02-audio-engine/02-01-SUMMARY.md: FOUND +- Commit 474b4a7 (chore: audio deps): FOUND +- Commit 5f75a19 (feat: MPVEngine): FOUND + +--- +*Phase: 02-audio-engine* +*Completed: 2026-04-06*