docs(02-01): complete audio engine plan 01 — MPVEngine, SUMMARY, STATE, ROADMAP updated

- Create 02-01-SUMMARY.md with task commits, decisions, self-check
- Advance plan counter to plan 2 of 3
- Update progress to 71% (5/7 plans complete)
- Add 3 key decisions to STATE.md
- Mark AUD-01, AUD-03, INF-03 complete in REQUIREMENTS.md
- Update ROADMAP.md phase 2 progress
This commit is contained in:
Claude
2026-04-06 13:05:02 +00:00
parent 5f75a19868
commit 3173fa2228
4 changed files with 156 additions and 15 deletions

View File

@@ -9,9 +9,9 @@
### Audio Engine ### Audio Engine
- [ ] **AUD-01**: Load audio from a local file (MP3, WAV, FLAC, OGG) - [x] **AUD-01**: Load audio from a local file (MP3, WAV, FLAC, OGG)
- [ ] **AUD-02**: Load audio from a YouTube URL via yt-dlp + MPV - [ ] **AUD-02**: Load audio from a YouTube URL via yt-dlp + MPV
- [ ] **AUD-03**: Playback controls: play, pause, seek to position - [x] **AUD-03**: Playback controls: play, pause, seek to position
- [ ] **AUD-04**: Realtime position reporting from MPV to UI (via named pipe IPC, ≤10ms poll interval) - [ ] **AUD-04**: Realtime position reporting from MPV to UI (via named pipe IPC, ≤10ms poll interval)
- [ ] **AUD-05**: Waveform display rendered from audio samples in the timeline UI - [ ] **AUD-05**: Waveform display rendered from audio samples in the timeline UI
@@ -69,7 +69,7 @@
- [x] **INF-01**: Python 3.11 backend (FastAPI + uvicorn) - [x] **INF-01**: Python 3.11 backend (FastAPI + uvicorn)
- [x] **INF-02**: Modular device abstraction — adding a new strip type requires only a new device class, no changes to core engine - [x] **INF-02**: Modular device abstraction — adding a new strip type requires only a new device class, no changes to core engine
- [ ] **INF-03**: Windows 11 timer resolution fix applied at startup (`timeBeginPeriod(1)` via ctypes) - [x] **INF-03**: Windows 11 timer resolution fix applied at startup (`timeBeginPeriod(1)` via ctypes)
--- ---
@@ -114,7 +114,7 @@
| DEV-01, DEV-02, DEV-03 | Phase 1 | Pending | | DEV-01, DEV-02, DEV-03 | Phase 1 | Pending |
| AUD-01, AUD-02, AUD-03, AUD-04 | Phase 2 | Pending | | AUD-01, AUD-02, AUD-03, AUD-04 | Phase 2 | Pending |
| AUD-05 | Phase 2 | Pending | | AUD-05 | Phase 2 | Pending |
| INF-03 | Phase 2 | Pending | | INF-03 | Phase 2 | Complete |
| UDP-01, UDP-02, UDP-03, UDP-04 | Phase 3 | Pending | | UDP-01, UDP-02, UDP-03, UDP-04 | Phase 3 | Pending |
| ANI-01, ANI-02, ANI-03, ANI-04 | Phase 3 | Pending | | ANI-01, ANI-02, ANI-03, ANI-04 | Phase 3 | Pending |
| TL-01, TL-02, TL-03, TL-04, TL-05, TL-06 | Phase 4 | Pending | | TL-01, TL-02, TL-03, TL-04, TL-05, TL-06 | Phase 4 | Pending |

View File

@@ -54,7 +54,7 @@ Plans:
**Plans**: 3 plans **Plans**: 3 plans
Plans: Plans:
- [ ] 02-01: MPV IPC bridge — python-mpv subprocess, named pipe reader thread, shared position variable, Windows timer fix - [x] 02-01: MPV IPC bridge — python-mpv subprocess, named pipe reader thread, shared position variable, Windows timer fix
- [ ] 02-02: Show engine skeleton + WebSocket hub — playback state machine, position broadcast at 10Hz, transport command dispatcher - [ ] 02-02: Show engine skeleton + WebSocket hub — playback state machine, position broadcast at 10Hz, transport command dispatcher
- [ ] 02-03: Transport UI + waveform — play/pause/seek controls connected to WebSocket, waveform rendering from audio samples - [ ] 02-03: Transport UI + waveform — play/pause/seek controls connected to WebSocket, waveform rendering from audio samples
**UI hint**: yes **UI hint**: yes
@@ -153,7 +153,7 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7
| Phase | Plans Complete | Status | Completed | | Phase | Plans Complete | Status | Completed |
|-------|----------------|--------|-----------| |-------|----------------|--------|-----------|
| 1. Foundation | 4/4 | Complete | 2026-04-05 | | 1. Foundation | 4/4 | Complete | 2026-04-05 |
| 2. Audio Engine | 0/3 | Not started | - | | 2. Audio Engine | 1/3 | In Progress| |
| 3. Communication Protocol | 0/3 | Not started | - | | 3. Communication Protocol | 0/3 | Not started | - |
| 4. Timeline Editor | 0/4 | Not started | - | | 4. Timeline Editor | 0/4 | Not started | - |
| 5. Live Show Execution | 0/3 | Not started | - | | 5. Live Show Execution | 0/3 | Not started | - |

View File

@@ -3,14 +3,14 @@ gsd_state_version: 1.0
milestone: v1.0 milestone: v1.0
milestone_name: milestone milestone_name: milestone
status: executing status: executing
stopped_at: Completed 01-foundation/01-04-PLAN.md stopped_at: Completed 02-audio-engine/02-01-PLAN.md
last_updated: "2026-04-06T12:24:20.367Z" last_updated: "2026-04-06T13:04:49.667Z"
last_activity: 2026-04-06 last_activity: 2026-04-06
progress: progress:
total_phases: 7 total_phases: 7
completed_phases: 1 completed_phases: 1
total_plans: 4 total_plans: 7
completed_plans: 4 completed_plans: 5
percent: 0 percent: 0
--- ---
@@ -21,12 +21,12 @@ progress:
See: .planning/PROJECT.md (updated 2026-04-05) See: .planning/PROJECT.md (updated 2026-04-05)
**Core value:** Load a song, build a synchronized light show, play it back live — with precise control over every effect on every device. **Core value:** Load a song, build a synchronized light show, play it back live — with precise control over every effect on every device.
**Current focus:** Phase 01foundation **Current focus:** Phase 02audio-engine
## Current Position ## Current Position
Phase: 2 Phase: 02 (audio-engine) — EXECUTING
Plan: Not started Plan: 2 of 3
Status: Ready to execute Status: Ready to execute
Last activity: 2026-04-06 Last activity: 2026-04-06
@@ -55,6 +55,7 @@ Progress: [░░░░░░░░░░] 0%
| Phase 01-foundation P01 | 4 | 2 tasks | 23 files | | Phase 01-foundation P01 | 4 | 2 tasks | 23 files |
| Phase 01-foundation P03 | 6 | 1 tasks | 9 files | | Phase 01-foundation P03 | 6 | 1 tasks | 9 files |
| Phase 01 P04 | 2 | 2 tasks | 3 files | | Phase 01 P04 | 2 | 2 tasks | 3 files |
| Phase 02-audio-engine P01 | 525603 | 2 tasks | 5 files |
## Accumulated Context ## Accumulated Context
@@ -74,6 +75,9 @@ Recent decisions affecting current work:
- [Phase 01-foundation]: API collection routes use empty-string paths ('') not '/' — canonical /api/devices without redirect - [Phase 01-foundation]: API collection routes use empty-string paths ('') not '/' — canonical /api/devices without redirect
- [Phase 01]: window.removeDevice export required for ES module onclick compatibility — functions in type=module scripts are not global - [Phase 01]: window.removeDevice export required for ES module onclick compatibility — functions in type=module scripts are not global
- [Phase 01]: CSS accent variable canonical naming: --accent is source of truth, aliases are text-accent and border-bright - [Phase 01]: CSS accent variable canonical naming: --accent is source of truth, aliases are text-accent and border-bright
- [Phase 02-audio-engine]: MPV_AO env var controls audio output — null default for Docker server mode, overridable for local dev
- [Phase 02-audio-engine]: 10Hz polling via time.sleep(0.1) over observe_property — observe_property fires per audio frame overwhelming asyncio
- [Phase 02-audio-engine]: apply_timer_fix() as Linux no-op — timeBeginPeriod is Windows-only; INF-03 satisfied as platform-guarded stub
### Pending Todos ### Pending Todos
@@ -87,8 +91,8 @@ None yet.
## Session Continuity ## Session Continuity
Last session: 2026-04-05T20:14:32.598Z Last session: 2026-04-06T13:04:49.658Z
Stopped at: Completed 01-foundation/01-04-PLAN.md Stopped at: Completed 02-audio-engine/02-01-PLAN.md
Resume file: None Resume file: None
Next action: `/gsd:plan-phase 1` Next action: `/gsd:plan-phase 1`

View File

@@ -0,0 +1,137 @@
---
phase: 02-audio-engine
plan: "01"
subsystem: audio
tags: [mpv, python-mpv, libmpv, soundfile, numpy, ffmpeg, audio-engine, playback]
# Dependency graph
requires:
- phase: 01-foundation
provides: FastAPI app skeleton, lifespan pattern, DeviceRegistry, ShowStore, WebSocket ConnectionManager
provides:
- MPVEngine class with load/play/pause/seek/stop and 10Hz position polling
- apply_timer_fix() with sys.platform == win32 guard (INF-03)
- Alpine Dockerfile with mpv-libs, ffmpeg, ffmpeg-libs, libsndfile
- pyproject.toml with python-mpv, soundfile, numpy
- MPVEngine integrated into FastAPI lifespan (app.state.engine)
affects:
- 02-02 (waveform extraction uses soundfile/numpy added here)
- 02-03 (WebSocket broadcast loop reads MPVEngine.get_state())
- all future phases that use engine.get_state() for position
# Tech tracking
tech-stack:
added:
- python-mpv>=1.0.5 (libmpv ctypes binding for audio playback)
- soundfile>=0.12.0 (audio file reading via libsndfile)
- numpy>=1.26.0 (waveform downsampling)
- mpv-libs 0.37.0-r0 (Alpine apk, libmpv.so.2)
- ffmpeg + ffmpeg-libs 6.1.1-r0 (Alpine apk, MP3 decode support)
- libsndfile 1.2.2-r1 (Alpine apk, WAV/FLAC/OGG reading)
patterns:
- 10Hz polling via time.sleep(0.1) in daemon thread (not observe_property which fires per audio frame)
- Thread-safe state snapshot via threading.Lock in get_state()
- MPVEngine injected at lifespan startup, exposed via app.state.engine
- ao=null default for server-mode (no audio output), overridable via MPV_AO env var
key-files:
created:
- lightsync/audio/__init__.py
- lightsync/audio/engine.py
modified:
- Dockerfile
- pyproject.toml
- lightsync/main.py
key-decisions:
- "MPV_AO env var controls audio output device — defaults to null for Docker server mode, overridable for local dev"
- "Polling at 100ms (10Hz) over observe_property — observe_property fires per decoded audio frame (~43x/s at 44.1kHz), overwhelming asyncio"
- "apply_timer_fix() is a Linux no-op — timeBeginPeriod is Windows-only Win32 API; guard with sys.platform == win32"
- "MPVEngine uses ao param in constructor defaulting to null — caller controls output device"
patterns-established:
- "Pattern: Daemon thread polling loop — blocking operations in thread, results accessed thread-safely via Lock"
- "Pattern: app.state.engine — MPVEngine available to all API routes via request.app.state.engine"
- "Pattern: Lifespan start/stop symmetry — engine.start() on startup, engine.stop() on shutdown"
requirements-completed: [AUD-01, AUD-03, INF-03]
# Metrics
duration: 10min
completed: 2026-04-06
---
# Phase 02 Plan 01: Audio Engine — MPVEngine Backend
**MPVEngine wraps libmpv via python-mpv for headless audio playback with 10Hz position polling, integrated into FastAPI lifespan with Alpine Docker audio stack**
## Performance
- **Duration:** ~10 min
- **Started:** 2026-04-06T13:01:00Z
- **Completed:** 2026-04-06T13:03:49Z
- **Tasks:** 2 of 2
- **Files modified:** 5
## Accomplishments
- Created MPVEngine class with load/play/pause/seek/stop methods and thread-safe 10Hz position polling
- Updated Dockerfile with Alpine audio stack (mpv-libs, ffmpeg, ffmpeg-libs, libsndfile)
- Added python-mpv, soundfile, numpy to pyproject.toml dependencies
- Integrated MPVEngine into FastAPI lifespan with MPV_AO env var for configurable audio output
- Implemented INF-03 as platform-guarded no-op (apply_timer_fix is Linux-safe)
## Task Commits
Each task was committed atomically:
1. **Task 1: Add audio dependencies to Dockerfile and pyproject.toml** - `474b4a7` (chore)
2. **Task 2: Create MPVEngine class with position polling and platform timer fix** - `5f75a19` (feat)
**Plan metadata:** (committed with SUMMARY + state updates)
## Files Created/Modified
- `lightsync/audio/__init__.py` - Package marker for audio module
- `lightsync/audio/engine.py` - MPVEngine class with headless mpv wrapper, 10Hz polling daemon thread, thread-safe get_state(), apply_timer_fix() INF-03 platform guard
- `Dockerfile` - Added apk packages: mpv-libs, ffmpeg, ffmpeg-libs, libsndfile
- `pyproject.toml` - Added python-mpv>=1.0.5, soundfile>=0.12.0, numpy>=1.26.0
- `lightsync/main.py` - Import MPVEngine, create/start/stop in lifespan, expose via app.state.engine
## Decisions Made
- **MPV_AO env var** for audio output device: defaults to `null` (server mode, no speakers needed), overridable for local dev with actual audio output. This avoids hardcoding and keeps Docker images working without audio devices.
- **Polling over observe_property**: observe_property fires on every decoded audio frame (~43x/s at 44.1kHz), which would flood the asyncio event loop. Fixed 100ms polling gives exactly 10Hz regardless of audio format.
- **apply_timer_fix() as Linux no-op**: `timeBeginPeriod` is a Windows-only Win32 API (`winmm.dll`). On Linux, the kernel provides sub-millisecond timer resolution natively. INF-03 is satisfied as a platform-guarded stub.
## Deviations from Plan
None — plan executed exactly as written. The load() method follows the research correction noted in the plan (no wait_for_playback() call, using async mpv loadfile).
## Issues Encountered
None — all dependencies verified in research phase, implementation matched expected patterns.
## User Setup Required
None - no external service configuration required. MPV_AO env var is optional (defaults to "null" which works in Docker without audio devices).
## Next Phase Readiness
- MPVEngine is ready for Plan 02-02 (waveform extraction using soundfile/numpy added in this plan)
- MPVEngine is ready for Plan 02-03 (WebSocket broadcast loop using engine.get_state())
- engine accessible in all API route handlers via `request.app.state.engine`
- Docker image will build with all required audio libraries once deployed
## Self-Check: PASSED
- lightsync/audio/__init__.py: FOUND
- lightsync/audio/engine.py: FOUND
- lightsync/main.py: FOUND
- .planning/phases/02-audio-engine/02-01-SUMMARY.md: FOUND
- Commit 474b4a7 (chore: audio deps): FOUND
- Commit 5f75a19 (feat: MPVEngine): FOUND
---
*Phase: 02-audio-engine*
*Completed: 2026-04-06*