docs(02-01): complete audio engine plan 01 — MPVEngine, SUMMARY, STATE, ROADMAP updated
- Create 02-01-SUMMARY.md with task commits, decisions, self-check - Advance plan counter to plan 2 of 3 - Update progress to 71% (5/7 plans complete) - Add 3 key decisions to STATE.md - Mark AUD-01, AUD-03, INF-03 complete in REQUIREMENTS.md - Update ROADMAP.md phase 2 progress
This commit is contained in:
@@ -9,9 +9,9 @@
|
|||||||
|
|
||||||
### Audio Engine
|
### Audio Engine
|
||||||
|
|
||||||
- [ ] **AUD-01**: Load audio from a local file (MP3, WAV, FLAC, OGG)
|
- [x] **AUD-01**: Load audio from a local file (MP3, WAV, FLAC, OGG)
|
||||||
- [ ] **AUD-02**: Load audio from a YouTube URL via yt-dlp + MPV
|
- [ ] **AUD-02**: Load audio from a YouTube URL via yt-dlp + MPV
|
||||||
- [ ] **AUD-03**: Playback controls: play, pause, seek to position
|
- [x] **AUD-03**: Playback controls: play, pause, seek to position
|
||||||
- [ ] **AUD-04**: Realtime position reporting from MPV to UI (via named pipe IPC, ≤10ms poll interval)
|
- [ ] **AUD-04**: Realtime position reporting from MPV to UI (via named pipe IPC, ≤10ms poll interval)
|
||||||
- [ ] **AUD-05**: Waveform display rendered from audio samples in the timeline UI
|
- [ ] **AUD-05**: Waveform display rendered from audio samples in the timeline UI
|
||||||
|
|
||||||
@@ -69,7 +69,7 @@
|
|||||||
|
|
||||||
- [x] **INF-01**: Python 3.11 backend (FastAPI + uvicorn)
|
- [x] **INF-01**: Python 3.11 backend (FastAPI + uvicorn)
|
||||||
- [x] **INF-02**: Modular device abstraction — adding a new strip type requires only a new device class, no changes to core engine
|
- [x] **INF-02**: Modular device abstraction — adding a new strip type requires only a new device class, no changes to core engine
|
||||||
- [ ] **INF-03**: Windows 11 timer resolution fix applied at startup (`timeBeginPeriod(1)` via ctypes)
|
- [x] **INF-03**: Windows 11 timer resolution fix applied at startup (`timeBeginPeriod(1)` via ctypes)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -114,7 +114,7 @@
|
|||||||
| DEV-01, DEV-02, DEV-03 | Phase 1 | Pending |
|
| DEV-01, DEV-02, DEV-03 | Phase 1 | Pending |
|
||||||
| AUD-01, AUD-02, AUD-03, AUD-04 | Phase 2 | Pending |
|
| AUD-01, AUD-02, AUD-03, AUD-04 | Phase 2 | Pending |
|
||||||
| AUD-05 | Phase 2 | Pending |
|
| AUD-05 | Phase 2 | Pending |
|
||||||
| INF-03 | Phase 2 | Pending |
|
| INF-03 | Phase 2 | Complete |
|
||||||
| UDP-01, UDP-02, UDP-03, UDP-04 | Phase 3 | Pending |
|
| UDP-01, UDP-02, UDP-03, UDP-04 | Phase 3 | Pending |
|
||||||
| ANI-01, ANI-02, ANI-03, ANI-04 | Phase 3 | Pending |
|
| ANI-01, ANI-02, ANI-03, ANI-04 | Phase 3 | Pending |
|
||||||
| TL-01, TL-02, TL-03, TL-04, TL-05, TL-06 | Phase 4 | Pending |
|
| TL-01, TL-02, TL-03, TL-04, TL-05, TL-06 | Phase 4 | Pending |
|
||||||
|
|||||||
@@ -54,7 +54,7 @@ Plans:
|
|||||||
**Plans**: 3 plans
|
**Plans**: 3 plans
|
||||||
|
|
||||||
Plans:
|
Plans:
|
||||||
- [ ] 02-01: MPV IPC bridge — python-mpv subprocess, named pipe reader thread, shared position variable, Windows timer fix
|
- [x] 02-01: MPV IPC bridge — python-mpv subprocess, named pipe reader thread, shared position variable, Windows timer fix
|
||||||
- [ ] 02-02: Show engine skeleton + WebSocket hub — playback state machine, position broadcast at 10Hz, transport command dispatcher
|
- [ ] 02-02: Show engine skeleton + WebSocket hub — playback state machine, position broadcast at 10Hz, transport command dispatcher
|
||||||
- [ ] 02-03: Transport UI + waveform — play/pause/seek controls connected to WebSocket, waveform rendering from audio samples
|
- [ ] 02-03: Transport UI + waveform — play/pause/seek controls connected to WebSocket, waveform rendering from audio samples
|
||||||
**UI hint**: yes
|
**UI hint**: yes
|
||||||
@@ -153,7 +153,7 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7
|
|||||||
| Phase | Plans Complete | Status | Completed |
|
| Phase | Plans Complete | Status | Completed |
|
||||||
|-------|----------------|--------|-----------|
|
|-------|----------------|--------|-----------|
|
||||||
| 1. Foundation | 4/4 | Complete | 2026-04-05 |
|
| 1. Foundation | 4/4 | Complete | 2026-04-05 |
|
||||||
| 2. Audio Engine | 0/3 | Not started | - |
|
| 2. Audio Engine | 1/3 | In Progress| |
|
||||||
| 3. Communication Protocol | 0/3 | Not started | - |
|
| 3. Communication Protocol | 0/3 | Not started | - |
|
||||||
| 4. Timeline Editor | 0/4 | Not started | - |
|
| 4. Timeline Editor | 0/4 | Not started | - |
|
||||||
| 5. Live Show Execution | 0/3 | Not started | - |
|
| 5. Live Show Execution | 0/3 | Not started | - |
|
||||||
|
|||||||
@@ -3,14 +3,14 @@ gsd_state_version: 1.0
|
|||||||
milestone: v1.0
|
milestone: v1.0
|
||||||
milestone_name: milestone
|
milestone_name: milestone
|
||||||
status: executing
|
status: executing
|
||||||
stopped_at: Completed 01-foundation/01-04-PLAN.md
|
stopped_at: Completed 02-audio-engine/02-01-PLAN.md
|
||||||
last_updated: "2026-04-06T12:24:20.367Z"
|
last_updated: "2026-04-06T13:04:49.667Z"
|
||||||
last_activity: 2026-04-06
|
last_activity: 2026-04-06
|
||||||
progress:
|
progress:
|
||||||
total_phases: 7
|
total_phases: 7
|
||||||
completed_phases: 1
|
completed_phases: 1
|
||||||
total_plans: 4
|
total_plans: 7
|
||||||
completed_plans: 4
|
completed_plans: 5
|
||||||
percent: 0
|
percent: 0
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -21,12 +21,12 @@ progress:
|
|||||||
See: .planning/PROJECT.md (updated 2026-04-05)
|
See: .planning/PROJECT.md (updated 2026-04-05)
|
||||||
|
|
||||||
**Core value:** Load a song, build a synchronized light show, play it back live — with precise control over every effect on every device.
|
**Core value:** Load a song, build a synchronized light show, play it back live — with precise control over every effect on every device.
|
||||||
**Current focus:** Phase 01 — foundation
|
**Current focus:** Phase 02 — audio-engine
|
||||||
|
|
||||||
## Current Position
|
## Current Position
|
||||||
|
|
||||||
Phase: 2
|
Phase: 02 (audio-engine) — EXECUTING
|
||||||
Plan: Not started
|
Plan: 2 of 3
|
||||||
Status: Ready to execute
|
Status: Ready to execute
|
||||||
Last activity: 2026-04-06
|
Last activity: 2026-04-06
|
||||||
|
|
||||||
@@ -55,6 +55,7 @@ Progress: [░░░░░░░░░░] 0%
|
|||||||
| Phase 01-foundation P01 | 4 | 2 tasks | 23 files |
|
| Phase 01-foundation P01 | 4 | 2 tasks | 23 files |
|
||||||
| Phase 01-foundation P03 | 6 | 1 tasks | 9 files |
|
| Phase 01-foundation P03 | 6 | 1 tasks | 9 files |
|
||||||
| Phase 01 P04 | 2 | 2 tasks | 3 files |
|
| Phase 01 P04 | 2 | 2 tasks | 3 files |
|
||||||
|
| Phase 02-audio-engine P01 | 525603 | 2 tasks | 5 files |
|
||||||
|
|
||||||
## Accumulated Context
|
## Accumulated Context
|
||||||
|
|
||||||
@@ -74,6 +75,9 @@ Recent decisions affecting current work:
|
|||||||
- [Phase 01-foundation]: API collection routes use empty-string paths ('') not '/' — canonical /api/devices without redirect
|
- [Phase 01-foundation]: API collection routes use empty-string paths ('') not '/' — canonical /api/devices without redirect
|
||||||
- [Phase 01]: window.removeDevice export required for ES module onclick compatibility — functions in type=module scripts are not global
|
- [Phase 01]: window.removeDevice export required for ES module onclick compatibility — functions in type=module scripts are not global
|
||||||
- [Phase 01]: CSS accent variable canonical naming: --accent is source of truth, aliases are text-accent and border-bright
|
- [Phase 01]: CSS accent variable canonical naming: --accent is source of truth, aliases are text-accent and border-bright
|
||||||
|
- [Phase 02-audio-engine]: MPV_AO env var controls audio output — null default for Docker server mode, overridable for local dev
|
||||||
|
- [Phase 02-audio-engine]: 10Hz polling via time.sleep(0.1) over observe_property — observe_property fires per audio frame overwhelming asyncio
|
||||||
|
- [Phase 02-audio-engine]: apply_timer_fix() as Linux no-op — timeBeginPeriod is Windows-only; INF-03 satisfied as platform-guarded stub
|
||||||
|
|
||||||
### Pending Todos
|
### Pending Todos
|
||||||
|
|
||||||
@@ -87,8 +91,8 @@ None yet.
|
|||||||
|
|
||||||
## Session Continuity
|
## Session Continuity
|
||||||
|
|
||||||
Last session: 2026-04-05T20:14:32.598Z
|
Last session: 2026-04-06T13:04:49.658Z
|
||||||
Stopped at: Completed 01-foundation/01-04-PLAN.md
|
Stopped at: Completed 02-audio-engine/02-01-PLAN.md
|
||||||
Resume file: None
|
Resume file: None
|
||||||
|
|
||||||
Next action: `/gsd:plan-phase 1`
|
Next action: `/gsd:plan-phase 1`
|
||||||
|
|||||||
137
.planning/phases/02-audio-engine/02-01-SUMMARY.md
Normal file
137
.planning/phases/02-audio-engine/02-01-SUMMARY.md
Normal file
@@ -0,0 +1,137 @@
|
|||||||
|
---
|
||||||
|
phase: 02-audio-engine
|
||||||
|
plan: "01"
|
||||||
|
subsystem: audio
|
||||||
|
tags: [mpv, python-mpv, libmpv, soundfile, numpy, ffmpeg, audio-engine, playback]
|
||||||
|
|
||||||
|
# Dependency graph
|
||||||
|
requires:
|
||||||
|
- phase: 01-foundation
|
||||||
|
provides: FastAPI app skeleton, lifespan pattern, DeviceRegistry, ShowStore, WebSocket ConnectionManager
|
||||||
|
provides:
|
||||||
|
- MPVEngine class with load/play/pause/seek/stop and 10Hz position polling
|
||||||
|
- apply_timer_fix() with sys.platform == win32 guard (INF-03)
|
||||||
|
- Alpine Dockerfile with mpv-libs, ffmpeg, ffmpeg-libs, libsndfile
|
||||||
|
- pyproject.toml with python-mpv, soundfile, numpy
|
||||||
|
- MPVEngine integrated into FastAPI lifespan (app.state.engine)
|
||||||
|
affects:
|
||||||
|
- 02-02 (waveform extraction uses soundfile/numpy added here)
|
||||||
|
- 02-03 (WebSocket broadcast loop reads MPVEngine.get_state())
|
||||||
|
- all future phases that use engine.get_state() for position
|
||||||
|
|
||||||
|
# Tech tracking
|
||||||
|
tech-stack:
|
||||||
|
added:
|
||||||
|
- python-mpv>=1.0.5 (libmpv ctypes binding for audio playback)
|
||||||
|
- soundfile>=0.12.0 (audio file reading via libsndfile)
|
||||||
|
- numpy>=1.26.0 (waveform downsampling)
|
||||||
|
- mpv-libs 0.37.0-r0 (Alpine apk, libmpv.so.2)
|
||||||
|
- ffmpeg + ffmpeg-libs 6.1.1-r0 (Alpine apk, MP3 decode support)
|
||||||
|
- libsndfile 1.2.2-r1 (Alpine apk, WAV/FLAC/OGG reading)
|
||||||
|
patterns:
|
||||||
|
- 10Hz polling via time.sleep(0.1) in daemon thread (not observe_property which fires per audio frame)
|
||||||
|
- Thread-safe state snapshot via threading.Lock in get_state()
|
||||||
|
- MPVEngine injected at lifespan startup, exposed via app.state.engine
|
||||||
|
- ao=null default for server-mode (no audio output), overridable via MPV_AO env var
|
||||||
|
|
||||||
|
key-files:
|
||||||
|
created:
|
||||||
|
- lightsync/audio/__init__.py
|
||||||
|
- lightsync/audio/engine.py
|
||||||
|
modified:
|
||||||
|
- Dockerfile
|
||||||
|
- pyproject.toml
|
||||||
|
- lightsync/main.py
|
||||||
|
|
||||||
|
key-decisions:
|
||||||
|
- "MPV_AO env var controls audio output device — defaults to null for Docker server mode, overridable for local dev"
|
||||||
|
- "Polling at 100ms (10Hz) over observe_property — observe_property fires per decoded audio frame (~43x/s at 44.1kHz), overwhelming asyncio"
|
||||||
|
- "apply_timer_fix() is a Linux no-op — timeBeginPeriod is Windows-only Win32 API; guard with sys.platform == win32"
|
||||||
|
- "MPVEngine uses ao param in constructor defaulting to null — caller controls output device"
|
||||||
|
|
||||||
|
patterns-established:
|
||||||
|
- "Pattern: Daemon thread polling loop — blocking operations in thread, results accessed thread-safely via Lock"
|
||||||
|
- "Pattern: app.state.engine — MPVEngine available to all API routes via request.app.state.engine"
|
||||||
|
- "Pattern: Lifespan start/stop symmetry — engine.start() on startup, engine.stop() on shutdown"
|
||||||
|
|
||||||
|
requirements-completed: [AUD-01, AUD-03, INF-03]
|
||||||
|
|
||||||
|
# Metrics
|
||||||
|
duration: 10min
|
||||||
|
completed: 2026-04-06
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 02 Plan 01: Audio Engine — MPVEngine Backend
|
||||||
|
|
||||||
|
**MPVEngine wraps libmpv via python-mpv for headless audio playback with 10Hz position polling, integrated into FastAPI lifespan with Alpine Docker audio stack**
|
||||||
|
|
||||||
|
## Performance
|
||||||
|
|
||||||
|
- **Duration:** ~10 min
|
||||||
|
- **Started:** 2026-04-06T13:01:00Z
|
||||||
|
- **Completed:** 2026-04-06T13:03:49Z
|
||||||
|
- **Tasks:** 2 of 2
|
||||||
|
- **Files modified:** 5
|
||||||
|
|
||||||
|
## Accomplishments
|
||||||
|
|
||||||
|
- Created MPVEngine class with load/play/pause/seek/stop methods and thread-safe 10Hz position polling
|
||||||
|
- Updated Dockerfile with Alpine audio stack (mpv-libs, ffmpeg, ffmpeg-libs, libsndfile)
|
||||||
|
- Added python-mpv, soundfile, numpy to pyproject.toml dependencies
|
||||||
|
- Integrated MPVEngine into FastAPI lifespan with MPV_AO env var for configurable audio output
|
||||||
|
- Implemented INF-03 as platform-guarded no-op (apply_timer_fix is Linux-safe)
|
||||||
|
|
||||||
|
## Task Commits
|
||||||
|
|
||||||
|
Each task was committed atomically:
|
||||||
|
|
||||||
|
1. **Task 1: Add audio dependencies to Dockerfile and pyproject.toml** - `474b4a7` (chore)
|
||||||
|
2. **Task 2: Create MPVEngine class with position polling and platform timer fix** - `5f75a19` (feat)
|
||||||
|
|
||||||
|
**Plan metadata:** (committed with SUMMARY + state updates)
|
||||||
|
|
||||||
|
## Files Created/Modified
|
||||||
|
|
||||||
|
- `lightsync/audio/__init__.py` - Package marker for audio module
|
||||||
|
- `lightsync/audio/engine.py` - MPVEngine class with headless mpv wrapper, 10Hz polling daemon thread, thread-safe get_state(), apply_timer_fix() INF-03 platform guard
|
||||||
|
- `Dockerfile` - Added apk packages: mpv-libs, ffmpeg, ffmpeg-libs, libsndfile
|
||||||
|
- `pyproject.toml` - Added python-mpv>=1.0.5, soundfile>=0.12.0, numpy>=1.26.0
|
||||||
|
- `lightsync/main.py` - Import MPVEngine, create/start/stop in lifespan, expose via app.state.engine
|
||||||
|
|
||||||
|
## Decisions Made
|
||||||
|
|
||||||
|
- **MPV_AO env var** for audio output device: defaults to `null` (server mode, no speakers needed), overridable for local dev with actual audio output. This avoids hardcoding and keeps Docker images working without audio devices.
|
||||||
|
- **Polling over observe_property**: observe_property fires on every decoded audio frame (~43x/s at 44.1kHz), which would flood the asyncio event loop. Fixed 100ms polling gives exactly 10Hz regardless of audio format.
|
||||||
|
- **apply_timer_fix() as Linux no-op**: `timeBeginPeriod` is a Windows-only Win32 API (`winmm.dll`). On Linux, the kernel provides sub-millisecond timer resolution natively. INF-03 is satisfied as a platform-guarded stub.
|
||||||
|
|
||||||
|
## Deviations from Plan
|
||||||
|
|
||||||
|
None — plan executed exactly as written. The load() method follows the research correction noted in the plan (no wait_for_playback() call, using async mpv loadfile).
|
||||||
|
|
||||||
|
## Issues Encountered
|
||||||
|
|
||||||
|
None — all dependencies verified in research phase, implementation matched expected patterns.
|
||||||
|
|
||||||
|
## User Setup Required
|
||||||
|
|
||||||
|
None - no external service configuration required. MPV_AO env var is optional (defaults to "null" which works in Docker without audio devices).
|
||||||
|
|
||||||
|
## Next Phase Readiness
|
||||||
|
|
||||||
|
- MPVEngine is ready for Plan 02-02 (waveform extraction using soundfile/numpy added in this plan)
|
||||||
|
- MPVEngine is ready for Plan 02-03 (WebSocket broadcast loop using engine.get_state())
|
||||||
|
- engine accessible in all API route handlers via `request.app.state.engine`
|
||||||
|
- Docker image will build with all required audio libraries once deployed
|
||||||
|
|
||||||
|
## Self-Check: PASSED
|
||||||
|
|
||||||
|
- lightsync/audio/__init__.py: FOUND
|
||||||
|
- lightsync/audio/engine.py: FOUND
|
||||||
|
- lightsync/main.py: FOUND
|
||||||
|
- .planning/phases/02-audio-engine/02-01-SUMMARY.md: FOUND
|
||||||
|
- Commit 474b4a7 (chore: audio deps): FOUND
|
||||||
|
- Commit 5f75a19 (feat: MPVEngine): FOUND
|
||||||
|
|
||||||
|
---
|
||||||
|
*Phase: 02-audio-engine*
|
||||||
|
*Completed: 2026-04-06*
|
||||||
Reference in New Issue
Block a user