docs(02-03): complete waveform display and transport UI plan — AUD-05 fulfilled
- SUMMARY.md for 02-03 (waveform extractor + transport UI) - STATE.md: checkpoint at human-verify task 3, decisions recorded - ROADMAP.md: phase 2 progress updated (3/3 plans) - REQUIREMENTS.md: AUD-05 marked complete
This commit is contained in:
Claude
129
.planning/phases/02-audio-engine/02-03-SUMMARY.md
Normal file
129
.planning/phases/02-audio-engine/02-03-SUMMARY.md
Normal file
@@ -0,0 +1,129 @@
|
||||
---
|
||||
phase: 02-audio-engine
|
||||
plan: "03"
|
||||
subsystem: audio
|
||||
|
||||
tags: [waveform, soundfile, numpy, ffmpeg, transport-ui, canvas, websocket]
|
||||
|
||||
requires:
|
||||
- phase: 02-audio-engine/02-01
|
||||
provides: MPVEngine with get_state(), play/pause/seek via WebSocket
|
||||
- phase: 02-audio-engine/02-02
|
||||
provides: WebSocket broadcast loop (tick messages), REST audio endpoints in lightsync/api/audio.py
|
||||
|
||||
provides:
|
||||
- extract_peaks() function in lightsync/audio/waveform.py (AUD-05)
|
||||
- GET /api/audio/waveform endpoint returning downsampled peak array
|
||||
- Transport bar UI with play/pause/stop/seek/load controls
|
||||
- Waveform canvas in timeline panel with position cursor
|
||||
- Real-time position cursor via WebSocket tick dispatch in handleMessage
|
||||
|
||||
affects: [03-timeline-editor, 04-show-builder]
|
||||
|
||||
tech-stack:
|
||||
added: [soundfile, numpy (waveform extraction), ffmpeg (mp3 via subprocess pipe)]
|
||||
patterns:
|
||||
- "MP3 via ffmpeg pipe: subprocess.run(['ffmpeg', '-f', 'f32le', 'pipe:1']) + np.frombuffer"
|
||||
- "asyncio.to_thread for waveform extraction (seconds-long operation)"
|
||||
- "Transport button visibility toggle: btnPlay/btnPause display swap on tick paused state"
|
||||
- "Seek bar _dragging flag prevents position jumps while user scrubs"
|
||||
- "Waveform click-to-seek: clientX/rect.left normalized to audioDuration"
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- lightsync/audio/waveform.py
|
||||
modified:
|
||||
- lightsync/api/audio.py
|
||||
- lightsync/frontend/index.html
|
||||
- lightsync/frontend/app.js
|
||||
- lightsync/frontend/style.css
|
||||
|
||||
key-decisions:
|
||||
- "ffmpeg subprocess pipe for MP3 (libsndfile/soundfile does not support MP3 natively)"
|
||||
- "peaks param clamped 100-5000 server-side; client requests 2000 for waveform display"
|
||||
- "CSS variables from existing palette used unchanged — --accent, --bg-primary, --bg-panel, --border-dim"
|
||||
- "Playback cursor uses CSS transition: left 0.1s linear for smooth 10Hz tick interpolation"
|
||||
|
||||
patterns-established:
|
||||
- "Pattern: waveform extraction returns list[float] 0.0-1.0, normalized to peak amplitude per chunk"
|
||||
- "Pattern: transport state driven by WebSocket tick — no polling from frontend"
|
||||
|
||||
requirements-completed: [AUD-05]
|
||||
|
||||
duration: 8min
|
||||
completed: 2026-04-06
|
||||
---
|
||||
|
||||
# Phase 02 Plan 03: Waveform Display and Transport UI Summary
|
||||
|
||||
**soundfile/ffmpeg waveform extractor, GET /api/audio/waveform endpoint, and full transport bar with play/pause/seek controls and real-time position cursor on canvas**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** ~8 min
|
||||
- **Started:** 2026-04-06T13:10:00Z
|
||||
- **Completed:** 2026-04-06T13:18:00Z
|
||||
- **Tasks:** 2 auto + 1 checkpoint (human-verify)
|
||||
- **Files modified:** 5 (1 created)
|
||||
|
||||
## Accomplishments
|
||||
|
||||
- Waveform extraction module handles MP3 via ffmpeg pipe and WAV/FLAC/OGG via soundfile+numpy (AUD-05)
|
||||
- GET /api/audio/waveform endpoint wraps extraction in asyncio.to_thread, returns {peaks, count, file}
|
||||
- Transport bar replaces placeholder: play/pause/stop buttons, seek slider, time display (M:SS.s), audio path input + LOAD button
|
||||
- Timeline panel waveform canvas draws peak bars using --accent color at device pixel ratio
|
||||
- Position cursor moves at 10Hz from WebSocket tick dispatch, play/pause button visibility toggles with state
|
||||
- Click-to-seek on waveform canvas, seek bar dragging, window resize redraws waveform
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: Waveform extraction module and HTTP endpoint** - `da474d9` (feat)
|
||||
2. **Task 2: Transport UI controls and waveform canvas** - `148f16b` (feat)
|
||||
3. **Task 3: Human verify** - awaiting checkpoint approval
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `lightsync/audio/waveform.py` — extract_peaks(), _extract_peaks_mp3(), _extract_peaks_soundfile(), _downsample_peaks()
|
||||
- `lightsync/api/audio.py` — Added GET /waveform endpoint with asyncio.to_thread + peaks clamp
|
||||
- `lightsync/frontend/index.html` — Timeline panel with waveform canvas + cursor; transport bar with all controls
|
||||
- `lightsync/frontend/app.js` — drawWaveform(), loadWaveform(), formatTime(), updatePosition(), transport event handlers, handleMessage tick dispatch
|
||||
- `lightsync/frontend/style.css` — .transport-btn, .transport-time, .seek-slider, .transport-file, .waveform-container, .playback-cursor
|
||||
|
||||
## Decisions Made
|
||||
|
||||
- Used ffmpeg subprocess pipe for MP3 extraction (soundfile/libsndfile doesn't support MP3 natively)
|
||||
- peaks query param server-side clamped to 100-5000; frontend requests 2000 for display
|
||||
- CSS used existing variable names unchanged (--accent, --bg-primary, --bg-panel, --border-dim) — no new color variables added
|
||||
- CSS transition on playback-cursor (left 0.1s linear) interpolates between 10Hz ticks for smooth movement
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
None — plan executed exactly as written, with CSS variable name adaptation (plan used --bg/--surface/--border, existing CSS uses --bg-primary/--bg-panel/--border-dim — adapted to match actual variable names).
|
||||
|
||||
---
|
||||
|
||||
**Total deviations:** 0
|
||||
**Impact on plan:** CSS variable name adaptation was minor and necessary for correctness. No scope creep.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
None during auto-tasks. Human verification checkpoint (Task 3) is pending.
|
||||
|
||||
## Known Stubs
|
||||
|
||||
None — waveform extraction and transport UI are fully wired. The position cursor and transport controls depend on WebSocket tick messages which are broadcast by Phase 02-02's broadcast loop.
|
||||
|
||||
## Next Phase Readiness
|
||||
|
||||
- GET /api/audio/waveform usable by any future phase needing waveform data
|
||||
- Transport controls drive audio via WebSocket commands (play/pause/seek/load)
|
||||
- Position cursor provides real-time visual feedback for timeline editor (Phase 3+)
|
||||
- All Phase 1 UI (devices panel, device form) preserved
|
||||
|
||||
---
|
||||
*Phase: 02-audio-engine*
|
||||
*Completed: 2026-04-06*
|
||||
Reference in New Issue
Block a user