led2/.planning/phases/02-audio-engine/02-03-SUMMARY.md

---
phase: 02-audio-engine
plan: "03"
subsystem: audio

tags: [waveform, soundfile, numpy, ffmpeg, transport-ui, canvas, websocket]

requires:
  - phase: 02-audio-engine/02-01
    provides: MPVEngine with get_state(), play/pause/seek via WebSocket
  - phase: 02-audio-engine/02-02
    provides: WebSocket broadcast loop (tick messages), REST audio endpoints in lightsync/api/audio.py

provides:
  - extract_peaks() function in lightsync/audio/waveform.py (AUD-05)
  - GET /api/audio/waveform endpoint returning downsampled peak array
  - Transport bar UI with play/pause/stop/seek/load controls
  - Waveform canvas in timeline panel with position cursor
  - Real-time position cursor via WebSocket tick dispatch in handleMessage

affects: [03-timeline-editor, 04-show-builder]

tech-stack:
  added: [soundfile, numpy (waveform extraction), ffmpeg (mp3 via subprocess pipe)]
  patterns:
    - "MP3 via ffmpeg pipe: subprocess.run(['ffmpeg', '-f', 'f32le', 'pipe:1']) + np.frombuffer"
    - "asyncio.to_thread for waveform extraction (seconds-long operation)"
    - "Transport button visibility toggle: btnPlay/btnPause display swap on tick paused state"
    - "Seek bar _dragging flag prevents position jumps while user scrubs"
    - "Waveform click-to-seek: clientX/rect.left normalized to audioDuration"

key-files:
  created:
    - lightsync/audio/waveform.py
  modified:
    - lightsync/api/audio.py
    - lightsync/frontend/index.html
    - lightsync/frontend/app.js
    - lightsync/frontend/style.css

key-decisions:
  - "ffmpeg subprocess pipe for MP3 (libsndfile/soundfile does not support MP3 natively)"
  - "peaks param clamped 100-5000 server-side; client requests 2000 for waveform display"
  - "CSS variables from existing palette used unchanged — --accent, --bg-primary, --bg-panel, --border-dim"
  - "Playback cursor uses CSS transition: left 0.1s linear for smooth 10Hz tick interpolation"

patterns-established:
  - "Pattern: waveform extraction returns list[float] 0.0-1.0, normalized to peak amplitude per chunk"
  - "Pattern: transport state driven by WebSocket tick — no polling from frontend"

requirements-completed: [AUD-05]

duration: 8min
completed: 2026-04-06
---

# Phase 02 Plan 03: Waveform Display and Transport UI Summary

**soundfile/ffmpeg waveform extractor, GET /api/audio/waveform endpoint, and full transport bar with play/pause/seek controls and real-time position cursor on canvas**

## Performance

- **Duration:** ~8 min
- **Started:** 2026-04-06T13:10:00Z
- **Completed:** 2026-04-06T13:18:00Z
- **Tasks:** 2 auto + 1 checkpoint (human-verify)
- **Files modified:** 5 (1 created)

## Accomplishments

- Waveform extraction module handles MP3 via ffmpeg pipe and WAV/FLAC/OGG via soundfile+numpy (AUD-05)
- GET /api/audio/waveform endpoint wraps extraction in asyncio.to_thread, returns {peaks, count, file}
- Transport bar replaces placeholder: play/pause/stop buttons, seek slider, time display (M:SS.s), audio path input + LOAD button
- Timeline panel waveform canvas draws peak bars using --accent color at device pixel ratio
- Position cursor moves at 10Hz from WebSocket tick dispatch, play/pause button visibility toggles with state
- Click-to-seek on waveform canvas, seek bar dragging, window resize redraws waveform

## Task Commits

Each task was committed atomically:

1. **Task 1: Waveform extraction module and HTTP endpoint** - `da474d9` (feat)
2. **Task 2: Transport UI controls and waveform canvas** - `148f16b` (feat)
3. **Task 3: Human verify** - awaiting checkpoint approval

## Files Created/Modified

- `lightsync/audio/waveform.py` — extract_peaks(), _extract_peaks_mp3(), _extract_peaks_soundfile(), _downsample_peaks()
- `lightsync/api/audio.py` — Added GET /waveform endpoint with asyncio.to_thread + peaks clamp
- `lightsync/frontend/index.html` — Timeline panel with waveform canvas + cursor; transport bar with all controls
- `lightsync/frontend/app.js` — drawWaveform(), loadWaveform(), formatTime(), updatePosition(), transport event handlers, handleMessage tick dispatch
- `lightsync/frontend/style.css` — .transport-btn, .transport-time, .seek-slider, .transport-file, .waveform-container, .playback-cursor

## Decisions Made

- Used ffmpeg subprocess pipe for MP3 extraction (soundfile/libsndfile doesn't support MP3 natively)
- peaks query param server-side clamped to 100-5000; frontend requests 2000 for display
- CSS used existing variable names unchanged (--accent, --bg-primary, --bg-panel, --border-dim) — no new color variables added
- CSS transition on playback-cursor (left 0.1s linear) interpolates between 10Hz ticks for smooth movement

## Deviations from Plan

### Auto-fixed Issues

None — plan executed exactly as written, with CSS variable name adaptation (plan used --bg/--surface/--border, existing CSS uses --bg-primary/--bg-panel/--border-dim — adapted to match actual variable names).

---

**Total deviations:** 0
**Impact on plan:** CSS variable name adaptation was minor and necessary for correctness. No scope creep.

## Issues Encountered

None during auto-tasks. Human verification checkpoint (Task 3) is pending.

## Known Stubs

None — waveform extraction and transport UI are fully wired. The position cursor and transport controls depend on WebSocket tick messages which are broadcast by Phase 02-02's broadcast loop.

## Next Phase Readiness

- GET /api/audio/waveform usable by any future phase needing waveform data
- Transport controls drive audio via WebSocket commands (play/pause/seek/load)
- Position cursor provides real-time visual feedback for timeline editor (Phase 3+)
- All Phase 1 UI (devices panel, device form) preserved

---
*Phase: 02-audio-engine*
*Completed: 2026-04-06*