docs(02): create phase 02 audio engine plans (3 plans, 3 waves)
This commit is contained in:
Claude
381
.planning/phases/02-audio-engine/02-01-PLAN.md
Normal file
381
.planning/phases/02-audio-engine/02-01-PLAN.md
Normal file
@@ -0,0 +1,381 @@
|
||||
---
|
||||
phase: 02-audio-engine
|
||||
plan: "01"
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- Dockerfile
|
||||
- pyproject.toml
|
||||
- lightsync/audio/__init__.py
|
||||
- lightsync/audio/engine.py
|
||||
- lightsync/main.py
|
||||
autonomous: true
|
||||
requirements: [AUD-01, AUD-03, INF-03]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "MPVEngine class can load an audio file and report position"
|
||||
- "play/pause/seek commands work on the MPVEngine"
|
||||
- "Windows timer fix is platform-guarded (no-op on Linux)"
|
||||
- "Docker image builds with mpv-libs, ffmpeg, libsndfile"
|
||||
artifacts:
|
||||
- path: "lightsync/audio/engine.py"
|
||||
provides: "MPVEngine class with load/play/pause/seek/get_state"
|
||||
exports: ["MPVEngine"]
|
||||
- path: "lightsync/audio/__init__.py"
|
||||
provides: "audio package init"
|
||||
- path: "Dockerfile"
|
||||
provides: "Alpine image with mpv-libs ffmpeg ffmpeg-libs libsndfile"
|
||||
contains: "mpv-libs"
|
||||
- path: "pyproject.toml"
|
||||
provides: "python-mpv, soundfile, numpy dependencies"
|
||||
contains: "python-mpv"
|
||||
key_links:
|
||||
- from: "lightsync/audio/engine.py"
|
||||
to: "mpv (libmpv.so.2)"
|
||||
via: "import mpv; mpv.MPV()"
|
||||
pattern: "import mpv"
|
||||
- from: "lightsync/main.py"
|
||||
to: "lightsync/audio/engine.py"
|
||||
via: "import and lifespan integration"
|
||||
pattern: "from lightsync.audio.engine import MPVEngine"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Create the MPVEngine audio backend — the headless mpv wrapper that loads files, controls playback, and polls position at 10Hz. Update Docker and Python dependencies. Implement INF-03 platform guard.
|
||||
|
||||
Purpose: MPVEngine is the master clock for the entire show system. Every subsequent phase depends on accurate audio position.
|
||||
Output: Working MPVEngine class, updated Dockerfile with audio libs, updated pyproject.toml.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@.planning/phases/02-audio-engine/02-RESEARCH.md
|
||||
|
||||
<interfaces>
|
||||
<!-- Existing codebase contracts the executor needs -->
|
||||
|
||||
From lightsync/main.py:
|
||||
```python
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.responses import FileResponse
|
||||
|
||||
from lightsync.devices.registry import DeviceRegistry
|
||||
from lightsync.store.show_store import ShowStore
|
||||
|
||||
registry: DeviceRegistry | None = None
|
||||
show_store: ShowStore | None = None
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
global registry, show_store
|
||||
registry = DeviceRegistry(Path("devices.json"))
|
||||
await registry.load()
|
||||
show_store = ShowStore(Path("shows"))
|
||||
show_store.ensure_dir()
|
||||
yield
|
||||
await registry.save()
|
||||
|
||||
def create_app() -> FastAPI:
|
||||
app = FastAPI(title="LightSync", lifespan=lifespan, redirect_slashes=False)
|
||||
from lightsync.api import shows, devices, ws
|
||||
app.include_router(shows.router, prefix="/api/shows")
|
||||
app.include_router(devices.router, prefix="/api/devices")
|
||||
app.include_router(ws.router)
|
||||
# ... static file serving ...
|
||||
return app
|
||||
```
|
||||
|
||||
From lightsync/api/ws.py:
|
||||
```python
|
||||
class ConnectionManager:
|
||||
active_connections: list[WebSocket]
|
||||
async def connect(self, ws: WebSocket) -> None
|
||||
def disconnect(self, ws: WebSocket) -> None
|
||||
async def broadcast(self, message: dict) -> None
|
||||
|
||||
manager = ConnectionManager()
|
||||
```
|
||||
|
||||
From Dockerfile:
|
||||
```dockerfile
|
||||
FROM python:3.11-alpine
|
||||
WORKDIR /app
|
||||
COPY pyproject.toml .
|
||||
RUN pip install --no-cache-dir .
|
||||
COPY lightsync/ lightsync/
|
||||
EXPOSE 8000
|
||||
CMD ["python", "-m", "lightsync"]
|
||||
```
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Add audio dependencies to Dockerfile and pyproject.toml</name>
|
||||
<files>Dockerfile, pyproject.toml</files>
|
||||
<read_first>Dockerfile, pyproject.toml</read_first>
|
||||
<action>
|
||||
1. Update `Dockerfile` — add Alpine apk packages for the audio stack. Insert BEFORE `RUN pip install`:
|
||||
|
||||
```dockerfile
|
||||
FROM python:3.11-alpine
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Audio stack: libmpv for python-mpv, ffmpeg for MP3 waveform, libsndfile for soundfile
|
||||
RUN apk add --no-cache \
|
||||
mpv-libs \
|
||||
ffmpeg \
|
||||
ffmpeg-libs \
|
||||
libsndfile
|
||||
|
||||
COPY pyproject.toml .
|
||||
RUN pip install --no-cache-dir .
|
||||
|
||||
COPY lightsync/ lightsync/
|
||||
|
||||
EXPOSE 8000
|
||||
|
||||
CMD ["python", "-m", "lightsync"]
|
||||
```
|
||||
|
||||
Key points:
|
||||
- `mpv-libs` (NOT full `mpv` package) — provides libmpv.so.2, ~7MB with 15 deps instead of 163
|
||||
- `ffmpeg` binary needed for MP3 waveform extraction pipe (not just ffmpeg-libs)
|
||||
- `ffmpeg-libs` needed for libmpv codec support
|
||||
- `libsndfile` for soundfile Python package (WAV/FLAC/OGG reading)
|
||||
|
||||
2. Update `pyproject.toml` — add audio dependencies to `[project] dependencies`:
|
||||
|
||||
```toml
|
||||
dependencies = [
|
||||
"fastapi>=0.115.0",
|
||||
"uvicorn[standard]>=0.34.0",
|
||||
"pydantic>=2.0.0",
|
||||
"aiofiles>=24.0.0",
|
||||
"python-dotenv>=1.0.0",
|
||||
"structlog>=24.0.0",
|
||||
"python-mpv>=1.0.5",
|
||||
"soundfile>=0.12.0",
|
||||
"numpy>=1.26.0",
|
||||
]
|
||||
```
|
||||
|
||||
Per AUD-01: python-mpv wraps libmpv for audio loading. Per AUD-05: soundfile+numpy for waveform extraction.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/claude/led2 && grep -q "mpv-libs" Dockerfile && grep -q "python-mpv" pyproject.toml && grep -q "soundfile" pyproject.toml && grep -q "numpy" pyproject.toml && echo "PASS"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- Dockerfile contains `apk add --no-cache` line with `mpv-libs`, `ffmpeg`, `ffmpeg-libs`, `libsndfile`
|
||||
- pyproject.toml dependencies list includes `python-mpv>=1.0.5`, `soundfile>=0.12.0`, `numpy>=1.26.0`
|
||||
- Dockerfile still uses `python:3.11-alpine` base image
|
||||
- Dockerfile still exposes port 8000 and runs `python -m lightsync`
|
||||
</acceptance_criteria>
|
||||
<done>Dockerfile has Alpine audio packages; pyproject.toml has python-mpv, soundfile, numpy</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Create MPVEngine class with position polling and platform timer fix</name>
|
||||
<files>lightsync/audio/__init__.py, lightsync/audio/engine.py, lightsync/main.py</files>
|
||||
<read_first>lightsync/main.py, lightsync/api/ws.py, .planning/phases/02-audio-engine/02-RESEARCH.md</read_first>
|
||||
<action>
|
||||
1. Create `lightsync/audio/__init__.py` — empty file (package marker).
|
||||
|
||||
2. Create `lightsync/audio/engine.py` with the `MPVEngine` class:
|
||||
|
||||
```python
|
||||
"""MPV-based audio engine — headless playback with 10Hz position polling."""
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
import mpv
|
||||
|
||||
|
||||
def apply_timer_fix() -> None:
|
||||
"""Apply 1ms timer resolution on Windows. No-op on Linux/macOS (INF-03)."""
|
||||
if sys.platform == "win32":
|
||||
import ctypes
|
||||
winmm = ctypes.WinDLL("winmm")
|
||||
winmm.timeBeginPeriod(1)
|
||||
|
||||
|
||||
class MPVEngine:
|
||||
"""Wraps libmpv for headless audio playback with position polling.
|
||||
|
||||
Usage:
|
||||
engine = MPVEngine()
|
||||
engine.start() # starts 10Hz polling thread
|
||||
engine.load("/path/to/song.mp3")
|
||||
engine.play()
|
||||
pos = engine.get_state() # {"position": 42.1, "paused": False, ...}
|
||||
engine.stop()
|
||||
"""
|
||||
|
||||
def __init__(self, ao: str = "null"):
|
||||
self._player: mpv.MPV = mpv.MPV(
|
||||
vo="null",
|
||||
ao=ao,
|
||||
input_default_bindings=False,
|
||||
input_vo_keyboard=False,
|
||||
)
|
||||
self._position: float = 0.0
|
||||
self._duration: float | None = None
|
||||
self._loaded_path: str | None = None
|
||||
self._lock = threading.Lock()
|
||||
self._running = False
|
||||
self._thread: threading.Thread | None = None
|
||||
|
||||
def start(self) -> None:
|
||||
"""Start the 10Hz position polling thread."""
|
||||
apply_timer_fix()
|
||||
self._running = True
|
||||
self._thread = threading.Thread(target=self._poll_loop, daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def _poll_loop(self) -> None:
|
||||
"""Poll time-pos at 10Hz. Do NOT use observe_property — fires per audio frame."""
|
||||
while self._running:
|
||||
pos = self._player.time_pos
|
||||
dur = self._player.duration
|
||||
with self._lock:
|
||||
self._position = pos if pos is not None else 0.0
|
||||
if dur is not None:
|
||||
self._duration = dur
|
||||
time.sleep(0.1)
|
||||
|
||||
def get_state(self) -> dict[str, Any]:
|
||||
"""Thread-safe snapshot of current playback state."""
|
||||
with self._lock:
|
||||
return {
|
||||
"position": self._position,
|
||||
"paused": bool(self._player.pause),
|
||||
"duration": self._duration,
|
||||
"loaded": self._loaded_path is not None,
|
||||
"file": self._loaded_path,
|
||||
}
|
||||
|
||||
def load(self, path: str) -> None:
|
||||
"""Load an audio file. Starts paused. Supports MP3, WAV, FLAC, OGG (AUD-01)."""
|
||||
self._player.loadfile(path, mode="replace")
|
||||
self._player.wait_for_playback() # don't use — blocks; instead:
|
||||
# Actually: loadfile is async in mpv. Set pause after a short wait.
|
||||
self._loaded_path = path
|
||||
self._player.pause = True
|
||||
|
||||
def play(self) -> None:
|
||||
"""Resume playback (AUD-03)."""
|
||||
self._player.pause = False
|
||||
|
||||
def pause(self) -> None:
|
||||
"""Pause playback (AUD-03)."""
|
||||
self._player.pause = True
|
||||
|
||||
def seek(self, seconds: float) -> None:
|
||||
"""Seek to absolute position in seconds (AUD-03)."""
|
||||
self._player.seek(seconds, reference="absolute")
|
||||
|
||||
def stop(self) -> None:
|
||||
"""Stop polling and terminate mpv."""
|
||||
self._running = False
|
||||
if self._thread:
|
||||
self._thread.join(timeout=2.0)
|
||||
try:
|
||||
self._player.terminate()
|
||||
except Exception:
|
||||
pass
|
||||
```
|
||||
|
||||
IMPORTANT corrections to research code:
|
||||
- Do NOT call `self._player.wait_for_playback()` in load() — it blocks until playback finishes. Instead, loadfile is already async in mpv; just set pause=True after calling it.
|
||||
- The `load()` method should be:
|
||||
```python
|
||||
def load(self, path: str) -> None:
|
||||
self._loaded_path = path
|
||||
self._player.loadfile(path, mode="replace")
|
||||
# loadfile is async — mpv loads in background
|
||||
# Pause will be applied once file is loaded
|
||||
```
|
||||
- Add `ao` parameter to constructor defaulting to `"null"` (server mode). Can be overridden via env var in main.py.
|
||||
|
||||
3. Update `lightsync/main.py` — add MPVEngine to lifespan:
|
||||
|
||||
Add `import os` at top. Add import: `from lightsync.audio.engine import MPVEngine`.
|
||||
Add module-level: `engine: MPVEngine | None = None`.
|
||||
|
||||
Update lifespan to create+start engine on startup, stop on shutdown:
|
||||
```python
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
global registry, show_store, engine
|
||||
# Startup
|
||||
registry = DeviceRegistry(Path("devices.json"))
|
||||
await registry.load()
|
||||
show_store = ShowStore(Path("shows"))
|
||||
show_store.ensure_dir()
|
||||
ao = os.environ.get("MPV_AO", "null")
|
||||
engine = MPVEngine(ao=ao)
|
||||
engine.start()
|
||||
app.state.engine = engine
|
||||
yield
|
||||
# Shutdown
|
||||
engine.stop()
|
||||
await registry.save()
|
||||
```
|
||||
|
||||
Preserve ALL existing code in main.py (create_app, static file serving). Only modify lifespan and add imports/module var.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/claude/led2 && python -c "from lightsync.audio.engine import MPVEngine, apply_timer_fix; print('import OK')" 2>&1 || echo "Import check requires mpv lib - verify file exists and has correct class"; test -f lightsync/audio/__init__.py && test -f lightsync/audio/engine.py && grep -q "class MPVEngine" lightsync/audio/engine.py && grep -q "apply_timer_fix" lightsync/audio/engine.py && grep -q "sys.platform" lightsync/audio/engine.py && grep -q "MPVEngine" lightsync/main.py && echo "PASS"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- File `lightsync/audio/__init__.py` exists
|
||||
- File `lightsync/audio/engine.py` exists with `class MPVEngine` containing methods: `start`, `_poll_loop`, `get_state`, `load`, `play`, `pause`, `seek`, `stop`
|
||||
- `lightsync/audio/engine.py` contains `def apply_timer_fix` with `sys.platform == "win32"` guard (INF-03)
|
||||
- `lightsync/main.py` imports `MPVEngine` from `lightsync.audio.engine`
|
||||
- `lightsync/main.py` lifespan creates MPVEngine with `ao` from `MPV_AO` env var (default "null")
|
||||
- `lightsync/main.py` lifespan calls `engine.start()` on startup and `engine.stop()` on shutdown
|
||||
- `lightsync/main.py` sets `app.state.engine = engine`
|
||||
- Existing main.py functionality (registry, show_store, create_app, static files) is preserved
|
||||
- MPVEngine constructor accepts `ao` parameter (default "null"), passes `vo="null"` to mpv.MPV
|
||||
- `_poll_loop` uses `time.sleep(0.1)` for 10Hz polling — does NOT use `observe_property`
|
||||
- `get_state()` returns dict with keys: position, paused, duration, loaded, file
|
||||
</acceptance_criteria>
|
||||
<done>MPVEngine class created with load/play/pause/seek/position polling; platform timer fix; lifespan integration in main.py</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- `lightsync/audio/engine.py` has MPVEngine with all methods
|
||||
- `apply_timer_fix()` has `sys.platform == 'win32'` guard
|
||||
- Dockerfile has `mpv-libs`, `ffmpeg`, `ffmpeg-libs`, `libsndfile`
|
||||
- pyproject.toml has `python-mpv`, `soundfile`, `numpy`
|
||||
- main.py lifespan creates and starts MPVEngine
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- MPVEngine class exists with load/play/pause/seek/get_state/stop methods
|
||||
- 10Hz polling loop in daemon thread (time.sleep(0.1), NOT observe_property)
|
||||
- INF-03 implemented as sys.platform guard
|
||||
- Docker image can build with audio dependencies
|
||||
- main.py integrates MPVEngine in lifespan
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/02-audio-engine/02-01-SUMMARY.md`
|
||||
</output>
|
||||
326
.planning/phases/02-audio-engine/02-02-PLAN.md
Normal file
326
.planning/phases/02-audio-engine/02-02-PLAN.md
Normal file
@@ -0,0 +1,326 @@
|
||||
---
|
||||
phase: 02-audio-engine
|
||||
plan: "02"
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on: ["02-01"]
|
||||
files_modified:
|
||||
- lightsync/api/ws.py
|
||||
- lightsync/api/audio.py
|
||||
- lightsync/main.py
|
||||
autonomous: true
|
||||
requirements: [AUD-02, AUD-04]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "WebSocket broadcasts position ticks at 10Hz to all connected clients"
|
||||
- "Browser receives {type: 'tick', position: N, paused: bool, duration: N} messages"
|
||||
- "Transport commands (play/pause/seek/load) sent from browser are dispatched to MPVEngine"
|
||||
- "Audio file can be loaded via REST POST /api/audio/load"
|
||||
artifacts:
|
||||
- path: "lightsync/api/ws.py"
|
||||
provides: "WebSocket command dispatch + broadcast_loop integration"
|
||||
contains: "broadcast_loop"
|
||||
- path: "lightsync/api/audio.py"
|
||||
provides: "REST endpoints for audio load"
|
||||
exports: ["router"]
|
||||
- path: "lightsync/main.py"
|
||||
provides: "broadcast_loop task in lifespan, audio router registered"
|
||||
contains: "broadcast_loop"
|
||||
key_links:
|
||||
- from: "lightsync/api/ws.py"
|
||||
to: "lightsync/audio/engine.py"
|
||||
via: "engine.play/pause/seek/load dispatch"
|
||||
pattern: "engine\\.(play|pause|seek|load)"
|
||||
- from: "lightsync/main.py"
|
||||
to: "lightsync/api/ws.py"
|
||||
via: "broadcast_loop asyncio task in lifespan"
|
||||
pattern: "asyncio.create_task.*broadcast_loop"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Wire the WebSocket hub to broadcast MPVEngine position at 10Hz and dispatch transport commands from the browser. Add REST endpoint for audio file loading.
|
||||
|
||||
Purpose: The browser needs real-time position data to drive the timeline cursor and transport UI. Commands must flow back to control playback.
|
||||
Output: Working bidirectional WebSocket (position ticks out, commands in), REST audio load endpoint.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@.planning/phases/02-audio-engine/02-RESEARCH.md
|
||||
@.planning/phases/02-audio-engine/02-01-SUMMARY.md
|
||||
|
||||
<interfaces>
|
||||
<!-- From 02-01: MPVEngine created in main.py lifespan -->
|
||||
|
||||
From lightsync/audio/engine.py (created in 02-01):
|
||||
```python
|
||||
class MPVEngine:
|
||||
def __init__(self, ao: str = "null"): ...
|
||||
def start(self) -> None: ...
|
||||
def get_state(self) -> dict[str, Any]:
|
||||
# Returns: {"position": float, "paused": bool, "duration": float|None, "loaded": bool, "file": str|None}
|
||||
def load(self, path: str) -> None: ...
|
||||
def play(self) -> None: ...
|
||||
def pause(self) -> None: ...
|
||||
def seek(self, seconds: float) -> None: ...
|
||||
def stop(self) -> None: ...
|
||||
```
|
||||
|
||||
From lightsync/main.py (after 02-01):
|
||||
```python
|
||||
engine: MPVEngine | None = None # module-level, set in lifespan
|
||||
# app.state.engine = engine # also available on app state
|
||||
```
|
||||
|
||||
From lightsync/api/ws.py (existing):
|
||||
```python
|
||||
class ConnectionManager:
|
||||
active_connections: list[WebSocket]
|
||||
async def connect(self, ws: WebSocket) -> None
|
||||
def disconnect(self, ws: WebSocket) -> None
|
||||
async def broadcast(self, message: dict) -> None
|
||||
|
||||
manager = ConnectionManager()
|
||||
|
||||
@router.websocket("/ws")
|
||||
async def websocket_endpoint(websocket: WebSocket):
|
||||
# Currently echoes back with type=ack
|
||||
```
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Wire WebSocket command dispatch and 10Hz broadcast loop</name>
|
||||
<files>lightsync/api/ws.py, lightsync/main.py</files>
|
||||
<read_first>lightsync/api/ws.py, lightsync/main.py, lightsync/audio/engine.py</read_first>
|
||||
<action>
|
||||
1. Update `lightsync/api/ws.py` — replace the echo stub with command dispatch:
|
||||
|
||||
Keep the existing `ConnectionManager` class and `manager` instance UNCHANGED.
|
||||
|
||||
Add a `broadcast_loop` async function at module level:
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
|
||||
async def broadcast_loop(get_engine) -> None:
|
||||
"""Broadcast playback state to all WebSocket clients at 10Hz (AUD-04).
|
||||
|
||||
Args:
|
||||
get_engine: callable returning the MPVEngine instance (avoids import cycle)
|
||||
"""
|
||||
while True:
|
||||
await asyncio.sleep(0.1)
|
||||
engine = get_engine()
|
||||
if engine and manager.active_connections:
|
||||
state = engine.get_state()
|
||||
await manager.broadcast({"type": "tick", **state})
|
||||
```
|
||||
|
||||
Replace the `websocket_endpoint` receive loop body — dispatch on `msg["type"]`:
|
||||
|
||||
```python
|
||||
@router.websocket("/ws")
|
||||
async def websocket_endpoint(websocket: WebSocket):
|
||||
await manager.connect(websocket)
|
||||
try:
|
||||
while True:
|
||||
data = await websocket.receive_text()
|
||||
msg = json.loads(data)
|
||||
msg_type = msg.get("type")
|
||||
|
||||
# Import engine reference from main (module-level var)
|
||||
from lightsync.main import engine
|
||||
|
||||
if engine is None:
|
||||
await websocket.send_text(json.dumps({"type": "error", "message": "engine not ready"}))
|
||||
continue
|
||||
|
||||
if msg_type == "play":
|
||||
engine.play()
|
||||
await websocket.send_text(json.dumps({"type": "ack", "command": "play"}))
|
||||
elif msg_type == "pause":
|
||||
engine.pause()
|
||||
await websocket.send_text(json.dumps({"type": "ack", "command": "pause"}))
|
||||
elif msg_type == "seek":
|
||||
position = float(msg.get("position", 0))
|
||||
engine.seek(position)
|
||||
await websocket.send_text(json.dumps({"type": "ack", "command": "seek", "position": position}))
|
||||
elif msg_type == "load":
|
||||
path = msg.get("path", "")
|
||||
if path:
|
||||
engine.load(path)
|
||||
await websocket.send_text(json.dumps({"type": "ack", "command": "load", "path": path}))
|
||||
else:
|
||||
await websocket.send_text(json.dumps({"type": "error", "message": "missing path"}))
|
||||
else:
|
||||
await websocket.send_text(json.dumps({"type": "ack", "echo": msg}))
|
||||
except WebSocketDisconnect:
|
||||
manager.disconnect(websocket)
|
||||
```
|
||||
|
||||
2. Update `lightsync/main.py` — start broadcast_loop as asyncio task in lifespan:
|
||||
|
||||
Add `import asyncio` at top.
|
||||
Add import: `from lightsync.api.ws import manager, broadcast_loop`.
|
||||
|
||||
In lifespan, AFTER engine.start(), add:
|
||||
```python
|
||||
task = asyncio.create_task(broadcast_loop(lambda: engine))
|
||||
```
|
||||
|
||||
In lifespan shutdown (before yield cleanup), add:
|
||||
```python
|
||||
yield
|
||||
task.cancel()
|
||||
try:
|
||||
await task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
engine.stop()
|
||||
await registry.save()
|
||||
```
|
||||
|
||||
The `get_engine` callable pattern avoids circular imports (ws.py does not import engine at module level).
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/claude/led2 && grep -q "broadcast_loop" lightsync/api/ws.py && grep -q "broadcast_loop" lightsync/main.py && grep -q "asyncio.create_task" lightsync/main.py && grep -q '"type": "tick"' lightsync/api/ws.py && grep -q 'msg_type == "play"' lightsync/api/ws.py && grep -q 'msg_type == "pause"' lightsync/api/ws.py && grep -q 'msg_type == "seek"' lightsync/api/ws.py && echo "PASS"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- `lightsync/api/ws.py` contains `async def broadcast_loop(get_engine)` function
|
||||
- broadcast_loop sends `{"type": "tick", "position": ..., "paused": ..., "duration": ..., "loaded": ..., "file": ...}` at 10Hz
|
||||
- broadcast_loop uses `await asyncio.sleep(0.1)` — NOT a threading timer
|
||||
- broadcast_loop only broadcasts when `manager.active_connections` is non-empty
|
||||
- WebSocket endpoint dispatches `play`, `pause`, `seek`, `load` commands to engine
|
||||
- Each command sends back `{"type": "ack", "command": "..."}` confirmation
|
||||
- `seek` command reads `msg["position"]` as float
|
||||
- `load` command reads `msg["path"]` as string
|
||||
- Unknown message types still get echo response (backward compat)
|
||||
- `lightsync/main.py` starts broadcast_loop via `asyncio.create_task` in lifespan
|
||||
- `lightsync/main.py` cancels the task in lifespan shutdown
|
||||
- ConnectionManager class is NOT modified
|
||||
</acceptance_criteria>
|
||||
<done>WebSocket dispatches transport commands to MPVEngine; broadcast_loop sends position ticks at 10Hz</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Add REST audio load endpoint with file validation</name>
|
||||
<files>lightsync/api/audio.py, lightsync/main.py</files>
|
||||
<read_first>lightsync/main.py, lightsync/api/shows.py</read_first>
|
||||
<action>
|
||||
1. Create `lightsync/api/audio.py` with REST endpoints:
|
||||
|
||||
```python
|
||||
"""Audio loading and status endpoints."""
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from pydantic import BaseModel
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
SUPPORTED_EXTENSIONS = {".mp3", ".wav", ".flac", ".ogg"}
|
||||
|
||||
|
||||
class LoadRequest(BaseModel):
|
||||
path: str
|
||||
|
||||
|
||||
class AudioState(BaseModel):
|
||||
position: float
|
||||
paused: bool
|
||||
duration: float | None
|
||||
loaded: bool
|
||||
file: str | None
|
||||
|
||||
|
||||
@router.post("/load")
|
||||
async def load_audio(req: LoadRequest, request: Request):
|
||||
"""Load an audio file by server path (AUD-01). YouTube URL deferred to Phase 6 (AUD-02 stub)."""
|
||||
engine = request.app.state.engine
|
||||
if engine is None:
|
||||
raise HTTPException(status_code=503, detail="Audio engine not ready")
|
||||
|
||||
p = Path(req.path)
|
||||
if not p.exists():
|
||||
raise HTTPException(status_code=404, detail=f"File not found: {req.path}")
|
||||
if p.suffix.lower() not in SUPPORTED_EXTENSIONS:
|
||||
raise HTTPException(status_code=400, detail=f"Unsupported format: {p.suffix}. Supported: {', '.join(sorted(SUPPORTED_EXTENSIONS))}")
|
||||
|
||||
await asyncio.to_thread(engine.load, req.path)
|
||||
return {"status": "loaded", "path": req.path}
|
||||
|
||||
|
||||
@router.get("/state")
|
||||
async def get_audio_state(request: Request):
|
||||
"""Get current audio playback state."""
|
||||
engine = request.app.state.engine
|
||||
if engine is None:
|
||||
raise HTTPException(status_code=503, detail="Audio engine not ready")
|
||||
return engine.get_state()
|
||||
```
|
||||
|
||||
Key points:
|
||||
- `POST /api/audio/load` accepts `{"path": "/path/to/file.mp3"}`
|
||||
- Validates file exists and extension is in SUPPORTED_EXTENSIONS
|
||||
- Uses `asyncio.to_thread` for `engine.load()` to avoid blocking event loop (per research pitfall 3)
|
||||
- `GET /api/audio/state` returns current engine state (useful for initial page load)
|
||||
- AUD-02 (YouTube) is noted as Phase 6 stub — not implemented here
|
||||
|
||||
2. Update `lightsync/main.py` — register the audio router:
|
||||
|
||||
In `create_app()`, add BEFORE the static file handler:
|
||||
```python
|
||||
from lightsync.api import shows, devices, ws, audio
|
||||
# ... existing router includes ...
|
||||
app.include_router(audio.router, prefix="/api/audio")
|
||||
```
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/claude/led2 && test -f lightsync/api/audio.py && grep -q "class LoadRequest" lightsync/api/audio.py && grep -q 'prefix="/api/audio"' lightsync/main.py && grep -q "SUPPORTED_EXTENSIONS" lightsync/api/audio.py && grep -q "asyncio.to_thread" lightsync/api/audio.py && echo "PASS"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- File `lightsync/api/audio.py` exists with `router = APIRouter()`
|
||||
- `POST /load` endpoint accepts `LoadRequest(path: str)`, validates file existence and extension
|
||||
- Supported extensions: `.mp3`, `.wav`, `.flac`, `.ogg`
|
||||
- `engine.load()` called via `asyncio.to_thread` (non-blocking)
|
||||
- Returns `{"status": "loaded", "path": "..."}` on success
|
||||
- Returns 404 if file not found, 400 if unsupported format, 503 if engine not ready
|
||||
- `GET /state` endpoint returns engine.get_state() dict
|
||||
- `lightsync/main.py` includes `audio.router` at prefix `/api/audio`
|
||||
- Audio router registered BEFORE static file catch-all handler
|
||||
</acceptance_criteria>
|
||||
<done>REST endpoints for audio load (with file validation) and state query; router wired in main.py</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- WebSocket sends position ticks with `type: "tick"` at 10Hz
|
||||
- WebSocket dispatches play/pause/seek/load commands
|
||||
- POST /api/audio/load validates and loads audio files
|
||||
- GET /api/audio/state returns playback state
|
||||
- broadcast_loop started in lifespan, cancelled on shutdown
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- Position ticks broadcast at 10Hz over WebSocket (AUD-04)
|
||||
- Transport commands dispatched from WebSocket to MPVEngine
|
||||
- REST audio load endpoint with file validation (AUD-01 integration)
|
||||
- AUD-02 noted as Phase 6 deferral
|
||||
- No blocking calls in async context (asyncio.to_thread for load)
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/02-audio-engine/02-02-SUMMARY.md`
|
||||
</output>
|
||||
623
.planning/phases/02-audio-engine/02-03-PLAN.md
Normal file
623
.planning/phases/02-audio-engine/02-03-PLAN.md
Normal file
@@ -0,0 +1,623 @@
|
||||
---
|
||||
phase: 02-audio-engine
|
||||
plan: "03"
|
||||
type: execute
|
||||
wave: 3
|
||||
depends_on: ["02-02"]
|
||||
files_modified:
|
||||
- lightsync/audio/waveform.py
|
||||
- lightsync/api/audio.py
|
||||
- lightsync/frontend/index.html
|
||||
- lightsync/frontend/app.js
|
||||
- lightsync/frontend/style.css
|
||||
autonomous: false
|
||||
requirements: [AUD-05]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "Waveform is rendered in the timeline panel from loaded audio"
|
||||
- "Play/pause/seek controls work in the browser transport bar"
|
||||
- "Position cursor moves in real time during playback"
|
||||
- "Transport bar shows current time, duration, and playback state"
|
||||
artifacts:
|
||||
- path: "lightsync/audio/waveform.py"
|
||||
provides: "extract_peaks() function for waveform data"
|
||||
exports: ["extract_peaks"]
|
||||
- path: "lightsync/api/audio.py"
|
||||
provides: "GET /api/audio/waveform endpoint"
|
||||
contains: "extract_peaks"
|
||||
- path: "lightsync/frontend/app.js"
|
||||
provides: "Transport controls, waveform canvas, position tick handler"
|
||||
contains: "waveform"
|
||||
- path: "lightsync/frontend/index.html"
|
||||
provides: "Transport bar HTML, waveform canvas element"
|
||||
contains: "transport"
|
||||
key_links:
|
||||
- from: "lightsync/frontend/app.js"
|
||||
to: "/api/audio/waveform"
|
||||
via: "fetch after audio load"
|
||||
pattern: "fetch.*api/audio/waveform"
|
||||
- from: "lightsync/frontend/app.js"
|
||||
to: "WebSocket tick messages"
|
||||
via: "handleMessage dispatches type=tick to update position display"
|
||||
pattern: 'msg.type.*tick'
|
||||
- from: "lightsync/api/audio.py"
|
||||
to: "lightsync/audio/waveform.py"
|
||||
via: "import extract_peaks"
|
||||
pattern: "from lightsync.audio.waveform import extract_peaks"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Build the waveform extraction module, wire the waveform HTTP endpoint, and create the browser transport UI (play/pause/seek buttons, time display, waveform canvas with position cursor).
|
||||
|
||||
Purpose: The user needs visual feedback — waveform shows song structure, transport controls audio, position cursor tracks playback in real time.
|
||||
Output: Working transport bar with play/pause/seek, waveform canvas in timeline panel, real-time position updates from WebSocket ticks.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@.planning/phases/02-audio-engine/02-RESEARCH.md
|
||||
@.planning/phases/02-audio-engine/02-01-SUMMARY.md
|
||||
@.planning/phases/02-audio-engine/02-02-SUMMARY.md
|
||||
|
||||
<interfaces>
|
||||
<!-- From 02-01: MPVEngine -->
|
||||
From lightsync/audio/engine.py:
|
||||
```python
|
||||
class MPVEngine:
|
||||
def get_state(self) -> dict[str, Any]:
|
||||
# Returns: {"position": float, "paused": bool, "duration": float|None, "loaded": bool, "file": str|None}
|
||||
```
|
||||
|
||||
<!-- From 02-02: WebSocket tick format and audio endpoints -->
|
||||
WebSocket tick message format:
|
||||
```json
|
||||
{"type": "tick", "position": 42.1, "paused": false, "duration": 180.5, "loaded": true, "file": "/path/to/song.mp3"}
|
||||
```
|
||||
|
||||
WebSocket commands accepted:
|
||||
```json
|
||||
{"type": "play"}
|
||||
{"type": "pause"}
|
||||
{"type": "seek", "position": 42.0}
|
||||
{"type": "load", "path": "/path/to/file.mp3"}
|
||||
```
|
||||
|
||||
REST endpoints:
|
||||
```
|
||||
POST /api/audio/load — body: {"path": "..."}
|
||||
GET /api/audio/state — returns engine state dict
|
||||
```
|
||||
|
||||
From lightsync/api/audio.py:
|
||||
```python
|
||||
router = APIRouter()
|
||||
# Already has POST /load and GET /state
|
||||
```
|
||||
|
||||
<!-- From Phase 1: Frontend structure -->
|
||||
From lightsync/frontend/index.html:
|
||||
- `<div id="timeline-panel">` — currently shows placeholder text
|
||||
- `<footer class="transport-bar" id="transport-panel">` — currently shows placeholder text
|
||||
- `<script src="/app.js" type="module">` — ES modules
|
||||
|
||||
From lightsync/frontend/app.js:
|
||||
- `class LightSyncClient` with `handleMessage(msg)`, `send(msg)`, `connect()`
|
||||
- `const client = new LightSyncClient()` — global instance
|
||||
- Uses `wss://` protocol detection: `${location.protocol === 'https:' ? 'wss' : 'ws'}://${location.host}/ws`
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Create waveform extraction module and HTTP endpoint</name>
|
||||
<files>lightsync/audio/waveform.py, lightsync/api/audio.py</files>
|
||||
<read_first>lightsync/api/audio.py, .planning/phases/02-audio-engine/02-RESEARCH.md</read_first>
|
||||
<action>
|
||||
1. Create `lightsync/audio/waveform.py`:
|
||||
|
||||
```python
|
||||
"""Waveform peak extraction for timeline display (AUD-05)."""
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import soundfile as sf
|
||||
|
||||
|
||||
def extract_peaks(path: str, num_peaks: int = 1000) -> list[float]:
|
||||
"""Extract downsampled peak amplitudes from audio file.
|
||||
|
||||
Returns a list of floats (0.0-1.0) representing peak amplitude per chunk.
|
||||
Handles MP3 via ffmpeg subprocess (libsndfile doesn't support MP3).
|
||||
|
||||
Args:
|
||||
path: Absolute path to audio file
|
||||
num_peaks: Number of peak samples to return (default 1000)
|
||||
|
||||
Returns:
|
||||
List of float peak values, length <= num_peaks
|
||||
"""
|
||||
p = Path(path)
|
||||
if p.suffix.lower() == ".mp3":
|
||||
return _extract_peaks_mp3(path, num_peaks)
|
||||
return _extract_peaks_soundfile(path, num_peaks)
|
||||
|
||||
|
||||
def _extract_peaks_soundfile(path: str, num_peaks: int) -> list[float]:
|
||||
"""Extract peaks using soundfile (WAV, FLAC, OGG)."""
|
||||
data, _ = sf.read(path, always_2d=True)
|
||||
mono = np.mean(data, axis=1)
|
||||
return _downsample_peaks(mono, num_peaks)
|
||||
|
||||
|
||||
def _extract_peaks_mp3(path: str, num_peaks: int) -> list[float]:
|
||||
"""Extract peaks from MP3 via ffmpeg pipe to float32 PCM."""
|
||||
cmd = [
|
||||
"ffmpeg", "-i", path,
|
||||
"-f", "f32le", "-ar", "44100", "-ac", "1",
|
||||
"pipe:1", "-loglevel", "quiet",
|
||||
]
|
||||
result = subprocess.run(cmd, capture_output=True, timeout=60)
|
||||
if result.returncode != 0:
|
||||
raise RuntimeError(f"ffmpeg failed for {path}: exit code {result.returncode}")
|
||||
mono = np.frombuffer(result.stdout, dtype=np.float32)
|
||||
return _downsample_peaks(mono, num_peaks)
|
||||
|
||||
|
||||
def _downsample_peaks(mono: np.ndarray, num_peaks: int) -> list[float]:
|
||||
"""Downsample mono audio to peak amplitude array."""
|
||||
if len(mono) == 0:
|
||||
return []
|
||||
chunk_size = max(1, len(mono) // num_peaks)
|
||||
peaks = []
|
||||
for i in range(0, len(mono), chunk_size):
|
||||
chunk = mono[i : i + chunk_size]
|
||||
peaks.append(float(np.max(np.abs(chunk))))
|
||||
return peaks[:num_peaks]
|
||||
```
|
||||
|
||||
2. Update `lightsync/api/audio.py` — add waveform endpoint:
|
||||
|
||||
Add import at top: `from lightsync.audio.waveform import extract_peaks`
|
||||
|
||||
Add endpoint AFTER the existing ones:
|
||||
|
||||
```python
|
||||
@router.get("/waveform")
|
||||
async def get_waveform(request: Request, peaks: int = 1000):
|
||||
"""Get waveform peak data for the currently loaded audio file (AUD-05).
|
||||
|
||||
Query params:
|
||||
peaks: Number of peak samples (default 1000, max 5000)
|
||||
"""
|
||||
engine = request.app.state.engine
|
||||
if engine is None:
|
||||
raise HTTPException(status_code=503, detail="Audio engine not ready")
|
||||
|
||||
state = engine.get_state()
|
||||
if not state["loaded"] or not state["file"]:
|
||||
raise HTTPException(status_code=400, detail="No audio file loaded")
|
||||
|
||||
num_peaks = min(max(peaks, 100), 5000)
|
||||
|
||||
try:
|
||||
data = await asyncio.to_thread(extract_peaks, state["file"], num_peaks)
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=500, detail=f"Waveform extraction failed: {e}")
|
||||
|
||||
return {"peaks": data, "count": len(data), "file": state["file"]}
|
||||
```
|
||||
|
||||
Uses `asyncio.to_thread` because waveform extraction can take seconds for large files.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/claude/led2 && test -f lightsync/audio/waveform.py && grep -q "def extract_peaks" lightsync/audio/waveform.py && grep -q "_extract_peaks_mp3" lightsync/audio/waveform.py && grep -q "_extract_peaks_soundfile" lightsync/audio/waveform.py && grep -q "waveform" lightsync/api/audio.py && grep -q "extract_peaks" lightsync/api/audio.py && echo "PASS"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- File `lightsync/audio/waveform.py` exists with `extract_peaks(path, num_peaks=1000) -> list[float]`
|
||||
- MP3 files handled via ffmpeg subprocess pipe (`_extract_peaks_mp3`)
|
||||
- WAV/FLAC/OGG handled via soundfile (`_extract_peaks_soundfile`)
|
||||
- `_downsample_peaks` shared helper produces list of floats 0.0-1.0
|
||||
- `lightsync/api/audio.py` has `GET /waveform` endpoint
|
||||
- Waveform endpoint returns `{"peaks": [...], "count": N, "file": "..."}`
|
||||
- Waveform extraction uses `asyncio.to_thread` (non-blocking)
|
||||
- Returns 400 if no file loaded, 503 if engine not ready
|
||||
- `peaks` query param clamped between 100-5000
|
||||
</acceptance_criteria>
|
||||
<done>Waveform extraction works for MP3/WAV/FLAC/OGG; HTTP endpoint returns peak data</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Build transport UI controls and waveform canvas in frontend</name>
|
||||
<files>lightsync/frontend/index.html, lightsync/frontend/app.js, lightsync/frontend/style.css</files>
|
||||
<read_first>lightsync/frontend/index.html, lightsync/frontend/app.js, lightsync/frontend/style.css</read_first>
|
||||
<action>
|
||||
1. Update `lightsync/frontend/index.html` — replace placeholders in transport bar and timeline panel:
|
||||
|
||||
Replace the transport-bar footer content:
|
||||
```html
|
||||
<footer class="transport-bar" id="transport-panel">
|
||||
<div class="transport-controls">
|
||||
<button id="btn-play" class="transport-btn" title="Play">▶</button>
|
||||
<button id="btn-pause" class="transport-btn" title="Pause" style="display:none">▮▮</button>
|
||||
<button id="btn-stop" class="transport-btn" title="Stop">■</button>
|
||||
</div>
|
||||
<div class="transport-time">
|
||||
<span id="time-current">0:00.0</span>
|
||||
<span class="text-dim">/</span>
|
||||
<span id="time-duration">0:00.0</span>
|
||||
</div>
|
||||
<div class="transport-seek">
|
||||
<input type="range" id="seek-bar" min="0" max="100" value="0" step="0.1" class="seek-slider">
|
||||
</div>
|
||||
<div class="transport-file">
|
||||
<input type="text" id="audio-path" placeholder="Audio file path on server..." class="audio-path-input">
|
||||
<button id="btn-load" class="transport-btn" title="Load audio">LOAD</button>
|
||||
</div>
|
||||
</footer>
|
||||
```
|
||||
|
||||
Replace the timeline-panel content (keep the panel div, replace inner content):
|
||||
```html
|
||||
<main class="main-area">
|
||||
<div class="panel" id="timeline-panel" style="flex: 1; display: flex; flex-direction: column;">
|
||||
<div class="panel-header">TIMELINE</div>
|
||||
<div class="waveform-container" style="flex: 1; position: relative; min-height: 100px;">
|
||||
<canvas id="waveform-canvas" style="width: 100%; height: 100%;"></canvas>
|
||||
<div id="playback-cursor" class="playback-cursor"></div>
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
```
|
||||
|
||||
2. Update `lightsync/frontend/style.css` — add transport and waveform styles:
|
||||
|
||||
Append these styles (preserve ALL existing styles):
|
||||
|
||||
```css
|
||||
/* Transport bar */
|
||||
.transport-controls {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.transport-btn {
|
||||
background: var(--surface);
|
||||
color: var(--accent);
|
||||
border: 1px solid var(--border-bright);
|
||||
padding: 4px 10px;
|
||||
font-family: 'JetBrains Mono', monospace;
|
||||
font-size: 0.85rem;
|
||||
cursor: pointer;
|
||||
min-width: 36px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.transport-btn:hover {
|
||||
background: var(--accent);
|
||||
color: var(--bg);
|
||||
}
|
||||
|
||||
.transport-time {
|
||||
font-family: 'JetBrains Mono', monospace;
|
||||
font-size: 0.85rem;
|
||||
white-space: nowrap;
|
||||
min-width: 120px;
|
||||
}
|
||||
|
||||
.transport-seek {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.seek-slider {
|
||||
width: 100%;
|
||||
accent-color: var(--accent);
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.transport-file {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.audio-path-input {
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
border: 1px solid var(--border);
|
||||
padding: 4px 8px;
|
||||
font-family: 'JetBrains Mono', monospace;
|
||||
font-size: 0.8rem;
|
||||
width: 200px;
|
||||
}
|
||||
|
||||
.audio-path-input:focus {
|
||||
border-color: var(--accent);
|
||||
outline: none;
|
||||
}
|
||||
|
||||
/* Transport bar layout */
|
||||
.transport-bar {
|
||||
display: flex;
|
||||
gap: 12px;
|
||||
align-items: center;
|
||||
padding: 6px 12px;
|
||||
}
|
||||
|
||||
/* Waveform */
|
||||
.waveform-container {
|
||||
background: var(--bg);
|
||||
border: 1px solid var(--border);
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.playback-cursor {
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 2px;
|
||||
height: 100%;
|
||||
background: var(--accent);
|
||||
pointer-events: none;
|
||||
transition: left 0.1s linear;
|
||||
z-index: 10;
|
||||
}
|
||||
```
|
||||
|
||||
IMPORTANT: Read style.css first to identify the existing CSS variable names. Use `var(--accent)`, `var(--bg)`, `var(--surface)`, `var(--text)`, `var(--border)`, `var(--border-bright)`, `var(--text-dim)` — whatever variable names are already defined. Do NOT create new color variables. The terminal/hacker aesthetic (UI-02) must be maintained.
|
||||
|
||||
3. Update `lightsync/frontend/app.js` — add transport logic, waveform rendering, and position tick handling:
|
||||
|
||||
Add AFTER the existing `LightSyncClient` class definition but BEFORE `const client = new LightSyncClient()`:
|
||||
|
||||
```javascript
|
||||
// --- Waveform rendering ---
|
||||
let waveformPeaks = [];
|
||||
let audioDuration = 0;
|
||||
let audioLoaded = false;
|
||||
|
||||
function drawWaveform(canvas, peaks) {
|
||||
const ctx = canvas.getContext('2d');
|
||||
const dpr = window.devicePixelRatio || 1;
|
||||
const rect = canvas.getBoundingClientRect();
|
||||
canvas.width = rect.width * dpr;
|
||||
canvas.height = rect.height * dpr;
|
||||
ctx.scale(dpr, dpr);
|
||||
|
||||
const w = rect.width;
|
||||
const h = rect.height;
|
||||
const mid = h / 2;
|
||||
const barWidth = w / peaks.length;
|
||||
|
||||
ctx.clearRect(0, 0, w, h);
|
||||
|
||||
// Draw waveform bars
|
||||
ctx.fillStyle = getComputedStyle(document.documentElement).getPropertyValue('--accent').trim() || '#0f0';
|
||||
for (let i = 0; i < peaks.length; i++) {
|
||||
const amp = peaks[i] * mid * 0.9;
|
||||
const x = i * barWidth;
|
||||
ctx.fillRect(x, mid - amp, Math.max(barWidth - 0.5, 0.5), amp * 2);
|
||||
}
|
||||
}
|
||||
|
||||
async function loadWaveform() {
|
||||
try {
|
||||
const res = await fetch('/api/audio/waveform?peaks=2000');
|
||||
if (!res.ok) return;
|
||||
const data = await res.json();
|
||||
waveformPeaks = data.peaks;
|
||||
const canvas = document.getElementById('waveform-canvas');
|
||||
if (canvas) drawWaveform(canvas, waveformPeaks);
|
||||
} catch (err) {
|
||||
console.error('[waveform]', err);
|
||||
}
|
||||
}
|
||||
|
||||
// --- Time formatting ---
|
||||
function formatTime(seconds) {
|
||||
if (seconds == null || isNaN(seconds)) return '0:00.0';
|
||||
const m = Math.floor(seconds / 60);
|
||||
const s = seconds % 60;
|
||||
return `${m}:${s < 10 ? '0' : ''}${s.toFixed(1)}`;
|
||||
}
|
||||
|
||||
// --- Position update from WebSocket tick ---
|
||||
function updatePosition(position, duration, paused) {
|
||||
audioDuration = duration || 0;
|
||||
|
||||
document.getElementById('time-current').textContent = formatTime(position);
|
||||
document.getElementById('time-duration').textContent = formatTime(duration);
|
||||
|
||||
// Update seek bar
|
||||
const seekBar = document.getElementById('seek-bar');
|
||||
if (duration > 0 && !seekBar._dragging) {
|
||||
seekBar.value = (position / duration) * 100;
|
||||
}
|
||||
|
||||
// Update playback cursor position
|
||||
const cursor = document.getElementById('playback-cursor');
|
||||
const container = document.querySelector('.waveform-container');
|
||||
if (cursor && container && duration > 0) {
|
||||
const pct = (position / duration) * 100;
|
||||
cursor.style.left = pct + '%';
|
||||
}
|
||||
|
||||
// Toggle play/pause button visibility
|
||||
const btnPlay = document.getElementById('btn-play');
|
||||
const btnPause = document.getElementById('btn-pause');
|
||||
if (btnPlay && btnPause) {
|
||||
btnPlay.style.display = paused ? '' : 'none';
|
||||
btnPause.style.display = paused ? 'none' : '';
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Modify the `handleMessage` method of `LightSyncClient`:
|
||||
```javascript
|
||||
handleMessage(msg) {
|
||||
if (msg.type === 'tick') {
|
||||
updatePosition(msg.position, msg.duration, msg.paused);
|
||||
audioLoaded = msg.loaded;
|
||||
} else {
|
||||
console.debug('[ws]', msg);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Add transport button event listeners AFTER `client.connect()` and `loadDevices()`:
|
||||
|
||||
```javascript
|
||||
// --- Transport controls ---
|
||||
document.getElementById('btn-play')?.addEventListener('click', () => {
|
||||
client.send({ type: 'play' });
|
||||
});
|
||||
|
||||
document.getElementById('btn-pause')?.addEventListener('click', () => {
|
||||
client.send({ type: 'pause' });
|
||||
});
|
||||
|
||||
document.getElementById('btn-stop')?.addEventListener('click', () => {
|
||||
client.send({ type: 'pause' });
|
||||
client.send({ type: 'seek', position: 0 });
|
||||
});
|
||||
|
||||
// Seek bar interaction
|
||||
const seekBar = document.getElementById('seek-bar');
|
||||
if (seekBar) {
|
||||
seekBar.addEventListener('mousedown', () => { seekBar._dragging = true; });
|
||||
seekBar.addEventListener('mouseup', () => {
|
||||
seekBar._dragging = false;
|
||||
const pos = (parseFloat(seekBar.value) / 100) * audioDuration;
|
||||
client.send({ type: 'seek', position: pos });
|
||||
});
|
||||
}
|
||||
|
||||
// Load audio button
|
||||
document.getElementById('btn-load')?.addEventListener('click', async () => {
|
||||
const pathInput = document.getElementById('audio-path');
|
||||
const path = pathInput?.value.trim();
|
||||
if (!path) return;
|
||||
try {
|
||||
const res = await fetch('/api/audio/load', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ path }),
|
||||
});
|
||||
if (!res.ok) {
|
||||
const err = await res.json();
|
||||
console.error('[audio] load failed:', err.detail);
|
||||
return;
|
||||
}
|
||||
// Wait briefly for mpv to initialize, then fetch waveform
|
||||
setTimeout(loadWaveform, 500);
|
||||
} catch (err) {
|
||||
console.error('[audio] load error:', err);
|
||||
}
|
||||
});
|
||||
|
||||
// Click on waveform to seek
|
||||
document.querySelector('.waveform-container')?.addEventListener('click', (e) => {
|
||||
if (audioDuration <= 0) return;
|
||||
const rect = e.currentTarget.getBoundingClientRect();
|
||||
const pct = (e.clientX - rect.left) / rect.width;
|
||||
const pos = pct * audioDuration;
|
||||
client.send({ type: 'seek', position: pos });
|
||||
});
|
||||
|
||||
// Resize waveform on window resize
|
||||
window.addEventListener('resize', () => {
|
||||
if (waveformPeaks.length > 0) {
|
||||
const canvas = document.getElementById('waveform-canvas');
|
||||
if (canvas) drawWaveform(canvas, waveformPeaks);
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
IMPORTANT: Keep ALL existing code in app.js (LightSyncClient class, loadDevices, removeDevice, escapeHtml, device form handler). Only ADD the new transport/waveform code and modify handleMessage.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/claude/led2 && grep -q "waveform-canvas" lightsync/frontend/index.html && grep -q "btn-play" lightsync/frontend/index.html && grep -q "playback-cursor" lightsync/frontend/index.html && grep -q "drawWaveform" lightsync/frontend/app.js && grep -q "updatePosition" lightsync/frontend/app.js && grep -q "formatTime" lightsync/frontend/app.js && grep -q "loadWaveform" lightsync/frontend/app.js && grep -q 'msg.type.*tick' lightsync/frontend/app.js && grep -q "playback-cursor" lightsync/frontend/style.css && grep -q "transport-btn" lightsync/frontend/style.css && echo "PASS"</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- `index.html` transport bar contains: play button (#btn-play), pause button (#btn-pause), stop button (#btn-stop), time display (#time-current, #time-duration), seek slider (#seek-bar), audio path input (#audio-path), load button (#btn-load)
|
||||
- `index.html` timeline panel contains: waveform canvas (#waveform-canvas), playback cursor (#playback-cursor)
|
||||
- `app.js` has `drawWaveform(canvas, peaks)` function that renders bars on canvas using --accent color
|
||||
- `app.js` has `updatePosition(position, duration, paused)` that updates time display, seek bar, cursor, and play/pause button visibility
|
||||
- `app.js` handleMessage dispatches `msg.type === 'tick'` to `updatePosition`
|
||||
- `app.js` play/pause/stop buttons send WebSocket commands via `client.send()`
|
||||
- `app.js` seek bar mouseup sends `{type: "seek", position: N}` with position calculated from bar value and duration
|
||||
- `app.js` load button POSTs to `/api/audio/load` then calls `loadWaveform()` after 500ms delay
|
||||
- `app.js` click on waveform-container calculates position from click x-coordinate and sends seek command
|
||||
- `style.css` has styles for `.transport-btn`, `.transport-time`, `.seek-slider`, `.waveform-container`, `.playback-cursor`
|
||||
- CSS uses existing variable names (--accent, --bg, --surface, --text, --border, --border-bright) — no new color variables
|
||||
- All existing app.js code (device list, device form, LightSyncClient class) preserved
|
||||
- Terminal/hacker aesthetic maintained (monospace fonts, dark theme)
|
||||
</acceptance_criteria>
|
||||
<done>Transport controls work in browser; waveform renders from audio data; position cursor moves in real time</done>
|
||||
</task>
|
||||
|
||||
<task type="checkpoint:human-verify" gate="blocking">
|
||||
<name>Task 3: Verify audio playback and UI</name>
|
||||
<files>lightsync/frontend/index.html</files>
|
||||
<action>
|
||||
Human verifies the complete audio engine end-to-end. Before verification, rebuild and redeploy the Docker container:
|
||||
|
||||
```bash
|
||||
cd /home/claude/led2 && ~/bin/deploy.sh /home/claude/led2
|
||||
```
|
||||
|
||||
Verification steps:
|
||||
1. Open https://lightsync.groll.cloud in browser (Authelia login if prompted)
|
||||
2. Place a test audio file on the server (e.g., copy to /app/ inside the container)
|
||||
3. Enter the file path in the audio path input box and click LOAD
|
||||
4. Verify: waveform appears in the timeline panel
|
||||
5. Click Play — verify position cursor moves, time display counts up
|
||||
6. Click Pause — verify playback stops, cursor stops
|
||||
7. Drag seek bar — verify cursor jumps to new position
|
||||
8. Click on waveform — verify cursor seeks to clicked position
|
||||
9. Let it play for 30+ seconds — verify no drift in position updates (ticks arrive smoothly)
|
||||
10. Check browser console for errors
|
||||
|
||||
Resume signal: Type "approved" or describe issues.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/claude/led2 && grep -q "waveform-canvas" lightsync/frontend/index.html && grep -q "btn-play" lightsync/frontend/index.html && echo "PASS"</automated>
|
||||
</verify>
|
||||
<done>User confirms: audio loads, waveform renders, play/pause/seek work, position cursor tracks playback in real time</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- Waveform extraction works for WAV/FLAC/OGG (soundfile) and MP3 (ffmpeg pipe)
|
||||
- GET /api/audio/waveform returns peak data array
|
||||
- Transport bar has play/pause/stop/seek controls
|
||||
- Waveform canvas renders peaks with terminal-aesthetic styling
|
||||
- Position cursor moves at 10Hz matching WebSocket ticks
|
||||
- Click-to-seek works on waveform
|
||||
- All existing Phase 1 UI (devices panel, device form) still works
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- Waveform rendered from audio samples in timeline UI (AUD-05)
|
||||
- Transport controls connected to WebSocket commands
|
||||
- Position cursor tracks audio position in real time
|
||||
- Terminal/hacker aesthetic maintained throughout
|
||||
- Human verification confirms end-to-end audio playback works
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/02-audio-engine/02-03-SUMMARY.md`
|
||||
</output>
|
||||
Reference in New Issue
Block a user