led2/.planning/phases/04-timeline-editor/04-01-PLAN.md

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves
04-timeline-editor	01	execute	1		lightsync/models/show.py lightsync/frontend/index.html lightsync/frontend/style.css lightsync/frontend/timeline/timeline.js	true	TL-01 TL-05	truths artifacts key_links Per-device horizontal tracks render on a time axis with device name labels Playback cursor moves across the timeline in realtime as audio plays Waveform peaks render as a background layer behind tracks CueModel has a duration field that defaults to 4.0 path provides contains lightsync/models/show.py CueModel with duration field, AnalysisBlock with calibration_offset duration: float = 4.0 path provides exports min_lines lightsync/frontend/timeline/timeline.js TimelineCanvas class with rendering loop TimelineCanvas 150 path provides contains lightsync/frontend/index.html Timeline canvas element, toolbar, inspector placeholder timeline-canvas path provides contains lightsync/frontend/style.css Timeline layout styles, toolbar, inspector strip #timeline-canvas from to via pattern lightsync/frontend/timeline/timeline.js HTML5 audio element requestAnimationFrame reads audio.currentTime for cursor position audio.currentTime from to via pattern lightsync/frontend/timeline/timeline.js /api/audio/waveform fetch waveform peaks on audio load api/audio/waveform from to via pattern lightsync/frontend/timeline/timeline.js /api/devices fetch device list to build tracks api/devices

Build the timeline canvas foundation — data model changes, HTML layout restructure, canvas rendering with per-device tracks, time axis, waveform background, and realtime playback cursor.

Purpose: This is the visual backbone of the DAW UI. Everything else (block interactions, beat overlay, inspector) builds on top of this canvas and coordinate system. Output: A visible, scrollable timeline with device tracks and a moving playback cursor synced to the HTML5 audio element.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/04-timeline-editor/04-CONTEXT.md @.planning/phases/04-timeline-editor/04-RESEARCH.md @.planning/phases/04-timeline-editor/04-UI-SPEC.md

@lightsync/models/show.py @lightsync/frontend/index.html @lightsync/frontend/style.css @lightsync/frontend/app.js

class CueModel(BaseModel): id: UUID = Field(default_factory=uuid4) timestamp: float mode: Literal["animation", "frame_sequence"] = "animation" animation: str | None = None params: dict[str, Any] = Field(default_factory=dict)

class AnalysisBlock(BaseModel): analysed_at: datetime | None = None tempo_bpm: float | None = None beat_times: list[float] = Field(default_factory=list) onset_times: list[float] = Field(default_factory=list)

// WebSocket client: client.send({ type, ... }) // Audio: document.getElementById('player') — HTML5 audio element // API: apiPost(path, data), apiDelete(path) // Devices: loadDevices() populates #device-list

--bg-primary: #0a0a0a; --bg-panel: #0f0f0f; --bg-panel-dark: #080808; --border-dim: #1a4a4a; --accent: #00ffff; --text-primary: #cccccc; --text-dim: #555555; --text-accent: var(--accent); --font-mono: JetBrains Mono chain

Task 1: Data model changes + HTML layout restructure lightsync/models/show.py, lightsync/frontend/index.html lightsync/models/show.py, lightsync/frontend/index.html, lightsync/frontend/style.css **1. lightsync/models/show.py — Add duration field to CueModel and calibration_offset to AnalysisBlock:**

class CueModel(BaseModel):
    id: UUID = Field(default_factory=uuid4)
    timestamp: float
    duration: float = 4.0  # block length in seconds; defaults for old files
    mode: Literal["animation", "frame_sequence"] = "animation"
    animation: str | None = None
    params: dict[str, Any] = Field(default_factory=dict)

class AnalysisBlock(BaseModel):
    analysed_at: datetime | None = None
    tempo_bpm: float | None = None
    beat_times: list[float] = Field(default_factory=list)
    onset_times: list[float] = Field(default_factory=list)
    calibration_offset: float = 0.03  # seconds; compensates librosa latency bias

Do NOT change schema_version (stays at 1). Duration defaults gracefully for old files via Pydantic.

2. lightsync/frontend/index.html — Replace the animation grid in main-area with timeline layout:

Replace the entire content of <main class="main-area"> (the .section-header ANIMATIONS div and the #animation-grid div) with this structure:

<main class="main-area">
    <div id="timeline-toolbar" class="timeline-toolbar">
        <label class="toolbar-field">
            <span class="toolbar-label">ZOOM</span>
            <input type="range" id="zoom-slider" min="20" max="400" step="10" value="100">
        </label>
        <div class="toolbar-sep"></div>
        <label class="toolbar-field">
            <span class="toolbar-label">BEAT OFFSET</span>
            <input type="number" id="beat-offset" step="0.005" min="-0.2" max="0.2" value="0.030">
            <span class="toolbar-unit">s</span>
        </label>
        <div class="toolbar-sep"></div>
        <button id="btn-snap" class="transport-btn snap-btn active" title="Snap to beat">SNAP</button>
    </div>
    <canvas id="timeline-canvas"></canvas>
    <div id="block-inspector" class="block-inspector" style="display:none;"></div>
</main>

Move the ANIMATIONS section into the sidebar. Add a second panel BELOW the existing DEVICES panel (inside <aside class="sidebar">), after the closing </div> of the first .panel:

<div class="panel" style="flex: 0 0 auto;">
    <div class="panel-header">ANIMATIONS</div>
    <div class="panel-content animation-list" id="animation-palette">
    </div>
</div>

The animation palette will be populated dynamically by JS (Task 2 wires this). For now the empty container is sufficient. Remove the old hardcoded .anim-tile buttons from main-area. cd /home/claude/led2 && python -c "from lightsync.models.show import CueModel, AnalysisBlock; c = CueModel(timestamp=1.0); assert c.duration == 4.0; a = AnalysisBlock(); assert a.calibration_offset == 0.03; print('OK')" && grep -q 'timeline-canvas' lightsync/frontend/index.html && grep -q 'timeline-toolbar' lightsync/frontend/index.html && grep -q 'block-inspector' lightsync/frontend/index.html && grep -q 'animation-palette' lightsync/frontend/index.html && echo "ALL PASS" <acceptance_criteria> - lightsync/models/show.py contains duration: float = 4.0 in CueModel - lightsync/models/show.py contains calibration_offset: float = 0.03 in AnalysisBlock - lightsync/frontend/index.html contains id="timeline-canvas" (canvas element) - lightsync/frontend/index.html contains id="timeline-toolbar" with zoom-slider, beat-offset, btn-snap - lightsync/frontend/index.html contains id="block-inspector" with style="display:none;" - lightsync/frontend/index.html contains id="animation-palette" inside the sidebar - The old animation-grid div is removed from main-area - CueModel(timestamp=1.0).duration == 4.0 (Python import succeeds) </acceptance_criteria> CueModel has duration field with default 4.0, AnalysisBlock has calibration_offset with default 0.03, index.html has timeline canvas layout with toolbar/canvas/inspector structure, animation palette moved to sidebar

Task 2: TimelineCanvas class + CSS + app.js wiring lightsync/frontend/timeline/timeline.js, lightsync/frontend/style.css, lightsync/frontend/app.js lightsync/frontend/index.html, lightsync/frontend/style.css, lightsync/frontend/app.js, lightsync/api/audio.py **1. Create `lightsync/frontend/timeline/timeline.js` — the TimelineCanvas class.**

This is the core canvas rendering engine. It must export a TimelineCanvas class with:

Constants:

const PIXELS_PER_SECOND = 100;  // default, adjustable via zoom
const TRACK_HEIGHT = 48;
const HEADER_WIDTH = 140;
const TIME_AXIS_HEIGHT = 24;

Constructor: constructor(canvasEl, audioEl) — stores references to the canvas element and audio element. Initializes:

• this.canvas = canvasEl
• this.ctx = canvasEl.getContext('2d')
• this.audio = audioEl
• this.pixelsPerSecond = PIXELS_PER_SECOND
• this.scrollX = 0
• this.tracks = [] — array of {device_id, device_name, strip_type, cues: []}
• this.waveformPeaks = []
• this.audioDuration = 0
• this.beatTimes = []
• this.calibrationOffset = 0.03
• this.snapEnabled = true
• this.selectedBlock = null
• this._dragState = null
• this._onSelectBlock = null — callback for inspector
• Calls this._resizeCanvas() and adds window resize listener to call _resizeCanvas()

Coordinate helpers (exposed as methods):

• timeToX(t) — returns HEADER_WIDTH + t * this.pixelsPerSecond - this.scrollX
• xToTime(x) — returns (x - HEADER_WIDTH + this.scrollX) / this.pixelsPerSecond
• trackIndexToY(i) — returns TIME_AXIS_HEIGHT + i * TRACK_HEIGHT
• yToTrackIndex(y) — returns Math.floor((y - TIME_AXIS_HEIGHT) / TRACK_HEIGHT), clamped to -1 if above time axis

_resizeCanvas() method — handles devicePixelRatio:

_resizeCanvas() {
    const dpr = window.devicePixelRatio || 1;
    const rect = this.canvas.getBoundingClientRect();
    this.canvas.width = rect.width * dpr;
    this.canvas.height = rect.height * dpr;
    this.ctx.setTransform(dpr, 0, 0, dpr, 0, 0);
}

render() method — clears canvas, draws layers in order:

1. Background fill #0a0a0a (full canvas)
2. Track row banding: even tracks #0a0a0a, odd tracks #0c0c0c — filled rects from HEADER_WIDTH to canvas width, each TRACK_HEIGHT tall starting at TIME_AXIS_HEIGHT
3. Waveform: if this.waveformPeaks.length > 0, draw vertical bars in #1a3a3a. For each peak, compute x via timeToX((i / peaks.length) * audioDuration), draw from midY-h to midY+h where h is peak * TRACK_HEIGHT * tracks.length * 0.4. Only draw peaks where x is within visible range (0 to canvasWidth).
4. Beat marks: for each beatTime in this.beatTimes, compute displayTime = beatTime - this.calibrationOffset, draw 1px wide vertical line in rgba(0, 255, 255, 0.4) from TIME_AXIS_HEIGHT to canvas bottom
5. Block rendering: for each track, for each cue in track.cues, compute x = timeToX(cue.timestamp), w = cue.duration * this.pixelsPerSecond, y = trackIndexToY(trackIdx). Fill rect with #0a2a2a (or #0d3a3a if selected). Border 1px #1a4a4a (or #00ffff if selected). Clip and draw animation name label in #00ffff at 10px uppercase inside block.
6. Drag ghost: if this._dragState && this._dragState.ghost, draw ghost rect with rgba(0,255,255,0.15) fill and 1px #00ffff border
7. Track headers: fill #0f0f0f rect from x=0 to HEADER_WIDTH, full height. Draw track separator lines. For each track draw device_name in #cccccc at 12px, vertically centered in track row.
8. Time axis: fill #080808 rect from x=0 to canvasWidth, height TIME_AXIS_HEIGHT. Draw second/minute markers as tick marks in #1a4a4a with time labels in #555555 at 10px. Show marks at every second, with longer ticks and labels at 5s and 10s intervals.
9. Playback cursor: if !this.audio.paused, compute x = timeToX(this.audio.currentTime). Draw 1px vertical line in #00ffff from TIME_AXIS_HEIGHT to canvas bottom.

startRenderLoop() method:

startRenderLoop() {
    const loop = () => { this.render(); requestAnimationFrame(loop); };
    requestAnimationFrame(loop);
}

loadTracks(devices) method — takes array of device objects from /api/devices, builds this.tracks:

loadTracks(devices) {
    this.tracks = devices.map(d => ({
        device_id: d.id,
        device_name: d.name,
        strip_type: d.strip_type,
        cues: []
    }));
}

loadWaveform(audioPath) async method:

async loadWaveform(audioPath) {
    try {
        const res = await fetch(`/api/audio/waveform?path=${encodeURIComponent(audioPath)}&peaks=2000`);
        const data = await res.json();
        this.waveformPeaks = data.peaks || [];
    } catch (e) { console.warn('[waveform]', e); }
}

loadBeats(audioPath) async method:

async loadBeats(audioPath) {
    try {
        const res = await fetch(`/api/audio/beats?path=${encodeURIComponent(audioPath)}`);
        const data = await res.json();
        this.beatTimes = data.beat_times || [];
    } catch (e) { console.warn('[beats]', e); }
}

Scroll handling — add wheel event on canvas:

this.canvas.addEventListener('wheel', (e) => {
    e.preventDefault();
    this.scrollX = Math.max(0, this.scrollX + e.deltaX + e.deltaY);
}, { passive: false });

Auto-scroll during playback — in render(), after computing cursor x: if cursor x > canvasWidth * 0.8, set this.scrollX so cursor is at 20% from left.

Export the class as export { TimelineCanvas }.

Empty state handling: In render(), if this.tracks.length === 0 and no audio loaded, draw centered dim text "NO DEVICES -- add a device in the sidebar to create tracks" in #555555. If tracks exist but this.audioDuration === 0, draw "LOAD AUDIO -- use the transport bar to load a file".

2. Add CSS to style.css:

Add these rules AFTER the existing .anim-desc rule block (replacing the existing .main-area animation grid styles is NOT needed — those stay for sidebar now):

/* ── Timeline ──────────────────────────────────── */
#timeline-canvas {
    flex: 1;
    display: block;
    min-height: 0;
    cursor: default;
}

.timeline-toolbar {
    height: 32px;
    background: var(--bg-panel-dark);
    border-bottom: 1px solid var(--border-dim);
    display: flex;
    align-items: center;
    gap: 16px;
    padding: 0 12px;
    flex-shrink: 0;
}

.toolbar-field {
    display: flex;
    align-items: center;
    gap: 6px;
}

.toolbar-label {
    font-size: 10px;
    text-transform: uppercase;
    color: var(--text-dim);
    letter-spacing: 0.1em;
    white-space: nowrap;
}

.toolbar-unit {
    font-size: 10px;
    color: var(--text-dim);
}

.toolbar-sep {
    width: 1px;
    height: 20px;
    background: var(--border-dim);
    flex-shrink: 0;
}

#zoom-slider {
    width: 100px;
    accent-color: var(--accent);
    height: 3px;
}

#beat-offset {
    width: 70px;
    font-size: 12px;
    padding: 2px 4px;
}

.snap-btn.active {
    background: var(--accent);
    color: var(--bg-primary);
    border-color: var(--accent);
}

/* ── Block Inspector Strip ─────────────────────── */
.block-inspector {
    height: 40px;
    background: var(--bg-panel-dark);
    border-top: 1px solid var(--border-dim);
    display: flex;
    align-items: center;
    gap: 16px;
    padding: 0 12px;
    flex-shrink: 0;
}

.inspector-label {
    color: var(--text-accent);
    font-size: 14px;
    font-weight: 600;
    text-transform: uppercase;
    flex-shrink: 0;
    min-width: 120px;
}

.inspector-field {
    display: flex;
    align-items: center;
    gap: 4px;
    font-size: 10px;
    text-transform: uppercase;
    color: var(--text-dim);
    letter-spacing: 0.1em;
}

.inspector-field input,
.inspector-field select {
    font-size: 12px;
    padding: 2px 4px;
    background: var(--bg-panel);
    border: 1px solid var(--border-dim);
    color: var(--text-primary);
    font-family: var(--font-mono);
}

.inspector-field input[type="number"] {
    width: 60px;
}

.inspector-color-swatch {
    width: 16px;
    height: 16px;
    border: 1px solid var(--border-dim);
    cursor: pointer;
    flex-shrink: 0;
}

.inspector-remove {
    margin-left: auto;
    background: none;
    border: none;
    color: var(--text-dim);
    font-size: 16px;
    cursor: pointer;
    padding: 0 4px;
}
.inspector-remove:hover {
    color: #ff3333;
    border: none;
}

3. Wire timeline into app.js:

At the TOP of app.js, add the import:

import { TimelineCanvas } from './timeline/timeline.js';

At the BOTTOM of app.js (after wireAudio();), add initialization:

// ── Timeline ─────────────────────────────────────────────────────────────────

let timeline = null;

function initTimeline() {
    const canvas = document.getElementById('timeline-canvas');
    const audio = document.getElementById('player');
    if (!canvas || !audio) return;

    timeline = new TimelineCanvas(canvas, audio);

    // Load device tracks
    fetch('/api/devices').then(r => r.json()).then(devices => {
        timeline.loadTracks(devices);
    }).catch(e => console.warn('[timeline]', e));

    // When audio loads, fetch waveform + beats
    audio.addEventListener('loadedmetadata', () => {
        const src = audio.src;
        const filename = src.split('/').pop();
        const path = `/app/shows/${filename}`;
        timeline.audioDuration = audio.duration;
        timeline.loadWaveform(path);
        timeline.loadBeats(path);
    });

    // Zoom slider
    document.getElementById('zoom-slider')?.addEventListener('input', (e) => {
        timeline.pixelsPerSecond = parseFloat(e.target.value);
    });

    // Beat offset
    document.getElementById('beat-offset')?.addEventListener('input', (e) => {
        timeline.calibrationOffset = parseFloat(e.target.value) || 0.03;
    });

    // Snap toggle
    document.getElementById('btn-snap')?.addEventListener('click', (e) => {
        timeline.snapEnabled = !timeline.snapEnabled;
        e.target.classList.toggle('active', timeline.snapEnabled);
    });

    // Populate animation palette in sidebar
    const palette = document.getElementById('animation-palette');
    if (palette) {
        const anims = ['chase', 'pulse', 'rainbow', 'strobe', 'color_wipe', 'fire', 'solid'];
        palette.innerHTML = anims.map(name => `
            <button class="anim-tile" data-anim="${name}" draggable="true">
                <span class="anim-cursor">&gt;</span>
                <span class="anim-name">${name.toUpperCase().replace('_', ' ')}</span>
            </button>
        `).join('');
    }

    timeline.startRenderLoop();
}

initTimeline();

Remove or comment out the old selectedAnim variable and the animation-grid click listener (lines that reference #animation-grid), since that element no longer exists in main-area. cd /home/claude/led2 && test -f lightsync/frontend/timeline/timeline.js && grep -q "export.*TimelineCanvas" lightsync/frontend/timeline/timeline.js && grep -q "timeToX" lightsync/frontend/timeline/timeline.js && grep -q "requestAnimationFrame" lightsync/frontend/timeline/timeline.js && grep -q "devicePixelRatio" lightsync/frontend/timeline/timeline.js && grep -q "import.*TimelineCanvas" lightsync/frontend/app.js && grep -q "#timeline-canvas" lightsync/frontend/style.css && grep -q ".block-inspector" lightsync/frontend/style.css && echo "ALL PASS" <acceptance_criteria> - lightsync/frontend/timeline/timeline.js exists and exports TimelineCanvas - timeline.js contains timeToX, xToTime, trackIndexToY, yToTrackIndex methods - timeline.js contains _resizeCanvas with devicePixelRatio handling - timeline.js contains render() method with layer drawing (background, waveform, beat marks, blocks, headers, time axis, cursor) - timeline.js contains startRenderLoop() using requestAnimationFrame - timeline.js contains loadTracks(), loadWaveform(), loadBeats() methods - timeline.js contains wheel scroll handler updating scrollX - app.js imports TimelineCanvas from './timeline/timeline.js' - app.js has initTimeline() function that creates TimelineCanvas, wires zoom/beat-offset/snap - app.js populates #animation-palette with 7 animation types - style.css contains #timeline-canvas, .timeline-toolbar, .block-inspector, .inspector-label rules - Old animation-grid click listener removed from app.js </acceptance_criteria> Timeline canvas renders per-device tracks with headers, time axis, waveform background, beat marks, and a playback cursor synced to audio.currentTime via requestAnimationFrame. Toolbar controls zoom, beat offset, and snap toggle. Animation palette rendered dynamically in sidebar.

1. Docker rebuild: `cd /home/claude/led2 && docker compose build && docker compose up -d` 2. Open browser: navigate to the app URL 3. Verify: per-device tracks visible (if devices exist), time axis with second markers, toolbar with ZOOM/BEAT OFFSET/SNAP controls 4. Load audio file: waveform peaks appear as dim background, beat marks appear as cyan lines 5. Press play: cursor moves across timeline in sync with audio 6. Scroll: mouse wheel scrolls timeline horizontally 7. Sidebar: ANIMATIONS panel visible below DEVICES with 7 animation types 8. Python model: `python -c "from lightsync.models.show import CueModel; c = CueModel(timestamp=0); print(c.duration)"` prints 4.0

<success_criteria>

• Timeline canvas fills the main area with per-device horizontal tracks
• Playback cursor moves in realtime synced to audio.currentTime
• Waveform peaks render behind tracks
• Beat marks overlay on timeline at correct time positions
• Toolbar zoom slider adjusts pixelsPerSecond
• CueModel.duration defaults to 4.0
• AnalysisBlock.calibration_offset defaults to 0.03 </success_criteria>

After completion, create `.planning/phases/04-timeline-editor/04-01-SUMMARY.md`