led2/.planning/phases/06-ai-sync/06-01-PLAN.md

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves, user_setup

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves	user_setup
06-ai-sync	01	execute	1		pyproject.toml docker-compose.prod.yml lightsync/api/audio.py lightsync/frontend/index.html lightsync/frontend/app.js lightsync/frontend/style.css	true	AUD-02	truths artifacts key_links A YouTube URL pasted into the transport bar downloads the audio to /app/shows/ as an MP3 file Download progress is displayed as a terminal-style block progress bar in the transport bar After download completes, the audio file auto-loads into the player and waveform/beats load identically to a local upload The URL input is disabled during download and re-enabled on completion or error path provides contains pyproject.toml yt-dlp and anthropic dependencies yt-dlp path provides contains lightsync/api/audio.py POST /api/audio/fetch endpoint with SSE progress streaming async def fetch_youtube path provides contains lightsync/frontend/index.html YouTube URL text input and FETCH button in transport bar yt-url path provides contains lightsync/frontend/app.js SSE ReadableStream reader, progress bar rendering, auto-load on completion fetchWithProgress from to via pattern lightsync/frontend/app.js /api/audio/fetch fetch POST with ReadableStream reader api/audio/fetch from to via pattern lightsync/api/audio.py yt-dlp subprocess asyncio.create_subprocess_exec yt_dlp from to via pattern lightsync/frontend/app.js audio element auto-load after download done event audio.src.*shows/	service why env_vars dashboard_config anthropic Optional LLM show generation (Plan 06-03) name source ANTHROPIC_API_KEY Anthropic Console -> API Keys (https://console.anthropic.com/settings/keys)

YouTube audio loading via yt-dlp with terminal-style progress streaming.

Purpose: Enables loading songs from YouTube URLs (AUD-02), the primary new input method for Phase 6. After download, the file enters the existing audio pipeline unchanged (waveform, beats, show saving all reuse existing wiring).

Output: Working YouTube fetch endpoint with SSE progress, transport bar URL input, and Docker image with yt-dlp + anthropic dependencies installed.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/06-ai-sync/06-CONTEXT.md @.planning/phases/06-ai-sync/06-RESEARCH.md

From lightsync/api/audio.py:

router = APIRouter()
SHOWS_DIR = Path("/app/shows")
SUPPORTED_EXTENSIONS = {".mp3", ".wav", ".flac", ".ogg"}

@router.post("/upload")
async def upload_audio(request: Request, file: UploadFile = File(...)):
    # Uploads file to SHOWS_DIR, runs beat analysis, caches in app.state.beats
    ...

@router.get("/beats")
async def get_beats(request: Request, path: str):
    ...

@router.get("/waveform")
async def get_waveform(path: str, peaks: int = 1000):
    ...

From lightsync/models/show.py:

class AudioRef(BaseModel):
    source_type: Literal["file", "youtube"] = "file"
    path: str | None = None
    yt_url: str | None = None
    duration_seconds: float | None = None

From lightsync/main.py:

app.include_router(audio.router, prefix="/api/audio")
# app.state.beats = {} in lifespan

From lightsync/frontend/app.js:

// Audio load flow after file selection:
// audio.src = `/shows/${filename}`; audio.load();
// Then loadedmetadata fires -> timeline.loadWaveform(path) + timeline.loadBeats(path)

async function refreshFiles(selectPath) { ... }
// Refreshes audio-select dropdown from /api/audio/files

From lightsync/frontend/index.html (transport bar, lines 84-101):

<div class="transport-sep"></div>
<label class="transport-btn" title="Upload MP3/WAV/FLAC/OGG" style="cursor:pointer;">
    + UPLOAD
    <input type="file" id="audio-upload" accept=".mp3,.wav,.flac,.ogg" style="display:none;">
</label>
<select id="audio-select" class="transport-file-select">
    <option value="">&#8212; no files &#8212;</option>
</select>
<button id="btn-load" class="transport-btn" title="Load selected file">LOAD</button>

Task 1: Add dependencies and rebuild Docker image pyproject.toml, docker-compose.prod.yml - pyproject.toml (current dependencies list) - docker-compose.prod.yml (current environment config) - Dockerfile (build steps) 1. Edit `pyproject.toml` dependencies array — add these two lines after the existing `"python-multipart>=0.0.9"` entry: ``` "yt-dlp>=2025.0.0", "anthropic>=0.40.0", ```

2. Edit docker-compose.prod.yml — add ANTHROPIC_API_KEY to the environment section (empty default so the app starts without it):

   environment:
     HOST: 0.0.0.0
     PORT: "8000"
     ANTHROPIC_API_KEY: "${ANTHROPIC_API_KEY:-}"
   

3. Rebuild and restart the Docker container:

   cd /home/claude/led2
   docker compose -f docker-compose.prod.yml build
   docker compose -f docker-compose.prod.yml up -d --force-recreate
   

4. Verify yt-dlp is available inside the container:

   docker exec lightsync python -c "import yt_dlp; print(yt_dlp.version.__version__)"
   docker exec lightsync python -m yt_dlp --version
   

docker exec lightsync python -c "import yt_dlp; print('yt-dlp OK')" && docker exec lightsync python -c "import anthropic; print('anthropic OK')" - pyproject.toml contains `"yt-dlp>=2025.0.0"` - pyproject.toml contains `"anthropic>=0.40.0"` - docker-compose.prod.yml contains `ANTHROPIC_API_KEY` - `docker exec lightsync python -c "import yt_dlp"` exits 0 - `docker exec lightsync python -c "import anthropic"` exits 0 Both yt-dlp and anthropic are importable inside the running container. ANTHROPIC_API_KEY env var is wired (empty by default). Task 2: YouTube download endpoint with SSE progress streaming lightsync/api/audio.py - lightsync/api/audio.py (existing endpoints and patterns) - lightsync/audio/beats.py (analyze_async pattern for post-download beat analysis) - lightsync/main.py (app.state.beats cache) Add a POST `/fetch` endpoint to `lightsync/api/audio.py` that:

1. Add imports at the top of the file:

   import asyncio
   import json
   import re
   import sys
   from fastapi import Request
   from fastapi.responses import StreamingResponse
   

2. Add constants:

   PROGRESS_RE = re.compile(r'\[download\]\s+([\d.]+)%')
   

3. Add Pydantic model for the request body:

   from pydantic import BaseModel
   
   class YouTubeFetchRequest(BaseModel):
       url: str
   

4. Add the endpoint:

   @router.post("/fetch")
   async def fetch_youtube(request: Request, body: YouTubeFetchRequest):
       """Download audio from YouTube URL via yt-dlp. Streams SSE progress events."""
   
       async def event_stream():
           filename = None
           try:
               proc = await asyncio.create_subprocess_exec(
                   sys.executable, '-m', 'yt_dlp',
                   '--extract-audio',
                   '--audio-format', 'mp3',
                   '--audio-quality', '0',
                   '--newline',
                   '--no-playlist',
                   '-o', str(SHOWS_DIR / 'yt_%(id)s.%(ext)s'),
                   body.url,
                   stdout=asyncio.subprocess.PIPE,
                   stderr=asyncio.subprocess.STDOUT,
               )
   
               async for line_bytes in proc.stdout:
                   line = line_bytes.decode('utf-8', errors='replace').rstrip()
                   # Progress line
                   m = PROGRESS_RE.search(line)
                   if m:
                       pct = float(m.group(1))
                       yield f"data: {json.dumps({'type': 'progress', 'pct': pct})}\n\n"
                   # Destination line — captures actual output filename
                   if 'Destination:' in line:
                       filename = line.split('Destination:')[-1].strip()
   
               await proc.wait()
   
               if proc.returncode != 0:
                   yield f"data: {json.dumps({'type': 'error', 'msg': f'yt-dlp exited with code {proc.returncode}'})}\n\n"
                   return
   
               if not filename:
                   # Fallback: find most recently created mp3 in SHOWS_DIR
                   mp3s = sorted(SHOWS_DIR.glob('yt_*.mp3'), key=lambda p: p.stat().st_mtime, reverse=True)
                   if mp3s:
                       filename = str(mp3s[0])
   
               if filename:
                   # Run beat analysis in background and cache
                   try:
                       beat_data = await analyze_async(filename)
                       if not hasattr(request.app.state, 'beats') or request.app.state.beats is None:
                           request.app.state.beats = {}
                       request.app.state.beats[filename] = beat_data
                   except Exception:
                       pass  # Non-fatal — beats can be loaded later
   
                   fname = Path(filename).name
                   yield f"data: {json.dumps({'type': 'done', 'path': filename, 'filename': fname})}\n\n"
               else:
                   yield f"data: {json.dumps({'type': 'error', 'msg': 'Download completed but output file not found'})}\n\n"
   
           except Exception as e:
               yield f"data: {json.dumps({'type': 'error', 'msg': str(e)})}\n\n"
   
       return StreamingResponse(event_stream(), media_type="text/event-stream")
   

Key design decisions per CONTEXT.md:

• D-01: Downloads to SHOWS_DIR (/app/shows/) — same directory as uploaded files
• Uses sys.executable -m yt_dlp to avoid PATH issues in Docker (Pitfall 6 from RESEARCH.md)
• Uses yt_%(id)s.%(ext)s output template for predictable filenames (Pitfall 1 from RESEARCH.md)
• --newline flag is critical for line-by-line progress parsing (anti-pattern from RESEARCH.md)
• Runs beat analysis after download completes, caches in app.state.beats docker exec lightsync python -c "from lightsync.api.audio import fetch_youtube; print('endpoint imported OK')" <acceptance_criteria>
  • lightsync/api/audio.py contains async def fetch_youtube
  • lightsync/api/audio.py contains PROGRESS_RE = re.compile
  • lightsync/api/audio.py contains sys.executable, '-m', 'yt_dlp'
  • lightsync/api/audio.py contains --newline
  • lightsync/api/audio.py contains yt_%(id)s.%(ext)s
  • lightsync/api/audio.py contains media_type="text/event-stream"
  • lightsync/api/audio.py contains class YouTubeFetchRequest
  • lightsync/api/audio.py contains analyze_async(filename) (beat analysis after download) </acceptance_criteria> POST /api/audio/fetch accepts {url: string}, spawns yt-dlp subprocess, streams SSE progress events, runs beat analysis on completion, returns done event with filename.

Task 3: YouTube URL input UI with terminal progress bar lightsync/frontend/index.html, lightsync/frontend/app.js, lightsync/frontend/style.css - lightsync/frontend/index.html (transport bar layout, line ~84-104) - lightsync/frontend/app.js (wireAudio, refreshFiles, upload handler patterns) - lightsync/frontend/style.css (transport-btn, transport-file-select, transport-sep classes) **index.html** — Add YouTube URL input + FETCH button inline in the transport bar. Insert BEFORE the `

` spacer (after the LOAD button for audio files, around line 101). Per D-03: same row, terminal aesthetic, flat, monospace, no rounded corners.

<div class="transport-sep"></div>
<input type="text" id="yt-url" class="yt-url-input" placeholder="YOUTUBE URL" disabled>
<button id="btn-fetch" class="transport-btn" title="Download from YouTube">FETCH</button>
<span id="yt-progress" class="yt-progress" style="display:none;"></span>

Insert this block between <button id="btn-load" ...>LOAD</button> and <div style="flex:1"></div>.

style.css — Add styles at the end of the file. Use existing CSS variables only (--accent, --bg-panel, --border-dim, --text-dim, --text-primary, --font-mono). Per D-03: flat, monospace, no rounded corners.

/* YouTube URL input (Phase 6) */
.yt-url-input {
    background: var(--bg-panel);
    border: 1px solid var(--border-dim);
    color: var(--text-primary);
    font-family: var(--font-mono);
    font-size: 12px;
    padding: 2px 6px;
    width: 180px;
    outline: none;
}
.yt-url-input:focus {
    border-color: var(--accent);
}
.yt-url-input:disabled {
    opacity: 0.4;
    cursor: not-allowed;
}
.yt-progress {
    font-family: var(--font-mono);
    font-size: 11px;
    color: var(--accent);
    white-space: nowrap;
    min-width: 200px;
}

app.js — Add YouTube fetch handler. Insert after the upload handler (audio-upload change listener) and before the btn-load click listener section. Add these two functions and the event wiring:

1. Add the fetchWithProgress function (from RESEARCH.md Pattern 4 — SSE via fetch ReadableStream):

async function fetchWithProgress(url, body, onProgress, onDone, onError) {
    const res = await fetch(url, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify(body),
    });
    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let buf = '';
    while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        buf += decoder.decode(value, { stream: true });
        const lines = buf.split('\n\n');
        buf = lines.pop();
        for (const chunk of lines) {
            if (chunk.startsWith('data: ')) {
                const msg = JSON.parse(chunk.slice(6));
                if (msg.type === 'progress') onProgress(msg.pct);
                if (msg.type === 'done') onDone(msg);
                if (msg.type === 'error') onError(msg.msg);
            }
        }
    }
}

2. Add the terminal progress bar renderer (per D-02 — stepped block fill, monospace):

function renderProgressBar(pct, width = 20) {
    const filled = Math.round((pct / 100) * width);
    const empty = width - filled;
    return `DOWNLOADING ${'█'.repeat(filled)}${'░'.repeat(empty)} ${Math.round(pct)}%`;
}

3. Wire the FETCH button click handler:

document.getElementById('btn-fetch')?.addEventListener('click', async () => {
    const urlInput = document.getElementById('yt-url');
    const btn = document.getElementById('btn-fetch');
    const progress = document.getElementById('yt-progress');
    const url = urlInput?.value.trim();
    if (!url) return;

    // D-02: Disable input during download
    urlInput.disabled = true;
    btn.disabled = true;
    btn.textContent = '...';
    progress.style.display = 'inline';
    progress.textContent = 'DOWNLOADING ░░░░░░░░░░░░░░░░░░░░ 0%';

    try {
        await fetchWithProgress('/api/audio/fetch', { url },
            (pct) => {
                progress.textContent = renderProgressBar(pct);
            },
            (msg) => {
                // D-01: After download, load identically to local upload
                progress.textContent = 'COMPLETE';
                const audio = document.getElementById('player');
                if (audio && msg.filename) {
                    audio.src = `/shows/${msg.filename}`;
                    audio.load();
                }
                refreshFiles(msg.path);
                setTimeout(() => { progress.style.display = 'none'; }, 2000);
            },
            (errMsg) => {
                progress.textContent = `ERROR: ${errMsg}`;
                progress.style.color = '#ff4444';
                setTimeout(() => {
                    progress.style.display = 'none';
                    progress.style.color = '';
                }, 5000);
            }
        );
    } catch (e) {
        progress.textContent = `ERROR: ${e.message}`;
        setTimeout(() => { progress.style.display = 'none'; }, 5000);
    } finally {
        urlInput.disabled = false;
        urlInput.value = '';
        btn.disabled = false;
        btn.textContent = 'FETCH';
    }
});

4. Enable the URL input on page load (it starts disabled in HTML as a placeholder state, but should be enabled once JS is ready). Add after wireAudio():

document.getElementById('yt-url').disabled = false;

docker exec lightsync python -c " import pathlib html = pathlib.Path('/app/lightsync/frontend/index.html').read_text() assert 'yt-url' in html, 'missing yt-url input' assert 'btn-fetch' in html, 'missing btn-fetch' js = pathlib.Path('/app/lightsync/frontend/app.js').read_text() assert 'fetchWithProgress' in js, 'missing fetchWithProgress' assert 'renderProgressBar' in js, 'missing renderProgressBar' assert 'api/audio/fetch' in js, 'missing fetch endpoint call' css = pathlib.Path('/app/lightsync/frontend/style.css').read_text() assert 'yt-url-input' in css, 'missing yt-url-input style' assert 'yt-progress' in css, 'missing yt-progress style' print('All frontend checks passed') " - lightsync/frontend/index.html contains `id="yt-url"` input element - lightsync/frontend/index.html contains `id="btn-fetch"` button element - lightsync/frontend/index.html contains `id="yt-progress"` span element - lightsync/frontend/app.js contains `function fetchWithProgress(` - lightsync/frontend/app.js contains `function renderProgressBar(` - lightsync/frontend/app.js contains string literal `DOWNLOADING` - lightsync/frontend/app.js contains `/api/audio/fetch` - lightsync/frontend/app.js contains `urlInput.disabled = true` (D-02 disable during download) - lightsync/frontend/app.js contains `audio.src = ` followed by shows path (D-01 auto-load) - lightsync/frontend/style.css contains `.yt-url-input` - lightsync/frontend/style.css contains `.yt-progress` - No rounded corners in new CSS (border-radius must not appear in yt-url-input or yt-progress rules) YouTube URL input field and FETCH button appear in the transport bar. Clicking FETCH downloads audio with a terminal-style progress bar. After download, the audio auto-loads into the player and appears in the file selector. 1. Paste a YouTube URL into the transport bar input, click FETCH — progress bar fills with block characters 2. After download completes, audio loads automatically — waveform and beats appear on timeline 3. The downloaded file appears in the audio file selector dropdown 4. Entering an invalid URL shows an error message that auto-clears 5. During download, the URL input and FETCH button are disabled

<success_criteria>

• YouTube URL → audio file download works end-to-end
• Terminal-style progress bar displays during download
• Downloaded audio enters existing pipeline (waveform + beats + show saving)
• yt-dlp and anthropic packages available in Docker container </success_criteria>

After completion, create `.planning/phases/06-ai-sync/06-01-SUMMARY.md`