---
phase: 06-ai-sync
plan: 03
subsystem: ai-sync
tags: [heuristic-generation, llm-generation, anthropic, chroma-analysis, beat-snapping, ui-panel]
dependency_graph:
  requires:
    - phase: 06-02
      provides: "extract_features_async, detect_segments_async — audio analysis foundation"
    - phase: 06-01
      provides: "YouTube audio loading, ANTHROPIC_API_KEY wiring"
  provides:
    - "POST /api/sync/generate — heuristic + LLM show generation endpoint"
    - "GET /api/sync/llm-available — API key presence check"
    - "AI Sync panel with GENERATE button, USE LLM toggle, section range support"
  affects: [lightsync/api/sync.py, lightsync/frontend]
tech-stack:
  added: []
  patterns:
    - "Heuristic rule mapper: RMS-to-animation-type, chroma pitch class-to-HSL-hue color, beat-snapped block placement"
    - "LLM lazy initialization: import anthropic inside function to avoid startup crash when API key missing"
    - "setInterval-based segment selection watcher for reactive UI without event subscription"

key-files:
  created:
    - lightsync/api/sync.py
  modified:
    - lightsync/main.py
    - lightsync/frontend/index.html
    - lightsync/frontend/app.js
    - lightsync/frontend/style.css

key-decisions:
  - "Heuristic always runs first; LLM is opt-in via use_llm flag (D-04)"
  - "LLM generates from scratch, not from heuristic output (D-06)"
  - "section_range param limits generation to selected segment time window (D-11)"
  - "LLM toggle disabled when ANTHROPIC_API_KEY not configured — Pitfall 4 guard"
  - "claude-3-haiku-20240307 chosen for LLM model (speed vs cost)"

patterns-established:
  - "API key guard pattern: check os.environ.get('ANTHROPIC_API_KEY') in async function, raise 503 if missing"
  - "Lazy anthropic import: import inside function avoids startup crash without key"

requirements-completed: [SYNC-03]

duration: 8min
completed: 2026-04-07
---

# Phase 06 Plan 03: AI-Assisted Show Generation Summary

**Heuristic + LLM show generator with dedicated AI Sync panel: fills timeline with beat-snapped, energy-matched, harmonically-colored animation blocks from audio analysis.**

## Performance

- **Duration:** ~8 min
- **Started:** 2026-04-07T11:39:00Z
- **Completed:** 2026-04-07T11:47:18Z
- **Tasks:** 2
- **Files modified:** 5

## Accomplishments

1. **AI sync backend** (`lightsync/api/sync.py`): `POST /api/sync/generate` endpoint with two generation modes:
   - `heuristic_generate`: section-level RMS-to-animation-type mapping, chroma pitch-class-to-HSL color, beat-grid-snapped block placement with density-adaptive grouping
   - `llm_generate`: Anthropic claude-3-haiku with compact feature payload, JSON parse + validation, lazy init to guard against missing API key
   - `GET /api/sync/llm-available` for frontend toggle gating
   - `section_range` parameter for per-section fill (D-11)

2. **AI Sync panel UI** (`index.html`, `app.js`, `style.css`):
   - AI SYNC button in toolbar toggles collapsible panel
   - Feature toggles: BEATS, ONSETS, CHROMA, RMS
   - GENERATE button disabled until audio loaded; re-enabled on loadedmetadata
   - USE LLM toggle: disabled when ANTHROPIC_API_KEY not set (checked via llm-available endpoint)
   - Confirmation dialog before replacing existing blocks (D-07)
   - Section-aware: if a segment band is selected, fills only that time range (D-11)
   - Info text shows FULL SHOW or SECTION: SEC N, updates via 200ms interval watcher
   - Terminal/flat aesthetic — no border-radius

## Task Commits

1. **Task 1: AI sync backend** - `b6aacad` (feat)
2. **Task 2: AI Sync panel UI + frontend wiring** - `70b86f7` (feat)

## Files Created/Modified

- `lightsync/api/sync.py` - New: `/api/sync/generate` endpoint, `heuristic_generate`, `llm_generate`, `SyncGenerateRequest`, `_chroma_to_rgb`, `_intensity_to_animation`
- `lightsync/main.py` - Register `sync.router` with prefix `/api/sync`
- `lightsync/frontend/index.html` - AI SYNC button, ai-sync-panel div with feature toggles and GENERATE button
- `lightsync/frontend/app.js` - Panel toggle, llm-available check, segment watcher, GENERATE handler with D-07/D-11 logic
- `lightsync/frontend/style.css` - AI sync panel styles (flat terminal aesthetic)

## Decisions Made

- Heuristic always available as baseline; LLM is opt-in (D-04) — no LLM dependency for core functionality
- LLM generates independently from scratch, not post-processing heuristic output (D-06)
- claude-3-haiku-20240307 for LLM: fast and cheap for structured JSON output
- `anthropic` imported lazily inside `llm_generate` to avoid import errors at container startup without API key (Pitfall 4)

## Deviations from Plan

None - plan executed exactly as written.

## Known Stubs

None - all functionality is fully wired end-to-end.

## Self-Check: PASSED

- `lightsync/api/sync.py` exists: contains `router = APIRouter()`, `def heuristic_generate(`, `async def llm_generate(`, `async def generate_sync(`, `async def llm_available(`, `class SyncGenerateRequest`, `section_range: list[float] | None`, `use_llm: bool = False`, `ANTHROPIC_API_KEY`, `claude-3-haiku`, `_chroma_to_rgb`, `_intensity_to_animation`
- `lightsync/main.py` contains `from lightsync.api import shows, devices, ws, audio, timeline, sync` and `sync.router`
- `lightsync/frontend/index.html` contains `id="ai-sync-panel"`, `id="btn-ai-sync-open"`, `id="btn-ai-generate"`, `id="ai-use-llm"`, `ai-use-beats`, `style="display:none;"`
- `lightsync/frontend/app.js` contains `api/sync/generate`, `api/sync/llm-available`, `Replace all existing blocks`, `section_range`, `segmentOverlay?.getSelected()`, `GENERATING...`
- `lightsync/frontend/style.css` contains `.ai-sync-panel`, `.ai-sync-header`, `.ai-sync-generate`
- Heuristic generates blocks with valid animation types and params with color/speed
- All 2 task commits exist: `b6aacad`, `70b86f7`