docs(06): capture phase context
This commit is contained in:
161
.planning/phases/06-ai-sync/06-DISCUSSION-LOG.md
Normal file
161
.planning/phases/06-ai-sync/06-DISCUSSION-LOG.md
Normal file
@@ -0,0 +1,161 @@
|
||||
# Phase 6: AI Sync - Discussion Log
|
||||
|
||||
> **Audit trail only.** Do not use as input to planning, research, or execution agents.
|
||||
> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered.
|
||||
|
||||
**Date:** 2026-04-07
|
||||
**Phase:** 06-ai-sync
|
||||
**Areas discussed:** YouTube Loading, AI Sync Engine, Segmentation Overlay UX
|
||||
|
||||
---
|
||||
|
||||
## YouTube Loading
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| yt-dlp download | Server runs yt-dlp to download to /app/shows/; reuses all existing wiring | ✓ |
|
||||
| yt-dlp extract + stream | Extract direct audio URL, pass to browser; no disk write | |
|
||||
| Add MPV backend | Introduce MPV as audio engine with --ytdl flag; replaces transport layer | |
|
||||
|
||||
**User's choice:** yt-dlp download to /app/shows/
|
||||
**Notes:** No MPV backend change. Download reuses the existing local file flow entirely.
|
||||
|
||||
---
|
||||
|
||||
### Loading UI
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Spinner + progress text | 'DOWNLOADING... [title]' in transport bar | |
|
||||
| Progress bar | Terminal-style % bar via yt-dlp --progress parsing | ✓ |
|
||||
| Just disable + wait | Disable UI, no feedback | |
|
||||
|
||||
**User's choice:** Terminal-style progress bar with download percentage
|
||||
|
||||
---
|
||||
|
||||
### URL input placement
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Transport bar | Inline with existing file controls | ✓ |
|
||||
| Separate panel / modal | Dedicated overlay | |
|
||||
|
||||
**User's choice:** Transport bar — inline text input + FETCH button
|
||||
|
||||
---
|
||||
|
||||
## AI Sync Engine
|
||||
|
||||
### Heuristics vs. LLM
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Heuristics only | beat/onset/chroma → rules, no API key needed | |
|
||||
| Heuristics + optional LLM | Heuristic base + optional LLM enhance step | ✓ |
|
||||
| LLM-first | LLM generates full show, heuristics as fallback | |
|
||||
|
||||
**User's choice:** Heuristics + optional LLM
|
||||
|
||||
---
|
||||
|
||||
### Audio features
|
||||
|
||||
| Feature | Description | Selected |
|
||||
|---------|-------------|----------|
|
||||
| Beats | librosa beat_track (existing) | ✓ |
|
||||
| Onsets | librosa.onset.onset_detect() | ✓ |
|
||||
| Chroma / key | librosa.feature.chroma_cqt() | ✓ |
|
||||
| RMS energy | librosa.feature.rms() | ✓ |
|
||||
|
||||
**User's choice:** All four features
|
||||
|
||||
---
|
||||
|
||||
### LLM path behavior
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Remap + rename | LLM rewrites heuristic animation assignments | |
|
||||
| Generate from scratch | LLM gets raw features, generates complete block list | ✓ |
|
||||
| Describe + confirm | LLM describes song, user reviews, heuristics execute | |
|
||||
|
||||
**User's choice:** Generate from scratch — LLM receives raw audio features, produces a full block list independently
|
||||
|
||||
---
|
||||
|
||||
### Conflict handling
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Prompt first | Confirmation dialog before replacing existing blocks | ✓ |
|
||||
| Always replace | Clear and fill, no prompt (undo recovers) | |
|
||||
| Fill gaps only | Only fill time ranges with no existing blocks | |
|
||||
|
||||
**User's choice:** Prompt first
|
||||
|
||||
---
|
||||
|
||||
### AI Sync button location
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Toolbar / timeline header | Button in timeline controls area | |
|
||||
| Transport bar | Next to play/pause | |
|
||||
| Context menu / separate panel | Hidden behind menu or dedicated panel | ✓ |
|
||||
|
||||
**User's choice:** Context menu / separate panel — keeps main UI clean
|
||||
|
||||
---
|
||||
|
||||
## Segmentation Overlay UX
|
||||
|
||||
### Visual style
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Colored bands with labels | Translucent colored rectangles + SEC 1, SEC 2... labels | ✓ |
|
||||
| Color bands only | Same but no labels | |
|
||||
| Region markers only | Vertical tick marks at section boundaries | |
|
||||
|
||||
**User's choice:** Colored bands with labels
|
||||
|
||||
---
|
||||
|
||||
### Label naming
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Generic numbered | SEC 1, SEC 2, SEC 3... | ✓ |
|
||||
| Structural guess | INTRO / VERSE / CHORUS / OUTRO heuristics | |
|
||||
| LLM-named | Feed segments to LLM for naming | |
|
||||
|
||||
**User's choice:** Generic numbered (SEC 1, SEC 2, ...)
|
||||
|
||||
---
|
||||
|
||||
### Interactivity
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Purely visual | No click behavior | |
|
||||
| Click to seek | Click band → seek to section start | |
|
||||
| Click to select + fill | Click selects section; AI Sync can fill only that section | ✓ |
|
||||
|
||||
**User's choice:** Click to select + fill — AI Sync respects selected section as target range
|
||||
|
||||
---
|
||||
|
||||
## Claude's Discretion
|
||||
|
||||
- librosa segmentation algorithm choice
|
||||
- LLM prompt design and feature payload structure
|
||||
- yt-dlp progress output parsing approach
|
||||
- Section color palette (from existing CSS variables)
|
||||
|
||||
## Deferred Ideas
|
||||
|
||||
- Per-section manual label renaming
|
||||
- LLM-named sections
|
||||
- Full MPV backend / streaming YouTube
|
||||
- Gap-only fill mode for AI Sync
|
||||
Reference in New Issue
Block a user