diff --git a/.planning/phases/02-app-core-audio/02-CONTEXT.md b/.planning/phases/02-app-core-audio/02-CONTEXT.md new file mode 100644 index 0000000..9dc066c --- /dev/null +++ b/.planning/phases/02-app-core-audio/02-CONTEXT.md @@ -0,0 +1,115 @@ +# Phase 2: App Core + Audio - Context + +**Gathered:** 2026-04-03 +**Status:** Ready for planning + + +## Phase Boundary + +Headless Pi-side Python application: song playback (MP3/FLAC/WAV), beat detection from system audio, choreography data model with JSON persistence, UDP transport client to ESP32, and a monotonic-clock event scheduler. Everything testable from the command line without any TUI. + + + + +## Implementation Decisions + +### Choreography File Format +- **D-01:** Choreography files are Pydantic-validated JSON. Schema: `{"song_path": str, "bpm": float|null, "events": [{"timestamp": float, "zone": str, "animation": str, "params": {...}, "loop": int|null, "duration": float|null}]}`. File-level metadata includes song path and optional BPM. +- **D-02:** Events are an ordered array sorted by timestamp. Each event specifies zone, animation name, params (matching ESP32 protocol), optional loop count, and optional duration. +- **D-03:** File extension: `.choreo.json` for choreography files. + +### Audio Pipeline +- **D-04:** Two separate threads: miniaudio for song playback (with sample-accurate position tracking), sounddevice+aubio for system audio capture and beat detection. Neither thread touches the other's audio device. +- **D-05:** Beat detection thread uses `call_from_thread()` bridge to post beat events into the asyncio event loop. Beat events carry the detected timestamp. +- **D-06:** Audio capture uses PipeWire's PulseAudio compatibility layer via sounddevice. Monitor source for system audio loopback. + +### Playback Clock +- **D-07:** Absolute monotonic timestamp scheduling. Record `start_time = time.monotonic()` at play start. Each event fires at `start_time + event.timestamp`. Recalculate remaining sleep each iteration — never accumulate relative delays. +- **D-08:** Drift tolerance: must stay under 20ms over 5 minutes of playback. Validate with test. +- **D-09:** Pause/resume adjusts `start_time` offset so event scheduling remains correct after resume. + +### UDP Transport Client +- **D-10:** Asyncio DatagramProtocol wrapping UDP socket. Sends JSON commands to ESP32 on port 4210. Matches the protocol defined in `docs/protocol.md` exactly. +- **D-11:** Heartbeat: optional periodic ping to detect ESP32 availability (not blocking — fire-and-forget). +- **D-12:** Connection state tracked (connected/disconnected) based on ESP32 STATUS responses. Displayed in CLI. + +### CLI Testing Interface +- **D-13:** Simple asyncio REPL for testing. Commands: `play `, `pause`, `resume`, `seek `, `stop`, `load `, `add [params_json]`, `save `, `status`, `quit`. +- **D-14:** Not a production interface — Phase 3 TUI replaces this. Minimal error handling, no fancy output. + +### Claude's Discretion +- Python package structure (src layout vs flat) +- asyncio task organization and cancellation patterns +- Pydantic model field validation details +- Error handling granularity in transport layer +- aubio configuration (buffer size, hop size, threshold) + + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +### ESP32 Protocol +- `docs/protocol.md` — UDP JSON protocol spec (port 4210, command structure, animation names, zone values). Pi-side transport MUST match this exactly. + +### Firmware Source (for integration understanding) +- `firmware/src/protocol.h` — LedCommand struct, CmdType enum, Zone enum +- `firmware/src/wifi_server.h` — UDP server interface (port, queue) + +### Project Context +- `.planning/PROJECT.md` — Project vision, hardware specs +- `.planning/REQUIREMENTS.md` — AUD-01..04, CHR-04, CHR-05 + +### Research +- `.planning/research/STACK.md` — miniaudio, sounddevice, aubio recommendations +- `.planning/research/ARCHITECTURE.md` — Component boundaries, async/thread architecture +- `.planning/research/PITFALLS.md` — Timeline drift, audio thread safety, PipeWire compatibility + +### Prior Phase Context +- `.planning/phases/01-esp32-firmware/01-CONTEXT.md` — Firmware decisions (JSON protocol, animation names, zone addressing) + + + + +## Existing Code Insights + +### Reusable Assets +- `docs/protocol.md` — Complete protocol specification, Pi-side client must implement this +- `firmware/src/protocol.h` — Reference for command structure (LedCommand, AnimParams, Zone, CmdType) + +### Established Patterns +- JSON command format: `{"v":1, "zone":"...", "animation":"...", "params":{...}}` +- Zone values: "schrank", "wand", "all" +- 8 animation names: chase, pulse, rainbow, strobe, color_wash, breathe, sparkle, gradient_sweep +- UDP port: 4210 + +### Integration Points +- UDP socket to ESP32 (port 4210) — fire-and-forget commands +- Choreography JSON files — load/save persistence +- System audio via PipeWire — beat detection input + + + + +## Specific Ideas + +- This phase produces a headless app that Phase 3 wraps with Textual TUI +- The asyncio event loop and playback clock must be designed for TUI integration from the start (Textual is asyncio-native) +- Beat detection must be a standalone module that Phase 5 (Live Reactive) can reuse +- Choreography data model must support the loop/repeat feature from CHR-03 (Phase 3) even if the CLI doesn't expose full loop editing + + + + +## Deferred Ideas + +None — discussion stayed within phase scope (auto mode) + + + +--- + +*Phase: 02-app-core-audio* +*Context gathered: 2026-04-03* diff --git a/.planning/phases/02-app-core-audio/02-DISCUSSION-LOG.md b/.planning/phases/02-app-core-audio/02-DISCUSSION-LOG.md new file mode 100644 index 0000000..b4542cb --- /dev/null +++ b/.planning/phases/02-app-core-audio/02-DISCUSSION-LOG.md @@ -0,0 +1,67 @@ +# Phase 2: App Core + Audio - Discussion Log + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. +> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered. + +**Date:** 2026-04-03 +**Phase:** 02-app-core-audio +**Areas discussed:** Choreography format, Audio pipeline, Playback clock, CLI interface +**Mode:** auto (all areas auto-selected, recommended defaults chosen) + +--- + +## Choreography File Format + +| Option | Description | Selected | +|--------|-------------|----------| +| Pydantic JSON with event list | Structured, validated, sorted events | [auto] | +| YAML with event list | More human-readable but slower parsing | | +| SQLite database | Overkill for file-based storage | | + +**User's choice:** [auto] Pydantic-validated JSON (recommended default) + +--- + +## Audio Pipeline + +| Option | Description | Selected | +|--------|-------------|----------| +| Separate threads (miniaudio + sounddevice) | Clean isolation, no device contention | [auto] | +| Single thread with multiplexing | Complex, potential blocking | | +| Subprocess for audio | Over-engineered for this use case | | + +**User's choice:** [auto] Separate threads with call_from_thread bridge (recommended default) + +--- + +## Playback Clock + +| Option | Description | Selected | +|--------|-------------|----------| +| Absolute monotonic timestamps | No drift, correct after pause/resume | [auto] | +| Relative sleep chains | Simple but drifts 100-200ms/minute | | +| Hardware timer | Overkill for Pi | | + +**User's choice:** [auto] Absolute monotonic timestamps (recommended default) + +--- + +## CLI Interface + +| Option | Description | Selected | +|--------|-------------|----------| +| Simple asyncio REPL | Lightweight, disposable | [auto] | +| Click/Typer CLI | Over-engineered for testing | | +| No CLI (scripts only) | Less interactive | | + +**User's choice:** [auto] Simple asyncio REPL (recommended default) + +--- + +## Claude's Discretion + +- Python package structure, asyncio task organization, Pydantic details, aubio config + +## Deferred Ideas + +None