# Architecture Patterns: LED Sync Studio **Domain:** Terminal-based LED choreography system (Raspberry Pi + ESP32) **Researched:** 2026-04-03 **Confidence:** HIGH for component boundaries and data flow; MEDIUM for exact protocol choices (verified against ESP32 docs and WLED/xLights ecosystem) --- ## System Overview LED Sync Studio splits cleanly into two physical contexts: the Raspberry Pi runs the user-facing application and all intelligence, while the ESP32-C3 is a dumb executor — it receives animation commands and runs them locally on the LED strips. The Pi never streams raw pixel frames. This is the defining architectural constraint from which everything else follows. ``` ┌─────────────────────────────────────────────────────────────┐ │ Raspberry Pi 4B │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ TUI Layer │ │ App Core │ │ Audio Layer │ │ │ │ (Textual) │◄──►│ (Async) │◄──►│ (PipeWire) │ │ │ └──────────────┘ └──────┬───────┘ └──────────────┘ │ │ │ │ │ ┌──────▼───────┐ │ │ │ Transport │ │ │ │ (UDP/WiFi) │ │ │ └──────┬───────┘ │ └─────────────────────────────┼───────────────────────────────┘ │ JSON commands ┌─────────▼─────────┐ │ ESP32-C3 │ │ │ │ ┌─────────────┐ │ │ │ WiFi Server │ │ │ └──────┬──────┘ │ │ │ │ │ ┌──────▼──────┐ │ │ │ Anim Engine │ │ │ └──────┬──────┘ │ │ │ │ │ ┌──────▼──────┐ │ │ │ LED Drivers │ │ │ │ WS2801(SPI) │ │ │ │ SK6812(RMT) │ │ │ └─────────────┘ │ └───────────────────┘ ``` --- ## Component Boundaries ### Pi-Side Components | Component | Responsibility | Communicates With | |-----------|---------------|-------------------| | TUI Layer | Render all UI views, accept keystrokes, display timeline/event list | App Core (reactive messages) | | App Core | State machine, playback clock, choreography scheduler | TUI Layer, Audio Layer, Transport | | Choreography Engine | Load/save JSON files, manage event sequences, calculate playback position | App Core, file system | | Audio Layer | Song playback (MP3/FLAC/WAV), real-time beat detection from system audio | App Core (beat events) | | Transport | JSON serialization, UDP/TCP socket to ESP32, connection health | App Core | ### ESP32-Side Components | Component | Responsibility | Communicates With | |-----------|---------------|-------------------| | WiFi Server | Listen for UDP datagrams or TCP connections, parse JSON | Animation Engine | | Animation Engine | Maintain animation state, run effect loops, handle parameter updates | WiFi Server, LED Drivers | | LED Drivers | Hardware-specific output: WS2801 via SPI, SK6812 via RMT peripheral | Hardware (GPIO/SPI) | | Zone Manager | Route animation commands to correct strip (zone A = WS2801, zone B = SK6812) | Animation Engine, LED Drivers | --- ## Data Flow ### Choreography Playback Mode ``` Song file → Audio decoder → PulseAudio/PipeWire sink │ elapsed time │ Playback Clock ──► Choreography Engine │ scheduled event fires │ JSON command built │ UDP datagram → ESP32 │ Animation Engine starts │ LEDs execute effect ``` ### Live Reactive Mode ``` System audio → PyAudio/sounddevice → Audio buffer │ FFT / onset detection │ beat event (aubio) │ App Core receives event │ mapped animation selected │ JSON command → UDP → ESP32 │ LEDs react in near-realtime ``` ### User Interaction Flow ``` SSH terminal → Textual event loop → UI widget action → App Core message │ state mutation │ reactive UI update (no direct DOM) ``` ### Choreography File Data Model ``` Choreography (file: .json / .yaml) ├── metadata │ ├── song_path: str │ ├── song_duration_s: float │ └── bpm: float | null └── events: [ { time_s: float, # offset from song start zone: "ws2801" | "sk6812" | "both", animation: str, # registered name on ESP32 params: { # animation-specific color: [r, g, b], speed: float, intensity: float, loop: bool }, duration_s: float | null # null = run until replaced } ] ``` ### JSON Command Protocol (Pi → ESP32) ```json { "zone": "ws2801", "animation": "pulse", "params": { "color": [255, 0, 128], "speed": 0.8, "intensity": 1.0 } } ``` A separate command handles stop/clear: ```json { "zone": "all", "animation": "off" } ``` --- ## Recommended Architecture Pattern: Event-Driven Async Core The App Core is an asyncio event loop (Textual already runs on asyncio). Everything hangs off it: - **Playback clock** — asyncio task, fires scheduled events from choreography - **Audio processor** — runs in a thread (audio callbacks are synchronous), posts beat events into the asyncio loop via `call_soon_threadsafe` - **Network sender** — async UDP socket (asyncio DatagramProtocol), non-blocking - **File I/O** — async file reads (aiofiles) to avoid blocking the TUI This pattern avoids the thread-safety trap: only one asyncio task ever mutates state, threads post events inward via thread-safe bridges. ### Critical Pattern: Beat Detection Threading ``` Thread: audio callback (sounddevice/PyAudio) └── aubio onset detector └── beat detected └── loop.call_soon_threadsafe(post_beat_event) └── asyncio coroutine handles it └── sends UDP command ``` Audio must run in a thread — audio callbacks cannot be async. This is the one mandatory thread boundary in the system. --- ## Transport Protocol Decision: UDP over TCP **Use UDP for animation commands.** Rationale: - Animation commands are stateless triggers, not streams. A dropped packet means a missed animation start — tolerable. A delayed retransmission (TCP) means a late animation start — worse for musical sync. - UDP latency on a local WiFi network is typically 1-5ms vs TCP 10-30ms (with Nagle's algorithm; TCP_NODELAY helps but UDP is still lower). - ESP32-C3's single RISC-V core handles UDP receive with less overhead than full TCP stack maintenance. - Command payloads are small (< 200 bytes), well under UDP MTU (1470 bytes on typical WiFi). - WLED (the leading ESP32 LED firmware) uses UDP for all real-time control protocols (E1.31, DDP, Art-Net). This is industry standard for LED sync. **Heartbeat mechanism:** Pi sends a UDP heartbeat every 5s; ESP32 detects loss and enters "safe mode" (off or last known state). **Confidence:** HIGH — verified against WLED protocol architecture and ESP32 forum discussions on latency optimization. --- ## ESP32-C3 Constraints and Firmware Architecture The ESP32-C3 SuperMini has: - 400 KB SRAM (tight) - 4 MB Flash - Single RISC-V core at 160 MHz - No hardware FPU (floating-point operations are slower) ### FreeRTOS Task Allocation ``` Core 0 (only core): Task 1 (high priority): LED output — drives SPI for WS2801, RMT for SK6812 Task 2 (medium priority): WiFi + JSON receive Task 3 (low priority): Animation tick — updates frame buffer, feeds to LED task ``` WiFi and LED output cannot share timing perfectly on one core. The approach: LED task uses FreeRTOS queue to receive new frame buffers from the animation task. Animation task runs at a fixed interval (e.g., 20ms = 50fps) and posts computed frames into the queue. LED task drains the queue and writes to hardware. ### LED Driver Architecture WS2801 (SPI): - Uses ESP-IDF SPI master driver - Clock + Data pins, speed ~1 MHz - Blocking write per frame is acceptable (160 LEDs * 3 bytes = 480 bytes, fast at 1 MHz) SK6812 (single-wire): - Uses ESP-IDF RMT (Remote Control Transceiver) peripheral - RMT generates precise timing pulses without CPU involvement - Non-blocking: CPU queues frame data, RMT transmits via DMA - SK6812 is RGBW (4 bytes/LED) → 300 LEDs * 4 bytes = 1200 bytes per frame Both strips can update concurrently: SPI is blocking but fast, RMT is DMA-driven. Sequence: start RMT (non-blocking), write SPI (blocking), both finish in roughly the same window. --- ## Textual TUI Architecture Textual uses a reactive widget tree with message passing. For this app: ``` App (root) ├── Header (song info, BPM, mode) ├── TransportBar (play/pause/stop, position scrubber) ├── MainContent (switches between views) │ ├── TimelineView (horizontal scrollable canvas) │ │ ├── TimeRuler │ │ ├── ZoneTrack (WS2801) │ │ └── ZoneTrack (SK6812) │ └── EventListView (DataTable of events) ├── AnimationPanel (sidebar: select animation, set params) └── StatusBar (ESP32 connection, audio status, confidence) ``` **Reactive data binding:** Textual's `reactive` descriptor triggers re-renders when playback position changes. The playback clock posts position updates at ~10 Hz, which drives the timeline cursor without flooding the event loop. **Timeline rendering:** Custom Textual Canvas or Rich markup — render events as colored blocks at proportional x positions. Horizontal scroll follows playback cursor. --- ## Suggested Build Order (Dependencies) The dependency graph drives this order: ``` 1. ESP32 Firmware (foundation — everything depends on having a working LED executor) └── LED drivers (WS2801 SPI, SK6812 RMT) └── Animation engine (effect registry, parameter system) └── WiFi UDP server + JSON parser 2. Transport Layer (Pi side — needed to test firmware without UI) └── UDP socket client └── JSON command builder └── Connection management + heartbeat 3. App Core + State (no UI yet — testable via CLI) └── Choreography data model (JSON load/save) └── Playback clock └── Event scheduler 4. Audio Layer (can be developed in parallel with App Core after firmware done) └── Song playback (pygame/miniaudio) └── Beat detection (aubio + sounddevice) └── Audio→App Core event bridge 5. TUI Layer (depends on App Core having stable API) └── Transport controls + status └── Event list view └── Timeline view (most complex — build last within TUI) └── Animation parameter panel 6. Integration + Polish └── Cyberpunk visual theme └── Choreography file management (browse, rename, delete) └── Live reactive mode (ties Audio Layer to Animation dispatch) ``` **Why firmware first:** Every other component is validated against a real ESP32. Developing the protocol without firmware means writing a mock, then potentially changing everything when the real hardware behaves differently. Ship firmware first, test with curl/netcat, then build the Pi application around proven commands. **Why Audio Layer parallel to App Core:** Beat detection and song playback have no dependency on the UI. They can be prototyped as standalone scripts, then integrated. --- ## Cross-Cutting Concerns ### Latency Budget for Musical Sync Musical sync at 120 BPM has 500ms between beats. A cue that fires 50ms early or late is perceptible. Target latency budget: | Stage | Budget | Mechanism | |-------|--------|-----------| | Playback clock tick accuracy | < 10ms | asyncio event loop, avoid blocking | | JSON serialization | < 1ms | ujson (faster than stdlib json) | | WiFi UDP transit | < 10ms | local network, UDP | | ESP32 receive to animation start | < 5ms | FreeRTOS high-priority LED task | | **Total** | **< 26ms** | Well within perceptible threshold | ### Zone Independence WS2801 (cabinet, 160 LEDs) and SK6812 (wall, 300 LEDs) must be independently addressable. The JSON protocol always specifies a zone. The ESP32 Zone Manager routes to the correct driver. Both zones can run different animations simultaneously — this is a first-class requirement, not an afterthought. ### Animation Registry Animations are named strings registered in firmware. The Pi does not know animation internals — it only sends a name + parameters. This keeps the protocol stable: new animations are added to firmware without changing the Pi application, as long as the parameter schema is consistent. Define a shared animation manifest (JSON file on Pi, #define registry in firmware) so the TUI can offer valid animation names and parameters without hardcoding them in two places. --- ## Anti-Patterns to Avoid ### Anti-Pattern 1: Frame Streaming from Pi to ESP32 **What:** Sending raw pixel arrays (160*3 + 300*4 = 1680 bytes) at 50fps = 84 KB/s continuously over WiFi UDP **Why bad:** Saturates single-core ESP32 receive pipeline. WiFi on ESP32-C3 has practical throughput around 1-2 Mbps but latency spikes under load. 84 KB/s is feasible but leaves no headroom for WiFi overhead. Any retransmit or congestion causes visible glitch. Also burns CPU on both ends doing serialization. **Instead:** Commands only. The project already decided this correctly — ESP32 runs animations locally. ### Anti-Pattern 2: Blocking Audio Processing on Asyncio Loop **What:** Running aubio beat detection in an `async def` handler directly **Why bad:** aubio's Python bindings are synchronous C extensions. Blocking the asyncio event loop for even 10ms causes TUI lag and missed events. **Instead:** Thread worker with `call_soon_threadsafe` bridge (documented above). ### Anti-Pattern 3: Polling Playback Position from TUI **What:** TUI widget calling `time.time()` on every render to calculate playback position **Why bad:** Playback position should be authoritative state in App Core. Multiple widgets reading independently diverge. If audio playback pauses, widgets don't know. **Instead:** App Core maintains `current_position_s` as reactive state. TUI subscribes to updates. ### Anti-Pattern 4: JSON Parsing on ESP32 ISR **What:** Parsing received UDP JSON inside the WiFi receive interrupt handler **Why bad:** JSON parsing is slow (malloc, string iteration) and will cause watchdog resets from ISR **Instead:** WiFi ISR copies raw bytes to a FreeRTOS queue. Separate task drains queue and parses JSON with enough stack space. --- ## Scalability Considerations This is a single-user, single-ESP32 system by design. Scalability means: | Concern | Current | Future if needed | |---------|---------|-----------------| | More LED strips | Add second ESP32, Pi sends duplicate commands | Multi-target transport layer | | More animations | Add to ESP32 firmware + manifest file | Plugin architecture on ESP32 | | Longer songs | JSON file size grows linearly — no issue | No change needed | | Multiple choreographies | File system already handles this | Directory browser UI | --- ## Sources - [WLED Project Documentation](https://kno.wled.ge/) — Industry reference for ESP32 LED control architecture, UDP protocol patterns - [WLED Control Interfaces (DeepWiki)](https://deepwiki.com/wled/WLED-Docs/6-control-interfaces) — Protocol specifics (E1.31, DDP, UDP) - [ESP32-C3 Technical Reference Manual](https://www.espressif.com/sites/default/files/documentation/esp32-c3_technical_reference_manual_en.pdf) — RMT peripheral, SPI, single-core constraints - [Getting Started with ESP32-C3 Super Mini](https://randomnerdtutorials.com/getting-started-esp32-c3-super-mini/) — Hardware specs (400KB RAM, 4MB Flash, 160MHz) - [Textual Workers Guide](https://textual.textualize.io/guide/workers/) — Background task architecture, thread safety - [Textual Events and Messages](https://textual.textualize.io/guide/events/) — Async message passing patterns - [Real-Time Beat Tracking (TISMIR 2024)](https://transactions.ismir.net/articles/10.5334/tismir.189) — Latency sources in beat tracking pipelines - [dancyPi: Audio-reactive LEDs on RPi + ESP](https://github.com/naztronaut/dancyPi-audio-reactive-led) — Reference architecture for similar system - [ESP32 UDP Latency Discussion](https://github.com/espressif/arduino-esp32/issues/1283) — UDP vs TCP latency on ESP32 - [aubio Real-Time Beat Prediction](https://www.maxhaesslein.de/notes/real-time-beat-prediction-with-aubio/) — aubio integration pattern for live beat detection