19 KiB
Architecture Patterns: LED Sync Studio
Domain: Terminal-based LED choreography system (Raspberry Pi + ESP32) Researched: 2026-04-03 Confidence: HIGH for component boundaries and data flow; MEDIUM for exact protocol choices (verified against ESP32 docs and WLED/xLights ecosystem)
System Overview
LED Sync Studio splits cleanly into two physical contexts: the Raspberry Pi runs the user-facing application and all intelligence, while the ESP32-C3 is a dumb executor — it receives animation commands and runs them locally on the LED strips. The Pi never streams raw pixel frames. This is the defining architectural constraint from which everything else follows.
┌─────────────────────────────────────────────────────────────┐
│ Raspberry Pi 4B │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ TUI Layer │ │ App Core │ │ Audio Layer │ │
│ │ (Textual) │◄──►│ (Async) │◄──►│ (PipeWire) │ │
│ └──────────────┘ └──────┬───────┘ └──────────────┘ │
│ │ │
│ ┌──────▼───────┐ │
│ │ Transport │ │
│ │ (UDP/WiFi) │ │
│ └──────┬───────┘ │
└─────────────────────────────┼───────────────────────────────┘
│ JSON commands
┌─────────▼─────────┐
│ ESP32-C3 │
│ │
│ ┌─────────────┐ │
│ │ WiFi Server │ │
│ └──────┬──────┘ │
│ │ │
│ ┌──────▼──────┐ │
│ │ Anim Engine │ │
│ └──────┬──────┘ │
│ │ │
│ ┌──────▼──────┐ │
│ │ LED Drivers │ │
│ │ WS2801(SPI) │ │
│ │ SK6812(RMT) │ │
│ └─────────────┘ │
└───────────────────┘
Component Boundaries
Pi-Side Components
| Component | Responsibility | Communicates With |
|---|---|---|
| TUI Layer | Render all UI views, accept keystrokes, display timeline/event list | App Core (reactive messages) |
| App Core | State machine, playback clock, choreography scheduler | TUI Layer, Audio Layer, Transport |
| Choreography Engine | Load/save JSON files, manage event sequences, calculate playback position | App Core, file system |
| Audio Layer | Song playback (MP3/FLAC/WAV), real-time beat detection from system audio | App Core (beat events) |
| Transport | JSON serialization, UDP/TCP socket to ESP32, connection health | App Core |
ESP32-Side Components
| Component | Responsibility | Communicates With |
|---|---|---|
| WiFi Server | Listen for UDP datagrams or TCP connections, parse JSON | Animation Engine |
| Animation Engine | Maintain animation state, run effect loops, handle parameter updates | WiFi Server, LED Drivers |
| LED Drivers | Hardware-specific output: WS2801 via SPI, SK6812 via RMT peripheral | Hardware (GPIO/SPI) |
| Zone Manager | Route animation commands to correct strip (zone A = WS2801, zone B = SK6812) | Animation Engine, LED Drivers |
Data Flow
Choreography Playback Mode
Song file → Audio decoder → PulseAudio/PipeWire sink
│
elapsed time
│
Playback Clock ──► Choreography Engine
│
scheduled event fires
│
JSON command built
│
UDP datagram → ESP32
│
Animation Engine starts
│
LEDs execute effect
Live Reactive Mode
System audio → PyAudio/sounddevice → Audio buffer
│
FFT / onset detection
│
beat event (aubio)
│
App Core receives event
│
mapped animation selected
│
JSON command → UDP → ESP32
│
LEDs react in near-realtime
User Interaction Flow
SSH terminal → Textual event loop → UI widget action → App Core message
│
state mutation
│
reactive UI update
(no direct DOM)
Choreography File Data Model
Choreography (file: .json / .yaml)
├── metadata
│ ├── song_path: str
│ ├── song_duration_s: float
│ └── bpm: float | null
└── events: [
{
time_s: float, # offset from song start
zone: "ws2801" | "sk6812" | "both",
animation: str, # registered name on ESP32
params: { # animation-specific
color: [r, g, b],
speed: float,
intensity: float,
loop: bool
},
duration_s: float | null # null = run until replaced
}
]
JSON Command Protocol (Pi → ESP32)
{
"zone": "ws2801",
"animation": "pulse",
"params": {
"color": [255, 0, 128],
"speed": 0.8,
"intensity": 1.0
}
}
A separate command handles stop/clear:
{
"zone": "all",
"animation": "off"
}
Recommended Architecture Pattern: Event-Driven Async Core
The App Core is an asyncio event loop (Textual already runs on asyncio). Everything hangs off it:
- Playback clock — asyncio task, fires scheduled events from choreography
- Audio processor — runs in a thread (audio callbacks are synchronous), posts beat events into the asyncio loop via
call_soon_threadsafe - Network sender — async UDP socket (asyncio DatagramProtocol), non-blocking
- File I/O — async file reads (aiofiles) to avoid blocking the TUI
This pattern avoids the thread-safety trap: only one asyncio task ever mutates state, threads post events inward via thread-safe bridges.
Critical Pattern: Beat Detection Threading
Thread: audio callback (sounddevice/PyAudio)
└── aubio onset detector
└── beat detected
└── loop.call_soon_threadsafe(post_beat_event)
└── asyncio coroutine handles it
└── sends UDP command
Audio must run in a thread — audio callbacks cannot be async. This is the one mandatory thread boundary in the system.
Transport Protocol Decision: UDP over TCP
Use UDP for animation commands. Rationale:
- Animation commands are stateless triggers, not streams. A dropped packet means a missed animation start — tolerable. A delayed retransmission (TCP) means a late animation start — worse for musical sync.
- UDP latency on a local WiFi network is typically 1-5ms vs TCP 10-30ms (with Nagle's algorithm; TCP_NODELAY helps but UDP is still lower).
- ESP32-C3's single RISC-V core handles UDP receive with less overhead than full TCP stack maintenance.
- Command payloads are small (< 200 bytes), well under UDP MTU (1470 bytes on typical WiFi).
- WLED (the leading ESP32 LED firmware) uses UDP for all real-time control protocols (E1.31, DDP, Art-Net). This is industry standard for LED sync.
Heartbeat mechanism: Pi sends a UDP heartbeat every 5s; ESP32 detects loss and enters "safe mode" (off or last known state).
Confidence: HIGH — verified against WLED protocol architecture and ESP32 forum discussions on latency optimization.
ESP32-C3 Constraints and Firmware Architecture
The ESP32-C3 SuperMini has:
- 400 KB SRAM (tight)
- 4 MB Flash
- Single RISC-V core at 160 MHz
- No hardware FPU (floating-point operations are slower)
FreeRTOS Task Allocation
Core 0 (only core):
Task 1 (high priority): LED output — drives SPI for WS2801, RMT for SK6812
Task 2 (medium priority): WiFi + JSON receive
Task 3 (low priority): Animation tick — updates frame buffer, feeds to LED task
WiFi and LED output cannot share timing perfectly on one core. The approach: LED task uses FreeRTOS queue to receive new frame buffers from the animation task. Animation task runs at a fixed interval (e.g., 20ms = 50fps) and posts computed frames into the queue. LED task drains the queue and writes to hardware.
LED Driver Architecture
WS2801 (SPI):
- Uses ESP-IDF SPI master driver
- Clock + Data pins, speed ~1 MHz
- Blocking write per frame is acceptable (160 LEDs * 3 bytes = 480 bytes, fast at 1 MHz)
SK6812 (single-wire):
- Uses ESP-IDF RMT (Remote Control Transceiver) peripheral
- RMT generates precise timing pulses without CPU involvement
- Non-blocking: CPU queues frame data, RMT transmits via DMA
- SK6812 is RGBW (4 bytes/LED) → 300 LEDs * 4 bytes = 1200 bytes per frame
Both strips can update concurrently: SPI is blocking but fast, RMT is DMA-driven. Sequence: start RMT (non-blocking), write SPI (blocking), both finish in roughly the same window.
Textual TUI Architecture
Textual uses a reactive widget tree with message passing. For this app:
App (root)
├── Header (song info, BPM, mode)
├── TransportBar (play/pause/stop, position scrubber)
├── MainContent (switches between views)
│ ├── TimelineView (horizontal scrollable canvas)
│ │ ├── TimeRuler
│ │ ├── ZoneTrack (WS2801)
│ │ └── ZoneTrack (SK6812)
│ └── EventListView (DataTable of events)
├── AnimationPanel (sidebar: select animation, set params)
└── StatusBar (ESP32 connection, audio status, confidence)
Reactive data binding: Textual's reactive descriptor triggers re-renders when playback position changes. The playback clock posts position updates at ~10 Hz, which drives the timeline cursor without flooding the event loop.
Timeline rendering: Custom Textual Canvas or Rich markup — render events as colored blocks at proportional x positions. Horizontal scroll follows playback cursor.
Suggested Build Order (Dependencies)
The dependency graph drives this order:
1. ESP32 Firmware (foundation — everything depends on having a working LED executor)
└── LED drivers (WS2801 SPI, SK6812 RMT)
└── Animation engine (effect registry, parameter system)
└── WiFi UDP server + JSON parser
2. Transport Layer (Pi side — needed to test firmware without UI)
└── UDP socket client
└── JSON command builder
└── Connection management + heartbeat
3. App Core + State (no UI yet — testable via CLI)
└── Choreography data model (JSON load/save)
└── Playback clock
└── Event scheduler
4. Audio Layer (can be developed in parallel with App Core after firmware done)
└── Song playback (pygame/miniaudio)
└── Beat detection (aubio + sounddevice)
└── Audio→App Core event bridge
5. TUI Layer (depends on App Core having stable API)
└── Transport controls + status
└── Event list view
└── Timeline view (most complex — build last within TUI)
└── Animation parameter panel
6. Integration + Polish
└── Cyberpunk visual theme
└── Choreography file management (browse, rename, delete)
└── Live reactive mode (ties Audio Layer to Animation dispatch)
Why firmware first: Every other component is validated against a real ESP32. Developing the protocol without firmware means writing a mock, then potentially changing everything when the real hardware behaves differently. Ship firmware first, test with curl/netcat, then build the Pi application around proven commands.
Why Audio Layer parallel to App Core: Beat detection and song playback have no dependency on the UI. They can be prototyped as standalone scripts, then integrated.
Cross-Cutting Concerns
Latency Budget for Musical Sync
Musical sync at 120 BPM has 500ms between beats. A cue that fires 50ms early or late is perceptible. Target latency budget:
| Stage | Budget | Mechanism |
|---|---|---|
| Playback clock tick accuracy | < 10ms | asyncio event loop, avoid blocking |
| JSON serialization | < 1ms | ujson (faster than stdlib json) |
| WiFi UDP transit | < 10ms | local network, UDP |
| ESP32 receive to animation start | < 5ms | FreeRTOS high-priority LED task |
| Total | < 26ms | Well within perceptible threshold |
Zone Independence
WS2801 (cabinet, 160 LEDs) and SK6812 (wall, 300 LEDs) must be independently addressable. The JSON protocol always specifies a zone. The ESP32 Zone Manager routes to the correct driver. Both zones can run different animations simultaneously — this is a first-class requirement, not an afterthought.
Animation Registry
Animations are named strings registered in firmware. The Pi does not know animation internals — it only sends a name + parameters. This keeps the protocol stable: new animations are added to firmware without changing the Pi application, as long as the parameter schema is consistent.
Define a shared animation manifest (JSON file on Pi, #define registry in firmware) so the TUI can offer valid animation names and parameters without hardcoding them in two places.
Anti-Patterns to Avoid
Anti-Pattern 1: Frame Streaming from Pi to ESP32
What: Sending raw pixel arrays (1603 + 3004 = 1680 bytes) at 50fps = 84 KB/s continuously over WiFi UDP Why bad: Saturates single-core ESP32 receive pipeline. WiFi on ESP32-C3 has practical throughput around 1-2 Mbps but latency spikes under load. 84 KB/s is feasible but leaves no headroom for WiFi overhead. Any retransmit or congestion causes visible glitch. Also burns CPU on both ends doing serialization. Instead: Commands only. The project already decided this correctly — ESP32 runs animations locally.
Anti-Pattern 2: Blocking Audio Processing on Asyncio Loop
What: Running aubio beat detection in an async def handler directly
Why bad: aubio's Python bindings are synchronous C extensions. Blocking the asyncio event loop for even 10ms causes TUI lag and missed events.
Instead: Thread worker with call_soon_threadsafe bridge (documented above).
Anti-Pattern 3: Polling Playback Position from TUI
What: TUI widget calling time.time() on every render to calculate playback position
Why bad: Playback position should be authoritative state in App Core. Multiple widgets reading independently diverge. If audio playback pauses, widgets don't know.
Instead: App Core maintains current_position_s as reactive state. TUI subscribes to updates.
Anti-Pattern 4: JSON Parsing on ESP32 ISR
What: Parsing received UDP JSON inside the WiFi receive interrupt handler Why bad: JSON parsing is slow (malloc, string iteration) and will cause watchdog resets from ISR Instead: WiFi ISR copies raw bytes to a FreeRTOS queue. Separate task drains queue and parses JSON with enough stack space.
Scalability Considerations
This is a single-user, single-ESP32 system by design. Scalability means:
| Concern | Current | Future if needed |
|---|---|---|
| More LED strips | Add second ESP32, Pi sends duplicate commands | Multi-target transport layer |
| More animations | Add to ESP32 firmware + manifest file | Plugin architecture on ESP32 |
| Longer songs | JSON file size grows linearly — no issue | No change needed |
| Multiple choreographies | File system already handles this | Directory browser UI |
Sources
- WLED Project Documentation — Industry reference for ESP32 LED control architecture, UDP protocol patterns
- WLED Control Interfaces (DeepWiki) — Protocol specifics (E1.31, DDP, UDP)
- ESP32-C3 Technical Reference Manual — RMT peripheral, SPI, single-core constraints
- Getting Started with ESP32-C3 Super Mini — Hardware specs (400KB RAM, 4MB Flash, 160MHz)
- Textual Workers Guide — Background task architecture, thread safety
- Textual Events and Messages — Async message passing patterns
- Real-Time Beat Tracking (TISMIR 2024) — Latency sources in beat tracking pipelines
- dancyPi: Audio-reactive LEDs on RPi + ESP — Reference architecture for similar system
- ESP32 UDP Latency Discussion — UDP vs TCP latency on ESP32
- aubio Real-Time Beat Prediction — aubio integration pattern for live beat detection