# Creative Director AI (Part 2): An A2A Music-Video Swarm You Can Actually Observe *How we evolved from our V1 Taskmaster on Google ADK into a specialized Agent-to-Agent production crew—and wired Grafana Cloud so spend, latency, and render failures aren't a black box.* **Author:** G2 (Founder, Creative Organization DAO / Creative Platform) **Series:** Behind the Build — Part 2 | [Read Part 1: Autonomous Music Video Generation via Google ADK](https://blog.creativeplatform.xyz/creative-director) **Live Cost Dashboard:** [Grafana Cost & Spend Overview](https://thecreative.grafana.net/goto/s9bnpl) --- ## The Story So Far: Recapping V1 In Part 1, we introduced the first iteration of Creative Director AI built for Track 1 (The Taskmaster) of the Google Agent Hackathon. In that build, our goal was zero-touch generation: an autonomous director agent built on the Google Agent Development Kit (ADK) in TypeScript that ingested raw music waveforms into Gemini, decomposed the song's BPM and emotional structure, planned scenes, and triggered structured Zod tool calls (`generate_video_cut` via Google Veo, `assemble_and_sync_timeline` via Pixels on Cloud Run, and `sign_c2pa_manifest` for provenance), deployed on Vertex AI Reasoning Engines. That V1 build proved that audio-native intelligence works. But when we pushed it beyond hackathon demos toward real production workflows, we ran into three fundamental bottlenecks: 1. **The Monolithic Director Trap:** A single agent trying to juggle high-level artistic vision, deep narrative scripting, shot-by-shot camera blocking, and timeline assembly quickly gets bloated. Real video productions aren't run by one person pretending to be everyone—they are run by specialized crafts. 2. **Brittle Genre Aesthetics:** V1 relied on three hard-coded style packs (dark-pop, hip-hop, generic). Music is vastly richer than three categories; artists need deep, nuanced visual worlds that honor specific subgenres. 3. **The Operational Black Box:** Video generation isn't text chat—Google Veo compute and multi-step agent reasoning cost real money and take significant render time. When a generation failed at step 3 or cost spiked, we had zero visibility into which tool choked, why latency ballooned, or how much token/render budget was burned. To solve this, we didn't just tweak prompts. We completely re-architected the system for V2: transforming our solo director into an Agent-to-Agent (A2A) Swarm and wiring it to a dedicated Control Room using Grafana Cloud. --- ## The Core Concept: From Solo Director to A2A Swarm Most "AI video" experiments rely on a single prompt or a solo agent trying to do everything at once. In practice: - A Writer shouldn't invent Google Cloud Storage (GCS) URLs. - A DP shouldn't approve a multi-dollar Veo render. - An Editor shouldn't invent clip sequences out of thin air. In V2, we decomposed the creative process into an Agent-to-Agent (A2A) swarm using Google ADK (TypeScript), where each specialist owns a distinct domain contract: | Role | Domain & Responsibilities | | --- | --- | | **Creative Director** | Orchestrates the entire swarm, selects and injects the resolved genre pack, manages state handoffs, and enforces Human-in-the-Loop (HITL) gates before invoking expensive render tools. | | **Writer** | Develops the high-concept treatment, overarching narrative arc, character motifs, and explicitly defines what *not* to show. | | **DP (Director of Photography)** | Translates the treatment and song stems into a beat-synced storyboard, defining camera motion (pan, tilt, dolly) and generating Veo-ready `visual_prompt` specifications. | | **Editor** | Constructs timeline assembly arguments strictly from verified, rendered tool outputs—never hallucinating URLs or guessing cut points. | **Architectural Guardrail:** Render tools (`generate_video_cut` via Veo, `assemble_and_sync_timeline` via Pixels, and `sign_c2pa_manifest` for provenance) remain strictly isolated on the Creative Director. Planning turns never claim a render happened until the human gives the green light. --- ## What We Shipped in V2 ### 1. Dynamic Genre System (Scaling Beyond 3 Hard-Coded Packs) In V1, artists were boxed into dark-pop, hip-hop, or generic. For V2, we engineered a scalable hybrid genre catalog: - **Alias Normalization:** Resolves Apple Music & Spotify-style tags and colloquial names (e.g., R&B / r-and-b / r and b). - **Deep Visual Bibles:** ~25 curated, deep visual style guides formatted in markdown (cinematography rules, lighting, palettes, cultural references). - **Style-Family Templates:** Any genre outside the core bibles dynamically expands from structured style families (urban, electronic, global, rock, ambient, etc.). - **Structured Resolution (`select_genre_pack`):** Returns structured metadata containing `catalogGenre`, source provenance (`deep` | `template` | `generic`), comprehensive pack guidelines, and non-fatal fallback warnings. - **Boundary-Aware Matching:** Word-boundary tokenization prevents matching failures (e.g., preventing "cinematic hip-hop" from mistakenly resolving as classical). Genre routing is treated as rigorous product logic rather than brittle regex. ### 2. Human-in-the-Loop (HITL) Production Pipeline In V1, the Taskmaster attempted zero-touch execution from audio to final file. In V2, we introduced a structured production path that respects human creative direction and budget: 1. **Resolve Genre:** Swarm parses brief and establishes aesthetic constraints. 2. **Writer Treatment:** Narrative structure and creative direction established. 3. **DP Storyboard:** Shot list and beat-synced prompts generated. 4. **HITL Gate:** Execution halts until a human operator reviews and explicitly approves the render. 5. **Render & Assemble:** Automated calls to `generate_video_cut` → `assemble_and_sync_timeline` → `sign_c2pa_manifest`. **Infrastructure:** - **Vertex AI:** Gemini models for agent reasoning; Google Veo for generative video cuts. - **Google Cloud Storage (GCS):** Scalable storage for raw audio stems and rendered cuts. - **Cloud Run:** Headless Pixels video assembly and synchronization engine. - **Google Agent Engine / Runtime:** Managed deployment and agent swarm lifecycle hosting. ### 3. First-Class Observability: The Grafana Control Room Running multi-agent swarms without observability is like flying blind in a storm. For V2, we built observability directly into the swarm loop: - **Grafana Cloud MCP:** The Director agent can directly query Loki, Tempo, and Prometheus tools via Model Context Protocol (MCP) to self-diagnose why a render cut failed or why a sub-agent timed out. - **Grafana Agent Observability (`@grafana/agento11y`):** Every agent turn, reasoning step, and tool invocation is recorded as a first-class distributed trace. - **OpenTelemetry (OTLP):** Native telemetry streaming directly into Grafana Cloud. - **Soft-Fail Fault Tolerance:** MCP calls are non-blocking; an observability hop or logging timeout never crashes the user's creative generation turn. - **Live Demo Dashboard:** Open dashboard tracking real-time agent spend, token consumption, Veo compute costs, and pipeline latency: [Live Cost & Spend Overview Dashboard](https://thecreative.grafana.net/goto/s9bnpl) --- ## Engineering Challenges: What Broke Along the Way Moving from a solo agent to an observable A2A swarm surfaced real production friction: - **ADK Dev UI "Session Not Found":** When restarting local development, sessions dropped state because the internal `App.name` didn't match the route app name (`agent`). - **Cloud MCP Headless Authentication:** ADK's Streamable HTTP transport does not support interactive user OAuth popups; automated execution required configuring dedicated service-account tokens. - **Optional Peer Dependencies (`@modelcontextprotocol/sdk`):** The MCP SDK failed to resolve inside ADK's temporary bundling environment. We resolved this by promoting it to a direct dependency and passing `NODE_PATH` during startup. - **Environment Pollution in Testing:** Agent Observability's internal dotenv override collided with our Grafana unit test suites, requiring sandboxed test environments. - **Inter-Agent Pack Handoffs:** Specialists cannot reliably access shared session memory across turns. We established explicit payload contracts, passing genre packs directly through AgentTool messages. --- ## Key Architectural Takeaways 1. **Contracts Beat Vibes:** Structured Zod schemas and explicit tool parameters prevent agent hallucination. Specialists only receive verified outputs, never fabricated URLs. 2. **HITL Before Burn:** Generative video compute carries non-trivial costs. Human-in-the-loop gates protect creators from accidental spend. 3. **Observability Is Operator UX:** Traces, spend breakdowns, and error forensics turn an unpredictable black-box swarm into a dependable production tool. 4. **Genre Routing Is a Catalog Problem:** Real scalability requires normalized aliases, deterministic tie-breakers, and inheritance templates rather than endless if/else statements. --- ## What's Next for V3 - **Live Catalog Sync:** Direct API integration with Spotify and Apple Music to ingest artist discography and style metadata automatically. - **Remote Cloud Run Specialists:** Decoupling Writer, DP, and Editor into independently scalable microservices on Cloud Run. - **Automated Budget Alerts:** Grafana alerting rules that automatically pause agent swarms if spend thresholds or render failure rates spike. - **On-Chain Distribution:** Connecting final C2PA masters to Pinata ERC-8004 Digital Twins and Creative TV for decentralized broadcast. ## Project Links & Series Navigation - [Part 1: Autonomous Music Video Generation via Google ADK](https://blog.creativeplatform.xyz/creative-director) - Part 2: You are reading Part 2 (A2A Swarm & Observability). - [Live Telemetry: Grafana Cloud Cost & Spend Dashboard](https://thecreative.grafana.net/goto/s9bnpl)