Claude Opus 5.5 Hand-Codes a Full Animated Music Video: Teardown & Open Template

Twitter user @NotinReality recently fed a lyric sheet and audio file to Claude Opus 5.5, instructing it to generate raw JavaScript code that renders a complete, hand-crafted, frame-by-frame animated music video. Zero image-generation or video-diffusion models were used in the pipeline.

This deep dive breaks down the iterative prompt engineering, the multi-agent parallel chapter architecture, and how you can leverage the author's open-source ClaudeAnimationBase template to build your own programmatic animations.

1. Core Architecture & Pipeline Breakdown

Unlike modern AI video workflows that rely on latent diffusion models (e.g., Sora, Runway Gen-3), this project utilizes a deterministic, code-driven rendering pipeline:

[Lyrics & Audio] ──> [Claude Opus 5.5 (xhigh)] ──> [9 Chapter JS Files (p5.js)] 
                                                          │
[FFmpeg MP4 Muxing] <── [Headless Chrome Frame Capture] <─┘
  1. LLM as the Renderer: Claude Opus 5.5 writes pure p5.js code to draw every single frame programmatically.
  2. Headless Execution: The system runs the JavaScript in a headless Chrome instance to capture frames sequentially.
  3. Deterministic Assembly: ffmpeg stitches the PNG frames into an MP4 container, multiplexing the original audio track in sync.

The resulting 156.6-second watercolor-styled music video is titled "I'm Upping My P(doom)"—a tongue-in-cheek reference to the AI safety community's term for existential risk. The visual aesthetic mimics an old-school children's storybook featuring watercolor textures, subtly jittering ink lines, rhythmically synced choreography, and expressive character acting.

The author utilized the original audio track from a 3D-animated music video produced by creator @slimer48484, but built an entirely 2D programmatic counterpart using Claude, with all source code publicly available in the PDoomVideo repository and packaged as ClaudeAnimationBase.

2. Technical Highlights & Iterative Teardown

The entire production cycle required only two primary iterations, with 100% of the code written autonomously by the model.

Iteration 1: The Flash-Style Prototype (Medium Reasoning)

  • Input: Lyrics and audio pasted directly to Claude with a brief prompt to make a "cute animation." Reasoning set to Medium.
  • Output: A baseline vector animation reminiscent of early 2000s Adobe Flash. It featured flat primary color blocks, heavy outlines, and excessive literal UI elements (e.g., floating text tags like "AGI?", "LOSS", "don't eat me!").

Iteration 2: The Watercolor Masterpiece (xhigh Reasoning)

  • Parameters: Reasoning cranked to xhigh; enforced the use of p5.brush (a library that layers organic watercolor and ink textures onto p5.js); mandated richer scene compositions, persistent visual motion, and seamless transitions between every scene.
  • Architectural Shifts: Opus authored a master storyboard (STORYBOARD.md) minimizing redundant on-screen text. Aside from bottom-aligned karaoke subtitles and a literal prop thermometer tracking the P(doom) score, text was restricted to punchy onomatopoeia (FOOM, CHOMP).
  • Execution Time: Approximately 45 minutes of autonomous code generation and refinement.

3. Multi-Agent Chapter Parallelism & Director Workflow

To scale beyond single-file token limits, Claude acted as an autonomous software director executing a structured workflow:

  1. Global Storyboarding (STORYBOARD.md): Opus conceived the video as an unfolding runaway stage play, bounded by opening and closing curtains. The recurring mascot, "Clawd," scales from a small desktop doodle to planetary size, interacting with various AI doom tropes. A physical "P(doom)" thermometer prop is iteratively pumped higher during choruses (8 -> 34 -> 61 -> 86 -> 99.9), resolving with a final meta-commentary line: "Was it all for show?"
  2. Horizontal Chapter Sharding: The track was broken down into 9 distinct chronological chapters, each mapped to an isolated JavaScript file.
  3. Sub-Agent Swarms (ANIMATION_GUIDE.md): Opus generated a strict brief for parallel sub-agents. Each agent was restricted to modifying only its assigned chapter file. If an agent encountered a bug in a shared utility, it was instructed to log an issue rather than refactor globally.
  4. Automated Verification: render.mjs handled headless rendering, allowing the model to visually inspect output frames and self-correct rendering artifacts.

4. Quickstart Guide: Running ClaudeAnimationBase

You don't need to be an experienced front-end developer to build programmatic animations. By pairing this repository with Claude Code (or your preferred coding agent), you can replicate the workflow locally.

Prerequisites & Setup

Ensure you have Node.js and ffmpeg installed globally on your machine:

# Clone the base repository
git clone https://github.com/NotinReality/ClaudeAnimationBase.git
cd ClaudeAnimationBase

# Install dependencies
npm install

Execution Pipeline

  1. Drop your target audio file into the working directory and update the config references.
  2. Use the provided prompt templates in the repo to initialize your storyboard with Claude Opus 5.5 (xhigh reasoning).
  3. Run the headless rendering script:
node render.mjs
  1. Mux the output frames to MP4 via ffmpeg:
ffmpeg -framerate 30 -i frames/frame_%04d.png -i audio.mp3 -c:v libx264 -pix_fmt yuv420p output.mp4

Advanced Extensibility

Advanced engineers (such as @Donald Jewkes) have already forked this paradigm to build hybrid pipelines—using fal.ai for static asset generation and SeaDance for video clips as foundational layers, overlaid and orchestrated entirely via programmatic JS execution managed by Claude over a 12-hour continuous build run.