← runs · all docs · view source on github

double-acts

Eval-driven generative loop for sketch comedy in the style of BBC Radio 4's John Finnemore's Double Acts. Generate concept → full sketch → audio, then score it against a critic that's been calibrated to tell real Finnemore apart from imitations.

uv sync
echo "GEMINI_API_KEY=..." > .env

uv run pipeline   # ingest 12 MP3s → extract → merge → calibrate autorater → analytics
uv run generate   # generate concept → full sketch → audio → score → iterate

Both binaries use absl.flags. Pass --help for options. --budget_usd N caps API spend.

How it works

Imagine you're trying to learn how to write comedy like John Finnemore. You'd listen to every episode, take notes on what makes them work, then try to write your own — show them to a tough critic, take the feedback, try again, keep going until they sound right. That's the project. The computer just does it really fast and never gets tired.

                    ╔═══════════════╗
                    ║   12 BBC MP3s ║   the gold standard
                    ╚═══════╤═══════╝
                            │
              ┌─────────────┴─────────────┐
              ▼                           ▼
       ┌────────────┐              ┌────────────┐
       │  STUDY     │              │  CALIBRATE │
       │ the corpus │─────────────▶│ the critic │
       └────────────┘              └─────┬──────┘
       transcript, cast,             does it score
       scenes, beats, etc.            real Finnemore
                                      higher than fakes?
                            ▼
                    "the critic is ready"
                            │
                            ▼
                    ╔═══════════════╗
                    ║  YOUR SEED    ║   one-line idea
                    ╚═══════╤═══════╝
                            │
                            ▼
                    ┌──────────────┐
                    │   WRITE      │  concept → full transcript → audio
                    └──────┬───────┘
                            │
                            ▼
                    ┌──────────────┐
                    │   CRITIQUE   │  the critic reads (and listens)
                    └──────┬───────┘
                            │
                       happy? ─── yes ──▶ ship it
                            │
                            no
                            │
                            ▼
                    ┌──────────────┐
                    │   REVISE     │  feedback folded into next attempt
                    └──────┬───────┘
                            │
                            └─▶ back to WRITE  (the hill-climb)

The six steps in plain English

STEP 1 — Ingest. Download the 12 real Double Acts episodes — the gold standard.

STEP 2 — Analyse. Per episode, ask Gemini ten questions while it listens to the audio (transcript, cast, scenes, beats…). Reconcile the ten answers into one canonical record.

STEP 3 — Evaluate. Build a critic with three layers: 30 deterministic lint rules, an 11-axis Gemini judge, and a Claude cross-model second opinion. Calibrate on real episodes vs deliberate fakes — the critic ships only when the corpus separates from fakes by ≥2 points (current: 4.65). Per-episode breakdowns (cast-lead share + beginning/middle/end thirds) run on both corpus and generated episodes.

STEP 4 — Generate. From a one-line seed: write the concept (cast, premise, scenes, beats), then expand into a full sketch one scene at a time with running state. Perform the sketch: Gemini 3.1 Flash TTS for voices, Lyria for music, ElevenLabs for SFX. The mixer sidechain-ducks music during dialogue and prepends a leading-silence gap so voices aren't drowned at the start.

STEP 5 — Critique. Text autorater scores the script. Audio judge listens to the mix and scores five performance axes, backed by deterministic instruments: STT roundtrip, voice distinctness, chunk-seam check, pacing, and a 4-layer voice-consistency check (pitch + speaker embeddings + Gemini diarization + Chirp 3 cross-check). The most important lessons get written to docs/lessons.md and folded into every future generation's prompt.

STEP 6 — Iterate. Steps 4–5 in a loop. Acceptance gate: 0.6·text + 0.4·audio must beat the current best. Five ways to produce the next attempt: selection-from-N, crossbreed top-2 concepts, tree search at scene boundaries, axis edit-mode (surgical patch on the weakest concept axis), scene edit-mode (regenerate just the weakest scene). Plus a listener panel writing in-voice notes and an adversarial red/green/refactor critic that attacks the script before patching it. A separate audio-only panel_loop mode polishes a finished episode by render → panel critique → patch delivery_notes → re-render, rolling back if the audio score regresses. Peaks get logged in docs/BEST_CANDIDATES.md.

The whole thing is eval-driven: we don't trust the LLM to judge its own output, we trust the calibrated critic — and the critic only grades because we proved it can tell Finnemore from not-Finnemore on the corpus first.

Surgical fine-tuning — hill-climb at the line level

Selection-from-N and tree search make whole new attempts. Once the system picks a winner, we narrow to per-line edits to fine-tune toward quality. Three modes via edit-episode:

--rerender re-renders only the affected TTS chunks (surgical re-render, ~80% cheaper than a full re-synth) and remixes with current ducking + leading-silence logic. The original episode.json is auto-backed up; the diff is saved as edit_diff.md next to it. Then A/B compare the edited version against the prior best to confirm the lift.

Quick start

uv sync                                                       # install
uv run pipeline                                               # one-time corpus + autorater setup (~$3, ~30 min)
uv run generate --mode=meta --seed="<your one-line seed>"     # generate a full episode
uv run generate --mode=panel_loop --episode_json=<path>       # polish audio on an existing episode

--budget_usd N caps spend on any subcommand.

For every other recipe (iterate-only, audio-only, scene-edit, crossbreed, overnight, remix, A/B, …), see CLAUDE.md.

Caveats

License

MIT.