Audio
ai podcast
audio summary

Podcast Generation

AI-generated podcast-style audio from text scripts or document inputs. NotebookLM-clone workflows combine TTS + dialogue generation.

Capability notes

End-to-end AI podcast generation chains multiple models into a single pipeline: script generation → speech synthesis (multiple TTS voices) → sound design → audio mastering — producing a complete, listenable podcast episode from a topic prompt or document. Unlike single-model tasks ([text-to-speech](/tasks/text-to-speech), [music-generation](/tasks/music-generation)), podcast generation is purely a pipeline orchestration problem — the quality depends on how well 4-6 separate models are integrated. The reference stack in 2026: **Script generation**: [Llama 3.3 70B](/models/llama-3-3-70b) or [DeepSeek V4](/models/deepseek-v4) via [Ollama](/tools/ollama) or [vLLM](/tools/vllm) — generates structured dialogue for 2-3 speakers with natural turn-taking, banter, and topic progression. Quality: passes as a human-written-but-slightly-stiff podcast script for most topics when given source material (article PDF, research paper, news transcript). Without source material (topic prompt only), outputs generic, surface-level content — the LLM knows the format but not fresh details. **Multi-voice TTS**: [XTTS-v2](/tools/xtts-v2) or [F5-TTS](/tools/f5-tts) for voice cloning, plus [Kokoro](/tools/kokoro) for pre-defined high-quality voices. The production pattern: generate each speaker's voice with a distinct TTS model (Host = Kokoro voice "Bella," Co-host = XTTS-v2 clone of a real podcaster, Guest = F5-TTS with cloned voice from source material). Voice differentiation is critical — if all speakers use the same TTS engine with different prompts, they sound like the same person with different accent settings. Cross-engine voice assignment (each character uses a different TTS engine) provides the most natural speaker differentiation. **Sound design**: [MusicGen](/tasks/music-generation) for intro/outro music and segment transitions, [AudioGen](/tools/audiocraft) for sound effects (applause, scene transitions, button clicks). The production pattern: generate 30s intro music → generate 5s transition sounds between segments → duck music under speech (sidechain compression, -18dB during speech) → layer ambient bed (MusicGen at 10-15% volume throughout).

If you just want to try this

Lowest-friction path to a working setup.

Start by generating a 5-minute two-speaker podcast from a short article. This is a weekend project using individual tools, not an integrated pipeline — you'll stitch components together manually to understand each step. Step 1 — Script: copy-paste a blog post (500-1,000 words) into [Ollama](/tools/ollama) with [Llama 3.3 70B](/models/llama-3-3-70b) or any 30B+ model. Prompt: "Convert this article into a 5-minute podcast script with two speakers, Alex (host) and Jordan (co-host). Include natural dialogue, turn-taking, and a brief intro/outro. Alex does most of the talking; Jordan adds commentary and asks questions. Format: ALEX: [dialogue] JORDAN: [dialogue]." Step 2 — Voice generation: assign Alex = Kokoro voice (pre-defined, no cloning needed) via [TTS Generation WebUI](/tools/tts-generation-webui). Assign Jordan = XTTS-v2 with a voice sample from a podcast you like (6-10s clip of a single speaker). Generate each speaker's lines separately — you now have individual WAV files for each turn. Step 3 — Music: generate 30s intro/outro music with MusicGen (medium model, prompt: "upbeat podcast intro music, modern electronic, clean production"). Trim to 15s intro + 10s outro. Step 4 — Assembly: use Audacity (free, GUI) or FFmpeg (CLI): arrange speaker turns in order, insert intro/outro music, duck music under speech, export as MP3. Audacity is easier for manual arrangement; FFmpeg for scripting. Total time for first 5-minute podcast: 2-4 hours (mostly learning the tools). Hardware: [RTX 3060 12GB](/hardware/rtx-3060-12gb) runs every step. Script: 10-30s via Ollama. TTS: 1-5 minutes for all dialogue. Music: 30-60s. Assembly: 15-30 minutes (manual). This is a learning exercise — you'll understand the pipeline's failure modes, which is the real value for your first attempt.

For production deployment

Operator-grade recommendation.

Production podcast generation is for content-at-scale use cases: converting a publishing backlog into audio, generating daily news podcasts, producing multi-language versions of the same podcast, or creating personalized audio content (your daily briefing, your newsletter read aloud). **Full pipeline architecture**: (1) Content ingestion → source preprocessing (article PDF/text, RSS feed, research paper, internal document) → (2) Script generation via LLM ([vLLM](/tools/vllm) or [Ollama](/tools/ollama) with 70B+ model) with structured output (JSON: array of {speaker_id, text, emotion, pacing_notes}) → (3) Script QA (factuality check: verify LLM claims against source document via RAG retrieval, toxicity check, length check) → (4) Per-speaker voice selection (speaker library maintained in [pgvector](/tools/pgvector) with voice embeddings) → (5) Per-line TTS generation (parallel across GPUs where possible) → (6) Sound design by segment type (intro = music + voice, body = voice + ambient, outro = music + voice + call-to-action) → (7) Audio assembly + mastering (FFmpeg: concatenation, crossfades, loudness normalization) → (8) Metadata + distribution. **Throughput**: [RTX 4090 24GB](/hardware/rtx-4090): ~4-8 fifteen-minute podcast episodes per hour (pipeline stages pipelined). For a daily 15-minute news podcast: ~15-30 minutes of compute per episode. Monthly: 30 episodes × 20 minutes = 10 GPU-hours. Cost: $5-20/month in GPU compute. This is 10-50× cheaper than hiring a podcast producer at $200-800/month for a weekly show. **Personalization at scale**: The killer feature of AI podcast generation. Generate "Alice's Daily Tech Briefing" personalized to Alice's interests (from her reading history) with her preferred voices (she chose "British male host + American female co-host" in preferences). Generate "Bob's Portfolio Update" with growth/shrink of his specific holdings delivered in his preferred podcast format. This level of personalization is impossible with human-produced podcasts. Self-hosted generation is required for privacy — Alice's reading history and Bob's portfolio must never leave local infrastructure. **Multi-language podcast generation**: Same-as-above pipeline, with translation layer inserted after script generation. Generate English script → translate via LLM to 5-10 languages → per-language TTS with native-speaker voices → per-language music (optional, or use same music bed). A 15-minute English podcast becomes versions in Spanish, French, German, Japanese, Mandarin with ~50% added pipeline time per extra language (translation + TTS generation parallelizable). For global organizations communicating to multilingual teams: one content piece → auto-distributed as localized podcast.

What breaks

Failure modes operators see in the wild.

**Monotonous dialogue (Flat Stan and Flat Sarah).** Symptom: the two AI hosts speak with identical pacing, vocabulary, sentence structure, and emotional range — 15 minutes of two bots politely agreeing. Cause: the LLM generates dialogue from its neutral-default tone; TTS renders with neutral-default prosody. Mitigation: inject per-speaker personality profiles in the LLM prompt ("Alex is energetic, uses short sentences, interrupts occasionally. Jordan is thoughtful, uses analogies, asks probing questions.") and per-speaker emotion conditioning in TTS. Test for personality distinctiveness before full generation. **Factual drift from source material.** Symptom: the podcast script contains claims, statistics, and quotes not present in the source article. Cause: LLMs generate plausible-sounding content that's semantically related but factually distinct from the source. Mitigation: RAG factuality check — after script generation, retrieve relevant passages from source document for each claim, verify alignment via a second LLM call ("does the script claim match the source passage?"). Flag for human review where alignment <0.8. For news content: this is mandatory; for creative fiction: optional. **Voice bleed (same person, different names).** Symptom: Alex and Jordan are supposed to be different people, but listeners can tell it's the same TTS engine with different settings — timbre sounds identical, just pitch-shifted. Cause: using the same TTS engine for all speakers with parameter variation only (pitch +0.2, speed -0.1). Mitigation: assign different TTS engines per speaker. Kokoro for Speaker A, XTTS-v2 for Speaker B, F5-TTS for Speaker C. Cross-engine voice assignment is the strongest signal of distinct speakers — the engines have different "sound signatures" that listeners subconsciously use to differentiate. **Rushed turn-taking (overlapping speech artifacts).** Symptom: one speaker ends, the next begins in 100ms — natural human conversation has 200-500ms gaps. The podcast feels anxious, rushed, inhuman. Mitigation: insert silence gaps between turns: 200ms for quick back-and-forth, 400ms for normal conversation, 800ms for topic transitions. This is an audio editing parameter, not an AI parameter — easily fixed in post but easily missed by pipeline builders.

Hardware guidance

**Minimum viable (8-12 GB VRAM)**: [RTX 3060 12GB](/hardware/rtx-3060-12gb) or [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb). Runs each pipeline stage sequentially: script LLM (~8-10 GB at Q4 for Llama 3.3 70B), TTS (XTTS-v2 ~4 GB, Kokoro ~2 GB), MusicGen (~8 GB). A 15-minute podcast in 15-30 minutes (0.5-1× real-time). Viable for one-off or daily single-episode generation. **SMB/production (16-24 GB)**: [RTX 4070 Ti Super 16GB](/hardware/rtx-4070-ti-super) or [RTX 4090 24GB](/hardware/rtx-4090). Pipeline stages pipelined on one GPU: LLM + TTS share GPU with context switching, MusicGen runs separately. Or: LLM on [RTX 4090](/hardware/rtx-4090), TTS + MusicGen on [RTX 4070](/hardware/rtx-4070) — two-GPU setup gives 2-3× throughput of single-GPU sequential. 15-min episode in 5-10 minutes (1.5-3× real-time). **Content farm/studio (24-48 GB)**: [RTX 5090 32GB](/hardware/rtx-5090) or [NVIDIA L40S](/hardware/nvidia-l40s) 48 GB. Concurrent episodes on one GPU: 2-3 fifteen-minute episodes simultaneously. For daily multi-topic podcast feed (e.g., "5 topics, each becomes a 10-min episode" = 50 min of content/day): single RTX 5090 handles in ~30-60 minutes. **Multi-GPU scaling**: Audio pipeline stages are independent — run LLM (script) on GPU 1, TTS (voice generation) on GPUs 2-4, MusicGen on GPU 5. Decouple stages with a queue: GPU 1 writes scripts to queue → GPUs 2-4 consume scripts, generate voice files → GPU 5 generates music → CPU assembles + masters. This pipelined architecture processes 5-10× more content than all-stages-on-one-GPU. **Memory scaling per stage**: LLM (70B Q4, 30B-class for script gen): 35-45 GB or 16-20 GB depending on model. XTTS-v2: 2-4 GB. F5-TTS: 4-8 GB. Kokoro: 1-2 GB. MusicGen medium: 6-8 GB. Total concurrent: 50-70 GB — multi-GPU is the practical deployment for concurrent pipelines.

Runtime guidance

**If building a podcast generation pipeline from scratch** → Python orchestration with per-stage tool APIs. [Ollama](/tools/ollama) or [vLLM](/tools/vllm) for LLM script generation (prompt → structured JSON dialogue). [XTTS-v2](/tools/xtts-v2) Python API for per-line TTS (`tts.tts_to_file(text=line["text"], speaker_wav=speaker_audio, file_path=f"line_{i}.wav", language="en")`). MusicGen Python API for intro/outro/transition music. FFmpeg for assembly: `ffmpeg -i intro.wav -i "concat:line1.wav|line2.wav|..." -i outro.wav -filter_complex "[0][1][2]concat=n=3:v=0:a=1,aloudnorm=I=-16:LRA=11:TP=-1.5" output.mp3`. **If you want a visual workflow (less coding)** → [ComfyUI](/tools/comfyui) handles the TTS and music portions via ComfyUI-AudioCraft and ComfyUI-TTS nodes. Script generation is external (LLM via Ollama). ComfyUI workflow: Text inputs (dialogue lines) → TTS node per speaker → Audio Mixer nodes (volume, ducking, transitions) → Audio Output. ComfyUI isn't designed for podcast assembly but its audio nodes are functional for shorter pieces (<5 min). For longer episodes, Python + FFmpeg is more reliable. **If you want near-one-click generation** → there is no local equivalent to [NotebookLM](https://notebooklm.google)'s one-click AI podcast in 2026. The local AI path requires scripting or manual tool chaining. The closest option: a Python wrapper that accepts a source URL/text → runs the full pipeline → outputs an MP3. Build once, reuse. The open-source project "podcastfy" (GitHub) provides a partial implementation — script generation + TTS, but manual assembly. It's a starting point, not a drop-in replacement for NotebookLM. **Integration with content management systems**: Connect RSS feed → daily podcast pipeline. Cron job: morning: fetch new articles from RSS → LLM generates 5-minute podcast per article → TTS + music → auto-publish to podcast host (Anchor/Spotify API, Transistor FM API). This creates a daily, fully automated podcast feed from existing written content. For news publishers: auto-convert the daily news articles into a 15-minute audio roundup, available by 6 AM. The pipeline runs overnight on the publisher's hardware — no cloud dependency, no content leaving the organization.

Setup walkthrough

  1. Install components:
    • pip install kokoro-onnx (TTS — fast, CPU-friendly)
    • ollama pull llama3.1:8b (script generation)
  2. Pipeline: LLM generates a dialogue script → each speaker's lines synthesized with different Kokoro voices → audio segments concatenated with slight gaps.
import ollama, soundfile as sf
from kokoro_onnx import Kokoro

kokoro = Kokoro("kokoro-v0_19.onnx", "voices.json")

# Step 1: Generate dialogue script
resp = ollama.chat(model="llama3.1:8b", messages=[{
    "role": "user",
    "content": "Write a 2-person podcast script about local AI: host 'Sarah' and guest 'Mike'. Topic: running AI models on your own computer. 3-minute script. Format: SARAH: <text>, MIKE: <text>."
}])
script = resp["message"]["content"]

# Step 2: Parse and synthesize each line
audio_segments = []
for line in script.split("\n"):
    if line.startswith("SARAH:"):
        samples, sr = kokoro.create(line[7:], voice="af_sarah")
    elif line.startswith("MIKE:"):
        samples, sr = kokoro.create(line[7:], voice="am_michael")
    audio_segments.append(samples)

# Step 3: Concatenate with 0.5s silence gaps
import numpy as np
silence = np.zeros(int(0.5 * sr))
final = np.concatenate([np.concatenate([seg, silence]) for seg in audio_segments])
sf.write("podcast.wav", final, sr)
  1. A 3-minute podcast generates in 1-3 minutes on CPU.

The cheap setup

Podcast generation is CPU-friendly. Kokoro TTS + Llama 3.2 3B (script writing) runs entirely on CPU. A $300 laptop generates a 10-minute podcast in 5-10 minutes. The LLM script generation is the bottleneck — 3B models write slower but for structured scripts (host+guest dialogue), quality is adequate. For higher-quality scripts: add a used GTX 1060 6 GB ($60) for the LLM at 40-60 tok/s. Total: ~$360. Podcast generation at $300-400 produces credible AI podcasts with distinct voices, natural pacing, and coherent dialogue. The output is good enough for internal communications, educational content, and hobby projects.

The serious setup

Used RTX 3060 12 GB (~$200-250, see /hardware/rtx-3060-12gb). Runs Llama 3.1 8B for high-quality script writing at 50-80 tok/s + Kokoro TTS for voice synthesis. A 30-minute podcast generates in 5-10 minutes. For production podcast pipelines (daily AI-generated news summaries, internal comms): batch generate overnight — 8 hours = ~50 episodes. Total build: ~$700-900. For voice variety: Kokoro has ~20 preset voices. For cloned voices (CEO/host voice cloning), F5-TTS adds VRAM overhead (+4-6 GB) but yields personalized podcasts.

Common beginner mistake

The mistake: Generating a 20-minute AI podcast script, running it through TTS with default settings, and publishing without listening to the whole thing. Why it fails: AI-generated scripts hallucinate facts and statistics. TTS mispronounces technical terms and names (Kokoro reads "Qwen" as "kwen," "Ollama" as "oh-la-ma"). A 20-minute podcast with 5 factual errors and 10 mispronunciations destroys credibility instantly. The fix: Always do a human listen-through before publishing. Fact-check every claim the LLM makes. Add pronunciation guides to the script: spell "Qwen" as "Queen" and "Ollama" as "Oh-la-ma." Edit the script for natural speech — written text ≠ spoken text. AI generates the first draft; human polish makes it publishable. An AI-generated podcast with obvious AI errors gets one thing: ignored.

Recommended setup for podcast generation

Recommended runtimes

Browse all tools for runtimes that fit this workload.

Reality check

Audio models are surprisingly forgiving on hardware. Whisper, Coqui, OpenAI Whisper-cpp all run well on 8-12 GB GPUs. The bottleneck is rarely the GPU; it's audio preprocessing and disk I/O for batch transcription.

Common mistakes

  • Overspending on GPU for audio-only workflows (8-12 GB is enough for Whisper)
  • Running audio + LLM concurrently without budgeting VRAM
  • Using fp32 weights when fp16 / int8 give 2-3x speedup with no quality loss
  • Forgetting audio preprocessing eats CPU cycles — a fast SSD helps more than expected

What breaks first

The errors most operators hit when running podcast generation locally. Each links to a diagnose+fix walkthrough.

Before you buy

Verify your specific hardware can handle podcast generation before committing money.

Hardware buying guidance for Podcast Generation

Voice cloning, TTS, and audio generation models trade VRAM for output quality — most operators undersize here.

Specialized buyer guides
Updated 2026 roundup