A Truly Free, API-First AI Video Pipeline

Script generation เขียนสคริปต์อัตโนมัติ脚本自动生成

The free-tier Gemini 2.5 Flash is more than capable for blueprint generation — JSON mode is solid, latency is under three seconds, and the daily quota is generous. Grab a free key from aistudio.google.com/apikey and put it in env.txt.txt as google-api:AIza....

# pip install google-genai
from google import genai
from google.genai import types
import json

client = genai.Client(api_key=GEMINI_KEY)

SYSTEM = """You are a senior video director. Output JSON only:
{
  "anchor_prompt":  string,
  "scene_prompts":  [string],
  "motion_prompts": [string],
  "narration":      [string],
  "music_prompt":   string,
  "voice_style":    string
}"""

resp = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Topic: a Lanna farmer at dawn launching khom loi lanterns. 6 scenes.",
    config=types.GenerateContentConfig(
        system_instruction=SYSTEM,
        response_mime_type="application/json",
        temperature=0.85,
    ),
)
blueprint = json.loads(resp.text)

Anchor image ภาพต้นแบบของตัวละคร角色锚定图

Pollinations runs a public FLUX endpoint that returns clean PNGs over a single HTTP GET — no API key, no signup. URL-encode the prompt, fix a seed for repeatability, and the image bytes come back in 5–15 seconds.

import requests, hashlib
from urllib.parse import quote
from pathlib import Path

def seed_from(text):
    return int(hashlib.sha256(text.encode()).hexdigest()[:8], 16) % 2**31

prompt = blueprint["anchor_prompt"] + ", cinematic portrait, 16:9, hyperreal lighting"
url = (
    f"https://image.pollinations.ai/prompt/{quote(prompt)}"
    f"?model=flux&width=1280&height=720&seed={seed_from(prompt)}"
    f"&nologo=true&enhance=false"
)
r = requests.get(url, timeout=180)
Path("out/anchor.png").write_bytes(r.content)

The result is a 1280×720 FLUX render — visually close to Nano Banana 2, slightly less character-locked. Regenerate (vary the seed) until you have a portrait you like. This single file is the character contract for every storyboard scene.

Storyboard stills (serial) ภาพแต่ละฉาก分镜批量生成

Loop the scene prompts. Repeat the anchor description in every prompt — Pollinations doesn't expose a true image-conditioning endpoint anonymously, so character coherence comes from prompt repetition plus deterministic seeding. Keep concurrency at 1: the anonymous endpoint returns 429 on parallel requests.

import asyncio, time

async def render_stills(blueprint, out_dir):
    sem = asyncio.Semaphore(1)             # serial — Pollinations 429s on parallel
    anchor_desc = blueprint["anchor_prompt"]

    async def one(i, scene_prompt):
        async with sem:
            full = f"{scene_prompt}, featuring: {anchor_desc}, 16:9 cinematic, matching character"
            await asyncio.to_thread(
                gen_still, full, seed_from(anchor_desc) + i,
                out_dir / f"scene_{i:02d}.png",
            )

    async with asyncio.TaskGroup() as tg:
        for i, p in enumerate(blueprint["scene_prompts"]):
            tg.create_task(one(i, p))

Narration เสียงพากย์配音生成

The edge-tts Python package talks to the same neural TTS engine that powers Microsoft Edge's "Read Aloud" feature. No API key. No daily cap. 300+ voices, including two Thai ones (th-TH-PremwadeeNeural, th-TH-NiwatNeural). Quality is comparable to Gemini TTS for most narration uses.

# pip install edge-tts
import asyncio, edge_tts

async def tts(text, out_path, voice="en-US-AvaNeural"):
    c = edge_tts.Communicate(text, voice)
    await c.save(str(out_path))

async def render_voice(blueprint, out_dir):
    for i, line in enumerate(blueprint["narration"]):
        await tts(line, out_dir / f"voice_{i:02d}.mp3")

Voice is generated before motion in the free pipeline. The duration of each narration clip becomes the duration of each visual clip — that way the timeline lines up without a separate alignment pass.

Motion — Ken Burns or HF Space ทำให้ภาพเคลื่อนไหว让画面动起来

The default motion backend is ffmpeg's zoompan filter — push, pull, pan-left, pan-right. It runs locally in seconds, never fails, and gives you a real cinematic camera move on each still. For most narrative-over-stills work this is what you want anyway.

def kenburns_clip(src, duration, kb_dir, out, w=1280, h=720, fps=30):
    frames = int(round(duration * fps))
    if kb_dir == "in":
        z = "if(eq(on,0),1.0,zoom+0.0008)"
        x = "iw/2-(iw/zoom/2)"; y = "ih/2-(ih/zoom/2)"
    elif kb_dir == "out":
        z = "if(eq(on,0),1.12,max(1.001,zoom-0.0008))"
        x = "iw/2-(iw/zoom/2)"; y = "ih/2-(ih/zoom/2)"
    # ... left / right omitted for brevity
    vf = (
        f"scale=2560:-1:flags=lanczos,"
        f"zoompan=z='{z}':d={frames}:s={w}x{h}:fps={fps}:x='{x}':y='{y}',"
        f"scale={w}:{h}:force_original_aspect_ratio=decrease,"
        f"pad={w}:{h}:(ow-iw)/2:(oh-ih)/2:black,setsar=1,format=yuv420p"
    )
    subprocess.run([
        "ffmpeg", "-y", "-loglevel", "error",
        "-loop", "1", "-i", str(src),
        "-vf", vf, "-t", f"{duration}", "-r", str(fps),
        "-c:v", "libx264", "-preset", "fast", "-crf", "20",
        "-pix_fmt", "yuv420p", "-an", str(out),
    ], check=True)

If you want real AI subject motion (waving grass, lantern flicker, walking gait) — opt into a free HuggingFace Space. Lightricks/ltx-video-distilled exposes /image_to_video via Gradio, runs on ZeroGPU, and is gated only by your free HF account's daily quota.

# pip install gradio_client    (only needed for --i2v hf)
from gradio_client import Client, handle_file

c = Client("Lightricks/ltx-video-distilled", token=HF_TOKEN, verbose=False)
res = c.predict(
    prompt=motion_prompt,
    negative_prompt="worst quality, blurry, watermark",
    input_image_filepath=handle_file(str(still)),
    input_video_filepath="",
    height_ui=512, width_ui=768,
    mode="image-to-video",
    duration_ui=4.0, ui_frames_to_use=25,
    seed_ui=42, randomize_seed=False,
    ui_guidance_scale=3.0, improve_texture_flag=False,
    api_name="/image_to_video",
)
# res["video"] is a temp filepath — copy to your clips/ dir

Assembly ตัดต่อด้วย ffmpeg用 ffmpeg 合成

One write of clips.txt, one concat-demuxer pass, one mux pass. The concat demuxer copies streams without re-encoding (fast and lossless), and the final mux is a single libx264 / AAC pass at -profile:v high -level 4.0 -pix_fmt yuv420p for guaranteed LINE / mobile-Safari playback.

# 1. Concat video clips
Path("out/clips.txt").write_text(
    "\n".join(f"file '{c.as_posix()}'" for c in clips)
)
subprocess.run([
    "ffmpeg", "-y", "-loglevel", "error",
    "-f", "concat", "-safe", "0", "-i", "out/clips.txt",
    "-c", "copy", "out/concat.mp4",
], check=True)

# 2. Concat narration mp3s the same way
#    (same pattern with voice_NN.mp3 -> voice_concat.mp3)

# 3. Mux video + voice into final.mp4
subprocess.run([
    "ffmpeg", "-y", "-loglevel", "error",
    "-i", "out/concat.mp4", "-i", "out/voice_concat.mp3",
    "-map", "0:v:0", "-map", "1:a:0",
    "-c:v", "libx264", "-preset", "medium", "-crf", "20",
    "-pix_fmt", "yuv420p", "-profile:v", "high", "-level", "4.0",
    "-c:a", "aac", "-b:a", "192k", "-shortest", "-movflags", "+faststart",
    "out/final.mp4",
], check=True)