Runway
Text-to-video, image-to-video, and a full editing studio — used by real filmmakers.
You need a 5-second shot of a person opening a door into a bright, unfamiliar room. You do not have a person, a door, or a camera. What you have is a laptop and three paragraphs describing the mood. Three years ago, that was not a production plan. It is now.
Why this tool matters
Runway is the AI video tool that real filmmakers use in production. Its Gen-3 model produces text-to-video and image-to-video clips at a quality that crosses the “professionally shippable” threshold — and its editor bundles dozens of AI-powered tools (inpainting, green-screen removal, motion tracking, camera-path generation, lip-sync) that would otherwise require a pipeline of specialty software.
Where Day 16–2020 tools (ElevenLabs, Otter, Descript) made audio production accessible, Runway is the equivalent leap for video. The practical experience: you write a 2-3 sentence description of a shot, pick a camera movement, wait about ninety seconds, and download a 5 or 10-second clip that looks cinematic. Chain a dozen of these and you have a short film. Add real footage from your phone and use Runway's editor to composite, color-grade, and ship it. That was a multi-week production pipeline for a solo creator until about eighteen months ago; it is now an evening.
For researchers, Runway is useful in a different way: visualizing abstract concepts for lectures (“a DNA molecule unwinding in slow motion,” “a neural network activation pattern as fluid dynamics”), creating illustrative b-roll for academic YouTube, and building explainer content for grant applications and outreach. Its learning curve is steeper than ChatGPT's, but the ceiling is dramatically higher.
Setup
Account: runwayml.com — free tier gives you 125 one-time credits (enough to generate ~25 seconds of video and try the editor). Standard ($15/mo), Pro ($35/mo), and Unlimited ($95/mo) unlock progressively more generation budget and longer clips. Serious users typically need at least Standard.
Bring: a reference image or two. Gen-3's strongest mode is image-to-video — generating motion from a still frame you already like. If you have Midjourney or DALL-E outputs, bring them.
Walkthrough
Step 1: Generate a text-to-video shot
Log in at runwayml.com. Open Gen-3 Alpha. In the prompt box: “A warm café at golden hour, slow dolly-in toward a steaming cup of coffee on a wooden table, shallow depth of field, cinematic.” Pick duration 5 seconds. Click Generate.
Step 2: Try image-to-video with your own reference
Generate a reference image in Midjourney or DALL-E (or upload one of your photos). In Runway, switch to Image to Video. Upload the image. Prompt the motion: “Gentle wind moves the leaves. Subtle camera push-in.” This mode produces the most consistent, controllable results.
Step 3: Use camera controls instead of prose
Runway's newer interface exposes discrete camera controls: Pan, Tilt, Zoom, Orbit, with intensity sliders. Use these rather than trying to describe camera moves in the prompt. The results are dramatically more predictable.
Step 4: Extend or remix a promising clip
When a generation lands, use Extend to add another 5 seconds from the final frame, preserving style and subject. Use Reprompt to generate variations on the same seed. This is how you turn a fluke good generation into a usable sequence.
Step 5: Edit in the Runway studio
Bring clips into Sessions (Runway's video editor). Trim, stack, add transitions, apply color presets. For live-action footage: use Green Screen (AI background removal — no green screen needed) and Motion Tracking to follow subjects. These are the features that would require a full Premiere and After Effects pipeline otherwise.
Step 6: Export in the right format
For YouTube: 1080p, MP4, h.264. For Instagram/TikTok: vertical 1080x1920. For LinkedIn: square 1080x1080. Runway exports all three directly; always match the platform's spec rather than letting it crop on upload.
Your turn
Basic: One cinematic shot, two ways
Pick a single shot you'd love to have: a specific mood, a specific subject, a specific lighting. Generate it two ways in Runway:
- Pure text-to-video from a detailed prompt.
- Image-to-video starting from a still image you generate in Midjourney or upload from your camera roll.
Compare. Note which approach gave you more control and which gave you more surprise. Keep the better one as a reference for future shots.
Advanced: A 30-second shot sequence
Plan a 30-second sequence: five or six shots that tell a tiny story or explore a single idea visually. Storyboard on paper first (one sketch per shot). For each shot, decide: text-to-video, image-to-video, or real footage you'll shoot on your phone.
Generate each shot. Iterate until each one stands on its own — don't move to the next until the current one is passable. Expect to burn a lot of credits.
Import all shots into Runway Sessions. Cut to length, add simple transitions, color-grade for consistency across shots. Add a 30-second music bed from Suno (Day 14) or royalty-free stock.
Export as a 1080p MP4. Share it with two people who know the topic and two who don't. Ask: does it communicate what you intended? Write a 200-word reflection on which shots worked, which didn't, and what you'd change on a second pass.
You just made your first AI-assisted short film.
Pitfalls and pro tips
Hands, text, and faces still betray AI. Gen-3 is good but not yet seamless on close-ups of hands, legible text, or sustained human faces in motion. For your hero shots, shoot real footage on your phone and use Runway for b-roll and establishing shots where imperfection reads as stylization.
Credit math gets expensive fast. Each generation is 5–10 credits. The 125-credit free tier gives you about 12–25 generations total, and maybe half of them will be usable. Generate a short test at 5 seconds before committing credits to a 10-second version.
Continuity breaks between shots. Runway doesn't maintain character or location identity across separate generations. A “woman in a red coat” looks different in shot 1 versus shot 2. For continuity, either shoot real footage, use the Extend feature to stay on the same generation, or embrace the shot-to-shot variation as stylistic.
How it compares
Runway competes with Luma Dream Machine (strong on motion, cheaper), Pika (fun, creative, less cinematic), Kling (covered in Course 1, outstanding on physical realism, slower), and Sora 2 (OpenAI, best overall quality as of this writing but access-restricted and expensive). Runway's edge is the editor: you can generate and edit in one place, and its non-generation tools (Green Screen, Motion Tracking, Inpainting) are category- leading. For a creator shipping video every week, Runway is usually the better platform even when Sora's raw quality is higher.
When to use — and when not to
Use Runway when you need cinematic-quality AI-generated video as part of a real production: short films, branded video, lecture b-roll, explainer content, music-video visuals. Also when you need the editor's AI features (background removal, motion tracking, color grading) on real footage.
Do not use Runway when you need a talking-head training video (HeyGen — Day 17 — is better), when you're converting long-form written content to social clips (Pictory — Day 18 — is faster), or when shooting simple phone video that doesn't need AI generation (Captions — Day 20 — is lighter-weight).