39 experiments toward a video editor with no servers

Today, ReelForge renders on our servers. That works, but it means your footage travels to us, every export costs us compute, and the editor can only be as fast as an upload. Browsers can now decode video in hardware, run GPU shaders and run small AI models. So we asked: could the whole editor live in a tab?

Rather than start building and discover the hard parts halfway through, we wrote the rule down first: do not skip experiments; each one exposes specific pitfalls. Between May and June 2026 we built 39 standalone apps, each proving — or disproving — one piece.

The architecture we’re aiming for#

Main thread ── React UI, timeline state (no playhead re-renders)
 │
 ├─ RenderWorker ── WebGPU compositor (WGSL)
 │                  frame cache: VRAM textures → RAM bitmaps
 ├─ DecodeWorker ── VideoDecoder + demuxer, byte-range reads from OPFS
 ├─ AudioWorker  ── SharedArrayBuffer ring → AudioWorklet (the master clock)
 ├─ ProxyWorker  ── 720p all-I-frame proxies for instant scrubbing
 └─ AIWorker     ── ONNX on WebGPU: masks, speech, detection

Five principles hold it together:

1. 1

   Local-first

   Media stays on your disk. The Origin Private File System is the I/O layer.
2. 2

   Worker-isolated rendering

   The compositor runs in a dedicated worker and never blocks the UI.
3. 3

   Zero-copy

   A decoded VideoFrame goes straight into a GPU texture without a round trip through the CPU.
4. 4

   Streaming I/O

   Files are read in byte-range chunks; nothing is loaded whole into RAM.
5. 5

   Proxy-first playback

   Scrub low-res proxies; export from the originals.

What the experiments taught us#

Decoding is full of foot-guns#

VideoDecoder.flush() resolves before all the frames have actually arrived. Frames must be fed in decode order but tagged with presentation time. And every VideoFrame must be closed by hand — leak a few at 4K and the GPU process falls over. A dedicated backpressure experiment exists just to prove 4K60 decode doesn’t grow VRAM.

Two demuxers, one verdict#

We built the same seek index with mp4box.js and mediabunny. Verdict: mediabunny for the decode path, mp4box where we need raw byte offsets, which mediabunny doesn’t expose.

The GPU is fast; the bus isn’t#

Drawing frames through Canvas2D at 1080p60 moves roughly half a gigabyte a second across the bus for nothing. importExternalTexture avoids the copy — with the catch that external textures expire every frame, so the bind group has to be rebuilt each time.

React shouldn’t know where the playhead is#

With 500 clips on a timeline, re-rendering on every frame is hopeless. Keeping the playhead in refs, outside React state, was the single biggest performance win of the whole programme.

Crashes are a design input#

A project journal in OPFS replays edits after a crash, because beforeunload is not enough. Web Locks keep one tab as the writer. And when the GPU device is lost, the compositor rebuilds itself from a resource registry — but gives up after five losses in ten seconds rather than flapping forever.

Where “agentic” comes from#

The experiments we’re most excited about aren’t plumbing. They’re the pieces of an editor that edits for you — all running on your device:

• Speech-to-text with Whisper-tiny and Moonshine on WebGPU — weights downloaded once and cached.
• Silence cutting with a ~2 MB voice-activity model and hysteresis so cuts don’t chatter.
• Auto-reframe from 16:9 to 9:16, tracking the subject with a small detector and smoothing the camera path.
• Smart-cut: transcript, audio energy and motion combined to rank the moments most worth keeping.
• Background removal and voice isolation, also on-device.

That’s the business case in one line. When the editor, the AI and the export all run on your machine, a free plan stops being a cost centre — and your footage never has to leave your laptop.