We Dropped ffmpeg.wasm for WebCodecs — Here's What That Actually Cost
📅 July 30, 2026 · ⏱️ 8 min read
Most of the WebCodecs-versus-ffmpeg.wasm writing on the internet frames the choice as a performance question — the comparison posts all quote big speedup multipliers. We didn’t switch for speed. We switched because we couldn’t legally ship the wasm build at all, and once we’d switched we discovered the comparison nobody writes: WebCodecs doesn’t replace FFmpeg, it hands you three of FFmpeg’s jobs.
This is what those three jobs cost us on a real, shipped product.
The wall we hit
The product is a browser-side file shrinker: you drop a clip in, it comes back under a Discord upload limit, and the bytes never leave your machine. The obvious build is ffmpeg.wasm. The wrappers — @ffmpeg/ffmpeg and @ffmpeg/util — are MIT and completely fine. The blocker is the thing that actually does the encoding: the core .wasm binary.
Rather than trust npm metadata, we pulled each core with npm pack, extracted it, and recovered the FFmpeg configure line that every ffmpeg.wasm core embeds verbatim in the binary — you can read it straight out with strings ffmpeg-core.wasm | grep -- --enable-gpl. Those flags are the ground truth for the license, because they name exactly which libraries were linked in.
@ffmpeg/core@0.12.10(the single-thread build matching wrapper 0.12.15) declares"license": "GPL-2.0-or-later"and ships no LICENSE or COPYING file at all — justpackage.jsonanddist/. Its embedded configure line contains--enable-gpl --enable-libx264 --enable-libx265. libx264 and libx265 are GPL-2.0-or-later; linking them into FFmpeg produces a GPL binary. That label is honest and correct.@ffmpeg/core@0.12.6is the version whose npm metadata says"license": "MIT", and it’s the one people cite when they want this problem to go away. Its embedded configure line is byte-identical to 0.12.10’s — same--enable-gpl, same libx264, same libx265. A build with those flags cannot be MIT. The field is the packager’s mistake, not the license.@ffmpeg/core@0.11.0, often described as “the LGPL one,” configures with--enable-gpl --enable-nonfree --enable-libx264 --enable-libx265 --enable-libfdk-aac.--enable-nonfreeplus libfdk-aac makes the result non-redistributable by FFmpeg’s own documentation. That’s worse than GPL for anyone shipping a static site.
We recorded sha256s for every file we inspected so the check is reproducible rather than a memory. For 0.12.10’s dist/umd/ffmpeg-core.wasm:
9f57947a5bd530d8f00c5b3f2cb2a3492faa7e5d823315342d6a8656d0a6b7b7Hash the file you actually downloaded and compare. If it differs, you are not looking at the build described here, and you owe yourself a fresh strings pass before trusting anything above.
So the decision wasn’t “wasm is slow.” It was: conveying that binary as part of a website puts the conveyed work under GPL-2.0-or-later and obliges us to offer complete corresponding source. Building our own LGPL-clean core (no --enable-gpl, no x264/x265) is genuinely possible, but it needs an Emscripten + FFmpeg toolchain, and the resulting core could only emit VP8/VP9/WebM — not the H.264/MP4 most people actually want to upload.
We dropped it and rebuilt the video path on WebCodecs — the browser’s own VideoEncoder/VideoDecoder, which is the platform’s codec, not a codec we ship — plus three vendored, pinned, permissively-licensed libraries for the container work: mp4-muxer 5.2.2 (MIT), webm-muxer 5.1.4 (MIT), and mp4box.js 2.4.1 (BSD-3-Clause) for demuxing frames back out of the source file. Those three do byte shuffling only. No encoder, no GPL, and nothing that phones home.
It works. Here’s the bill.
Cost 1: codec availability is a runtime question, not a build-time one
With ffmpeg.wasm you ship the codec, so you know what you have. With WebCodecs the codec belongs to the browser, the OS, and sometimes the GPU — and it varies. AAC encoding in particular is missing on a lot of Linux builds. You cannot hard-code a codec string and hope.
So the first thing we had to write was a capability probe: call VideoEncoder.isConfigSupported() and AudioEncoder.isConfigSupported() with the exact configs you intend to emit, and branch on the answer. Note exact — isConfigSupported() answers for the config you asked about, so a true for avc1.42001f at 1280×720 means “this realistic config works,” not “H.264 works.”
That pushed a ladder into the code where FFmpeg had a single flag. The H.264 path tries avc1.640028, then 4d0028, 640020, 4d401f, 42001f, 42e01e — High down to Baseline — and takes the first one the browser accepts. Audio tries mp4a.40.2 (AAC-LC), falls back to Opus, and if neither configures, the output is honestly video-only. Offering VP9/WebM as an alternative container adds another rule you now own: WebM cannot carry AAC, so the VP9 branch’s audio candidate list is Opus alone.
The user-visible consequence is that the UI has to say what this browser can do rather than assert what the tool can do. That’s more honest than the wasm version would have been, but it is real code you have to write, and it’s code FFmpeg was doing for you.
Cost 2: you scale frames yourself, by drawing them
-vf scale=1280:-2 is one of the most-typed strings in video work. WebCodecs has no equivalent. VideoEncoder takes VideoFrame objects at whatever size they already are; if you want a smaller output you produce smaller frames.
The actual mechanism is: create an OffscreenCanvas at the target size, drawImage() each decoded frame onto it, and construct a new VideoFrame from the canvas — carrying the original timestamp and duration forward, because a frame with the wrong timestamp desynchronises the mux. That’s a handful of lines, but it comes with a trapdoor we walked into: the encoder is configured once, with fixed dimensions, and every frame you feed it must match. If OffscreenCanvas isn’t available, or getContext('2d') returns null, you cannot quietly continue — native-size frames going into a smaller-configured encoder is a hard error. The correct handling is either fall back to native resolution before configuring the encoder, or fail out loud.
Then there’s the part scaling was for. Bitrate alone doesn’t decide quality; quality is bitrate spread over the pixels you encode. 300 kbps at 1920×1080 is a smear and the same 300 kbps at 640×360 is clean. FFmpeg’s rate control hides this. We had to write the decision explicitly, around a watchable floor expressed in bits per pixel per frame: at 0.03 bpp the floor works out to roughly 1.87 Mbps for 1080p30, ~830 kbps for 720p30, ~370 kbps for 480p30. If the byte budget clears the native floor, keep native size; otherwise pick the largest aspect-preserving, even-dimensioned downscale whose floor the budget does clear (H.264 requires even dimensions, so that rounding isn’t optional); only if you’ve hit a minimum sane resolution do you start dropping frame rate. That heuristic is ours now, and it is a planning estimate rather than a guarantee — real encoders track an average with tolerance.
Cost 3: the byte budget must be planned, not dialled in
This is the one that surprised us most. A command-line workflow lets you tweak -crf and look at the result. A web tool gets one shot in front of a user who is waiting, so you have to predict the size before you encode.
The arithmetic is simple — bytes ≈ bitrate × duration — but the details are where files come back oversized. You reserve a container/mux overhead margin off the top (ours defaults to 3% of the target), then split what’s left: audio first, with a floor of 32 kbps, and if honouring the requested audio squeezes video below ~100 kbps then audio drops to its floor and video takes the remainder. If even that leaves video under the floor, the honest answer is that the target is infeasible and you say so instead of producing a smear.
And you will still miss sometimes, which means a retry loop, which has its own trap. The obvious correction after an overshoot is proportional: next = bitrate × target / output. That’s wrong, because not every output byte scales with the video bitrate — the audio track, the container and the index are roughly fixed. Concretely: 8 MB target, 9 MB output, 1 MB of fixed overhead, video asked for 1,000,000 bps. Proportional gives 888,888 bps, whose predicted output is 8 MB × 8/9 + 1 MB ≈ 8.11 MB — still over, before any backoff. Subtracting the fixed part first (an affine correction) gives 875,000 bps → 7 MB + 1 MB = exactly 8.00 MB. We cap the ladder at three attempts, because a user watching a spinner has a patience budget too.
Our end-to-end gate encodes a committed 462,884-byte clip in a real browser and reads blob.size off the produced download. Two findings worth stealing: at a 150 KB target the first bitrate guess overshoots and the fit is only earned on the retry — that case exists specifically to keep the ladder honest. And at the 10 MB Free tier the same 463 KB source comes back larger, around 1.2 MB, because a re-encoder aimed at a tier spends the budget it’s given. That’s a legitimate quality choice, but it means “compress” is the wrong word for what the tool does at a generous limit, and the copy has to admit it.
What we’d tell you before you copy this
Two honest limits on our own claims. First, we ran no benchmark. WebCodecs is hardware-backed and ffmpeg.wasm is not, and every comparison post quotes a large multiplier, but we never measured one and won’t repeat a number we didn’t produce. Speed was not why we moved. Second, the “VP9 needs 20–40% fewer bits than H.264 at equal quality” figure our planner leans on is a published ecosystem range, not our measurement — we plan against the conservative end of it and treat it as an estimate, never a promise, because real encoder behaviour varies with content.
The trade is: you give up one dependency that does everything and a licence you can’t ship, and you take back three jobs — capability negotiation, frame scaling, and budget planning — that FFmpeg was quietly doing for you. For a tool whose whole promise is that your file never leaves your device, that was worth it. For a batch pipeline on a server, it obviously isn’t; run real FFmpeg there and comply with the GPL like everyone else.
The finished version of all of this is running on smallcord if you want to drop a file in and watch the probe, the downscale decision and the retry ladder do their thing.
More build notes and postmortems: the DankDev blog.
— DankDev
Tags: WebCodecs · ffmpeg · Licensing · Browser APIs · Video · JavaScript
Follow new posts by RSS: paste dankdev.com/feed.xml into any feed reader. It is a plain XML file — no signup, no email address, no account. Posts land there the same day they go up, and there is nothing to unsubscribe from later.