Catching Visual Regressions in Remotion with Frame Diffing

Remotion’s central promise is that a composition is a pure function: given the same frame number, the same props, and the same viewport, you get the same pixels. That determinism is what makes server-side rendering possible, and it is also what makes systematic visual regression testing tractable. If frame 45 of your title card looked correct when you merged the PR, it will still look correct on the next deploy, provided nothing has changed in between.

The problem is that “something changed” covers a wider surface area than most teams expect. A dependency bump that tweaks spring() damping defaults, a refactor that shuffles a <Sequence> offset by two frames, a font-face rule that now resolves to a different system fallback on the CI runner: none of these will fail a TypeScript build, none will trip a Jest unit test, and all of them can ship a broken video to customers long before anyone catches it.

Frame diffing closes that gap. Rather than rendering the entire video on every CI run, you pick a small set of semantically important frames, render them as PNGs, compare them pixel-by-pixel against stored golden images, and fail the build if the mismatch rate crosses a threshold. The rest of this article walks through how to build that pipeline in a way that is fast, deterministic, and tuned to catch the regressions that actually matter.

Why Remotion Makes Frame Diffing Unusually Reliable

Visual regression tools for browser UIs are notoriously noisy. Subpixel anti-aliasing, system font substitution, GPU compositing: any of these can shift a pixel by one unit on a Tuesday for no apparent reason, burying real regressions beneath a layer of false-positive noise that eventually makes the tool feel unreliable.

Remotion sidesteps most of this because @remotion/renderer downloads and pins its own Chromium build. Every machine that runs renderFrames() (your laptop, your colleague’s laptop, the GitHub Actions Ubuntu runner) uses the same browser binary at the same version. That single binary pin removes the most common driver of flakiness in headless screenshot pipelines.

The remaining sources of drift are narrower and predictable. Chromium’s font fallback stack can resolve system font names differently on macOS versus Linux. Any data fetched inside a delayRender/continueRender pair at render time introduces network variability. And Date.now() or Math.random() calls inside a composition inject non-determinism that you can eliminate by deriving values from useCurrentFrame instead.

Knowing these failure modes lets you guard against them, which the setup below addresses directly.

Choosing Your Golden Frames

Rendering every frame for diffing defeats the purpose. A 10-second, 30 fps composition has 300 frames, and rendering all of them in CI takes roughly the same time as a full export. The goal is to pick the minimum set of frames that gives maximum signal.

Three categories of frames are worth testing.

Scene entry frames are the first frame after a <Sequence> starts. They catch timing regressions where an animation begins two frames early, or where the whole composition has shifted because a clip was reordered.

Peak animation frames are those where a spring() or interpolate() call reaches its final resting value. For a spring with stiffness: 80 and damping: 20, that typically settles around frame 18 to 22. Testing frame 20 verifies the keyframe state rather than a transient in-between value.

Threshold crossing frames are those where opacity transitions from 0 to something visible, or where a translate completes. Off-by-one errors in <Sequence from={}> surface exactly here.

For a typical 150-frame composition with three scenes, six to nine golden frames give you solid coverage without extending CI time by more than 30 to 45 seconds.

Project Setup

Install the renderer and diffing dependencies:

npm install --save-dev @remotion/renderer pixelmatch pngjs
npm install --save-dev @types/pngjs

Create a scripts/ directory at the project root for the test harness. The renderer can run outside of a Remotion dev server; it bundles your composition internally.

The Render Harness

The harness bundles the composition, renders only the frames you need, and writes them to a temp directory:

// scripts/render-golden.ts
import path from 'node:path';
import fs from 'node:fs/promises';
import { bundle } from '@remotion/bundler';
import { renderFrames, selectComposition } from '@remotion/renderer';

// The frames we care about, keyed by a human-readable label.
// Label is used for the output filename so diffs are self-documenting.
const TEST_FRAMES: Record<string, number> = {
  'title-entry': 0,       // frame 0 — composition opens on title card
  'title-peak': 20,       // frame 20 — spring animation settled
  'scene2-entry': 60,     // frame 60 — second sequence begins
  'scene2-peak': 78,      // frame 78 — second animation settled
  'outro-entry': 120,     // frame 120 — outro sequence begins
  'outro-final': 149,     // frame 149 — last frame before hold
};

const COMPOSITION_ID = 'MainVideo';

async function main() {
  const outputDir = path.resolve('./test-frames/rendered');
  await fs.mkdir(outputDir, { recursive: true });

  // bundle() inlines your webpack config; no dev server needed
  const bundleLocation = await bundle({
    entryPoint: path.resolve('./src/index.ts'),
    // Pass your existing webpack override if you have one
    onProgress: (progress) => process.stdout.write(`\rbundling ${progress}%`),
  });
  process.stdout.write('\n');

  const composition = await selectComposition({
    serveUrl: bundleLocation,
    id: COMPOSITION_ID,
    // inputProps must be stable across runs — don't read from env here
    inputProps: {},
  });

  for (const [label, frame] of Object.entries(TEST_FRAMES)) {
    await renderFrames({
      composition,
      serveUrl: bundleLocation,
      outputDir,
      inputProps: {},
      imageFormat: 'png',  // never jpeg — compression artifacts cause false positives
      scale: 1,            // always render at native resolution for pixel-accurate diffs
      frameRange: [frame, frame], // render only this single frame
      timeoutInMilliseconds: 30_000,
      // rename each output from frame-000XXX.png to our label
      onFrameUpdate: () => {},
    });

    // renderFrames names outputs by frame number; rename to our label
    const paddedNum = String(frame).padStart(8, '0');
    const generatedName = path.join(outputDir, `frame-${paddedNum}.png`);
    await fs.rename(generatedName, path.join(outputDir, `${label}.png`));

    console.log(`rendered ${label} (frame ${frame})`);
  }
}

main().catch((e) => { console.error(e); process.exit(1); });

Run this once locally to generate your initial golden frames, then commit them to the repo:

npx ts-node scripts/render-golden.ts
# copy rendered/ → golden/
cp -r test-frames/rendered/* test-frames/golden/
git add test-frames/golden/
git commit -m "chore: add frame-diff golden images"

If your PNGs are large (1920×1080), consider Git LFS. A 1080p PNG of a typical title card runs between 200 and 500 KB; with a dozen goldens that is still under 6 MB uncompressed, which is fine for most repos without LFS.

The Diff Script

// scripts/diff-frames.ts
import path from 'node:path';
import fs from 'node:fs';
import pixelmatch from 'pixelmatch';
import { PNG } from 'pngjs';

// pixelmatch's threshold is per-channel, 0–1.
// 0.1 means a channel difference of 10% of 255 (~25 levels) is ignored.
// This is tight enough to catch real regressions but loose enough to absorb
// 1-bit rounding differences between Chromium patch releases.
const PIXEL_THRESHOLD = 0.1;

// Total mismatched pixels as a fraction of the frame's pixel count.
// 0.001 = 0.1% of pixels — roughly a 2×2 pixel cluster on a 1920×1080 frame.
const MAX_MISMATCH_RATIO = 0.001;

const goldenDir = path.resolve('./test-frames/golden');
const renderedDir = path.resolve('./test-frames/rendered');
const diffDir = path.resolve('./test-frames/diff');

fs.mkdirSync(diffDir, { recursive: true });

const labels = fs.readdirSync(goldenDir).filter((f) => f.endsWith('.png'));
let failed = false;

for (const filename of labels) {
  const goldenPath = path.join(goldenDir, filename);
  const renderedPath = path.join(renderedDir, filename);

  if (!fs.existsSync(renderedPath)) {
    console.error(`MISSING rendered frame: ${filename}`);
    failed = true;
    continue;
  }

  const golden = PNG.sync.read(fs.readFileSync(goldenPath));
  const rendered = PNG.sync.read(fs.readFileSync(renderedPath));

  if (golden.width !== rendered.width || golden.height !== rendered.height) {
    console.error(
      `SIZE MISMATCH ${filename}: golden ${golden.width}×${golden.height} vs rendered ${rendered.width}×${rendered.height}`
    );
    failed = true;
    continue;
  }

  const { width, height } = golden;
  const diffPng = new PNG({ width, height });

  const mismatchedPixels = pixelmatch(
    golden.data,
    rendered.data,
    diffPng.data,
    width,
    height,
    { threshold: PIXEL_THRESHOLD }
  );

  const mismatchRatio = mismatchedPixels / (width * height);

  // Always write the diff image — even on pass — so CI artifacts are always available
  fs.writeFileSync(
    path.join(diffDir, filename),
    PNG.sync.write(diffPng)
  );

  if (mismatchRatio > MAX_MISMATCH_RATIO) {
    console.error(
      `FAIL ${filename}: ${mismatchedPixels} pixels (${(mismatchRatio * 100).toFixed(3)}%) exceed threshold`
    );
    failed = true;
  } else {
    console.log(
      `PASS ${filename}: ${mismatchedPixels} pixels (${(mismatchRatio * 100).toFixed(3)}%)`
    );
  }
}

process.exit(failed ? 1 : 0);

Writing diff images for every frame, including passing ones, is intentional. When you upload the entire test-frames/diff/ directory as a CI artifact, reviewers can open every diff without re-running the job, even for frames that passed. Debugging a regression that passed diffing but looks wrong is much easier when you have the diff overlay in front of you.

The Font Trap on Linux Runners

This is where most teams’ first attempts break. Remotion pins Chromium, but Chromium still resolves font names through the OS font stack. If your composition uses fontFamily: 'system-ui' or any font that exists on macOS but not on the Ubuntu runner, Chromium will fall back to a different glyph, change the text layout, and fail every text-containing frame.

The fix is to declare explicit font stacks with system fonts that exist on both platforms, or to embed the font as a base64 @font-face rule in your composition’s global CSS.

The latter is more robust:

// src/fonts.ts — import this at the top of your root composition
import { useEffect } from 'react';

// Embed the font as a data URI so the same bytes render on every OS.
// Generate with: base64 -i YourFont.woff2 | tr -d '\n'
const FONT_DATA_URI = 'data:font/woff2;base64,d09GRgAB...'; // truncated

export function EmbedFonts() {
  useEffect(() => {
    const style = document.createElement('style');
    style.textContent = `
      @font-face {
        font-family: 'BrandFont';
        src: url('${FONT_DATA_URI}') format('woff2');
        font-weight: 400 700;
        font-display: block;
      }
    `;
    document.head.appendChild(style);
    return () => document.head.removeChild(style);
  }, []);
  return null;
}

This approach is also the correct path for production Remotion renders, not only for CI. External font URLs add latency and introduce a network failure point inside the renderer.

Guarding Against delayRender Non-Determinism

If a composition fetches data inside a delayRender block, you need to mock that data source for frame-diff tests. The simplest approach is an inputProps flag:

// src/compositions/DataDrivenScene.tsx
import { delayRender, continueRender, useCurrentFrame } from 'remotion';
import { useEffect, useState } from 'react';

type Props = {
  dataUrl: string;
  // When true, skip the network call and use staticData instead
  useStaticData?: boolean;
  staticData?: ChartData;
};

export function DataDrivenScene({ dataUrl, useStaticData, staticData }: Props) {
  const [data, setData] = useState<ChartData | null>(staticData ?? null);
  const handle = delayRender('fetch chart data');
  const frame = useCurrentFrame();

  useEffect(() => {
    if (useStaticData && staticData) {
      continueRender(handle);
      return;
    }
    fetch(dataUrl)
      .then((r) => r.json())
      .then((d) => { setData(d); continueRender(handle); })
      .catch(() => continueRender(handle)); // never let a failed fetch hang the renderer
  }, []);

  // ... render chart using data
}

In your render harness, pass useStaticData: true and a known fixture. The golden frames and the CI-rendered frames will both use the same data, so layout regressions caused by data shape changes will show up but network variability will not.

GitHub Actions Workflow

# .github/workflows/frame-diff.yml
name: Frame Diff

on:
  pull_request:
    paths:
      - 'src/**'
      - 'package.json'
      - 'package-lock.json'

jobs:
  frame-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - run: npm ci

      # Remotion bundles its own Chromium but still needs these system libs on Ubuntu
      - name: Install Chromium system dependencies
        run: |
          sudo apt-get update -qq
          sudo apt-get install -y --no-install-recommends \
            libnss3 libdbus-1-3 libatk1.0-0 libgbm-dev \
            libasound2 libxrandr2 libxkbcommon0 libxfixes3 \
            libxcomposite1 libxdamage1

      # Pre-download the pinned Chromium so it doesn't happen mid-render
      - name: Ensure Remotion browser
        run: npx remotion browser ensure

      - name: Render test frames
        run: npx ts-node scripts/render-golden.ts

      - name: Run pixel diff
        run: npx ts-node scripts/diff-frames.ts

      # Upload diff images even on failure so reviewers can inspect regressions
      - name: Upload diff artifacts
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: frame-diffs
          path: test-frames/diff/
          retention-days: 14

The paths filter keeps this job from running on documentation-only PRs. The if: always() on the artifact upload step is critical: without it, CI cleans up the diff images on failure, precisely when you need them most.

Updating Golden Frames

When you intentionally change the visual output (a redesign, a timing tweak, a new animation), the goldens need to be updated. Add a script entry:

{
  "scripts": {
    "frames:update": "ts-node scripts/render-golden.ts && cp -r test-frames/rendered/* test-frames/golden/"
  }
}

Run npm run frames:update locally, review the diff images with your eyes, then commit the new goldens in the same PR as the visual change. The PR diff will show the golden PNGs changed, which acts as a forcing function for code review. Nobody can silently regress the visuals without also touching the golden files.

Wrapping Up

Frame diffing for Remotion CI is genuinely more reliable than the equivalent for browser UIs, because the pinned Chromium binary eliminates most cross-machine variance. The gaps that remain (system font resolution, external data fetches, non-deterministic JavaScript inside compositions) are all fixable at the composition level with a bit of defensive design.

The practical checklist:

• Pick 6 to 10 golden frames per composition: scene entries, animation peaks, threshold crossings
• Always render with imageFormat: 'png' and scale: 1
• Embed fonts as data URIs rather than relying on system font stacks
• Mock external data sources via inputProps when the composition uses delayRender
• Set pixelmatch threshold to 0.1 and mismatch ratio ceiling to 0.1% as a starting point, then tighten after your first few green CI runs
• Upload diff images as CI artifacts unconditionally; you will want them on passing runs too

This pattern works well for custom compositions and scales naturally to larger template libraries. If you are starting from an existing template (like the ones in the RenderComp catalog), the golden frames are often already meaningful because the templates are built around stable, well-defined keyframe positions.

The investment is an hour of setup and a handful of golden PNGs. The payoff is a CI run that tells you, within 60 seconds of a push, whether frame 78 still looks the way it did when you shipped it.