Use Playwright Screenshots to Catch Remotion Frame Regressions

Remotion treats video as a pure function of time: f(frame, props) → pixels. Given the same frame number and the same input props, useCurrentFrame() always returns the same integer, spring() always returns the same float, and interpolate() always produces the same mapped value. There is no hidden temporal state. That determinism is what makes regression testing genuinely reliable: render frame 42 today and frame 42 after a dependency upgrade, and any pixel difference is a real change, not noise.

The gap most teams fall into is reaching for renderStill once to verify that a composition doesn’t crash, then shipping without a baseline comparison system. That catches panics, but not the subtler regressions that travel silently: a spring that stopped overshooting after @remotion/renderer bumped its internal Chromium, a text element that shifted four pixels when you refactored a layout component, a brand color that desaturated when you touched a CSS variable file. Without committed baselines, those changes slip through code review and reach production inside the next rendered video.

Playwright adds the second layer. Its toHaveScreenshot API gives you managed baseline storage, configurable pixel diff thresholds, and failure artifacts showing exactly which pixels changed, all applied to your compositions running in a real browser via @remotion/player. This article covers the complete setup: choosing keyframes, building a test harness, writing Playwright tests, handling non-determinism, and hardening the pipeline for CI.

---

What Makes a Keyframe Worth Testing

Testing every frame is wasteful and makes baseline updates painful. The productive strategy is to identify animation phase boundaries, frames where the visual state changes qualitatively rather than incrementally.

For a composition that opens with an enter animation, holds, then exits, the natural checkpoints are:

• Frame 0 is the initial state, with everything at opacity 0 or off-screen.
• The spring settle frame is where the enter animation has reached roughly 99% of its target.
• A stable mid-clip frame sits well past the enter transition and before the exit begins.
• The final frame shows exit animations fully resolved, with no overshoot tail.

The spring settle frame is not a guess; it’s computable. A spring({ stiffness: 180, damping: 12, mass: 1 }) at 30fps crosses 0.99 of its target at frame 18. A softer { stiffness: 80, damping: 10 } takes until roughly frame 35. You can find the exact value without eyeballing:

// utils/keyframes.ts
import { spring } from 'remotion';

/**
 * Returns the frame at which a spring reaches `threshold` of its final value.
 * Use this to pick regression keyframes instead of guessing.
 */
export function springSettleFrame(
  config: { stiffness: number; damping: number; mass: number },
  fps: number,
  threshold = 0.99,
): number {
  for (let frame = 0; frame < 600; frame++) {
    if (spring({ frame, fps, config }) >= threshold) return frame;
  }
  return 600;
}

// For a PromoCard with stiffness:180, damping:12 at 30fps → frame 18
// For a softer SlideIn  with stiffness:80,  damping:10 at 30fps → frame 35

For a 150-frame clip at 30fps, a keyframe set that covers enter, held, and exit states looks like this:

// tests/keyframes.ts
export const PROMO_VIDEO_KEYFRAMES = [
  0,    // before any animation starts
  9,    // mid-rise of the enter spring
  18,   // spring settled — held state begins
  90,   // mid-clip, tests any looping or continuous motion
  149,  // last frame, tests exit animations and clamped interpolations
] as const;

---

Approach 1: renderStill Node.js Baselines

The fastest path to frame comparison is @remotion/renderer’s renderStill. It spins up a headless Chromium internally, renders one frame, and writes a PNG, with no server setup or browser launch overhead from your test runner.

// scripts/render-baselines.ts
import { bundle } from '@remotion/bundler';
import { getCompositions, renderStill } from '@remotion/renderer';
import path from 'path';
import { mkdirSync } from 'fs';

const OUTPUT_DIR = path.resolve('./baselines');
mkdirSync(OUTPUT_DIR, { recursive: true });

async function captureBaselines() {
  // bundle() returns a local directory path; renderStill serves it internally
  const serveUrl = await bundle({
    entryPoint: path.resolve('./src/index.ts'),
  });

  const compositions = await getCompositions(serveUrl);
  const comp = compositions.find((c) => c.id === 'PromoVideo');
  if (!comp) throw new Error('Composition PromoVideo not found in bundle');

  const frames = [0, 9, 18, 90, 149];

  for (const frame of frames) {
    const output = path.join(OUTPUT_DIR, `promo-video-frame-${frame}.png`);
    await renderStill({
      composition: comp,
      serveUrl,
      output,
      frame,
      // Pin inputProps so any seed-driven randomness is deterministic
      inputProps: { seed: 42 },
    });
    console.log(`✓ frame ${frame} → ${output}`);
  }
}

captureBaselines();

Commit the resulting PNGs. In CI, re-render those same frames and diff against the committed files:

// tests/still-regression.test.ts
import { renderStill, getCompositions } from '@remotion/renderer';
import { bundle } from '@remotion/bundler';
import pixelmatch from 'pixelmatch';
import { PNG } from 'pngjs';
import { readFileSync } from 'fs';
import path from 'path';
import { describe, it, expect, beforeAll } from 'vitest';

let serveUrl: string;

beforeAll(async () => {
  serveUrl = await bundle({ entryPoint: path.resolve('./src/index.ts') });
}, 60_000);

describe('PromoVideo frame regressions', () => {
  const frames = [0, 9, 18, 90, 149];

  for (const frame of frames) {
    it(`frame ${frame} matches baseline`, async () => {
      const actualPath = path.join('/tmp', `actual-frame-${frame}.png`);
      const compositions = await getCompositions(serveUrl);
      const comp = compositions.find((c) => c.id === 'PromoVideo')!;

      await renderStill({
        composition: comp,
        serveUrl,
        output: actualPath,
        frame,
        inputProps: { seed: 42 },
      });

      const baseline = PNG.sync.read(
        readFileSync(path.join('./baselines', `promo-video-frame-${frame}.png`)),
      );
      const actual = PNG.sync.read(readFileSync(actualPath));
      const { width, height } = baseline;
      const diff = new PNG({ width, height });

      const diffPixels = pixelmatch(
        baseline.data, actual.data, diff.data, width, height,
        { threshold: 0.1 }, // per-pixel color distance tolerance (0–1)
      );

      // Allow up to 0.1% of pixels to differ — sub-pixel anti-aliasing noise
      const allowedDrift = Math.floor(width * height * 0.001);
      expect(diffPixels).toBeLessThanOrEqual(allowedDrift);
    }, 30_000);
  }
});

The renderStill approach is Chromium-only and tests the Remotion renderer’s internals rather than the runtime your users experience when watching a video embedded via @remotion/player on your site. That gap is where Playwright earns its keep.

---

Approach 2: Playwright with @remotion/player

@remotion/player renders compositions directly in the browser DOM, running the same React tree your users see. Testing through Playwright means CSS variable resolution, self-hosted font rendering, and any browser-specific layout quirks all participate in the comparison.

Build a Frame Harness Page

The harness reads the target frame from the URL query string so Playwright can navigate to an exact frame without JavaScript injection:

// src/test-harness/FrameHarness.tsx
import { Player } from '@remotion/player';
import { PromoVideo } from '../compositions/PromoVideo';

const params = new URLSearchParams(window.location.search);
const initialFrame = Number(params.get('frame') ?? '0');
const seed = Number(params.get('seed') ?? '42');

export default function FrameHarness() {
  return (
    // Fixed pixel dimensions: no responsive scaling, no scroll, no viewport surprises
    <div
      data-testid="player-frame"
      style={{ width: 1920, height: 1080, overflow: 'hidden' }}
    >
      <Player
        component={PromoVideo}
        durationInFrames={150}
        fps={30}
        compositionWidth={1920}
        compositionHeight={1080}
        // initialFrame renders at this exact frame without autoplay
        initialFrame={initialFrame}
        inputProps={{ seed }}
        style={{ width: 1920, height: 1080 }}
        // Player shows no controls by default — explicit here for clarity
        controls={false}
      />
    </div>
  );
}

Route this page at /frame-harness in your dev server. The data-testid attribute gives Playwright a stable selector that won’t break if Remotion’s internal DOM structure changes.

Write the Playwright Tests

// tests/frame-regression.spec.ts
import { test, expect } from '@playwright/test';

const BASE_URL = 'http://localhost:5173';

const SUITES = [
  { id: 'PromoVideo',      frames: [0, 9, 18, 90, 149] },
  { id: 'CountdownTimer',  frames: [0, 15, 30, 59]     },
] as const;

for (const suite of SUITES) {
  for (const frame of suite.frames) {
    test(`${suite.id} · frame ${frame}`, async ({ page }) => {
      // The frame is baked into the URL — no seekTo() calls needed
      await page.goto(
        `${BASE_URL}/frame-harness?comp=${suite.id}&frame=${frame}&seed=42`,
      );

      // Wait for self-hosted fonts and any preloaded assets to finish loading.
      // 'networkidle' means no in-flight requests for 500ms — reliable for
      // @font-face assets that trigger on first paint.
      await page.waitForLoadState('networkidle');

      const playerEl = page.getByTestId('player-frame');

      await expect(playerEl).toHaveScreenshot(
        `${suite.id}-frame-${frame}.png`,
        {
          // Allow up to 2% of pixels to differ — covers sub-pixel AA variance
          // between local dev (macOS) and CI (Linux).
          maxDiffPixelRatio: 0.02,
          // Per-pixel color distance tolerance (0–1). 0.15 passes minor
          // hinting differences while catching layout and color regressions.
          threshold: 0.15,
        },
      );
    });
  }
}

On the first run, Playwright writes baseline PNGs under tests/__snapshots__/. Commit them. After intentional design changes, regenerate with:

npx playwright test --update-snapshots

The diff images in test-results/ show exactly which pixels changed, which is useful in pull request review to confirm that a refactor only touched what it was supposed to.

---

The Non-Determinism Trap

If your composition calls Math.random() or new Date() anywhere in the render path, the pixel output changes every run and baselines become useless. Common culprits:

• Particle systems that scatter elements on mount
• Generative art compositions with random placement or color variation
• Data-driven compositions that format a live timestamp

The fix is to accept a seed prop and pass it through every component that needs randomness. Use a seeded pseudo-random generator scoped to the frame rather than a module-level singleton:

// compositions/ParticleIntro.tsx
import seedrandom from 'seedrandom';

interface Props {
  seed: number; // always thread this in — never call Math.random() directly
}

export const ParticleIntro: React.FC<Props> = ({ seed }) => {
  const frame = useCurrentFrame();

  // Scoping the seed to `frame` means the same particles appear at the same
  // positions regardless of which frame Remotion renders first.
  const rng = seedrandom(`${seed}-${frame}`);

  const particles = Array.from({ length: 20 }, () => ({
    x: rng() * 1920,
    y: rng() * 1080,
    radius: rng() * 8 + 4,
  }));

  return (
    <AbsoluteFill>
      {particles.map((p, i) => (
        <div
          key={i}
          style={{
            position: 'absolute',
            left: p.x,
            top: p.y,
            width: p.radius * 2,
            height: p.radius * 2,
            borderRadius: '50%',
            background: 'white',
          }}
        />
      ))}
    </AbsoluteFill>
  );
};

Pass seed=42 in every test URL and renderStill call. The particle positions are then identical across every CI run and every local developer machine.

---

CI Configuration

Chrome Flags

Headless Chromium inside Docker containers requires --no-sandbox and --disable-dev-shm-usage. Wire these into your Playwright config alongside a webServer entry so the dev server starts automatically before the test suite:

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  projects: [
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome'],
        launchOptions: {
          args: ['--no-sandbox', '--disable-dev-shm-usage'],
        },
        // Fix the viewport to exactly the composition dimensions
        viewport: { width: 1920, height: 1080 },
      },
    },
  ],
  webServer: {
    command: 'npx vite --port 5173',
    url: 'http://localhost:5173',
    reuseExistingServer: !process.env.CI,
    timeout: 30_000,
  },
  // Strip platform suffix from snapshot filenames so one baseline file
  // covers both macOS (local) and Linux (CI) — combined with the threshold
  // settings above, this avoids dual-platform baseline maintenance.
  snapshotPathTemplate: '{testDir}/__snapshots__/{testFilePath}/{arg}{ext}',
});

Platform Pixel Drift

Sub-pixel font hinting and anti-aliasing differ between macOS and Linux. If you generate baselines locally on macOS and run comparisons on a Linux CI runner, you’ll hit failures that represent rendering infrastructure differences rather than actual code regressions. Two approaches:

1. Generate baselines in CI by adding a --update-snapshots step to a dedicated CI job, committing the resulting PNGs, and always comparing on the same Linux environment. This is the right call for templates with precise brand colors where 1% pixel drift matters.

2. Relax the threshold by setting maxDiffPixelRatio: 0.05 and threshold: 0.2. This accepts more noise but avoids dual-platform baseline maintenance, making it suitable for motion-heavy compositions where anti-aliasing variation is expected and the regressions you care about are layout shifts and color changes.

---

Wrapping Up

The pattern in practice:

• Pick keyframes computationally. Use springSettleFrame() to find animation phase boundaries rather than choosing arbitrary round numbers. Frame 0, the spring settle point, and the final frame cover most regression surfaces.
• Use renderStill for fast Node.js diffs. No browser setup, runs in seconds, catches pure render regressions from Remotion version bumps.
• Use Playwright for the full-browser layer. Catches font loading failures, CSS variable drift, and @remotion/player embed regressions that bypass renderStill.
• Pin seed in every composition that touches randomness. Module-level Math.random() calls make baselines useless. Thread a seed prop through your component tree and scope it to the frame.
• Generate baselines on Linux CI. Platform-specific hinting differences cause spurious failures if baselines are committed from macOS.

The investment pays off when a Remotion patch changes internal spring behavior by 0.003 at one stiffness value, or when a CSS refactor shifts a title by a single pixel. Without regression snapshots, the change ships inside the next rendered video. With them, the CI pipeline catches it at the diff stage and produces an image showing exactly which pixels moved, giving you the same guarantee you’d expect from UI component tests applied to the output that becomes your customers’ videos. Templates like those in the RenderComp catalog are built around this testing foundation precisely because even subtle visual drift is visible to clients who approved a specific frame.