Test the image bytes, not just the Open Graph tag

A blog page advertises a 1200 by 630 social card, yet Slack shows the generic QABattle image. The page is healthy and the <meta property="og:image"> tag exists. The failure lives one request later, in the generated image route.

Social sharing has two HTTP contracts. The first page response publishes metadata that points to an image. The second response must deliver the intended image bytes. Checking only the page proves that a URL was written into HTML. Checking only a convenient image route can miss a stale slug or wrong canonical host in the metadata. A reliable test follows the same link a crawler follows and keeps the two results distinct.

The repository contains three useful variants. Blog cards use the opengraph-image.tsx file convention under a dynamic [slug] segment. Battle and profile cards are ordinary route handlers under /api/og/. Each variant can return a valid PNG while showing fallback content, so status and file type are necessary but not sufficient assertions.

Trace the two requests that create one preview

src/app/blog/[slug]/page.tsx builds a canonical article URL from SITE_URL and the frontmatter slug. Its generateMetadata() result sets the Open Graph image to ${canonical}/opengraph-image, declares 1200 by 630 dimensions, and uses the article title as alt text. The Twitter metadata points to the same image URL.

Next.js treats opengraph-image.tsx as a specialized route handler. The current framework documentation says the function can receive dynamic route parameters and return an ImageResponse. This repository's image file exports size, contentType, and a generic alt, then awaits params to read the slug. The renderer returns a 1200 by 630 image.

The content lookup is where a subtle defect already has a recognizable shape. postMeta contains one explicit article, how-to-write-test-cases. Every other slug falls through to QABattle QA Guide and the guide track. Those fallbacks are deliberate expressions in the code, so the server can return 200 and a structurally perfect PNG for a route whose card is not article-specific.

That behavior gives the test suite four independent obligations:

1. The page must publish an absolute image URL for its own article path.
2. The image request must succeed and identify itself as PNG.
3. The binary file must have the declared dimensions and valid PNG structure.
4. The rendered content must come from the requested slug, or follow a documented fallback policy.

Do not merge those into one boolean named ogWorks. When the first assertion fails, metadata construction owns the bug. When binary validation fails, routing, rendering, or an intermediary owns it. When only content identity fails, inspect the slug lookup and data source. That separation makes a red CI result useful to someone who did not write the test.

There are also two alt-text sources in the current code: the page's metadata object uses the article title, while the image file exports QABattle QA guide. Framework precedence and generated tags can change across versions. Inspect the rendered head and assert the public result you intend. Do not decide that both source declarations must appear or guess which one wins from filenames alone.

The API routes have different data policies. The battle card truncates the requested slug to 120 characters, queries a published non-teaser battle, and uses generic battle copy when no row matches. The profile card truncates the username to 30 characters, queries a non-banned user, and shows a generic profile state when no user matches. Neither handler returns 404 for a missing record. A test expecting 404 would contradict the application rather than protect it.

These policies mean that “dynamic” describes input resolution, not guaranteed uniqueness. Two unknown blog slugs currently produce the same generic image. Two missing battles can do the same. A profile that exists but is banned is intentionally indistinguishable from an absent profile at this public boundary. Tests should assert uniqueness only for controlled records that are supposed to resolve. Requiring every arbitrary URL to produce different bytes would reject privacy and fallback behavior.

Page metadata and route metadata can also drift without either endpoint throwing. The page declares the article title as image alt text and 1200 by 630 dimensions before the image is fetched. If the renderer later changes to a different size, the HTML can continue advertising the old dimensions. That is why the binary parser compares the response with the public declaration. It protects an agreement across two resources, not simply two isolated constants in source code.

Check URL identity before decoding a pixel

Begin at a real article page. Read the metadata from the browser DOM because that is what the shipped framework output exposes. Parse each value with URL, then compare protocol, host policy, and pathname separately. In local CI, the metadata may correctly name the production canonical host while the test server runs on loopback. Request the path from the local server instead of accidentally turning a local test into a production dependency.

// e2e/blog-og-metadata.spec.ts
import { expect, test } from "@playwright/test";

const slug = "playwright-locators-guide";

test("article metadata points to its own generated image", async ({
  page,
  request,
}) => {
  await page.goto(`/blog/${slug}`);

  const ogValues = await page
    .locator('meta[property="og:image"]')
    .evaluateAll((elements) =>
      elements
        .map((element) => element.getAttribute("content") ?? "")
        .filter(Boolean),
    );
  const twitterValues = await page
    .locator('meta[name="twitter:image"]')
    .evaluateAll((elements) =>
      elements
        .map((element) => element.getAttribute("content") ?? "")
        .filter(Boolean),
    );

  expect(ogValues.length).toBeGreaterThan(0);
  expect(twitterValues.length).toBeGreaterThan(0);

  const expectedPath = `/blog/${slug}/opengraph-image`;
  for (const value of [...ogValues, ...twitterValues]) {
    const imageUrl = new URL(value);
    expect(imageUrl.protocol).toBe("https:");
    expect(imageUrl.pathname).toBe(expectedPath);
    expect(imageUrl.search).toBe("");
    expect(imageUrl.hash).toBe("");
  }

  const response = await request.get(expectedPath);
  expect(response.ok(), `image returned ${response.status()}`).toBe(true);
  expect(response.headers()["content-type"]).toMatch(/^image\/png(?:;|$)/i);
});

This test rejects several plausible regressions. Copying metadata from another article leaves a different slug in the path. A preview deployment host accidentally published as canonical fails the host-policy assertion if you add the expected hostname. A query string used as an undocumented cache buster appears in search. A redirect to a generic image can still end in 200, so compare response.url() with the intended local endpoint when redirect behavior matters.

Keep host policy configurable but explicit. PLAYWRIGHT_BASE_URL tells Playwright where to execute. It does not define the site's public identity. The expected canonical host should come from release configuration or a test constant that represents product policy. Deriving it from localhost creates an assertion that asks production metadata to be local.

If the page has multiple og:image tags by design, test the ordered set and role of each image. Do not blindly select the first and ignore the rest. Conversely, requiring exactly one can reject a legitimate multi-image policy. This repository currently declares one article image, so a stricter count is reasonable after inspecting the actual rendered output for the installed Next.js version.

A metadata failure can look like an image failure in a social debugger. If the advertised URL contains the old slug, requesting the new route manually proves only that an unreferenced route works. Preserve the exact content value from the page whenever triaging a preview. The crawler does not invent the URL you wish the page had published.

One useful negative case changes only the article slug in a controlled fixture while leaving its title unchanged. A title-based assertion alone still passes because the visible page looks identical. The path assertion fails and shows that shares continue requesting the previous image route. This is common after content migrations, where redirects keep the article readable but metadata construction still uses stale identity.

A second negative case changes SITE_URL for a preview build. The local path remains valid and the PNG test passes, yet the head advertises the preview hostname. Whether that is correct depends on release policy. Production smoke tests should require the production canonical origin, while preview tests may either require their own origin or validate only the pathname. Encode that choice in configuration instead of weakening every environment to “any HTTPS URL.”

When redirects are allowed, test them deliberately. Playwright's request client follows redirects by default, so a final 200 can hide a 301 from the article-specific image to a generic asset. Compare the final response.url() with the requested identity, or create a request context with redirects disabled when the redirect itself is the subject. Do not infer the redirect chain from the final status alone.

Verify the response is a 1200 by 630 PNG

An HTTP header is a claim about bytes, not proof of them. A proxy or error handler can send HTML with Content-Type: image/png. A one-pixel fallback can have the correct type. A nonempty-body threshold catches truncation but cannot distinguish any of these cases.

PNG files have a fixed eight-byte signature. Their first chunk must be IHDR, whose data begins with four-byte width and height values in network byte order. Reading those fields gives a deterministic structural assertion without an image-processing dependency.

// e2e/blog-og-png.spec.ts
import { expect, test } from "@playwright/test";

const pngSignature = Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]);

function readPngSize(body: Buffer): { width: number; height: number } {
  if (body.length < 24) throw new Error(`PNG is truncated at ${body.length} bytes`);
  if (!body.subarray(0, 8).equals(pngSignature)) {
    throw new Error("response does not start with the PNG signature");
  }
  const firstChunkLength = body.readUInt32BE(8);
  const firstChunkType = body.toString("ascii", 12, 16);
  if (firstChunkType !== "IHDR" || firstChunkLength !== 13) {
    throw new Error(
      `expected a 13-byte IHDR first chunk, got ${firstChunkType}:${firstChunkLength}`,
    );
  }
  return {
    width: body.readUInt32BE(16),
    height: body.readUInt32BE(20),
  };
}

test("generated blog card has the declared PNG dimensions", async ({
  request,
}, testInfo) => {
  const response = await request.get(
    "/blog/playwright-locators-guide/opengraph-image",
  );
  expect(response.status()).toBe(200);
  expect(response.headers()["content-type"]).toMatch(/^image\/png(?:;|$)/i);

  const body = await response.body();
  expect(readPngSize(body)).toEqual({ width: 1200, height: 630 });

  await testInfo.attach("generated-open-graph-card", {
    body,
    contentType: "image/png",
  });
});

Every branch can fail because of a real product mutation. Return an error page and the signature check fails. Truncate the body and the minimum header length fails. Change the renderer to 600 by 315 and the dimension assertion fails. This is stronger than asserting byteLength > 100, which accepts almost any unrelated payload of moderate size.

The attachment gives reviewers the exact image from the failed attempt. It is evidence, not the automated oracle. Human inspection is valuable for typography and clipping, but a person should not have to open every artifact to discover a 500 response or wrong dimensions.

Avoid asserting an exact compressed file size. PNG encoding, fonts, framework versions, and harmless renderer changes can alter the number of bytes without changing pixels or dimensions. Hashes are useful to compare two supposedly identical responses from the same release, not as permanent semantic contracts unless the asset is intentionally immutable.

The same parser works for the battle and profile PNG routes because they also return ImageResponse with 1200 by 630 options. Keep route-specific status and content policies separate. A generic profile card for an unknown username may be correct, while generic copy for a known seeded username is a data-resolution failure.

Run the PNG parser against a known bad fixture once to prove the oracle. Feed it an HTML buffer, a truncated signature, and a syntactically valid 1 by 1 PNG. The first two cases should throw and the last should report the wrong dimensions. This mutation exercise is cheap and prevents a helper refactor from turning the test into a body-exists check with a reassuring name.

Dimension validation does not prove the complete PNG is decodable. The IHDR can be correct while later chunks are truncated or corrupt. A browser load assertion covers that next layer: wait for the image element, require naturalWidth and naturalHeight, and check that it completed without the broken-image state. The visual snapshot example performs that browser decode for a representative card. For a high-risk image service, add a real PNG decoder in a focused integration test rather than attempting to implement the entire format in test code.

Color space, embedded profiles, and compression settings rarely belong in a social-card smoke test. Assert them only when a downstream platform or brand workflow has a documented requirement. Every extra binary rule increases coupling to the encoder. Status, type, decodability, dimensions, and intended pixels cover the failures readers and crawlers actually experience in this application.

Catch a valid image with the wrong content

Structural checks will happily accept the current generic blog fallback. To protect content identity, make the card-copy lookup testable and ensure the image route can read the complete post inventory. The existing route exports the Edge runtime while src/lib/blog.ts imports Node's file-system module. Importing that loader into the Edge route would be an invalid fix.

One valid design is to run this particular image route in the Node.js runtime and reuse getPostBySlug(). The trade-off is important: the route now loads and parses the content inventory in a Node environment, and it gives up the deployment characteristics that motivated the Edge runtime. Measure cold generation and build behavior before adopting it for a large catalog.

// src/app/blog/[slug]/opengraph-image.tsx
import { ImageResponse } from "next/og";
import { getPostBySlug } from "@/lib/blog";

export const runtime = "nodejs";
export const alt = "QABattle QA guide";
export const size = { width: 1200, height: 630 };
export const contentType = "image/png";

export default async function OgImage({
  params,
}: {
  params: Promise<{ slug: string }>;
}) {
  const { slug } = await params;
  const post = getPostBySlug(slug);
  const title = post?.title ?? "QABattle QA Guide";
  const track = post?.track ?? "guide";

  return new ImageResponse(
    (
      <div
        style={{
          width: "100%",
          height: "100%",
          display: "flex",
          flexDirection: "column",
          justifyContent: "space-between",
          padding: 72,
          background: "#19140f",
          color: "#f3efe7",
          fontFamily: "sans-serif",
        }}
      >
        <div
          style={{
            display: "flex",
            justifyContent: "space-between",
            color: "#f2b240",
            fontSize: 26,
            fontWeight: 800,
          }}
        >
          <span>QABATTLE</span>
          <span>{track.toUpperCase()}</span>
        </div>
        <div style={{ display: "flex", fontSize: 76, fontWeight: 900 }}>
          {title}
        </div>
        <div style={{ display: "flex", fontSize: 28, color: "#b8ad9d" }}>
          The Testing Academy
        </div>
      </div>
    ),
    size,
  );
}

Another valid design keeps the Edge renderer and supplies a generated, Edge-safe metadata module containing only slug, title, and track. That preserves the runtime choice but adds a generation step and another artifact that can drift from Markdown. Its QA contract must compare the generated metadata slug set with the source inventory. Hand-maintaining a large object is the least attractive option because every new article creates a second manual publishing step.

A third design passes card text through a signed or otherwise controlled query, but that changes the public URL contract and expands the validation surface. Arbitrary title text in a query can create cache fragmentation, excessively long URLs, or untrusted rendering input. It also duplicates metadata in the link. Use it only when the route cannot access a trusted content source and the team is prepared to validate length, encoding, and authenticity.

Whichever design wins, keep resolution separate from layout. A pure function can accept a trusted post record and return the title, label, and optional summary used by the renderer. Unit tests cover known, missing, and boundary records without decoding PNGs. Integration tests then prove that the route passes the resolved values into ImageResponse. Pixel tests remain focused on layout. This split makes a failed title lookup different from a font or flexbox regression.

The fallback itself is not automatically wrong. An unknown slug may intentionally receive a branded generic card. Make that a named branch and test it with a value guaranteed not to exist. Then test two known posts with different titles and tracks. A resolver that always returns the generic object fails the known-post cases but passes the unknown case.

Pixel comparison covers the final layout contract. Use a small representative set rather than all posts: a normal title, the longest supported title fixture, punctuation and non-ASCII text, and missing optional content. A browser page can load the image into an <img> and let Playwright compare the rendered element.

import { expect, test } from "@playwright/test";

test("representative article card matches the approved layout", async ({ page }) => {
  await page.goto("/");
  await page.setContent(`
    <style>html, body { margin: 0; } img { display: block; }</style>
    <img data-testid="card"
         alt=""
         src="/blog/playwright-locators-guide/opengraph-image">
  `);

  const card = page.getByTestId("card");
  await expect(card).toHaveJSProperty("naturalWidth", 1200);
  await expect(card).toHaveJSProperty("naturalHeight", 630);
  await expect(card).toHaveScreenshot("blog-card-normal-title.png", {
    animations: "disabled",
  });
});

The snapshot catches generic text, clipping, unexpected font fallback, color changes, and layout movement. Its cost is baseline review. A deliberate brand change updates the image and every affected baseline. Keep the environment's fonts and browser version controlled, and never approve snapshots in bulk without examining the diff.

Separate renderer, data, and cache failures

Consider three incidents that look identical to a social-media user. In the first, the image URL returns 500. The page metadata is correct, the route status is not, and server logs point to image generation. Unsupported styling, a failed font read, or an exception in data access can live here. The useful artifact is the response status, short text excerpt if it is not an image, request URL, and correlated server error. Updating metadata will not repair the renderer.

In the second incident, the response is a valid 1200 by 630 PNG but contains generic copy. The structural test passes and the representative snapshot fails. For a blog route, inspect whether the requested slug exists in the card metadata source. For a battle, confirm the seeded record is published, is not a teaser, and survives the 120-character lookup rule. For a profile, confirm the exact username identifies a non-banned user. The handlers intentionally convert “no matching row” into branded fallback content, so a database miss is observable in pixels rather than status.

In the third incident, a direct request to the exact published URL returns the current correct bytes, while one sharing platform still shows an old card. Record a digest of the origin response, its headers, the deployment identifier, and the time observed. Repeat from a clean network client before blaming application caching. If the origin remains correct and only the platform is stale, use that platform's documented refresh mechanism or wait for its cache policy. Do not add random query parameters to production metadata without deciding whether each new URL should become a separate public asset identity.

Application caching can still be responsible. Next.js documentation states that generated metadata images are statically optimized by default unless dynamic APIs or uncached data make them dynamic. Ordinary /api/og/ handlers and deployment intermediaries can have different behavior. Assert the behavior your deployed route actually promises, and record cache-control, etag, age, and the final URL when present. Do not hard-code a framework header copied from one environment if the application has not adopted it as a contract.

Long text creates a fourth failure class. Status, type, dimensions, and lookup can all pass while a title wraps over the author line or disappears outside the canvas. That belongs to representative pixel tests with controlled boundary fixtures. Do not claim support for an arbitrary character count because one hand-written long title happened to fit. Define the editorial limit or design the card to clamp and test that exact rule.

Character rendering needs similar discipline. Include text with punctuation and the scripts the product truly supports. If a required glyph is absent from the configured font, a fallback may change line breaks without throwing. The visual diff exposes it. A test that searches PNG bytes for the title cannot work because rendered text is pixels, not embedded plain text.

Database timing can produce a valid transient fallback. A battle may be published in one transaction while a share URL is generated before a read replica observes the row. The route then returns generic content and a cache can preserve it. Prove this scenario with database timestamps, the exact data source used by the image handler, and response cache evidence before changing renderer code. A retry against the primary database that succeeds shows data visibility, not a rendering fix.

There is a similar deployment race for static metadata maps. If Markdown and the generated lookup module come from different artifacts, the article page can publish a new slug while its image renderer still has the old catalog. Record the commit or artifact identity for both page and image responses when the platform exposes it. A cache purge cannot reconcile two genuinely different deployments.

For failures that occur only with one title, reduce the input while preserving the bad character or length. If a shorter ASCII title works, the renderer and route are generally available. The remaining suspects are font coverage, text layout, encoding, or an input limit. Keep the original PNG and reduced fixture together so a layout fix can be reviewed against the real incident.

Keep request evidence small but complete. Attach the failed PNG, headers, URL, slug, expected data identity, and server log correlation key if one exists. Avoid logging entire user records fetched for profile cards. The public card needs a username, XP, streak, and level output, not email or internal identifiers.

Roll out useful coverage without snapshotting the catalog

Put structural checks on every pull request that changes metadata, image routes, framework versions, or shared layout code. They are fast, deterministic, and cover every response class. Add resolver tests for known and unknown slugs close to the data lookup. Run a handful of visual snapshots for layout boundaries. Use a deployed smoke check for canonical host and origin response after promotion.

CI can keep these layers visible as separate commands. The configuration below assumes dependencies and browsers were provisioned earlier by the shared job setup. It does not ask an external social platform to scrape the preview, so release success does not depend on a third-party cache.

steps:
  - name: Verify Open Graph metadata and PNG structure
    run: >-
      pnpm exec playwright test
      e2e/blog-og-metadata.spec.ts
      e2e/blog-og-png.spec.ts

  - name: Compare representative card pixels
    run: pnpm exec playwright test e2e/blog-og-visual.spec.ts

  - name: Upload failed social cards
    if: failure()
    uses: actions/upload-artifact@v4
    with:
      name: open-graph-card-evidence
      path: test-results/
      if-no-files-found: error

Start with the existing generic behavior documented as a known failure, then fix the data source and approve baselines only after design review. Do not generate baselines from a red incident and call the current pixels correct. The reviewer should see the requested slug, resolved title and track, and final image side by side.

During rollout, separate baseline creation from baseline enforcement. First publish PNG attachments for the representative cases without comparing them. Design and QA agree on expected wrapping, safe margins, labels, and fallback copy. Commit only those approved images, then enable blocking comparison. This prevents the first automated run from silently defining the product design.

Treat framework and browser upgrades as deliberate snapshot events. Run the structural suite first. If dimensions, status, and data identity remain correct but many pixels change, inspect font rasterization and renderer release notes before updating. A mass diff can be legitimate, but it deserves one reviewed cause. Updating every baseline because the job is red destroys the historical signal.

The cost of this suite is not only execution time. Binary artifacts consume storage, visual diffs need human attention, and known-record tests need stable fixtures. Retain successful images briefly or not at all, while keeping failed images long enough for triage. Seed only public fields needed by the card and clean them through the same test-data lifecycle as other integration records.

Avoid a snapshot for every article. Most cards share one renderer, so a thousand nearly identical baselines multiply storage and review cost without creating a thousand independent layout rules. Test the renderer's meaningful input classes and test the full slug inventory at the resolver layer. Add a specific visual baseline only when an article has an intentionally unique design.

Do not use a generated route when every page intentionally shares one immutable card. A static PNG is simpler, cheaper to cache, and easier to verify. Dynamic rendering earns its operational cost only when the content actually varies or when generation removes a valuable manual step.

Do not require an unknown battle or profile to return 404 unless product policy changes. Current code promises a branded fallback. Likewise, do not mistake a correct fallback test for proof that known records resolve. Both branches need their own fixtures and assertions.

Finally, keep social-platform scraping outside the deterministic application gate. It can be a useful scheduled monitor, but it depends on external network behavior, crawler rules, and caches the test does not control. The release gate should prove that the page advertises the right URL and that the origin serves the right card. Platform-specific display is a separate observation with a separate owner.