When a thousand Markdown posts turn builds into a release risk

Your 999-post build passes. Add one Markdown file, and CI either runs out of memory or ships without the new URL. A homepage smoke test stays green because it never asks whether the content inventory and generated routes still agree.

That is the uncomfortable part of a large static blog: “the build passed” is not the same claim as “every publishable article became the right public page.” One is a process result. The other is an inventory contract. A useful QA strategy checks both, then keeps enough evidence to tell a malformed article from a capacity limit or a stale deployment.

This repository makes the contract unusually clear. Markdown files live in content/blog. src/lib/blog.ts reads them, validates the filename against frontmatter, and builds the post objects. src/app/blog/[slug]/page.tsx returns every post slug from generateStaticParams(). src/app/sitemap.ts uses the same post inventory for public discovery. Those connections are valuable, but shared inputs can also create shared blind spots. If the loader silently excludes a file, both the page list and sitemap can agree on the same wrong set.

Follow one article from disk to its public URL

Start with the path a real article takes. getAllPosts() calls readPosts() the first time its module-level cache is empty. The loader reads every file ending in .md from content/blog, parses frontmatter with gray-matter, calculates reading statistics, extracts headings, and sorts the resulting posts by publication date. The parser rejects an empty required string, a bad track, a malformed FAQ collection, or a frontmatter slug that differs from the filename.

The route then calls getAllPosts() inside generateStaticParams() and returns objects shaped like { slug: post.slug }. Current Next.js documentation says that, during next build, generateStaticParams runs before the corresponding pages or layouts are generated. In this application the page also exports dynamic = "force-static", so the intended contract is explicit: the known blog inventory participates in static generation.

There are four separate questions hidden in that short path:

1. Did the file system contain every article the release expected?
2. Did the loader accept every file and preserve the correct slug?
3. Did static parameter generation register every accepted post?
4. Did the built application expose every registered route and discovery URL?

A count at only one layer cannot answer all four. Imagine that alpha.md disappears and an accidental alpha-copy.md arrives. The total stays constant. A count assertion passes. Set comparison reports one missing slug and one unexpected slug. The difference is not academic when hundreds of files are edited by concurrent publishing jobs.

The same warning applies to success sampled from one page. Opening the oldest, newest, or featured post proves that one route works. It says nothing about a file whose YAML contains an unescaped colon, a slug that differs by one character, or a relation that points to a non-existent post. Large collections need inventory-shaped assertions because their failure surface is a collection.

File identity deserves its own review when developers work across operating systems. A case-insensitive laptop can make Timeouts.md and a requested lowercase path appear compatible during local work, while a case-sensitive CI file system treats names exactly. The current loader derives the expected slug from the filename and requires frontmatter to match it, which is a good first defense. The HTTP comparison then proves the serialized URL uses the intended spelling. When this fails only in CI, print the exact code points and filenames rather than lowercasing both sides in the test. Normalization would make the test pass by erasing the distinction the deployment still observes.

Publication dates create a different risk. Sorting posts by new Date(publishedAt) affects order, but it does not decide whether a post enters generateStaticParams() in the current loader. A future test that assumes “future date means draft” would invent policy the application does not implement. If editorial scheduling is added, make the eligibility rule explicit in one production function and build fixtures on both sides of the time boundary. Until then, inventory expectations should include every accepted Markdown file.

The module cache improves repeated reads within a loaded module instance, but a test should not turn that implementation detail into a universal performance promise. Build tooling may evaluate work in processes or contexts that do not share arbitrary application memory. Measure the build as a process and test the loader as a function. Do not infer process-wide memory behavior from the existence of postsCache.

Route completeness also differs from page correctness. A generated route can exist and render the wrong article if lookup or parameter handling is broken. Conversely, the loader can parse every post correctly while a deployment serves an older artifact. The test layers below keep those cases separable.

Prove the source inventory before paying for a full build

The fastest failure is the one found before framework compilation. A content audit should parse the exact publishing directory, retain every error, and exit nonzero after reporting the full batch. Stopping at the first bad file forces a slow edit-build-fail loop when several authors made mistakes in the same release.

The following script is deliberately independent of getAllPosts(). It uses the same file format but builds its own expected set. That independence matters. Calling the production loader and then asserting that the loader returned what it returned creates an oracle that cannot detect omission.

// scripts/check-blog-inventory.ts
import fs from "node:fs";
import path from "node:path";
import matter from "gray-matter";

type ParsedPost = {
  file: string;
  slug: string;
  related: string[];
};

const directory = path.join(process.cwd(), "content/blog");
const files = fs
  .readdirSync(directory)
  .filter((file) => file.endsWith(".md"))
  .sort();

const errors: string[] = [];
const posts: ParsedPost[] = [];
const seen = new Map<string, string>();

for (const file of files) {
  const fileSlug = file.replace(/\.md$/, "");
  try {
    const source = fs.readFileSync(path.join(directory, file), "utf8");
    const { data, content } = matter(source);
    const slug = typeof data.slug === "string" ? data.slug.trim() : "";
    const related = Array.isArray(data.related) ? data.related : [];

    if (slug !== fileSlug) {
      errors.push(`${file}: frontmatter slug ${JSON.stringify(slug)} must equal ${fileSlug}`);
    }
    if (typeof data.title !== "string" || data.title.trim() === "") {
      errors.push(`${file}: title must be a non-empty string`);
    }
    if (typeof data.description !== "string" || data.description.trim() === "") {
      errors.push(`${file}: description must be a non-empty string`);
    }
    if (content.trim() === "") {
      errors.push(`${file}: article body is empty`);
    }

    const routeKey = fileSlug.toLowerCase();
    const previous = seen.get(routeKey);
    if (previous) {
      errors.push(
        `${file} and ${previous} both resolve to /blog/${routeKey}; two files may not differ only by letter case`,
      );
    } else {
      seen.set(routeKey, file);
    }

    if (related.some((value) => typeof value !== "string")) {
      errors.push(`${file}: related must contain only strings`);
    } else {
      posts.push({ file, slug, related });
    }
  } catch (error) {
    errors.push(`${file}: ${error instanceof Error ? error.message : String(error)}`);
  }
}

const fileSlugs = new Set(files.map((file) => file.replace(/\.md$/, "")));
for (const post of posts) {
  for (const relatedSlug of post.related) {
    if (!fileSlugs.has(relatedSlug)) {
      errors.push(`${post.file}: related article ${relatedSlug} does not exist`);
    }
  }
}

if (errors.length > 0) {
  console.error(errors.map((error) => `BLOG_AUDIT ${error}`).join("\n"));
  process.exitCode = 1;
} else {
  console.log(`BLOG_AUDIT accepted=${files.length}`);
}

Every assertion in that script has a real mutation that makes it fail. Rename a file without changing its slug, remove a title, add a second file whose name differs from an existing one only by letter case, empty a body, or point related at a missing file. The audit does not congratulate a hard-coded fixture for agreeing with itself.

The collision key deserves a note, because the obvious choice is a check that cannot fire. Keying the map on the frontmatter slug looks natural, and it is useless here. The script has already required slug === fileSlug, and filenames are unique inside one directory, so any input that reaches the duplicate branch must have failed the equality check first and produced a second error on the same line of the report. Worse, the message names whichever file the sort visited first, which can be the correct one. Give alpha.md the wrong slug beta while beta.md is perfectly valid, and the audit prints that beta.md is a duplicate declared by alpha.md, sending the owner to the innocent file. Keying on the case-folded filename removes both problems. Two names that differ only in capitalization are distinct files on a case-sensitive CI file system, cannot both own one lowercase URL, and each of them can be internally valid, so the collision fires on its own with nothing else in the report. Naming both files in one message also blames neither.

Suppose an editor saves retry-policy.md with slug: "retry-policies". The diagnostic identifies the file and both values before Next.js starts. That is more useful than a later generic build failure because the owner can fix the content contract without studying compiler output. If the YAML parser throws, the catch branch reports that file and continues to the next article.

The case-folded key also connects the audit to the file-identity risk described earlier. A developer on a case-insensitive laptop cannot create Timeouts.md and timeouts.md in the same folder, so the mistake is invisible locally and arrives through a merge. Verify the rule the way you would verify any gate: create both files on a case-sensitive volume, give each a matching frontmatter slug, and confirm the audit reports exactly one error and exits nonzero. A version keyed on the frontmatter slug accepts that same directory and reports two healthy articles.

There is a near-miss here. A source audit passing does not prove that React can render the Markdown, that a component imported by the page compiles, or that the build has enough memory. It establishes only that the publishable inventory is internally coherent. Keep its job narrow so a failure remains easy to assign.

Run the audit whenever files under content/blog, the frontmatter schema, heading extraction, or relation rules change. It is cheap enough for every pull request. Record the derived accepted count as context, not as a permanent threshold. A hard-coded “must equal 1000” gate fails the day a legitimate article is retired and can pass after a missing file is replaced by junk.

Keep schema validation and editorial scoring separate. The audit should block an invalid slug or missing relation because the application cannot honor those contracts. It should not reject an article because a sentence-length heuristic or keyword-density score changed unless publishing has deliberately adopted that rule. Mixing subjective content scoring into the route gate makes an infrastructure signal noisy and encourages authors to game the checker. Run quality analysis as a named report with its own owners, then promote only rules that have clear false-positive handling.

Also preserve the file list used by the audit as an artifact when the inventory is assembled outside a normal checkout. If another job generates or copies content before the build, record a checksum per file after that step. A later mismatch can then answer whether the build saw different bytes, not merely whether two jobs reported the same count. This matters when publication pipelines use downloaded artifacts, sparse checkouts, or workspace caching. In a plain git checkout, the commit and clean dependency lock are usually enough, so do not add checksum machinery without that extra boundary.

Compare the built route set with the source set

After the application is built and started, inspect a public artifact assembled from the same release. The sitemap is useful because src/app/sitemap.ts emits a blog URL for every post returned by getAllPosts(). It is not sufficient by itself, because page generation and sitemap generation share that loader. The source side of the assertion must come directly from the files, while the actual side comes through HTTP from the built server.

Use set equality rather than count equality. Preserve duplicates long enough to report them. Compare URL pathnames rather than the local server origin because this repository constructs sitemap entries from the canonical SITE_URL, while Playwright may be talking to 127.0.0.1. The two origins have different jobs.

// e2e/blog-inventory.spec.ts
import fs from "node:fs";
import path from "node:path";
import { expect, test } from "@playwright/test";

const expectedPaths = fs
  .readdirSync(path.join(process.cwd(), "content/blog"))
  .filter((file) => file.endsWith(".md"))
  .map((file) => `/blog/${file.replace(/\.md$/, "")}`)
  .sort();

test("every Markdown article is discoverable in the built sitemap", async ({
  page,
  request,
}) => {
  const response = await request.get("/sitemap.xml");
  expect(response.ok(), `sitemap returned ${response.status()}`).toBe(true);
  expect(response.headers()["content-type"]).toContain("xml");

  const xml = await response.text();
  const locations = await page.evaluate((source) => {
    const document = new DOMParser().parseFromString(source, "application/xml");
    const parserError = document.querySelector("parsererror")?.textContent;
    if (parserError) throw new Error(parserError);
    return Array.from(document.getElementsByTagNameNS("*", "loc"), (node) =>
      node.textContent?.trim() ?? "",
    );
  }, xml);

  const actualPaths = locations
    .map((location) => new URL(location).pathname)
    .filter((pathname) => /^\/blog\/[^/]+$/.test(pathname));
  const duplicates = actualPaths.filter(
    (value, index, all) => all.indexOf(value) !== index,
  );

  expect(duplicates, `duplicate blog paths: ${duplicates.join(", ")}`).toEqual([]);
  expect(actualPaths.sort()).toEqual(expectedPaths);

  const samples = [
    expectedPaths[0],
    expectedPaths[Math.floor(expectedPaths.length / 2)],
    expectedPaths.at(-1),
  ].filter((value): value is string => Boolean(value));

  for (const pathname of new Set(samples)) {
    const article = await request.get(pathname);
    expect(article.ok(), `${pathname} returned ${article.status()}`).toBe(true);
    expect(article.headers()["content-type"]).toContain("text/html");
  }
});

This test can fail in ways a build exit code cannot. A missing path means source inventory and public discovery differ. An unexpected path suggests stale or extra content in the artifact. A duplicate points to route assembly. A sampled HTML request that fails while its sitemap entry exists points to page rendering, routing, or deployment packaging rather than inventory selection.

The three page samples are not a mathematical proof that all pages render. They are a low-cost smoke layer chosen across the sorted slug range. The exact set comparison already covers registration. A scheduled release crawl can request every path with a small concurrency limit to catch article-specific render faults. Avoid launching a thousand simultaneous requests from a fully parallel test, because that measures an artificial burst and can obscure the one route that is malformed.

A complete crawl should validate more than status without duplicating every component test. Require HTML content type, the canonical pathname expected for that slug, and one article-specific identity such as the page heading or metadata title derived from frontmatter. Status alone can accept a friendly not-found page if middleware or hosting rewrites it with 200. A generic site heading alone can accept the blog index. The identity assertion needs to vary with each source file.

Keep the expected title beside the slug when parsing the source inventory. If a page for alpha renders the title from beta, the route exists and the sitemap is complete, but the product result is still wrong. That defect can come from a stale lookup cache, parameter handling, or an artifact assembled from mismatched content. Report the requested slug, expected title, actual title, final URL, and deployment identifier together. Those five fields usually narrow the investigation without attaching the full page.

If every route must be requested, partition the slug list into deterministic chunks and cap active requests. Save the failing pathname, status, content type, and a short response excerpt. Do not dump a thousand complete HTML bodies into CI logs. Useful evidence narrows the bad route without turning the report into another capacity problem.

A second worked example shows why both sides matter. Assume a future loader filters posts without a published flag, but old articles do not have that field. generateStaticParams() and sitemap() would both omit them. A test that compares the sitemap with getAllPosts() passes because both use the same faulty filter. Direct filesystem expectation exposes every omission. The fix then belongs in publishing policy or migration, not Playwright.

Tell a content defect from a capacity failure

Large builds fail at different layers, and the last line in a log is not always the cause. Preserve the command status and the complete build log before classifying anything. On a Linux runner, GNU time can add measured peak resident memory and elapsed time. Those values belong to that run only. They are evidence for comparison, not universal performance claims.

#!/usr/bin/env bash
set -uo pipefail

artifact_dir="artifacts/static-blog-build"
mkdir -p "$artifact_dir"

node --version > "$artifact_dir/toolchain.txt"
pnpm --version >> "$artifact_dir/toolchain.txt"

set +e
/usr/bin/time -v pnpm build \
  > >(tee "$artifact_dir/next-build.log") \
  2> >(tee "$artifact_dir/next-build.stderr.log" >&2)
build_status=$?
set -e

printf 'BUILD_STATUS=%s\n' "$build_status" | tee "$artifact_dir/result.txt"
exit "$build_status"

Treat the resulting evidence in order. A frontmatter exception naming a slug or field is a content rejection. A TypeScript or bundler diagnostic with a source location is a code problem. A nonzero status accompanied by runner telemetry showing that the process was terminated at its memory limit is a capacity failure. On many Linux systems status 137 is consistent with a process receiving SIGKILL, but the number alone does not prove an out-of-memory kill. Check the runner or container event that says why the signal was sent.

Build duration regression is another category. Compare the same command, lockfile, cache condition, runner class, and content set. A cold build and a warm build are not interchangeable samples. Neither are a developer laptop and a constrained hosted runner. Store the conditions beside the duration so a future engineer knows what changed.

One malformed article can resemble a scale threshold because it happens to be the newest file that crossed a round number. Remove that file and the build passes, but the reason may be invalid MDX-like markup or YAML, not “1000 pages is too many.” The source audit and the named build error distinguish the two. Replacing the runner with a larger machine would hide the content defect without fixing it.

The reverse near-miss also occurs. Splitting a large file or deleting a few posts can make a memory-bound build pass. That does not identify a particular article as corrupt. Re-run the same commit on a runner with known capacity, inspect measured peak memory, and look for termination evidence. Capacity diagnosis needs process evidence, not folklore about corpus size.

A deployed 404 after a successful local build is a third look-alike. Request the exact deployment identifier, inspect whether its sitemap contains the slug, and compare the deployed commit with the tested commit. If the local artifact contains the route but production serves an older artifact, rebuilding content code is not the first fix. Deployment promotion or cache invalidation owns that incident.

Watch for partial success when logs are streamed through multiple commands. Without pipefail, a failing build piped into tee can leave the shell reporting the logger's successful exit instead of the build's failure. The diagnostic script enables pipefail and captures the build command directly. Test that wrapper once with a command that intentionally exits nonzero. Otherwise a well-written application test can be undermined by CI plumbing whose oracle observes the wrong process.

Disk exhaustion can resemble memory pressure because both may stop late in compilation and leave an incomplete artifact. Runner metrics, filesystem errors, and available-space telemetry separate them. Do not respond to either condition by retrying until one attempt lands on a less busy machine. A passing retry is useful evidence of environmental variability, but the first attempt remains a failed release candidate and should retain its log and resource context.

Add the gates without building the app twice

The normal pull-request path should fail cheaply, build once, and reuse that built server for route checks. This repository's Playwright configuration already starts its managed server with pnpm build followed by next start. Running pnpm build in one CI step and then invoking that same managed configuration causes another build unless the configuration is changed. Account for that cost explicitly.

A minimal job can run the independent inventory script first and let Playwright own the single build-start-test lifecycle. Environment provisioning and dependency installation can remain in the organization's shared setup; the important wiring is the order and the absence of a duplicate build.

steps:
  - name: Validate the Markdown inventory
    run: pnpm exec tsx scripts/check-blog-inventory.ts

  - name: Build once and verify public blog routes
    run: pnpm exec playwright test e2e/blog-inventory.spec.ts
    env:
      CI: "true"

  - name: Upload build diagnostics after a failure
    if: failure()
    uses: actions/upload-artifact@v4
    with:
      name: static-blog-build-diagnostics
      path: artifacts/static-blog-build/
      if-no-files-found: ignore

If a separate build step is mandatory, point Playwright at an already started server through PLAYWRIGHT_BASE_URL so its configuration does not create another one. The owning shell must start the exact artifact, wait for readiness, preserve server logs, and stop the process. That arrangement is more complex, but it can be worthwhile when several test jobs share one immutable deployment.

Roll this out in stages. First run the inventory audit in report-only mode and fix every existing mismatch. Then make it blocking. Add the sitemap set comparison next, because it has a clear expected set and low request volume. Add representative page requests after the built environment is stable. Finally, schedule the complete crawl and establish a measured budget from several comparable runs.

During migration, tag failures by check rather than placing all assertions in one giant test. source-inventory, sitemap-set, sample-render, full-crawl, and build-capacity are useful identities. A team can temporarily quarantine a noisy full crawl without disabling the frontmatter gate. It can also see that a framework upgrade changed build capacity while route completeness stayed intact. One monolithic red result loses that information and makes temporary exceptions dangerously broad.

Changes to the Markdown parser deserve the widest content sample because every article passes through that code. A CSS-only change usually needs the cheap inventory gate plus representative rendering, not a fresh semantic audit of every body. A change to generateStaticParams, getAllPosts, the sitemap, canonical URL construction, or deployment packaging should trigger the exact set comparison and complete crawl. This risk-based selection keeps the ordinary path fast while spending time where one line can affect the entire corpus.

Do not set a build-duration gate from one observation. Collect real durations under controlled conditions, choose a percentile or headroom policy that matches release needs, and document how cache hits are treated. When the threshold fires, publish the observed value, baseline window, runner type, and content count. “Build got slower” is not actionable evidence.

Ownership should follow the failed layer. Content schema failures go to the publishing change. Route-set differences go to loader or sitemap owners. Compilation errors go to the referenced code owner. Confirmed memory termination goes to the build platform and application team together. A stale deployed artifact goes to release engineering. One generic “static blog failed” alert sends all five groups searching in the dark.

Know when full static generation is the wrong contract

Do not optimize away static generation merely because the corpus looks large in a spreadsheet. A thousand simple pages may fit comfortably, while a much smaller set of pages that performs expensive data work may not. Measure the current pipeline. The inputs that matter include per-page computation, shared data loading, source size, framework version, cache state, runner resources, and how often content changes.

Full build-time generation is a strong choice when every article must be ready at deployment, releases are reasonably infrequent, and a failed article should block the whole artifact. Its cost is coupled release latency: one content change can make the pipeline revisit a large route set, and the release needs capacity for the high-water mark.

Generating only a subset at build time or producing pages later can reduce deployment work. The cost moves elsewhere. A first request may do more work, caches need an explicit freshness policy, and QA must cover cold as well as warm behavior. Next.js exposes route-generation options, but selecting one is a product and operations decision. Do not copy a configuration flag into an existing site without checking how unpublished paths, revalidation, and failures are supposed to behave in the installed framework version.

Server rendering can decouple content volume from build duration, yet it introduces request-time dependencies and latency. A database or content service outage can now affect readers rather than only the release. Tests must cover availability, caching, and stale-content policy. The static approach avoided those request-time failure modes by paying earlier.

Client-side loading is not a free escape either. It changes discoverability, initial rendering, error handling, and the amount of JavaScript sent to readers. A blog whose main value is readable content usually has little reason to hide the article behind a browser fetch solely to shorten CI.

Avoid a full route crawl on every small code change when the sitemap set and targeted render tests already protect the altered boundary. The complete crawl costs server time and report volume. Put it on release candidates, scheduled runs, or changes to the parser and page renderer. Keep a smaller deterministic gate on every pull request so failures arrive while the change is still fresh.

Finally, do not raise memory and timeout limits without recording what that buys. More capacity can be the right fix for a healthy, growing static workload. It also increases cost and delays the next threshold. Pair the change with measured before-and-after evidence, a projected content range, and the same completeness assertions. Faster failure is useful, but a slower pipeline that provably publishes every intended article is still better than a quick green build with a hole in the route set.