Test MCP stdio Servers for stdout Corruption

MCP stdio stdout corruption testing proves that a local server reserves stdout for valid, UTF-8, newline-delimited MCP messages. A startup banner, debug print, stack trace, blank line, embedded newline, or partial frame can break a client parser. Capture stdout and stderr independently, then validate every stdout frame before testing higher-level capabilities.

The current MCP stdio transport specification states the transport contract directly: a client launches the server as a subprocess, the peers exchange JSON-RPC messages through stdin and stdout, messages are delimited by newlines and contain no embedded newlines, and a server must not write non-MCP content to stdout. The transport and handshake challenges in src/db/seed-data/challenges/expansion-mcp-ai.ts make that rule the repository evidence for a process-level test.

What is MCP stdio stdout corruption?

Corruption is any stdout byte sequence that cannot be consumed as the next MCP message under the stdio framing contract. The server may be functioning internally and the printed text may look harmless to a developer. At the pipe boundary, however, every stdout line is protocol data.

Typical corruption sources include:

• A framework banner printed during module import or process startup.
• print, console.log, or equivalent debug output sent to stdout.
• Pretty-printed JSON that spans several lines.
• A stack trace written by a default exception handler.
• A warning emitted by a dependency before the server initializes.
• A partial message followed by a hang or process exit.
• Bytes that are not valid UTF-8.

The first invalid frame can prevent initialization, tool discovery, or any useful error response. That is why this topic is narrower than testing MCP servers. A full protocol suite can report missing capabilities when the actual defect happened earlier: the client never decoded the server's response.

Do not "repair" frames in the test harness by skipping unknown lines or joining pretty-printed fragments. A diagnostic proxy may expose a tolerant mode for local investigation, but the conformance test should report the first invalid line exactly where it occurred.

Why must stdout contain only valid MCP messages?

Stdio has no separate channel marker inside stdout. The newline is the frame boundary, and the line is expected to contain one complete JSON-RPC message. A client cannot reliably decide that one line is "just a log" while another is a protocol response without parsing both. Guessing based on a leading brace or known phrase creates an undocumented alternate protocol.

The server can write UTF-8 logs to stderr. That independent pipe preserves operator diagnostics without mixing them into MCP framing. The client may capture, forward, or ignore stderr, and should not assume that any stderr text means the protocol operation failed. The official MCP debugging guide repeats the practical rule for local stdio servers: do not log to stdout.

This boundary has two testable consequences. First, a valid response on stdout does not become invalid because stderr contains a warning. Second, a clean stderr stream does not excuse one banner on stdout. Keep evidence from both pipes, but apply different assertions to them.

Capability negotiation begins only after framing works. MCP protocol-version negotiation tests cover initialization fields and supported versions. A stdout probe should run before those tests or wrap them so a parse failure is classified as transport corruption rather than capability absence.

Which output mistakes break stdio clients?

Use a matrix that states the expected parser outcome. This prevents a helper from slowly becoming more permissive as new incidents appear.

The blank-line rule is worth making explicit. A blank line is not a JSON-RPC message. Some readers may choose to ignore it, but a portable server should not depend on client tolerance that is absent from the transport contract.

Embedded newlines are different from a normal terminating newline. The final newline separates messages. A newline inside serialized JSON splits one message into multiple frames. JSON string serializers escape line breaks inside values as characters such as \n; they do not insert a literal framing newline into the line.

For upgrades and authentication failures on other transports, use MCP protocol upgrade and OAuth scenarios. Those failures should not be mixed into an stdio framing matrix.

How do you launch an MCP server as a test subprocess?

Launch the real entry command with stdin, stdout, and stderr connected to separate pipes. Pass a minimal environment rather than inheriting every CI secret. Avoid shell interpolation when arguments can be supplied as a list. Record the executable version and sanitized arguments for diagnosis.

from __future__ import annotations

import asyncio
import json
import os
from dataclasses import dataclass, field

@dataclass
class McpProcess:
    child: asyncio.subprocess.Process
    stdout_buffer: bytearray = field(default_factory=bytearray)

async def start_server(command: list[str]) -> McpProcess:
    env = {
        "PATH": os.environ["PATH"],
        "PYTHONUNBUFFERED": "1",
        "MCP_TEST_MODE": "1",
    }
    child = await asyncio.create_subprocess_exec(
        *command,
        stdin=asyncio.subprocess.PIPE,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
        env=env,
    )
    return McpProcess(child)

async def send_message(process: McpProcess, message: dict) -> None:
    assert process.child.stdin is not None
    frame = json.dumps(message, separators=(",", ":")).encode("utf-8") + b"\n"
    process.child.stdin.write(frame)
    await process.child.stdin.drain()

Asynchronous binary pipes let the harness detect invalid UTF-8 without blocking the test runner or accepting decoder replacement characters. They are supported by Python's subprocess event-loop implementation on the major operating-system families, avoiding the platform limits that synchronous selectors can have with Windows pipes. Decode each complete frame with strict error handling. The test command must point to the same packaged entry path used by real clients; testing only an in-process handler can miss import banners and default process exception behavior.

Startup ownership also matters. Do not wait forever for a response without checking whether the child exited. A bounded reader should report exit-before- frame, timeout-with-partial-frame, invalid frame, and valid response as different terminal states.

Playwright MCP server setup addresses a specific browser automation server. The subprocess pattern here is transport-focused and applies before any browser permission or tool safety assertion.

How do you validate newline-delimited JSON-RPC frames?

Read one bounded line from stdout, retain the raw bytes under an artifact cap, remove exactly the trailing framing newline, and reject an empty payload. Then decode strict UTF-8, parse one JSON value, and validate the JSON-RPC shape against the applicable MCP schema.

import asyncio
import json

MAX_FRAME_BYTES = 1_048_576

def _is_jsonrpc_id(value: object) -> bool:
    return isinstance(value, (str, int)) and not isinstance(value, bool)

def validate_jsonrpc_envelope(payload: bytes) -> dict:
    if not payload:
        raise ValueError("Blank stdout frame")

    text = payload.decode("utf-8", errors="strict")
    message = json.loads(text)
    if not isinstance(message, dict):
        raise ValueError("JSON-RPC message must be an object")
    if message.get("jsonrpc") != "2.0":
        raise ValueError("Missing JSON-RPC 2.0 marker")

    has_method = "method" in message
    has_result = "result" in message
    has_error = "error" in message

    if has_method:
        if has_result or has_error:
            raise ValueError("Request or notification cannot contain result/error")
        if not isinstance(message["method"], str):
            raise ValueError("JSON-RPC method must be a string")
        if "params" in message and not isinstance(message["params"], dict):
            raise ValueError("MCP params must be an object")
        if "id" in message and not _is_jsonrpc_id(message["id"]):
            raise ValueError("MCP request ID must be a string or integer")
        return message

    if has_result == has_error:
        raise ValueError("Response must contain exactly one of result or error")
    if "id" not in message:
        raise ValueError("JSON-RPC response is missing id")
    if message["id"] is not None and not _is_jsonrpc_id(message["id"]):
        raise ValueError("Response ID must be string, integer, or null")
    if has_result and message["id"] is None:
        raise ValueError("Successful response cannot use a null ID")
    if has_error:
        error = message["error"]
        if not isinstance(error, dict):
            raise ValueError("JSON-RPC error must be an object")
        if not isinstance(error.get("code"), int) or isinstance(
            error.get("code"), bool
        ):
            raise ValueError("JSON-RPC error code must be an integer")
        if not isinstance(error.get("message"), str):
            raise ValueError("JSON-RPC error message must be a string")
    return message

async def _read_next_frame(process: McpProcess) -> dict:
    assert process.child.stdout is not None

    while True:
        newline_at = process.stdout_buffer.find(b"\n")
        if newline_at >= 0:
            if newline_at > MAX_FRAME_BYTES:
                raise ValueError("Stdout frame exceeds the configured byte cap")
            payload = bytes(process.stdout_buffer[:newline_at])
            del process.stdout_buffer[: newline_at + 1]
            return validate_jsonrpc_envelope(payload)

        if len(process.stdout_buffer) > MAX_FRAME_BYTES:
            raise ValueError("Stdout frame exceeds the configured byte cap")

        chunk = await process.child.stdout.read(4096)
        if not chunk:
            if process.stdout_buffer:
                raise EOFError(
                    f"EOF with {len(process.stdout_buffer)} partial frame bytes"
                )
            raise EOFError(f"Server exited with {process.child.returncode}")
        process.stdout_buffer.extend(chunk)

async def read_frame(process: McpProcess, timeout_s: float) -> dict:
    try:
        return await asyncio.wait_for(_read_next_frame(process), timeout_s)
    except asyncio.TimeoutError as exc:
        partial = len(process.stdout_buffer)
        raise TimeoutError(
            f"No complete stdout frame before deadline; partial_bytes={partial}"
        ) from exc

The buffer consumes arbitrary byte chunks and searches for the framing newline itself, so one readable byte cannot turn into an unbounded readline() call. asyncio.wait_for owns the complete-frame deadline, including the partial-line case. The validator rejects non-object JSON, mixed response members, malformed error objects, invalid IDs, and request fields that do not satisfy the MCP envelope boundary.

This validator is still a diagnostic boundary, not a complete replacement for an MCP SDK. The official MCP schema defines message fields beyond these basic guards. Use the production SDK for complete semantic validation, while retaining the raw-frame probe to locate contamination before SDK parsing.

Correlate response IDs with requests. A valid JSON object with the wrong ID is not stdout corruption, but the same harness can classify it as a protocol correlation failure. Keep transport framing, JSON-RPC shape, MCP semantics, and request lifecycle as distinct diagnostic layers.

How do you separate stdout protocol data from stderr logs?

Drain both pipes without merging them. If stderr fills its operating-system pipe while the test reads only stdout, the child may block and look like a protocol timeout. A practical harness uses reader threads, asynchronous streams, or selectors appropriate to its runtime so neither pipe is neglected.

Apply a separate stderr policy:

• Decode as UTF-8 when the server contract promises UTF-8 logging.
• Cap retained bytes to avoid unbounded artifacts.
• Redact tokens, credentials, user content, paths, and environment values.
• Record chronology relative to requests and stdout frames.
• Do not parse stderr lines as JSON-RPC messages.
• Do not fail merely because a permitted diagnostic line exists.

The assertion should still catch a log accidentally routed to stdout. Configure the test server to emit a known canary log at startup. Prove the canary appears only in stderr and the initialize response remains the first stdout frame.

This separation is a useful precursor to MCP server safety and compliance evaluation. Transport evidence can itself contain sensitive values, so collection and redaction must be designed together.

How do you reproduce startup banners and stack-trace leaks?

Build tiny fixture servers that intentionally violate one rule at a time. Do not rely on production dependencies to keep producing an accidental banner. One fixture prints text before reading stdin. Another pretty-prints a response. A third raises an uncaught exception after receiving a request. A fourth writes the same exception summary to stderr and a valid error response to stdout.

import pytest

@pytest.mark.parametrize(
    ("mode", "expected"),
    [
        ("debug-print", "invalid-json"),
        ("startup-banner", "invalid-json"),
        ("pretty-json", "split-message"),
        ("blank-line", "empty-frame"),
        ("invalid-utf8", "decode-error"),
        ("stderr-log", "valid-stdout"),
    ],
)
def test_stdio_corruption_fixture(mode, expected, probe_fixture_server):
    evidence = probe_fixture_server(mode)
    assert evidence.classification == expected
    assert evidence.child_reaped is True

Make each fixture executable through the same process launcher as the real server. The useful assertion is not only that an exception occurred. Verify the frame index, raw-byte classification, request phase, stderr separation, and cleanup result.

When a real server fails, preserve the smallest reproduction command and first offending frame. Debugging MCP tools missing after capability negotiation becomes relevant only if initialization and subsequent frames are valid.

How should the harness handle hangs and partial frames?

Use deadlines for startup, each expected response, and process shutdown. A single broad test timeout cannot explain whether the child never started, received no request, wrote half a frame, blocked on stderr, or ignored termination. Record the phase when the deadline expires.

The controlled workflow is:

1. Start the subprocess with separate binary pipes.
2. Send a valid initialize request ending in one newline.
3. Read stdout with a bounded deadline and artifact cap.
4. Decode the complete frame as strict UTF-8.
5. Parse exactly one JSON value from the frame.
6. Validate JSON-RPC and applicable MCP message shape.
7. Correlate response ID with the initialize request.
8. Drain and sanitize stderr independently.
9. Close input, terminate when necessary, wait, and reap the child.

For a partial line, report the bytes collected before EOF or deadline without trying to parse them as complete JSON. Cap the excerpt and include a hash when the full payload is too large or sensitive. If the process remains alive, request normal termination first, then apply the test runner's controlled force policy.

A dropped or hung process is not permission to leave it running. Teardown should execute in a finally block and itself produce an assertion or evidence field. Leaked subprocesses can consume ports, files, or credentials and turn a single framing failure into unrelated CI noise.

Which stdio corruption checks belong in CI?

Run the packaged server smoke probe on changes to entry points, logging, dependencies, exception handling, serialization, or transport code. Send initialize, a representative discovery request, and a controlled invalid request. Validate every stdout frame and prove a known stderr log remains separate.

Add fixture-based negative cases for banner text, debug output, multiline JSON, blank lines, invalid UTF-8, partial frames, stack traces, and a hung process. The fixture suite tests the harness itself. The packaged-server smoke test finds real import and configuration regressions.

CI evidence should retain the case, process exit status, frame number, sanitized offending excerpt, classification, stderr excerpt, timeout phase, and cleanup result. Contract testing tool schema evolution can run after framing passes. MCP list-changed subscription tests likewise depend on a healthy transport before notification semantics can be trusted.

Do not reuse these exact framing assertions for Streamable HTTP. That transport has HTTP and streaming rules rather than subprocess stdout framing. JSON-RPC message-shape helpers may be shared, but the ownership of boundaries and failure evidence differs.

Probe the packaged entry point, not only source code

Stdout contamination often appears outside the server handler. A package manager can print a warning, a runtime loader can announce configuration, or a dependency can log during import. An in-process unit test that calls the request handler bypasses all of those stages. Keep the unit test, but add a process probe that launches the distributed command exactly as a client does.

Build or install the package before the probe when production clients execute built artifacts. Record the artifact version and entry command, then compare the environment with the real installation path. Test development and production logging configurations separately if they select different sinks. The server should remain conformant in either mode; debug mode is not permission to use protocol stdout.

Run the probe from a working directory that does not contain the source tree when feasible. That catches accidental relative imports and configuration files that can change startup output. Pass only required environment variables. A test that inherits a developer's logging override may pass locally and fail in a clean client process.

Attribute the first bad byte

The first invalid byte or frame matters more than the cascade that follows. A banner can make the client report a JSON parser error; the server then keeps writing valid messages that the client no longer correlates; teardown finally produces a broken-pipe stack trace. Saving only the last exception points at the wrong component.

Give each raw stdout frame a sequence number and monotonic observation time. When decoding fails, record a bounded hexadecimal prefix as well as a sanitized text representation. Hex evidence distinguishes invalid UTF-8, a byte-order marker, terminal color codes, and ordinary ASCII text without asking the reporter to paste unrestricted output.

Do not continue the conformance run after the first corrupt frame as if later responses were trustworthy. The harness can drain capped output for diagnosis, but the terminal classification should remain the first violation. This makes incident comparisons stable even when later logging changes.

Test backpressure and pipe ownership

A server that writes large stderr logs can block if the client never drains stderr. That is a client harness defect even though stderr logging is allowed. Create a fixture that emits bounded but substantial stderr content while returning one valid stdout response. The harness should consume both streams and correlate the response before its deadline.

The reverse condition matters too. A server should not depend on a client reading stdout forever after the client has closed the session. Close stdin, stop readers under controlled teardown, and assert the child exits or responds to termination according to the test policy. Keep pipe closure errors out of the protocol channel.

If reader threads are used, give them owned shutdown signals and join them. If asynchronous tasks are used, cancel and await them. A child process can be reaped while a reader remains blocked, leaving test-runner resources behind. Record reader cleanup separately from process cleanup so a green exit code cannot hide a leaking harness.

Distinguish framing, schema, and lifecycle failures

A line can be valid JSON but invalid JSON-RPC. It can be valid JSON-RPC but invalid for the selected MCP schema. It can satisfy message shape but arrive with the wrong request ID or before initialization. Report these as separate layers:

1. Byte decoding and newline framing.
2. JSON syntax.
3. JSON-RPC message shape.
4. MCP schema and method semantics.
5. Request correlation and session lifecycle.

That order makes ownership visible. Adding a missing jsonrpc field is not a newline fix. Correcting a response ID is not a UTF-8 fix. A diagnostic should name the lowest failing layer and attach higher-level checks only when the message reached them.

Use the same layered explanation in code reviews and interviews. The broader MCP testing questions for agentic systems help teams practice the protocol boundary, while the executable probe supplies evidence for this specific failure.

Add release-time installation tests

Package changes can reintroduce banners even when application source is unchanged. Run the smoke probe against the release artifact after installation in a clean temporary directory. Send initialize and at least one known request, then close the session and prove the process is reaped.

Test the documented command on each supported runtime and operating-system family in the project's actual support matrix. Do not claim portability beyond those executions. Store only sanitized failure frames, package version, runtime version, and command metadata.

Finally, add a negative control to the CI job itself. Launch the intentional banner fixture and require the probe to fail with the expected first-frame classification. If that fixture starts passing, the harness has become tolerant and the packaged-server result no longer proves stdout purity.

Keep the probe output readable for developers who do not know the transport internals. Report that stdout line one was reserved for an MCP message, show the sanitized classification, and point to stderr as the logging channel. Include the command and runtime version without environment secrets. A precise report shortens diagnosis without teaching the harness to ignore the offending line.

After the repair, rerun both the original packaged command and the intentional negative fixture. The first must pass and the second must continue to fail. That pair proves the server was corrected while the detector retained its ability to catch contamination.

FAQ

Can an MCP server log to stderr?

Yes. The stdio transport permits a server to write UTF-8 log text to stderr, and clients may capture, forward, or ignore it. Tests should keep stderr separate from protocol stdout, avoid treating every stderr line as a protocol failure, and verify sensitive values are redacted before retention.

Are blank stdout lines allowed?

A blank stdout line is not a valid MCP JSON-RPC message, so a strict corruption probe should report it rather than silently accepting contaminated protocol output. If an SDK reader ignores blanks, keep a lower-level process test that detects them because another client may not share that tolerance.

Can JSON-RPC messages contain embedded newlines?

Not on MCP stdio. Messages are newline-delimited and must not contain embedded newline characters. Serialize each JSON-RPC message on one line and terminate it with a newline. Tests should reject pretty-printed or split payloads even when concatenating the text would produce valid JSON.

How should tests stop a hung MCP server?

Use bounded reads and a deadline owned by the test. On expiry, record sanitized stdout and stderr evidence, close stdin when appropriate, request normal termination, wait for a controlled period, and force termination only if needed. Always reap the child process.

Does Streamable HTTP have the same stdout rule?

No. The stdout restriction belongs to local stdio where standard output carries MCP messages. Streamable HTTP has separate HTTP request, response, session, and streaming rules. Share JSON-RPC shape assertions where applicable, but do not reuse newline framing or subprocess-pipe assumptions as HTTP requirements.

What evidence should CI retain?

Retain the case name, sanitized command, exit status, request ID, frame number, offending bytes or text, parser classification, stderr excerpt, timeout phase, and cleanup result. Cap artifact size and redact before upload. Evidence should identify the first corrupt frame without publishing credentials or full sessions.

Conclusion

An stdio MCP server has a simple but unforgiving channel contract: stdout is protocol data, stderr is the logging channel, and one newline ends one complete UTF-8 JSON-RPC message. Validate that contract at the process boundary before trusting SDK-level errors about initialization, tools, or capabilities.

A useful corruption test does more than catch invalid JSON. It identifies the first bad frame, separates stderr, bounds hangs and partial reads, correlates request IDs, and proves the child was reaped. That evidence turns a vague "server disconnected" report into a specific transport defect.