| |

MCP Server Test Plan for Tool-Calling Agents

ByPromode
MCP server test plan for tool-calling agents covering contracts, auth, transports, and observability

MCP server test plan work is no longer optional for teams building tool-calling agents. The Model Context Protocol has moved fast enough that a server can look healthy in a demo, then fail under a real agent because discovery, auth, streaming, schema validation, or observability was never tested as a contract.

I see the same pattern in QA teams adopting agent tooling: the agent gets the attention, while the MCP server becomes a thin integration layer nobody owns. That is risky. A broken tool response can create a false production change, leak an internal resource, or turn a simple support workflow into a retry storm.

Table of Contents

Contents

Why an MCP Server Test Plan Matters in 2026

Agents fail differently from normal APIs

A normal REST client usually calls one endpoint with a predictable payload. A tool-calling agent behaves differently. It discovers capabilities, chooses a tool, sends model-shaped arguments, interprets structured content, and may ask the server for another round trip before it completes the user task.

That means a pass on one happy-path curl command tells you very little. Your MCP server needs tests for discovery, schema contracts, malformed arguments, permission boundaries, transport behavior, and telemetry. The agent is only as safe as the server contract it receives.

The adoption signal is already visible

The numbers are not tiny anymore. The official modelcontextprotocol/typescript-sdk GitHub repository showed 13,103 stars when I checked it for this article. The modelcontextprotocol/python-sdk repository showed 23,937 stars. The npm downloads API reported 194,785,321 downloads for @modelcontextprotocol/sdk in the last month ending 2026-08-06.

Those numbers do not prove every download is production usage. They do prove that MCP is not a small side experiment anymore. If your QA team waits until the first incident to write tests, you are late.

A test plan gives the server an owner

The biggest benefit of an MCP server test plan is ownership. Instead of saying “the agent failed,” you can isolate the problem to one layer:

  • Tool schema changed without a versioned contract.
  • Auth middleware allowed a forbidden resource.
  • Streaming transport dropped keep-alive frames.
  • Server returned text where structured content was expected.
  • Observability missed the tool name, session ID, or error class.

That language is useful in standups. It is also useful in incident reviews, release gates, and SDET interviews.

Release Context: TypeScript SDK 1.30.0 and Python SDK 2.0.0

What changed in the TypeScript SDK release

The TypeScript SDK 1.30.0 release was published on 2026-07-27. The release notes list practical changes QA engineers should care about: an end-to-end test suite, a v1 stdio buffer limit, Content-Type validation by parsed media type, Server-Sent Events keep-alive fixes, and a dependency update around GHSA-frvp-7c67-39w9.

That is not just maintenance noise. Each item maps to a test category. Content-Type validation maps to negative HTTP tests. SSE keep-alive fixes map to long-running stream checks. The stdio buffer limit maps to large payload and backpressure tests.

What changed in the Python SDK release

The Python SDK v2.0.0 release was published on 2026-07-28. Its release notes say v2 supports the 2026-07-28 revision of MCP, serves earlier revisions from the same server, and makes pip install mcp install the 2.x line. The notes also say the v1.x line is in maintenance mode and should use a requirement such as mcp>=1.28,<2 if a project is not ready to migrate.

For QA, that means version compatibility is not theoretical. You need a matrix. Test a v2 server with old clients, new clients, stdio, Streamable HTTP, and auth-enabled routes. If a team is migrating from v1, test both server behavior and dependency constraints.

The specification is the source of truth

Do not treat blog posts as the contract. Use the official Model Context Protocol specification, the transport documentation, the authorization documentation, and the tools documentation as your baseline.

I use release notes to decide what changed. I use the spec to decide what must never break.

Contract Tests for Tools, Resources, and Prompts

Start with discovery

The first test in every MCP server suite should be discovery. If the agent cannot discover a tool, the workflow never starts. If it discovers the wrong schema, the model sends arguments your server cannot process.

Write contract assertions for every tool:

  • Tool name is stable and descriptive.
  • Description explains when to use the tool.
  • Input schema marks required fields correctly.
  • Enums reject unknown values.
  • Response content uses the shape your agent expects.
  • Dangerous tools include explicit guardrails.

This is similar to OpenAPI contract testing, but the consumer is an agent. That changes the failure mode. A vague tool description can be as damaging as a wrong JSON type.

Test tool calls as product behavior

Here is a TypeScript-style Playwright API test I would put in the first commit. It is intentionally boring. Boring tests catch expensive production bugs.

import { test, expect, request } from '@playwright/test';

test('MCP tool discovery exposes stable QA tools', async ({ playwright }) => {
  const api = await request.newContext({
    baseURL: process.env.MCP_BASE_URL,
    extraHTTPHeaders: { Authorization: `Bearer ${process.env.MCP_TOKEN}` }
  });

  const response = await api.post('/mcp', {
    data: {
      jsonrpc: '2.0',
      id: 'tools-1',
      method: 'tools/list',
      params: {}
    }
  });

  expect(response.status()).toBe(200);
  const body = await response.json();
  const tools = body.result.tools.map((tool: any) => tool.name);

  expect(tools).toContain('create_bug_report');
  expect(tools).toContain('search_test_cases');
});

This test does not prove your server is perfect. It proves the discovery surface did not silently disappear. Add one test like this per domain-critical tool.

Validate structured output, not just status code

A 200 response can still be a failure. Your test should assert the response shape, content type, and error semantics. For a tool that creates a test case, check the returned ID format, title, priority, and trace link. For a tool that searches logs, check pagination and empty results.

I also recommend snapshotting the public tool contracts. Not the full response with timestamps. Snapshot names, descriptions, input schemas, and dangerous permissions. A small schema diff in pull request review is easier to discuss than a broken agent workflow after deployment.

Auth and Negative Cases That Catch Real Bugs

Auth is part of the contract

The MCP authorization documentation exists for a reason. Tool servers often sit near internal systems: Jira, GitHub, test management, databases, CI logs, cloud dashboards, and customer support data. If auth is wrong, the agent becomes a friendly interface to a security bug.

Minimum auth tests should cover:

  1. No token returns 401.
  2. Expired token returns 401 with a safe error.
  3. Valid token without scope returns 403.
  4. Valid token with read scope cannot call write tools.
  5. Tenant A token cannot read Tenant B resources.
  6. Tool descriptions do not expose secrets in error text.

This is where many AI demos become production liabilities. The demo tests the happy path with an admin token. Production users do not all have admin permissions.

Negative cases should be first-class tests

Negative tests are not optional for MCP servers. Agents generate imperfect inputs. Users paste messy prompts. LLMs infer values that look plausible but are invalid. Your server must reject bad arguments deterministically.

Here is a compact Playwright test for malformed arguments:

test('create_bug_report rejects invalid severity', async ({ request }) => {
  const response = await request.post('/mcp', {
    headers: { Authorization: `Bearer ${process.env.MCP_TOKEN}` },
    data: {
      jsonrpc: '2.0',
      id: 'bad-severity',
      method: 'tools/call',
      params: {
        name: 'create_bug_report',
        arguments: {
          title: 'Checkout button freezes',
          severity: 'very very bad'
        }
      }
    }
  });

  const body = await response.json();
  expect(body.error.code).toBeDefined();
  expect(JSON.stringify(body)).not.toContain(process.env.MCP_TOKEN!);
});

The second assertion matters. Error handling is a data leak vector. A test that checks secrets are absent can save you from an embarrassing incident.

Abuse tests belong in QA, not only security

SDETs do not need to replace the security team, but they should own repeatable abuse checks. Send large payloads. Send unknown tool names. Send HTML where JSON is expected. Send a valid request twice and verify idempotency. Send parallel calls and check race conditions.

If your MCP server writes to a ticketing system, add a duplicate prevention test. If it triggers CI, add a test that a user cannot trigger a protected deployment workflow. If it reads logs, test redaction on stack traces.

Transport, Streaming, and Version Compatibility Checks

Do not test only one transport

MCP servers commonly support stdio and Streamable HTTP. The TypeScript SDK 1.30.0 notes mention a v1 stdio buffer limit and Streamable HTTP SSE keep-alive fixes. Those release details are a strong hint: transport bugs happen in the boring layer.

Your transport matrix can start small:

  • stdio startup and shutdown.
  • stdio large response near configured buffer limits.
  • HTTP Content-Type validation for valid and invalid media types.
  • SSE keep-alive behavior during long-running tool calls.
  • Client disconnect cleanup.
  • Retry behavior after network interruption.

Run this matrix in CI for every server change. Run the long stream tests nightly if they slow down pull requests.

Version compatibility needs explicit cases

The Python SDK v2.0.0 release notes say one SDK serves the 2026-07-28 revision and earlier revisions from the same server. That is convenient for migration. It also creates a testing responsibility.

Create a compatibility table before the first rollout:

  • Python SDK v2 server with new client: must pass.
  • Python SDK v2 server with 2025-era client: must pass or fail with documented limits.
  • TypeScript SDK client against Python server: core tool flows must pass.
  • Stdio client against HTTP-only deployment: must fail clearly.
  • Unsupported protocol revision: must return a safe, actionable error.

This table is not busywork. It turns migration anxiety into visible risk. Managers understand a matrix with green and red cells.

Content-Type bugs deserve their own tests

One TypeScript SDK 1.30.0 change validates Content-Type by parsed media type instead of substring matching. I like this detail because it is exactly the kind of bug that slips past casual API testing.

Add tests for application/json, application/json; charset=utf-8, missing Content-Type, text/plain, and a misleading value such as text/application-json-ish. The expected behavior should be documented in the test name, not hidden inside a helper.

Observability and CI Gates for MCP Servers

Logs must answer five questions

Observability is part of the MCP server test plan because agent failures are hard to debug after the fact. A failed tool call should answer five questions without opening a debugger:

  • Which tool was called?
  • Which client or session called it?
  • Which protocol revision and transport were used?
  • Which validation rule failed?
  • Was the failure safe, retried, or user-visible?

Do not log full prompts or secrets by default. Log identifiers, counts, durations, and error classes. Your test suite should assert that sensitive values are redacted.

Turn the plan into CI gates

I like a three-level gate for MCP servers:

  1. Pull request gate: discovery, schemas, critical tools, auth negative tests.
  2. Nightly gate: transport matrix, load, retries, long-running SSE checks.
  3. Release gate: backward compatibility, dependency audit, staged production smoke.

This structure works because every test has a job. Pull request tests stay fast. Nightly tests catch timing issues. Release tests protect migration and customer-facing behavior.

If your team already runs Playwright API tests, reuse that foundation. ScrollTest has related playbooks on analytics event verification in Playwright, PromptFoo regression gates, and MCP Python SDK 2.0 validation. The mechanics are familiar: arrange data, call API, assert contract, report risk.

Add release-note watchers

The official SDK releases are useful test triggers. When the TypeScript SDK adds an end-to-end suite or fixes SSE keep-alive behavior, that tells you what the maintainers had to harden. When the Python SDK marks v1 as maintenance mode, that tells you dependency constraints need review.

A practical release-note watcher creates a ticket with:

  • SDK name and version.
  • Release date.
  • Changed transport, auth, or schema behavior.
  • Impacted MCP servers in your org.
  • Required tests before upgrade.

This is a good fit for QA-owned workflow automation. It keeps the team ahead of breakage instead of reacting to it.

India SDET Context: Who Should Own This?

This is a senior SDET skill

In India, I see more job descriptions asking for API automation, CI ownership, observability, and AI tooling in the same role. The strong SDET in 2026 is not just writing selectors. They are validating systems where LLMs call tools that touch real products.

For engineers targeting ₹25-40 LPA product-company roles, MCP testing is a strong portfolio topic. Build one small server, add a contract suite, publish the matrix, and explain the failure modes. That is more impressive than another login test framework.

Manual testers can enter through contracts

You do not need to start with LLM internals. Start with the server contract. Learn JSON-RPC basics, HTTP status codes, auth scopes, schema validation, and logs. Then add Playwright API tests. Then add CI.

This path is realistic for manual testers moving toward automation. It gives you concrete artifacts: a test plan, a GitHub repo, a CI badge, and a short write-up. The Testing Academy audience understands this because the skill stack is visible, not magical.

Managers should not bury this under platform teams

If a platform team owns the MCP server but QA owns the release risk, create a shared contract. Platform writes the server. QA owns the tests and release gates. Security reviews scopes and audit logs. Product defines safe user-facing behavior.

That split is simple enough to work in TCS, Infosys, service companies, and product teams. The difference is speed. Product companies may ship the gate in one sprint. Large services accounts may need a formal change request. The test plan stays the same.

Implementation Playbook: A 7-Day MCP Test Sprint

Day 1 and 2: Inventory and risk mapping

List every MCP server, transport, exposed tool, backing system, and owner. Mark tools as read, write, destructive, or external-facing. A tool that reads a public catalog is low risk. A tool that creates Jira tickets, runs CI, or queries customer data is high risk.

By the end of Day 2, you should have one spreadsheet or markdown file with at least these columns: server name, tool name, transport, auth scope, owner, risk level, log source, and release cadence.

Day 3 and 4: Contract and auth tests

Add discovery tests, tool schema snapshots, required-field checks, invalid enum checks, and authorization tests. Keep the first suite small. Ten high-value tests are better than 70 shallow assertions nobody trusts.

Use this command pattern in CI:

MCP_BASE_URL=https://mcp-staging.example.com \
MCP_TOKEN=$MCP_TEST_TOKEN \
npx playwright test tests/mcp-contract.spec.ts --reporter=line

Store tokens in CI secrets. Never commit them. Add a test that a fake token fails.

Day 5: Transport and compatibility

Add stdio startup tests if you support local clients. Add HTTP Content-Type tests if you expose Streamable HTTP. Add one long-running tool call to check keep-alive behavior and cleanup. If you support old clients during migration, add the exact client versions to the matrix.

Do not aim for perfect load testing on Day 5. Aim for a repeatable signal that catches obvious transport regressions before release.

Day 6 and 7: Observability and release gate

Connect test failures to logs. For one forced validation error, verify a structured log event exists with tool name, error class, transport, and correlation ID. For one auth failure, verify no token or prompt is logged.

Then add the release gate. A simple GitHub Actions job is enough:

name: mcp-contract-gate
on: [pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm ci
      - run: npx playwright test tests/mcp-contract.spec.ts
        env:
          MCP_BASE_URL: ${{ secrets.MCP_BASE_URL }}
          MCP_TOKEN: ${{ secrets.MCP_TOKEN }}

This is not a fancy setup. That is the point. A plain CI gate that blocks a broken tool contract is better than a polished demo that nobody can trust.

Key Takeaways

An MCP server test plan should protect the contract between tool-calling agents and the systems they touch. The recent TypeScript SDK 1.30.0 and Python SDK 2.0.0 releases give QA teams concrete areas to test: discovery, schema validation, auth, transport behavior, compatibility, and observability.

  • Do not stop at a happy-path agent demo; test the MCP server directly.
  • Use official release notes and the MCP specification as source material for test cases.
  • Prioritize auth negative cases because MCP tools often sit near sensitive systems.
  • Run fast contract tests in pull requests and slower transport tests nightly.
  • For Indian SDETs, MCP testing is a strong 2026 portfolio skill because it combines API automation, CI, security thinking, and AI systems.

If you want a practical next step, take one existing tool server and write five tests this week: discovery, required field validation, invalid enum rejection, forbidden scope, and structured log redaction. That small suite will teach you more than reading 20 generic AI agent threads.

FAQ

What is the focus of an MCP server test plan?

The focus is server behavior, not model quality. You test discovery, tool schemas, resources, prompts, auth, transports, compatibility, error handling, and logs. Model evaluation is still useful, but it sits above the server contract.

Should I use Playwright for MCP server testing?

Yes, Playwright API testing is a practical choice for HTTP-based MCP servers. It gives you fixtures, assertions, retries, reporters, and CI integration. For stdio-only servers, pair Playwright with a small Node or Python harness that starts the process and sends JSON-RPC messages.

How many tests should a first MCP suite have?

Start with 10 to 20 tests. Cover discovery, two critical tools, three auth cases, three malformed inputs, one transport check, and one observability check. Expand after you see real failures.

Do I need separate tests for TypeScript and Python SDK servers?

You need separate compatibility cases if your organization uses both SDKs or both client ecosystems. The protocol contract should be consistent, but release notes show implementation details can change. Test the combinations you actually support.

Where should MCP tests run?

Run fast contract and auth tests on every pull request. Run slower streaming, retry, and compatibility tests nightly. Run a final smoke test after deployment to staging and before production promotion.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.