Cross-client testing
One passing tool call doesn't mean your server behaves.
Each client connects, authorizes, and calls your server a little differently. MCPJam tests the whole chain the way each host will: protocol compliance, OAuth conformance, MCP Apps rendering, and evals with real models, so a pass means it works where your users are.
Four checks behind every pass
Protocol compliance
The doctor validates connection, capabilities, and primitives end to end, and compatibility checks return per-host verdicts for Claude, ChatGPT, Cursor, Copilot, and Codex.
Security
OAuth conformance exercises your auth across every spec version and registration method clients use, DCR included, plus EMA for enterprise Cross-App Access.
Reliability
Eval suites generated from your server's tools run with real models, measuring accuracy, latency, and whether each host completes the jobs your users will ask for.
Self-improvement
Run Insights names each failure pattern, failing cases become regression tests, and pass rates trend across runs, so the next release is better than the last.
The whole gate, from the terminal
Inspection and conformance are CLI commands with pass/fail exit codes, and evals run in your test runner via the SDK, so the same checks run locally and in CI. See the docs for the full flag reference.
# Inspect: probe, connect, and validate your server
mcpjam server doctor --url https://your-server.com/mcp
# Conformance: OAuth and MCP Apps, per spec version
mcpjam oauth conformance --url https://your-server.com/mcp
mcpjam apps conformance --url https://your-server.com/mcp
# Evals: run your @mcpjam/sdk suites in your test runner
npm testFrequently asked questions
Evals are one of four layers. The other three: protocol compliance (connection, capabilities, primitives), OAuth conformance across spec versions, and MCP Apps rendering. Each layer gets per-host verdicts, so you see exactly which client breaks and where.
Whether a real model selects the right tool, sends the right arguments, and completes the job across many realistic requests (evaluation, not assertions), plus accuracy, latency, tokens, and tool-call metrics.
MCPJam generates them from your server's tools. In testing against a Google Calendar server it produced eight cases, from a simple meeting event to scheduling multiple events in one request.
Yes. The CLI and SDK wire into GitHub Actions or any pipeline to run evals on every PR, with accuracy tracked over time. See the CI/CD actions page.
Find out how models use your server.
Generate a suite, run it with a real model, and see the failure patterns. Free, in about two minutes.
Open source core · no credit card · ~2 min