Don't ship MCP servers that fail

You know your users now live in ChatGPT, Claude, & Copilot, but are they actually succeeding when using your product? We've launched our new platform to answer this question with Swarms, User Testing, Evals, and CI/CD integration.

Start testing in web

September 2026

Swarms, User Testing, Evals, & CI/CD

Swarms

01

Simulate many user scenarios across ChatGPT, Claude, and more in parallel. Catch the edge cases your tests miss.

Leave

Create a swarm of your users

Simulated users run through your server so you can see what breaks.

Choose the clients and servers your users will interact with. *

This will shape the swarm's behavior. You will be able to compare behavior across clients side by side.

ChatGPTClaudeCursor+ Add client
acme-payments· v2.15.0-rc.1 · 5 tools

Describe your users and their behavior. *

We build the user goals from your input. Your inputs are not final. You can edit personas and goals on the next screen.

Support agents refunding duplicate charges, finance ops reconciling Tuesday's payouts, and developers wiring a failed-payment webhook.

Generated personas

  • MCMaya Chen · Support agentFind the duplicate charge on order A-2291 and refund it.8 × 3 clients
  • JHJordan Hale · Finance opsMatch payout po_7Hd2 to its charges and flag anything missing.8 × 3 clients
  • PSPriya Shah · DeveloperCreate an invoice, fail its payment, and confirm the webhook fires.8 × 3 clients

User Testing

02

See how real users interact with your product. Measure sentiment and uncover usability gaps.

Refund & reconcile · beta

Access settings

Anyone with the link (guests included)

Has access

Ratings

Per-turn ratings

Preview

ChatGPTCopy link

Refund the duplicate charge on order A-2291

list_charges· refund_charge

I found two charges for order A-2291: ch_3RkA9 for $129.00 and ch_3RkQ9 for $129.00, 41 seconds apart. I refunded ch_3RkQ9 as a duplicate. The customer will see it in 5 to 10 business days.

Rate this response
Message…

AI can make mistakes. Please double-check responses.

Previewing as you, signed in. A tester opening this link gets the guest experience, which can differ.

Evals

03

Turn user workflows into repeatable, durable test suites. Re-run them automatically across your preferred clients.

Evaluate

OverviewSuites

Suite health

98%average across 12 runs

acme-paymentsPayments operations
100%95%
9/29/39/49/59/69/79/89/99/109/119/129/13

Runs

PlatformSuiteClientServer
RunSuiteClientResultRatePlatformDateLatencyTokensCalls
#128Payments operationsFailed96%69/72GitHubSep 13, 2:00 AM18.4s16.2k44
#127Payments operationsPassed98%71/72CLISep 12, 2:00 AM14.2s12.1k41
#126Treasury reconciliationPassed100%48/48SDKSep 12, 1:14 AM16.8s15.6k38
#125Billing exception handlingFailed94%45/48CLISep 11, 11:42 PM21.1s18.9k47
#124Payments operationsPassed99%71/72GitHubSep 11, 2:00 AM11.3s9.4k29
#123Dispute resolutionPassed100%36/36SDKSep 10, 6:18 PM15.5s11.8k33
#122Treasury reconciliationPassed97%70/72GitHubSep 10, 2:00 AM17.9s14.7k40

CI/CD

04

Block failures before they ship. Make durable cross client evaluation part of every release.

acme/acme-paymentsPrivate
CodeIssuesPull requestsActions

list_charges: return full charge objects #482

Openm.okafor wants to merge 1 commit into release/2.15
ConversationCommits 1Checks 5

Some checks were not successful

1 failing and 4 successful checks

  • BuildSuccessfulRequired
  • Unit testsSuccessfulRequired
  • MCPJam / Protocol conformanceSuccessfulRequired
  • MCPJam / OAuth & securitySuccessfulRequired
  • MCPJam / Evals89% pass rate · 95% requiredRequired

Merging is blocked

MCPJam Evals must pass before this pull request can be merged.

Get started

Start with your agent, launch locally, or however you'd like.

Get your agent set up with MCPJam

Copy a prompt into your coding agent to set up MCPJam and check your MCP server.

Onboard onto MCPJam, use it to test the MCP server in this repo on a branch, and tell me how it did: https://www.mcpjam.com/agent-setup.md

Use a coding agent with terminal access, or an MCP-capable client. Your agent will explain any login or setup step it needs help with.

Launch locally