MCPJamMCPJam
Back to all posts

We added frontier models to our MCP playground for free

Matthew Wang5 min read

We're excited to announce that we're providing frontier proprietary/open source models from OpenAI, Anthropic, Gemini, and more to be used for free in MCPJam's LLM playground. You no longer have to bring your own API key to access the best MCP server testing experience. It's on us.

Frontier models available in MCPJam playground

Models we're releasing

Anthropic logoAnthropic

  • Claude Sonnet 4.5
  • Claude Haiku 4.5

Google logoGoogle Gemini

  • Gemini 2.5 Flash
  • Gemini 2.5 Flash Preview

OpenAI logoOpenAI

  • GPT-5
  • GPT-5 Codex
  • GPT-5 Mini

xAI Grok logoxAI

  • Grok 4 Fast

Moonshot AI logoMoonshot AI

  • Kimi K2

Zhipu AI logoZhipu AI

  • GLM 4.6

From our observations so far, we noticed that GPT-5 has the best tool call accuracy, followed by Claude, Grok, then Gemini.

Why we're doing this

We're on a mission to help developers build the best MCP servers, and we believe that allowing users that don't have access to these models will help us reach this goal faster. As a server developer, you have to think about how different clients interact with your server, along with their different language models.

We want to give developers the certainty that their server will perform exactly how they intend to in all these clients, and have it all live in a single platform to help with fast iteration.

Models have wildly different performances when interacting with MCP servers. Salesforce AI research team benchmarked the performance of models across different MCP servers of varying tasks in their MCP-Universe paper. This highlights the importance of testing your MCP server with different models for your server's production-readiness.

MCP-Universe paper

We also think releasing frontier models is necessary for the OpenAI Apps SDK developer experience. By providing GPT-5 series models, we can emulate how ChatGPT (GPT-5) interacts with your ChatGPT app.

How to start testing

You can start using MCPJam's LLM playground by running the following command:

npx @mcpjam/inspector@latest

This will spin up a local React instance of the MCPJam inspector. You'll need to create an account on MCPJam to use the frontier models. Navigate to the playground tab and start testing!