hn · Show HN · 09-17 19:23 59 分 ← 返回列表

Show HN: MCPJam - the first testing & evaluations platform for MCP servers

原文链接 ↗ ·prathmeshmcp

Prathmesh, CEO of MCPJam here. Users now start in ChatGPT, Claude, Cursor, and other AI clients. They reach your product through your MCP server. That means your users often aren’t in your product anymore. You can’t see what they prompted for, how the agent interpreted it, or whether your server helped them get the result they wanted. I saw this firsthand leading MCP technical strategy at Asana, including our ChatGPT and Claude launches. We were building high-stakes enterprise integrations, but

五维评分

付费意愿

需求真实度

个人可落地

竞争空间

新鲜度

AI 分析

痛点：开发者构建 MCP 服务器后，无法观测用户在 ChatGPT/Claude/Cursor 等客户端中的真实 prompt、agent 如何理解、以及最终是否给用户带来想要的结果，缺少像传统软件那样可测试、可回归的评估手段。

目标用户：正在为企业客户构建 MCP 服务器/AI agent 集成的开发团队与平台工程师（类似 Asana 这类做 ChatGPT、Claude 集成的公司）。

证据：We were building high-stakes enterprise integrations, but we had no reliable way to test them the way we test normal software- or to know whether they worked once they reached real users. / I joined the team because I kept hearing the same issue from other companies building for agents.

变现：SaaS 订阅（按项目/CI 用量或席位收费），提供免费开源 CLI + 云端 Swarms/Evals 面板；面向企业做私有化部署与合规版本。

MVP：两周内做一个 MCP 服务器测试 CLI：录制/回放 MCP 工具调用、用 LLM 生成多角色模拟用户跑一批 golden prompts、输出通过率与失败用例报告，并支持 GitHub Action 集成。

竞品：MCPJam（本帖本身）、Braintrust、LangSmith、Promptfoo、Inspect AI

摘要：MCPJam 是号称首个针对 MCP 服务器的测试与评估平台，由前 Asana MCP 技术负责人创立，主打 Swarms 模拟用户、User Testing 和可回归的 Evals，并已接入 CI/CD。它揭示了一个真实且新鲜的需求：随着用户入口转向 ChatGPT/Claude/Cursor，MCP 服务器开发者无法观测和测试真实交互效果。但帖子热度很低（6 分 1 评论），尚无公开付费数据，属于新兴但有竞争涌入的早期市场。