2.6 KiB
2.6 KiB
Category 1 — Functional MCP Gateway Capability (weight 8)
Verbatim criteria / gates / questions from the criteria Google Doc. Fill Score / Evidence / Findings / Answers locally; the human pastes into the Google Doc. 1–5 scale; anchors at 1/3/5.
Scores
| # | Criterion (verbatim) | Score (1–5) | Evidence / note |
|---|---|---|---|
| 1 | Implements MCP protocol correctly — tool listing, tool invocation, error responses. | ||
| 2 | Gateway tool curation — ability to expose a subset of tools from underlying servers to a given doorway. | ||
| 3 | Per-user tool scoping — different users see different tool lists based on their explicit grants. | ||
| 4 | Supports all required MCP clients without custom adapters (Claude Code, Cursor, LangGraph, internal agent frameworks). | ||
| 5 | Tool execution isolation — one user's tool call cannot access another user's tokens or context. | ||
| 6 | Supports mixing prebuilt (global catalog) and custom (self-hosted) servers behind a single gateway URL. | ||
| 7 | Gateway is pure metadata — adding or removing tools does not require server redeployment. | ||
| 8 | Dynamic tool registration — new tools become available without gateway restart. |
Average: ___ Category score: ___
Score anchors
- 1 — Basic MCP server, no per-user scoping or curation
- 3 — Gateway curation works; per-user scoping requires workarounds
- 5 — Full per-user tool scoping, mixed-server gateways, zero-config for MCP clients
Benchmark questions
| # | Question (verbatim) | Answer | Evidence |
|---|---|---|---|
| 1 | Can a Claude Code client connect to the gateway and see only the tools granted to the current user? | ||
| 2 | Can the same gateway URL serve two different users with different tool lists? | ||
| 3 | Can we add a tool to the gateway without restarting any server or the Engine? | ||
| 4 | Can we expose tools from both a prebuilt connector and a custom self-hosted server through one gateway endpoint? | ||
| 5 | What happens when a client requests a tool the user has not been granted? |
Suggested pass/fail gates
| Gate | Pass condition (verbatim) | Result | Evidence |
|---|---|---|---|
| MCP protocol compliance | Any compliant MCP client connects without custom adapters | ||
| Tool curation | Gateway tool list matches exactly the configured allow-list | ||
| Per-user isolation | User A cannot see or invoke tools granted only to User B | ||
| Mixed server gateway | Prebuilt and custom server tools coexist behind one gateway URL |