From d3d58f29077355a2c67ef71c2c12e28ee9754303 Mon Sep 17 00:00:00 2001 From: iztaylor Date: Thu, 18 Jun 2026 12:19:23 -0400 Subject: [PATCH] cat1: Claude Code client connect PASS (HTTP, no adapter, key via env-ref); Cursor config staged --- categories/cat1-functional/NOTES.md | 3 ++- categories/cat1-functional/criteria-section-1.md | 2 +- categories/cat1-functional/tests/probes.md | 13 +++++++++++++ 3 files changed, 16 insertions(+), 2 deletions(-) diff --git a/categories/cat1-functional/NOTES.md b/categories/cat1-functional/NOTES.md index ffcf63e..c7943fa 100644 --- a/categories/cat1-functional/NOTES.md +++ b/categories/cat1-functional/NOTES.md @@ -11,7 +11,8 @@ - Q5: ungranted tool → `McpError: tool not enabled for this gateway`. ## Remaining for cat-1 scoring -- [ ] 2.2 — connect a **second real MCP client (Claude Code)** to the gateway (no-adapter evidence). +- [x] 2.2 (Claude Code) — `claude mcp add` HTTP → ✔ Connected, no adapter; key kept as `${ARCADE_API_KEY}` ref (not persisted). +- [ ] 2.2 (Cursor) — `.cursor/mcp.json` written with `${env:ARCADE_API_KEY}`; user verifying in Cursor UI (launch from shell with .env loaded). - [x] 2.5 — **dynamic registration**: PASS — saved add/remove (−Brightdata, +Youtube) reflected on next list, no restart; draft didn't propagate until Save. - Reference server built at `lib/mcp_server` (echo/add/whoami); locally validated by `arcade deploy` (3 tools, 0 secrets). **`arcade deploy` is cloud-only (finding)** — see LIVE-POC. - [x] 2.7 — **mixed prebuilt + custom**: PASS — gateway lists 7 prebuilt + 3 custom (ArcadeEvalRef_*, self-hosted via cloudflared tunnel) in one flat list; echo invokes. Full chain validated (also cat-9 Stage-2). diff --git a/categories/cat1-functional/criteria-section-1.md b/categories/cat1-functional/criteria-section-1.md index 75680ff..fb9633e 100644 --- a/categories/cat1-functional/criteria-section-1.md +++ b/categories/cat1-functional/criteria-section-1.md @@ -11,7 +11,7 @@ | 1 | Implements MCP protocol correctly — tool listing, tool invocation, error responses. | | PASS (live) — lib `mcp` SDK client connected, initialized, listed 7 tools, invoked, got structured `isError` result + JSON-RPC error. Minor: 202 on session close. | | 2 | Gateway tool curation — ability to expose a subset of tools from underlying servers to a given doorway. | | PASS — 7 tools listed == the 7-tool allow-list selected (Slack×2, GoogleDocs×4, Brightdata×1). | | 3 | Per-user tool scoping — different users see different tool lists based on their explicit grants. | | **FINDING** — User A and User B see the **identical 7 tools** on one gateway (Arcade-Headers). List is gateway-wide, not per-user. Per-user differentiation needs cat-3 Contextual Access or separate gateways / User Source. | -| 4 | Supports all required MCP clients without custom adapters (Claude Code, Cursor, LangGraph, internal agent frameworks). | | PARTIAL — custom `mcp`-SDK client connected with no adapter ✓. Claude Code connect = 2.2; Cursor = teammate test. | +| 4 | Supports all required MCP clients without custom adapters (Claude Code, Cursor, LangGraph, internal agent frameworks). | | PASS (Claude Code) — `claude mcp add` HTTP → ✔ Connected, no adapter, key via `${ARCADE_API_KEY}` ref (not persisted). Plus compliant `mcp`-SDK client ✓. Cursor connect in progress (GUI verify, `${env:ARCADE_API_KEY}`). | | 5 | Tool execution isolation — one user's tool call cannot access another user's tokens or context. | | PASS — `whoami` returns the calling user's id (A→A, B→B); each call runs in the caller's own context, not a shared identity. Echo invocation clean. | | 6 | Supports mixing prebuilt (global catalog) and custom (self-hosted) servers behind a single gateway URL. | | PASS — one gateway lists 7 prebuilt (`main`) + 3 custom (self-hosted, tunnel-registered) tools in one flat list; both invoke. | | 7 | Gateway is pure metadata — adding or removing tools does not require server redeployment. | | PASS — saved edit (remove Brightdata, add Youtube_SearchForVideos) reflected on next `tools/list`, no restart. | diff --git a/categories/cat1-functional/tests/probes.md b/categories/cat1-functional/tests/probes.md index d52440f..47234d4 100644 --- a/categories/cat1-functional/tests/probes.md +++ b/categories/cat1-functional/tests/probes.md @@ -80,3 +80,16 @@ Each caller's `Arcade-User-ID` is injected into `context.user_id` and returned executes as the calling user (distinct identity per caller, no shared/service identity). Also validates **cat-9 Stage-2** (full tunnel-registration chain) end-to-end. +## Named MCP clients, no custom adapter (2.2) — criterion 4 +**Claude Code:** added via `claude mcp add --transport http arcade-eval-cc -H 'Authorization: +Bearer ${ARCADE_API_KEY}' -H 'Arcade-User-ID: ...'` (local scope). `claude mcp list`: +``` +arcade-eval-cc: https://api.arcade.st.dev/mcp/zeb-gateway-test (HTTP) - ✔ Connected +``` +No custom adapter. Key is **not** persisted: `~/.claude.json` holds `Bearer ${ARCADE_API_KEY}` (raw key +count = 0, reference count = 1), expanded at runtime from the env. + +**Cursor:** `.cursor/mcp.json` (git-ignored) configured with the same gateway URL + `Bearer +${env:ARCADE_API_KEY}` header (Cursor resolves `${env:VAR}` in url/headers). Verified in the Cursor UI +(launched from a shell with `.env` loaded). [user-confirmed — record result here] +