cat1: live probes — protocol/curation PASS, per-user list gateway-wide (finding), ungranted rejected

2026-06-18 11:01:07 -04:00
parent 30d19d42eb
commit 2d02ddbd35
5 changed files with 103 additions and 20 deletions
@@ -0,0 +1,46 @@
+# Cat-1 live probes — raw evidence
+
+Target: `https://api.arcade.st.dev/mcp/zeb-gateway-test` (auth mode: Arcade Headers — API key + `Arcade-User-ID`).
+Client: `lib/mcp_client.py` (official `mcp` SDK, streamable HTTP). Date: 2026-06-18.
+
+## Connect + list (as User A) — protocol compliance
+`connect_and_list` initialized an MCP session and returned 7 tools, no custom adapter:
+```
+Brightdata_ScrapeAsMarkdown
+GoogleDocs_CreateBlankDocument
+GoogleDocs_CreateDocumentFromText
+GoogleDocs_GetDocumentAsDocmd
+GoogleDocs_GetDocumentById
+Slack_SendMessage
+Slack_WhoAmI
+```
+Minor: client logs `Session termination failed: 202` on session close (server returns 202 to the
+streamable-HTTP session DELETE; benign — listing/calls succeed). Note as a protocol nit.
+
+## Per-user visibility (User A vs User B) — criterion 3
+Listed as `ARCADE_USER_A` and `ARCADE_USER_B`:
+```
+A count: 7 | B count: 7
+A == B set?: True
+only in A: set()  | only in B: set()
+```
+**Finding:** on a single gateway in Arcade-Headers mode the curated tool list is **gateway-wide,
+identical for every `Arcade-User-ID`** — not per-user. Per-user list differentiation would need
+Contextual Access (cat 3) or separate gateways / a User Source. (Token vault is still per-user_id.)
+
+## Ungranted / unknown tool — benchmark Q5
+Calling a tool not exposed by the gateway, as User A:
+```
+call_tool("Github_CreateIssue", {}) -> McpError: tool not enabled for this gateway
+```
+Clean rejection at the Engine; no leak, no execution.
+
+## Invocation of an auth-requiring tool (Slack_WhoAmI) as User A — error responses
+```
+isError: True
+content: "unsupported authorization provider type ID '' (providerID 'slack')"
+```
+The Engine processed the call and failed cleanly because the Slack OAuth provider isn't configured
+(the gateway's "Missing Requirements"). Demonstrates: invocation routes through the Engine, returns a
+structured error result, and does **not** fall back to a shared credential. Clean per-call invocation
+without per-user OAuth + a project secret (e.g. serp_api_key) is deferred to the reference server (`echo`).