Cat-1 live probes — raw evidence

Target: https://api.arcade.st.dev/mcp/zeb-gateway-test (auth mode: Arcade Headers — API key + Arcade-User-ID). Client: lib/mcp_client.py (official mcp SDK, streamable HTTP). Date: 2026-06-18.

Connect + list (as User A) — protocol compliance

connect_and_list initialized an MCP session and returned 7 tools, no custom adapter:

Brightdata_ScrapeAsMarkdown
GoogleDocs_CreateBlankDocument
GoogleDocs_CreateDocumentFromText
GoogleDocs_GetDocumentAsDocmd
GoogleDocs_GetDocumentById
Slack_SendMessage
Slack_WhoAmI

Minor: client logs Session termination failed: 202 on session close (server returns 202 to the streamable-HTTP session DELETE; benign — listing/calls succeed). Note as a protocol nit.

Per-user visibility (User A vs User B) — criterion 3

Listed as ARCADE_USER_A and ARCADE_USER_B:

A count: 7 | B count: 7
A == B set?: True
only in A: set()  | only in B: set()

Finding: on a single gateway in Arcade-Headers mode the curated tool list is gateway-wide, identical for every Arcade-User-ID — not per-user. Per-user list differentiation would need Contextual Access (cat 3) or separate gateways / a User Source. (Token vault is still per-user_id.)

Ungranted / unknown tool — benchmark Q5

Calling a tool not exposed by the gateway, as User A:

call_tool("Github_CreateIssue", {}) -> McpError: tool not enabled for this gateway

Clean rejection at the Engine; no leak, no execution.

Invocation of an auth-requiring tool (Slack_WhoAmI) as User A — error responses

isError: True
content: "unsupported authorization provider type ID '' (providerID 'slack')"

The Engine processed the call and failed cleanly because the Slack OAuth provider isn't configured (the gateway's "Missing Requirements"). Demonstrates: invocation routes through the Engine, returns a structured error result, and does not fall back to a shared credential. Clean per-call invocation without per-user OAuth + a project secret (e.g. serp_api_key) is deferred to the reference server (echo).

Dynamic registration (2.5) — criteria 7 & 8, benchmark Q3

Edited zeb-gateway-test in the dashboard: removed Brightdata_ScrapeAsMarkdown, added Youtube_SearchForVideos. A fresh tools/list:

added since first probe:   ['Youtube_SearchForVideos']
removed since first probe: ['Brightdata_ScrapeAsMarkdown']

PASS — change took effect on the next list with no engine/server restart (gateway = pure metadata). Corollary: the edit did not propagate while unsaved (draft); it appeared only after Save — correct/expected, not a defect. Propagation was effectively immediate (next poll).

2.6 KiB Raw Blame History