cat1: FINALIZE scorecard (draft 4/5); STATUS + cat-5 NOTES ready for fresh-session handoff

This commit is contained in:
2026-06-22 09:55:01 -04:00
parent 8b48f5813e
commit 53f960409e
5 changed files with 95 additions and 24 deletions
+8 -2
View File
@@ -11,8 +11,14 @@
- Q5: ungranted tool → `McpError: tool not enabled for this gateway`.
## Remaining for cat-1 scoring
- [x] 2.2 (Claude Code) — `claude mcp add` HTTP → ✔ Connected, no adapter; key kept as `${ARCADE_API_KEY}` ref (not persisted).
- [ ] 2.2 (Cursor) — `.cursor/mcp.json` written with `${env:ARCADE_API_KEY}`; user verifying in Cursor UI (launch from shell with .env loaded).
- [x] 2.2 (Claude Code) — connected with NO adapter in both modes: Arcade-Headers (`claude mcp add`) AND Entra User-Source OAuth (`/mcp` login → tools loaded in-session; echo/whoami ran). Key kept as `${ARCADE_API_KEY}` ref (not persisted).
- [~] 2.2 (Cursor/LangGraph/internal) — not exercised this round; no adapter expected (same transport). Cursor config currently empty.
- [x] 2.8 — scorecard FINALIZED (draft 4/5) in criteria-section-1.md; awaiting user paste into Google Doc.
## Side evidence generated (handed to other lanes)
- cat-2: Entra IdP login works; identity = opaque `sub`; downstream OAuth consent-bind mismatch (see LIVE-POC).
- cat-4/8/9: `arcade deploy` is cloud-only → self-hosted servers use the register path.
- cat-9: full tunnel-registration chain validated end-to-end (client→gateway→Engine→tunnel→local server).
- [x] 2.5 — **dynamic registration**: PASS — saved add/remove (Brightdata, +Youtube) reflected on next list, no restart; draft didn't propagate until Save.
- Reference server built at `lib/mcp_server` (echo/add/whoami); locally validated by `arcade deploy` (3 tools, 0 secrets). **`arcade deploy` is cloud-only (finding)** — see LIVE-POC.
- [x] 2.7 — **mixed prebuilt + custom**: PASS — gateway lists 7 prebuilt + 3 custom (ArcadeEvalRef_*, self-hosted via cloudflared tunnel) in one flat list; echo invokes. Full chain validated (also cat-9 Stage-2).