servicetitan/arcade-eval
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fixture: reference MCP server (echo/add/whoami); finding: arcade deploy is cloud-only

Browse Source
This commit is contained in:
iztaylor
2026-06-18 11:14:50 -04:00
parent 619e6d833e
commit beb17c0a8f
6 changed files with 108 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
categories/cat1-functional/NOTES.md
+3 -2
View File
@@ -13,6 +13,7 @@
## Remaining for cat-1 scoring
- [ ] 2.2 — connect a **second real MCP client (Claude Code)** to the gateway (no-adapter evidence).
- [x] 2.5 — **dynamic registration**: PASS — saved add/remove (−Brightdata, +Youtube) reflected on next list, no restart; draft didn't propagate until Save.
- [ ] 2.7 — **mixed prebuilt + custom**: compose a gateway with a `main` tool + a `lib/mcp_server` tool. (needs reference server → `arcade login`/`arcade deploy`)
- [ ] 2.4 — **`whoami` execution proof** that calls run as the calling user. (needs reference server)
- Reference server built at `lib/mcp_server` (echo/add/whoami); locally validated by `arcade deploy` (3 tools, 0 secrets). **`arcade deploy` is cloud-only (finding)** — see LIVE-POC.
- [ ] 2.7 — **mixed prebuilt + custom**: needs the ref server behind the self-hosted Engine via the **register path** (run `server.py --transport http` + cloudflared tunnel + dashboard Add Server), then compose a gateway (a `main` tool + `echo`). Doubles as cat-9 Stage-2.
- [ ] 2.4 — **`whoami` execution proof**: once registered, call whoami as A vs B (expect A→A, B→B).
- [ ] 2.8 — finalize scores once the above land.