# Category 9 — Developer Experience (weight 5)

> Verbatim criteria/gates from the criteria Google Doc. Fill Score/Evidence locally; **the human
> pastes**. 1–5 scale; anchors at 1/3/5.

## Scores
| # | Criterion (verbatim) | Score (1–5) | Evidence / note |
|---|---|---|---|
| 1 | Local development loop is productive — stdio mode enables tool development without cloud infrastructure (Stage 1: code runs locally, MCP client spawns the server directly). |  |  |
| 2 | Tunnel-based development loop is supported — a developer can expose their locally running MCP server through a tunnel (Cloudflare, ngrok) and register it against a shared dev Arcade instance to exercise the full request chain (gateway → Engine → tunnel → local server) without deploying to Kubernetes. This is the primary development pattern for custom server authors. |  |  |
| 3 | A shared dev Arcade instance is available for ServiceTitan developers to register tunnel endpoints against — no need to provision a personal Arcade org for every developer. |  |  |
| 4 | Cloudflare tunnel (or equivalent) is the standardized proxy mechanism — documented, with a permanent named-tunnel option so the registered server URL does not change on every session restart. |  |  |
| 5 | SDK documentation is complete, accurate, and has working examples. |  |  |
| 6 | Error messages are actionable — auth failures, misconfigurations, and policy blocks identify the root cause. |  |  |
| 7 | MCP client integration requires no custom adapters or wrappers. |  |  |
| 8 | Gateway and server management are automatable via API. |  |  |

**Average:** ___   **Category score:** ___

## Score anchors
- **1** — No tunnel support; local development requires full Kubernetes deployment to test the gateway chain
- **3** — Tunnel registration works but is underdocumented; no shared dev instance; engineers figure it out individually
- **5** — Tunnel loop is documented with a standard Cloudflare recipe; shared dev instance available; engineers are productive without platform hand-holding

## Benchmark tests
| # | Test (verbatim) | Result | Evidence |
|---|---|---|---|
| 1 | **Stage 1 — local stdio:** Time an engineer from SDK install to first successful local tool call (stdio mode, no Arcade infrastructure). Target: under 2 hours. |  |  |
| 2 | **Stage 2 — tunnel registration (the key test):** Developer runs a local MCP server in HTTP mode, opens a Cloudflare tunnel, registers the tunnel URL as a self-hosted server in the dev Arcade instance, and makes a tool call that flows: Claude Code → gateway → Engine → Cloudflare tunnel → local server. Verify the full chain works including auth and Contextual Access. Measure time from working local server to first successful gateway-mediated call. Target: under 1 day. |  |  |
| 3 | Verify Cloudflare named tunnel (permanent hostname) — confirm the registered URL survives session restarts without re-editing the server registration. |  |  |
| 4 | Intentionally misconfigure an OAuth provider. Measure how quickly the error message identifies the root cause. |  |  |
| 5 | Integrate with Claude Code from scratch — time from gateway URL to working tool invocation in Claude Code. Target: under 30 minutes. |  |  |

## Suggested pass/fail gates
| Gate | Pass condition (verbatim) | Result | Evidence |
|---|---|---|---|
| Tunnel loop | Full gateway → Engine → Cloudflare tunnel → local server chain works end-to-end |  |  |
| Permanent tunnel URL | Named tunnel hostname persists across session restarts without re-registration |  |  |
| Shared dev instance | ServiceTitan developers can register local servers against a shared dev Arcade org without individual account provisioning |  |  |
| Time to first call | Engineer reaches a working gateway-mediated tool call in under 1 day from scratch |  |  |

## Findings
-