arcade-eval/docs/how-the-stack-works.md

How the stack works — Automation Hub, Tool Hub, and the two gateways (plain language)

A plain-terms companion to the technical seam map in categories/cat3-access-policy/integration-architecture.md. Same architecture, no jargon. Grounded in servicetitan/automation-hub @ master and servicetitan/tool-hub @ master (source-verified 2026-06-22).

The one-paragraph version

Automation Hub is the warehouse of ~5,000+ things an agent can do inside ServiceTitan. Tool Hub is the smart front desk that makes that giant catalog usable for an AI and acts as the single bouncer (per-user permissions + audit). The MCP Gateway (Arcade) plugs in beside Automation Hub to add outside tools (GitHub, Slack, Google) with per-user login — the one thing neither of the others can do. The AI Gateway is one toll booth that every model/AI call passes through (keys, cost, rate limits), added by configuration, not a rebuild.

---

1. Automation Hub — the warehouse of actions

Where ServiceTitan keeps everything an agent can actually do: "create a job," "look up a customer," "send an invoice" — 5,000+ actions today.

• It holds the catalog (every action + what inputs it needs) and does the execution (actually calls ServiceTitan's internal APIs).
• Its login is ServiceTitan-identity only. It can act as a ServiceTitan user/bot, but it has no way to log into GitHub / Slack / Google on your behalf — and that's deliberate (AH's roadmap lists third-party OAuth as a non-goal).

AH = the internal action warehouse. Great at ServiceTitan, blind to outside SaaS.

2. Tool Hub — the smart front desk

Handing an AI the raw list of 5,000 tools (heading to 200,000) blows its context window and it picks the wrong tool. Tool Hub is the front desk between the agent and the warehouse. It does three things:

1. Aggregates — every source (AH today, others later) becomes one clean, unified list. The agent sees one front desk, not many warehouses.
2. Discovers progressively — the agent never reads the whole catalog. It asks:
   • "What tools do something like X?" → search_tools returns a short shortlist (names + one-line summaries only).
   • "How exactly do I use this one?" → get_tool_details returns full instructions for just the 1–3 it actually wants.
   • "Run it." → execute_tool.
   • (Plus resume_execution, list_namespaces, cancel_execution.) It finds tools by meaning, not keywords — semantic search over a vector database (pgvector + HNSW), embedded by Voyage, descriptions enriched by Claude, then reranked.
3. Permission-filters — before the shortlist ever reaches the agent, it removes any tool you're not allowed to use. You can't see, let alone call, what you don't have access to.

Tool Hub = the brain and the bouncer. It runs as its own central service (two autoscaled Kubernetes deployments + an admin UI), not a sidecar — and it's the single place policy, permissions, and audit live.

The flow so far:

Agent  →  Tool Hub (front desk: search · filter · decide)  →  Automation Hub (execute)  →  ServiceTitan APIs

3. Where the two gateways fit

Two real gaps remain. Each gateway plugs one.

MCP Gateway (Arcade) — the gap = outside tools

Tool Hub + AH are great for internal ServiceTitan actions, but neither can log into GitHub/Slack/Google as you. That's Arcade's one job: a second warehouse for outside SaaS tools, with per-user login built in. Tool Hub already has an empty "plug in another source" slot (the mcp_proxy adapter), so Arcade plugs in right beside Automation Hub:

flowchart LR
  Agent["LLM Agent"]
  TH["Tool Hub<br/>(brain + bouncer:<br/>search · per-user filter · audit)"]
  AH["Automation Hub<br/>(internal actions)"]
  AR["MCP Gateway — Arcade<br/>(outside tools + per-user login)"]
  ST["ServiceTitan APIs"]
  SaaS["GitHub · Slack · Google"]
  Agent --> TH
  TH --> AH --> ST
  TH --> AR --> SaaS
  classDef new fill:#ffe8cc,stroke:#e8860c,stroke-width:2px,color:#000;
  class AR new;

Tool Hub stays the single front desk and bouncer for both paths. The only difference: for an outside tool it hands off to Arcade, and Arcade handles the messy per-user OAuth login (that's the "authorize GitHub" pop-up). Tool Hub never stores your GitHub token — Arcade does.

AI Gateway — the gap = the model calls themselves

Everything above quietly uses AI models: semantic search uses Voyage embeddings, catalog descriptions are written by Claude, the agent itself calls a model to think. The AI Gateway is one toll booth all of that passes through — so keys, cost tracking, rate limits, and routing live in one place.

The key point: this is configuration, not a rebuild. Every component already calls models through a swappable address; you just repoint those addresses at the gateway.

flowchart LR
  A["Agent thinking"] --> GW
  B["Tool Hub — search (Voyage)"] --> GW
  C["Tool Hub — descriptions (Claude)"] --> GW
  GW["AI Gateway<br/>(one toll booth: keys · cost · limits)"] --> P["Anthropic · Voyage · OpenAI"]
  classDef new fill:#ffe8cc,stroke:#e8860c,stroke-width:2px,color:#000;
  class GW new;

4. The whole picture in one breath

Piece	What it is (simple)	The gap it fills
Automation Hub	Warehouse of 5,000+ internal ServiceTitan actions; executes them (ST-login only)	— (the base)
Tool Hub	Smart central front desk: makes the catalog usable for an AI (search → details → run) + the one bouncer (per-user filter + audit)	Scale + governance
MCP Gateway (Arcade)	Plugs in beside AH to add outside tools (GitHub/Slack/Google) with per-user login	The thing neither AH nor Tool Hub can do
AI Gateway	One toll booth for all model/AI calls	One place for keys/cost/limits — added by config

The design win: adding both gateways is mostly plugging into seams that already exist — Tool Hub stays the single authority, Automation Hub is untouched, and the only genuinely new capability (logging into third-party apps as you) lives inside Arcade.