arcade-eval/docs/how-the-stack-works.md

# How the stack works — Automation Hub, Tool Hub, and the two gateways (plain language)

> A plain-terms companion to the technical seam map in
> `categories/cat3-access-policy/integration-architecture.md`. Same architecture, no jargon.
> Grounded in `servicetitan/automation-hub` @ master and `servicetitan/tool-hub` @ master
> (source-verified 2026-06-22).

## The one-paragraph version

**Automation Hub** is the warehouse of ~5,000+ things an agent can *do* inside ServiceTitan.
**Tool Hub** is the smart front desk that makes that giant catalog usable for an AI and acts as
the single bouncer (per-user permissions + audit). The **MCP Gateway (Arcade)** plugs in beside
Automation Hub to add *outside* tools (GitHub, Slack, Google) **with per-user login** — the one
thing neither of the others can do. The **AI Gateway** is one toll booth that every model/AI call
passes through (keys, cost, rate limits), added by **configuration, not a rebuild**.

---

## 1. Automation Hub — the warehouse of actions

Where ServiceTitan keeps everything an agent can actually *do*: "create a job," "look up a
customer," "send an invoice" — 5,000+ actions today.

- It holds the **catalog** (every action + what inputs it needs) and does the **execution**
  (actually calls ServiceTitan's internal APIs).
- Its login is **ServiceTitan-identity only.** It can act as a ServiceTitan user/bot, but it has
  **no way to log into GitHub / Slack / Google on your behalf** — and that's deliberate (AH's
  roadmap lists third-party OAuth as a non-goal).

> AH = the internal action warehouse. Great at ServiceTitan, blind to outside SaaS.

## 2. Tool Hub — the smart front desk

Handing an AI the raw list of 5,000 tools (heading to 200,000) blows its context window and it
picks the wrong tool. Tool Hub is the front desk between the agent and the warehouse. It does
three things:

1. **Aggregates** — every source (AH today, others later) becomes one clean, unified list. The
   agent sees **one front desk**, not many warehouses.
2. **Discovers progressively** — the agent never reads the whole catalog. It asks:
   - *"What tools do something like X?"* → `search_tools` returns a **short shortlist**
     (names + one-line summaries only).
   - *"How exactly do I use this one?"* → `get_tool_details` returns full instructions for just
     the **1–3** it actually wants.
   - *"Run it."* → `execute_tool`.
   - (Plus `resume_execution`, `list_namespaces`, `cancel_execution`.)
   It finds tools by **meaning, not keywords** — semantic search over a vector database
   (pgvector + HNSW), embedded by **Voyage**, descriptions enriched by **Claude**, then reranked.
3. **Permission-filters** — before the shortlist ever reaches the agent, it **removes any tool
   you're not allowed to use.** You can't see, let alone call, what you don't have access to.

> Tool Hub = the brain *and* the bouncer. It runs as its **own central service** (two
> autoscaled Kubernetes deployments + an admin UI), **not** a sidecar — and it's the single
> place policy, permissions, and audit live.

**The flow so far:**

```
Agent  →  Tool Hub (front desk: search · filter · decide)  →  Automation Hub (execute)  →  ServiceTitan APIs
```

## 3. Where the two gateways fit

Two real gaps remain. Each gateway plugs one.

### MCP Gateway (Arcade) — the gap = *outside tools*

Tool Hub + AH are great for internal ServiceTitan actions, but neither can **log into
GitHub/Slack/Google as you**. That's Arcade's one job: a second warehouse for **outside SaaS
tools, with per-user login built in.** Tool Hub already has an empty "plug in another source"
slot (the `mcp_proxy` adapter), so Arcade plugs in **right beside** Automation Hub:

```mermaid
flowchart LR
  Agent["LLM Agent"]
  TH["Tool Hub<br/>(brain + bouncer:<br/>search · per-user filter · audit)"]
  AH["Automation Hub<br/>(internal actions)"]
  AR["MCP Gateway — Arcade<br/>(outside tools + per-user login)"]
  ST["ServiceTitan APIs"]
  SaaS["GitHub · Slack · Google"]
  Agent --> TH
  TH --> AH --> ST
  TH --> AR --> SaaS
  classDef new fill:#ffe8cc,stroke:#e8860c,stroke-width:2px,color:#000;
  class AR new;
```

Tool Hub stays the single front desk and bouncer for **both** paths. The only difference: for an
outside tool it hands off to Arcade, and **Arcade handles the messy per-user OAuth login** (that's
the "authorize GitHub" pop-up). Tool Hub never stores your GitHub token — Arcade does.

### AI Gateway — the gap = *the model calls themselves*

Everything above quietly uses AI models: semantic search uses **Voyage** embeddings, catalog
descriptions are written by **Claude**, the agent itself calls a model to think. The **AI
Gateway** is **one toll booth** all of that passes through — so keys, cost tracking, rate limits,
and routing live in one place.

The key point: this is **configuration, not a rebuild.** Every component already calls models
through a swappable address; you just **repoint those addresses at the gateway.**

```mermaid
flowchart LR
  A["Agent thinking"] --> GW
  B["Tool Hub — search (Voyage)"] --> GW
  C["Tool Hub — descriptions (Claude)"] --> GW
  GW["AI Gateway<br/>(one toll booth: keys · cost · limits)"] --> P["Anthropic · Voyage · OpenAI"]
  classDef new fill:#ffe8cc,stroke:#e8860c,stroke-width:2px,color:#000;
  class GW new;
```

## 4. The whole picture in one breath

| Piece | What it is (simple) | The gap it fills |
|---|---|---|
| **Automation Hub** | Warehouse of 5,000+ internal ServiceTitan actions; executes them (ST-login only) | — (the base) |
| **Tool Hub** | Smart central front desk: makes the catalog usable for an AI (search → details → run) + the one bouncer (per-user filter + audit) | Scale + governance |
| **MCP Gateway (Arcade)** | Plugs in beside AH to add outside tools (GitHub/Slack/Google) **with per-user login** | The thing neither AH nor Tool Hub can do |
| **AI Gateway** | One toll booth for **all** model/AI calls | One place for keys/cost/limits — added by config |

**The design win:** adding both gateways is mostly **plugging into seams that already exist** —
Tool Hub stays the single authority, Automation Hub is untouched, and the only genuinely new
capability (logging into third-party apps as you) lives inside Arcade.