MCP makes it easy to go from “agent” to “agent that takes action.” The trap is that success compounds: every new system becomes a new server, every team ships “just one more tool,” and soon your integration surface is too large to reason about, too inconsistent to secure, and too messy to operate.
Meanwhile, the model gets blamed for failure modes that are actually integration design problems. Tool definitions balloon. Selection accuracy drops. Context gets eaten before anyone types a prompt. Anthropic’s Tool Search work is a strong acknowledgement of this reality: tool definitions for 50+ MCP tools can consume tens of thousands of tokens, and on-demand discovery can dramatically reduce that overhead while improving accuracy.
Tool search helps with context economics. It does not, by itself, solve the governance and operations problems that show up once MCP is powering real workflows.
That’s where the MCP gateway pattern becomes the difference between a working demo and a sustainable platform.
What an MCP gateway is
An MCP gateway is a single MCP entrypoint that federates tools from multiple MCP servers into one managed tool surface. The value is not “one more hop.” The value is that the gateway becomes the natural place to centralize cross-cutting concerns: authentication, policy, routing, and telemetry.
A crisp definition (using Arcade’s language because it’s explicit and current): gateways “federate the tools from multiple MCP Servers into a single collection,” allow mixing tools across servers, and “not all tools from a MCP server need to be available to the same LLM.”
Two practical implications:
- You stop wiring every client/agent to an ever-growing mesh of servers.
- You gain a controlled way to expose different tool surfaces to different agents, workflows, IDEs, and users—without duplicating backend servers.
Registry vs gateway: discovery is not governance
As the ecosystem grows, you need both:
- A registry: a discovery layer that helps clients find servers and their metadata (think “app store for MCP servers”). The official MCP Registry project explicitly positions itself this way.
- A gateway: a runtime control plane that decides what is exposed, to whom, under what policy, with what auditing.
Conflating these leads to fragile systems. Registries reduce fragmentation by standardizing discovery metadata. Gateways reduce fragmentation by standardizing runtime control.
A reference architecture that holds up in production
A durable architecture is:
MCP clients/hosts (IDE, agent runtime, copilots) → Gateway (single MCP endpoint + policy + telemetry) → Backend MCP servers (system capabilities + orchestration) → Upstream systems/APIs
Transport: default to Streamable HTTP for real deployments
If you’re building anything that’s remotely accessed or used by multiple clients, Streamable HTTP is the practical backbone. MCP’s transport spec also makes the security posture explicit for HTTP-based transports—this is not optional “hygiene.”
Specifically, when implementing Streamable HTTP, servers must validate the Origin header (DNS rebinding), should bind to localhost when running locally, and should implement authentication.
Why “gateway thinking” is now mandatory: exposed servers are already a problem
This is not theoretical. Bitsight’s TRACE team reported finding roughly 1,000 internet-exposed MCP servers with no authorization and demonstrated that they could retrieve tool lists and other metadata from them.
Two takeaways:
- Do not expose MCP endpoints to the public internet without a real authorization mechanism.
- If you are going to make MCP broadly usable across agents, IDEs, and users, you need a consistent “front door” that enforces baseline security and policy.
A gateway is that front door.
Authorization: follow MCP’s model, and avoid the two most common mistakes
MCP authorization (for HTTP transports) is based on OAuth 2.1 concepts: the MCP server is a resource server, the MCP client is an OAuth client, and token issuance is handled by an authorization server.
Three points from the spec matter disproportionately in practice:
- Authorization is optional in MCP implementations. That fact is one reason you see so many unsecured deployments.
- Audience binding is mandatory when you validate tokens: servers must validate that tokens were issued for them as the intended audience.
- Token passthrough is explicitly forbidden: servers must not accept/transit arbitrary tokens; upstream APIs require separate tokens issued by the upstream authorization server.
This maps cleanly to a gateway design principle:
Separate front-door identity from downstream authorization
You want two distinct controls:
- Caller identity at the gateway (who is invoking this MCP surface?)
- Tool-level authorization (what can be invoked, and under what scopes/permissions?)
This separation is the difference between “we put a key in a config file” and “this is safe for multi-user production.”
A concrete example of how a gateway can support multi-user environments (without turning into a bespoke auth system): Arcade’s gateway docs describe a production mode where a service authenticates with an API key and passes an end-user identifier separately (Arcade-User-ID)—so the gateway can enforce per-user policy while the calling app remains the authenticated client.
You can implement the same concept vendor-neutrally: authenticate the calling workload, propagate a user identity claim, and apply least-privilege policy for tool execution on behalf of that user.
Tool surfaces: your gateway should curate, not aggregate
The easiest gateway implementation is “proxy everything.” It’s also the fastest way to recreate tool overload, just one layer up.
A production gateway should support curated tool surfaces:
- Allowlisted tools (explicitly choose what is exposed)
- Per-agent / per-workflow views (different clients get different subsets)
- LLM-facing usage instructions at the surface (how this tool collection should be used)
This is the integration-era lesson: reuse comes from stable building blocks, but reliability comes from purpose-built surfaces.
Scaling beyond “a few tools”: combine curation with on-demand discovery
Tool overload is now mainstream enough that model providers are building first-class mechanisms to address it. Anthropic’s Tool Search approach is explicit: instead of loading all tool definitions upfront, you mark tools as deferred (defer_loading: true) and discover them on demand.
Two important clarifications for accuracy:
- Tool search is model/client tooling (e.g., Claude), not a core MCP protocol feature.
- It complements gateways: the gateway reduces what’s eligible, and the model-side mechanism reduces what’s loaded right now.
A practical rule of thumb (supported by Claude’s docs): keep a small set of frequently-used tools non-deferred, and defer the rest so they’re only loaded via search.
This gives you a scalable pattern:
- Gateway: policy + curation + stable naming/contracts
- Client/model: dynamic discovery to preserve context and improve selection accuracy
Operations and governance: treat MCP like an integration platform, not a feature
Once MCP is involved in real workflows, you need the same controls that made API programs survivable:
- Centralized auth and policy enforcement
- Routing and request shaping (which backend server/tool is used)
- Rate limiting / throttling to protect upstreams and contain abuse
- Audit logs and tracing for “who did what, when, and why”
This isn’t speculative—these are well-established “gateway policies” and responsibilities in modern API gateway practice (auth, rate limiting, monitoring, CORS, etc.).
One nuance for MCP: be careful with “helpful” caching. Anything influenced by caller identity or policy (including “what tools are available”) should be treated as identity-scoped or not cached.
Build order: the shortest path to a safe, operable MCP gateway
If you’re implementing this pattern, this sequence minimizes rework:
- One Streamable HTTP endpoint for clients.
- Origin validation + safe binding defaults (per MCP transport guidance).
- Front-door authentication on every request; fail closed.
- Tool allowlists and named tool surfaces (treat tool names as APIs).
- Per-agent/per-workflow views (least privilege by default).
- Surface instructions (make the tool contract explicit to the model).
- Token discipline: audience binding, no passthrough, separate upstream tokens.
- Structured audit logging for tool calls (inputs, outputs, actor, policy decision).
- Rate limiting by caller and by tool class.
- On-demand tool discovery where supported (deferred loading + tool search).
Closing
MCP is winning because it standardizes how agents connect to real systems. But standardization increases composability faster than it increases operability. Registries accelerate discovery. Tool search reduces context cost. Neither gives you governance.
An MCP gateway is the missing control plane: it turns “a pile of servers” into a managed integration surface with consistent security and predictable operations—without sacrificing the interoperability that made MCP compelling in the first place.
