Residency-Aware Routing: Making Routing Decisions When Data Can't Leave the Country

Every routing decision already has three dimensions. Most gateways stop there.

Ask any AI gateway how it picks a model for a given request, and you'll get some version of the same answer: cost, quality, latency. Route the cheap, simple stuff to a cheap, simple model. Route what needs real reasoning to a stronger one. Prefer whichever provider responds fastest. That's a reasonable, well-understood optimization problem, and most of the industry has converged on solving it well.

What's usually missing is a fourth dimension: is this provider even allowed to receive this request in the first place. Not "which model is best" — "which models are permitted." For most gateways, that question doesn't exist, because it was never part of the routing model to begin with.

What "residency-aware" means, precisely

Residency-aware routing means the routing layer knows, per request, which providers or regions are permissible destinations — and treats that as a constraint to satisfy before it optimizes for cost, quality, or latency. Not a preference. A constraint. A request that isn't allowed to leave a given jurisdiction shouldn't be routed to the cheapest available model if that model happens to sit outside the allowed set; it should be routed to the cheapest, fastest, best model within the allowed set.

That ordering matters. Bolting a residency check on as a post-hoc filter — route normally, then reject anything that landed somewhere it shouldn't have — means you find out about the violation after it already happened. Treating it as a routing input means it never happens.

This isn't the same as "pick a region once"

A lot of infrastructure handles residency by picking a deployment region at setup time — you choose the EU cluster, or the US one, and everything runs there. That's a real solution for a lot of problems, but it's a static one: it assumes every request from that application has the same residency requirement, forever.

In practice, that assumption often doesn't hold. The same application frequently handles requests with genuinely different requirements — some data is sensitive enough to need to stay within a jurisdiction, some isn't, and that distinction usually lives at the request level, not the application level.

A concrete example

Take a customer-facing AI assistant at a Saudi bank. It answers two broad categories of question: general product FAQs ("what's your minimum balance requirement"), and account-specific requests that touch a customer's actual financial data ("why was this transaction flagged"). The first category carries no meaningful residency requirement — it's public information, phrased conversationally. The second category is exactly the kind of workload that falls under SAMA's cloud computing framework for what a bank can send to an external service.

A gateway with no residency dimension treats both categories identically — same routing logic, same candidate models, optimized purely on cost and quality. A residency-aware one narrows the candidate set for the second category to whatever's permitted before it optimizes anything else, and leaves the first category free to route however's cheapest and fastest. Same application, same integration, two different routing outcomes, decided per request rather than per deployment.

Why most gateways don't have this dimension

It's not an oversight so much as a market artifact. Global AI gateways were built primarily for US and EU customers, and the EU's regulatory environment already pushed some of that infrastructure toward in-region processing as one option among several. Saudi Arabia's PDPL, SDAIA's AI Adoption Framework, and SAMA's cloud framework for financial institutions are a comparatively newer and more specific set of constraints for this category of infrastructure to account for — and building for them requires treating residency as a first-class routing input rather than a regional deployment choice, which is a different design decision than most existing gateways made.

What this doesn't solve

Being direct about the limits here: residency-aware routing enforces a policy. It doesn't write one for you, and it doesn't make a non-compliant provider compliant by routing around it carefully. Whether a specific workload needs to stay within a specific jurisdiction, and which providers actually satisfy your organization's obligations under PDPL, SDAIA's framework, or SAMA's rules, is a determination your own compliance and legal teams make — see LLM Gateway for Saudi Arabia for the fuller regulatory picture. What changes with residency-aware routing is that once that determination is made, enforcing it becomes an infrastructure property instead of a manual process someone has to remember to follow.

That's the piece we built Mizan's routing layer around: residency as one of the inputs to every routing decision, not a separate system bolted on after the fact.

Route by where data can go, not just what it costs to send it there.