Solution 02 — Local intelligent gateway

One gateway. Every model. No blind spots.

Deploy the CIPHER gateway inside your network and get a single OpenAI-compatible endpoint for local models, CIPHER cloud, and third-party APIs — with routing, caching, and spend control in one place.

Book a demo See the console

Container

to deploy

App changes

same SDK

Deployment shapes

VPC, on-prem, edge

1 day

To first request

typical VPC install

The problem

Sprawling AI usage is an invisible risk

Every team buys its own AI

Different vendors, different keys, and nobody can total the monthly spend.

Sensitive prompts hit public clouds

One careless integration sends customer data to an endpoint outside your control.

One provider outage takes everything down

Your AI features inherit the availability of whichever vendor you picked.

No logs, no evals, no governance

You cannot answer what was sent, by whom, at what cost, or whether it was accurate.

One request, four decisions

Arrive

One endpoint replaces five integrations.

Your services keep calling the same OpenAI-compatible URL. Behind it, the gateway knows which upstreams exist, which are healthy, and which are allowed for this request.

Request path

1/5

Application
Gateway
Policy
Upstream
Trace

Any OpenAI-compatible client, unchanged.

Decide

Policy runs before the model does.

Data classification, team budget, residency, and latency target are evaluated in-process. The decision is recorded, so it can be explained later.

Request path

2/5

Application
Gateway
Policy
Upstream
Trace

Authentication, quotas, and the policy engine, in-process.

Reuse

Repeat questions never reach a GPU.

Semantic cache answers near-duplicates at the edge. On support and retrieval workloads this is usually the largest single cost reduction.

Request path

3/5

Application
Gateway
Policy
Upstream
Trace

Classification, budget, residency, and latency rules evaluated per request.

Answer

Failover is a property of the route, not a runbook.

If the first upstream degrades, the next one in the chain answers. The client sees one successful response and one trace line.

Request path

4/5

Application
Gateway
Policy
Upstream
Trace

Local model, CIPHER cloud, or your own provider contract.

Policy as code

Rules you can review in a pull request

gateway.yaml

# gateway.yaml — reviewed in git, applied by CI
routes:
  - match: { team: "support", contains_pii: true }
    model: meta-llama/llama-4-70b
    residency: eu-only

  - match: { team: "marketing" }
    model: deepseek/deepseek-v4-flash
    max_cost_per_request: 0.02

  - default:
      model: anthropic/claude-sonnet-5.5
      fallback: [qwen/qwen3-235b, mistralai/mistral-large-3]

limits:
  support: { monthly_eur: 400 }
  marketing: { monthly_eur: 150 }

The same file defines which model may see which data, what a request may cost, and what happens when an upstream degrades. It is applied by your CI, not by a dashboard click.

Capabilities

Unified endpoint

OpenAI-compatible; aggregates local, cloud, and third-party upstreams.

Intelligent routing

Route by cost, latency, quality, or a compliance rule you define.

Semantic caching

Repeat and near-repeat requests answered before they reach a GPU.

Failover and balancing

Automatic multi-upstream failover with health checks.

PII guardrails

Sensitive fields stay local; only redacted text leaves the network.

Budgets and quotas

Limits at team, key, and project level.

Observability

Per-request traces with latency, tokens, and cost.

Key vault

Centralised, encrypted storage for upstream credentials.

Deployment

Three places it can live

Where	How	Egress	Lead time
Your VPC	Terraform module, 1 container	Outbound HTTPS only	1 day
On-premises	Helm chart, air-gapped option	None	1–2 weeks
Edge	Container at the PoP nearest your users	Outbound to upstreams	2–3 days

The gateway needs outbound HTTPS to the upstreams you allow and nothing inbound beyond your own network. State stays in your database.

Operationally it is one process with a Postgres or SQLite store. Health checks, structured logs, and Prometheus metrics are included; no sidecar, no agent, no vendor-specific control plane required.

Compare

What changes for your team

	Today	With the gateway
Model access	A key per vendor per service	One key, one endpoint
Sensitive data	Policy by convention	Policy by route
Spend	Reconciled monthly	Live per team
Failover	Manual, if at all	Automatic per model
Model migration	Code change per service	One policy edit
Audit	Scattered provider logs	One trace store

Requirements

What you need on your side

A Kubernetes cluster or a host with Docker — one container, 1 vCPU and 512 MB is enough to start.
Outbound HTTPS to the upstreams you allow; no inbound rule beyond your own network.
A Postgres 14+ database, or SQLite for single-node installs.
An identity provider if you want SSO on the console (Enterprise).

FAQ

Do we have to run it ourselves?

No. We can operate the gateway for you inside your VPC under a management agreement, or you run it with our support. The software is the same either way.

How does semantic caching avoid wrong answers?

The cache is opt-in per route and scoped by a similarity threshold you set. Anything below the threshold goes upstream. For high-stakes routes, leave it off.

Which upstreams are supported?

Any OpenAI-compatible endpoint, plus local runtimes such as vLLM, TGI, and Ollama. CIPHER cloud is one upstream among several, not a requirement.

What does it store?

Keys, policy, budgets, and request metadata in your database. Prompt and completion bodies are stored only if you enable trace retention, and then for the period you choose.

How is this different from an API gateway we already run?

An API gateway routes HTTP. This routes by model semantics, token cost, and data classification, and it speaks the chat-completions schema. Most teams run it behind their existing gateway.

See the gateway on your own workload.

We will run it against a sample of your real traffic and show you the routing and cost decisions it makes.

Book a demo