Skip to content

Concepts · Sovereign runtime

From one workstation to a full air gap.

"Runs where your data lives" is a claim about hardware, networks and trust boundaries. Here are the actual deployment shapes — and the routing pattern that keeps small models fast without ever calling a cloud.

The shapes

Three deployment envelopes, one principle.

Every shape keeps data, model and orchestration inside a single trust boundary you control. They differ in where that boundary sits.

Workstation class

Task-tuned 4–8B, quantized

One GPU server in your rack — or a locked cabinet in your office. Runs a fine-tuned small model with retrieval at interactive speed for a team or a document pipeline. This is the shape that makes sovereign AI a line item instead of a project.

Node class

70B-class, or several models

A multi-GPU server in your datacentre or your Swiss cloud tenancy. Runs larger models, multiple concurrent workloads, and the escalation tier of the router below. The shape for organisation-wide rollouts.

Air-gapped / confidential

Any size, maximum isolation

Either physically disconnected infrastructure with offline update runbooks, or confidential-compute VMs with remote attestation — memory encrypted with keys the operator never holds. Chip-level detail in the FAQ below.

What makes the workstation shape possible at all is aggressive, evaluation-verified quantization.

The pattern

Small-first, escalate local.

The routing architecture that gets small-model economics without giving up on the hard tail — or on sovereignty.

  1. 01
    The small model sees every request first

    The task-tuned 4–8B answers first — fast, cheap, on the workstation-class shape. For a well-scoped workflow, that's the overwhelming majority of volume.

  2. 02
    A router watches for trouble

    Schema violations and failed tool calls are unambiguous triggers. Confidence and out-of-distribution signals are harder — we calibrate them per task against your evaluation set, and the escalation threshold is a measured trade-off, not a default we trust blindly.

  3. 03
    Escalation goes to a larger local model

    Flagged requests re-run on the node-class tier: a bigger open-weight model on your own infrastructure. The cloud version of this pattern escalates to a US API — ours never does. On workstation-only deployments, flagged requests route to a human review queue instead: a small system that knows when to hand off beats a large one that never does.

  4. 04
    Everything is logged and re-scored

    Escalation rates and outcomes feed the evaluation loop: a rising rate means the task drifted or the small model needs a refresh — visible in the metrics weeks before users would notice.

The boundary

Zero egress is a design property, not a promise.

You shouldn't have to trust a vendor's word for "nothing leaves". You should be able to verify it at the firewall.

The runtime is containerised and self-contained: model weights, retrieval index, orchestration and monitoring all run inside your network segment. No component calls out — no telemetry, no license pings, no "anonymous usage statistics". Your firewall rules can say deny all egress, and the system keeps working. That's the test, and you can run it yourself.

Monitoring stays inside too: dashboards, logs and evaluation scores are served locally, and log retention follows your policy — inference logs contain your data. The self-hosted tracing stack behind this is on the orchestration page.

On the workstation and air-gapped shapes, this is also a demo we like giving: pull the network cable, keep working.

FAQ

Runtime questions.

What does the hardware actually cost?
The workstation shape — a single-GPU server that runs a task-tuned 4–8B — is typically a CHF 5,000–15,000 machine that lives in your existing rack or a locked office cabinet. The node shape for 70B-class models is a mid-five-figure investment or a rental in your Swiss cloud tenancy. Exact sizing (and committed throughput numbers) is part of every proposal — and Pilot hardware can be ours, so infrastructure spend starts only after the evaluation clears.
What is confidential compute, concretely?
Hardware-encrypted virtual machines — AMD SEV-SNP or Intel TDX — where memory is encrypted with keys the cloud operator never holds, and remote attestation lets you cryptographically verify the launch state (firmware, image, configuration) before any data enters. Practical meaning — you can rent Swiss infrastructure while shifting the provider out of the data-confidentiality trust boundary. One caveat we raise before your security team does — a CPU enclave alone doesn't cover the GPU where inference actually runs; extending the boundary to the accelerator takes confidential-computing-capable GPUs (NVIDIA H100 class, with encrypted transfers and GPU attestation), and we size for exactly that when the threat model requires it.
How do air-gapped systems get updates?
Deliberately and verifiably — never over a wire. Model updates, retrieval-index rebuilds and software patches arrive as signed, checksummed offline bundles on scheduled maintenance windows, installed from documented runbooks your own staff can execute. Every update is re-validated against your evaluation set before it replaces the running version. Slower than cloud auto-update — which is exactly what your security team wants.

Which shape fits your constraints?

Describe your data-governance constraints in the scoping call — we'll map them to a deployment shape and put the sizing and throughput in the proposal.

Book a call See how it works