Skip to content

Live demo · on-device inference

A working model, running inside this page.

Below, a 1.7-billion-parameter language model triages support messages — category, urgency, one-line summary — computed on your own GPU, in this browser tab. No API, no server, no data egress. It is the most extreme form of the argument this site makes: models that run where the data lives.

Why this matters

The objection to sovereign AI is capability. The answer is scope.

Regulated organisations can't send client data to third-party AI APIs — banking secrecy, medical confidentiality and procurement rules all say the same thing. The usual counter-argument is that anything you can run yourself is too weak to be useful. That's true if you expect one model to do everything.

It stops being true when the task is narrow. A small model given one well-defined job — classify this message, extract these fields, draft this standard reply — can be evaluated, made deterministic, and run on hardware you control. Quantization shrinks it until "hardware you control" can mean a workstation, a phone — or, as here, a browser tab.

This page is the working proof: a real model with a real job, and a 0-byte network footprint for everything you type.

The use case

Support triage: a canonical narrow task.

Every service organisation has an inbox: complaints, outage reports, sales questions, password resets. Someone — or something — reads each message and decides what it is and how urgent it is before anyone acts on it. That decision is a perfect fit for a small model: the input is one message, the output is a closed set of labels plus a one-line summary, and every answer is checkable.

The demo below does exactly that. Paste a message (or use an example) and the model returns Category · Urgency · Summary. Each message is triaged independently — no history, no drift. Switch to the Chat tab to talk to the same model freely and calibrate your expectations of what 1.7 billion 1-bit parameters can and cannot do.

In a client project this is where the work starts, not ends: the taxonomy becomes your categories, the model is fine-tuned on your labelled messages, and an evaluation harness measures it against your bar before it touches production — the same method as our published ablations.

Runs entirely in this tab on WebGPU. First run downloads the model — ~237 MB, one time — then it's cached and works offline. Desktop Chrome or Edge with a GPU recommended.

Model download from huggingface.co · your messages never leave this device · standalone version ↗

What's running

Bonsai-1.7B — an open 1-bit model.

Bonsai-1.7B is an open model by Prism ML (Apache-2.0): built from Qwen3-1.7B and trained end-to-end for 1-bit weights. The WebGPU engine — hand-written compute kernels that decode the packed weights on your GPU — was published by the webml-community for the 27B flagship (MIT); we ported it to the plain qwen3 architecture of the 1.7B for this page. It is an independent demonstrator, not one of our Apertus fine-tunes.

1.125
Bits per weight

Q1_0: packed sign bits + FP16 group scales — about 14× smaller than the same model in FP16

0.24 GB
One-time download

Cached by your browser after the first load; runs offline from then on

32k
Token context window

The full Qwen3-1.7B context length, executed on your GPU

0
Bytes of data egress

Weights come down from Hugging Face; nothing you type goes anywhere

From demo to production

The pattern is the product.

What you just used is a pattern: a small, task-scoped model, quantized to fit the hardware you actually have, running where the data lives. In client work we apply the same pattern with different ingredients — Apertus, the open Swiss model, fine-tuned on your data, deployed in your jurisdiction, and held to an evaluation bar agreed before the project starts.

If your organisation has an inbox, a document stream or a form pipeline that people currently sort by hand, that's usually the right first project: narrow, measurable, and sovereign from day one.

Questions

The fine print.

Does anything I type leave my browser?
No. The model weights are downloaded once from Hugging Face and cached by your browser; after that, all computation happens on your own GPU via WebGPU. Your messages are never sent to any server — there is no server.
Why is the model only 0.24 GB?
Bonsai-1.7B stores its weights at roughly 1.125 bits each (packed sign bits plus small group scales) instead of 16 bits — about 14× smaller than the same model in FP16. Extreme quantization is one of the levers that makes sovereign, on-premise deployment affordable; our quantization concept page covers the trade-offs.
Is this the model sysf.io deploys for clients?
No — Bonsai-1.7B is an independent open model by Prism ML, used here purely to demonstrate on-device inference. Client systems are built on Apertus, fine-tuned on your data and evaluated against your task; the pattern is the same, the model is yours.
What do I need to run it?
A desktop browser with WebGPU (recent Chrome or Edge) and a GPU. The first run downloads ~237 MB, one time. If WebGPU isn't available the page will say so — nothing breaks.

Map your first narrow task.

A 30-minute scoping call — we'll tell you honestly whether a small task-tuned model fits.

Book a call See how it works