Concepts · Small models
Small models, measured.
Two questions decide a sovereign build: how big a model does the task need, and which base do you start from. This page answers both — capability versus size, what each size class can actually do, and the spectrum from fully-Swiss Apertus to European and international open-weight models — with the honest limits and the compliance framing included.
Size
How much model does the task need?
Capability rises with size — but it flattens, and the returns get expensive fast. Most regulated work doesn't live at the frontier; it lives in the bands a small model can reach.
Model size is measured in parameters — from ~1B up past 200B. More parameters buy broader knowledge and deeper reasoning, but the curve bends: the jump from 1B to 8B is transformative, 8B to 70B is real but costly, and beyond that you pay a lot for a little on most tasks.
The trick is that the size you need is set by the task, not by ambition. Extraction, classification, routing and structured drafting sit in the lower bands — a small model's home turf. And the red arrow below is the whole point of this page: fine-tuning a small model on one narrow task lifts its effective capability into the band above, so a tuned 8B beats a far larger generalist on the work you actually run.
The effect
Specialisation beats scale on narrow tasks.
General benchmarks reward breadth. Your workflow rewards depth — your templates, your terminology, your formats, your edge cases.
Out of the box, a general-purpose small model is a generalist that happens to speak your language. Fine-tuned on a few thousand examples of your task, it becomes a specialist that has read your corpus. How much that buys depends on where the base model starts — which is exactly what the ablation in every Pilot measures, on your documents, before you commit.
The public reference point in Switzerland: the Canton of Ticino's translation service went from 90% to 94% accuracy after fine-tuning Apertus-8B on cantonal data, running entirely on cantonal infrastructure. Four points at the top of the scale — in a production service, the difference between spot-checking output and re-doing it.
A frontier API has two structural handicaps on this ground: it has never seen your documents, and it never can — because they're the documents you can't send away. The small model's advantage isn't intelligence. It's access.
The method
Fine-tuning teaches behaviour. Retrieval supplies facts.
The two are routinely conflated — and the conflation is where AI projects go wrong.
Fine-tuning
Teaches the task: your terminology, document formats, tone, output structure, refusal behaviour. It does not reliably memorise facts — using it as a knowledge store is a design error we refuse to ship.
Retrieval (RAG)
Supplies the facts: the current version of your policies, contracts and records, cited back to the source page. Knowledge updates the moment the document does — no retraining.
The ablation
Every Pilot scores all four arms — base, retrieval, fine-tune, combined — on your evaluation set. The table says which system is worth buying.
Measured, the split is stark: in our FINMA ablation, fine-tuning alone took format compliance from ~2% to 100% yet left answer-plus-citation accuracy at 1.6% — behaviour learned, facts not. Retrieval alone: 22.4%. Combined: 45.4%.
The base
Fully Swiss, or as flexible as your compliance allows.
Apertus is our default — Switzerland's fully open, fully transparent LLM. But sovereignty isn't only Swiss. Where your policy permits, a European or international open-weight base can win your evaluation and still run entirely on your infrastructure.
Fully Swiss
Sovereign & fully transparent
Apertus
Open weights and open training data, Swiss origin. The choice when every layer has to be auditable — public sector, or the strictest compliance postures.
European
EU-based, open weights
Mistral · Teuken · EuroLLM
European providers, open weights, strong EU-language coverage. Provenance stays inside Europe; training data is typically not published.
International
Best open performance
Llama · Qwen · Gemma
The strongest open-weight models regardless of origin. Maximum capability and ecosystem — the trade is training-data transparency and, for some, country of origin.
Does the EU AI Act let you use a model like Qwen?
Short version: the Act regulates by use-case risk and places transparency obligations on the provider of a general-purpose model — it does not restrict a base model by its country of origin. Running any open-weight model on your own infrastructure keeps your data in your jurisdiction regardless of where the weights were trained.
What can rule a model out is your own procurement or data-governance policy: several Swiss and EU public bodies restrict Chinese-origin technology by internal policy, not by the Act. That's a decision we map with you in scoping — and it's exactly why the fully-Swiss option exists. Not legal advice; we work alongside your compliance and legal teams.
The catalogue
The base models we build on.
A working shortlist, grouped by origin. Sizes and hardware are facts; performance and speed are qualitative — the real ranking always comes from the evaluation on your task.
Swiss — maximum sovereignty
Apertus
EPFL · ETH Zurich · CSCS (Switzerland)
- Sizes
- 8B · 70B
- Hardware
- 8B → workstation · 70B → node
Switzerland's sovereign LLM — the only base here that publishes its training data and recipe, not just its weights. Multilingual by construction, including Swiss German and Romansh.
Best for: Public sector, regulated Q&A, translation — anywhere full auditability is the requirement
Performance & speed: Strong on tuned narrow tasks and Swiss-national languages; trails the largest international models on open-ended general work
+ Strength
Complete data lineage an auditor can inspect; Swiss origin and jurisdiction
△ Trade-off
Younger ecosystem and fewer sizes than the incumbents
Europe — EU-based, open weights
Mistral
Mistral AI (France)
- Sizes
- 8B · ~24B · larger
- Hardware
- 8B → workstation · 24B → single 24–48 GB GPU
Efficient French open-weight family with permissive (Apache-2.0) releases and strong European-language coverage.
Best for: Multilingual EU document work, drafting and summarisation on a tight hardware budget
Performance & speed: Excellent capability-per-parameter; a frequent evaluation winner on European multilingual tasks
+ Strength
EU-based, permissive licences, very strong performance for the size
△ Trade-off
Training data not published; the largest tiers aren't all open
Teuken · EuroLLM
OpenGPT-X / EU consortia (Europe)
- Sizes
- 7B · 9B
- Hardware
- → workstation
Publicly-funded European models trained for broad EU-language coverage — all 24 official languages — with unusually open documentation.
Best for: EU-sovereign deployments that want a European public-research provenance story and wide-language coverage
Performance & speed: Solid multilingual coverage at small size; a smaller ecosystem and fewer tuned checkpoints than Mistral
+ Strength
European public-research provenance; broad official-language coverage
△ Trade-off
Smaller and less battle-tested than the market leaders
International — best open performance
Llama
Meta (USA)
- Sizes
- 8B · 70B · larger
- Hardware
- 8B → workstation · 70B → node
Meta's open-weight family — the most mature tooling and community ecosystem of any open model.
Best for: General-purpose baselines and agentic tooling where ecosystem maturity matters
Performance & speed: Strong general capability across sizes; a reliable evaluation contender
+ Strength
Unmatched ecosystem, tooling and fine-tuning support
△ Trade-off
US origin; a community licence with use restrictions; training data not public
Qwen
Alibaba (China)
- Sizes
- 7B–72B dense · 235B MoE
- Hardware
- 8B → workstation · 32B → single GPU · 72B / MoE → node
Among the strongest open models on reasoning, coding and multilingual work, with the widest size range — including memory-efficient mixture-of-experts.
Best for: The most demanding open-weight tasks where capability is the priority and policy permits it
Performance & speed: Frontier-open-tier on reasoning and multilingual benchmarks; MoE variants run faster than a dense model of the same total size
+ Strength
Top open performance, permissive licences, a size for every hardware budget
△ Trade-off
China origin — some Swiss and EU public-sector procurement policies exclude it; training data not public
Gemma
Google (USA)
- Sizes
- 2B · 9B · 27B
- Hardware
- → workstation (all sizes)
Google's compact open-weight family, tuned for strong quality at small and mid sizes that fit a single GPU.
Best for: Latency-sensitive single-GPU and edge workloads
Performance & speed: Competitive quality-per-parameter in the small and mid range
+ Strength
Small, fast, easy to run on modest hardware
△ Trade-off
US origin; capped at 27B; training data not public
Sizing the hardware behind these is its own discipline — see quantization for how a 70B fits in ~40 GB and an 8B in under 5 GB, and the sovereign runtime for the deployment shapes.
The economics
An order of magnitude cheaper to run — and faster.
Small models don't just fit your compliance envelope. They fit your budget — in francs and in milliseconds.
Serving cost: running a 7–8B model costs roughly an order of magnitude less than a 70B-class model — the difference between one workstation-class GPU and a multi-GPU node, between a purchase order and a procurement round.
Latency: every cloud API call pays a round-trip tax — network hops, TLS, queueing behind other tenants — before your tokens even reach a GPU; first tokens routinely arrive only after several hundred milliseconds. A small model in your own rack skips that tax entirely, and its faster prefill and decode compound on top — which is what makes high-volume document pipelines and interactive assistants feel instant.
The sovereignty multiplier: every parameter you don't need is compliance you don't have to buy. A task-tuned small model runs on one machine — the workstation shape in the sovereign runtime — which opens the air-gapped and edge deployments (shop floor, clinic, field service) where hosted platforms and US APIs structurally cannot follow.
The limits
When a small model is the wrong choice.
An expert who won't tell you when their favourite tool loses isn't an expert. Small models lose here:
Open-ended assistance
"Answer anything about anything" rewards scale. If you need a general chat assistant over public knowledge, a frontier model is better at it.
Broad world knowledge
Small models know less. Retrieval closes the gap only where your corpus has the answer — it can't retrieve what nobody wrote down.
Deep multi-hop reasoning
Long chains of novel reasoning still favour larger models. Routine volume goes small; the hard tail escalates to a larger local model.
Non-sensitive, low-volume work
If the data can leave and the volume is trivial, an API subscription is simpler. We'll say so in the scoping call.
The dividing line isn't opinion — it's the evaluation set. That's why every engagement starts with one: evaluation-first delivery.
FAQ
Small-model questions.
Is an 8B model really good enough for German legal or clinical text?
Can you use a non-Swiss base like Mistral or Qwen — and does the EU AI Act allow it?
What about reasoning-heavy work?
Doesn't fine-tuning become obsolete every time a better base model ships?
Would a small model clear your bar?
One workflow, your documents, our harness. We'll build the evaluation set and show you the ablation — in your jurisdiction.