Sovereign generative AI, in production at the Calvados département

The principle: an open stack, controlled end to end

The technical stack: open building blocks, operated end to end

• 01

  An open weight model, served by the platform

  Mistral Small 3.2 24B Instruct (version 2506), released by Mistral AI under the Apache 2.0 licence, runs in a vLLM container on the GPU server and is queried through an OpenAI-compatible API. The language model and the embedding model share a single GPU: an NVIDIA A100 80 GB by default, with automatic fallback to an H100 or an H200 when it is unavailable.

• 02

  Sourced answers

  Extracted text is split into chunks, embedded by an embedding model also served by vLLM on the same GPU, then searched semantically. The BAAI bge-reranker-v2-m3 reranker, also served by vLLM, re-orders the retrieved chunks before they are passed to the model, for sourced answers.

• 03

  Multi-format extraction

  PDF (native text, tables converted to Markdown, GPU OCR of images with orientation detection using docTR), recent Word, PowerPoint and Excel documents (.docx, .pptx, .xlsx) and HTML pages using Docling, and text files (txt, csv, md, json, xml, source code). The extraction engine writes each document to a temporary file, deletes it after processing and sends the extracted text back to the portal.

• 04

  A collaborative portal

  The interface is Open WebUI: chat, documents and knowledge bases. The portal is configured through its API to use the platform's own extraction engine and reranker.

• 05

  Daytime compute, in the EU

  From Monday to Friday, a scheduled task creates and configures the model's GPU server in the morning, then shuts it down and deletes it in the evening: it is rebuilt from scratch each morning and is not created at weekends. This server is only rented in data centres located in the European Union.

• 06

  An encrypted tunnel, monitored operations

  At every start, the model's GPU server joins a WireGuard encrypted tunnel that links it to the portal. Each internal service (model, embeddings, reranker, extraction) requires an access token. Prometheus monitors the GPUs and the inference engine, and an email report is sent at every start and every shutdown, with an alert if anything fails.

What this demonstrates

• A reproducible model. The stack first deployed in a conurbation is in production in a département, with the same model and the same architecture: a local authority does not have to start from a blank page.
• Sovereignty built into the architecture. The language model, embeddings, re-ranking and extraction are served by the platform itself, rather than called from an AI provider. The model's server is rented in the European Union and linked to the portal by an encrypted tunnel.
• Costs contained by design. The model's GPU server is only rented Monday to Friday during the day, the model and the embeddings share a single GPU, and before each start a check verifies that the compute credit covers the day. The actual cost depends on the hosting provider's offer and on the other services, which may stay on.
• Automated operations. Starting, configuring and shutting down the model's GPU server, as well as the reports, need no manual intervention; monitoring covers the GPUs and the inference engine.