RAG, visually
How an LLM answers questions on your documents: you index once, then every question retrieves the right extracts before a sourced answer is generated.
- LLM (large language model)
- A neural network trained on huge volumes of text to understand and generate language. It is the "brain" of a generative AI. See our LLM models page.
- RAG (Retrieval-Augmented Generation)
- A technique that connects an LLM to your documents: the system retrieves the relevant passages, then hands them to the model, which answers while citing its sources. This is what makes an AI genuinely useful on your business data.
- Parameters (e.g. 24B)
- The number of "weights" in a model, in billions (B). The more there are, the more capable the model, but the more GPU memory it consumes. Estimate that memory with our VRAM calculator.
- Quantisation (FP16, INT8, INT4)
- Reducing the precision of the weights in order to lower the memory required. Moving from FP16 (16 bits) to INT4 (4 bits) divides the memory taken by the weights by four, usually with only a small loss of quality.
- VRAM (GPU memory)
- The memory of the graphics card where the model resides during inference. It has to hold the weights, the KV cache and a safety margin. On Apple Silicon, this is called unified memory.
- KV cache
- The memory that stores the context of every request in progress. It grows with the context length and the number of simultaneous requests, and it is often the item that explodes when you serve many users.
- Open source vs open weight
- An open weight model publishes its weights (freely usable and deployable); an open source model goes further by also opening the code and sometimes the data. Both allow sovereignty and reversibility.
- Embeddings (vectors)
- A numerical representation of the meaning of a text. Embeddings enable semantic search: finding the documents closest to a question, even when the keywords are not identical. A core building block of RAG.
- Context window
- The amount of text (in tokens) a model can take into account at once. A 128k token window makes it possible to analyse long documents in full.
- Hallucination
- A plausible but false answer generated by an LLM. RAG and sourced answers sharply reduce this risk by anchoring the answers in your real documents.
- Fine-tuning
- Adjusting a pre-trained model on specific data. RAG is often enough and avoids the cost of fine-tuning; fine-tuning remains useful for very particular styles or tasks.
- Digital sovereignty
- Keeping control of your data, your tools and your technology choices, without depending on a foreign player. Applied to AI, it rests on EU hosting and open source. See Sovereignty.
- CLOUD Act
- A US law allowing US authorities to demand access to data held by American companies, wherever that data is. It is the major legal risk of AI hosted by US vendors.
- SSO (single sign-on)
- Single Sign-On: your users log in to the AI with their existing account (the organisation's directory), with no new password. Better security and better comfort.
- Inference
- Running an already trained model to produce an answer. This is what consumes the GPU day to day, measured in tokens per second.
From theory to your own sovereign AI
We turn these concepts into a concrete solution, deployed and operated for you.
Request a demonstration