Most “AI for business” still means sending your documents to a US cloud and trusting a policy page. There is another way: a private assistant that runs on infrastructure you control, trained on your own content, with nothing leaving your boundary. Here is what that actually takes — and where the honest trade-offs are.
Local model or European API?
The first fork is whether the model runs on your own hardware or behind an API.
Self-hosted, open-weight models
Open-weight models you run yourself — Mistral's open models are the leading European example, alongside families like Llama and Qwen — keep every inference inside your perimeter. Nothing is sent to a third party because there is no third party. The cost is hardware: a capable GPU with enough VRAM, and someone to operate it.
A European, private API
If self-hosting a model is too much, a European provider's API — Mistral's hosted models, for instance — keeps data under European jurisdiction without you running GPUs. It is not the same as fully local, but it is a world away from piping your documents to a US hyperscaler, and it is often the pragmatic middle path.
Making it useful: RAG over your own documents
A raw model knows nothing about your business. The technique that fixes this is RAG — retrieval-augmented generation. In plain terms:
- Your documents are split into chunks and turned into embeddings — numeric representations of meaning — stored in a vector database.
- When someone asks a question, the system retrieves the most relevant chunks and hands them to the model as context.
- The model answers from your actual content, with citations, instead of guessing or hallucinating.
Crucially, in a private setup the documents, the embeddings and the vector store all live on your infrastructure. The assistant gets smarter about your business without your business leaving the building.
Access and networking
A private assistant still has to be reachable by your team — securely, without exposing it to the open internet. A modern approach uses a zero-trust mesh network such as NetBird (built on WireGuard): only enrolled, authenticated devices can reach the service, and nothing is published publicly. Hosting can sit on a European GPU provider like OVHcloud, or on your own hardware where latency and control matter most.
The honest trade-offs
- Capability — the very largest frontier models still lead on the hardest tasks; for most business document work, open and European models are more than enough.
- Cost — self-hosting trades per-token API fees for hardware and operations; the maths depends on volume.
- Effort — a private stack is more to set up and run than signing up for a SaaS, which is precisely why it is worth doing properly, once.
Done well, the payoff is a genuinely private assistant: useful on your own documents, sovereign by construction, and never a line item on someone else's training data.
Frequently asked questions
- Does a private AI mean my data is used to train someone's model?
- No — that is the point. With a self-hosted open-weight model, nothing leaves your infrastructure. With a European private API, your data stays under European jurisdiction and reputable providers do not train on business API traffic. Either way, your documents are not feeding a US training set.
- What is RAG, simply?
- Retrieval-augmented generation. Instead of relying on what a model memorised, the system retrieves the relevant passages from your own documents and gives them to the model as context, so answers are grounded in your content and can cite their source.
- Do I need expensive GPUs?
- For fully local models, yes — a suitable GPU with enough VRAM. If that is overkill for your needs, a European hosted API gives you privacy and jurisdiction without buying hardware. We help you size the right option for the workload.
- Is a self-hosted assistant as capable as ChatGPT?
- For the hardest frontier tasks, the largest commercial models still lead. For grounded work over your own documents — search, drafting, summarising, answering policy questions — a well-built private setup is more than capable, and it keeps your data yours.