What does an internal generative AI really cost?

A word on method before we start: this article gives a calculation structure, not a price list. The amounts depend far too much on context, number of users, document volume, availability requirements, hosting, for any published price to mean anything. What does transfer, on the other hand, is the list of lines you must not forget and the logic that tips the decision.

The five real cost lines

An internal generative AI project breaks down into five lines, and experience shows that the last three are systematically underestimated in early costings.

GPU compute

This is the only line that is genuinely calculable in advance, and that is fortunate: it depends on the size of the model, the quantisation chosen, the context length and the number of simultaneous requests. That is exactly what our VRAM calculator does. One important point: sizing must be based on real concurrency, not on total headcount. In a deployment covering several hundred staff members, only a handful query the model in the same second.

Operations, the forgotten line

An inference server is not a piece of software you install and forget. It has to be monitored, the document index has to be backed up, patches applied, the model upgraded when a better version appears, accounts managed and users answered. Over three years, this line often weighs as much as the hardware.

That is precisely what separates an "installed" infrastructure from an "operated" one. A costing that does not show this line is an incomplete costing: the workload will exist anyway, it will simply be absorbed by your IT department without having been budgeted.

Enablement

An AI nobody uses costs exactly its purchase price for zero benefit. What determines the return on investment is not model performance but the adoption rate: identifying use cases department by department, training, tooling, measuring. That is the purpose of our automation audit by department, which translates existing processes into hours that could be freed up.

The break-even point

The difference in nature between the two models is the key to the decision:

• A per-seat licence is a linear cost: doubling the number of staff members equipped doubles the bill, indefinitely.
• An internal infrastructure is a stepped cost: the same server serves 50 or 300 users, until the moment you have to add a card.

The break-even point is therefore written simply:

threshold = full annual cost of the infrastructure ÷ (price per user per month × 12)

Beyond that number of users, the internal infrastructure becomes cheaper, and the gap then widens with every new user, since each one is free. Below it, the subscription remains the more rational option: that is an honest answer, and it holds for many small organisations.

Two nuances shift that threshold, always in the same direction:

• The step is wide. In our deployment at Annemasse, a single GPU serves 340 users, and we have validated more than 200 concurrent users in production. Pooling is the real economic lever of internal AI.
• The infrastructure serves several uses. The same server hosts the conversational assistant, document search, minute summarisation and writing assistance. Under a licence model, each new use is often charged as an extra.

Our return on investment simulator sets out this calculation with your own assumptions: time saved, infrastructure cost, data risk avoided and compliance savings.

Three common calculation mistakes

1. Comparing a purchase with a subscription

Putting the purchase price of a server next to a monthly invoice mechanically distorts the comparison. Both must be brought back to the same unit: a full annual cost over an explicit depreciation period, three years being a reasonable convention for GPU hardware, five years if usage is stable.

2. Sizing for the absolute peak

Planning the infrastructure for the case where every staff member queries the AI at the same time leads to overspecifying by a considerable factor. Sizing is based on the load actually observed at peak hours, with a margin, not on a theoretical scenario.

In the same spirit, an internal AI has no reason to consume power at night or at weekends. Our deployments include a scheduled shutdown of the GPUs outside working hours: over an 8am to 6pm weekday window, roughly two thirds of machine time is not billed, nor is the associated electricity consumption.

3. Forgetting the cost of doing nothing

This is the most expensive mistake, because it is invisible in the spreadsheet. With no internal tool, staff members are already using consumer AI services from their browser, with working documents. This is not a hypothesis: it is what we observe in almost every organisation we meet.

That cost does not show up on an invoice but as an exposure: data leaving the perimeter, no traceability, and a risk of sanction that the GDPR and the European AI regulation make very concrete. The method we use to quantify it is published on our risk-avoided calculation method page, and the full regulatory analysis is on our AI & GDPR and AI Act pages.

Reversibility is an economic argument too

One last line never appears in comparisons: the cost of leaving. A proprietary solution whose data, indexes and conversation histories live at the vendor creates a dependency that gets paid for at contract renewal, which is the moment when the price per user goes up.

An open source and open weight stack, hosted by you, removes that asymmetry: models are interchangeable, the index belongs to you, and changing supplier does not mean starting from scratch. That freedom has no line in the first year's budget. It has one in the fourth.

Key takeaways

The right reflex is not to ask "how much does it cost" but "from how many users, and over what period". An internal AI is a stepped investment that becomes more and more advantageous the more it is used, the exact opposite of a per-seat subscription. And for most organisations with more than a hundred staff members, the threshold is crossed far sooner than people imagine.