What does an internal generative AI really cost?

A word on method before we start: this article gives a calculation structure, not a price list. The amounts depend far too much on context, number of users, document volume, availability requirements, hosting, for any published price to mean anything. What does transfer, on the other hand, is the list of lines you must not forget and the logic that tips the decision.

The five real cost lines

An internal generative AI project breaks down into five lines, and experience shows that the last three are systematically underestimated in early costings.

The lines to budget for

Line	Nature	What makes it vary
GPU compute	Purchase or rental	Model size, number of concurrent users, context length
Hosting	Recurring	In-house room or datacentre, electricity, cooling, redundancy
Software licences	Recurring	Zero if the stack is open source, this is the line that disappears
Operations	Recurring	Monitoring, backups, model updates, user support
Enablement	One-off then recurring	Training, use cases, change management, governance

GPU compute

This is the only line that is genuinely calculable in advance, and that is fortunate: it depends on the size of the model, the quantisation chosen, the context length and the number of simultaneous requests. That is exactly what our VRAM calculator does. One important point: sizing must be based on real concurrency, not on total headcount. In a deployment covering several hundred staff members, only a handful query the model in the same second.

Operations, the forgotten line

An inference server is not a piece of software you install and forget. It has to be monitored, the document index has to be backed up, patches applied, the model upgraded when a better version appears, accounts managed and users answered. Over the depreciation period, this line can weigh as much as the hardware, depending on the level of service expected.

That is precisely what separates an "installed" infrastructure from an "operated" one. A costing that does not show this line is an incomplete costing: the workload will exist anyway, it will simply be absorbed by your IT department without having been budgeted.

Enablement

An AI nobody uses costs exactly its purchase price for zero benefit. What determines the return on investment is not model performance but the adoption rate: identifying use cases department by department, training, tooling, measuring. That is the purpose of our automation audit by department, which translates existing processes into hours that could be freed up.

The break-even point

The difference in nature between the two models is the key to the decision:

• A per-seat licence is a linear cost: doubling the number of staff members equipped doubles the bill, indefinitely.
• An internal infrastructure is a stepped cost: the same server takes on additional users at no extra cost, until the moment concurrent load means you have to add a card.

The break-even point is therefore written simply:

threshold = full annual cost of the infrastructure ÷ (price per user per month × 12)

To set the order of magnitude of the denominator, here are the public prices recorded in October 2026 on the vendors' official pages. In euros, Microsoft 365 Copilot is listed at €26.00 per user per month excluding VAT, paid yearly (€27.30 paid monthly, with an annual commitment), on top of a qualifying Microsoft 365 licence. Microsoft 365 Copilot Business, available only to customers with an eligible Microsoft 365 Business plan and for up to 300 users, is listed at €18.20 excluding VAT, paid yearly, reduced to €15.60 for the first year under a promotional offer on annual subscriptions, valid from 1 July to 31 December 2026. At OpenAI, a Standard ChatGPT Business seat costs $20 per user per month billed annually, or $25 billed monthly, a price that applies in most countries but may vary by country and currency; ChatGPT Enterprise is priced on request. These prices change: check them on the day you run your calculation.

Beyond that number of users, the internal infrastructure becomes cheaper, and the gap then widens with every new user, since each one is free. Below it, the subscription remains the more rational option: that is an honest answer, and it holds for many small organisations.

Two nuances shift that threshold, always in the same direction:

• The step is wide. A single server serves all equipped users, because only some of them query it at any given moment: as long as the concurrent load fits on the card, each additional user costs nothing more. Pooling is the real economic lever of internal AI.
• The infrastructure serves several uses. Within its capacity, the same server hosts the conversational assistant, document search, meeting-minute summaries and writing assistance. Under a licence model, usage beyond the included limits is paid for on top: with ChatGPT Business, for example, through workspace credits or through Premium seats at $100 per user per month billed annually.

Sizing depends on actual concurrent requests, context and model. Equipped user counts are not inference benchmarks; a dated test must specify throughput, median/p95 latency and failure rate.

Our return on investment simulator sets out this calculation with your own assumptions: spending actually eliminated, infrastructure cost, time saved and data risk avoided, the last two being assessed separately from the budget.

Three common calculation mistakes

1. Comparing a purchase with a subscription

Putting the purchase price of a server next to a monthly invoice mechanically distorts the comparison. Both must be brought back to the same unit: a full annual cost, calculated over the same horizon for both options, with the hardware depreciated over an explicit period. Use the period in your own depreciation schedule rather than a generic convention; by default our simulator works over a five-year horizon, which you can change, with no residual value.

2. Sizing for the absolute peak

Planning the infrastructure for the case where every staff member queries the AI at the same time leads to overspecifying by a considerable factor. Sizing is based on the load actually observed at peak hours, with a margin, not on a theoretical scenario.

In the same spirit, an internal AI has no reason to consume power at night or at weekends. Our deployments include a scheduled shutdown of the model's GPU server outside working hours: over an 8am to 6pm weekday window, counted as in our simulator over 52 weeks of 5 days, the machine runs for 2,600 of the 8,760 hours in a year and stays off roughly 70% of the time. Under hourly rental, those idle hours are not billed, subject to the host's terms (minimum billing, resources kept during shutdown, restart time); the associated electricity consumption disappears, apart from standby.

3. Forgetting the cost of doing nothing

This is the most expensive mistake, because it is invisible in the spreadsheet. With no internal tool, staff members are already using consumer AI services from their browser, with working documents. This is not a hypothesis: it is what we observe in almost every organisation we meet.

That cost does not show up on an invoice but as an exposure: data leaving the perimeter, no traceability, and a risk of sanction that the GDPR and the European AI regulation make very concrete. The method we use to quantify it is published on our risk-avoided calculation method page, and the full regulatory analysis is on our AI & GDPR and AI Act pages.

Reversibility is an economic argument too

One last line never appears in comparisons: the cost of leaving. A proprietary solution whose data, indexes and conversation histories live at the vendor creates a dependency that gets paid for at contract renewal, which is the moment when the price per user goes up.

An open source and open weight stack, hosted by you, removes that asymmetry: models are interchangeable, the index belongs to you, and changing supplier does not mean starting from scratch. That freedom has no line in the first year's budget. It has one in the fourth.

Key takeaways

The right reflex is not to ask "how much does it cost" but "from how many users, and over what period". An internal AI is a stepped investment that becomes more and more advantageous the more it is used, the exact opposite of a per-seat subscription. And that threshold is not something to guess but something to calculate: with your actual headcount, current prices and a full infrastructure cost, operations included.