Short answer. A production AI system usually costs several times the pilot that justified it, and the driver is volume, not the model. A single agentic request can trigger 5 to 50 model calls once tool use, retries, and verification loops are counted, and each call is billed per token. Add real traffic, messy production data, mandatory logging, and human oversight, and the running cost separates from the pilot fast. More than 85% of banks under European supervision now run AI, so the teams that budget only for the proof of concept are the ones that get surprised.
A pilot and a production system are different cost problems. A pilot prices one use case, on a small clean dataset, with a handful of internal testers. Production prices the same system under real load, with retries when calls fail, guardrail checks on every request, and storage for the logs a regulator will ask for. The unit price of a model call has fallen, the OECD price index for language models dropped by nearly 80% between January 2024 and April 2026, yet volume rises faster than price falls, so the bill grows.
Three examples of where the money actually goes:
The economic center of AI has moved from training to inference. Training is a one-time cost, inference is paid every time the system runs. In production, that recurring cost is the budget line that decides whether the case holds.
For a bank or insurer, production cost is not only infrastructure, it is compliance. The EU AI Act treats credit scoring and insurance pricing as high-risk uses. High-risk systems must keep automatic logs (Article 12), run post-market monitoring (Article 72), and hold human oversight (Article 14). Each obligation is a standing cost: log storage, a monitoring function, and staff time on review. Deployers must retain those logs for at least six months. None of this appears in a pilot that never touched a real customer.
DORA adds a second layer. An AI service that supports a critical or important function must be resilient, testable, and auditable, with the evidence held inside the bank’s own governance boundary rather than a vendor’s. That means logging you control, testing you can repeat, and third-party terms you can prove. The ECB has named risks from banks’ use of AI a supervisory priority for 2026 to 2028, so the monitoring and evidence cost is not optional and will be examined.
A production AI system usually costs several times its pilot, and volume, not the model price, drives the gap.
Four drivers move the number. First, volume: real users generate far more requests than a test group, and agentic workflows multiply the calls per request. Second, error handling: retries, fallbacks, and validation add calls that scale with configuration, not with user count. Third, data: production data is messy, so the system needs cleaning, enrichment, and re-processing the pilot never did. Fourth, egress and infrastructure: moving data between services carries fees that can reach 15 to 30 percent of cloud AI cost and rise with every interaction. A pilot hides all four because it runs small, clean, and short.
Forecast from the production shape, not the demo. Estimate real request volume per month, then multiply by the calls per request the design actually needs, including retries and guardrail checks. Price that against current token rates, and model the storage and monitoring the regulation requires. Run a load test at a fraction of real volume and extrapolate, rather than reading cost off a ten-user pilot. The forecast that survives contact with production is built on request volume times calls per request times token price, plus the fixed cost of logging and oversight. A team that has shipped production systems can put real ranges on each of those inputs. A team that has only run pilots is guessing.
Three recurring lines. Logging and retention: high-risk systems log automatically and keep records for months, which is continuous storage and pipeline cost. Human oversight: a person must be able to review and override decisions, so oversight is a staffed role, not a checkbox. Monitoring and reporting: post-market monitoring under the EU AI Act and incident reporting under DORA both require tooling and the people who run them. These costs are small next to a fine, the EU AI Act sets penalties up to 35 million euros or 7% of global turnover, but they are real and they recur every month the system runs.
Treat cost as a governed metric, not a surprise. Track spend per use case, set a cost budget per request the way you set a latency budget, and alert when it drifts. Cut calls where an agent loops more than the task needs, cache repeated retrievals, and route simple requests to smaller models. Right-size retries so error handling does not quietly triple the bill. The banks that stay in control measure cost the day the system goes live, not the quarter after finance flags it.
Ableneo shipped 34 production AI projects in 2025, and 94% of them use large language models, so the running cost of an LLM system in production is a number Ableneo forecasts from delivery data, not from a demo. Roughly four of five projects reach production, which means the cost model is tested against real traffic, real retries, and the logging that regulated clients require. That is the difference between a pilot budget and a production budget. Ableneo’s AI transformation work prices the system a bank will actually run, with the monitoring and oversight the EU AI Act and DORA demand built into the plan.
Key takeaways
Planning AI in a regulated business? Ableneo takes systems from classification to governed production.