Short answer. Default to a hosted LLM under a signed data processing agreement, and self-host only when a specific function crosses a hard line. There are two lines. The first is data that cannot legally or contractually leave the bank’s control. The second is volume: a dedicated deployment starts to beat a managed API somewhere around 100 to 256 million tokens per month. Model quality is rarely the deciding factor in 2026, because the gap between the best hosted and the best open-weight models has closed. Data residency, DORA third-party obligations, and cost at your real volume decide it.
A “hosted LLM” means you call a model through a provider’s API and the provider runs the infrastructure. “Self-hosting” means you run open-weight models on your own compute, in your own cloud tenant or data center, and you own the operations. The choice is not all or nothing. Most banks end up with both, split by use case.
The pattern that works is to sort use cases by data sensitivity and volume, not by a blanket policy. A customer-facing assistant handling account data and a back-office document summarizer handling public filings sit in different risk classes and belong on different infrastructure.
Ableneo has shipped 34 production AI projects across financial services and other regulated sectors, and 94% of them use LLMs. The consistent lesson is that the infrastructure decision is a per-use-case decision, taken once you can see the data class and the expected volume, not a single procurement choice made at the top.
For a bank, this is a compliance decision before it is a technology decision. The Digital Operational Resilience Act, Regulation (EU) 2022/2554, has applied directly since 17 January 2025. It treats an LLM provider that supports a critical or important function as an ICT third-party service provider, with obligations for contracts, monitoring, exit plans, and entry in the register of information the bank reports to its competent authority at least once a year.
The EU AI Act adds a second layer. High-risk AI systems, which include AI used in credit scoring and creditworthiness assessment, face obligations for logging, human oversight, and record-keeping that reach full application in August 2026. A hosted model does not remove these duties. The bank remains the deployer and stays accountable for oversight and audit trails, whichever way the model is run. The ECB has been direct on the point: technology is neutral, governance is not.
Default to a hosted LLM under a data processing agreement, and self-host only when data cannot leave the bank’s control or volume passes roughly 100 to 256 million tokens per month.
Self-hosting pays off in two situations, and it is worth being precise about both. The first is a legal or contractual constraint: data that cannot leave the bank’s control, a supervisory expectation on residency, or a client contract that forbids sending data to a third-party processor. Here self-hosting is the compliant option regardless of cost. The second is scale. Once a use case runs at high, stable token volume, roughly above the 100 to 256 million tokens per month range depending on the model, a dedicated deployment on reserved compute is cheaper per token than a metered API, and the gap widens as volume grows.
Below that line, self-hosting usually costs more, not less, once you count GPU capacity, the engineering team to run inference reliably, patching, and the security controls a bank needs. The mistake is to self-host for a sense of control on a low-volume workload, then carry a fixed cost that a hosted API would have handled for a fraction of the price.
If a hosted LLM supports a critical or important function, DORA requires the bank to treat the provider as an ICT third party across the full lifecycle. That means a written contract with defined service levels and audit rights, an ex-ante risk assessment before onboarding, continuous monitoring, and a documented, tested exit plan so the bank can move off the provider without disrupting the function. The provider goes into the register of information, together with any subcontractors in the chain. The bank keeps full responsibility for compliance at all times; it cannot be delegated to the vendor.
Concentration risk is the part banks underestimate. The ECB has noted that cloud services account for roughly 15% of outsourcing contracts, and about half of those cover critical or important functions, in a market served by only a few large providers. Routing every AI use case through one hosted provider recreates that dependency. A workable exit plan, and often a second sourcing option, is part of doing this properly.
Data residency is the constraint that most often forces the decision. Under GDPR, an LLM provider that processes personal data is a data processor, which requires a data processing agreement and clarity on where the data is stored and who can access it. Many hosted providers now offer in-region deployment and a zero-retention or no-training commitment, and for a large share of banking use cases that is enough to satisfy residency and supervisory expectations without self-hosting.
Where it is not enough, because the data is too sensitive or a contract forbids any external processing, a private deployment inside the bank’s own in-region cloud tenant, or fully on-premise inference, keeps the data on infrastructure the bank controls and can show an auditor. The practical rule is to define the data class first, confirm what residency and retention the use case requires, then pick the lightest deployment that meets it.
Most banks do not need a binary answer, and the strongest architectures are hybrid. A dedicated or provisioned deployment gives a bank an isolated instance of a hosted provider’s model, in a chosen region, with capacity reserved for that bank alone. It removes multi-tenant and residency concerns while leaving the operations to the provider. Open-weight models running in the bank’s own tenant cover the cases where nothing may leave at all.
The design that holds up routes each use case to the lightest option that satisfies its data class and volume, behind one governance layer that logs every call, enforces access control, and keeps the audit trail the EU AI Act and DORA both expect. The infrastructure underneath can be mixed. The controls above it cannot be.
Ableneo builds the governance and integration layer that makes this decision auditable, not just technical. Across 34 production AI projects, with about 4 of every 5 reaching production, the pattern is consistent: banks that decide per use case, on data class and real volume, ship faster and pass audit more cleanly than banks that set one blanket rule. Our work in financial services covers the full path, from classifying use cases and sizing volume to standing up the logging and human-oversight controls that DORA and the EU AI Act require. See the Ableneo AI Transformation FAQ for the decisions that surround this one.
Key takeaways
Planning AI in a regulated business? Ableneo takes systems from classification to governed production.