Klarna’s AI assistant now handles 2.3 million customer conversations a month and cut resolution time from eleven minutes to under two. JPMorgan runs upward of 450 AI use cases in production, on a platform that touches roughly $10 trillion in daily transactions. McKinsey puts the total addressable value of generative and agentic AI in banking at close to $2 trillion a year, once revenue, cost, and risk are counted together, the equivalent of 9 to 15 percent of the industry’s operating profit from productivity gains alone.
None of that is hype. It’s in the filings and the case studies, not the keynote slides.
Here’s the number that doesn’t make it into the keynote: only 38% of banks that have adopted AI report seeing an actual financial return. MIT’s research on enterprise AI puts the pilots-that-reach-production rate at roughly 5%, with more than half of generative AI proofs of concept abandoned before they ship. Gartner’s banking CIOs are past “should we do AI” and stuck on “why doesn’t it show up in the numbers yet” — 57% report pressure to improve productivity from AI, 52% to cut costs with it, and most are still waiting.
So the $2 trillion is real, and so is the 38%. Both numbers describe the same industry, the same models, often the same vendors, sometimes the same bank running two pilots six months apart with opposite outcomes. The difference between the banks in the McKinsey number and the banks in the Gartner number isn’t the AI. I’ve spent 20+ years building the layer underneath — target architectures, system integration programs, data governance at regulated-bank scale, in institutions with billions of dollars in assets and hundreds of interconnected systems. I can tell you exactly where that difference lives. It’s not in the model. It’s in four layers of architecture that either exist or don’t, and most banks skip straight past them to the demo.
This piece is that architecture, in the order it actually has to be built, with the regulatory deadline that just made ignoring it a lot more expensive.
Look closely at the banks that are actually collecting on the AI promise, and a pattern shows up fast: the win never comes from the model getting smarter. It comes from the model getting access.
JPMorgan’s Contract Intelligence platform reclaimed roughly 360,000 hours of lawyer time a year, not because a large language model reads contracts better than a lawyer, but because someone built the pipeline that lets the model see every contract, in a structured form, with a workflow on the other end that acts on what it finds. Klarna’s $60 million in savings came from an assistant that can actually resolve a refund or a dispute end to end, not one that summarizes the policy and hands the customer back to a queue. Across HSBC, Citi, UBS, DBS, and ING, the reported pattern is consistent: 20–40% lower cost to serve, 10–30% revenue uplift, concentrated in the domains where the bank rewired the full workflow — data, decision, and action — rather than dropping a chat widget onto the existing one.
IDC’s figure for agentic AI specifically is a 2.3x return within thirteen months, and it’s trending upward as deployments mature. McKinsey’s banking-specific breakdown shows 3–15% higher revenue per relationship manager and 20–40% lower cost to serve, again concentrated wherever a bank did the unglamorous work of connecting a model to a real process with real permissions.
Notice what those two levers are: cost to serve, and revenue per relationship. Neither moves because a model got better at language. They move because a workflow that used to require a human handoff now doesn’t, or because a relationship manager who used to spend four hours a week assembling a client view now spends four minutes reviewing one an agent assembled. That’s not intelligence. That’s plumbing. Good plumbing, expensively built, but plumbing.
That’s the part vendor demos never show you, because the demo doesn’t need permissions. The demo needs a slide.
The industry-wide numbers on pilot failure are ugly enough to make you wonder if the technology works at all. It does. What doesn’t work is bolting it onto what’s already there.
Sixty-eight percent of bank CTOs name legacy systems as their single biggest obstacle to AI, with typical delays running twelve to eighteen months before anything ships. The pattern behind that number is almost always the same, and it has nothing to do with model quality: an AI agent can only act where the underlying architecture lets it act. Most core banking platforms expose configuration screens, not logic. Credit decisioning rules, workflow state, product eligibility, it’s all locked inside a vendor’s proprietary layer that a model can query, sometimes, but never actually operate. You end up with a very articulate agent standing in front of a locked door, describing the room on the other side.
The market has started to notice. Fiserv, one of the three incumbent core banking providers most banks still run on — launched “agentOS” in May 2026, co-built with OpenAI and AWS, specifically to give agents a place to act inside a core that was never designed for them. Half a dozen institutions are running it in pilot, two of them live. That’s not a feature announcement. That’s an admission, from inside the industry, that the core was the actual blocker the whole time — and a tell that the vendors who own the core know it before most of their clients do.
I’ve watched this exact failure mode play out at 90-billion-dollar-asset scale, long before “agentic AI” was a phrase anyone used. Give a smart system access to seventy applications that don’t agree on what a “customer” is, and the smart system inherits the disagreement, then confidently reports it as fact. The model isn’t the bottleneck. It never was. It’s the most replaceable part of the whole stack, which is exactly why it gets all the attention and none of the budget that actually needs to move.
Here’s the model I use, built from two decades of watching banks try to bolt intelligence onto systems that weren’t built to receive it. It has four layers. Skip one, and the ceiling on what AI can do for you drops hard, usually down to “chatbot with a wrapper,” which is a 1.2x outcome wearing a 10x pitch deck.
Before an agent can act, something has to separate “the logic a decision depends on” from “the system that happens to run it today.” This is the oldest problem in enterprise architecture wearing a new costume: you cannot automate what you cannot address. A composable core — APIs that expose actual business logic, not just data reads — is what turns “the agent can see the loan file” into “the agent can move the loan file.” Everything downstream depends on this layer existing first, which is exactly why it’s the one banks skip. It’s the least glamorous, the most expensive, and the only one that doesn’t demo well.
In a multi-billion-dollar asset management target-architecture program I led, the first eighteen months weren’t spent on anything resembling AI. They were spent mapping and decoupling seventy-plus applications so the organization finally had one consistent picture of its own systems. That unglamorous mapping work cut regulatory reporting time in half. It also happens to be the exact prerequisite an agent needs today to do anything beyond read-only summarization. Nobody called it “AI-readiness” in year one. It was AI-readiness anyway, most durable architecture work is retroactively AI-readiness, which is a useful thing to remember the next time a modernization budget gets deprioritized in favor of a pilot.
An agent that can act on your core still needs to know what the words mean. Ask three departments in any bank what “revenue,” “active customer,” or “at-risk exposure” means, and you’ll get three answers, all defensible, all incompatible. A model trained on that ambiguity doesn’t resolve it — it picks one, silently, and is wrong for two of your three departments without telling anyone. That’s arguably worse than a system that fails loudly, because nobody goes looking for a bug that reports a confident, plausible number.
The semantic layer is the governed, versioned, machine-readable contract that says: this is what these terms mean, here, in this bank, and here’s who owns the definition when it changes. I built exactly this kind of layer across a 150-plus-system integration program at a tier-one corporate and investment bank, streaming through an ELK/Kafka platform at over 15,000 documents a second. The volume isn’t the point. The point is that every one of those systems now agreed on what a “position” was. That agreement is what lets an agent reason across systems instead of hallucinating a bridge between them — and it’s the layer most vendor RAG pitches quietly assume already exists, because building it isn’t their problem to solve.
This is the layer people mean when they say “agentic AI,” and it’s the one that gets the most hype and the least architectural rigor. Orchestration is not a chatbot with more steps. It’s a workflow layer that knows which decisions an agent can make alone, which ones need a human, which ones need two humans and a four-eyes control, and what happens when the agent is uncertain. Loan origination, KYC remediation, fraud triage, these are increasingly agent-run end to end in production banks, and the ones that work share one trait: the orchestration layer was designed with the same rigor as a settlement system, because functionally, that’s what it is.
The failure mode here isn’t the agent getting something wrong. It’s a bank that never defined what “wrong” costs, or who’s accountable when it happens, discovering both in production, usually in front of a regulator or a journalist rather than in a design review where it would have been cheap to fix.
This is the layer that decides whether the first three matter at all, and it’s the one where regulated banking stops being optional homework and starts being the whole game.
The EU AI Act’s high-risk enforcement threshold took effect on August 2, 2026 — a week ago, as I’m writing this. Credit scoring, insurance underwriting, and most customer-facing decisioning systems fall under Annex III as high-risk AI, which means this isn’t only a banking story; asset managers and insurers building on the same agentic patterns inherit the same obligations. That means documented risk management, data governance, human oversight assignment, a completed Fundamental Rights Impact Assessment, and registration in the EU database — not as a policy memo, but as an operating requirement with a market surveillance authority now empowered to open proceedings against you. Layer that against DORA’s operational resilience requirements for any AI supporting a critical function, including your third-party model providers, and “governance” stops being a compliance checkbox appended after deployment. It has to be a property of the architecture from day one, the same way double-entry bookkeeping isn’t a report you generate — it’s a constraint the whole ledger is built around.
I traced financial flows across 2 million transactions a month, 492 branches, and €20 million in daily volume for a European retail bank, and brought unexplained entries down from €35 million to €7,000. Nobody on that program thought of it as “AI governance,” because there was no AI yet. But traceability, human accountability, and an audit trail that survives a regulator’s questions, that’s precisely the discipline Layer 4 requires now. The regulated-banking playbook didn’t need to be reinvented for AI. It needed to be extended to cover a new kind of decision-maker.
I want to sit on this a second longer, because most of the AI-and-banking commentary I read treats regulation as friction to route around. In a regulated industry, that’s backwards. The AI Act’s Annex III list — credit, insurance, most consequential customer decisioning, reads like a checklist of exactly the domains where the $2 trillion opportunity lives. The banks that already had governance-by-design as an architectural habit, because regulated banking has demanded it since long before anyone said “agentic,” are the ones that can move fastest into high-risk AI use cases now. Everyone else is discovering, this month, that they need a Fundamental Rights Impact Assessment for a system they already shipped.
Compliance isn’t the tax on the 10x outcome. In banking specifically, it’s a precondition for reaching the domains large enough to produce one. A bank that can’t operate in credit and underwriting because its governance layer isn’t ready is, by definition, locked out of the two domains where McKinsey’s numbers are largest.
One more data point worth holding in view, not because it changes what you should build, but because it’s the honest counterweight to a $2 trillion headline. The Bank for International Settlements flagged the AI infrastructure buildout, north of a trillion dollars in hyperscaler capex through 2026, as one of four top risks to global financial stability in its June 2026 annual report, alongside sticky inflation and strained public finances. Its models show the payoff-minus-cost curve on that capex bending downward as competitive spending accelerates.
I’m not going to tell you what that means for markets — that’s not my lane, and anyone offering you a confident answer over LinkedIn probably shouldn’t be trusted with your balance sheet. What it means for architecture is narrower and more useful: build for optionality, not for a single vendor’s roadmap. A bank whose Layer 3 orchestration is portable across models, and whose critical operations have a sovereign or local fallback for the day a cloud dependency gets expensive or simply unavailable, is hedged against a capex cycle turning. A bank that wired its core irreversibly to one provider’s stack is making a bet it never chose to make on purpose.
None of the four layers above is optional, and none of them can be bought as a single line item from a vendor, no matter what the pitch deck implies. But they can be sequenced, and sequencing is most of the job. A ninety-day version of this usually looks like: weeks one through four are spent auditing the core and finding where the real logic lives versus where it’s just a configuration screen; weeks five through eight go into naming the owners of your most contested definitions and writing the first version of the semantic contract; the back half is spent on a single, narrow, well-governed workflow, not a portfolio of pilots, with the accountability question answered in writing before a single transaction runs through it.
The order matters more than the speed. A bank that rushes Layer 3 without Layer 1 gets an agent that’s very good at describing what it can’t do. A bank that does all four, in order, gets the number McKinsey is talking about, not because the model was smarter, but because for the first time, it was finally allowed to work.
I spent 20+ years architecting the systems underneath banks like the ones in this piece — regulated data governance, multi-billion-dollar target architectures, system integration at scale — before “agentic AI” was a term anyone used. If your bank is somewhere between the pilot and the platform, that’s the conversation worth having. Follow along for more architecture and strategy on[nganiet.com].
Sources: McKinsey — Generative AI in banking · McKinsey — Agentic AI frontline teams · Gartner — Banking & Investment Services AI priorities · Gartner — CIO Agenda 2026 · IDC agentic AI ROI, via Neurons Lab · MIT pilot-to-production research, via Jinba · Fiserv agentOS, via TechTimes · JPMorgan / Klarna / HSBC / Citi / UBS / DBS / ING case data, via AI Monk · EU AI Act financial-services obligations · EU AI Act Annex III scope · DORA + AI Act compliance stack · BIS Annual Economic Report 2026 · BIS AI-boom risk coverage, via CNBC The 10x Stack: The Architecture Banks Need Before AI Can 10x Anything was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.