Someone asks how much they can afford to overpay on their mortgage. You look at their income and outgoings, work out what’s genuinely spare, and give them a figure along with how you got there. The arithmetic holds up. A decent spreadsheet would have produced the same number.
According to the people who built the system that gave that answer, it was still wrong.
The example comes from Where the Model Stops, a white paper from MoneyMeans, the company behind an AI financial guide called Aida. During user testing earlier this year someone put exactly that question to Aida, and it answered it well. What it never mentioned was their pension. In the paper’s words, “Mortgage overpayment and pension contribution compete for the same money over roughly the same horizon. For a lot of people in that position, the pension is the better home for it.” Writing separately in TechRound, the firm’s CTO Tom Richards says the detail that decided it was an employer contribution that wasn’t in the data Aida could see.
Richards, who wrote the paper, put it this way in FT Adviser’s report: “A language model can produce a fluent, persuasive answer to the wrong-shaped question. The language model is the mouth. Controlled financial-planning logic must be the brain.”
If you advise on mortgages, you’ll know the wrong-shaped question well. Clients bring them to us all the time. “How much can I borrow?” is rarely the real question, and neither is “two-year fix or five?” Behind each one there’s usually something else. A job that might change, a relationship that might not last, a parent who may need help, a pension nobody has looked at since the last employer. The paper describes what an adviser does with that kind of question better than I could: “They don’t answer. They widen the question first.”
The obvious response is to wait for the next model. Richards says they tried that. In internal testing, more capable models still gave polished answers about overpaying the mortgage without first looking at the customer’s wider circumstances. The paper says the gap between answering well and advising well “is not a gap in model quality, and it does not close as models improve.”
I think he’s right, and the reason is fairly simple. A general-purpose assistant is built to answer the question you ask, and on its own terms that’s good behaviour. Nothing in the client’s question suggested the pension mattered. Nothing in their data did either. The link between overpaying a mortgage and paying into a pension is a fact about how money works, and it would have been just as true for anyone else asking. So somebody has to build it into the system before the model starts talking.
Aida’s answer is what it calls decision frameworks. For each decision a customer might face, there’s a record of what that decision touches and the facts it can’t responsibly be made without. The interesting part is what happened when they first fed those frameworks to the model as instructions. It didn’t reliably work, because a customer asking a narrow question is more persuasive to the model than a background note telling it to think wider. The paper sums it up in two sentences I’d happily see in every firm’s AI policy: “Instructions raise the probability of the right behaviour. They do not establish it.”
So the checks now sit outside the model, in code. If the flow needs to know whether there’s an employer match going unclaimed, it doesn’t move on until it has found out. Figures come from a calculation service that logs the inputs and assumptions it used. A rules engine decides which action comes first. The model does the talking.
You could read all that and decide the model is just a friendly voice in front of a calculator. Aida’s own numbers push back on that. Several of its journeys started life as simple flows: a set of questions, then a calculator, then an answer. The numbers were right. Yet in one of those calculator flows only about a quarter of the people who started it reached the end. In its conversational flows, more than three quarters did.
The paper’s explanation is simple. “People trust the adviser, not the advice.”
That cuts both ways. For anyone building technology, it says the conversation isn’t decoration around the calculation. For brokers who worry about being replaced, it suggests the part clients value most could be the part that feels least like work, like explaining why you need to know about their pension when they came in to talk about their mortgage.
Aida currently gives guidance rather than regulated advice, and MoneyMeans operates as an appointed representative of Smith and Wardle Financial Planning. Every response is checked by what the firm calls a boundary judge, which classifies it against the line between guidance and advice. Compliant answers go through, minor failures are rewritten and major ones are replaced. The firm says the judge is around 99% accurate against a human-labelled set of more than a thousand scenarios, and that when it does get it wrong it errs on the strict side. To its credit, the paper also names the limit. The test set was written by the people who built the system, so it can only contain the failures someone has already thought of.
Near the end, the paper lists what would need to be true before Aida moved into regulated advice. Knowing a person’s objectives and financial situation. Their attitude to risk and, separately, their capacity for loss. Whether anything about their circumstances makes them vulnerable. Being able to show afterwards that the recommendation was suitable on the evidence held at the time. The authors point out that this is “almost word for word, a description of the fact find a good adviser has always done before opening their mouth.”
That lands at an interesting moment. FT Adviser reported this week that FCA chief executive Nikhil Rathi told a Treasury select committee hearing that “at some point we may need a perimeter shift” on AI and advice, calling it “a societal choice”. If that line does move, firms that can show precisely where their own boundary sits, and what changed when they moved it, will be far better placed than those whose boundary lives somewhere inside a prompt. Aida’s phrase for it is a good one: “Crossing into regulated advice will be a decision, not a drift.”
A fair word of caution. This is a white paper from a company with a product to launch, the figures come from its own testing, and it admits that its competitive edge is “a claim we are making rather than one we have proved”. None of that makes the central argument wrong. Just read it knowing who wrote it.
The detail I keep coming back to is a small one. A technology firm had to write code to stop its system answering a mortgage question until it had asked about the pension. Advisers are meant to do that by instinct, and most good ones do. But on a Friday afternoon, with a client who knows exactly what they want and a case that needs to be on the lender’s system before five, how often does the question get answered exactly as it was asked?
The paper closes by saying the model “is no longer the thing that decides what counts as an answer.” In your firm, who does?
Sources:
FT Adviser, “‘Generic’ AI will not bridge advice gap”, Tom Dunstan, October 2026
Aida by MoneyMeans, “Where the Model Stops” white paper, August 2026
TechRound, “The Most Important AI Design Decision Is Where The Model Stops”, Tom Richards, 12 October 2026
FT Adviser, “FCA: we may need perimeter shift when it comes to AI and advice”, Tom Dunstan, October 2026