The Legal AI Industry Is Solving the Wrong Problem

A lawyer asks an AI assistant to find authority for a proposition. It returns a case name, a citation, a court, a year, and a confident one-line summary of the holding. Every part of it looks right. None of it exists.

This is not a rare failure mode. It is the defining failure mode of legal AI, and it has already put real names in front of real judges: practitioners sanctioned for filing submissions built on citations their AI tool invented. The profession's instinct, understandably, has been to treat this as a quality problem with the technology. The model isn't good enough yet, so wait for the next one, or train a specialist "legal" model that knows the law properly.

That instinct is wrong, and it's expensive. The hallucination problem in legal AI is not a model problem. It is a data problem. And almost the entire industry is spending its money on the wrong layer.

The two bets everyone is making

There are two dominant strategies for "solving" hallucination, and both are bets on the model.

The first is wait for a better one. The reasoning goes: the frontier models keep improving, hallucination rates keep falling, so give it another release or two and the problem takes care of itself.

The second is fine-tune your own. Take a strong base model, train it further on a large body of case law and legal writing, and you get a model that has "read the law" and therefore won't make things up.

Both sound reasonable. Both are the wrong place to invest, and the reasons are worth spelling out, because they're the reasons a serious buyer should push back the next time a vendor's pitch leans on either one.

The model is no longer the moat

Start with the "wait for a better model" bet. The uncomfortable truth for anyone selling a model-centric product is that models are commoditising fast.

As of mid-2026, the gap between the best closed, API-only models and the best free, open-weight models has collapsed to a matter of months and a handful of benchmark points. Open-weight models from Chinese labs, such as Kimi, GLM and DeepSeek, now sit level with the flagship offerings from the largest Western labs on real-world tasks, and in several published coding and tool-use benchmarks they lead. The lag between the frontier and the free tier has narrowed from six-to-ten months through 2025 to roughly four-to-seven months now.

Two caveats keep this honest. Many of these headline numbers are vendor-reported rather than independently replicated, so no single score should be load-bearing. And "open-weight caught up" is not the same as "open-weight won", because the newest closed releases still hold a real edge on the hardest reasoning and agentic-reliability work. But the direction is unambiguous, and one number tells the whole story: the open models cost roughly an eighth of the flagship price per token, and you can host them yourself. When capability converges and price collapses by 8x, the model stops being a durable advantage. It becomes a utility, like bandwidth, or compute. Nobody builds a defensible business on having slightly better bandwidth than the firm down the road.

If your legal AI's value proposition is "we use the best model", you have tied your product to a depreciating asset on a moving walkway. Whatever edge that model gives you today, a cheaper one erases in a few months.

And the trajectory points one way: local. As open weights close the gap and the hardware to run them gets cheaper, the natural home for a lot of this work becomes a model you run yourself, on your own infrastructure, under your own control. In law that is not a nice-to-have. Client confidentiality, privilege and data residency all pull in the same direction, away from piping privileged material through a third party's API and towards a model that never leaves your control. It also reorders who holds the value. The frontier labs are spending extraordinary sums to stay a few months ahead of models anyone can download for nothing, and it is not obvious that lead is defensible, or that the economics of it survive contact with a commoditised, self-hostable alternative. My own view is that the current giants are far more fragile than their valuations suggest. You should not build a legal product on the assumption that any one of today's big AI companies will still be the obvious choice, or even still be standing, in five years. Build on the opposite assumption: that the model underneath you will keep changing, keep getting cheaper, and increasingly run on your own machines.

Fine-tuning is the same bet, made worse

The "train our own legal model" bet is the first bet with extra steps, and a trap of its own.

A fine-tuned model is frozen at the moment you tuned it. The frontier keeps moving; your custom model does not. Within a release cycle or two, a general model that costs you nothing to adopt is out-reasoning the specialist model you spent months and a substantial budget building. You now carry all the cost of a bespoke model and none of the durability, and every time the frontier jumps you face the same expensive choice again: re-tune, or fall behind.

But there's a deeper problem, and it's the one that matters for hallucination specifically. Fine-tuning teaches a model to sound like a lawyer. It does not give it a reliable, checkable record of what the law actually says. Training bakes patterns into weights, not facts into a lookup you can verify. A model fine-tuned on a million judgments learns the cadence, the structure, the phrasing of legal reasoning exquisitely well, which is precisely what makes its fabrications more convincing, not less. It has learned the shape of a citation without a source of truth for whether any given citation is real.

You do not fix a fabrication problem by making the fabrications more fluent. Facts do not belong in the weights. They belong in a database.

The real lever: context, not the model

If the model isn't the moat, what is?

The answer is unglamorous, and it's the reason this argument matters. The durable advantage in applied AI is context engineering: controlling exactly what information the model is given at the moment you ask it to reason. And context engineering, stripped of the hype, is mostly just good software. Retrieval, structure, data modelling, indexing. The fundamentals. Databases.

A model that is handed the right, real, specific source material and asked to reason over that is a fundamentally different tool from a model asked to answer from memory. The first is grounded. The second is guessing fluently. The entire difference between a legal AI you can trust in front of a court and one that gets you sanctioned lives in that distinction, and it is a data-architecture decision, not a model decision.

This is why the smart bet is to stay deliberately loosely coupled to any one model. Treat the model as a swappable component. When a cheaper or better one appears, and it will, every few months, you switch it out in an afternoon. What you invest in, what compounds, what a competitor cannot copy by signing a different API contract, is the data layer underneath.

Why "give the AI the full case" isn't enough

Here is where legal AI vendors who have understood the data point still stop short.

The common grounded approach is retrieval: instead of letting the model answer from memory, give it the actual case text and have it reason over that. Better. But the usual implementation, an API that fetches a whole judgment and stuffs it into the model's context, has two problems.

The first is cost. Full judgments are long. Feeding an entire case, or several, into the model on every query burns an enormous number of tokens, and you pay per token, every time. It does not scale to the way lawyers actually work, which is across many authorities at once.

The second problem is the one that actually causes bad answers, and it is conceptual, not economic. A case read in isolation cannot tell you whether it is still good law.

A judgment does not carry, within its own four corners, the information that it was distinguished five years later, or narrowed on appeal, or quietly overruled by a higher court. That information does not live in the case. It lives in the network of other cases that cite it. An AI that reads a single judgment perfectly and reports its holding with total accuracy can still be catastrophically wrong, because the holding it just summarised so faithfully was overturned, and nothing in the document it read said so.

This is the trap. Give the model one case and it will tell you what that case says. It will not tell you whether anyone should still rely on it. For a lawyer, that second question is the entire job.

The value is in the edges

Think about how a good lawyer actually establishes that a proposition is safe to rely on. They don't read one case. They read the case, then they read the cases that cite it, to see how it's been treated. Followed? Distinguished? Overruled? Confined to its facts? The authority of any single judgment is a property of the network around it, not of the document itself.

It is, to put it bluntly, like checking your answer against everyone else's homework. One student's answer might be wrong. But when you can see who copied from whom, who corrected whom, and which answer the whole class converged on, you can tell, with confidence a single sheet of paper can never give you, which answer to trust.

This is the layer that almost nobody is building, because it isn't a model problem and it doesn't demo well. It's slow, expensive, unglamorous data engineering. And it's the whole game.

It's what we've built. Our legal graph holds 393,000 UK authorities across 39 courts, connected by over two million citation edges: a machine-readable map of which case relies on which, extracted paragraph by paragraph. Because plain-text citations miss the countless times a judgment refers to a case by name alone, we separately capture those too, nearly 293,000 name-only references that a naive citation scanner would drop on the floor.

The point of the graph is not that it's big. The point is what the edges let you do. When the model answers a question, it isn't handed one case in a vacuum. It's handed the case and its neighbourhood: the judgments that cite it, the treatment they gave it, the position it occupies in the network. Every claim it makes is anchored to a real node with a real, checkable provenance, and its standing as good law is a property you can read off the graph rather than a fact the model has to remember and might invent.

That is also what makes it cheap. You don't feed the model whole cases hoping the answer is buried in there. You feed it the structurally relevant slice, this holding, its treatment, the specific paragraphs that matter, which is a fraction of the tokens and a far better answer.

What this means if you're buying legal AI

Strip away the demos and the model names, and the questions that separate a trustworthy legal AI from a liability are data questions.

• Where does every answer come from? If the tool can't show you the real source for each claim, a specific case, a specific paragraph, it is reasoning from memory, and memory fabricates.
• How do you know it's still good law? If the tool reads cases in isolation, it structurally cannot know. Ask what it does with the citation network. If the answer is "nothing", the confident summary you get back is only ever a summary of a document, not a statement about the law.
• What happens when the model changes? A vendor whose advantage is their data layer will shrug and swap the model and move on. A vendor whose advantage is the model has just told you where their moat is, and it's the one thing that will be commoditised out from under them.

The hallucination problem was never going to be solved by a bigger model or a cleverer fine-tune. A fabricated citation is not a sign that the model didn't try hard enough. It's a sign that we asked it to remember something it should have been able to look up, and gave it one page when the answer was in the network.

Fix the data, and the model has nothing left to make up.