Legal AI Hallucination and How Citation Verification Stops It

Legal AI hallucination is not a mysterious defect. It is the predictable output of a system trained to produce text that looks right, applied to a task where only text that is right is acceptable. Citation verification is the control that closes the gap, and it works only if you understand precisely what it checks, what it cannot check, and where the residual risk lands. For Indian practitioners, that risk lands on the advocate whose name appears on the pleading.

What a hallucinated citation actually is

A hallucinated citation is a reference that is well-formed and false. It comes in degrees, and the mild forms are more dangerous than the obvious ones because they survive a skim by an experienced reader.

• A case that does not exist at all: invented party names, a plausible court, a plausible year and a plausible reporter reference.
• A real case with a wrong identifier: correct parties but wrong year, wrong volume or page, or attributed to the wrong court or bench.
• A real case cited for a proposition it does not contain, often adjacent to something it does say.
• A real proposition attributed to the wrong part of the judgment: a dissent, a concurring opinion, a submission the court was recording rather than accepting, or obiter presented as the holding.
• A real case, correctly quoted, that has since been overruled or rendered obsolete because the provision it construed has been amended.

Only the first is a fabricated case. The remaining four are citation errors. Professionally, they carry the same consequence, because a court does not distinguish between an authority you invented and an authority you misrepresented.

Why the format is always perfect

A language model generates the next token conditioned on the preceding text. Legal citations are highly regular strings in any large training corpus. Party names follow recognisable patterns. Court names and reporter abbreviations form a closed vocabulary. Years, volumes and page numbers are numerals in predictable positions. The result is that a citation is unusually easy for a model to imitate and unusually hard for it to get truthfully right.

When a model is asked to support a proposition and has no retrieved source in front of it, it does the only thing it can do: it completes the pattern. It produces a string with the statistical shape of a citation. The shape is learned; the referent is not. In a bare generation there is no lookup step, so there is no moment at which the system could discover that the case does not exist.

The corollary is the part practitioners most need to internalise. The confidence of the output tells you nothing about its truth. Fluency is a property of the model, not evidence about the world. A fabricated citation and a correct one are produced by identical machinery and read with identical authority. There is no tone of voice to listen for.

The three checks that matter

Check one: existence

Does the cited authority exist, and is this the correct identifier for it? The citation must resolve to a retrievable document, from the court named, of the date claimed. Neutral citations and stable case identifiers are more reliable anchors than reporter references, because reporter numbering varies between publishers and can be transposed. The rule is binary: if a citation cannot be resolved to a document, it fails. No benefit of the doubt, and no assuming a typo in the volume number.

Check two: accuracy of the cited passage

Does the document actually say what is claimed? This is the check most easily skipped, because passing the existence check feels like validation. It is not. Verification here means locating the specific paragraph relied on and comparing it against the proposition asserted. The recurring failure modes are worth knowing by name.

• The quoted text does not appear in the judgment in that form, or has been smoothed into cleaner language than the court used.
• The passage comes from a dissenting or concurring opinion rather than the majority.
• The sentence is the court recording counsel's submission or the appellant's contention, not its own conclusion.
• The proposition is drawn from an editorial headnote rather than the judgment itself. Headnotes are publisher summaries and are not the court's words.
• The passage is real and from the majority, but a qualification in the following sentence changes its effect.
• The proposition is accurate but drawn from reasoning expressly confined to materially different facts, so it is not authority for the use being made of it.

Check three: currency

Is the authority still good law? A verified quotation from an overruled judgment is a verified quotation from an overruled judgment. Currency means two separate enquiries. First, the treatment history: has the proposition been overruled by a larger bench, doubted, referred, or narrowed by repeated distinguishing? Second, the statutory foundation: does the provision the court construed still read as it did, or has it been amended, repealed and re-enacted, or displaced by a validating amendment? The second enquiry is easy to overlook, and not every citation tool covers it. Establish what any given tool actually checks rather than assuming that statutory currency is included.

Existence, accuracy, currency. A citation that passes only the first two is a very well-documented mistake.

Why retrieval reduces the problem but does not remove it

Retrieval-augmented generation grounds the model's output in documents fetched from a real corpus. This removes a conspicuous and severe failure mode, the wholly invented case, and it is a genuine advance. It does not remove the rest, and treating retrieval as a solved problem is how teams end up with false assurance.

• Retrieval can succeed while summarisation fails. The model may generalise a fact-bound holding, drop a qualifying clause, or merge propositions from two documents into one sentence that neither supports.
• Retrieval can return the right case for the wrong reason. A document surfaced because it shares vocabulary with the query is not necessarily authority for the query's proposition.
• Chunking can sever context. A paragraph retrieved without the sentence that qualifies it reads differently from the same paragraph in the judgment.
• Coverage gaps are silent. If the corpus does not include the relevant tribunal, bench or period, the system answers from what it has rather than reporting what it lacks.
• Indexes go stale. A corpus current to last quarter cannot know about last month's larger-bench decision or last week's amendment notification.

Grounding and verification are therefore different controls doing different jobs. Grounding shapes what the model sees before it writes. Verification tests what the model produced after it writes. A system that performs the first and describes it as the second is mislabelled, and the mislabelling is what causes reviewers to relax.

Where the risk lands in Indian practice

The consequence of a bad citation does not attach to the software vendor. It attaches to the professional who filed the document. Under the Advocates Act 1961, professional misconduct is dealt with under Section 35 by the State Bar Council, and standards of professional conduct and etiquette are prescribed by the Bar Council of India in rules framed under that Act. An advocate's duty of candour to the court is not diluted by the tool used to prepare the draft. Pleadings verified under Order VI Rule 15 of the Code of Civil Procedure 1908 are verified by a person. Depending on the facts, a false or misleading statement to a court can attract costs, adverse orders, disciplinary proceedings before the Bar Council and, where the ingredients are made out, action under the Contempt of Courts Act 1971.

Internationally, there have been reported instances in which lawyers filed submissions containing citations fabricated by generative AI tools, and courts responded with adverse orders, costs consequences and referrals to disciplinary authorities. The procedural specifics differ between systems; the pattern does not. In each instance the tool produced the error and the professional carried it. That allocation of responsibility is the correct one, and it is also the reason verification cannot be treated as optional.

What a verification-first system looks like

If verification is to happen reliably, the workflow has to make it cheap. Six design properties do most of that work. These are criteria worth testing in any tool, whoever built it.

• Every proposition offered as law carries a resolvable link to a source document. No source, no proposition.
• Quoted text is matched against the source text rather than paraphrased into it, so a mismatch is visible immediately.
• The pinpoint is captured, not just the case: paragraph-level reference, so the reviewer lands on the passage rather than the first page.
• Treatment status is surfaced alongside the citation, including subsequent history, pending larger-bench references and whether the underlying provision has been amended.
• The deciding forum is surfaced, so the reviewer can see at a glance whether the authority binds or merely persuades the court they are in.
• A finding of nothing relevant is a permitted and clearly presented answer, rather than a gap the system fills with something plausible.

LexVio's Indian court research covers the Supreme Court, High Courts, NCLT, ITAT, CCI and CESTAT, with citation graph visualisation and filtering by jurisdiction, so the citing relationships and the deciding forum are visible next to the authority. Vio is the single-document copilot, scoped to the document you have open. Customer data is not used to train models, and encryption is applied at rest using AES-256 and in transit using TLS 1.3. Those are retrieval and navigation features, not a verification verdict. LexVio does not classify subsequent treatment and does not tell you whether an authority is still good law: reading the citing decisions and deciding how each one treats the proposition remains work for the professional.

The human step that cannot be automated away

Verification checks correspondence between a claim and a document. That is a mechanical relationship and it can largely be automated. What cannot be automated is the question that actually decides matters: whether this authority, before this forum, on these facts, against this opponent, helps this client. That is an assessment of weight, strategy and risk appetite, and it is precisely the part of the work that carries professional accountability.

The division of labour is clean once stated. Software can compress the mechanical half: assembling candidate authorities, gathering the subsequent citing history, matching quotations against source text, and surfacing amendments to the underlying provision. The lawyer decides which authority to lead with, which to distinguish pre-emptively, what must be disclosed, and what goes on the record. Speed on the mechanical half is worth having for exactly one reason, which is that it leaves more time for the half that cannot be delegated.

A short operating rule for teams adopting these tools: treat every AI-surfaced citation as an unverified lead until it resolves to a document, the pinpoint matches the proposition, and the subsequent history has been read. A well-built system can shorten the first two steps and show you where the third begins. None of them will sign the pleading.