Intrinsic and extrinsic hallucination

The material a model can see when it generates is its context — the run of text placed in the prompt before it produces anything, which in a support system typically means a system prompt, some retrieved documents and the customer's message. Everything a model asserts is either consistent with that text, inconsistent with it, or unrelated to it, and those three cases have sharply different causes. A claim that conflicts with a sentence sitting in the prompt is a reading failure. A claim about a subject the prompt never raises is a generation failure, produced from parameters rather than from evidence.

Both of those descriptions rest on one fact about how supplied text acts on a model, and being exact about it prevents a great deal of confused reasoning later. Text placed in the prompt is not loaded into a separate store the model consults. It becomes more tokens in the same sequence, and its only effect is on the distribution over the next token — a retrieved policy makes 14 a likelier continuation after "the refund window is" than it would otherwise have been. That effect is real and frequently decisive. It is also a competing influence rather than an overriding one, sitting alongside whatever the weights already favour at that position.

The two failure types fall straight out of that. An extrinsic hallucination happens where the context contributes nothing at a position, because nothing in it bears on the subject, so the weights decide alone and produce whatever the pattern suggests. An intrinsic hallucination happens where the context does bear on the position and loses anyway — to a strong prior, to an awkwardly phrased passage, to a relevant sentence sitting a long way from where it is needed. The first is the model working without evidence. The second is the model working with evidence that failed to win. Naming which of the two is in front of you is naming whether the fix is to supply something or to enforce something.

That split is the taxonomy this topic runs on, and the reason to establish it before measuring anything is that a system counting both as "one hallucinated response" has thrown away the information needed to fix either. The terminology comes from summarisation research, where the same distinction was needed for the same reason: a summary can contradict its source document or it can add material the source never mentioned. A Survey on Hallucination in Large Language Models is the reference statement of it. This chapter defines both terms precisely, crosses them with a second axis — truth in the world — that settles most labelling arguments before they start, works through a single response containing both, and then covers the two edges where the definitions decide a surprising number of labels.

The two definitions the rest of this topic runs on#

An intrinsic hallucination is a claim in the output that contradicts content present in the supplied context. If a pasted policy says the refund window is 14 days and the answer says 30, that is intrinsic: the correct value was in the prompt, the model had it, and it produced something inconsistent with it. An extrinsic hallucination is a claim that has no counterpart in the supplied context at all — neither supported nor contradicted, simply absent. If the same policy is silent about how long processing takes and the answer states a 3-to-5-business-day turnaround, that is extrinsic: there is nothing in the context for the claim to agree or disagree with.

The dividing line is the supplied material, and only the supplied material. That makes the labels relative to what you put in the prompt rather than absolute properties of a sentence. The same output sentence — "refunds are processed in 3 to 5 business days" — is extrinsic against a policy with no processing-time section and supported against one that has it. Nothing about the sentence changed; the evidence available to it did. This relativity is not a weakness of the taxonomy but the point of it, because it makes the label a statement about your system's inputs, and a statement about your system's inputs is something you can act on.

Being unsupported and being false are different questions#

The second axis is truth in the world, which is independent of support in the context. Crossing the two gives four cases, and keeping them apart is what stops labelling from drifting into an argument about who is right rather than what the evidence says.

The bottom-left cell is where most disagreements about labelling start, and it is worth settling now. When the model says 30 days and 30 days is genuinely the current policy, people want to call the output correct and move on. Resist that, because the system had no way to distinguish this response from the one directly above it, and neither did the reviewer. Both were produced by the same mechanism, a pattern-fit that ignored the supplied text; one of them landed on the truth by coincidence. Grading on the coincidence rewards a process that will be wrong the next time the coincidence does not hold.

The sharper version of that argument is about what a label is for. A label earns its place only if it predicts something, and what these labels predict is how the system will behave on the next case of the same shape. Two responses produced by identical mechanics — a pattern-fit that ignored the supplied text — have identical prospects on the next question, whatever happened to be true this time. Marking one of them correct records the outcome of a coin toss as though it were a property of the system, and a rate assembled from such marks will drift with the world rather than with anything you change.

What you are measuring when you label against the context is faithfulness — does the output follow from the evidence supplied — and faithfulness is the property a grounded system can actually be built to have. Factuality, correspondence with the world, is a different and much harder target, and it is not what a per-claim check against a passage measures. The distinction has a practical edge in the top-right cell too: a model faithfully reporting a stale document is behaving exactly as designed, and no amount of hallucination work will fix it. That is a content pipeline problem, and mislabelling it as a hallucination sends a team off to tune prompts when they needed to fix an ingestion job.

The difference between the two is practical rather than metaphysical, and the practical part is what makes it worth keeping. Faithfulness is decidable by somebody with the prompt in front of them, in seconds, under a rule two annotators can be brought to agree on. Factuality requires knowing what is true, which for most claims in a support corpus means asking whoever owns the process and for some means that nobody currently knows. A bar a system can be held to has to be built out of the decidable one, and the honest way to say so is to state which of the two you measured rather than letting a reader assume you measured the other.

One response, both kinds, tangled together#

A single response carrying both failure types at once, interleaved with claims that are perfectly fine, turns out to be the normal case rather than an edge case. Written out with the source beside it, the structure is visible.

{
  "ticket_id": 48213,
  "context": [
    "REFUND POLICY v4.2 — Section 2. Customers may request a refund within",
    "14 days of the original purchase date. Section 3. Requests outside this",
    "window may be escalated to a manager at the manager's discretion.",
    "Section 4. Refunds are issued to the original payment method."
  ],
  "question": "I bought this 19 days ago and want a refund. Am I eligible?",
  "answer": [
    "Our refund window is 30 days from purchase,",
    "so at 19 days you are still eligible.",
    "Refunds go back to your original payment method",
    "and finance processes them in 3 to 5 business days."
  ]
}

Four assertions come out of that answer. The window figure contradicts Section 2. The eligibility conclusion follows validly from the wrong figure, which makes it a downstream consequence rather than an independent invention — it inherits its wrongness. The payment-method claim is directly supported by Section 4 and is simply correct. The processing time is absent from all four sections. So the response is one part intrinsic, one part inherited, one part supported, one part extrinsic, and the single label "hallucinated" flattens all of it.

The second of those four rewards dwelling on, because it is where a counting scheme usually goes wrong. "At 19 days you are still eligible" is a correct deduction: given a 30-day window and a purchase 19 days old, eligibility follows, and the reasoning at that step is sound. What is defective is the premise it reasoned from, and that premise has already been counted one line above. Recording the conclusion as a second contradiction charges twice for one error and inflates the density figure, so that a four-claim answer containing a single wrong number would score as half defective rather than a quarter. Marking it inherited keeps the error where it happened and preserves the property that makes the count useful, which is that fresh defects are a count of independent things having gone wrong.

The mixture is not an accident of this example. Retrieval-augmented systems — the ones that search a document store for passages matching the question and paste the best of them into the prompt, which chapter 4 sets out mechanically — produce it structurally, because the model is doing two things in the same breath: reading supplied text, where it can misread, and drawing on parameters, where it can invent. A response that summarises a passage and then adds one helpful-sounding operational detail has done both. That is why the labelling unit has to be smaller than the response, and it is the reason chapter 3 exists.

There is a sharper way to say why that mixture is structural rather than incidental. Position by position through a response, the balance between what the supplied text favours and what the weights favour shifts, and it shifts because different sentences are about different things while the retrieved passages cover some of those things and not others. A response is therefore not produced under one regime but under a regime that changes from clause to clause, which is precisely what a single label per response cannot represent. The unit of measurement has to be the unit over which that regime is roughly constant, and the closest workable approximation to it is a claim.

Two edges of the definitions that decide a lot of labels#

Silence is not denial, and treating it as one is the most common way the labels get misapplied. A refund policy that does not mention processing times does not thereby contradict a claim about them. The processing-time sentence is unsupported, so extrinsic; calling it a contradiction would be asserting that the policy's silence means "there is no processing time", which it does not. The practical test is whether you can point at a span that says something incompatible. If you cannot point, the label is unsupported, however obviously wrong the claim feels.

The second edge is what counts as supplied context, and the answer is wider than the retrieved documents. Everything in the prompt is supplied: the system prompt, the retrieved passages, the results of any tool calls, the earlier turns of the conversation, and the customer's own message. A reply telling a customer who wrote "I bought this 19 days ago" that they purchased nine days ago contradicts supplied context and is intrinsic, even though no policy document is involved. This matters more than it sounds, because in a support setting a large fraction of the facts in play come from the user rather than from any document store, and a system that only checks against retrieved documents will label those errors as unsupported or miss them entirely.

There is a corollary for systems that supply nothing. A closed-book assistant answering from parameters alone has no context for a claim to contradict, so every defective claim it produces is extrinsic by construction and the intrinsic count is structurally zero. That is not a sign of a well-behaved system; it is a sign that there is no evidence to be faithful to. When you compare such a system to a grounded one, the two intrinsic columns are not measuring the same thing unless you fix the evidence you judge against, which is a design decision chapter 5 has to make explicitly before its experiment can be read.

That corollary follows from the definition rather than from anything observed about such systems, and keeping the two apart matters. The definition draws its line at the edge of the prompt. Move that edge inward until nothing sits inside it, and one of the two categories has no members left. The system did not start behaving better; the category was defined by reference to something that is now empty. Any comparison across systems supplying different amounts of evidence therefore has to fix the yardstick separately from the input, which is the first thing chapter 5's experiment does.

What the taxonomy does not tell you#

It is not a severity scale. An extrinsic claim is not automatically less serious than an intrinsic one; in ticket 48213 the invented processing time was the sentence that generated a second angry ticket, while the wrong window figure was caught by someone who knew the policy. The two labels tell you where the failure happened — misreading supplied evidence versus generating past it — not how much it costs you. Severity is a separate dimension you assign per claim from the business context, and conflating it with the type is a good way to end up optimising the wrong number. On that product the ordering ran roughly: an invented entitlement or price is a refund the company has to honour, an invented timeline is a follow-up ticket, and a wrong clause reference is an embarrassment the agent usually catches. None of that ordering correlates with which of the two labels applies.

It also does not cover every way a response can be bad. An answer entirely supported by the context and completely unhelpful — restating the question, refusing, answering a different question, or burying the eligibility ruling under four paragraphs of hedging — has zero hallucinations of either kind and is still a failure. Rudeness, leaked personal data, an unsafe recommendation and a broken JSON payload are all out of scope here. Hallucination measurement is one axis of response quality and needs to sit alongside the others rather than standing in for them, and a team that drives its unsupported-claim rate to 1% while its resolution rate collapses has optimised itself into a worse product.

Finally, the labels say nothing about why the model produced what it produced. Intrinsic errors have several distinct mechanisms behind them — a long context where the relevant sentence sits in the middle and gets underweighted, a prompt in which pretraining priors are stronger than the supplied text, an ambiguous passage resolved the wrong way, a plain reading error on a number. The taxonomy is a classification of the output against its evidence, observable from the outside, and that is exactly what makes it usable as a measurement. Diagnosis of cause comes after counting, not before.

Two words, one dividing line, and the line is drawn at the edge of the prompt. Intrinsic means the evidence was there and the answer went against it; extrinsic means there was no evidence and the answer went ahead anyway. Everything later in this topic — what retrieval fixes, what an instruction fixes, what a verifier can check, what you can put in a written commitment — depends on which of the two a given failure is, because interventions that move one of them reliably leave the other where it was.

Counting is where this goes next. A reply like ticket 48213's needs to become several numbers rather than one verdict, and that means a procedure for splitting an answer into claims, aligning each one to the source, and recording the span that settles it.