When the answer must be the source clause, a paraphrase is a liability

It is 02:40 on a medical ward. A nurse has two prepared syringes and one running line, the patient is due both drugs within the hour, and the question is whether they can go through the same line or whether the line has to be flushed and the second dose delayed. The medication policy answers this. It is 340 documents and about 2,100 pages, it lives on the intranet behind a search box that returns PDFs, and at 02:40 with an infusion pump alarming nobody is going to find the right page. So the nurse asks the colleague at the next bed, who is fairly sure, and the dose goes through.

That is the problem a hospital administrator is describing when they ask for a chatbot so nurses can ask questions about the medication policy. The request names a solution, and the underlying observation is correct: the policy is unreachable at the moment it is needed. What follows is what to build instead, and why the obvious answer is the dangerous one.

The standard answer fails in a way the demo will not show you#

The reflex build is retrieval over the policy corpus — searching the 2,100 pages for the passages that look relevant to the question and pasting those passages into the model's prompt, so the answer is written from the policy rather than from whatever the model absorbed in training — wrapped in a chat interface, with citations under each answer and a disclaimer reminding the user to verify against the source. It demos extremely well. Ask it the compatibility question and it produces three clear sentences, a document link, and a line of small grey text about clinical judgement. Everyone in the room can see that it would have helped at 02:40.

Three things are wrong with it, and none of them show up in a demo.

The paraphrase is what is on screen, so the paraphrase is what gets acted on. When the model writes "these two agents may be co-administered through a single peripheral line", that sentence is now the operative instruction, and it was produced by a system that is right most of the time rather than by the document that governs. The failure mode is not gibberish; it is a sentence that is nearly right — dropping a rate qualifier, generalising a statement that applied to central lines only, merging two adjacent policy provisions that happen to sit in the same section. Those are the errors a fluent summariser makes, and they are indistinguishable from correct answers at the point of use.

The citation is decoration under time pressure. A link below the answer is a mechanism for verification that assumes the reader will stop, open a 60-page PDF, find the section, and read it. Nobody does that at 02:40, and the ones who would do it are the ones who did not need the assistant. A citation that is never followed provides the appearance of traceability without the traceability.

The disclaimer moves liability rather than the decision. "Verify against the source" tells the nurse to do the thing they came here because they could not do. It changes who is at fault when the answer is wrong; it does not change the probability that the wrong answer is acted on, and building a safety story on a sentence in grey text is building it on the one part of the interface designed to be ignored.

The answer is the passage, and the refusal is a feature#

The surface is a policy lookup that lives inside the administration workflow, not a separate place the nurse has to think to visit. The electronic medication administration record — the eMAR, the screen a nurse signs each dose off on — already knows the patient, the two drug orders, the route and the time. Those are the inputs; the nurse should not be typing them into a text box, because they are already on screen and re-entering known facts is exactly the interaction cost this whole topic is about. A single control next to the two due doses — compatibility — is the entire input surface.

Embedding it there rather than beside it is not a convenience. A separate assistant, however good, has to be remembered, opened, and told which patient and which drugs — three steps performed with one hand while an alarm sounds, and three opportunities to decide it is faster to ask the colleague at the next bed. The measured effect at the pilot site was blunt: an intranet search box available to every nurse for four years had been used 62 times that year across the whole hospital, while the in-workflow control was used 1,900 times in its first eight weeks on two wards. Availability at the moment of the decision is most of what determines whether a policy is consulted at all.

What comes back is the governing text, verbatim, with everything needed to know which text it is.

{
  "query": {
    "drug_a": {"order_id": "ORD-88431", "rxcui": "1116635", "route": "IV peripheral"},
    "drug_b": {"order_id": "ORD-88437", "rxcui": "308047",  "route": "IV peripheral"},
    "asked_at": "2026-03-14T02:40:11Z",
    "asked_by": "staff:41203"
  },
  "coverage": "governed",
  "passage": {
    "verbatim": "Vancomycin and piperacillin-tazobactam must not be co-administered through a shared peripheral line. Flush with 20 mL 0.9% sodium chloride between agents, or use a separate access site.",
    "document_id": "MED-POL-014",
    "document_title": "Intravenous Therapy: Compatibility and Line Management",
    "version": "7.2",
    "section": "4.3.1",
    "effective_from": "2025-11-01",
    "supersedes": "7.1",
    "page": 22
  },
  "retrieval": {"score": 0.83, "threshold": 0.62, "candidates_considered": 6}
}

The retrieval block at the bottom is the search step reporting on itself: the index scores each candidate passage for how well it matches the query, on its own scale from 0 to 1, and threshold is the score a passage has to beat before the system is willing to say that anything governs the question. The rendered screen shows the paragraph in the policy's own words, the document title and version, the effective date, and a link to the full section for anyone who wants it. There is no generated prose above it summarising what it says, because the paragraph is four lines long and the summary would be the liability.

The other outcome matters as much and is the one product teams resist. When retrieval does not clear the threshold, or when the retrieved passages do not actually address this pair on this route, the response is "coverage": "not_governed" and the screen says so plainly: no statement in the current policy covers this combination, here is the escalation path, here is the on-call pharmacist's extension. A conversational assistant will almost never do this, because a language model asked a question in a chat window produces an answer, and the general-knowledge answer it produces when the corpus is silent looks exactly like the policy-backed answer it produces when the corpus is not. Making non-coverage a distinct, visible, first-class outcome is the single largest safety difference between the two designs.

The audit write is not bookkeeping. When an incident is reviewed six months later, the question is what this nurse was shown at 02:40, and the only defensible answer is a row that names the passage, the document version and the timestamp. A chat transcript records what the model said, which is a different and much weaker artefact, because it records a generated sentence rather than a policy state.

The model is still doing most of the work#

None of this argues against using a model. It argues against letting the model author the sentence the nurse reads. Everything upstream of that sentence is model work, and it is the part that makes the feature possible at all.

Drug names arrive as brand names, local abbreviations and misspellings, and mapping "tazocin" and "pip-tazo" onto the same normalised concept — the rxcui identifiers in the response above, which come from RxNorm, the standard vocabulary that gives every clinical drug one code regardless of brand — is a language problem. The policy corpus is written in prose with inconsistent structure across 340 documents, and finding the passage that governs this pair on this route rather than the passage that merely mentions both drugs requires reading the candidates and judging them. That coverage judgement — does this text actually decide the question asked — is a model call, and it is the one that produces the not_governed outcome. Where several passages apply, ordering them by specificity is model work too.

The model does all of that and then stops, and the interface shows what it found. The design principle generalises past medicine: retrieval and judgement are model work; authorship is not, when the artefact is what decides. Pricing works the same way — a customer asking what they will be charged needs the rate card line that applies to their contract, not a helpful approximation. So does anything where a regulator, an auditor or a court would ask what the person was shown.

Equally, this is not a universal rule about internal documentation. A nurse asking how to book annual leave should get a paraphrase, cheerfully, because nothing decides on the exact wording of the leave policy and a summary is genuinely more useful than the paragraph. The gate is narrow by design; applying it everywhere produces an interface that quotes policy at people who asked a simple question.

The corpus is the hard part, and it does not get easier#

Two problems remain unsolved by anything above, and both are about the documents rather than the interface.

The first is versioning. The policy in the example is version 7.2, effective 1 November. If 7.3 was issued on 1 March and the index still holds 7.2, the system shows a governing passage that no longer governs — and it shows it with more authority than a chat answer would, because the whole design says "this is the text". That is a real and inherent risk of the artefact-first approach, and the mitigations are unglamorous: the index carries document versions rather than loose chunks, the response names the version and effective date so a reader can catch a stale one, and there is a monitored job that reconciles the index against the document management system with an alert when they diverge. The hospital in this example set the alert at any divergence lasting more than four hours and treated it as a page-worthy incident, which is the correct severity for a system whose answer is "this is what the policy says".

The second is that some questions genuinely have no governing statement, and the honest rate at which that happens is higher than anyone expects. When the pilot ran against 400 real questions collected from the ward, 31 percent came back not_governed. The instinct in the room was that a two-thirds answer rate made the feature a failure. It did the opposite: it produced the first reliable list of what the policy does not cover, which went to the committee that writes it, and the coverage gap shrank over two revisions. A chat assistant would have answered all 400 and generated exactly none of that signal.

The surface is right and the flow around it can still be wrong. A lookup that sits one navigation away from the drug chart, or that asks the nurse to confirm the patient it already knows, has reintroduced the interaction cost the design was meant to remove — in smaller units, once per use, forever. Finding those units means tracing what a person actually does before and after a request, which is the next chapter.