Rejection criteria for third-party servers

Two documents can be written about installing third-party MCP servers, and they behave completely differently under pressure. One says what a good server looks like. The other says what disqualifies one. They can contain the same technical judgements and produce opposite outcomes on the same Tuesday, which is why the form is worth more attention than the content.

The first form is the one everybody writes. It reads as a list of properties a server should have — verified publisher, active maintainer, minimal dependencies, secure credential handling, least privilege, no critical CVEs, pinned version, reviewed, re-reviewed annually — and it collects agreement easily, because nobody disagrees with any of it. Agreement is the problem: a document nobody argues with is a document nobody has tested against a deadline.

Why a list of security properties loses to a deadline#

MCP Server Security Requirements  (v0.9, adopted 4 March)

A third-party server SHOULD:
 1. be published by a verified organisation
 2. have an active maintainer and recent releases
 3. declare a minimal dependency footprint
 4. handle credentials securely
 5. avoid unnecessary network access
 6. support least-privilege configuration
 7. have no known critical CVEs
 8. be pinned to a specific version
 9. be reviewed by a member of the platform team
10. be re-reviewed annually

A property list is evaluated at the worst possible moment, by the person with the most to lose from a no, against criteria that are all matters of degree. "Minimal dependency footprint" has no failing value. "Handles credentials securely" is a conclusion rather than an observation, so the discussion is about whether the reviewer's discomfort is proportionate. Under a deadline, the burden of proof sits with whoever wants to block, and the blocker has to argue from a document that never committed to a threshold.

There is a structural problem underneath the wording. Ten properties evaluated together produce a score, and a score invites trading: three strong items against one weak one feels like a pass. The failures that matter here do not trade. A server that posts your tool arguments to its own endpoint is not redeemed by an excellent test suite, and the distinction between a gap the author would close and a feature the author shipped is exactly the one a scored checklist erases, because it flattens "fails to do something it intended" and "does something it intended" onto the same scale.

The defect in that is arithmetic before it is anything else. A score is a sum, and a sum presumes its terms share a unit — that a point of dependency hygiene and a point of credential handling are the same kind of quantity and can therefore be exchanged for one another. Some of the ten items are that kind of quantity and some are not. A missing containment check and an unmaintained parser are gaps between what an author meant and what an author wrote, and running the thing carefully closes them, so they sit on a scale of how much care the install needs. A destination the author chose is the implementation matching the intent exactly, and no amount of care on your side moves it, so it sits on no scale at all: it is a yes or a no. Adding a yes-or-no to a how-much and reading off a total is the operation a property list performs, and the total means nothing, which is why the conversation that follows one is always a conversation about how people feel.

The fix is to invert the document. Written as rejection criteria, the intent becomes a single sentence: no third-party MCP server may be installed if it exhibits any of the named provenance, credential-handling or egress gaps below, and everything else about a server is a configuration decision for the installing team. Three things change with that sentence. The default flips, so the burden moves to whoever wants the install rather than whoever wants to stop it. The decision moves earlier in time, because the criteria are written before any particular server is on the table, when nobody has a launch date attached to the answer. And the failure mode becomes visible, because a server that hits a criterion and is installed anyway produces a written exception with a name on it instead of eleven minutes in a channel.

Three criteria, and what decides each#

Each criterion names a gap, not a virtue, and each one names the artefact that settles it.

The three are chosen because each is a fact rather than a judgement, each is cheap to establish relative to the cost of being wrong, and each corresponds to a failure with no configuration remedy. Provenance is first because it is the precondition for the other two meaning anything: a review of source that does not correspond to the running artefact establishes nothing, which makes it the criterion most worth enforcing and the one most often skipped because the package "obviously" comes from its repository.

That ordering is a dependency rather than a preference, and it is worth seeing why the other two collapse without it. The credential criterion is settled by reading the auth path in the source and sweeping canaries through a run. The egress criterion is settled by reading every base URL in the source and checking the logs from that same run. Both procedures take an artefact as their input, and both produce a conclusion about whichever artefact they were given. If nothing ties the artefact you examined to the artefact that will run, the conclusion is about a file on your laptop while the install is about a different file, and the two hours have bought a statement about the wrong object. Provenance is therefore not the first of three tests of equal standing. It is the condition under which the other two are tests at all, which is why ten minutes of it is what makes the other two hours worth spending.

Just as important is what is deliberately not on the list. Dependency count is not a rejection criterion, because there is no defensible threshold and the number correlates weakly with risk. Open CVEs in the tree are not, because they are defects with fixes and belong to the configuration conversation and to the scanning process the company already runs. Star count, release cadence, test coverage and code quality are not, for the reason the chapter on source review gave about tidy codebases with telemetry pipelines: none of those properties bears on where the data goes. A short list of criteria that are genuinely disqualifying survives contact with a deadline; a long list containing items everyone privately knows are proxies teaches people that the document is negotiable, and that lesson transfers to the items that were not.

There is a cheap way to find out whether a set of criteria is calibrated before anyone has to live with it, and it is the discriminating-power test that topic 6.13 applies to an isolation suite, pointed at a document instead. Run the criteria backwards over the servers already installed. A set that rejects none of them returns the same answer whatever it is asked about, so running it tells you nothing you did not know before you ran it. A set that rejects nearly all of them has the identical defect from the other end, with an extra property attached: it will be overridden every time, which teaches everybody that the document is negotiable, and that lesson transfers to the criteria nobody was arguing about. Only a set whose answer depends on which server it was asked about is worth adopting, and running it retroactively is how you find out which kind you have written — before the answer matters, rather than during the eleven minutes when it does.

The path that exists when a team hits one#

A rejection criterion with no exception path is a rule people route around, so the path is part of the policy rather than an admission that the policy is soft. What makes it work is that it is more expensive than compliance and less expensive than lying, and that it produces a record.

# exceptions/2026-06-11-schedulr-mcp.yaml
server: "@schedulr/calendar-mcp@3.1.2"
digest: "sha256:1c9a…"
criterion_hit: egress-by-design            # status ping to status.schedulr.dev
requested_by: team-billing
business_reason: >
  Calendar sync for the 19 June launch; no first-party alternative exists
  and building one is estimated at 3 weeks.
compensating_controls:
  - egress allowlist denies status.schedulr.dev; server tolerates the failure
  - profile ide-assist only; denied on any profile with write scope
  - separate vault path, token scoped to one calendar
expires: 2026-09-09                         # 90 days, no auto-renewal
approved_by:
  - name: "R. Iyer"                         # director, billing
  - name: "M. Okafor"                       # security on-call, week 24
review_owner: "team-billing"

Four features of that record carry the weight. A named person, because "the platform team approved it" is nobody and cannot be asked about it in October. An expiry with no automatic renewal, so that the exception must be re-argued in a quarter when the launch is over and the argument is easier to lose. Compensating controls written as specifics that someone can verify — compensating meaning that they shrink the exposure the criterion was aimed at without claiming the gap has been closed — since "we will be careful" is not a control. And a business reason with an estimate attached, which is what makes the exception log readable later as a list of the things the company chose to buy with risk. Teams that keep this log discover the same thing every time they read it back: two or three exceptions account for most of the residual, and one of them has been renewed four times because nobody ever budgeted the three weeks.

The expiry is the part that gets removed first when the policy is trimmed for being bureaucratic, and it is the part doing the most work. Without it, an exception is a permanent change to the policy made by one team under deadline pressure, which is precisely the outcome the rewrite was meant to prevent.

The date in that file is checkable in the way the rest of the record is supposed to be, and checking it is worth a moment, because the arithmetic is the enforcement. The eleventh of June to the ninth of September is the rest of June, then the whole of July, then the whole of August, then nine days — which adds to ninety, the interval the policy names. Writing the resulting date rather than the interval is what makes the expiry enforceable by something other than memory, since a script can compare today against a date and nobody has to recall when the quarter started. It also puts the re-argument after the nineteenth of June launch the exception was bought for, which is the property to check whenever somebody proposes a different number: an expiry falling before the thing it was granted for is a renewal request wearing a deadline.

What a green pipeline can and cannot mean#

The reasonable objection to all of this is that a document is weaker than a gate, and that the checklist should be automated in CI: mechanise every item that can be mechanised, require sign-off for the rest, and let nothing install without a green run. Automation is right for the mechanisable subset, and it manufactures a confidence that is dangerous in a specific way.

The three rows in the middle and bottom are the ones that decide installs, and none of them is machine-checkable. The hazard is not that the pipeline is useless; it is that a green run is read downstream as "this was reviewed", by people who will never see which columns the pipeline covered. You end up with a rigorous gate on the easy questions and no gate at all on the hard ones, and the rigour of the first makes the absence of the second invisible. Automate the top three rows, label the check honestly as what it is, and keep the human decision explicitly separate rather than folding it into a status badge.

What makes a badge dangerous is a specific substitution rather than general laziness, and it is worth naming because it is invisible to the person making it. A green check is one symbol standing for a set of questions, and the set is not visible at the point the symbol is read. Somebody three steps downstream — a reviewer approving a configuration change, a director signing off a launch — sees the symbol and infers the largest set it could plausibly stand for, because that is the reading that requires no additional work of them. The gap between the set the pipeline covers and the set the reader infers is where the whole risk lives, and it widens as the pipeline improves: the more rigorous the automated part, the more confidently the symbol is read, and the less anyone asks what was left outside it. Naming the check after the three rows it actually covers is the cheap correction, because a symbol that states its own scope cannot be read as covering more than it does.

A policy written as three named gaps, each with its evidence and its cost, is short enough to be read, specific enough to be argued against in advance, and structured so that being overruled leaves a file with a date and two names on it. That is the whole of what a document can contribute here, and it is more than a list of virtues contributes, because it survives the one afternoon when it matters.

Its limits are worth naming in the same breath. It does not decide which loops a server may sit in, which is a per-deployment pairing rather than a per-server verdict. It does not govern servers your own team writes, where the provenance criterion is trivially satisfied and the interesting questions are the multi-tenancy ones of topic 6.13, on isolating tenants inside a server you operate. And it is not a vendor risk questionnaire, since the answers to a questionnaire are supplied by the party being assessed, which is the same category of evidence as a README.

The larger limitation is that every criterion above assumes controls that have never been exercised. The compensating controls in that exception file claim that an allowlist denies a host, that a profile has no write scope, and that a token covers one calendar, and the policy takes all three on trust. Nobody has watched a credential try to leave, checked whether an alert fires, or timed how long a rotation takes when it turns out the token is in six places.