How we test

A product that fails your spec can't come back as the answer.

Not because a model is careful — because the screen is code. A hard constraint is a SQL filter over typed columns, the model only ever sees products that survived it, and we attack that property with generated cases on every change. The numbers below are measured, and they carry their date.

The property under test

A product that violates a stated hard constraint must be eliminated — with a one-sentence reason and the source line it was judged against. If we can't say why a product was ruled out, the screen is wrong, and we fix the screen.

01
Typed, then filtered
Every requirement becomes a typed constraint, and every hard constraint is enforced as a SQL filter. Similarity never decides a verdict.
02
One verdict each
Every product is evaluated exactly once, and the counts must add up — recommended, to verify, and ruled out reconcile against the catalogue count, or the result says so.
03
Silence is not a pass
Where the data sheet is silent, the verdict is unknown and the missing value is named. Absence of evidence never converts into a pass.
04
Survivors only
The model drafts from products that survived every hard constraint. A failed product is not in what it reads — so it can't resurface as a match.

Measured, not promised

One number we refuse to round: false passes.

A false pass is a product that violates a stated hard constraint and survives the screen anyway — the one failure a spec screen must never produce.

FALSE PASSES
0
of more than 56,000 surviving verdicts checked — false-pass rate 0.000000 across more than 200,000 generated cases, run 2026-08-25 at code_fp 5c14510b48bce40c.
GENERATED CASES
200,000+
more than 200,000 generated for the 2026-08-25 run: numeric boundaries, unit traps, composition contradictions, injection payloads.
EVERY CHANGE
Re-run in CI
the full bank screens in seconds with zero model calls, so it runs on every pull request — one violation fails the build.
SECOND GATE
Model-in-the-loop
every screening change must pass a model-in-the-loop evaluation at zero false passes before it merges.

Conditions, because a number without them is a slogan: measured 2026-08-25 on the engine build this page describes (fingerprint 5c14510b). The judge is an independently written oracle that re-checks every surviving product against the stated constraints — never the screening code grading itself. When the engine changes, the number is re-measured, not quoted forward.

Show, don't tell

Real output, with the receipts.

This is the deployed engine's own answer to the first example inquiry on the home page — the fictional demo catalogue, the real engine. Every line below is quoted from the live response, byte for byte.

Screened result deterministic engine

Inquiry

Matting agent for a waterborne wood coating. Particle size under 10 microns, PTFE-free, VOC under 50 g/L.

Parsed constraints

max particle size 10PTFE-free truewaterborne truemax VOC 50
No product in this catalogue meets every requirement — the binding one was PTFE-free. That is the honest answer, and it is the one a rep can send.

Three of the thirty receipts

Ruled out
Polyfluor 905
composition states a fluoropolymer (not PTFE-free)
Sheet line it was judged against: "100% micronized PTFE"
Ruled out
Fischertex 200
not waterborne
Sheet line it was judged against: "Fischer-Tropsch wax"
Unknown
Mattex 201
PTFE-free status not stated on the data sheet — a recognized non-fluoropolymer composition names the polymer backbone and does not disclose additives or processing aids; needs review
Sheet line it was judged against: "Untreated silica matting agent"
30 screened0 recommended17 to verify13 ruled out

Reproduced 2026-08-25 from the same public endpoint the home-page demo calls. The example inquiries use stored typed constraints, so this result is deterministic — no model call, same answer every run. Seventeen of thirty products land in to verify because their sheets never state PTFE-free either way: refusing to guess is the behaviour under test, not a gap in the demo.

The honest edges

What this measures — and what it doesn't.

  • The guarantee is about the screen — the step that decides. Reading your customer's prose into typed constraints is a separate step with its own tests, and the rep sees the parsed constraints and can correct them before anything is sent.
  • The bank is generated, plus a fixed inquiry set — it is not your traffic. Your catalogue gets its own measurement when we ingest your line card: where a sheet is silent, GRADE returns unknown and names the value it needs, and we measure that gap per field.
  • A checker can be wrong too. That is why the oracle is written independently of the screening code, and why a case that finds a real defect is added to the bank permanently — the attack that worked once is re-run forever.

See it yourself

Watch it enforce a spec.

Paste an inquiry on the home page and screen the demo catalogue with the deployed engine — or send us a real one from your inbox.

Try the live demo → Talk to us