Preflight
ChecksDocsBillingAll the Possibles

All four checks

Flag agent output that parses as valid JSON but holds no real answer.

Output check, called check_output over MCP.

What gets better

A schema check passes this output; this check finds the three identical retry steps and the field that holds only TBD.

Before: what you give it

The output as a JSON string or parsed value, plus an optional list of fields that must be present.

{
 "output": {
  "summary": "TBD",
  "owner": "unknown",
  "steps": [
   {
    "action": "retry"
   },
   {
    "action": "retry"
   },
   {
    "action": "retry"
   }
  ],
  "risk": ""
 },
 "expectedFields": [
  "summary",
  "risk",
  "deadline"
 ]
}

After: what comes back

Output that passes a schema check: every field exists and every type is right. The report below was produced by the scanner, then its findings listed one per line.

verdict: suspicious
HIGH      Expected field "risk" is missing or empty [risk]
HIGH      Expected field "deadline" is missing or empty [deadline]
MEDIUM    Placeholder value at summary: "TBD" [summary]
MEDIUM    Placeholder value at owner: "unknown" [owner]
HIGH      3 identical items at steps [steps]

What you get

A verdict (looks-real, suspicious or degenerate) and a finding for each placeholder value, missing field and run of identical items, with its path.

What it cannot do

It looks for a fixed set of degenerate patterns. It cannot tell whether a well-formed, plausible answer is correct.

Who it is for

  • Developers running agent loops
  • Teams that gate downstream actions on model output

Free and Pro

Free: 20 requests a minute per IP address, no sign-in. Pro adds: Nothing. It is pure and runs the same for everyone.

Full reference

Use it from your assistant

Add Preflight as an MCP server once, then call check_output by name or describe the job.

curl -X POST https://preflight.allthepossibles.com/api/check-output -H 'content-type: application/json' -d '{"output":"{\"summary\":\"TBD\",\"owner\":\"unknown\",\"steps\":[{\"action\":\"retry\"},{\"action\":\"retry\"},{\"action\":\"retry\"}],\"risk\":\"\"}","expectedFields":["summary","risk","deadline"]}'

Try it here

The box starts with the example input. Run it, then swap in your own.