faultgate 0.3.1 · fixes from the first outside test, 180 real tau2-bench runsRead the changelog →
faultgate docs

Current Limits

What it doesn’t do yet

  • Premature stop (the agent answered with an intermediate entity of the chain). Detecting it needs the traversal chain, not just the final answer.
  • Semantic injection checks (the agent obeyed text it read). FAULTLINE P9 has the taxonomy. It needs a judge that scores.
  • Running your agent. faultgate reads traces. It never executes anything.

Policy checks and semantic injection

Structural policy checks catch tool calls that break a rule: an unlisted tool, a denied argument, a broken limit, or a missing prerequisite. They do not judge what the agent was persuaded to do. When the agent obeys text it read in a document, that is a semantic injection. It is a judge question, not a rule, and it is not in this release. See Business Rules for the structural keys.

Judge context limits

Laya’s context is 512 tokens. faultgate compacts the trace to fit, keeping the answer and termination first and eliding middle steps. It counts every call that still overflows, and the count is printed. Treat a judge verdict on a heavily compacted trace with care. See Trust & Calibration for how the judges score.