QAEDA-TST-006 · procedural safeguard

Require notice, reasons, and recourse

If the system materially restricts a person, can they understand what happened, contest the relevant evidence, and obtain meaningful human review?

Pass condition

What a defensible proposal should be able to show.

Consequential actions provide intelligible reason categories, disclose the governing rule, permit correction or appeal, and track whether review can actually reverse erroneous decisions.

Warning signs

Signals that the proposal is drifting.

  • Visibility, access, eligibility, or account standing changes without notice.
  • Appeals are routed into the same automated classifier with no independent review.
  • The operator cannot identify which rule or evidence drove the action.
  • Error rates and reversal rates are unavailable even in aggregate.

Evidence to demand

What should exist before confidence rises.

  • Appeal latency and overturn-rate data.
  • Examples of reason notices for representative cases.
  • Evidence that reviewers have authority to reverse automated decisions.
  • Independent or adversarial evaluation of false positives.
Failure mode

Opaque automation can convert uncertain inference into unreviewable deprivation and train users to conform to invisible rules.

Decision discipline

No single passing test licenses the intervention.

Apply the full sequence. A defined harm does not excuse a weak nexus; a narrow mechanism does not excuse secret secondary use; an appeal process does not prove efficacy.