QAEDA-ARG-012 · methodological rule

Audit behavior, not only policy promises

A meaningful audit should compare documented policy with observable system behavior and should keep evidence, methodology, and normative scoring criteria explicit.

Strongest objection

What pushes against this view?

Black-box testing can misinterpret nondeterministic systems, violate terms or law if done carelessly, and overstate what a small benchmark proves.

Current response

Why QAEDA still holds it.

Testing should be bounded, lawful, reproducible, privacy-preserving, and explicit about sampling limits. QAEDA’s public audit framework is methodology, not permission for intrusive testing or a certification badge.

Revision rule

What would change the argument?

Revise audit categories and metrics when evidence shows they are unreliable, easy to game, non-reproducible, or poorly connected to real cognitive-liberty outcomes.

Open question

What remains unresolved?

Which audit measurements best predict real user agency and privacy rather than merely compliance theater?

Challenge the reasoning

The graph is a map of current arguments, not a demand for agreement.

Point to a missing premise, stronger objection, counterexample, source, or revision condition. Stable IDs make disagreement specific.