Agent Control Lab: Three Approval Gates, Four Synthetic Tests

What happens when alternative agent-control designs fail in synthetic tests?

In this four-case experiment, a gate checking only approval presence incorrectly allows a changed artifact and a changed recipient. Adding artifact identity fixes one failure; binding both artifact and recipient matches all four expected outcomes. These are observed deterministic fixture results, not measurements of a language model or proof of production security.

The test fixture

The fictional approved action is to send memo-v1 to reviewer@example.invalid. The artifact and recipient are simple strings. The expected policy allows only that exact approved pair and denies an action with no approval. The .invalid addresses are test identifiers, not real recipients.

• T1: memo-v1, approved recipient, approval present. Expected: allow.
• T2: memo-v2, approved recipient, approval present for memo-v1. Expected: deny.
• T3: memo-v1, outsider@example.invalid, approval present for the original recipient. Expected: deny.
• T4: memo-v1, original recipient, no approval. Expected: deny.

Three implementations

Design A returns hasApproval. Design B returns hasApproval AND artifact equals approvedArtifact. Design C returns hasApproval AND artifact equals approvedArtifact AND recipient equals approvedRecipient.

These deliberately small predicates isolate one design question: what information is bound to an approval? They do not authenticate the approver, secure storage or handle concurrency. The NIST generative-AI profile is a broader voluntary risk-management reference; these particular tests are our original experiment. [S1]

• A: approval presence only.
• B: approval plus artifact identity.
• C: approval plus artifact and recipient identity.
• Comparison rule: an actual boolean result passes only if it equals the fixture's expected result.

Observed results and failure interpretation

Design A produced allow, allow, allow, deny and passed 2 of 4 fixtures. Design B produced allow, deny, allow, deny and passed 3 of 4. Design C produced allow, deny, deny, deny and passed 4 of 4.

The failure in T2 is approval reuse after the output changes. The failure in T3 is approval reuse for a different destination. Both are possible in an apparently successful workflow: the system can send exactly what its current instruction says while violating the scope of the prior approval.

• A fails T2 and T3.
• B fails T3.
• C matches this fixture policy in all four cases.
• These counts describe only four hand-authored cases; they are not an estimated real-world failure rate.

Reproduce and extend the experiment

Create four records containing artifact, recipient, approvedArtifact, approvedRecipient, hasApproval and expected. Evaluate each of the three predicates above against every record. Compare each result to expected and retain both the boolean and the mismatch reason.

For a more realistic follow-up, replace artifact labels with verified content hashes, test approval expiry and revocation, add multiple recipients and simulate a document change between check and send. Those extensions have not been run here. A control that passes this small suite still needs evaluation against the actual system and threat model.

• Keep the policy oracle independent of the predicate being tested.
• Include an allowed case so an always-deny implementation cannot look successful.
• Add unknown-outcome and retry scenarios before testing external actions.
• Do not connect the experiment to live sending merely to demonstrate the logic.

Scope and limitations

Synthetic local JavaScript experiment executed November 5, 2026. No network messages, real documents or accounts were used.

Only the stated deterministic predicates were executed. No LLM, live workflow, authentication, cryptographic binding or production deployment was evaluated.

Sources

1. NIST AI 600-1: Generative Artificial Intelligence Profile — Cross-sector companion to the voluntary AI RMF for generative AI risk management. Checked 2026-11-05

Explore further

This lab provides a concrete way to think about human review controls when exploring Agentic Attorney at https://agenticattorney.com/. It does not describe or test the central project's implementation. Explore Agentic Attorney.