Cross-Version Evaluation of Guardrail Execution Precedence and External Side-Effect Commitment in Agent Runners

1. Introduction & Problem Statement

Agent orchestration frameworks delegate multi-step decision loops to Large Language Models (LLMs). When agents interact with external APIs, databases, or financial services, security governance requires strict control plane invariants:

Guardrail Evaluation (T₁) ≺ Tool Side-Effect Commitment (T₃)

Audit A02-07 documented that v0.22.3 under parallel execution lacked a synchronization barrier prior to tool execution. Audit A02-08 investigates whether this behavior is an isolated artifact of v0.22.3 or a persistent architectural trait across multiple SDK releases, and evaluates its impact when tools interact with process-isolated HTTP network endpoints.

2. Experimental Corpus Accounting (N = 3,300 Total Scope)

To maintain strict scientific rigor and transparency, the experimental corpus is categorized into four distinct accounting categories:

Note: The 300 master-DB matches belong strictly to the 900-trial forensic replication suite. The 2,000 original evaluable trials are preserved separately in JSON traces.

3. Chronological vs. Causal Evidence Synthesis

3.1 Within-Runner Monotonic Initiation (T₂ < T₁)

Inside the runner process using high-resolution monotonic timestamps, we measure the relative sequence between tool HTTP dispatch initiation (T₂) and guardrail verdict logging (T₁):

• Observed Rate: In 100/100 parallel DENY trials per SDK version (300/300 total across v0.20.0, v0.21.1, and v0.22.3 in the forensic replication), T₂ < T₁ was recorded under the tested 50ms guardrail vs 5ms tool latency parameterization.
• Scientific Boundary: T₂ < T₁ demonstrates that tool HTTP dispatch was initiated before the guardrail finished evaluation. Because clocks across independent OS processes cannot be assumed synchronized without physical cross-process hardware clocks, T₂ < T₁ alone must never be used to claim server-side commitment before verdict (T₃ < T₁).

3.2 Cross-Process Causal Transport & Exception Precedence (T₃ ≺ T₄)

Using correlated trial IDs, request headers, and idempotency keys across process boundaries:

Runner Dispatch (T₂) ≺ Server Socket Receive (S_REC) ≺ Server Commit (T₃) ≺ HTTP Response (T_2,resp) ≺ Tripwire Exception (T₄)

For trials where full transport and server event correlation was confirmed (the 300 Condition A parallel DENY forensic trials):

1. The tool function sends the HTTP request (T₂) and awaits the HTTP response (T_2,resp) before returning.
2. The server receives (S_REC), commits (T₃), and logs the event in the master database prior to responding.
3. The runner only communicates/raises the InputGuardrailTripwireTriggered exception (T₄) after asyncio.gather collects task results.

Therefore, for all verified trials with confirmed HTTP responses, external side-effect commitment (T₃) is causally established prior to tripwire observation (T₄).

4. Multi-Version Empirical Findings Matrix

5. Automated Adversarial Assessment Note (JEV)

The TypeSafe AI JEV adversarial service conducted an automated evaluation (jev-1.13.0, 6,165 input tokens, HTTP status 200 OK) of the preliminary A02-08 report and initial harness code. The automated model output attributed the observed behavior to the SDK's asyncio task scheduling structure under default parallel configuration.

6. Scope Boundaries & Methodological Limitations

1. Localhost Process-Level Scope: The target HTTP microservice was hosted locally on localhost (127.0.0.1:8888). No remote internet endpoints or production third-party systems were contacted.
2. Synthetic Latency Ratios: The 100/100 violation rate under parallel DENY is conditional on the tested 50ms guardrail vs 5ms tool latency parameterization and does not represent a static probability for arbitrary network conditions.
3. Audited Version Bounds: Empirical observations apply strictly to OpenAI Agents SDK releases v0.20.0, v0.21.1, and v0.22.3.

7. Recommended Governance Mitigations

To enforce happens-before safety in agentic workflows:

1. Enforce Sequential Mode for Critical Actions: Configure run_in_parallel=False on input guardrails protecting high-consequence tools.
2. Implement Pre-Commit Authorization Barriers: Require tool functions to await explicit guardrail completion flags before issuing side-effect payloads.
3. Deploy External Control-Plane Proxies: Use an out-of-process sidecar proxy that holds HTTP transport payloads until signed guardrail clearance is verified.

8. Methodology, Reproducibility & Data Availability

• Execution Environment: Windows 11 AMD64, Python 3.11.15, OpenAI Agents SDK releases v0.20.0, v0.21.1, and v0.22.3 isolated in versioned virtual environments.
• Model Adapter: Offline deterministic adapter (ScriptedModelAdapter) generating constant function-call decisions to eliminate LLM non-determinism.
• HTTP Microservice: Process-isolated Python HTTP server (local_http_service.py) maintaining a WAL-mode append-only SQLite database (external_service_audit_master.db).
• Reproducibility Material: Full test harnesses (a02_08_remediation_harness.py), evidence manifest (A02-08_EVIDENCE_MANIFEST.json), and master database are archived locally. Private raw traces and API credentials are withheld to maintain security governance.