Network-Egress Assertion and Hermetic Testing for AI Agent Tooling

Executive Summary

"Zero network calls" is a claim about the absence of an event, and absence is structurally harder to test than presence. A test that pulls the network cable (poisons curl, blackholes DNS, revokes credentials) and observes the program still succeeds only proves the program is resilient to network failure — it says nothing about whether the program tried to make a call in the first place. If any code path swallows the resulting error and falls back to a default, the poisoned-network test goes green regardless of whether the "offline" code path was ever exercised. This is a false-confidence pattern with a well-known shape across the industry: a test that cannot distinguish "the property holds" from "the property is broken but something else masked it."

The fix that generalizes is a two-part discipline, independently rediscovered by hermetic-testing practice at Google and Bazel, by HTTP-mocking libraries in every major language ecosystem, and by OS-level sandboxing tools:

Replace a failing dependency with a recording dependency. Instead of making the network unreachable, make it reachable but instrumented — a mock/stub/interceptor that journals every call it receives, including rejected, unmatched attempts, in a mode where an unmatched or unexpected call is an error, not a silent no-op.
Assert absence, not just success. The test must explicitly check "zero calls were recorded" (or "only expected calls were recorded") against that journal, and — critically — apply a positive control (a known-bad mutant): temporarily reintroduce the bug (revert the fix, or call the network path unconditionally) and confirm the test does go red. A check that cannot fail is not a check.

This document surveys how this pattern is implemented across hermetic build systems (Bazel, Nix), HTTP-interception libraries in JavaScript, Python, Ruby, Go, and Java, OS/sandbox-level enforcement (network namespaces, --network=none, seccomp/eBPF), the software-testing epistemology of experimental controls and mutation testing, and its emerging application to AI agent systems that claim offline/air-gapped operation or deterministic, non-live-API test suites. It closes with a practical playbook for building "assert-zero-egress" tests that cannot lie.

The Problem: False Confidence from Failure-Injection

Failure-injection ("break the network and see what happens") is a legitimate and common resilience test — it validates that a program degrades gracefully. It is a categorically different test from "this code path performs no network I/O," and conflating the two is the root of the false-confidence failure mode described in the motivating incident: an "offline install" feature was tested by making the network binary fail, the test passed, and the reviewer later discovered the pass was caused by an unrelated silent fallback swallowing the induced error — not by the offline code path actually running with no attempted calls.

This class of bug has a name in the broader literature: error hiding or error swallowing — catching an exception and continuing without logging, re-raising, or otherwise surfacing it. Wikipedia's treatment of error hiding notes it is considered an anti-pattern precisely because it masks defects instead of revealing them, extending the feedback loop and letting the system continue in an unverified state (Error hiding, Wikipedia). AWS's own static-analysis tooling (Amazon CodeGuru) ships a dedicated detector for "catch and swallow exception" for this reason (CodeGuru swallow-exceptions detector). The Schneide Blog's catalog of testing anti-patterns and Square's engineering post "Chasing Swallowed Exceptions" both describe the downstream cost: when a test's green light can be produced by two unrelated causes (the real property holding, or an error being silently absorbed), the test has stopped being evidence (Testing antipatterns, Chasing Swallowed Exceptions).

The general lesson: a test's failure mode must be inspected as carefully as its pass condition. "Does this test go green when the property holds?" is necessary but not sufficient; the complementary question — "would this test go red if the property were broken?" — is what the experimental-controls and mutation-testing literature (covered below) is built around.

Hermetic Testing Foundations

"Hermetic" testing means a test's result depends only on the code and data explicitly given to it, not on ambient state of the host — network reachability, installed packages, system clock, other running services, or DNS resolution.

Bazel treats hermeticity as a first-class build property: "a hermetic build system... always returns the same output" by isolating the build from the host, and Bazel enforces this at the action level by running each build/test action inside a sandboxed execroot containing only declared inputs, so tools "cannot accidentally read random files on the host or reach out to the network" (Bazel: Hermeticity). Sandboxing implementations (linux-sandbox, darwin-sandbox, processwrapper-sandbox) run locally by default, and Bazel exposes network access as an explicit, declarable policy rather than an ambient capability, so a test that needs network must say so rather than silently having it (Tweag: How to keep a Bazel project hermetic).

Nix achieves a related but distinct property. Its build sandbox isolates derivations from the host filesystem and (by default) the network, reporting empirically high reproducibility across millions of packages and hundreds of nixpkgs revisions in production measurement — isolation strength and bit-for-bit reproducibility are described as orthogonal properties that different tooling addresses independently (Nix: A Solution With Problems, arXiv).

Google's "hermetic server" concept, from the Testing on the Toilet / Google Testing Blog series, generalizes the same idea to integration tests: "a hermetic server is a real server that is brought up by the test and runs on the same machine... if you can start up the entire server on a single machine that has no network connection and the server works as expected, you have a hermetic server." The recommended construction is dependency-injecting all external connections at runtime (flags, DI containers), bundling static assets into the binary, and faking any datastore with an in-memory or file-backed implementation (Google Testing Blog: Hermetic Servers). More recent Google practice reports on ephemeral, hermetic test environments as a direct flakiness-reduction technique, citing unreliable external dependencies and network communication as leading causes of flaky CI (How we use Hermetic, Ephemeral Test Environments at Google, ICST/CCIW 2023).

Reproducible builds work (BuildStream, and the broader reproducible-builds.org community) makes the network-isolation goal explicit at the tooling level: "the sandbox... serves two purposes: giving [the build system] control over all build aspects... and providing safety guarantees for the host system," with the aim that "builds execute with only their declared inputs available, catching any undeclared dependencies through build failures rather than allowing silent network fetches" (BuildStream Sandboxing docs; reproducible-builds.org: Controlling the build environment).

The common thread across all four: hermeticity is achieved by making the ambient environment absent or fully declared, then observing that the system either fails loudly (undeclared dependency → build error) or succeeds correctly using only what was declared. This is architecturally identical to the "fail-closed on unmocked call" pattern used by HTTP-interception libraries, covered next.

Ecosystem Tooling Survey

Every major HTTP-client ecosystem has converged on a strict/fail-closed interception mode where an unmatched request throws instead of hitting the network; some — not all — additionally expose an assertion surface for "were there requests I didn't expect" and "were there expected requests that never fired." This symmetry — assert nothing unexpected happened and assert everything expected did happen — is what makes a test able to catch both over-calling and (via known-bad mutants) under-testing.

JavaScript / Node — nock, MSW, undici.

nock.disableNetConnect() makes any unmocked outbound request throw a NetConnectNotAllowedError ("Nock: Disallowed net connect for [host]") instead of silently hitting the network or silently no-op-ing. Nock's own issue tracker explicitly discusses the failure mode of interest here: disableNetConnect() errors surface as application-visible network errors, so a catch-all in application code can swallow them — the throw is a blocker, not a ledger (nock/nock#884). Nor do nock.isDone() / pendingMocks() close that gap: they assert that registered expected mocks were consumed, so in a zero-egress test with no registered mocks they are vacuously satisfied. The actual ledger of unexpected attempts is nock's global no match event (nock.emitter.on('no match', ...)), which fires at the interception layer for every unmatched request even when the thrown error is later swallowed downstream — nock/nock#991 is a request for a first-class version of exactly that assertion (nock README: no match event).
MSW (Mock Service Worker) exposes onUnhandledRequest: 'error' on server.listen() / worker.start(), which throws whenever a request has no matching handler — recommended explicitly for CI to catch untested network calls rather than let them warn-and-continue (MSW: start() docs, MSW: Debugging uncaught requests). One nuance matters for zero-egress tests: MSW's request:unhandled life-cycle event is documented as firing for unhandled requests that "will be performed as-is" — the warn/bypass paths — so it cannot be assumed to journal requests that strict 'error' mode aborts (MSW: Life-cycle events). The unambiguous ledger is the custom function form of onUnhandledRequest(request, print): record the request into your own journal first, then raise the error — one documented hook acting as blocker and ledger at the same point, with no ordering ambiguity (MSW: start() docs).
undici's MockAgent (the interceptor underlying Node's built-in fetch) has mockAgent.disableNetConnect(), which throws "... was not allowed (net.connect disabled)" for any non-matching request, with a corresponding enableNetConnect() to scope exceptions to specific origins (undici MockAgent docs).

Python — pytest-socket, responses, VCR.py.

pytest-socket disables the socket layer itself (pytest --disable-socket, or addopts = --disable-socket in pytest.ini), so any library using sockets — not just a specific HTTP client — raises SocketBlockedError on first attempted connection. It supports per-test overrides (--force-enable-socket, @pytest.mark.allow_hosts()) (pytest-socket GitHub). Its role should be named precisely: it is a fail-closed blocker, not a recorder — it guarantees an attempted connection errors, but keeps no journal of attempts, so pair it with an interception-layer ledger (or an OS-level egress log) when the assertion is "zero attempts were made" rather than "no attempt succeeded".
responses (Sentry) mocks the requests library specifically and is also a native all-attempt ledger: in its unmatched branch it records the rejected request into responses.calls before raising ConnectionError, so assert len(responses.calls) == 0 holds even when the application swallows the error (responses source: unmatched branch). Separately, RequestsMock(assert_all_requests_are_fired=True) raises if any registered mock was not called — the "expected call actually happened" half of the assertion pair (responses GitHub).
VCR.py's record_mode='none' "replays previously recorded interactions and causes an error to be raised for any new requests" — an explicit assert-no-new-egress mode intended for CI so tests are "completely isolated and deterministic" (vcrpy docs: Usage).

Ruby — WebMock. WebMock.disable_net_connect! blocks all outbound HTTP by default and requires an explicit :allow allowlist (string, regex, or lambda) for anything permitted through; RSpec suites commonly wrap this in an around hook scoped per test (WebMock GitHub). WebMock is likewise a native ledger: its HTTP-library adapters register every request signature in RequestRegistry before stub lookup, passthrough, or the NetConnectNotAllowedError raise, and the public a_request(...) / have_been_made matchers read that registry — so zero-attempt assertions work without custom recording stubs (WebMock source: Net::HTTP adapter).

Go — http.RoundTripper replacement / httptest. Go has no single dominant mocking library because the standard library already exposes the seam: http.Client.Transport is an http.RoundTripper interface, so tests substitute a custom RoundTripper that records every request it sees (and can be made to error on unrecognized ones), or point the client at an in-process httptest.Server. Both patterns let a test assert "zero calls to the real transport" directly, since the record-keeping is just application code inside the fake transport (How to Test HTTP Outbound in Go Using Just the Standard Library).

Java — WireMock. WireMock's verification API supports verify(0, getRequestedFor(urlEqualTo("/path"))) to assert a specific request was never made, and a dedicated "verify zero interactions" capability (modeled explicitly on Mockito's verifyZeroInteractions()) to assert no interaction occurred with the mock server at all — useful, per WireMock's own docs, for confirming "a client's app doesn't involve any kind of network interaction when it finds data in local cache" — precisely the offline/cache-hit scenario in the motivating incident. WireMock also keeps a true request journal: every received request, including unmatched ones, is recorded and queryable (findUnmatchedRequests(), GET /__admin/requests/unmatched), making it a genuine ledger rather than only a blocker (WireMock: Verifying, WireMock zero-interactions PR).

Across these ecosystems, three distinct roles keep getting conflated, and a zero-egress test needs them separated explicitly: (a) fail-closed blockers (pytest-socket, disableNetConnect()-style throws) guarantee an attempted call errors instead of escaping — but the error is delivered into the application, where a catch-all can swallow it; (b) ledgers/journals (nock's no match event, a custom MSW onUnhandledRequest callback that records before erroring, WireMock's request journal, a recording Go RoundTripper) retain every attempted call including rejected, unmatched ones, at the interception layer the application cannot reach; (c) expected-mock-consumption assertions (isDone(), assert_all_requests_are_fired=True) verify registered mocks were used — and are vacuously green in a test that registers none. A trustworthy "makes zero calls" test combines (a) and (b): block and journal, then assert the journal is empty. Concretely, in Node:

const attempts = [];
nock.emitter.on('no match', (req) => attempts.push(req));
nock.disableNetConnect();
await runOfflineInstall();               // even if it swallows NetConnectNotAllowedError...
assert.strictEqual(attempts.length, 0);  // ...the attempt was journaled, and this fails

The blocker alone can reproduce the false-green mechanism (swallowed error, green test); the interception-layer journal assertion is what actually goes red when a swallowed attempt occurred.

OS- and Sandbox-Level Enforcement

When you don't trust the application layer to honor a language-level mock (untrusted or partially-audited code, native extensions, subprocess-spawned tools), the enforcement point moves down to the OS.

Linux network namespaces / unshare --net: creates a process (or process tree) with its own network stack containing only a down loopback interface — no route to any external network exists at the kernel level, so there is nothing for a swallowed exception to "quietly work around" (Sandboxed network interfaces using unshare, Restricting network access using Linux Network Namespaces).
Docker --network=none: same idea at the container level — "only the loopback device created within the container... not connected to the outside world," commonly used to give CI a hard guarantee that a build/test stage has zero egress rather than a policy that egress should be zero (Docker: None network driver).
Firejail --net=none: a SUID sandbox combining namespaces, seccomp-bpf, and capability dropping; --net=none "denies network access to programs that don't really need network access," with the caveat that some applications crash outright rather than degrade gracefully when the network is absent this way — which is itself informative (it demonstrates the app was not actually tolerant of "no network," it was tolerant only of "network present but request failed") (Firejail network restrictions, Firejail manpage).
seccomp / eBPF egress observation: rather than blocking, kernel-level hooks can observe and log every socket/connect syscall for later assertion — closer in spirit to the "recording marker" pattern than to a hard block. GitHub's step-security/harden-runner installs eBPF hooks before any workflow step executes so "no network call can bypass egress controls," supports an audit mode that logs without blocking (useful for building the initial allowlist), and maintains anomaly detection against a learned baseline of past outbound connections — a CI-native implementation of record-then-assert (harden-runner GitHub, Unified Network Egress View, StepSecurity).
Supply-chain motivation: this machinery is not academic. Dependency-confusion attacks (publishing a higher-versioned public package with the same name as an internal one) and malicious npm/PyPI lifecycle scripts routinely exfiltrate CI secrets and source code during install/build, before the application under test even runs — which is why hardened CI runners restrict build-time egress to known package registries and block everything else by default (Detecting and preventing dependency confusion, Snyk, NPM Supply Chain Attacks Explained).

OS-level enforcement and language-level mocking are complementary, not competing: the mock gives you a fast, precise, in-process ledger of what was called with what arguments (good for asserting a specific offline code path); the sandbox gives you a coarse but unconditional guarantee that nothing gets out even if the mock is bypassed, misconfigured, or the code under test spawns a subprocess the mock never sees.

Positive Controls, Known-Bad Mutants, and Test Epistemology

The core epistemological move in the motivating incident — "revert the fix and confirm the test goes red" — is a positive control, in the experimental-science sense: a sample known to contain the condition under detection, run through the apparatus to prove the detector can actually raise the alarm. NIST's glossary makes the polarity explicit: a positive control is a known sample expected to produce a positive response, while a negative control is one from which no response is expected — the latter guards against false positives, not false negatives (NIST: positive control, NIST: negative control). Reintroducing a bug that must turn the test red is therefore a positive control — in software terms, a known-bad mutant. In software testing this maps directly onto mutation testing: a tool systematically introduces small code changes ("mutants" — flipping a fallback branch, deleting an early return, changing a comparison) and reruns the test suite; a mutant that survives (no test fails) is proof that the suite cannot detect that class of bug, regardless of how many tests are green (Mutation testing, Wikipedia). The Journal of Engineering paper "Mutation Testing Approach to Negative Testing" frames this explicitly: "an effective test case should pass when executed against the original program, but fail against a mutant... if no test fails, the test suite is not able to detect the mutant" (Strug, 2016).

This gives a concrete heuristic teams can apply without adopting a full mutation-testing pipeline: for any test asserting a negative property ("no network call," "no write to X," "no retry beyond N," "the fallback never triggers in the happy path"), explicitly ask "what mutant would this test catch?" and run that mutant once, by hand, as part of writing the test. If reverting the fix (or reintroducing the bug) does not turn the test red, the test is not verifying the property — it is verifying something correlated with the property, which is exactly the gap that let the poisoned-network test pass in the original incident. This is the same idea behind the "fail fast" principle in the testing literature (errors should surface immediately and loudly rather than be absorbed and continue) and behind classic anti-pattern catalogs warning against tests with "missing fail conditions" — a test that "might pass silently" if an expected exception isn't thrown, or if a contract violation produces no observable symptom (Fail Fast principle, Enterprise Craftsmanship; Code smell: missing test wrong assert).

Generalized false-confidence patterns worth naming explicitly when reviewing "proves absence" tests: catch-and-continue (the incident's root cause); retry loops that eventually give up silently, masking the fact that N failed attempts occurred; feature flags/env checks that default to the "safe" branch in test environments regardless of the code path under test, making the test pass for the wrong reason; and mocks that return plausible defaults instead of erroring, which look identical to "no call happened" in the test's assertions.

Implications for AI Agent Systems

The pattern maps onto AI agent tooling with unusual directness, because agent frameworks have exactly the same structural risk as the original "offline install" feature: a promise of bounded behavior (no network egress, no live LLM calls in tests, air-gapped operation) implemented on top of code with fallback paths (retries, default providers, silent re-authentication, tool-call error handlers) that can mask whether the bounded path was actually exercised.

Deterministic agent test harnesses. The VCR-cassette pattern (record real interactions once, replay deterministically thereafter, error on anything unrecorded) has been ported directly to LLM API testing: pytest-recording wraps VCR.py as a pytest fixture/decorator so agent tests "run once with live LLM API calls and subsequent test runs replay the recorded responses instead of making live calls" — with VCR's record_mode='none' giving the same assert-no-new-calls guarantee used for ordinary HTTP APIs (Eliminating Flaky Tests: Using VCR tests for LLMs). A more sophisticated variant, langchain-replay, records not the raw HTTP bytes but the agent's decisions (which tool was invoked, with what arguments, what text was returned) and replays those decisions while still executing the real tool code — deliberately keeping tool-execution code paths live under test while making the LLM call itself deterministic and network-free (sixty-north/langchain-replay, Deterministic Testing for LangChain Agents). LangChain's own integration-testing docs and community practice converge on the same rule surfaced in the ecosystem survey above: "mock at the boundary — the API call — not deep inside the framework," so the mock's ledger is unambiguous (LangChain: Integration testing).

Egress policy for autonomous agents. Because agents execute tool calls and, increasingly, generated code, industry guidance for 2025–2026 has shifted from "sandbox the agent" toward "sandbox the agent's network" as a distinct control: egress filtering that "blocks all outbound connections by default and whitelists only required API endpoints," with agents operating "under an explicit allowlist of destinations they are permitted to reach" (The Outbound Sandbox, DEV Community; Northflank: How to sandbox AI agents in 2026). The OWASP Top 10 for Agentic Applications (December 2025) formalizes this as ASI05: Unexpected Code Execution — agents generating or running code that attackers can leverage to bypass traditional controls — and its guidance is unambiguous: "Never execute agent-generated code without strict sandboxing, input validation, and allowlisting" (OWASP Top 10 for Agentic Applications). A real 2026 incident cited in this literature — an OpenAI evaluation agent reaching Hugging Face's production infrastructure because its sandbox had open internet access — is the agent-era analogue of a build script silently phoning home: the failure was not that the agent tried to misbehave, it's that nothing structurally prevented the call, and nothing was asserting its absence.

"No un-mocked LLM/API call" as a first-class test assertion. The same two-part discipline applies directly: (1) run agent tests behind a strict interceptor — VCR record_mode='none', or an OS-level egress block for the whole test process — so any un-cassetted call throws instead of silently hitting a real (possibly billed, possibly non-deterministic) endpoint; (2) assert the recorded-call ledger matches expectations exactly (zero calls for a "should short-circuit from cache" test, exactly the expected tool-call sequence for a scripted scenario) and run the positive control — deliberately break the cache-hit / short-circuit logic (a known-bad mutant) and confirm the previously-passing test now fails on an unexpected network entry in the ledger. Static/live "network kill switch" tests (poisoning DNS or the LLM SDK's HTTP client) remain valuable for verifying an agent's resilience to API outages, but — exactly as in the original incident — they cannot substitute for a recording-and-asserting test when the property under test is "this path makes no calls," because a resilient agent's fallback is exactly the thing that will make a poisoned-network test pass for the wrong reason.

Practical Playbook

A checklist for teams asserting "this code makes zero network calls" (or "zero un-mocked LLM calls," "zero writes to X," or any similar negative property):

Never test a negative property by breaking the dependency. Failure-injection (poison curl, blackhole DNS, revoke credentials) tests resilience, not absence. Use it for resilience tests explicitly labeled as such; do not reuse it to claim "offline mode makes no calls."
Install a fail-closed blocker. Any unmatched call must error instead of escaping: nock.disableNetConnect() / a custom MSW onUnhandledRequest / undici MockAgent.disableNetConnect() (JS); pytest-socket --disable-socket or VCR.py record_mode='none' (Python); WebMock.disable_net_connect! (Ruby); a fake http.RoundTripper that rejects unknown requests (Go); WireMock as the only reachable endpoint (Java). A blocker guarantees no attempt succeeds; it does not by itself prove no attempt happened — its error is delivered into the application, where a catch-all can swallow it. That proof is step 3's job.
Pair the blocker with a concrete all-attempt ledger, and assert on the ledger. The ledger must be populated at the interception layer, out of the application's reach, because a blocker error the application swallows never reaches the test. Where that ledger comes from differs by ecosystem, and some blockers have none in-process: JS/nock — subscribe to the global no match event; JS/MSW — the custom onUnhandledRequest(request) callback journals before raising; JS/undici — MockAgent documents no all-attempt journal for blocked requests, so record via a wrapping dispatcher or an OS-level layer (step 5); Python — responses journals the rejected attempt into responses.calls before raising, so assert len(responses.calls) == 0 is a native in-process ledger assertion; pytest-socket and VCR.py record_mode='none' are blocker-only (the rejected request rides the raised exception but lands in no journal), so when one of them is the only interception layer, supply the ledger with a hand-rolled recording transport/adapter or an OS-level egress log (step 5); Ruby/WebMock — a native ledger: every request signature enters RequestRegistry before stub lookup or the NetConnectNotAllowedError raise, and the a_request(...) / have_been_made matchers read it, so zero-attempt assertions need no custom recording stub; Go — the fake RoundTripper is the ledger (append every request it sees before rejecting); Java/WireMock — the request journal records all received requests including unmatched ones (findUnmatchedRequests()). Zero-call tests then assert the ledger is empty, as a first-class assertion. Expected-mock-consumption helpers (isDone(), assert_all_requests_are_fired=True) are a third, different mechanism: they catch expected calls that never fired, and are vacuously green in a test that registers no mocks.
Run a positive control (known-bad mutant) before trusting the test. Temporarily reintroduce the bug (revert the fix, remove the cache-hit short-circuit, force the fallback branch) and confirm the test goes red. If it doesn't, the test isn't testing what you think it's testing. Do this once at authoring time; consider codifying it as a mutation-testing target if the property is safety-critical.
Add OS-level enforcement as defense in depth for untrusted or opaque code. For subprocess-spawned tools, native extensions, or agent-generated code that a language-level mock can't see, layer unshare --net, --network=none, or eBPF-based egress monitoring (e.g. harden-runner-style audit-then-block) underneath the application-level mock.
Distinguish audit mode from enforce mode during rollout. Start new egress policies in observe/log-only mode to build an accurate allowlist baseline before flipping to fail-closed, mirroring how harden-runner and similar CI egress tools stage adoption.
For agent/LLM code specifically: cassette or record-mode the model/API boundary (not deep framework internals), assert the recorded interaction count for offline/cached paths, and treat any agent capable of running generated code or spawning subprocesses as requiring OS-level egress allowlisting in addition to SDK-level mocking — SDK mocks do not see network calls made outside the SDK.
Treat "can this check ever fail?" as a required question in code review for any test making a negative/absence claim. If the answer isn't demonstrable in under a minute (comment the fix out, watch it go red), the test needs rework before it can be trusted as evidence.

Key Takeaways

Failure-injection tests resilience; recording-and-asserting tests absence. They are not interchangeable, and using the former to justify the latter is the specific failure mode that produced a false-positive "zero network calls" claim.
Fail-closed blocking is convergent design across nock, MSW, undici, pytest-socket, responses, VCR.py, WebMock, Go RoundTripper doubles, and WireMock — but the full blocker-plus-ledger combination is an ecosystem-level composition, not a per-tool capability: a documented all-attempt journal exists in several stacks (nock's no match event, MSW's custom callback, responses.calls, WebMock's RequestRegistry, WireMock's request journal, a hand-rolled RoundTripper), while blocker-only tools (pytest-socket, VCR.py record_mode='none', undici's MockAgent) need the ledger supplied by an added layer (recording transport or OS-level egress log).
Hermetic build/test systems (Bazel, Nix, Google's hermetic servers) achieve the same guarantee structurally, by making the ambient environment absent or fully declared so undeclared dependencies fail loudly rather than silently succeed.
OS-level enforcement (network namespaces, --network=none, seccomp/eBPF egress monitoring) is the right layer when application-level mocks can't see everything — untrusted code, subprocesses, or agent-generated commands.
A positive control — deliberately breaking the property (a known-bad mutant) and confirming the test detects it — is not optional polish; it is the only thing that distinguishes "this test verifies X" from "this test happens to pass when X holds." Mutation testing formalizes the same check at scale.
AI agent systems inherit this problem directly: VCR-for-LLM tooling (pytest-recording, langchain-replay) and OS-level egress allowlisting are both necessary, because SDK-level cassettes don't see calls made outside the SDK, and OS-level blocks don't tell you which code path attempted to call.