Codex Reviews, vLLM Remembers

If You Only Read One Thing

The fastest way to expand an agent may be to move its waiting somewhere cheaper. Codex Delegates the Gate assigns approval prompts to an automatic reviewer; vLLM Puts Memory on Disk sends evicted attention state to NVMe and reports reloads beating its DRAM path. Both spend abundant machine capacity to release scarce human attention or GPU memory.

Codex Delegates the Gate

Approval prompts have been the speed limit on safe coding agents: every request to cross a sandbox boundary stops until a person decides. Codex 0.147 turns that pause into another machine-mediated step.

The August 7 release adds --approve-for-me, which routes escalation requests through automatic review. The implementation is more constrained than the flag sounds. It fixes the approval policy at on-request, keeps the agent in a workspace-write sandbox and conflicts with both a custom sandbox mode and the explicitly dangerous no-approval, no-sandbox option. Think of automatic review as code review for privilege: the working agent proposes an action outside its ordinary lane, a separate decision step judges the proposal, and the sandbox still defines the maximum reachable surface.

Why it matters: This changes the scarce resource in supervised autonomy. Manual approval spends human attention on every exceptional command, including routine package installs or network access; full bypass eliminates the queue by eliminating the check. Codex now offers a third structure in which reviewer compute absorbs the queue while a machine-enforced sandbox limits the cost of a bad decision. The same release reinforces that division of labor by requiring trust for unfamiliar projects and denying plugin network access when policy updates fail. The reviewer handles judgment; the runtime retains the hard stop.

That distinction makes reviewer quality a new capability claim. The merged flag change documents configuration and conflict tests, but it does not publish the reviewer’s model, allow/deny criteria, false-approval rate or behavior under prompt injection. A system can therefore be meaningfully safer than unrestricted execution while still being difficult to evaluate. The evidence that would settle the question is an approval corpus with risky and benign escalations, decision traces and measured false allows across resumed and compacted sessions.

Room for disagreement: Automating the prompt can reduce safety if users mistake reviewer judgment for policy enforcement. The stronger counterargument is that repetitive human prompts already invite reflexive approval or complete bypass; a reviewer inside an enforced sandbox may improve the real choice users make. Codex has shipped the safer shape, but not yet the evidence needed to price its judgment.

vLLM Puts Memory on Disk

Disk is supposed to be the slow tier. In vLLM’s new coding-agent benchmark, its local NVMe path reloaded evicted context more than ten times faster than the existing DRAM offload path.

The state being moved is the key-value cache, the intermediate attention data a model keeps for prior tokens. Think of it as leaving a long book open with every relevant passage marked: retaining the cache lets the next turn continue, while losing it forces the model to reread the conversation. Long agent sessions make those marks consume scarce GPU memory, so offloading moves inactive state elsewhere without discarding it.

The merged vLLM implementation adds a native disk backend with no external storage service. It writes GPU cache blocks through small pinned-memory buffers into preallocated, per-rank files on local NVMe. Direct I/O avoids the operating system’s page cache, while two buffers overlap GPU transfers with disk reads and writes. The scheduler uses the same least-recently-used bookkeeping as CPU offload, so disk becomes another capacity tier rather than a separate serving system.

Why it matters: The self-reported test used an H100, 128GB of host memory, a 1.7TB Solidigm NVMe drive and 610 coding-agent conversations averaging 66.3 turns. Cold performance was effectively equal. On reload after GPU eviction, median time to first token fell from 15.831 seconds on the DRAM path to 1.527 seconds on disk, while throughput rose from 10 to 85 tokens per second. The surprising result is not that NAND is physically faster than DRAM; it is that a pipelined storage path can beat a slower end-to-end memory implementation. Software scheduling, not the medium’s datasheet, determined the experienced latency.

July 28’s briefing described vLLM turning object storage into a secondary cache tier. This change brings the same hierarchy onto one inference node, removes an external dependency and supplies a coding-agent workload with measured reload behavior. Cheap NVMe can now preserve more dormant sessions per GPU, shifting the economic question from maximum context length to how frequently old context is reused and how predictably it can be prefetched.

Room for disagreement: This is one contributor’s benchmark on one machine, one model and a workload with a shared 12,000-token prefix. It does not measure sustained concurrency, drive endurance or performance when disk also serves model weights. Independent tests must separate the new double-buffer implementation from the storage medium before the 10.4× result becomes a general deployment expectation.

The Contrarian Take

Everyone says: Automatic approvals remove the human safety brake, while disk offload is a desperate substitute for buying enough GPU memory.

Here’s why that’s wrong (or at least incomplete): Both claims confuse the resource with the control. Codex retains an on-request policy and workspace sandbox; it changes who reviews exceptions, not the maximum authority available. vLLM’s NVMe result does not prove disk is faster than memory; it shows that overlapped movement can beat an inefficient DRAM path on reload. The durable advantage comes from placing each wait in the cheapest tier that can still meet the deadline.

Under the Radar

A retry can create two bills — Vercel AI SDK 7.0.56 now creates one idempotency key outside its video-generation retry loop, so a lost response does not automatically start another billable job when the gateway retries. The guarantee still depends on the provider honoring that key, which makes idempotency a contract spanning application, gateway and model host.
Eight-bit MLX shapes find the fast lane — An MLX dispatcher fix recognizes that six- and eight-bit matrix-vector kernels need 256-element alignment, not 512. Newly eligible eight-bit shapes ran 18.0% faster at one tested dimension and 10.6% faster at another on an M4 Pro, with unchanged numerical tests. A literal constant had been withholding an existing optimization.

Quick Takes

Agent State Becomes Importable

Codex 0.147 can discover Cursor-managed skills and synchronize changes from imported Claude and Cursor conversations without creating duplicates. The immediate convenience masks a larger portability shift: agent differentiation increasingly lives in accumulated procedures and session state, so the host that imports those assets can lower switching costs while becoming the new system of record. (Source)

MLX Links Statically on Windows

MLX fixed its installed Windows package so downstream CMake projects can link mlx.lib without requiring mlx.dll. The patch reconstructs CUDA, cuDNN and OpenBLAS dependencies and was manually verified against a C++ inference project. This is build-system work, but it widens the route from a research framework checkout to a redistributable native inference application. (Source)

Inspect Preserves Container Lifecycles

Inspect AI now accepts Docker Compose’s restart, stdin_open and tty fields and passes them through to sandbox providers. Those fields previously caused some evaluation tasks to fail during construction. The change matters because an eval environment’s lifecycle is part of the task: rejecting or silently dropping it can alter whether a service survives, accepts input or behaves like the system being measured. (Source)

The Thread

The shared idea is queue placement. A person waiting on every privilege request constrains agent throughput; GPU memory holding every dormant conversation constrains session density. Codex inserts reviewer compute before the human queue, while vLLM inserts local storage behind the GPU queue. Neither substitute is free: the reviewer can authorize badly, and NVMe can miss a latency target. But once the expensive resource stops handling every case, system performance depends on how accurately work is routed to the cheaper tier and how cleanly exceptions return.

Predictions

New predictions:

I predict: By October 31, 2026, at least one production coding-agent runtime other than Codex will release a named automatic approval-review mode that retains an enforced sandbox. This is wrong if no public release note names both automated review and a sandboxed execution boundary by the check date. (Confidence: medium; Check by: 2026-10-31)

Coming Next Week

Next week, I’ll examine whether automatic approval reviewers are becoming a distinct agent model class, and which eval would distinguish useful delegation from prompt-fatigue automation.

Issue date: 2026-08-07 · Generated: 2026-08-07 03:49 AM EDT