Claude Gains More From Reasoning
7 stories · ~7 min read

Listen
If You Only Read One Thing
An agent can spend more and learn less. Surge's coding experiment found that extra reasoning beat its tested planning-and-review additions, while Anthropic's memory experiments show why saving more information can make future work worse. The shared question is where extra computation changes a decision. Another reviewer or another remembered fact can consume the budget without correcting the mistake.
Claude Gains More From Extra Reasoning Than Tested Add-ons
More reasoning improved Claude's engineering judgment more than the tested extra skills and reviewers did. Surge compared setups on 100 software-engineering tasks using Opus 5.5. Raising vanilla Claude Code from medium to xhigh effort added 11.3 points; the strongest workflow additions added roughly two to three. These are points on expert grading criteria, rather than a count of successfully completed tasks.
That gain wasn't cheap. Xhigh cost 3.5 times as much as medium. Claude Code's combined goal and code-review plugins gained 3.3 points at 1.8 times the cost. The cheaper setup may suit a constrained budget; the larger quality gain may justify spending more on a difficult task.
Surge's sudo L7 benchmark asks agents to question risky requests and investigate surrounding behavior, alongside making code work. That changes what extra reasoning can buy. A request may be easy to implement yet wrong for the product; repeatedly checking the implementation won't necessarily expose that problem.
The expensive example involved emailing a Zoom passcode to event registrants. A setup with 37 added skills spawned 17 agents, but nobody caught that the email preceded payment. Unpaid registrants would receive access. Across the experiment, Compound Engineering cost eight times as much without a statistically clear improvement over vanilla.
Reviewers can share a blind spot when they inherit the same interpretation of the task. Dividing a mistaken plan into feasibility, security and code reviews doesn't guarantee that anyone asks whether the plan serves the user. Independent review needs room to challenge the premise, alongside checking the patch.
This puts a price on a familiar architectural choice. A single agent given more time to trace consequences can outperform a larger organization of agents following procedures. For assumption-heavy work, reasoning effort deserves its own comparison before orchestration gets credit for improvement.
Surge tested one model and particular configurations, with the harness comparisons at medium effort. It hasn't tested the combinations at equal time or dollar budgets, or xhigh with the plugins. The result supports a simpler comparison, not a universal high-effort default. The conclusion would weaken if matched-budget runs showed the added reviewers delivering better rubric scores than extra reasoning alone.
Anthropic's Memory Works Better With Source Search
Anthropic's memory experiments favor organized notes that keep access to their sources. Over an internal document corpus, a wiki plus search met 61% of graded criteria, versus 53% for search alone. Opus 5 built the wiki. Replacing search with the wiki would discard part of the advantage.
The November 6 engineering report examines memory across sessions, rather than longer conversations or model training. In a database task, each fresh session could consult a persistent memory directory. Opus 5.5's mean reward rose from 0.655 without memory to 0.838 with it. Agents received correct answers after questions, giving them feedback from which to form and test rules.
That feedback is essential to the interpretation. A folder of plausible conclusions isn't the same intervention as a folder of rules checked against outcomes. Persistent memory can spread a mistaken explanation just as efficiently as a correct one. The value comes from improving what the next session believes.
Anthropic also describes offline consolidation, called dreaming: the agent revisits transcripts, checks notes and reorganizes them between active sessions. The extra work can happen after the immediate task, when preserving every detail no longer competes directly with finishing it.
Learning systems need more than compressed history. A correction pass can ask which observation supported a rule and whether the note accurately represents it. A concise instruction is still dangerous if it distills the wrong lesson. Source access allows revision when another task exposes the error.
The tradeoff is maintenance. These are Anthropic-run evaluations, and the document experiment used only 25 questions. Building and updating a wiki consumes compute; on tasks answered from one document, the report found no clear gain over search. A memory system can become another expensive structure to navigate.
Anthropic's Dreams API makes that revision reviewable: it produces a separate output store while preserving the inputs. That allows a rebuilt memory to be inspected before future sessions use it. It also separates successful consolidation from merely finishing a background job.
An agent can therefore try revised notes without losing the originals, then compare the answers each version produces. This goes beyond the durable task state discussed on November 2. State records what happened; learned rules propose what will work again. Revised notes would earn their maintenance cost if they improved answers to unseen questions across several documents without adding claims the originals contradict.
The Contrarian Take
Everyone says: More review makes an agent's work more trustworthy.
Here's why that's wrong: Review only helps when it can discover something the original process missed. Surge's 17-agent example left the payment assumption intact, despite separate planning and code reviews. Another model can produce a fresh answer while accepting the same flawed premise. The scarce contribution is a different line of inquiry: whether the email should be sent at that point at all. Counting approvals rewards activity even when nobody tests the assumption that determines whether the feature is safe.
Under the Radar
-
Uber calibrates guardrails against production traffic. Its November 6 engineering account describes shadow deployment and replay to tune false blocks and false allows before enforcement. Narrowly defined intents reduced opportunities for manipulated actions, according to Uber, at the cost of flexibility. This is an operator account, without a published controlled prevention rate. The useful extension is testing security changes against legitimate behavior too: a guardrail that prevents the intended task can quietly drive users toward less protected paths.
-
Botpress’s support access exposed customer secrets. Its November 6 disclosure says an attacker exploited an internal sign-in proxy, obtained a staff API credential, and read bot configurations in 16 customer workspaces. Those configurations can contain model-provider keys and other integration secrets. Botpress can’t determine which credentials were valid and tells affected customers to treat them as exposed. A secure agent still inherits the permissions of the platform’s support tools.
Quick Takes
-
Meta brings Wonder-trained behavior to everyday models. Specialized versions of existing models are rolling into Muse and Meta AI with memory that retrieves relevant earlier information and behavior trained to persist through obstacles. Meta reports correct use of critical context rising from 76% to 90% on its memory evaluation. That's a vendor result, but the availability change matters: longer-running personal tasks get improved behavior without requiring users to select the largest frontier model. General coding superiority isn't demonstrated. (Source)
-
Mercury's speed doesn't win Plotly's cost comparison. Plotly's November 6 test gives Mercury 3 a 9.3-second median per analytics question, with 35 of 43 passed at $0.41 list-price cost for the exam. GPT-6 Luna passed 38 for $0.13. One run per model limits the ranking, but the choice is concrete: Mercury buys lower waiting time, while Luna produced more correct answers for less money in this harness. (Source)
-
ARTEX's operators expose their own infrastructure. Censys counted 1,465 exposed-console hosts, including about 650 in two identically configured hosting blocks; roughly 815 deployments looked distinct. This adds deployment evidence to October 31's coordination story. Exposed attack infrastructure creates opportunities for misuse and investigation, but host counts can't measure malicious operators or successful intrusions. Authorized testers also use the tool. (Source)
The Thread
Agent systems are accumulating procedures faster than they can prove which ones help. That creates a replacement problem: adding a reviewer, memory rule or guardrail is easier than showing that an older component can be removed. Surge's plain-install comparison gives complexity a measurable alternative. Anthropic's separate output store gives learned knowledge a reversible replacement path. Together they suggest a different standard for improving an agent system: each addition should identify which existing work it makes unnecessary. Otherwise every lesson becomes a permanent cost, and improvements accumulate into a system nobody can afford to simplify. A successful experiment can justify deleting machinery as well as adding it.
Predictions
New predictions:
- I predict: By December 7, Surge will publish a follow-up to its harness experiment testing xhigh effort combined with at least one of the workflow additions it evaluated. Surge explicitly names that combination as an unrun experiment; this forecasts a published test, not a winning result. (Confidence: low; Check by: 2026-12-08)
2026-11-07 · 03:30 AM ET
Tomorrow morning in your inbox.
Subscribe for free. 10-minute read, every weekday.