AI Speeds Anthropic's Research
7 stories · ~7 min read

Listen
If You Only Read One Thing
AI can accelerate its own development before it can replace the people doing it. METR's assessment of Anthropic puts assistance ahead of autonomous research. Kimi K3.1 Pro's released weights make another distinction concrete: access to a capable model doesn't require owning its training, but running it means owning the service. Both developments reward choosing where people and infrastructure still matter.
Anthropic's Research Accelerates Through AI Assistance
Anthropic appears to be improving its models faster because its researchers use AI, without having handed its most important research over to autonomous agents. That's a consequential result for anyone deciding whether AI only pays off once it can finish an entire job alone.
METR's November 5 assessment estimates that Anthropic's internal capability-benchmark scores have improved 1.9 times as fast since October 2025 as before it. METR judges that AI use probably explains most of the acceleration. Staff accounts put assistance ahead of fully autonomous contributions. The number describes a rate of progress, not a model becoming twice as intelligent.
Tuesday's GDPx coverage showed agents struggling to finish demanding professional assignments. These findings can coexist. A researcher can retain responsibility for choosing an experiment and interpreting its result while a model helps with the intervening work. Failure to replace that researcher doesn't prevent a meaningful contribution to the research program.
Humans can connect AI-assisted steps that an autonomous agent still can't complete reliably. An autonomous system has to handle every essential dependency before it can deliver unattended. An assistant can improve one recurring part of a process while a person handles the difficult transitions. For example, faster experiment implementation could let a researcher test more ideas without giving the model authority to decide which conclusions deserve belief. That's a mechanism for acceleration, not a claim that METR isolated that particular contribution.
The attribution is harder to check than the benchmark trend. METR ruled out faster growth in training compute, but didn't find decisive evidence separating all the other causes. Researchers can misjudge their own gains, and Anthropic redacted much of the most informative evidence. This is an independent assessment with unusual access, not a randomized productivity trial.
Counting only fully delegated projects would miss this kind of gain. Accepted experiments per researcher-week would capture whether faster implementation produces more usable research after review. That's a proposed comparison for other teams, not a measured result from METR.
If that comparison showed flat output after including human review, the case for copying this division of work elsewhere would weaken.
Kimi's Released Weights Enable Private Deployment
Kimi K3.1 Pro can now run under somebody else's control. Its released weights and working serving recipes open a private deployment choice for coding and research agents. They don't turn a frontier model into a laptop installation, or make its largest context window cheap to fill.
October 24's briefing covered the capability jump. The change is the downloadable model, with runtime teams publishing serving paths on November 4. AMD describes 2.8 trillion total parameters, of which 104 billion are active per token. Activating a fraction saves computation, but the rest of the weights still need storage.
AMD's recipe runs the compressed checkpoint across eight MI355X accelerators. Its vLLM configuration also reserves about 1.64 TiB of host memory to move stored conversation state off the GPUs. The model's size is only part of the capacity bill; the conversations it keeps in progress consume memory too.
That recipe exposes a trap in reading server settings. It permits a one-million-token context and sets a limit of 48 concurrent sequences. AMD explicitly warns that those settings don't guarantee room for 48 simultaneous million-token requests. Multiplying two advertised limits would overstate what the server promises. For a private research agent, the relevant capacity is how many actual conversations fit together, with their accumulated history.
The serving method also changes how long a person waits. The vLLM team's comparison cuts time to first token from 15.89 to 9.42 seconds for a 340,000-token prompt on eight GB300 GPUs. Dividing prompt processing differently across the GPUs produced the reported 1.69-times speedup. That's a measured configuration change, not a gain for every Kimi request.
The license deserves a more precise reading than “commercial restrictions.” It requires separate terms for model-service businesses above its revenue threshold and branding for certain large commercial products. Those provisions explicitly exempt internal use. A company evaluating private deployment needs that distinction before pricing a licensing obstacle that may not apply.
Private execution can be worth paying extra for when code or data can't leave the organization. The published recipes still don't show lower cost per accepted code change than a hosted frontier model. If idle GPU time, retries and operating labor erase the token savings at actual request volume, self-hosting hasn't won on price, even if it wins on data control.
The Contrarian Take
Everyone says: AI has to become an autonomous researcher before it can materially accelerate AI research.
Here's why that's wrong: METR attributes most current acceleration at Anthropic to AI use while reporting that assistance, rather than fully autonomous contributions, does most of the work. Humans remaining in charge doesn't impose a ceiling of trivial gains. The error is treating a job as indivisible: either the model owns it or it contributes little. A process can move faster because several recurring steps improve, even while its hardest judgments stay with people. METR hasn't established that every research organization will get the same benefit.
Under the Radar
-
A mostly healthy API can still break a time-sensitive workflow. In a November 5 Google developer-forum exchange, a customer reports 90 image-request replays: 88 finished within 14 seconds, one took 115 seconds and one failed after 127 seconds. Google attributes 503 errors to overload. It's one customer's reproduction, not a service-wide failure rate. Average speed would hide the two requests that held the workflow open.
-
Sysdig moves agent protection to execution. Its November 4 announcement makes AI Defense generally available, describing controls that judge and stop agent actions from laptops to cloud systems. Enforcement outside the conversation can constrain an action even when a model wants to proceed. The announcement supplies no independently measured containment rate, so effectiveness still depends on which execution paths the installed controls cover.
Quick Takes
-
GPT-6 Cyber is available to approved customers. OpenAI's November 5 specialist release is classified Critical for cyber capability, following the Linux research covered yesterday. API use requires Daybreak Red approval, project setup and IP allowlisting. Standard short-context input/output prices are $20/$100 per million tokens. Access is a separate decision from ordinary coding-model selection; a subscription alone won't qualify. (Source)
-
Inkling 2 Small is hosted before it's downloadable. Artificial Analysis's November 5 article index identifies it as the highest-scoring prospective open-weight model outside China, pending release. Thinking Machines says inference and fine-tuning are available through Tinker. That makes customization a current hosted option; private serving still depends on the promised weights. An open-weight ranking shouldn't be mistaken for a downloadable checkpoint. (Source)
-
Nanbeige makes local terminal work more competitive. Artificial Analysis's November 5 report puts the open-weight 33B model at 30% on Terminal-Bench 4.0 versus Qwen3.8 27B's 6%, despite equal overall index scores. Terminal-Bench tests work in a command-line environment. That makes Nanbeige a local-agent candidate, while interactive speed on a particular laptop needs its own measurement. (Source)
The Thread
Open weights can spread a model's capabilities without equalizing how quickly organizations improve them. Kimi's checkpoint transfers the result of training; it doesn't transfer the research process that produced it. METR's assessment suggests that process can itself get faster through AI assistance. Put those together and catching up requires more than downloading today's strong model: the next target may be arriving sooner. A company using open weights can still benefit enormously without trying to match a frontier lab. But evidence that a model is widely available says little about whether the capability gap is closing. The revealing comparison is how quickly each side turns its current models into better successors, rather than how many copies of the current checkpoint exist.
Predictions
- I predict: By November 30, a commercial inference host other than Moonshot will publish a new Kimi K3.1 Pro serving benchmark naming its hardware, input/output lengths and concurrency. Released weights and multiple runtime implementations give that comparison a concrete starting point. (Confidence: medium; Check by: 2026-12-01)
Coming Next Week
We'll examine where AI assistance beats full delegation in research and engineering. The comparison will focus on completed work and human time, rather than how long an agent can run alone.
Issue date: 2026-11-06 · Generated 04:10 AM ET
Tomorrow morning in your inbox.
Subscribe for free. 10-minute read, every weekday.