Grok Gains Depend on Fit

If You Only Read One Thing

A model upgrade can improve capability while making an existing agent worse. Grok Needs Its Own Rhythm follows XBOW’s revealing comparison; MiMo Broadens the Cheap Frontier examines Xiaomi’s new open-weight challenger. Together they complicate the habit of swapping model names inside an unchanged application. Better weights expand the available choices, but the surrounding system determines which improvements become useful work.

Grok Needs Its Own Rhythm

Grok 4.7 deserves a fresh model-choice decision, including a fresh look at the agent running it. Keeping every surrounding component fixed can conceal an improvement that depends on how the model acts.

SpaceXAI’s September 21 release is available through its API, Cursor and Grok Build. The company says it trained a larger base model for harder, longer tasks. Starting token prices remain $2 per million input and $6 per million output, matching Grok 4.6.

Independent evidence supports a real advance in coding work. Artificial Analysis puts Grok 4.7 with Grok Build at 56 on its Coding Agent Index, up from 47 for 4.6 at the same extra-high effort setting. That result evaluates the model and its native agent together; it does not establish the same gain inside another product.

XBOW’s early-access evaluation found precisely that distinction. Across three large targets, three workload sizes and four systems, Build-based systems rose from 42 to 68 correct findings on an equal-weight average. But external systems using the old model had scored 72. Improvement inside Build does not establish that switching to Build beats the incumbent system; outside Build, performance declined slightly.

The mechanism is interaction rhythm. Think of an agent as alternating between action and feedback: one system rewards a short command followed by inspection, while another expects a substantial script before checking progress. XBOW observed Grok 4.7 favoring smaller commands. A fixed action limit can penalize that rhythm. Yet XBOW found no obvious token-efficiency advantage either: changing the denominator did not rescue the old setup.

The practical extension to an established multi-agent workflow is a paired comparison: old and new models inside both the incumbent agent and the native one. Task, dollar budget and deadline stay comparable. That separates a weaker model from a poor model–agent fit; adding another reviewer cannot diagnose the distinction.

There is also a useful cheaper setting. CursorBench 4.0 reports Grok 4.7 high at 43.9% and $4.69 per task, versus 46.3% and $6.01 at extra-high. The extra score costs about 28% more. Cursor cautions that small score differences may reflect variance. The cost gap still makes effort a deployment choice, extending August 23’s effort comparison with a new model and measured agent interaction.

The decisive test is whether the gain within the native agent survives on ordinary repository work under the same dollar and time ceilings.

MiMo Broadens the Cheap Frontier

Xiaomi’s MiMo-V2.6-Pro adds a serious open-weight option to the model shortlist. Its significance is the combination of independently measured capability, inexpensive hosted access and downloadable weights, rather than another vendor claiming a benchmark victory.

Artificial Analysis’s launch snapshot gives Pro an Intelligence Index score of 46 and a cost of $0.13 per index task. It ranks first in the open-weight comparison shown on the page. The index combines several kinds of reasoning and practical work; that thirteen-cent figure is an evaluation average, not the price of completing a production pull request.

Xiaomi’s September 22 announcement releases Pro and Flash weights alongside API access, keeping the prior generation’s API pricing. This changes the available alternative today. A team can examine the hosted model before deciding whether operating the weights would justify its own infrastructure.

The ownership option comes with a substantial physical constraint. Pro has roughly a trillion total parameters, with 42 billion active for each token. In a mixture-of-experts model, only selected parts of the network perform each calculation. Think of a large workshop using a small specialist crew for one job: fewer workers are busy, but the workshop still has to exist. Sparse computation reduces work per token without making all the other weights disappear.

The published deployment instructions make that distinction concrete. Xiaomi supplies SGLang and vLLM recipes; its SGLang example spans two nodes and specifies sixteen-way tensor parallelism, splitting the model across devices. The MIT license permits commercial use, but permission to operate a model says nothing about the cost of keeping its serving machinery occupied.

The strongest objection is that broad benchmark leadership may not survive a particular coding workflow. Grok’s result explains why that objection matters. MiMo’s score does not establish tool-call reliability inside an existing agent, and a million-token context window is capacity rather than proof of accurate long-session work.

My read is that the API belongs on a cost-sensitive shortlist now; self-hosting needs a separate utilization case. Low hosted prices can actually make ownership harder to justify, because idle hardware must compete with a service charging only for use. The next useful evidence is a matched hosted-versus-self-hosted workload reporting accepted-task cost, including retries and idle capacity.

The Contrarian Take

Everyone says: A model released at the same token price gives its users a straightforward efficiency upgrade.

Here’s why that’s incomplete: Artificial Analysis measured about 81,000 output tokens per Intelligence Index task for Grok 4.7 extra-high, versus 38,000 for Grok 4.6 extra-high. That is more than twice the output at the same unit rate, although the new model also does better work. Cursor’s coding test points the other way: extra-high 4.7 scores better than extra-high 4.6 while its average task cost falls slightly, from $6.10 to $6.01. These results can coexist because the tasks and surrounding agents differ. “Same price” describes the meter; efficiency depends on how much work the system buys from it. The evidence supports workload-specific substitution, not a universal savings claim.

Under the Radar

Grok’s API carries state the visible answer does not explain. The September 21 release notes say Grok 4.7 always returns encrypted reasoning content through the Responses API, even when the request does not explicitly include it. This is a concrete compatibility condition for clients that assume a requested field list defines the whole response; the announcement establishes the returned data shape, not that every third-party agent preserves it correctly.
An approval wait can outlive the agent session. Elastic’s new reproducible workflow stores execution state while separate agents diagnose and execute a remediation, with approval before action and verification afterward. The useful extension beyond a notification-and-reply loop is durable suspension: work can resume after days without keeping one reasoning session alive, although persistence alone does not establish that the target system is unchanged when approval arrives.

Quick Takes

Jev gets a human-label test, with a five-case boundary. LangChain’s experiment froze five weather-agent runs and judged each 100 times; Jev matched all 500 repeated binary labels from one human reviewer. Unlike September 17’s launch claims, this exposes a reproducible comparison. The method separates judge variation from agent variation, but 500 repetitions do not become 500 distinct cases. It supports testing narrow evaluators, not replacing every semantic reviewer. (Source)
Python agent backends gain an edge deployment route. Cloudflare made Python Workers generally available September 21. Networking changes let libraries including openai, langchain and mcp communicate from the Workers runtime, and a published example joins queues, workflows, inference and storage. The practical change is a Python orchestration path without a separate Python server; it is not evidence that an arbitrary local Python environment will migrate unchanged. (Source)
Xiaomi’s small release is a different deployment proposition. MiMo-V2.6-Distill-Qwen-9B fine-tunes Qwen3.5-9B on MiMo-generated examples, transferring behavior from a larger model into smaller weights. Xiaomi reports SWE Pro rising from 32.0 to 44.6, averaged across three runs, and supplies an SGLang recipe. This is a local-agent research candidate, not the trillion-parameter flagship compressed without loss; the result remains vendor-reported. (Source)

The Thread

Model competition is creating a choice between portability and specialization. A common agent interface makes providers easy to substitute. But if a model learns a particular rhythm of tools, feedback and delegation, that common interface may leave some of its capability unused. Grok’s result makes this tension measurable; MiMo’s open weights make adapting the surrounding system possible without requiring the original provider’s permission. This extends September 21’s focus on preserving behavior through replacement: today’s question is whether preserving the old workflow also preserves its limitations. My inference is that model selection increasingly has two legitimate tracks. One compares drop-in replacements under a shared agent. The other compares complete systems after a bounded adaptation effort. Keeping those tracks distinct reveals whether an improvement comes from the model, integration work, or both. A small team may rationally prefer the weaker drop-in result when the stronger pairing requires ongoing maintenance; a heavily used workflow can amortize that adaptation across many tasks.

Predictions

I predict: Grok 4.7 with Grok Build will remain in the top five of Artificial Analysis’s Coding Agent Index among models evaluated in their native agents on October 6. Its current fourth-place position is supported across the index’s component tasks; a wave of stronger releases is the main falsifier. (Confidence: medium; Check by: 2026-10-06)

Issue date: 2026-09-22. Prepared at 03:39 AM ET.