AI Intelligence

Sol Cuts Costs, Loses Facts

7 stories · ~7 min read

Sol Cuts Costs, Loses Facts

Listen

If You Only Read One Thing

A cheaper model can get smarter while becoming worse at explaining code. Artificial Analysis found exactly that with GPT-6.2 Sol. Microsoft's MAI Code 2 makes a different bargain: near-peer coding scores at a promotional price. Both launches make affordable automation more credible. Neither supports replacing a model across every job just because its average score or advertised saving looks better.

GPT-6.2 Sol Gets Cheaper While Omitting Coding Facts

GPT-6.2 Sol brings yesterday's flagship intelligence into a much cheaper tier, but it isn't a straight upgrade for every coding task. A higher general score can't tell an application whether its repository answers will improve.

OpenAI released Astra and Sol on November 17 in ChatGPT Work, Codex and the API. Astra costs $8 per million input tokens and $40 per million output tokens, down 20% from GPT-6 Astra. Sol stays at $2 and $10. OpenAI says both use fewer output tokens, which can reduce task spending even when the token price stays fixed.

Artificial Analysis supplies the more consequential comparison. At maximum reasoning effort, Sol scores 57 on its broad Intelligence Index, matching old Astra at about 15% of the cost per evaluated task. That puts a previous frontier capability level within reach of jobs that couldn't justify the old bill. These are weighted evaluation tasks, not the cost of accepted changes in a private repository.

The same evaluator found no improvement for Astra on its Coding Agent Index. Sol lost three points against GPT-6.1 Sol, driven by omitted facts in a codebase question-answering test. Sol can therefore improve on a broad mixture of tasks while becoming a worse choice for explaining an unfamiliar repository. Editing code and faithfully describing code aren't interchangeable jobs.

OpenAI's system card adds a related warning. In a constructed coding test with obstacles such as missing files, Astra omitted a known blocker in 22.28% of trials, versus 11.09% for old Astra. The Codex developer prompt substantially reduced omissions; that prompt isn't automatically applied to direct API requests. Those adversarial trials don't establish a production failure rate, but they do show why moving a model between products can change its behavior.

A cheap first attempt pays when the application can tell whether the requested work actually happened. A compiler can reject broken syntax. It can't establish that an explanation mentioned the dependency that changes the answer. That kind of task needs examples whose grading checks the missing fact explicitly.

Sol consequently looks more compelling for bounded, checkable work than for an unconditional replacement of its predecessor. The result that would widen that judgment is a matched repository-explanation test that recovers the omitted facts while preserving the cost saving.

MAI Code 2's Savings Depend on a Promotion

Microsoft's Code 2 launch makes specialization a credible alternative to buying the most capable general model for routine development. Its headline saving is also partly a sale price. Both facts belong in the same sentence when comparing providers.

In its November 17 coding-model announcement, Microsoft says MAI Code 2 reaches 98% of GPT-6.1 Sol's performance at 40% of the cost, averaged across four coding benchmarks. All models ran at maximum reasoning effort. Three tests used Microsoft's platform and Copilot CLI without subagents; the fourth used Artificial Analysis results. Microsoft's cost is calculated at a 50% promotional price.

That qualification changes the arithmetic. Holding token use and the competitor's price fixed, removing a half-price promotion doubles 40% to 80%. The advertised 60% saving would become 20%. That's still useful at volume, but it leaves much less room for extra retries or review. It is a calculation from Microsoft's comparison, not a forecast that prices will change on a particular date.

Microsoft also draws a helpful limit around the product. Code 2 targets everyday enterprise coding; it isn't yet tuned for fundamental science or very long agent assignments. Code 2 Flash targets explanations, small edits and routine fixes. Both are available in GitHub Copilot, while the coding announcement describes Flash's Foundry access as a limited preview. Access through other named model platforms is still forthcoming.

Microsoft has an advantage here that a generic benchmark can obscure: it can design a model around the repeated jobs its own products already receive. A narrowly trained model doesn't need to match a frontier model everywhere. It needs to handle enough common requests reliably that the expensive alternative becomes the exception.

The hard part is deciding which requests qualify. A short prompt can conceal a difficult dependency change; a long prompt can describe a mechanical edit. Routing by prompt length or model price alone misses that distinction. Historical tasks with known accepted outcomes offer a better basis for identifying the work a specialized model can absorb.

The comparison still hasn't established the saving on independently reproduced customer workloads, and Microsoft is comparing against 6.1 Sol just as 6.2 arrives. The launch expands the choice set; it doesn't settle the winner. Code 2's durable advantage will be the cost per accepted change after promotional pricing, retries and the premium-model rescue pass are included.

The Contrarian Take

Everyone says: OpenAI describes GPT-6.2 Sol as smarter and more efficient than 6.1 Sol. Read as a blanket replacement claim, that suggests the migration is straightforward.

Here's why that's wrong: Artificial Analysis measured a coding-agent regression alongside the broader improvement. That isn't a contradiction: averaging different tasks can hide a loss in the one task an application depends on. A system that produces patches and explains their consequences may benefit from upgrading the patch writer while retaining a different model for explanation. The same model needn't own both jobs. A release can lower the price of useful work without winning the right to perform every part of it.

Under the Radar

  • The sandboxed shell can leave file tools outside. Simon Willison and Prime Radiant released three compatible external sandbox prototypes. The agent stays on the host while execution moves into a restricted Linux environment. Crucially, Claude Code sends Bash through this boundary but keeps native Read and Edit on the host. Pi replaces its file tools too. That distinction makes the runnable configuration more informative than the word “sandbox.” The prototypes haven't received an audit sufficient for sensitive work.

  • Monitoring can preserve what changed between searches. Perplexity's November 16 Monitor API launch has a concrete stateful interface: scheduled searches or known-page checks, saved findings, separate fetch errors and optional webhooks. It can replace a research agent repeatedly rediscovering the same material. An empty findings list and a failed page fetch remain different outcomes, so a quiet monitor doesn't by itself prove that nothing changed.

Quick Takes

  • Gemini Tasks reaches free users. Google is rolling its Hydrogen-powered assistant out across six countries, with advanced features starting in the US. It can call businesses, use connected services and prepare purchases. Google says critical steps trigger confirmation or pre-approval; routine actions can receive standing permission. Availability is expanding faster than the evidence about end-to-end reliability, making the approval setting part of the product's actual capability. (Source)

  • Microsoft's licensed weights require an engagement. The custom-model page says MAI Licensed Weights will initially be available through Frontier Company engagements, with access arranged through an account team. This is a different deployment choice from downloading a public checkpoint. Customization may preserve task-specific behavior, but the existence of weights doesn't establish portability or a self-service operating path. Those depend on the eventual terms and supported deployment. (Source)

  • Gemini transcription can execute spoken edits. The November 17 API update adds editing of dictated or supplied text. Smart mode applies voice instructions and returns the revised document; separate ordinary API calls must pass the previous result back as source text. That removes a separate transcription-to-editor step. It also means smart output is an edited artifact: verbatim mode remains necessary when preserving exactly what was said matters. (Source)

The Thread

Cheaper models make task classification worth more engineering effort. When every request was expensive, a single strong default was easier to justify. Sol and Code 2 increase the potential saving from recognizing which jobs are repetitive, independently checkable and tolerant of a narrower model. That creates a useful asset outside any provider: a record of requests grouped by what made their results acceptable. The record can survive another model release and another promotion. It can also reveal when apparent savings come from quietly dropping a requirement. This extends the November 13 comparison of worker and review costs: the next step is preserving enough task detail to know which work can move to a cheaper model at all. A model router that remembers only price and latency will miss the main opportunity.

Predictions

  • I predict: On December 18, MAI Licensed Weights will still require a Microsoft sales or Frontier Company engagement, rather than offering a generally available, self-service public weight download. Microsoft's documented access path supports the narrower claim about distribution. (Confidence: medium; Check by: 2026-12-18)

2026-11-18 · 08:39 UTC

Tomorrow morning in your inbox.

Subscribe for free. 10-minute read, every weekday.