AI Intelligence

Inkling Runs Agents On Laptops

7 stories · ~7 min read

Inkling Runs Agents On Laptops

Listen

If You Only Read One Thing

More tokens needn't mean more expensive work. Inkling 2 Mini's independent evaluation shows how a small model can spend more tokens yet cost less per task. GPT-6.2 reverses that bargain: repeated instructions can cost more even when requests keep working. Both stories turn on what the system processes again, something a token count alone can't tell us.

Inkling Matches Qwen With Less Active Compute

Inkling 2 Mini makes local agent work more credible without claiming a frontier-model replacement. Thinking Machines released the weights on November 20, and Ollama provides a roughly 20GB version for Macs with at least 32GB of memory. That puts the model within reach of an ordinary development machine rather than a dedicated server.

Artificial Analysis scores Mini alongside Qwen3.8 27B at its highest reasoning setting. Its tested service costs $0.59 per evaluation task against Qwen's $3.53. Those are weighted benchmark tasks using hosted services and prices; they don't measure laptop speed or successful private-repository fixes.

Mini gets there by reusing part of its network. Twelve of its 24 layers run three times for each generated token. Instead of storing an entirely larger model, it applies the same middle layers again before producing the next token. This repeat inside the model approach trades extra computation for a smaller collection of stored weights.

Repeated layers also complicate the size comparison. Mini stores weights that each token doesn't use, then applies some of the selected weights several times. Counting only unique active weights misses those extra passes. Memory must still hold all the stored weights and the growing conversation.

Artificial Analysis found Mini used more output tokens than Qwen for the same overall score, yet estimated less computation overall. It estimated compute by multiplying active parameters by output tokens, rather than measuring electricity or laptop runtime. More words can still be cheaper when each word requires less work.

The attractive job is a bounded local agent with tools and an inspectable result. Ollama's launch path requires its preview build; existing installations aren't automatically ready. Local execution also keeps the inference request on the machine, although an agent's search and shell tools can still reach outside it.

Mini's model card warns about weak factual recall, repetitive debugging and destructive file operations without confirmation. I wouldn't infer dependable unattended coding from the aggregate score. The next comparison that changes its job assignment is local completion time on the same accepted tasks as Qwen, including failed turns and repairs, rather than a faster stream of text.

GPT-6.2 Reprices Stable Agent Instructions

GPT-6.2 can make a working agent more expensive because of where its client puts the system prompt. LiteLLM says customers encountered exactly that after the November 17 release. No error alert fired: the requests succeeded while stable instructions were billed at full input price on every call.

This extends Wednesday's model-cost comparison with an integration cost that a leaderboard can't settle. A long-running agent repeatedly sends project rules, tool guidance and the conversation so far. Prompt caching normally avoids charging the full processing rate for eligible text the service has already seen. But identical words aren't enough; their position in the rendered request matters.

OpenAI now places GPT-6.2's top-level instructions after input, beyond eligible cache breakpoints. That supports guidance that changes for the current turn while leaving earlier material reusable. OpenAI's documentation explicitly says those instruction tokens incur the uncached rate even when repeated. This is documented behavior, although early reports understandably called it a bug.

The change caught a translation layer. LiteLLM had put system prompts in instructions when converting Chat Completions or Anthropic-format traffic into Responses requests. Its incident account says the old placement cached through GPT-6.1. It now sends the reusable prompt as a leading developer message for GPT-6.2, with fixes backported across five release lines. The stable 1.110 line's repair is version 1.110.2.

LiteLLM's repair preserves the system prompt's words while changing where OpenAI receives them. That makes request placement part of the cost calculation, alongside prompt length and model price. Savings still depend on how much of each request repeats and how much the model generates; the incident account doesn't supply a matched total-bill comparison.

The reusable pattern is to separate persistent guidance from turn-specific guidance in the request itself. Stable rules go first as a developer message; changing instructions can remain at the end. An explicit breakpoint can mark the reusable text, and later requests preserve that message. This avoids paying repeatedly for the stable part without requiring a shorter policy file.

There is a behavioral cost to check alongside the accounting. Moving guidance from the end to the beginning changes what surrounds it, and conversation handling must preserve the intended instructions. A warmed replay that restores cache reads but weakens instruction following would be a bad repair. The relevant outcome is both restored reuse and unchanged behavior across the same multi-turn cases.

The Contrarian Take

Everyone says: The cache reports were initially framed as a GPT-6.2 bug, implying that an eventual provider fix would restore the old economics.

Here's why that's wrong: OpenAI documents the placement and billing as intentional. Waiting leaves affected clients paying the uncached rate. LiteLLM changed its request translation because the remedy belongs partly in the integration. The more consequential failure was an alerting gap: successful responses concealed a changed cost contract. A system can remain available while becoming uneconomic for the job it was built to do.

Under the Radar

  • A claimed proof still needs an independently checked claim. Levent Alpöge's November 20 announcement says he and Claude proved the Riemann Hypothesis and formalized the argument, with downloadable Lean files. That's an extraordinary claim, not an independently validated result here. For AI-generated software, the transferable issue is precise: a proof checker can check a formal statement, while people still have to establish that the statement expresses the intended requirement. The announcement merits attention without becoming evidence that ordinary coding agents can now prove arbitrary programs correct.

  • Isolated worktrees still shared a destructive lifecycle. Claude Code 2.1.330 fixes exiting one session deleting a worktree another session was using. It also repairs concurrent saves losing settings, plugins and task metadata. The maintainer changelog describes concrete failures in parallel operation, not measured productivity gains. Separate directories isolate edits; deletion and shared configuration still need coordination across processes. That's a different constraint from merging the resulting code.

Quick Takes

  • Cairn's agents found an exploitable flaw without step-by-step human direction. Amazon's November 20 disclosure reconstructs a September campaign: agents reached arbitrary file writes in about 11 minutes, then used a later session to obtain root access through Python startup code. Other targets resisted lengthy attempts. This is observed offensive capability with a concrete execution chain, rather than a benchmark-only demonstration or a general success rate. (Source)

  • Hy4's full weights change deployment options. Tencent has published instruction, FP8 and base checkpoints after its earlier hosted launch. The main language backbone has 770 billion parameters, with 49 billion active per token; the model card includes vLLM and SGLang deployment paths. Downloadable weights permit a different custody and customization choice, but the Tencent Hy License differs from Apache 2.0. This is server-scale availability, not another laptop alternative to Mini. (Source)

  • An idle voice connection can stop doing inference. OpenAI's November 19 update adds standby for GPT-Live WebRTC sessions. Speech can wake inference, and an idle timeout returns it to standby while retaining the connection. The docs warn that the first short reply after waking may take longer. Voice-controlled coding or research agents can therefore trade response delay against active-session expense without stopping their separate backend worker; no measured whole-workflow saving is published. (Source)

The Thread

Compatibility now needs an economic definition. An agent that returns the same answer through the same endpoint can still have changed how often it processes instructions, how much computation each token represents, or whether a quiet connection keeps inference running. That suggests a useful addition to migration records: the resource behavior the application expects to stay stable. A warmed second request should reuse its long prefix. An idle voice session should enter the chosen standby state. A local model should finish inside the machine's memory and time budget. These are concrete expectations that can fail while ordinary answer-quality tests keep passing. Recording them alongside functional results would make it easier to tell a model improvement from a cost shifted into the surrounding software.

Predictions

  • I predict: By December 21, Ollama's Inkling 2 Mini instructions will offer a stable-release installation path rather than requiring the 0.40.17-rc3 preview. (Confidence: medium; Check by: 2026-12-21)

Issue date: November 21, 2026. Generated at 03:30 AM ET.

Tomorrow morning in your inbox.

Subscribe for free. 10-minute read, every weekday.