Smaller Models, Bigger Systems

If You Only Read One Thing

An 8-billion-parameter model and a four-worker research team now compete on the same axis: how much useful computation survives outside a frontier checkpoint. Granite Makes 8B Useful compresses tool behavior into smaller weights; Apodex Makes the Loop Larger expands one model through parallel search and verification. The model is no longer the whole product, but it still sets the ceiling.

Granite Makes 8B Useful

Granite 4.2 does not put IBM at the model frontier. It makes an 8-billion-parameter checkpoint credible for local agent work and exposes why the 30B sibling is not automatically better.

IBM released three dense models on August 25 at 3B, 8B, and 30B parameters. All carry Apache 2.0 terms and 128,000-token native context; the 30B advertises a 512,000-token extension. Weights run through Hugging Face, Ollama, OpenRouter, and IBM’s API.

The important change sits in training. Granite 4.1 mainly learned to follow instructions. Granite 4.2 adds agentic reinforcement learning, or practice inside tool-using environments where code execution, terminal work, and search produce checkable outcomes. The 8B and 30B models practiced choosing tools and recovering across steps instead of learning only from static examples. IBM also reports 15 trillion pretraining tokens and one trillion tokens of synthetic code in the technical release.

Independent evidence makes the 8B more interesting than the family headline. Artificial Analysis scores it at 20, versus 24 for the 30B, but measures the 8B at 145.1 output tokens per second and prices it at $0.06 per million input tokens and $0.25 per million output tokens on IBM’s API. A separate six-model local test found the 8B passed all nine coding tasks, while the 30B passed seven. The smaller Granite also sustained its full 131,072-token tested context; the 30B fell below 90% of baseline speed after 58,368 tokens.

That test is too small to crown an 8B winner. Granite 8B used a median 11,947 tokens on the coding set, nearly four times Gemma 4 31B’s 3,168, and its intelligence score remains far below frontier APIs. Dense attention also creates a large memory bill: the local test measured 160 kilobytes of cache per context token for the 8B.

The payoff is a better model-choice boundary. Granite 8B is a cheap, portable agent candidate when the task has executable checks and the context fits; Granite 30B buys modest general capability at a steep memory and throughput cost. A fixed-harness SWE-bench Pro run that puts the 8B within five points of the 30B at less than half the task cost would confirm that IBM found a real agent sweet spot.

Apodex Makes the Loop Larger

Apodex 1.1 shows the other way to make a model more capable: leave the weights alone and spend the extra system budget on decomposition, retrieval, and a separate verification pass. The result is an agent-architecture claim with a live API, not merely a research diagram.

The production documentation exposes OpenAI-compatible Chat Completions, native Anthropic Messages, and a Responses API for submit, resume, cancel, and disconnect-safe background work. Apodex also opened FrontierAgent, the terminal and evaluation runtime behind its long-running workflows. That makes the before-and-after concrete: an ordinary request can become a durable job with multiple workers, saved artifacts, and a final reviewer.

The cleanest evidence is FrontierSearchBench, a set of 41 web-research tasks with executable scorers. One task requires all eleven Starship test flights, checked across 23 dimensions, with false details reducing the score. Apodex 1.1 scored 57.0 in a standard reason-and-act loop. The same model reached 69.1 when four workers split source collection and a final verifier checked the merged answer. On the Starship task, that verifier caught eight detail errors after 210 retrieval actions and lifted the score from 0.30 to 0.61.

This is scaffolded scale. A model’s working capacity expands because the surrounding system holds parallel state, revisits primary evidence, and rejects unsupported claims. The model does not need to remember every branch inside one transcript. Apodex’s 12.1-point gain is therefore a systems result: orchestration changes which errors survive to the final answer.

The counterargument is substantial. Apodex designed the benchmark and ran every published row, including GPT-5.6 Sol at 67.4 and Claude Opus 5 at 64.4. BenchLM withholds a composite rank because only ten sourced benchmark rows exist; comparable task cost, token price, and speed are missing. Verification also consumed more retrieval and model work, so higher quality may simply be a higher bill.

The structural lesson still survives the vendor claim. For long research jobs, architecture can move task quality more than a model upgrade, but it must report the added compute and the errors the verifier itself introduces. An independent rerun of the 41 open tasks that preserves at least an eight-point verifier gain under a fixed dollar budget would turn scaffolded scale from a promising demo into a deployment result.

The Contrarian Take

Everyone says: Small and open models are catching the frontier, making raw parameter scale less important.

Here's why that's wrong (or at least incomplete): Granite 4.2 8B scores 20 on Artificial Analysis while frontier systems remain far ahead, and Apodex 1.1 in a plain loop trails its own orchestrated version by 12.1 points. The smaller systems become competitive only after training absorbs agent behavior or the runtime supplies extra search and checking. Scale has not disappeared. It has moved from one model file into a larger execution stack whose memory, tokens, and verification work still have to be paid for.

Under the Radar

“Done” becomes an executable claim — Unlazy writes acceptance gates before an agent begins, binds approvals to the exact command and working directory, and re-runs completed checks instead of trusting the final report. The current 2.1 source is untagged and its old performance ratios are not reproducible, but the design correctly treats agent self-certification as untrusted data.
Documents acquire typed relationships — Docling Graph converts documents into validated Pydantic objects and directed graphs with stable IDs, explicit edges, and page-level provenance. That changes retrieval from “find similar text” to “follow this instrument, dependency, or measurement back to its source,” while still supporting Ollama, vLLM, and API models.

Quick Takes

Deuz makes agent state injectable — The zero-dependency TypeScript Deuz SDK injects clocks, randomness, network calls, keys, and logging, then checkpoints durable runs to an application-owned database. The project is early, but deterministic side effects make crash recovery and tests part of the runtime contract rather than provider behavior. (Source)
Agnes is cheap, fast, and verbose — Artificial Analysis puts Agnes 2.5 Pro Beta at an index score of 49, 153.3 output tokens per second, and $0.10 input / $0.30 output per million tokens. It also generated 150 million evaluation tokens against a peer median of 66 million, so low token prices do not settle its task economics. (Source)
Compaction is an authority policy — The Compaction Cliff studies 396,934 agent configurations from 54,628 GitHub repositories and ships a corpus, classifier, and reference implementation. Its practical warning is simple: when safety rules and episodic state compete for the same context, a summarizer decides which authority survives. No major daily runtime has integrated the detector yet. (Source)

The Thread

A useful model-size measure now has three budgets: learned behavior in the weights, working state in the runtime, and evidence checked after generation. Granite spends training to compress agent behavior into 8B parameters. Apodex spends inference to expand one model into a research team. Unlazy, Docling Graph, and Deuz move intent and state into files, graphs, and databases. Smaller weights do not eliminate scale; they make the location of scale an architectural choice.

Predictions

New predictions:

I predict: vLLM or SGLang will ship Granite 4.2 reasoning parsing as a built-in named option, removing IBM’s custom parser step from the documented serving command. (Confidence: medium; Check by: 2026-10-31)

Issue date: 2026-08-30 · Generated: 03:44 ET