Opus Scores 12% On GDPx

If You Only Read One Thing

Agents can be poor substitutes for professionals and still make effective attackers. GDPx’s professional-task results put Opus first at just 12%; WorkOS’s breach shows AI-assisted attacks reaching customer conversations through a sales tool. There’s no contradiction: finishing a legitimate assignment means satisfying its requirements, while an attacker needs only one route to something valuable. Those are very different capability thresholds.

GDPx puts Opus at 12% on demanding professional work

The best agent for a short coding task isn’t necessarily ready to own the whole assignment. SemiAnalysis’s November 2 GDPx release makes that gap harder to ignore: Claude Opus 5.5 leads its broad professional-work score at 12%, with an average run costing about $51.

GDPx covers 206 tasks across 11 equally weighted domains. It tests models in their native agent software where available: Claude Code, Codex and Antigravity, for example. The default allowance is eight hours, with shorter limits for some tasks. This is a comparison of working systems, including their tools and restrictions, rather than isolated model answers.

The 12% needs care. GDPx combines all-or-nothing grading with partial-credit grading, then averages across domains and three attempts. It doesn't mean Opus completed exactly 12% of the 206 tasks, or that professionals can automate 12% of their jobs. The public results describe performance on a deliberately difficult selection of work.

For coding, the failures are more familiar than the headline sounds. The authors report that 76% of unsuccessful software-engineering runs involved requirements around the central request: preserving existing behavior, integration and edge cases. An agent could make the requested change while leaving the larger job unfinished. That's an extension of October 28’s FrontierCode finding, now appearing inside a broader professional-task evaluation.

That changes how a model comparison earns its keep. Checking whether the requested feature appeared misses whether the feature still works under existing configurations. A separate record of those surrounding requirements makes a failure diagnosable: the agent never investigated one, misunderstood it, or tried and failed. Another reviewer helps most when it receives those unresolved obligations, rather than just the patch and its author's explanation.

There’s a good objection to treating GDPx as a verdict on everyday usefulness. Its authors selected unusually demanding domains, and provider safety restrictions affect some results. A system can save substantial human work without completing an assignment alone. But partial progress and unattended completion need separate prices; a polished intermediate result can still leave expensive checking behind.

The released transcripts let that checking become concrete. They expose what agents actually attempted, rather than only their final claims. For a coding-model switch, the result that would change my view is fewer missed surrounding requirements on the same repository tasks, not merely a higher aggregate rank.

WorkOS’s sales tool exposed customer conversations

WorkOS says an AI-assisted intrusion reached customer relationship and support information through an internal sales application. Its November 2 disclosure puts an often overlooked asset into the agent-security discussion: the context that makes a fraudulent request sound authentic.

The company identified unauthorized access on October 30. Its security bulletin lists business contacts, account histories, excerpts and summaries of calls, support discussions and customer Slack conversations, plus product-use and connection-health metadata. WorkOS says it has no evidence that production services or WorkOS application data were accessed. The investigation continues.

Production and support information serve different purposes, but both can help an attacker. A stolen API key grants access directly. A real support conversation can persuade someone to grant it later. Imagine a request that correctly names an integration problem, the previous support contact and the customer's connection trouble. Those details make the request credible without proving the sender has authority.

The same problem reaches assistants that search support tickets or draft replies. Retrieved history can help an assistant understand a request. It can't establish that the person making it may change credentials or receive confidential records. A request to change credentials needs verification through an established account or contact route. Accurate recollections of a support call aren't a substitute.

This develops the October 31 ARTEX coverage with a named company's disclosure and a specific set of exposed records. WorkOS attributes automated reconnaissance and exploitation attempts to an AI-assisted offensive framework. That doesn't tell us how much of the intrusion was autonomous or how much faster AI made it. The confirmed consequence is access to business context, not a measured replacement of human attackers.

Defenders are putting models into the same investigative loop. In a separate November 2 announcement, OpenAI described its next security sprint with Cathedral. It says the previous sprint assigned a quarter of product engineers to security work, using Codex to investigate findings, reproduce vulnerabilities in isolation and prepare patches for review. Finding weaknesses and confirming repairs remain separate jobs.

WorkOS disabled external access to the affected tool and brought in an independent forensics firm. Its eventual account of the entry point, application permissions and reachable records will show whether one exposed tool could read more customer context than it needed.

The Contrarian Take

Everyone says: Agents must become dependable enough to replace professionals before they can have a large effect on professional work.

Here’s why that’s wrong: Dependability constrains the defender much more than the attacker. A legitimate agent has to finish the assignment, preserve existing behavior and respect access limits. An attacker benefits from a single successful disclosure, even if most attempts fail. GDPx’s low scores and WorkOS’s incident therefore fit together. The mistake is using an average completion score to estimate exposure to a determined adversary. Low average competence offers little comfort when attempts are cheap and one success is enough.

Under the Radar

Legora evaluates agents without reading customer traces. In its November 2 operator interview, the company describes three engineers and three lawyers recreating customer work, including a 10,000-file due-diligence environment. It maps performance by task and practice area, then checks known failure categories in production. Confidentiality makes copying live sessions into a test set impossible. Reconstructing the task preserves a way to compare models, although the account supplies no controlled accuracy gain and simulated work can miss production failures.
Willison treats generated pull requests as bug reports with candidate fixes. His November 2 LLM 0.37 release account credits fixes from 12 authors, almost all AI-assisted. He uses Opus or Astra to help identify valuable contributions, sometimes accepting the patch and sometimes writing his own repair. The contribution can supply a reproducible problem without earning automatic acceptance of its solution. That's a maintainer's reported workflow, not a measured reduction in review time; it also depends on someone who can judge the underlying project.

Quick Takes

Anthropic forecasts research acceleration; it hasn't declared the threshold crossed. Its November 2 amended filing says expectations for future models may soon meet its policy threshold for dramatically faster AI progress attributable to AI. The same passage explicitly says no current model has been designated above it. The capability implication is faster model development with lagging measurement, not demonstrated autonomous creation of successor models. Published acceleration measurements would add considerably more than the forecast. (Source)
LongCat-2.5 adds another agent-focused model candidate. Meituan’s November 2 announcement describes a native multimodal model aimed at extended agent tasks, including coding. The company's comparison names older GPT and Claude releases, so those vendor results can't settle its position against the current frontier. For model selection, support for images and long tasks makes it relevant to investigate; it doesn't yet justify replacing a proven coding route. (Source)
Gemini 2.5 names will stop preserving model identity. Google’s November 2 notice says December 1 requests for 2.5 Pro will route to 3.8 Flash, while 2.5 Flash will route to 3.5 Flash-Lite, at the destination models’ rates. Function calling and reasoning controls also change. Keeping the old string won't preserve the service contract: a previously passing integration may send parameters or tool history the replacement rejects. (Source)

The Thread

AI capability is becoming harder to describe with a single standard of “good enough.” A model that can't finish a complex assignment may still help someone investigate it. A model that produces unreliable attacks may still expose one valuable application. Each use has a different cost of failure and a different beneficiary of partial progress. That shifts the economic question from how intelligent the model is to who absorbs the unfinished work. In a legitimate workflow, the customer pays for checking and repair. In an intrusion, the victim pays for the one attempt that worked. Spending more on verification can therefore be rational even while model prices fall: the expensive part is deciding which incomplete results are safe to act on.

Predictions

New predictions:

I predict: By December 1, Google will implement both Gemini 2.5 automatic routing changes described in its November 2 notice, rather than extending the old endpoints unchanged. This follows a dated provider commitment, not an assumption about model improvement. (Confidence: high; Check by: 2026-12-02)

November 3, 2026 · 03:35 AM ET