Menu

Crypto sections

The new multimodal knowledge agent benchmark tests whether AI can handle blueprints, schematics and clinical scans like a working professional

Tue 10 Nov 2026 · 13:21 EST · 3 min read

In AI

Share

Google DeepMind and Artificial Analysis have released a new yardstick for AI agents, and it involves a lot more squinting at diagrams than most benchmarks do.

The Multimodal Knowledge Agent benchmark, or MKA, grades autonomous agents on complex, visually heavy professional work. It skips trivia questions and puzzle-solving. It is closer to a performance review.

What MKA actually tests

The two organizations unveiled MKA on November 10, 2026. The release coincided with a DeepMind blog post and an accompanying technical paper.

The benchmark contains 200 private tasks spread across 10 industry verticals. Each task was shaped with input from specialists representing 83 different occupations.

Agents must interpret architectural blueprints, engineering schematics, clinical scans and other dense visual material. Agents then have to produce real deliverables, including spreadsheets, slide decks and marked-up images.

The tasks are modeled on multi-step projects that typically take human professionals 3 to 8 hours to complete.

Scoring follows strict expert rubrics. Those rubrics weigh factual accuracy alongside visual design, so a correct answer buried in an ugly, unreadable slide deck still loses points.

Grading the graders

MKA leans on an agentic autorater, an AI system that scores the outputs of other AI systems.

According to the research findings, the autorater reached a 96.2% agreement rate with human experts on the evaluation criteria.

AI, tech, and the markets they move—in one daily briefing.

Daily. Free. Join 34,000+ readers across crypto, finance, and policy.

We respect your privacy. Unsubscribe anytime.

The private nature of the tasks matters too. Because the 200 tasks are not public, developers cannot easily train their models on the answers ahead of time.

Early results: quality versus cost

The first leaderboard already contains a notable data point for Google. Gemini 4 Carbon (high) ranked second in quality on MKA.

The more eye-catching figure is efficiency. Gemini 4 Carbon (high) completed tasks at roughly 6x lower cost than the model that took the top spot.

It is worth noting the obvious. DeepMind co-built the benchmark, and its own model scored well on cost-adjusted performance. That does not invalidate the results, but readers should weigh them with that relationship in mind.

Why visual reasoning is the missing piece

Many existing AI benchmarks rely on text-only tasks. MKA is designed to fill that gap by pulling visual reasoning into the evaluation itself.

That gap matters because professional work is rarely pure text. Architects work from drawings, engineers from schematics, and clinicians from scans.

By requiring both visual comprehension and polished output, MKA tests the full loop of knowledge work: take in complex material, reason about it, and hand back a usable product.

What this means for AI developers and buyers

For model developers, MKA raises the stakes on multimodal capability. It also puts design quality on the scorecard. Developers can no longer treat formatting as an afterthought if expert rubrics penalize cluttered or confusing deliverables.

Two hundred tasks across 10 sectors is substantial for a hand-curated benchmark, but it still covers a slice of professional work, and results in one vertical may not transfer neatly to another.

The autorater’s 96.2% agreement with human experts is strong, yet the remaining disagreement could cluster in the hardest or most subjective tasks.

Artificial Analysis brings a third-party evaluation brand to the project, while DeepMind brings the research muscle and a direct competitive interest. How other labs respond, and whether they submit models and accept the methodology, will determine whether MKA becomes an industry standard or remains one benchmark among many.

Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.

The news about what you hold

Follow your stocks and crypto. Read the news about them from hundreds of publishers in Pro.

Privacy policy