Google and Artificial Analysis Launch MKA Benchmark
Editorial & Technical Analysis
AI Benchmarking · Multimodal Agents
Google and Artificial Analysis Launch MKA Benchmark
The Multimodal Knowledge Agent benchmark tests how AI agents handle visually complex files in realistic, multi-step knowledge-work tasks, using a shared evaluation setup and a rubric that prioritizes factual accuracy.
What Google Announced
On November 10, 2026, Google announced the Multimodal Knowledge Agent benchmark in partnership with Artificial Analysis. MKA is intended to evaluate agents on work that requires reasoning across visually complex files, rather than relying only on text-based questions with known answers. The central idea is to evaluate AI against tasks shaped more like professional knowledge work: prompts accompanied by multiple multimodal inputs, with a substantial project to complete.
The announcement addresses a gap in evaluation. Many familiar language-model tests can be answered from text alone. Yet professional work often depends on information distributed across documents and visual material. Google gives examples such as cross-referencing architectural blueprints, tracing engineering schematics, and synthesizing findings into polished decks, reports, and spreadsheets. In these settings, an agent may need to use several files together, not simply produce a plausible response to a short prompt.
MKA is a benchmark, not an announcement of a new model release. The Google post introduces an evaluation framework and says that people can explore the benchmark and evaluate their own models on Artificial Analysis. It also makes a model-specific claim: Gemini 4 Carbon provides “top quality at a fraction of the price of similar models.” The information available here does not give specific prices or enough comparative detail to independently quantify that statement, so it should be understood as Google’s positioning rather than a numerical cost comparison.
How MKA Tasks Are Designed
Ready to apply enterprise AI inside your own operations?
Turn AI breakthroughs into custom internal workflows, inventory trackers, and client portals without heavy engineering overhead.
The task structure is one of the announcement’s most useful details. Each MKA task reflects a multi-step project that a seasoned professional would take an estimated three to eight hours to complete. A task pairs a realistic prompt with a bundle of multimodal input files. It is then evaluated against an expert-written rubric containing roughly 70 yes/no checks.
The rubric gives 75% of its weight to factual accuracy and 25% to visual design. That split signals that an answer is not judged solely on whether it looks polished. Factual correctness is the larger component, while design quality is also part of the evaluation. For work such as preparing a report, spreadsheet, or presentation from visual source material, both dimensions can matter: a visually organized deliverable is useful only if its claims are accurate, and accurate information can still be difficult to use if presented poorly.
The duration estimate describes the professional work represented by each task; it should not be confused with a claim about how long an AI model takes to run or how long an evaluation session lasts. Likewise, “roughly 70” describes the rubric’s approximate number of yes/no checks, not the total number of MKA tasks. The announcement details provided here do not establish an overall task count.
This design moves evaluation toward completed work products and multi-step projects. It asks more than whether a system can recognize an image or answer a question about one page. The agent must address a prompt in the presence of a bundle of inputs, and the result is assessed across a sizable set of criteria. That makes the benchmark relevant to developers building agents for visually rich work, while leaving the particular task examples and detailed scoring implementation to the benchmark materials themselves.
Why the Shared Stirrup Setup Matters
Google says every model runs in the same open-source agent harness, Stirrup, with the same tools, sandbox, and turn budget. This is a concrete part of the evaluation design: models are placed in a common testing environment rather than being described only through their standalone model capabilities. A shared harness and common evaluation constraints can help readers interpret comparisons by clarifying that these aspects of the setup are held consistent across evaluated models.
That does not, by itself, answer every question a technical evaluator might ask. The announcement details here do not provide a complete account of every tool, the exact turn limit, all task materials, or the mechanics of every scoring decision. Readers should consult the benchmark on Artificial Analysis for the available implementation and evaluation information, and should avoid inferring unspecified configuration details from the fact that the harness is shared.
For developers, the harness is important because an agent’s outcome reflects more than the underlying model in isolation. It is evaluated as a system operating within the announced setup. The common tools, sandbox, and turn budget provide part of the context needed when reading results; the benchmark’s published task and scoring details provide the rest. Results should be interpreted within these conditions rather than treated as a universal prediction of performance in every deployed environment.
What the Scoring Weights Tell Us
The 75% factual-accuracy and 25% visual-design weighting makes MKA’s priorities unusually clear in the announcement. Accuracy is the dominant criterion, while visual design contributes a meaningful but smaller share. For knowledge work, this is a practical distinction. A deck or report can be attractive but misleading; conversely, a collection of correct facts may not meet the needs of someone who must use the output. MKA’s stated weighting recognizes both, while putting substantially more emphasis on whether the content is right.
The roughly 70 yes/no checks give the rubric a finer-grained structure than a single overall judgment. They provide multiple criteria against which a task result can be assessed. However, the source information does not describe the exact criteria, how each check is assigned, how partial performance is aggregated, or whether results include category-level breakdowns. Those details matter when diagnosing why an agent does well or poorly, and they should not be assumed without consulting the benchmark documentation.
For users comparing results, a score should therefore be read alongside the task definitions and rubric information available on the benchmark. A single aggregate result cannot necessarily tell a buyer whether a system’s weakness lies in extracting a visual detail, connecting evidence across files, producing a correct conclusion, or meeting presentation expectations. The published weighting is a valuable starting point, but the finer interpretation depends on what MKA makes available for individual tasks and results.
Why Visually Complex Knowledge Work Is a Useful Test
Visual information is not limited to photographs. In professional workflows, meaning can depend on layout, the relationship between elements, or details spread across several files. A blueprint, engineering schematic, chart, slide, spreadsheet, or report can carry information that is difficult to capture by reading isolated text alone. The examples in Google’s announcement underscore this broader scope: tracing a schematic or cross-referencing a blueprint involves making connections across visual material, while producing a deck, report, or spreadsheet calls for synthesis into a usable deliverable.
That is why MKA’s focus is more specific than a general claim that an AI model accepts images. Image input alone does not demonstrate that an agent can complete a project using multiple types of evidence. A benchmark focused on visually complex tasks can help test whether systems are useful for work where documents and visual structure matter together. MKA’s project-based format and weighted rubric are the disclosed elements that support this framing.
It is still important to distinguish benchmark performance from production readiness. MKA results can provide evidence about performance on the benchmark’s tasks and under its announced shared evaluation setup. They cannot, on their own, establish how an agent will perform on every organization’s files, workflows, tools, or operational constraints. Organizations should use benchmark results as one source of evidence, then test relevant systems on representative work before relying on them for consequential processes.
What MKA Could Mean for GCC and Middle East Organizations
Organizations across the GCC and wider Middle East also work with visual and document-heavy information, including the kinds of blueprints, schematics, reports, spreadsheets, and presentations highlighted in Google’s announcement. For regional teams evaluating AI agents, MKA offers a useful way to follow progress on tasks that involve more than text-only question answering. It does not establish that any particular agent meets a specific regional organization’s requirements, but it points to evaluation questions worth applying locally.
Teams can begin by identifying representative tasks from their own work: for example, a workflow that requires an employee to consult several approved files and produce a structured deliverable. They can define what factual correctness means for that task and what visual quality makes the output usable. Comparing those internal requirements with MKA’s published focus on multi-step projects, factual accuracy, and design can help teams build a more disciplined evaluation plan.
Organizations should not assume that an external benchmark covers their own language needs, document formats, internal policies, or deployment requirements unless that coverage is explicitly documented. Nor should a headline comparison substitute for testing with relevant materials and oversight. For procurement and engineering teams, the practical value of MKA is as a reference point for agent evaluation—not as a universal certification or guarantee.
How Developers and Buyers Should Read MKA Results
Developers should first inspect what the available benchmark materials say about tasks and evaluation, then compare results only within the stated testing conditions. The same open-source Stirrup harness, tools, sandbox, and turn budget provide a common basis, but the announced facts do not justify assumptions about details that have not been provided. A careful comparison should keep the benchmark’s scope visible and avoid converting a task score into a broad claim about every kind of agent work.
Buyers can use MKA as one signal when shortlisting systems for visually complex knowledge work. They should also run their own evaluations on tasks that reflect the actual documents and deliverables their employees handle. That internal process can assess whether an agent produces accurate results on the organization’s materials and whether the output is useful for the intended workflow. The amount of human review required and the consequences of an error are decisions each organization must consider for its own use case; the announcement does not prescribe a deployment policy.
For researchers, the task length, rubric size, scoring weights, and shared harness make the announced benchmark design more concrete than a bare statement of intent. The next questions are about the details exposed through the benchmark itself: what tasks are available, how results are reported, and what evidence supports comparisons. Those questions are a reason to inspect the evaluation, not a reason to invent missing specifications.
What to Watch Next
The immediate next step is to explore MKA on Artificial Analysis, as Google invites readers to do, and review the information available there before drawing conclusions. Developers can examine the benchmark’s model evaluation offering, while researchers and enterprise teams can judge how the disclosed tasks relate to their own needs. The shared Stirrup environment and the rubric’s emphasis on accuracy and visual design are especially relevant when deciding how to interpret comparisons.
It will also be important to separate the benchmark’s reported evaluation details from Google’s accompanying statement about Gemini 4 Carbon. The claim that Gemini 4 Carbon provides top quality at a fraction of the price of similar models is in the announcement, but the context supplied here does not include pricing figures or detailed results to substantiate a numerical comparison. Readers seeking procurement guidance should look for those details rather than extrapolating from the phrase alone.
Gate of AI Analysis
MKA addresses a meaningful evaluation challenge: professional knowledge work often requires an agent to handle visual and textual evidence as part of a multi-step project. Its disclosed design is concrete in several important ways. Tasks represent work estimated at three to eight hours for a seasoned professional, pair a prompt with multimodal files, and use a rubric of roughly 70 checks weighted toward factual accuracy. Models are run in a shared Stirrup harness with the same tools, sandbox, and turn budget.
Those details make MKA more informative than an announcement that simply says a new benchmark exists. They also establish a sensible boundary for claims about it. The benchmark’s stated purpose and methodology are clear enough to understand why it may matter; its results and any model comparisons must still be read in the context of the published tasks and evaluation conditions. For developers, researchers, and organizations in the Middle East and elsewhere, the useful response is to examine the benchmark on Artificial Analysis and pair external comparisons with relevant, task-specific evaluation.
Editorial attribution: Gate of AI Editorial & Engineering Teams. GateOfAI, LLC is a Delaware, USA company.
Explore Live AI Calculators, Leaderboards & Side-by-Side Comparisons
LLM API Cost Calculator
Accurately estimate token consumption and operational costs across 50+ frontier LLMs including GPT-4o, Claude 3.5, Gemini, and DeepSeek.
AI Model Leaderboard
Evaluate frontier AI models side-by-side: reasoning benchmarks, code generation accuracy, latency scores, and price-to-performance rankings.
AI Comparison Matrix
Explore head-to-head architectural comparisons, pricing structures, feature breakdowns, and verified alternatives across leading AI software.
Ready-to-Deploy Production Solutions & SaaS
Bespoke Enterprise AI Build (Escrow)
Need custom LLM fine-tuning, RAG pipelines, or bespoke agents? Hire vetted AI architects under 100% Milestone Escrow.
LiveAssist Secure Cockpit
Bilingual real-time audio copilot for Zoom/Teams calls. Connects corporate RAG memory & guides sales/support live.
SkillForge AI – AI-Powered Learning Management System
An AI-powered learning platform for discovering courses, generating learning content, managing students and instructors, and delivering a complete…