Multilingual LLM Benchmark Audit: Fact Check

Verification Status: The Microsoft Audit Claim Is Unsupported

The original draft attributes a detailed multilingual large language model evaluation audit to Microsoft Research India. It names an under-review survey, gives a document date, and reports precise counts and percentages. None of those claims is supported by the verified context supplied for this review.

Specifically, the source set does not include a Microsoft Research India paper titled The State and Fate of Multilingual, Contextual Evaluation in the NLP World. It does not establish that Microsoft Research reviewed 51 multilingual benchmarks, 242 datasets, or 219 languages. It does not substantiate the claims that 36% of languages appeared in one benchmark, that lower-resource languages received 1–3 task categories while higher-resource languages received 14, or that 56% of dataset-language instances were translated from English.

The supplied context likewise does not verify the draft’s statements about near-zero benchmark coverage in Oceania, the Americas, and Central Asia; a Microsoft document date of April 2026; benchmark-contamination findings attributed to the alleged survey; or any conclusion about the state of multilingual evaluation reached by Microsoft Research. These are material claims, not editorial colour. They require a primary source before publication.

As a result, this article does not repeat the draft’s numerical findings as facts. A numerical claim can appear authoritative because it is specific, but specificity is not verification. Until an original paper or an official Microsoft Research publication is supplied and checked, readers should treat the proposed audit, its methodology, its statistics, and its conclusions as unverified.

What the Verified Record Actually Contains

The verified record supports three more limited points. First, Google DeepMind published a retrospective titled 2023: A Year of Groundbreaking Advances in AI and Computing on December 22, 2023. The post describes progress in AI research and practical applications and refers to Google’s perspective on developing useful and beneficial applications while mitigating risks. It is not a multilingual LLM benchmark audit and should not be cited as evidence for one.

Second, the National Institute of Standards and Technology published Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST AI 100-2e2025, in March 2025. The document is a trustworthy and responsible AI publication that addresses adversarial machine-learning attacks and mitigations. Its verified scope is AI security terminology and taxonomy; it does not establish performance figures for multilingual models or validate the Microsoft-related claims in the draft.

Third, G42 published its Responsible AI Transparency Report in October 2025. The report documents a responsible-AI strategy and governance approach that includes policies and guidelines, a generative AI policy, data policy and data-governance framework, a frontier AI safety framework, risk and impact assessment, procurement risk assessment, red teaming, documentation repositories, and model cards. The report also states alignment with UAE AI strategy and UAE AI policy guidance.

These sources are useful because they demonstrate that responsible AI work is broader than a benchmark score. However, they cannot fill the missing evidence for the alleged Microsoft survey. A source should be used for the proposition it actually supports, not for a more appealing or more expansive proposition that appears nearby in an AI discussion.

Why Evidence Discipline Matters for Multilingual AI