Key Takeaways

• Meta published AutoformBench on October 13, 2026, as a benchmark for advanced mathematical formalization in Lean 4.
• The benchmark contains 341 tasks drawn from 24 advanced mathematical sources.
• For each task, Lean-experienced mathematicians formalized and audited the target statement. The evaluated agent receives an informal proof and must translate it into Lean, constructing missing definitions, intermediate lemmas, and supporting infrastructure.
• Meta compares frontier models and reports a largely shared difficulty frontier: success depends less on mathematical domain than on the ability to build missing formal infrastructure.
• AutoformBench focuses on a broader challenge than completing an isolated theorem proof: establishing the formal development that lets the proof connect to a library.

What Is Meta’s AutoformBench?

AutoformBench: A Lean Formalization Benchmark for Advanced Mathematics is a Meta research benchmark for evaluating AI systems on mathematical formalization. Published by Meta AI Research on October 13, 2026, it comprises 341 Lean 4 formalization tasks drawn from 24 advanced mathematical sources.

The benchmark addresses a gap in how AI mathematics capabilities can be tested. Many theorem-proving evaluations begin with a formal statement and an environment that already contains much of the necessary supporting theory. The task is then to find a proof of that supplied goal. This can measure useful proof-construction ability, but it does not necessarily test whether an agent can build the formal setting needed to express and prove a result in the first place.

Meta frames real mathematical formalization as a connected development rather than an isolated answer. An agent must translate an informal proof into Lean while constructing any missing definitions, intermediate lemmas, and supporting infrastructure. Those pieces help connect the target result to the formal library. The benchmark therefore evaluates work around the proof as well as the proof itself.

Lean is a programming language and proof assistant used to express mathematical definitions and proofs in a form a computer can check. In a formalization task, the goal is not merely to produce prose that sounds mathematically convincing. The relevant mathematical objects and reasoning have to be expressed in Lean and accepted by its checker. AutoformBench uses this setting to examine whether AI systems can build the formal structure necessary for advanced mathematics.

AutoformBench at a Glance

How the Benchmark Tasks Work

AutoformBench’s task design starts with advanced mathematical material and a target statement prepared by people with Lean experience. According to Meta, mathematicians formalized and audited each target statement. The proof work is then left to the evaluated agent: it receives the informal proof and must translate that reasoning into Lean.

The agent’s responsibility extends beyond filling in a proof after all the important choices have been made. It must construct missing definitions and intermediate lemmas, along with the supporting infrastructure needed to connect the result to a formal library. In practical terms, the agent faces a formalization problem: it must express the relevant mathematics in the proof assistant’s language and establish the steps that make the target result work in that setting.

This is...