OpenAI Math Release Omits Elements of Advisory Guidance

OpenAI's release of hundreds of math manuscripts from an unreleased model omitted parts of advisory guidance, according to TechCrunch. The Advisory Group on Mathematics and Artificial Intelligence had recommended that labs stop testing advanced math problems on proprietary models, while OpenAI described its work as an evaluation of proprietary models on open research questions.

The release provided the model's chain of thought for 10 of 719 manuscripts. It also lacked metadata linking the natural language explanations to formal proofs, another item included in the group's guidance.

A paper from mathematicians at the University of Cambridge and King's College London documented at least two discrepancies between a natural language proof and Lean code for a problem derived from the Navier-Stokes equations. The discrepancies do not invalidate either solution, but the paper concluded that automated formal proofs require the same peer review and scrutiny as other proofs.