# How good are universal MLIPs for alloys? A benchmark is only as trustworthy as its controls

Article · Gus L. W. Hart, Jacob N. Pursley

- Archive id: arxiv:2610.30641
- Published: 2026-10-22
- Venue: Research Square
- DOI: https://doi.org/10.21203/rs.3.rs-10320842/v1
- arXiv: https://arxiv.org/abs/2610.30641
- Citations: 0
- Source URL: https://arxiv.org/abs/2610.30641
- PDF: https://arxiv.org/pdf/2610.30641
- Provenance: arxiv

## Abstract

Universal machine-learned interatomic potentials (uMLIPs) are ranked by their formation-energy error against a DFT database, as if that database were ground truth, but independent DFT databases disagree. We benchmark fourteen uMLIPs on 1.45 million ordered intermetallics against AFLOW, which none trained on, and compare every energy and stability error with that disagreement. For half the structures, AFLOW, Alexandria, and OQMD differ by less than 6-8 meV/atom, but a minority differ by far more, raising the average to 16-30. That disagreement sets a floor on any model's measurable error. Re-referenced, the best uMLIP agrees with AFLOW as closely as Alexandria does, and the next four sit 2-7 meV/atom above the floor. Once broken elemental references are screened, the energy and stability rankings agree almost everywhere. Three apparent model failures shrink or vanish under the right control: poorly correlated shear moduli, thousands of "bad" DFT references, and the leader's inflated lead.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [AI for Science](https://space-kitty.com/topics/ai-for-science)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.