# Mathematical Reasoning via Self-supervised Skip-tree Training

Preprint · Markus N. Rabe, Dennis Lee, Kshitij Bansal, Christian Szegedy

- Archive id: arxiv:2006.04757
- Published: 2020-06-08
- arXiv: https://arxiv.org/abs/2006.04757
- Citations: 24
- Source URL: https://arxiv.org/abs/2006.04757
- PDF: https://arxiv.org/pdf/2006.04757
- Provenance: arxiv

## Abstract

We examine whether self-supervised language modeling applied to mathematical formulas enables logical reasoning. We suggest several logical reasoning tasks that can be used to evaluate language models trained on formal mathematical statements, such as type inference, suggesting missing assumptions and completing equalities. To train language models for formal mathematics, we propose a novel skip-tree task. We find that models trained on the skip-tree task show surprisingly strong mathematical reasoning abilities, and outperform models trained on standard skip-sequence tasks. We also analyze the models' ability to formulate new conjectures by measuring how often the predictions are provable and useful in other proofs.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [AI for Mathematics & Formal Proof](https://space-kitty.com/topics/ai-mathematics)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.