# SuperJEPA: Pretraining with a Supervision-Augmented Joint-Embedding Predictive Architecture for High Energy Physics

Preprint · Dylan Sheldon Rankin, Ho Fung Tsoi

- Archive id: arxiv:2610.36121
- Published: 2026-10-26
- arXiv: https://arxiv.org/abs/2610.36121
- Citations: 0
- Source URL: https://arxiv.org/abs/2610.36121
- PDF: https://arxiv.org/pdf/2610.36121
- Provenance: arxiv

## Abstract

Foundation models often obtain powerful results through very large unlabeled training datasets that dwarf available labeled datasets in size, but high energy physics already provides large samples of high-fidelity labeled simulation. We therefore ask whether self-supervised objectives improve over supervised pretraining alone when large labeled samples already exist. We introduce SuperJEPA, a joint-embedding predictive architec- ture augmented with a supervised objective, and find improved downstream jet-tagging performance across multiple tasks, model sizes, and pretraining dataset sizes. The gains persist for both frozen and fine-tuned encoders, motivating predictive self-supervision in future large-scale particle-physics foundation models.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [LLM Architectures & Pretraining](https://space-kitty.com/topics/llm-architectures)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.