# Viral Proteins Reveal Geometry of Protein Language Models

Preprint · Arthur Bigot, Harmon Bhasin, Core Francisco Park, Eugene Shakhnovich, Dianzhuo Wang

- Archive id: arxiv:2606.12609
- Published: 2026-06-10
- arXiv: https://arxiv.org/abs/2606.12609
- Citations: 0
- Source URL: https://arxiv.org/abs/2606.12609
- PDF: https://arxiv.org/pdf/2606.12609
- Provenance: arxiv

## Abstract

Protein language models are trained on highly imbalanced datasets, raising the question of how they represent underrepresented biological sequences. Using viral proteins as a case study across ESM model families, we identify a dominant nativeness axis in embedding space, aligned with masked reconstruction perplexity, that orders sequences from well-modeled cellular proteins through viral proteins to shuffled and random sequences. Scaling contracts this axis unevenly across viral families. Despite this, protein language model embeddings retain viral-specific signal: viral proteins remain linearly separable beyond zero-shot perplexity and shallow sequence features. Together, these results suggest that pLM representations are structured by a general notion of nativeness while preserving information specific to distinct biological groups.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [AI Protein Design & Structure Prediction](https://space-kitty.com/topics/protein-design)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.