# Symbolic Mechanistic Data Attribution: Tracing Training Influence to Learned Behavioral Policies

Preprint · Reza Habibi, Darian Lee, Magy Seif El-Nasr

- Archive id: arxiv:2606.29171
- Published: 2026-06-28
- arXiv: https://arxiv.org/abs/2606.29171
- Source URL: https://arxiv.org/abs/2606.29171
- PDF: https://arxiv.org/pdf/2606.29171
- Provenance: arxiv

## Abstract

While existing data attribution methods can identify which training examples build specific mechanistic circuits, they cannot explain how training data shapes the behavioral decisions a model learns. We introduce Symbolic Mechanistic Data Attribution (SMDA), a framework that attributes training pairs to the interpretable symbolic policies governing model behavior. SMDA fits a closed-form Ridge regression over sparse autoencoder (SAE) features to model a behavior, then decomposes how each supervised fine-tuning (SFT) example shifts that policy through feature-activation ($ΔX$) and output-probability ($ΔY$) pathways. We distill a symbolic policy for refusal in Llama-3.2-3B-Instruct and analyze 200 SFT pairs. Our analysis reveals (1)~the symbolic policy's coefficients expose systematic gaps in the base model's safety behavior, such as religious stereotyping; (2)~per-feature $ΔX/ΔY$ decomposition mechanistically explains why harmful and harmless pairs exert qualitatively different influences on certain features; and (3)~individual pairs routinely exhibit cross-feature interference, allowing SMDA to identify pairs whose dominant effect falls on unintended features. Retraining without such pairs improved accuracy by up to 4.5 percentage points over full fine-tuning. These results show that combining mechanistic interpretability with data attribution yields a diagnostic tool both more fine-grained than black-box influence functions and more scalable than manual circuit analysis.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.