# Exact Dynamics and Finite-Sample Trajectory Recovery of Linear Recursive Feature Machines

Preprint · Andrew Cheng, Bobak T. Kiani, Yue M. Lu, Adityanarayanan Radhakrishnan

- Archive id: arxiv:2610.09196
- Published: 2026-10-06
- arXiv: https://arxiv.org/abs/2610.09196
- Citations: 0
- Source URL: https://arxiv.org/abs/2610.09196
- PDF: https://arxiv.org/pdf/2610.09196
- Provenance: arxiv

## Abstract

Recursive feature machines (RFMs) learn representations of data by alternating between fitting a predictor to a dataset and updating features of that predictor using the average gradient outer product (AGOP). Connections between AGOPs and feature learning in neural networks motivate linear RFMs as a simple setting for analyzing how representations evolve during training. Here, we study the dynamics and statistics of linear RFM in noisy multi-output regression with isotropic sub-Gaussian input data and targets generated by a low-rank teacher matrix of dimension $d$. We extend the known connection between linear RFM and iteratively reweighted least squares from the interpolating setting to ridge-regularized multi-output regression with noise. We show that the learned feature matrix remains close to its infinite-data ideal counterpart at every iteration. Namely, for $n$ samples, we show the error in the feature matrix decays as $O(\\sqrt{d/n})$ with high probability. Experiments on real-world text and single-cell gene-expression data illustrate the features learned by this simple linear model.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [Single-Cell & Spatial Omics](https://space-kitty.com/topics/single-cell-spatial)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.