{"id":"arxiv:2602.18998","kind":"preprint","title":"Evaluating Test-Time Scaling of General LLM Agents","authors":["Xiaochuan Li","Ryan Ming","Pranav Setlur","Abhijay Paladugu","Andy Tang","Hao Kang","Shuai Shao","Rong Jin","Chenyan Xiong"],"author_count":9,"year":2026,"published":"2026-02-22","venue":null,"cited_by":1,"doi":null,"arxiv_id":"2602.18998","nct_id":null,"source_url":"https://arxiv.org/abs/2602.18998","archive_url":"https://space-kitty.com/papers/arxiv:2602.18998","updated":"2026-09-29","pmid":null,"pmcid":null,"openalex_id":"W7131400538","pdf_url":"https://arxiv.org/pdf/2602.18998","abstract":"LLM agents are increasingly expected to operate as general-purpose systems that resolve real-world user requests, yet their dynamic scaling behavior in realistic environments remains poorly understood. In this paper, we systematically investigate two principal test-time scaling axes of LLM agents: sequential scaling through extended interaction and parallel scaling through trajectory sampling. We first introduce a realistic benchmark that provides one unified framework for evaluating LLM agents across search, coding, reasoning, and tool-use domains, more faithfully reflecting the heterogeneity of real-world deployments. Evaluating ten leading LLM agents reveals substantial performance degradation when transitioning from domain-specific evaluations to this realistic setting. Building on this foundation, we progressively scale test-time compute along fine-grained increments to characterize the performance upper bound. We find that neither scaling axis can consistently yield meaningful gains from additional test-time compute in realistic environments, a phenomenon we attribute to two fundamental limitations: the scaling plateau that bottlenecks sequential scaling and the verification gap that undermines parallel scaling. Code is publicly available at https://github.com/cxcscmu/General-AgentBench.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.AI","cs.CL"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"reasoning-models","name":"Reasoning Models & Test-Time Compute","domain_slug":"ai"}],"cited_in_dossiers":[]}