{"id":"arxiv:2412.21139","kind":"preprint","title":"Training Software Engineering Agents and Verifiers with SWE-Gym","authors":["Jiayi Pan","Xingyao Wang","Graham Neubig","Navdeep Jaitly","Heng Ji","Alane Suhr","Yizhe Zhang"],"author_count":7,"year":2024,"published":"2024-12-30","venue":null,"cited_by":2,"doi":null,"arxiv_id":"2412.21139","nct_id":null,"source_url":"https://arxiv.org/abs/2412.21139","archive_url":"https://space-kitty.com/papers/arxiv:2412.21139","updated":"2025-06-06","pmid":null,"pmcid":null,"openalex_id":"W4405957747","pdf_url":"https://arxiv.org/pdf/2412.21139","abstract":"We present SWE-Gym, the first environment for training real-world software engineering (SWE) agents. SWE-Gym contains 2,438 real-world Python task instances, each comprising a codebase with an executable runtime environment, unit tests, and a task specified in natural language. We use SWE-Gym to train language model based SWE agents, achieving up to 19% absolute gains in resolve rate on the popular SWE-Bench Verified and Lite test sets. We also experiment with inference-time scaling through verifiers trained on agent trajectories sampled from SWE-Gym. When combined with our fine-tuned SWE agents, we achieve 32.0% and 26.0% on SWE-Bench Verified and Lite, respectively, reflecting a new state-of-the-art for open-weight SWE agents. To facilitate further research, we publicly release SWE-Gym, models, and agent trajectories.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.SE","cs.CL"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"ai-agents","name":"LLM Agents & Tool Use","domain_slug":"ai"}],"cited_in_dossiers":[]}