# SWE-Bench Mobile: Can Large Language Model Agents Develop Industry-Level Mobile Applications?

Preprint · Muxin Tian, Zhe Wang, Blair Yang, Zhenwei Tang, Kunlun Zhu, Honghua Dong, Hanchen Li, Xinni Xie, Guangjing Wang, Jiaxuan You

- Archive id: arxiv:2602.09540
- Published: 2026-02-10
- Venue: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Vol. 2, pp. 8077-8087, 2026
- DOI: https://doi.org/10.1145/3770855.3818488
- arXiv: https://arxiv.org/abs/2602.09540
- Source URL: https://arxiv.org/abs/2602.09540
- PDF: https://arxiv.org/pdf/2602.09540
- Provenance: arxiv

## Abstract

Can large language model agents develop industry-level mobile applications? We introduce \\textbf{SWE-Bench Mobile}, a benchmark for evaluating coding agents on realistic software engineering tasks derived from a production iOS codebase. Unlike existing benchmarks that focus on isolated problems or bug fixes, SWE-Bench Mobile captures the full complexity of industrial development: multi-modal inputs (PRDs and Figma designs), a large-scale mixed Swift/Objective-C codebase, and comprehensive test suites. We evaluate 22 agent-model configurations across four coding agents -- three commercial (Cursor, Codex, Claude Code) and one open-source (OpenCode) -- and find that even the best configurations achieve only 12\\% task success rate. Our analysis reveals that (1) agent design matters as much as model capability -- the same model shows up to 6$\\times$ performance gap across agents, (2) commercial agents consistently outperform open-source alternatives, and (3) simple \`\`Defensive Programming'' prompts outperform complex ones by 7.4\\%. These findings highlight a significant gap between current agent capabilities and industrial requirements, while providing actionable insights for practitioners and researchers. We release SWE-Bench Mobile as a \\textit{hosted benchmark challenge} to prevent data contamination and ensure fair evaluation. The public leaderboard and development toolkit are available at https://swebenchmobile.com.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [LLM Agents & Tool Use](https://space-kitty.com/topics/ai-agents)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.