# Exploring the Potential of Conversational Test Suite Based Program Repair on SWE-bench

Preprint · Anton Cheshkov, Pavel Zadorozhny, Rodion Levichev, Evgeny Maslov, Ronaldo Franco Jaldin

- Archive id: arxiv:2410.04485
- Published: 2024-10-06
- arXiv: https://arxiv.org/abs/2410.04485
- Citations: 1
- Source URL: https://arxiv.org/abs/2410.04485
- PDF: https://arxiv.org/pdf/2410.04485
- Provenance: arxiv

## Abstract

Automatic program repair at project level may open yet to be seen opportunities in various fields of human activity. Since the SWE-Bench challenge was presented, we have seen numerous of solutions. Patch generation is a part of program repair, and test suite-based conversational patch generation has proven its effectiveness. However, the potential of conversational patch generation has not yet specifically estimated on SWE-Bench. This study reports experimental results aimed at evaluating the individual effectiveness of conversational patch generation on problems from SWE-Bench. The experiments show that a simple conversational pipeline based on LLaMA 3.1 70B can generate valid patches in 47\\% of cases, which is comparable to the state-of-the-art in program repair on SWE-Bench.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [LLM Agents & Tool Use](https://space-kitty.com/topics/ai-agents)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.