# Direct Preference Optimization for Chatbot Fine-Tuning: An Empirical Study

Preprint · Dezhi Yu, Yvonne Qiu, ShuoJia Fu

- Archive id: arxiv:2606.12881
- Published: 2026-06-11
- arXiv: https://arxiv.org/abs/2606.12881
- Source URL: https://arxiv.org/abs/2606.12881
- PDF: https://arxiv.org/pdf/2606.12881
- Provenance: arxiv

## Abstract

We present an approach to fine-tuning large language models using Direct Preference Optimization (DPO), a reinforcement learning technique. Our experimental results demonstrate that DPO simplifies the training pipeline, improves computational efficiency, and achieves competitive performance. The evaluation using BLEU, ROUGE, and cosine similarity metrics indicates effective learning and convergence, though further investigation is needed to address observed training instability.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.