# Linear-DPO: Linear Direct Preference Optimization for Diffusion and Flow-Matching Generative Models

Preprint · Kesong Li, Yixuan Xu, Kuo-kun Tseng, Weiyi Lu, Kan Liu, Tao Lan

- Archive id: arxiv:2605.21123
- Published: 2026-05-20
- arXiv: https://arxiv.org/abs/2605.21123
- Citations: 0
- Source URL: https://arxiv.org/abs/2605.21123
- PDF: https://arxiv.org/pdf/2605.21123
- Provenance: arxiv

## Abstract

Direct Preference Optimization (DPO) is successful for alignment in LLMs but still faces challenges in text-to-image generation. Existing studies are confined to denoising diffusion models while overlooking flow-matching, and suffer from an objective mismatch when applying discrete NLP-based DPO to regression-based generative tasks.\\ In this paper, we derive a generalized DPO objective that covers both diffusion and flow-matching via a unified reverse-time SDE framework, and point out from a gradient perspective that the standard DPO objective is suboptimal for text-to-image generation. Consequently, we propose Linear-DPO, which replaces the aggressive sigmoid-based utility function with a sustained linear utility and incorporates an EMA-updated reference model. Qualitative and quantitative experiments on diffusion models (SD1.5, SDXL) and flow-matching model (SD3-Medium) demonstrate the superiority of our approach over existing baselines.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [Generative & Multimodal Models](https://space-kitty.com/topics/generative-multimodal)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.