# Cross-lingual Human-Preference Alignment for Neural Machine Translation with Direct Quality Optimization

Preprint · Kaden Uhlig, Joern Wuebker, Raphael Reinauer, John DeNero

- Archive id: arxiv:2409.17673
- Published: 2024-09-26
- arXiv: https://arxiv.org/abs/2409.17673
- Source URL: https://arxiv.org/abs/2409.17673
- PDF: https://arxiv.org/pdf/2409.17673
- Provenance: arxiv

## Abstract

Reinforcement Learning from Human Feedback (RLHF) and derivative techniques like Direct Preference Optimization (DPO) are task-alignment algorithms used to repurpose general, foundational models for specific tasks. We show that applying task-alignment to neural machine translation (NMT) addresses an existing task--data mismatch in NMT, leading to improvements across all languages of a multilingual model, even when task-alignment is only applied to a subset of those languages. We do so by introducing Direct Quality Optimization (DQO), a variant of DPO leveraging a pre-trained translation quality estimation model as a proxy for human preferences, and verify the improvements with both automatic metrics and human evaluation.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.