# Improving Medical Visual Representations via Radiology Report Generation

Preprint · Keegan Quigley, Miriam Cha, Josh Barua, Geeticka Chauhan, Seth Berkowitz, Steven Horng, Polina Golland

- Archive id: arxiv:2310.19635
- Published: 2023-10-30
- arXiv: https://arxiv.org/abs/2310.19635
- Citations: 0
- Source URL: https://arxiv.org/abs/2310.19635
- PDF: https://arxiv.org/pdf/2310.19635
- Provenance: arxiv

## Abstract

Vision-language pretraining has been shown to produce high-quality visual encoders which transfer efficiently to downstream computer vision tasks. Contrastive learning approaches have increasingly been adopted for medical vision language pretraining (MVLP), yet recent developments in generative AI offer new modeling alternatives. This paper introduces RadTex, a CNN-encoder transformer-decoder architecture optimized for radiology. We explore bidirectional captioning as an alternative MVLP strategy and demonstrate that RadTex's captioning pretraining is competitive with established contrastive methods, achieving a CheXpert macro-AUC of 89.4%. Additionally, RadTex's lightweight text decoder not only generates clinically relevant radiology reports (macro-F1 score of 0.349), but also provides targeted, interactive responses, highlighting the utility of bidirectional captioning in advancing medical image analysis.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [AI in Clinical Medicine](https://space-kitty.com/topics/ai-medicine)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.