# LIMITR: Leveraging Local Information for Medical Image-Text Representation

Preprint · Gefen Dawidowicz, Elad Hirsch, Ayellet Tal

- Archive id: arxiv:2303.11755
- Published: 2023-03-21
- arXiv: https://arxiv.org/abs/2303.11755
- Source URL: https://arxiv.org/abs/2303.11755
- PDF: https://arxiv.org/pdf/2303.11755
- Provenance: arxiv

## Abstract

Medical imaging analysis plays a critical role in the diagnosis and treatment of various medical conditions. This paper focuses on chest X-ray images and their corresponding radiological reports. It presents a new model that learns a joint X-ray image & report representation. The model is based on a novel alignment scheme between the visual data and the text, which takes into account both local and global information. Furthermore, the model integrates domain-specific information of two types -- lateral images and the consistent visual structure of chest images. Our representation is shown to benefit three types of retrieval tasks: text-image retrieval, class-based retrieval, and phrase-grounding.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.