{"id":"arxiv:2411.04709","kind":"article","title":"TIP-I2V: A Million-Scale Real Text and Image Prompt Dataset for Image-to-Video Generation","authors":["Wenhao Wang","Yi Yang"],"author_count":2,"year":2024,"published":"2024-11-05","venue":"IEEE/CVF International Conference on Computer Vision (ICCV)","cited_by":2,"doi":"10.1109/iccv51701.2025.01382","arxiv_id":"2411.04709","nct_id":null,"source_url":"https://arxiv.org/abs/2411.04709","archive_url":"https://space-kitty.com/papers/arxiv:2411.04709","updated":"2025-07-09","pmid":null,"pmcid":null,"openalex_id":"W4404404779","pdf_url":"https://arxiv.org/pdf/2411.04709","abstract":"Video generation models are revolutionizing content creation, with image-to-video models drawing increasing attention due to their enhanced controllability, visual consistency, and practical applications. However, despite their popularity, these models rely on user-provided text and image prompts, and there is currently no dedicated dataset for studying these prompts. In this paper, we introduce TIP-I2V, the first large-scale dataset of over 1.70 million unique user-provided Text and Image Prompts specifically for Image-to-Video generation. Additionally, we provide the corresponding generated videos from five state-of-the-art image-to-video models. We begin by outlining the time-consuming and costly process of curating this large-scale dataset. Next, we compare TIP-I2V to two popular prompt datasets, VidProM (text-to-video) and DiffusionDB (text-to-image), highlighting differences in both basic and semantic information. This dataset enables advancements in image-to-video research. For instance, to develop better models, researchers can use the prompts in TIP-I2V to analyze user preferences and evaluate the multi-dimensional performance of their trained models; and to enhance model safety, they may focus on addressing the misinformation issue caused by image-to-video models. The new research inspired by TIP-I2V and the differences with existing datasets emphasize the importance of a specialized image-to-video prompt dataset. The project is available at https://tip-i2v.github.io.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":null,"categories":["cs.CV"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"generative-multimodal","name":"Generative & Multimodal Models","domain_slug":"ai"}],"cited_in_dossiers":[]}