{"id":"arxiv:2605.05241","kind":"preprint","title":"DexSim2Real: Foundation Model-Guided Sim-to-Real Transfer for Generalizable Dexterous Manipulation","authors":["Zijian Zeng","Fei Ding","Huiming Yang","Xianwei Li","Yuhao Liao"],"author_count":5,"year":2026,"published":"2026-05-03","venue":null,"cited_by":0,"doi":null,"arxiv_id":"2605.05241","nct_id":null,"source_url":"https://arxiv.org/abs/2605.05241","archive_url":"https://space-kitty.com/papers/arxiv:2605.05241","updated":"2026-05-03","pmid":null,"pmcid":null,"openalex_id":"W7160625947","pdf_url":"https://arxiv.org/pdf/2605.05241","abstract":"Sim-to-real transfer remains a critical bottleneck for deploying dexterous manipulation policies learned in simulation to real-world robots. Existing approaches rely on manually designed domain randomization or task-specific adaptation, limiting their generalizability across diverse manipulation scenarios. We present DexSim2Real, an integrated framework that leverages vision-language foundation models to bridge the sim-to-real gap for dexterous manipulation. Our system combines three components: (1) Foundation Model-Guided Domain Randomization (FM-DR), which uses a vision-language model as a visual realism critic to optimize simulation parameters via closed-loop CMA-ES, complementing text-based approaches like DrEureka with direct visual feedback; (2) a Tactile-Visual Cross-Attention Policy (TVCAP) that adapts cross-attention visuo-tactile fusion to zero-shot sim-to-real RL; and (3) a Progressive Skill Curriculum (PSC) that builds on LLM-based task decomposition with a difficulty scheduler tailored to contact-rich dexterous tasks. Extensive experiments on six challenging manipulation tasks with blinded evaluation demonstrate that DexSim2Real achieves a 78.2% average real-world success rate, outperforming DrEureka and DeXtreme while reducing the sim-to-real performance gap to only 8.3%.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.RO","cs.LG"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"robotics-embodied","name":"Robot Learning & Embodied AI","domain_slug":"ai"}],"cited_in_dossiers":[]}