# Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions

Article · Zhuochen Miao, Jun Lv, Hongjie Fang, Yang Jin, Cewu Lu

- Archive id: arxiv:2506.21057
- Published: 2025-06-26
- Venue: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- DOI: https://doi.org/10.1109/iros60139.2025.11246479
- arXiv: https://arxiv.org/abs/2506.21057
- Citations: 0
- Source URL: https://arxiv.org/abs/2506.21057
- PDF: https://arxiv.org/pdf/2506.21057
- Provenance: arxiv

## Abstract

Imitation learning has emerged as a powerful paradigm in robot manipulation, yet its generalization capability remains constrained by object-specific dependencies in limited expert demonstrations. To address this challenge, we propose knowledge-driven imitation learning, a framework that leverages external structural semantic knowledge to abstract object representations within the same category. We introduce a novel semantic keypoint graph as a knowledge template and develop a coarse-to-fine template-matching algorithm that optimizes both structural consistency and semantic similarity. Evaluated on three real-world robotic manipulation tasks, our method achieves superior performance, surpassing image-based diffusion policies with only one-quarter of the expert demonstrations. Extensive experiments further demonstrate its robustness across novel objects, backgrounds, and lighting conditions. This work pioneers a knowledge-driven approach to data-efficient robotic learning in real-world settings. Code and more materials are available on https://knowledge-driven.github.io/.

_Abstract reproduced under CC0-1.0 (arXiv descriptive metadata)._

Topics: [Robot Learning & Embodied AI](https://space-kitty.com/topics/robotics-embodied)

---
Space Kitty Archive · https://space-kitty.com · API: https://space-kitty.com/api/v1 · MCP: https://space-kitty.com/mcp · Premium: $0.20/request via x402 (Base Sepolia (testnet)). Metadata: arXiv (CC0), OpenAlex (CC0), Europe PMC (CC BY/CC0 works), ClinicalTrials.gov.