{"topic":"robotics-embodied","sort":"top","limit":25,"offset":0,"count":25,"papers":[{"id":"doi:10.1145/3054912","kind":"article","title":"Imitation Learning","authors":["Ahmed majeed Hussein","Mohamed Medhat Gaber","Eyad Elyan","Chrisina Jayne"],"author_count":4,"year":2017,"published":"2017-04-06","venue":"ACM Computing Surveys","cited_by":1059,"doi":"10.1145/3054912","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1145/3054912","archive_url":"https://space-kitty.com/papers/doi:10.1145/3054912","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W2604382266","pdf_url":null,"abstract":null,"abstract_license":null,"license":"cc-by-nc","is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Reinforcement Learning in Robotics"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"doi:10.1007/s10994-010-5223-6","kind":"article","title":"Policy search for motor primitives in robotics","authors":["Jens Kober","Jan Peters"],"author_count":2,"year":2010,"published":"2010-11-05","venue":"Machine Learning","cited_by":477,"doi":"10.1007/s10994-010-5223-6","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1007/s10994-010-5223-6","archive_url":"https://space-kitty.com/papers/doi:10.1007/s10994-010-5223-6","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W2012392077","pdf_url":"https://link.springer.com/content/pdf/10.1007/s10994-010-5223-6.pdf","abstract":null,"abstract_license":null,"license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Reinforcement Learning in Robotics"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"arxiv:1809.03531","kind":"article","title":"PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning","authors":["Guillaume Sartoretti","Justin H. Kerr","Yunfei Shi","Glenn Wagner","T. K. Satish Kumar","Sven Koenig","Howie Choset"],"author_count":7,"year":2019,"published":"2019-03-06","venue":"IEEE Robotics and Automation Letters","cited_by":424,"doi":"10.1109/lra.2019.2903261","arxiv_id":"1809.03531","nct_id":null,"source_url":"https://doi.org/10.1109/lra.2019.2903261","archive_url":"https://space-kitty.com/papers/arxiv:1809.03531","updated":"2019-02-20","pmid":null,"pmcid":null,"openalex_id":"W2892258706","pdf_url":"https://arxiv.org/pdf/1809.03531","abstract":"Multi-agent path finding (MAPF) is an essential component of many large-scale, real-world robot deployments, from aerial swarms to warehouse automation. However, despite the community's continued efforts, most state-of-the-art MAPF planners still rely on centralized planning and scale poorly past a few hundred agents. Such planning approaches are maladapted to real-world deployments, where noise and uncertainty often require paths be recomputed online, which is impossible when planning times are in seconds to minutes. We present PRIMAL, a novel framework for MAPF that combines reinforcement and imitation learning to teach fully-decentralized policies, where agents reactively plan paths online in a partially-observable world while exhibiting implicit coordination. This framework extends our previous work on distributed learning of collaborative policies by introducing demonstrations of an expert MAPF planner during training, as well as careful reward shaping and environment sampling. Once learned, the resulting policy can be copied onto any number of agents and naturally scales to different team sizes and world dimensions. We present results on randomized worlds with up to 1024 agents and compare success rates against state-of-the-art MAPF planners. Finally, we experimentally validate the learned policies in a hybrid simulation of a factory mockup, involving both real-world and simulated robots.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Robotic Path Planning Algorithms"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.1109/mra.2010.936947","kind":"article","title":"Learning and Reproduction of Gestures by Imitation","authors":["Sylvain Calinon","Florent D'halluin","Eric L. Sauser","Darwin G. Caldwell","Aude Billard"],"author_count":5,"year":2010,"published":"2010-06-01","venue":"IEEE Robotics & Automation Magazine","cited_by":461,"doi":"10.1109/mra.2010.936947","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1109/mra.2010.936947","archive_url":"https://space-kitty.com/papers/doi:10.1109/mra.2010.936947","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W2154543878","pdf_url":null,"abstract":null,"abstract_license":null,"license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Robot Manipulation and Learning"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"arxiv:1909.12271","kind":"article","title":"RLBench: The Robot Learning Benchmark & Learning Environment","authors":["Stephen James","Zicong Ma","David Rovick Arrojo","Andrew J. Davison"],"author_count":4,"year":2020,"published":"2020-02-18","venue":"IEEE Robotics and Automation Letters","cited_by":403,"doi":"10.1109/lra.2020.2974707","arxiv_id":"1909.12271","nct_id":null,"source_url":"https://doi.org/10.1109/lra.2020.2974707","archive_url":"https://space-kitty.com/papers/arxiv:1909.12271","updated":"2019-09-26","pmid":null,"pmcid":null,"openalex_id":"W3007769740","pdf_url":"https://arxiv.org/pdf/1909.12271","abstract":"We present a challenging new benchmark and learning-environment for robot learning: RLBench. The benchmark features 100 completely unique, hand-designed tasks ranging in difficulty, from simple target reaching and door opening, to longer multi-stage tasks, such as opening an oven and placing a tray in it. We provide an array of both proprioceptive observations and visual observations, which include rgb, depth, and segmentation masks from an over-the-shoulder stereo camera and an eye-in-hand monocular camera. Uniquely, each task comes with an infinite supply of demos through the use of motion planners operating on a series of waypoints given during task creation time; enabling an exciting flurry of demonstration-based learning. RLBench has been designed with scalability in mind; new tasks, along with their motion-planned demos, can be easily created and then verified by a series of tools, allowing users to submit their own tasks to the RLBench task repository. This large-scale benchmark aims to accelerate progress in a number of vision-guided manipulation research areas, including: reinforcement learning, imitation learning, multi-task learning, geometric computer vision, and in particular, few-shot learning. With the benchmark's breadth of tasks and demonstrations, we propose the first large-scale few-shot challenge in robotics. We hope that the scale and diversity of RLBench offers unparalleled research opportunities in the robot learning community and beyond.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Robot Manipulation and Learning"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:1811.06711","kind":"article","title":"An Algorithmic Perspective on Imitation Learning","authors":["Takayuki Osa","Joni Pajarinen","Gerhard M. Neumann","James Andrew Bagnell","Pieter Abbeel","Jan Peters"],"author_count":6,"year":2018,"published":"2018-03-27","venue":"Foundations and Trends in Robotics","cited_by":405,"doi":"10.1561/2300000053","arxiv_id":"1811.06711","nct_id":null,"source_url":"https://doi.org/10.1561/2300000053","archive_url":"https://space-kitty.com/papers/arxiv:1811.06711","updated":"2018-11-16","pmid":null,"pmcid":null,"openalex_id":"W3101780148","pdf_url":"https://www.nowpublishers.com/article/Download/ROB-053","abstract":"As robots and other intelligent agents move from simple environments and problems to more complex, unstructured settings, manually programming their behavior has become increasingly challenging and expensive. Often, it is easier for a teacher to demonstrate a desired behavior rather than attempt to manually engineer it. This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning. This work provides an introduction to imitation learning. It covers the underlying assumptions, approaches, and how they relate; the rich set of algorithms developed to tackle the problem; and advice on effective tools and implementation. We intend this paper to serve two audiences. First, we want to familiarize machine learning experts with the challenges of imitation learning, particularly those arising in robotics, and the interesting theoretical and practical distinctions between it and more familiar frameworks like statistical supervised learning theory and reinforcement learning. Second, we want to give roboticists and experts in applied artificial intelligence a broader appreciation for the frameworks and tools available for imitation learning.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Reinforcement Learning in Robotics"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.3390/app12020937","kind":"article","title":"Smart Industrial Robot Control Trends, Challenges and Opportunities within Manufacturing","authors":["Jānis Ārents","Modris Greitāns"],"author_count":2,"year":2022,"published":"2022-01-17","venue":"Applied Sciences","cited_by":324,"doi":"10.3390/app12020937","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.3390/app12020937","archive_url":"https://space-kitty.com/papers/doi:10.3390/app12020937","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W4205837126","pdf_url":"https://www.mdpi.com/2076-3417/12/2/937/pdf?version=1642436013","abstract":"Industrial robots and associated control methods are continuously developing. With the recent progress in the field of artificial intelligence, new perspectives in industrial robot control strategies have emerged, and prospects towards cognitive robots have arisen. AI-based robotic systems are strongly becoming one of the main areas of focus, as flexibility and deep understanding of complex manufacturing processes are becoming the key advantage to raise competitiveness. This review first expresses the significance of smart industrial robot control in manufacturing towards future factories by listing the needs, requirements and introducing the envisioned concept of smart industrial robots. Secondly, the current trends that are based on different learning strategies and methods are explored. Current computer-vision, deep reinforcement learning and imitation learning based robot control approaches and possible applications in manufacturing are investigated. Gaps, challenges, limitations and open issues are identified along the way.","abstract_license":"CC-BY (publisher license, via OpenAlex)","license":"cc-by","is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Robot Manipulation and Learning"],"details":null,"provenance":"openalex","attribution":"Abstract of \"Smart Industrial Robot Control Trends, Challenges and Opportunities within Manufacturing\" by Jānis Ārents, Modris Greitāns, https://doi.org/10.3390/app12020937, used under its Creative Commons Attribution license (CC-BY); markup removed, otherwise unmodified."},{"id":"arxiv:2301.04195","kind":"article","title":"Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments","authors":["Mayank Mittal","Calvin Yu","Qinxi Yu","Jingzhou Liu","Nikita Rudin","David Hoeller","Jia Lin Yuan","Ritvik Singh","Yunrong Guo","Hammad Mazhar","Ajay Mandlekar","Buck Babich","Gavriel State","Marco Hutter","Animesh Garg"],"author_count":15,"year":2023,"published":"2023-04-25","venue":"IEEE Robotics and Automation Letters","cited_by":269,"doi":"10.1109/lra.2023.3270034","arxiv_id":"2301.04195","nct_id":null,"source_url":"https://doi.org/10.1109/lra.2023.3270034","archive_url":"https://space-kitty.com/papers/arxiv:2301.04195","updated":"2024-02-16","pmid":null,"pmcid":null,"openalex_id":"W4367016688","pdf_url":"https://arxiv.org/pdf/2301.04195","abstract":"We present Orbit, a unified and modular framework for robot learning powered by NVIDIA Isaac Sim. It offers a modular design to easily and efficiently create robotic environments with photo-realistic scenes and high-fidelity rigid and deformable body simulation. With Orbit, we provide a suite of benchmark tasks of varying difficulty -- from single-stage cabinet opening and cloth folding to multi-stage tasks such as room reorganization. To support working with diverse observations and action spaces, we include fixed-arm and mobile manipulators with different physically-based sensors and motion generators. Orbit allows training reinforcement learning policies and collecting large demonstration datasets from hand-crafted or expert solutions in a matter of minutes by leveraging GPU-based parallelization. In summary, we offer an open-sourced framework that readily comes with 16 robotic platforms, 4 sensor modalities, 10 motion generators, more than 20 benchmark tasks, and wrappers to 4 learning libraries. With this framework, we aim to support various research areas, including representation learning, reinforcement learning, imitation learning, and task and motion planning. We hope it helps establish interdisciplinary collaborations in these communities, and its modularity makes it easily extensible for more tasks and applications in the future.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":"cc-by-sa","is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Reinforcement Learning in Robotics"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2102.03861","kind":"article","title":"Dynamic movement primitives in robotics: A tutorial survey","authors":["Matteo Saveriano","Fares J. Abu‐Dakka","Aljaž Kramberger","Luka Peternel"],"author_count":4,"year":2023,"published":"2023-09-23","venue":"The International Journal of Robotics Research","cited_by":267,"doi":"10.1177/02783649231201196","arxiv_id":"2102.03861","nct_id":null,"source_url":"https://doi.org/10.1177/02783649231201196","archive_url":"https://space-kitty.com/papers/arxiv:2102.03861","updated":"2021-02-07","pmid":null,"pmcid":null,"openalex_id":"W3128528928","pdf_url":"https://journals.sagepub.com/doi/pdf/10.1177/02783649231201196","abstract":"Biological systems, including human beings, have the innate ability to perform complex tasks in versatile and agile manner. Researchers in sensorimotor control have tried to understand and formally define this innate property. The idea, supported by several experimental findings, that biological systems are able to combine and adapt basic units of motion into complex tasks finally lead to the formulation of the motor primitives theory. In this respect, Dynamic Movement Primitives (DMPs) represent an elegant mathematical formulation of the motor primitives as stable dynamical systems, and are well suited to generate motor commands for artificial systems like robots. In the last decades, DMPs have inspired researchers in different robotic fields including imitation and reinforcement learning, optimal control,physical interaction, and human-robot co-working, resulting a considerable amount of published papers. The goal of this tutorial survey is two-fold. On one side, we present the existing DMPs formulations in rigorous mathematical terms,and discuss advantages and limitations of each approach as well as practical implementation details. In the tutorial vein, we also search for existing implementations of presented approaches and release several others. On the other side, we provide a systematic and comprehensive review of existing literature and categorize state of the art work on DMP. The paper concludes with a discussion on the limitations of DMPs and an outline of possible research directions.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":"cc-by-nc","is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Motor Control and Adaptation"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.1080/0951192x.2015.1130251","kind":"article","title":"Human – robot interaction review and challenges on task planning and programming","authors":["Panagiota Tsarouchi","Sotiris Makris","George Chryssolouris"],"author_count":3,"year":2016,"published":"2016-02-29","venue":"International Journal of Computer Integrated Manufacturing","cited_by":376,"doi":"10.1080/0951192x.2015.1130251","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1080/0951192x.2015.1130251","archive_url":"https://space-kitty.com/papers/doi:10.1080/0951192x.2015.1130251","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W2312797433","pdf_url":null,"abstract":null,"abstract_license":null,"license":null,"is_open_access":false,"retracted":false,"peer_reviewed":null,"categories":["Robot Manipulation and Learning"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"openalex:W2963094133","kind":"article","title":"One-Shot Imitation Learning","authors":["Yan Duan","Marcin Andrychowicz","Bradly C. Stadie","Jonathan Ho","Jonas Schneider","Ilya Sutskever","Pieter Abbeel","Wojciech Zaremba"],"author_count":8,"year":2017,"published":"2017-03-21","venue":"neural information processing systems","cited_by":372,"doi":null,"arxiv_id":null,"nct_id":null,"source_url":"http://papers.nips.cc/paper/6709-one-shot-imitation-learning.pdf","archive_url":"https://space-kitty.com/papers/openalex:W2963094133","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W2963094133","pdf_url":null,"abstract":null,"abstract_license":null,"license":null,"is_open_access":false,"retracted":false,"peer_reviewed":null,"categories":["Reinforcement Learning in Robotics"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"arxiv:2309.02473","kind":"article","title":"A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges","authors":["Maryam Zare","Parham Mohsenzadeh Kebria","Abbas Khosravi","Saeid Nahavandi"],"author_count":4,"year":2024,"published":"2024-07-18","venue":"IEEE Transactions on Cybernetics","cited_by":229,"doi":"10.1109/tcyb.2024.3395626","arxiv_id":"2309.02473","nct_id":null,"source_url":"https://doi.org/10.1109/tcyb.2024.3395626","archive_url":"https://space-kitty.com/papers/arxiv:2309.02473","updated":"2023-09-05","pmid":"39024072","pmcid":null,"openalex_id":"W4400770903","pdf_url":"https://arxiv.org/pdf/2309.02473","abstract":"In recent years, the development of robotics and artificial intelligence (AI) systems has been nothing short of remarkable. As these systems continue to evolve, they are being utilized in increasingly complex and unstructured environments, such as autonomous driving, aerial robotics, and natural language processing. As a consequence, programming their behaviors manually or defining their behavior through reward functions (as done in reinforcement learning (RL)) has become exceedingly difficult. This is because such environments require a high degree of flexibility and adaptability, making it challenging to specify an optimal set of rules or reward signals that can account for all possible situations. In such environments, learning from an expert's behavior through imitation is often more appealing. This is where imitation learning (IL) comes into play - a process where desired behavior is learned by imitating an expert's behavior, which is provided through demonstrations. This paper aims to provide an introduction to IL and an overview of its underlying assumptions and approaches. It also offers a detailed description of recent advances and emerging areas of research in the field. Additionally, the paper discusses how researchers have addressed common challenges associated with IL and provides potential directions for future research. Overall, the goal of the paper is to provide a comprehensive guide to the growing field of IL in robotics and AI.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":"cc-by-sa","is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Reinforcement Learning in Robotics"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:1709.04905","kind":"preprint","title":"One-Shot Visual Imitation Learning via Meta-Learning","authors":["Chelsea Finn","Tianhe Yu","Tianhao Zhang","Pieter Abbeel","Sergey Levine"],"author_count":5,"year":2017,"published":"2017-09-14","venue":null,"cited_by":341,"doi":null,"arxiv_id":"1709.04905","nct_id":null,"source_url":"http://arxiv.org/abs/1709.04905","archive_url":"https://space-kitty.com/papers/arxiv:1709.04905","updated":"2017-09-14","pmid":null,"pmcid":null,"openalex_id":"W2755546070","pdf_url":"https://arxiv.org/pdf/1709.04905","abstract":"In order for a robot to be a generalist that can perform a wide range of jobs, it must be able to acquire a wide variety of skills quickly and efficiently in complex unstructured environments. High-capacity models such as deep neural networks can enable a robot to represent complex skills, but learning each skill from scratch then becomes infeasible. In this work, we present a meta-imitation learning method that enables a robot to learn how to learn more efficiently, allowing it to acquire new skills from just a single demonstration. Unlike prior methods for one-shot imitation, our method can scale to raw pixel inputs and requires data from significantly fewer prior tasks for effective learning of new skills. Our experiments on both simulated and real robot platforms demonstrate the ability to learn new tasks, end-to-end, from a single visual demonstration.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"retracted":false,"peer_reviewed":false,"categories":["Reinforcement Learning in Robotics"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.1163/016918611x558261","kind":"article","title":"Imitation Learning of Positional and Force Skills Demonstrated via Kinesthetic Teaching and Haptic Input","authors":["Petar S. Kormushev","Sylvain Calinon","Darwin G. Caldwell"],"author_count":3,"year":2011,"published":"2011-01-01","venue":"Advanced Robotics","cited_by":279,"doi":"10.1163/016918611x558261","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1163/016918611x558261","archive_url":"https://space-kitty.com/papers/doi:10.1163/016918611x558261","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W2031580264","pdf_url":null,"abstract":null,"abstract_license":null,"license":null,"is_open_access":false,"retracted":false,"peer_reviewed":null,"categories":["Robot Manipulation and Learning"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"arxiv:2101.01993","kind":"article","title":"A Survey of Deep RL and IL for Autonomous Driving Policy Learning","authors":["Zeyu Zhu","Huijing Zhao"],"author_count":2,"year":2021,"published":"2021-12-22","venue":"IEEE Transactions on Intelligent Transportation Systems","cited_by":240,"doi":"10.1109/tits.2021.3134702","arxiv_id":"2101.01993","nct_id":null,"source_url":"https://doi.org/10.1109/tits.2021.3134702","archive_url":"https://space-kitty.com/papers/arxiv:2101.01993","updated":"2021-01-06","pmid":null,"pmcid":null,"openalex_id":"W3121045039","pdf_url":"https://arxiv.org/pdf/2101.01993","abstract":"Autonomous driving (AD) agents generate driving policies based on online perception results, which are obtained at multiple levels of abstraction, e.g., behavior planning, motion planning and control. Driving policies are crucial to the realization of safe, efficient and harmonious driving behaviors, where AD agents still face substantial challenges in complex scenarios. Due to their successful application in fields such as robotics and video games, the use of deep reinforcement learning (DRL) and deep imitation learning (DIL) techniques to derive AD policies have witnessed vast research efforts in recent years. This paper is a comprehensive survey of this body of work, which is conducted at three levels: First, a taxonomy of the literature studies is constructed from the system perspective, among which five modes of integration of DRL/DIL models into an AD architecture are identified. Second, the formulations of DRL/DIL models for conducting specified AD tasks are comprehensively reviewed, where various designs on the model state and action spaces and the reinforcement learning rewards are covered. Finally, an in-depth review is conducted on how the critical issues of AD applications regarding driving safety, interaction with other traffic participants and uncertainty of the environment are addressed by the DRL/DIL models. To the best of our knowledge, this is the first survey to focus on AD policy learning using DRL/DIL, which is addressed simultaneously from the system, task-driven and problem-driven perspectives. We share and discuss findings, which may lead to the investigation of various topics in the future.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Autonomous Vehicle Technology and Safety"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2007.05156","kind":"article","title":"A survey on autonomous vehicle control in the era of mixed-autonomy: From physics-based to AI-guided driving policy learning","authors":["Xuan Sharon Di","Rongye Shi"],"author_count":2,"year":2021,"published":"2021-03-12","venue":"Transportation Research Part C Emerging Technologies","cited_by":235,"doi":"10.1016/j.trc.2021.103008","arxiv_id":"2007.05156","nct_id":null,"source_url":"https://doi.org/10.1016/j.trc.2021.103008","archive_url":"https://space-kitty.com/papers/arxiv:2007.05156","updated":"2020-07-10","pmid":null,"pmcid":null,"openalex_id":"W3138181334","pdf_url":"https://arxiv.org/pdf/2007.05156","abstract":"This paper serves as an introduction and overview of the potentially useful models and methodologies from artificial intelligence (AI) into the field of transportation engineering for autonomous vehicle (AV) control in the era of mixed autonomy. We will discuss state-of-the-art applications of AI-guided methods, identify opportunities and obstacles, raise open questions, and help suggest the building blocks and areas where AI could play a role in mixed autonomy. We divide the stage of autonomous vehicle (AV) deployment into four phases: the pure HVs, the HV-dominated, the AVdominated, and the pure AVs. This paper is primarily focused on the latter three phases. It is the first-of-its-kind survey paper to comprehensively review literature in both transportation engineering and AI for mixed traffic modeling. Models used for each phase are summarized, encompassing game theory, deep (reinforcement) learning, and imitation learning. While reviewing the methodologies, we primarily focus on the following research questions: (1) What scalable driving policies are to control a large number of AVs in mixed traffic comprised of human drivers and uncontrollable AVs? (2) How do we estimate human driver behaviors? (3) How should the driving behavior of uncontrollable AVs be modeled in the environment? (4) How are the interactions between human drivers and autonomous vehicles characterized? Hopefully this paper will not only inspire our transportation community to rethink the conventional models that are developed in the data-shortage era, but also reach out to other disciplines, in particular robotics and machine learning, to join forces towards creating a safe and efficient mixed traffic ecosystem.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Traffic control and management"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.3390/robotics7020017","kind":"article","title":"Robot Learning from Demonstration in Robotic Assembly: A Survey","authors":["Zuyuan Zhu","Huosheng Hu"],"author_count":2,"year":2018,"published":"2018-04-16","venue":"Robotics","cited_by":256,"doi":"10.3390/robotics7020017","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.3390/robotics7020017","archive_url":"https://space-kitty.com/papers/doi:10.3390/robotics7020017","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W2796864868","pdf_url":"https://www.mdpi.com/2218-6581/7/2/17/pdf?version=1525348666","abstract":"Learning from demonstration (LfD) has been used to help robots to implement manipulation tasks autonomously, in particular, to learn manipulation behaviors from observing the motion executed by human demonstrators. This paper reviews recent research and development in the field of LfD. The main focus is placed on how to demonstrate the example behaviors to the robot in assembly operations, and how to extract the manipulation features for robot learning and generating imitative behaviors. Diverse metrics are analyzed to evaluate the performance of robot imitation learning. Specifically, the application of LfD in robotic assembly is a focal point in this paper.","abstract_license":"CC-BY (publisher license, via OpenAlex)","license":"cc-by","is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Robot Manipulation and Learning"],"details":null,"provenance":"openalex","attribution":"Abstract of \"Robot Learning from Demonstration in Robotic Assembly: A Survey\" by Zuyuan Zhu, Huosheng Hu, https://doi.org/10.3390/robotics7020017, used under its Creative Commons Attribution license (CC-BY); markup removed, otherwise unmodified."},{"id":"doi:10.3390/s21041278","kind":"review","title":"Learning for a Robot: Deep Reinforcement Learning, Imitation Learning, Transfer Learning","authors":["Jiang Hua","Liangcai Zeng","Gongfa Li","Zhaojie Ju"],"author_count":4,"year":2021,"published":"2021-02-11","venue":"Sensors","cited_by":231,"doi":"10.3390/s21041278","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.3390/s21041278","archive_url":"https://space-kitty.com/papers/doi:10.3390/s21041278","updated":null,"pmid":"33670109","pmcid":null,"openalex_id":"W3128707967","pdf_url":"https://www.mdpi.com/1424-8220/21/4/1278/pdf?version=1613722477","abstract":"Dexterous manipulation of the robot is an important part of realizing intelligence, but manipulators can only perform simple tasks such as sorting and packing in a structured environment. In view of the existing problem, this paper presents a state-of-the-art survey on an intelligent robot with the capability of autonomous deciding and learning. The paper first reviews the main achievements and research of the robot, which were mainly based on the breakthrough of automatic control and hardware in mechanics. With the evolution of artificial intelligence, many pieces of research have made further progresses in adaptive and robust control. The survey reveals that the latest research in deep learning and reinforcement learning has paved the way for highly complex tasks to be performed by robots. Furthermore, deep reinforcement learning, imitation learning, and transfer learning in robot control are discussed in detail. Finally, major achievements based on these methods are summarized and analyzed thoroughly, and future research challenges are proposed.","abstract_license":"CC-BY (publisher license, via OpenAlex)","license":"cc-by","is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Reinforcement Learning in Robotics"],"details":null,"provenance":"openalex","attribution":"Abstract of \"Learning for a Robot: Deep Reinforcement Learning, Imitation Learning, Transfer Learning\" by Jiang Hua, Liangcai Zeng, Gongfa Li et al., https://doi.org/10.3390/s21041278, used under its Creative Commons Attribution license (CC-BY); markup removed, otherwise unmodified."},{"id":"doi:10.1016/j.rcim.2022.102360","kind":"article","title":"Robot learning towards smart robotic manufacturing: A review","authors":["Zhihao Liu","Quan Liu","Wenjun Xu","Lihui Wang","Zude Zhou"],"author_count":5,"year":2022,"published":"2022-04-20","venue":"Robotics and Computer-Integrated Manufacturing","cited_by":222,"doi":"10.1016/j.rcim.2022.102360","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1016/j.rcim.2022.102360","archive_url":"https://space-kitty.com/papers/doi:10.1016/j.rcim.2022.102360","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W4224221644","pdf_url":null,"abstract":"Robotic equipment has been playing a central role since the proposal of smart manufacturing. Since the beginning of the first integration of industrial robots into production lines, industrial robots have enhanced productivity and relieved humans from heavy workloads significantly. Towards the next generation of manufacturing, this review first introduces the comprehensive background of smart robotic manufacturing within robotics, machine learning, and robot learning. Definitions and categories of robot learning are summarised. Concretely, imitation learning, policy gradient learning, value function learning, actor-critic learning, and model-based learning as the leading technologies in robot learning are reviewed. Training tools, benchmarks, and comparisons amongst different robot learning methods are delivered. Typical industrial applications in robotic grasping, assembly, process control, and industrial human-robot collaboration are listed and discussed. Finally, open problems and future research directions are summarised.","abstract_license":"CC-BY (publisher license, via OpenAlex)","license":"cc-by","is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Reinforcement Learning in Robotics"],"details":null,"provenance":"openalex","attribution":"Abstract of \"Robot learning towards smart robotic manufacturing: A review\" by Zhihao Liu, Quan Liu, Wenjun Xu et al., https://doi.org/10.1016/j.rcim.2022.102360, used under its Creative Commons Attribution license (CC-BY); markup removed, otherwise unmodified."},{"id":"arxiv:2109.12098","kind":"preprint","title":"CLIPort: What and Where Pathways for Robotic Manipulation","authors":["Mohit Shridhar","Lucas Manuelli","Dieter Fox"],"author_count":3,"year":2021,"published":"2021-09-24","venue":null,"cited_by":223,"doi":null,"arxiv_id":"2109.12098","nct_id":null,"source_url":"http://arxiv.org/abs/2109.12098","archive_url":"https://space-kitty.com/papers/arxiv:2109.12098","updated":"2021-09-24","pmid":null,"pmcid":null,"openalex_id":"W3203511201","pdf_url":"https://arxiv.org/pdf/2109.12098","abstract":"How can we imbue robots with the ability to manipulate objects precisely but also to reason about them in terms of abstract concepts? Recent works in manipulation have shown that end-to-end networks can learn dexterous skills that require precise spatial reasoning, but these methods often fail to generalize to new goals or quickly learn transferable concepts across tasks. In parallel, there has been great progress in learning generalizable semantic representations for vision and language by training on large-scale internet data, however these representations lack the spatial understanding necessary for fine-grained manipulation. To this end, we propose a framework that combines the best of both worlds: a two-stream architecture with semantic and spatial pathways for vision-based manipulation. Specifically, we present CLIPort, a language-conditioned imitation-learning agent that combines the broad semantic understanding (what) of CLIP [1] with the spatial precision (where) of Transporter [2]. Our end-to-end framework is capable of solving a variety of language-specified tabletop tasks from packing unseen objects to folding cloths, all without any explicit representations of object poses, instance segmentations, memory, symbolic states, or syntactic structures. Experiments in simulated and real-world settings show that our approach is data efficient in few-shot settings and generalizes effectively to seen and unseen semantic concepts. We even learn one multi-task policy for 10 simulated and 9 real-world tasks that is better or comparable to single-task policies.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"retracted":false,"peer_reviewed":false,"categories":["Robot Manipulation and Learning"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.1061/(asce)co.1943-7862.0002154","kind":"article","title":"Human–Robot Collaboration in Construction: Classification and Research Trends","authors":["Ci‐Jyun Liang","Xi Wang","Vineet R. Kamat","Carol C. Menassa"],"author_count":4,"year":2021,"published":"2021-08-04","venue":"Journal of Construction Engineering and Management","cited_by":223,"doi":"10.1061/(asce)co.1943-7862.0002154","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1061/(asce)co.1943-7862.0002154","archive_url":"https://space-kitty.com/papers/doi:10.1061/(asce)co.1943-7862.0002154","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W3188255222","pdf_url":null,"abstract":null,"abstract_license":null,"license":null,"is_open_access":false,"retracted":false,"peer_reviewed":null,"categories":["Robot Manipulation and Learning"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"arxiv:2203.13251","kind":"article","title":"Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation","authors":["Sridhar Pandian Arunachalam","Sneha Silwal","Ben Evans","Lerrel Pinto"],"author_count":4,"year":2022,"published":"2022-03-24","venue":"Proceedings - IEEE International Conference on Robotics and Automation/Proceedings","cited_by":84,"doi":"10.1109/icra48891.2023.10160275","arxiv_id":"2203.13251","nct_id":null,"source_url":"https://arxiv.org/abs/2203.13251","archive_url":"https://space-kitty.com/papers/arxiv:2203.13251","updated":"2022-03-24","pmid":null,"pmcid":null,"openalex_id":"W4383108491","pdf_url":"https://arxiv.org/pdf/2203.13251","abstract":"Optimizing behaviors for dexterous manipulation has been a longstanding challenge in robotics, with a variety of methods from model-based control to model-free reinforcement learning having been previously explored in literature. Perhaps one of the most powerful techniques to learn complex manipulation strategies is imitation learning. However, collecting and learning from demonstrations in dexterous manipulation is quite challenging. The complex, high-dimensional action-space involved with multi-finger control often leads to poor sample efficiency of learning-based methods. In this work, we propose 'Dexterous Imitation Made Easy' (DIME) a new imitation learning framework for dexterous manipulation. DIME only requires a single RGB camera to observe a human operator and teleoperate our robotic hand. Once demonstrations are collected, DIME employs standard imitation learning methods to train dexterous manipulation policies. On both simulation and real robot benchmarks we demonstrate that DIME can be used to solve complex, in-hand manipulation tasks such as 'flipping', 'spinning', and 'rotating' objects with the Allegro hand. Our framework along with pre-collected demonstrations is publicly available at https://nyu-robot-learning.github.io/dime.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":null,"categories":["cs.RO","cs.AI","cs.CV","cs.LG"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"doi:10.1016/j.robot.2014.03.001","kind":"article","title":"Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions","authors":["Seyed Mohammad Khansari-Zadeh","Aude Billard"],"author_count":2,"year":2014,"published":"2014-03-17","venue":"Robotics and Autonomous Systems","cited_by":229,"doi":"10.1016/j.robot.2014.03.001","arxiv_id":null,"nct_id":null,"source_url":"https://doi.org/10.1016/j.robot.2014.03.001","archive_url":"https://space-kitty.com/papers/doi:10.1016/j.robot.2014.03.001","updated":null,"pmid":null,"pmcid":null,"openalex_id":"W1991667713","pdf_url":null,"abstract":null,"abstract_license":null,"license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Reinforcement Learning in Robotics"],"details":null,"provenance":"openalex","attribution":"Metadata and citation counts: OpenAlex (CC0)."},{"id":"arxiv:2112.03227","kind":"article","title":"CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks","authors":["Oier Mees","Lukás Hermann","Erick Rosete-Beas","Wolfram Burgard"],"author_count":4,"year":2022,"published":"2022-06-03","venue":"IEEE Robotics and Automation Letters","cited_by":188,"doi":"10.1109/lra.2022.3180108","arxiv_id":"2112.03227","nct_id":null,"source_url":"https://doi.org/10.1109/lra.2022.3180108","archive_url":"https://space-kitty.com/papers/arxiv:2112.03227","updated":"2022-07-13","pmid":null,"pmcid":null,"openalex_id":"W4226073959","pdf_url":"https://arxiv.org/pdf/2112.03227","abstract":"General-purpose robots coexisting with humans in their environment must learn to relate human language to their perceptions and actions to be useful in a range of daily tasks. Moreover, they need to acquire a diverse repertoire of general-purpose skills that allow composing long-horizon tasks by following unconstrained language instructions. In this paper, we present CALVIN (Composing Actions from Language and Vision), an open-source simulated benchmark to learn long-horizon language-conditioned tasks. Our aim is to make it possible to develop agents that can solve many robotic manipulation tasks over a long horizon, from onboard sensors, and specified only via human language. CALVIN tasks are more complex in terms of sequence length, action space, and language than existing vision-and-language task datasets and supports flexible specification of sensor suites. We evaluate the agents in zero-shot to novel language instructions and to novel environments and objects. We show that a baseline model based on multi-context imitation learning performs poorly on CALVIN, suggesting that there is significant room for developing innovative agents that learn to relate human language to their world models with this benchmark.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["Robot Manipulation and Learning"],"details":null,"provenance":"openalex+arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."},{"id":"arxiv:2204.12490","kind":"article","title":"From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation","authors":["Yuzhe Qin","Hao Su","Xiaolong Wang"],"author_count":3,"year":2022,"published":"2022-04-26","venue":"IEEE Robotics and Automation Letters","cited_by":104,"doi":"10.1109/lra.2022.3196104","arxiv_id":"2204.12490","nct_id":null,"source_url":"https://arxiv.org/abs/2204.12490","archive_url":"https://space-kitty.com/papers/arxiv:2204.12490","updated":"2023-01-18","pmid":null,"pmcid":null,"openalex_id":"W4225003209","pdf_url":"https://arxiv.org/pdf/2204.12490","abstract":"We propose to perform imitation learning for dexterous manipulation with multi-finger robot hand from human demonstrations, and transfer the policy to the real robot hand. We introduce a novel single-camera teleoperation system to collect the 3D demonstrations efficiently with only an iPad and a computer. One key contribution of our system is that we construct a customized robot hand for each user in the physical simulator, which is a manipulator resembling the same kinematics structure and shape of the operator's hand. This provides an intuitive interface and avoid unstable human-robot hand retargeting for data collection, leading to large-scale and high quality data. Once the data is collected, the customized robot hand trajectories can be converted to different specified robot hands (models that are manufactured) to generate training demonstrations. With imitation learning using our data, we show large improvement over baselines with multiple complex manipulation tasks. Importantly, we show our learned policy is significantly more robust when transferring to the real robot. More videos can be found in the https://yzqin.github.io/dex-teleop-imitation .","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":true,"retracted":false,"peer_reviewed":null,"categories":["cs.RO","cs.CV","cs.LG"],"details":null,"provenance":"arxiv+openalex","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv."}],"next":"https://space-kitty.com/api/v1/topics/robotics-embodied/papers?sort=top&limit=25&offset=25"}