{"id":"arxiv:2610.37533","kind":"preprint","title":"Which Hand Was Late? Diagnosis for Robot Failure Recovery in Bimanual Tool Use","authors":["Baptiste Bonin","Caro Strickland","Audrey Durand"],"author_count":3,"year":2026,"published":"2026-10-27","venue":null,"cited_by":0,"doi":null,"arxiv_id":"2610.37533","nct_id":null,"source_url":"https://arxiv.org/abs/2610.37533","archive_url":"https://space-kitty.com/papers/arxiv:2610.37533","updated":"2026-10-27","pmid":null,"pmcid":null,"openalex_id":"W7224083087","pdf_url":"https://arxiv.org/pdf/2610.37533","abstract":"Failure diagnosers for robot manipulation are increasingly vision--language models (VLMs), trained and evaluated on single-arm failures produced by perturbing a successful demonstration. Bimanual tool use adds failures of other kinds: a tool that slips out of its socket while still held, two correct gestures made at different times, and attempts that fail for two reasons at once. On a simulated bimanual tool-use task whose failure causes are read from the recorded trace, we measure what each sensing channel lets a diagnoser tell. The tool's slip is read on the tool hand; which hand was late is read on both arms, while a sensor on one hand reads its own hand's delay and at most part of the other's; and whether a late attempt was also pushed too weakly is not identifiable from passive sensing when the tool hand is late. Within two attempts, a repair policy that escalates when the cause is not identifiable repairs as many failures as one told the true cause. A VLM's internal state carries the late hand almost perfectly, along a direction no closer to the one its answer reads than a random one; whether the answer reports it depends on the wording of the question.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.RO","cs.AI"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"ai-agents","name":"LLM Agents & Tool Use","domain_slug":"ai"}],"cited_in_dossiers":[]}