SuFIA: Language-Guided Augmented Dexterity for Robotic Surgical Assistants

Document detail

ID

oai:arXiv.org:2405.05226

Topic

Computer Science - Robotics

Author

Moghani, Masoud Doorenbos, Lars Panitch, William Chung-Ho Huver, Sean Azizian, Mahdi Goldberg, Ken Garg, Animesh

Year

2024

listing date

5/15/2024

Keywords

augmented dexterity sub-tasks robotic sufia surgical

Metrics

Abstract

In this work, we present SuFIA, the first framework for natural language-guided augmented dexterity for robotic surgical assistants.

SuFIA incorporates the strong reasoning capabilities of large language models (LLMs) with perception modules to implement high-level planning and low-level control of a robot for surgical sub-task execution.

This enables a learning-free approach to surgical augmented dexterity without any in-context examples or motion primitives.

SuFIA uses a human-in-the-loop paradigm by restoring control to the surgeon in the case of insufficient information, mitigating unexpected errors for mission-critical tasks.

We evaluate SuFIA on four surgical sub-tasks in a simulation environment and two sub-tasks on a physical surgical robotic platform in the lab, demonstrating its ability to perform common surgical sub-tasks through supervised autonomous operation under challenging physical and workspace conditions.

Project website: orbit-surgical.github.io/sufia

Moghani, Masoud,Doorenbos, Lars,Panitch, William Chung-Ho,Huver, Sean,Azizian, Mahdi,Goldberg, Ken,Garg, Animesh, 2024, SuFIA: Language-Guided Augmented Dexterity for Robotic Surgical Assistants