CS-8395-04: Special Topics – Intelligent Surgical Robots
This course will provide a broad overview of current topics in improving the capabilities of surgical robots. With robot-assisted surgical procedures, we can capture data about the surgeon’s motions and register to the patient anatomy in ways that were simply unavailable previously. This provides an unprecedented opportunity for machine learning to improve patient care. While some specialties have made use of this by automating certain treatments (with LASIK being a well-known application), most general surgery robots are still manually teleoperated. In this course, we will discuss why this is and what steps we can take to use machine learning to improve surgeries and the training of surgeons.
Topics covered include modeling robot motions and signals, modeling the patient anatomy, and how pre-operative imaging may be used to guide intraoperative decisions. By the end of the course, you should have an understanding of different aspects of surgical robots and how they fit together.
Location: Olin Hall 134
Time: Tuesdays and Thursdays 2:45 pm – 4:00 pm
Instructor: Jie Ying Wu
Office hours: 4:00 pm – 5:00 pm in Featheringill 364
Homeworks and projects will be submitted and graded through Brightspace. Please use Brightspace discussion for homework questions. Readings are posted on Perusall (access through Brightspace) and discussions on readings will take place there.
Prerequisites: This course is aimed at graduate students and advanced undergraduate students with an interest in applying computer science to healthcare. It assumes a background in deep learning (some familiarity with how neural networks work and experience with implementing one in any framework) and computer vision (calibration and frame transformations). We will be using Pytorch for homeworks so familiarity with it is useful though the goal of the first homework will be to bring everyone up to speed on the specifics of Pytorch.
If you do not have access to a GPU, there will be Google Cloud credits available (thanks to their generous academic program!) to run homeworks and projects.
Tentative schedule
Week 1 – Intro to surgical robots
Week 2 – Kinematics and teleoperation
Week 3 – Intro to different imaging devices and calibration
Week 4 – Neural networks for robot instrument segmentation
Week 5 – Image-guided interventions and tissue tracking
Week 6 – Anatomy modeling
Week 7 – Soft-tissue modeling
Week 8 – Losses and challenges in evaluation
Week 9 – Video gesture analysis and unsupervised modeling
Week 10 – Guest lectures from Dr. Peter Kazanzides and Dr. Florian Richter
Week 11 – Kinematics-based surgical gestures and skill identification
Week 12 – Imitation learning for surgical subtask automation
Week 13 – Virtual fixtures and controls
Week 14 – Happy Thanksgiving!
Week 15 – Endoscope camera manipulation
Week 16 – Project preparation and presentation
Assignments
There will be three homeworks, one paper presentation, and two group projects.