ISMAR 2018
IEEEIEEE computer societyIEEE vgtc


Platinum Apple
Silver MozillaIntelDaqriPTCAmazon
Bronze FacebookUmajinDisney Research
SME EnvisageAR
Academic TUMETHZ

Chengyuan Lin, Daniel Andersen, Voicu Popescu, Edgar Rojas-Mu noz, Maria Eugenia Cabrera, Brian Mullis, Ben Zarzaur, Kathryn Anderson, Sherri Marley, and Juan Wachs. A first-person mentee second-person mentor ar interface for surgical telementoring. In Adjunct Proceedings of the IEEE and ACM International Symposium for Mixed and Augmented Reality 2018 (To appear). 2018.


This application paper presents the work of a multidisciplinary group of designing; implementing; and testing an Augmented Reality (AR) surgical telementoring system. The system acquires the surgical field with an overhead camera; the video feed is transmitted to the remote mentor; where it is displayed on a touch-based interaction table; the mentor annotates the video feed; the annotations are sent back to the mentee; where they are displayed into the mentee's field of view using an optical see-through AR head-mounted display (HMD). The annotations are reprojected from the mentor's second-person view of the surgical field to the mentee's first-person view. The mentee sees the annotations with depth perception; and the annotations remain anchored to the surgical field as the mentee moves their head. Average annotation display accuracy is 1.22cm. The system was tested in the context of a user study where surgery residents (n = 20) were asked to perform a lower-leg fasciotomy on cadaver models. Participants who benefited from telementoring using our system received a higher Individual Performance Score; and they reported higher usability and self confidence levels.