External Validation of an Automated Surgical Step Recognition Model for Robotic Distal Gastrectomy (RDG) Using a Multicenter Dataset

Background: Innovations in artificial intelligence (AI) are revolutionizing surgical practices, enhancing the analysis and outcomes of complex procedures. Recent advances in AI-based computer vision have enabled our team to develop a novel artificial intelligence model that can recognize defined steps of robotic distal gastrectomy (RDG). Methods: This study assessed 130 robotic surgical videos from two institutions, 69 and 61 videos, respectively. The AI model used TeCNO, a multi-stage temporal convolutional network, and was trained using annotated videos with surgical steps defined by qualified surgeons. RDG step recognition predicted by the model was assessed using accuracy, precision, recall, and F-value metrics, and statistical analysis was assessed. Results: Three data sets were established to train and test the model. AI trained on single institution training sets performs moderately well at predicting RDG surgical steps with accuracies ranging from 56% to 63%, whereas AI trained on the multi-institutional data yielded a step recognition accuracy of 86%. These results were confirmed with F-scores and precision tests. Conclusions: We demonstrated that an AI step recognition model for RDG can predict surgical steps in an external video dataset with moderate accuracy. Furthermore, we conclude that training an AI model on a multi-institutional dataset significantly increases its step recognition capabilities. These results confirm that our model can be utilized by external institutions, and that a diverse training set of RDG procedures from multiple institutions is valuable to developing an AI model with precise step recognition capabilities in new institutions.