Biomaterial Database

Deep learning for surgical phase recognition using endoscopic videos. 2021

Annetje C P Guédon, and Senna E P Meij, and Karim N M M H Osman, and Helena A Kloosterman, and Karlijn J van Stralen, and Matthijs C M Grimbergen, and Quirijn A J Eijsbouts, and John J van den Dobbelsteen, and Andru P Twinanda

Department of Clinical Physics, Spaarne Gasthuis, Spaarnepoort 1, 2134TM, Hoofddorp, the Netherlands. aguedon@spaarnegasthuis.nl.

Operating room planning is a complex task as pre-operative estimations of procedure duration have a limited accuracy. This is due to large variations in the course of procedures. Therefore, information about the progress of procedures is essential to adapt the daily operating room schedule accordingly. This information should ideally be objective, automatically retrievable and in real-time. Recordings made during endoscopic surgeries are a potential source of progress information. A trained observer is able to recognize the ongoing surgical phase from watching these videos. The introduction of deep learning techniques brought up opportunities to automatically retrieve information from surgical videos. The aim of this study was to apply state-of-the art deep learning techniques on a new set of endoscopic videos to automatically recognize the progress of a procedure, and to assess the feasibility of the approach in terms of performance, scalability and practical considerations. A dataset of 33 laparoscopic cholecystectomies (LC) and 35 total laparoscopic hysterectomies (TLH) was used. The surgical tools that were used and the ongoing surgical phases were annotated in the recordings. Neural networks were trained on a subset of annotated videos. The automatic recognition of surgical tools and phases was then assessed on another subset. The scalability of the networks was tested and practical considerations were kept up. The performance of the surgical tools and phase recognition reached an average precision and recall between 0.77 and 0.89. The scalability tests showed diverging results. Legal considerations had to be taken into account and a considerable amount of time was needed to annotate the datasets. This study shows the potential of deep learning to automatically recognize information contained in surgical videos. This study also provides insights in the applicability of such a technique to support operating room planning.

UI	MeSH Term	Description	Entries
D010535	Laparoscopy	A procedure in which a laparoscope (LAPAROSCOPES) is inserted through a small incision near the navel to examine the abdominal and pelvic organs in the PERITONEAL CAVITY. If appropriate, biopsy or surgery can be performed during laparoscopy.	Celioscopy,Laparoscopic Surgical Procedures,Peritoneoscopy,Surgical Procedures, Laparoscopic,Laparoscopic Assisted Surgery,Laparoscopic Surgery,Laparoscopic Surgical Procedure,Procedure, Laparoscopic Surgical,Procedures, Laparoscopic Surgical,Surgery, Laparoscopic,Surgical Procedure, Laparoscopic,Celioscopies,Laparoscopic Assisted Surgeries,Laparoscopic Surgeries,Laparoscopies,Peritoneoscopies,Surgeries, Laparoscopic,Surgeries, Laparoscopic Assisted,Surgery, Laparoscopic Assisted
D006801	Humans	Members of the species Homo sapiens.	Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D000077321	Deep Learning	Supervised or unsupervised machine learning methods that use multiple layers of data representations generated by nonlinear transformations, instead of individual task-specific ALGORITHMS, to build and train neural network models.	Hierarchical Learning,Learning, Deep,Learning, Hierarchical
D016571	Neural Networks, Computer	A computer architecture, implementable in either hardware or software, modeled after biological neural networks. Like the biological system in which the processing capability is a result of the interconnection strengths between arrays of nonlinear processing nodes, computerized neural networks, often called perceptrons or multilayer connectionist models, consist of neuron-like units. A homogeneous group of units makes up a layer. These networks are good at pattern recognition. They are adaptive, performing tasks by example, and thus are better for decision-making than are linear learning machines or cluster analysis. They do not require explicit programming.	Computational Neural Networks,Connectionist Models,Models, Neural Network,Neural Network Models,Neural Networks (Computer),Perceptrons,Computational Neural Network,Computer Neural Network,Computer Neural Networks,Connectionist Model,Model, Connectionist,Model, Neural Network,Models, Connectionist,Network Model, Neural,Network Models, Neural,Network, Computational Neural,Network, Computer Neural,Network, Neural (Computer),Networks, Computational Neural,Networks, Computer Neural,Networks, Neural (Computer),Neural Network (Computer),Neural Network Model,Neural Network, Computational,Neural Network, Computer,Neural Networks, Computational,Perceptron
D017081	Cholecystectomy, Laparoscopic	Excision of the gallbladder through an abdominal incision using a laparoscope.	Cholecystectomy, Celioscopic,Laparoscopic Cholecystectomy,Celioscopic Cholecystectomies,Celioscopic Cholecystectomy,Cholecystectomies, Celioscopic,Cholecystectomies, Laparoscopic,Laparoscopic Cholecystectomies