监测上消化道盲区智能内镜影像分析系统的构建及验证

Construction and verification of intelligent endoscopic image analysis system for monitoring upper gastrointestinal blind spots

摘要:

目的 构建监测上消化道盲区的智能内镜影像分析系统，并验证其监测性能。方法 回顾性收集武汉大学人民医院消化内镜中心2016—2020年的上消化道内镜图片87 167张（数据集1），其中训练集75 551张，测试集11 616张；回顾性收集来自武汉大学人民医院消化内镜中心2016—2020年的咽部图片2 414张（数据集2），其中训练集2 233张， 测试集181张。分别构建上消化道盲区监测27分类模型（模型1，区分图像为咽部、食管、胃腔等27个解剖学部位）、咽部盲区监测5分类模型（模型2，区分上颚、咽后壁、喉部、左梨状窝、右梨状窝）。基于数据集1、2对上述模型进行训练和图片测试，基于keras框架的EfficientNet‑B4、ResNet50、VGG16模型进行训练。进一步回顾性收集来自武汉大学人民医院消化内镜中心2021年的完整上消化道内镜检查视频30个，在视频中测试模型2盲区监测性能。结果 模型1在图片中识别上消化道27个解剖学部位准确率的横向对比结果显示，EfficientNet‑B4、ResNet50、VGG16在上消化道盲区监测27分类模型的平均准确率分别为90.90%、90.24%、89.22%，其中EfficientNet‑B4模型的表现最优，EfficientNet‑B4模型各个部位监测的准确率介于80.49%~97.80%。模型2在图片中识别咽部5个解剖学部位准确率的横向对比结果显示，EfficientNet‑B4、ResNet50、VGG16在咽部盲区监测5分类模型的平均准确率分别为99.40%、98.56%、97.01%，其中EfficientNet‑B4模型的表现最优，其各个部位监测的准确率介于96.15%~100.00%；模型2在视频中识别咽部5个解剖学部位的总体准确率为97.33%（146/150）。结论 基于深度学习构建的可监测上消化道盲区的智能内镜影像分析系统，耦合了咽部盲区监测及食管、胃腔、十二指肠盲区监测功能，在静止图像及视频中均具有较高识别准确率，有望应用于临床辅助医生实现上消化道视野全覆盖。

Abstract:

Objective To construct an intelligent endoscopic image analysis system that could monitor the blind spot of the upper gastrointestinal tract, and to test its performance. Methods A total of 87 167 upper gastrointestinal endoscopy images (dataset 1) including 75 551 for training and 11 616 for testing, and a total of 2 414 pharyngeal images (dataset 2) including 2 233 for training and 181 for testing were retrospectively collected from the Digestive Endoscopy Center of Renmin Hospital of Wuhan University between 2016 to 2020. A 27‑category‑classification model for blind spot monitoring in the upper gastrointestinal tract (model 1, which distinguished 27 anatomical sites such as the pharynx, esophagus, and stomach) and a 5‑category‑classification model for blind spot monitoring in the pharynx (model 2, which distinguished palate, posterior pharyngeal wall, larynx, left and right pyriform sinuses) were constructed. The above models were trained and tested based on dataset 1 and 2, respectively, and trained based on the EfficientNet‑B4, ResNet50 and VGG16 models of the keras framework. Thirty complete upper gastrointestinal endoscopy videos were retrospectively collected from the Digestive Endoscopy Center of Renmin Hospital of Wuhan University in 2021 to test model 2 blind spot monitoring performance. Results The cross‑sectional comparison results of the accuracy of model 1 in identifying 27 anatomical sites of the upper gastrointestinal tract in images showed that the mean accuracy of EfficientNet‑B4, ResNet50, and VGG16 were 90.90%, 90.24%, and 89.22%, respectively, with the EfficientNet‑B4 model performance the best, and the accuracy of EfficientNet‑B4 model for each site ranged from 80.49% to 97.80%. The cross‑sectional comparison results of the accuracy of model 2 in identifying the 5 anatomical sites of the pharynx in the images showed that the mean accuracy of EfficientNet‑B4, ResNet50, and VGG16 were 99.40%, 98.56%, and 97.01%, respectively, in which the EfficientNet‑B4 model had the best performance, and the accuracy of EfficientNet‑B4 model for each site ranged from 96.15% to 100.00%. The overall accuracy of model 2 in identifying the 5 anatomical sites of the pharynx in the video was 97.33% (146/150). Conclusion The intelligent endoscopic image analysis system based on deep learning can monitor blind spots in the upper gastrointestinal tract, coupled with pharyngeal blind spot monitoring and esophagogastroduodenal blind spot monitoring functions. The system shows high accuracy in both images and videos, which is expected to have a potential role in clinical practice and assisting endoscopists to achieve full observation of the upper gastrointestinal tract.