基于人工智能的自动内镜下病灶尺寸测量系统（含视频）

An artificial intelligence‑based system for measuring the size of gastrointestinal lesions under endoscopy (with video)

摘要:

目的 开发一个基于人工智能的自动内镜下病灶尺寸测量系统，并测试其实时测量白光内镜下病灶尺寸的能力。方法 测量系统由3个模型组成：首先由模型1识别视频的连续图片中有无活检钳，有钳者标记钳叶轮廓；随后由模型2对有钳图片进行分类，分为张钳图片和未张钳图片；与此同时，模型3识别视频的连续图片中有无病灶，有病灶者标记边界；最后系统根据活检钳钳叶轮廓与病灶边界的像素对比，实时计算出病灶尺寸。数据集1由回顾性收集的武汉大学人民医院2017年1月1日—2019年11月30日4 835张图片组成，用于模型的训练和验证；数据集2由前瞻性收集的武汉大学人民医院内镜中心2019年12月1日—2020年6月4日检查拍摄的图片组成，用于测试模型分割活检钳边界和病灶边界的能力；数据集3由151个模拟病灶的302张图片组成，每个模拟病灶包括活检钳倾斜角度较大（与病灶垂直线夹角45°）和倾斜角度较小（与病灶垂直线夹角10°）情况下的图片各1张，用于测试模型在活检钳不同状态下测量病灶尺寸的能力；数据集4为视频测试集，由前瞻性收集的武汉大学人民医院内镜中心2019年8月5日—2020年9月4日检查拍摄的视频组成。以内镜医师复核后结果或内镜手术病理作为金标准，观察模型1识别有无活检钳的准确率、模型2分类活检钳状态（张钳或未张钳）的准确率和模型3识别有无病灶的准确率，用交并比（intersection over union，IoU）评价模型1的活检钳钳叶分割效果和模型3的病灶分割效果，用绝对误差和相对误差评价系统的病灶尺寸测量能力。结果 （1）数据集2共纳入1 252张图片，有钳图片821张（其中张钳图片401张、未张钳图片420张）、无钳图片431张；包含病灶图片640张、不包含病灶图片612张。模型1判断无钳图片433张（430张准确）、有钳图片819张（818张准确），识别有无活检钳的准确率为99.68%（1 248/1 252），以818张模型1准确判断有钳图片的数据统计模型1的活检钳钳叶分割效果，平均IoU为0.91（95%CI：0.90～0.92）。使用模型1准确判断的818张有钳图片评价模型2的活检钳状态分类准确率，模型2判断张钳图片384张（382张准确）、未张钳图片434张（416张准确），模型2的活检钳状态分类准确率为97.56%（798/818）。模型3判断包含病灶图片654张（626张准确）、不包含病灶图片598张（584张准确），识别有无病灶的准确率为96.65%（1 210/1 252），以626张模型3准确判断有病灶图片的数据统计模型3的病灶分割效果，平均IoU为0.86（95%CI：0.85～0.87）。（2）数据集3中：活检钳倾斜角度较小状态下系统病灶尺寸测量的平均绝对误差为0.17 mm（95%CI：0.08～0.28 mm），平均相对误差为3.77%（95%CI：0.00%～10.85%）；活检钳倾斜角度较大状态下系统病灶尺寸测量的平均绝对误差为0.17 mm（95%CI：0.09～0.26 mm），平均相对误差为4.02%（95%CI：2.90%～5.14%）。（3）数据集4共纳入59例患者的59个内镜检查视频的780张图片，系统病灶尺寸测量的平均绝对误差为0.24 mm（95%CI：0.00～0.67 mm），平均相对误差为9.74%（95%CI：0.00%～29.83%）。结论 基于人工智能的自动内镜下病灶尺寸测量系统可以实现内镜下对病灶尺寸的准确测量，有望提高内镜医师对病灶尺寸估计的准确率。

Abstract:

Objective To develop an artificial intelligence-based system for measuring the size of gastrointestinal lesions under white light endoscopy in real time. Methods The system consisted of 3 models. Model 1 was used to identify the biopsy forceps and mark the contour of the forceps in continuous pictures of the video. The results of model 1 were submitted to model 2 and classified into open and closed forceps. And model 3 was used to identify the lesions and mark the boundary of lesions in real time. Then the length of the lesions was compared with the contour of the forceps to calculate the size of lesions. Dataset 1 consisted of 4 835 images collected retrospectively from January 1, 2017 to November 30, 2019 in Renmin Hospital of Wuhan University, which were used for model training and validation. Dataset 2 consisted of images collected prospectively from December 1, 2019 to June 4, 2020 at the Endoscopy Center of Renmin Hospital of Wuhan University, which were used to test the ability of the model to segment the boundary of the biopsy forceps and lesions. Dataset 3 consisted of 302 images of 151 simulated lesions, each of which included one image of a larger tilt angle (45° from the vertical line of the lesion) and one image of a smaller tilt angle (10° from the vertical line of the lesion) to test the ability of the model to measure the lesion size with the biopsy forceps in different states. Dataset 4 was a video test set, which consisted of prospectively collected videos taken from the Endoscopy Center of Renmin Hospital of Wuhan University from August 5, 2019 to September 4, 2020. The accuracy of model 1 in identifying the presence or absence of biopsy forceps, model 2 in classifying the status of biopsy forceps (open or closed) and model 3 in identifying the presence or absence of lesions were observed with the results of endoscopist review or endoscopic surgery pathology as the gold standard. Intersection over union (IoU) was used to evaluate the segmentation effect of biopsy forceps in model 1 and lesion segmentation effect in model 3, and the absolute error and relative error were used to evaluate the ability of the system to measure lesion size. Results (1)A total of 1 252 images were included in dataset 2, including 821 images of forceps (401 images of open forceps and 420 images of closed forceps), 431 images of non-forceps, 640 images of lesions and 612 images of non-lesions. Model 1 judged 433 images of non-forceps (430 images were accurate) and 819 images of forceps (818 images were accurate), and the accuracy was 99.68% (1 248/1 252). Based on the data of 818 images of forceps to evaluate the accuracy of model 1 on judging the segmentation effect of biopsy forceps lobe, the mean IoU was 0.91 (95%CI: 0.90-0.92). The classification accuracy of model 2 was evaluated by using 818 forceps pictures accurately judged by model 1. Model 2 judged 384 open forceps pictures (382 accurate) and 434 closed forceps pictures (416 accurate), and the classification accuracy of model 2 was 97.56% (798/818). Model 3 judged 654 images containing lesions (626 images were accurate) and 598 images of non-lesions (584 images were accurate), and the accuracy was 96.65% (1 210/1 252). Based on 626 images of lesions accurately judged by model 3, the mean IoU was 0.86 (95%CI: 0.85-0.87). (2) In dataset 3, the mean absolute error of systematic lesion size measurement was 0.17 mm (95%CI: 0.08-0.28 mm) and the mean relative error was 3.77% (95%CI: 0.00%-10.85%) when the tilt angle of biopsy forceps was small. The mean absolute error of systematic lesion size measurement was 0.17 mm (95%CI: 0.09-0.26 mm) and the mean relative error was 4.02% (95%CI: 2.90%-5.14%) when the biopsy forceps was tilted at a large angle. (3) In dataset 4, a total of 780 images of 59 endoscopic examination videos of 59 patients were included. The mean absolute error of systematic lesion size measurement was 0.24 mm (95%CI: 0.00-0.67 mm), and the mean relative error was 9.74% (95%CI: 0.00%-29.83%). Conclusion The system could measure the size of endoscopic gastrointestinal lesions accurately and may improve the accuracy of endoscopists.