Cascaded retinal vessel segmentation network guided by a skeleton map

JIANG Da-guang; LI Ming-ming; CHEN Yu-zhong; DING Wen-da; PENG Xiao-ting; LI Rui-rui

doi:10.13374/j.issn2095-9389.2021.01.13.005

Volume 43 Issue 9

Sep. 2021

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Engineering > 2021 > 43(9): 1244-1252

JIANG Da-guang, LI Ming-ming, CHEN Yu-zhong, DING Wen-da, PENG Xiao-ting, LI Rui-rui. Cascaded retinal vessel segmentation network guided by a skeleton map[J]. Chinese Journal of Engineering, 2021, 43(9): 1244-1252. doi: 10.13374/j.issn2095-9389.2021.01.13.005

Citation:

JIANG Da-guang, LI Ming-ming, CHEN Yu-zhong, DING Wen-da, PENG Xiao-ting, LI Rui-rui. Cascaded retinal vessel segmentation network guided by a skeleton map[J]. Chinese Journal of Engineering, 2021, 43(9): 1244-1252. doi: 10.13374/j.issn2095-9389.2021.01.13.005

Citation:

PDF( 1575 KB)

Cascaded retinal vessel segmentation network guided by a skeleton map

doi: 10.13374/j.issn2095-9389.2021.01.13.005

1.
School of Information Science and Technology, Beijing University of Chemical and Technology, Beijing 100029, China
2.
Beijing Airdoc Technology Co., Ltd, Beijing 100089, China
3.
Beijing Futong Dongfang Technology Co., Ltd, Beijing 100010, China

More Information

Corresponding author: E-mail: ilydouble@gmail.com
Received Date: 2020-12-30
Available Online: 2021-09-07
Publish Date: 2021-09-18

Abstract

Abstract

Accurate identification of retinal vessels is essential for assisting doctors in screening early fundus diseases. Diabetes, hypertension, and cardiovascular disease can cause abnormalities of the retinal vascular structure. Retinal vessel segmentation maps can be quickly obtained using the automated retinal vessel segmentation technology, which saves time and cost of manually identifying retinal vessels. Aiming at the problem of incomplete and inaccurate extraction of fine retinal vessels, this paper explored the design of a multitask convolutional neural network and the topological relationship of retinal vessels. A cascaded retinal vessel segmentation network framework guided by a skeleton map was proposed. The auxiliary task of skeleton extraction was used to extract vessel centerlines, which could maximally preserve topological structure information. SAFF cascaded the two modules by remaining embedded between their feature layers. This process could effectively fuse the structural features with the vessel local features by learning pixel-wise fusion weight and thus enhancing the structural response of features in the vessel segmentation module. To obtain a complete skeleton map, the skeleton map extraction module introduced a graph-based regularization loss function for training. Compared with the latest vessel segmentation methods, the proposed approach wins the first place among the three public retinal image datasets. F1 metrics of the proposed method achieved 83.1%, 85.8%, and 82.0% on the DRIVE, STARE, and CHASEDB1 datasets, respectively. Ablation studies have shown that skeleton map-guided vessel segmentation is more effective, and graph-based regularization loss further improves accuracy of the retinal vessel segmentation compared to the vanilla network. Moreover, the framework generality is verified by replacing the skeleton map extraction and vessel segmentation modules with various convolutional networks.
- skeleton extraction,
- retinal vessel segmentation,
- multitask,
- cascaded network,
- graph-based regularization

FullText(HTML)

References(52)

References

[1]	叢明, 吳童, 劉冬, 等. 基于監督學習的前列腺MR/TRUS圖像分割和配準方法. 工程科學學報, 2020, 42(10):1362 Cong M, Wu T, Liu D, et al. Prostate MR/TRUS image segmentation and registration methods based on supervised learning. Chin J Eng, 2020, 42(10): 1362
[2]	馬博淵, 姜淑芳, 尹豆, 等. 圖像分割評估方法在顯微圖像分析中的應用. 工程科學學報, 2021, 43(1):137 Ma B Y, Jiang S F, Yin D, et al. Image segmentation metric and its application in the analysis of microscopic image. Chin J Eng, 2021, 43(1): 137
[3]	Tso M O M, Jampol L M. Pathophysiology of hypertensive retinopathy. Ophthalmology, 1982, 89(10): 1132 doi: 10.1016/S0161-6420(82)34663-1
[4]	Yu S, Xiao D, Kanagasingam Y. Machine learning based automatic neovascularization detection on optic disc region. IEEE J Biomed Heal Inform, 2018, 22(3): 886 doi: 10.1109/JBHI.2017.2710201
[5]	Becker C, Rigamonti R, Lepetit V, et al. Supervised feature learning for curvilinear structure segmentation // International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, 2013: 526
[6]	Tolias Y A, Panas S M. A fuzzy vessel tracking algorithm for retinal images based on fuzzy clustering. IEEE Trans Med Imaging, 1998, 17(2): 263 doi: 10.1109/42.700738
[7]	Soares J V B, Leandro J J G, Cesar R M, et al. Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification. IEEE Trans Med Imaging, 2006, 25(9): 1214 doi: 10.1109/TMI.2006.879967
[8]	Sebbe R, Gosselin B, Coche E, et al. Segmentation of opacified thorax vessels using model-driven active contour // 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference. Shanghai, 2006: 2535
[9]	Pal S, Chatterjee S, Dey D, et al. Morphological operations with iterative rotation of structuring elements for segmentation of retinal vessel structures. Multidimens Syst Signal Process, 2019, 30(1): 373 doi: 10.1007/s11045-018-0561-9
[10]	Chang C C, Lin C C, Pai P Y, et al. A novel retinal blood vessel segmentation method based on line operator and edge detector // 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. Kyoto, 2009: 299
[11]	Zhang Y S, Chung A C S. Deep supervision with additional labels for retinal vessel segmentation task // International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, 2018: 83
[12]	Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation // International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, 2015: 234
[13]	Guo C L, Szemenyei M, Hu H, et al. Channel attention residual U-Net for retinal vessel segmentation [J/OL]. arXiv preprint (2020-10-20) [2021-6-10]. https://arxiv.org/abs/2004.03702
[14]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, 2016: 770
[15]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need// Proceedings of the 31st Conference on neural information processing systems (NIPS 2017). Long Beach, 2017: 5998
[16]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018: 7132
[17]	Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions //Proceedings of the 4th International Conference on Learning Representations. San Juan, 2016
[18]	Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell, 2018, 40(4): 834 doi: 10.1109/TPAMI.2017.2699184
[19]	Chen L C, Szemenyei M, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation[J/OL]. arXiv preprint (2017-12-5) [2021-6-10].https://arxiv.org/abs/1706.05587
[20]	Chen L C, Papandreou G, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation // Computer Vision – ECCV 2018. Munich, 2018: 833
[21]	Song H M, Wang W G, Zhao S Y, et al. Pyramid dilated deeper ConvLSTM for video salient object detection // Computer Vision – ECCV 2018. Munich, 2018: 744
[22]	Xie S N, Tu Z W. Holistically-nested edge detection // 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, 2015: 1395
[23]	Orlando J I, Blaschko M. Learning fully-connected CRFs for blood vessel segmentation in retinal images // International Conference on Medical Image Computing and Computer-Assisted Intervention. Boston, 2014: 634
[24]	Ricci E, Perfetti R. Retinal blood vessel segmentation using line operators and support vector classification. IEEE Trans Med Imaging, 2007, 26(10): 1357 doi: 10.1109/TMI.2007.898551
[25]	Ganin Y, Lempitsky V. N⁴-fields: Neural network nearest neighbor fields for image transforms // Asian Conference on Computer Vision. Singapore, 2015: 536
[26]	Dollár P, Zitnick C L. Structured forests for fast edge detection // 2013 IEEE International Conference on Computer Vision. Sydney, 2013: 1841
[27]	Maninis K K, Pont-Tuset J, Arbeláez P, et al. Deep retinal image understanding // Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016. Athens, 2016: 140
[28]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition //Proceedings of the 3th International Conference on Learning Representations. San Diego, 2015
[29]	Zhang S H, Fu H Z, Yan Y G, et al. Attention guided network for retinal image segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 797
[30]	Guo C L, Szemenyei M, Yi Y G, et al. SA-UNet: spatial attention U-net for retinal vessel segmentation [J/OL]. arXiv preprint (2020-10-20) [2021-6-10]. https://arxiv.org/abs/2004.03696
[31]	Mou L, Zhao Y T, Chen L, et al. CS-net: Channel and spatial attention network for curvilinear structure segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 721
[32]	Jiang Y, Tan N, Peng T T, et al. Retinal vessels segmentation based on dilated multi-scale convolutional neural network. IEEE Access, 2019, 7: 76342 doi: 10.1109/ACCESS.2019.2922365
[33]	Hatamizadeh A, Hosseini H, Liu Z Y, et al. Deep dilated convolutional nets for the automatic segmentation of retinal vessels[J/OL]. arXiv preprint (2019-7-21) [2021-6-10]. https://arxiv.org/abs/1905.12120
[34]	Gu Z, Cheng J, Fu H, et al. CE-net: Context encoder network for 2D medical image segmentation. IEEE Trans Med Imaging, 2019, 38(10): 2281 doi: 10.1109/TMI.2019.2903562
[35]	Mo J, Zhang L. Multi-level deep supervised networks for retinal vessel segmentation. Int J Comput Assist Radiol Surg, 2017, 12(12): 2181 doi: 10.1007/s11548-017-1619-0
[36]	Hu K, Zhang Z Z, Niu X R, et al. Retinal vessel segmentation of color fundus images using multiscale convolutional neural network with an improved cross-entropy loss function. Neurocomputing, 2018, 309: 179 doi: 10.1016/j.neucom.2018.05.011
[37]	Yan Z Q, Yang X, Cheng K T. Joint segment-level and pixel-wise losses for deep learning based retinal vessel segmentation. IEEE Trans Biomed Eng, 2018, 65(9): 1912 doi: 10.1109/TBME.2018.2828137
[38]	Zhang Z J, Fu H Z, Dai H, et al. ET-net: A generic edge-aTtention guidance network for medical image segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 442
[39]	Kang H, Gao Y Q, Guo S, et al. AVNet: A retinal artery/vein classification network with category-attention weighted fusion. Comput Methods Programs Biomed, 2020, 195: 105629 doi: 10.1016/j.cmpb.2020.105629
[40]	Zhang S H, Fu H Z, Xu Y W, et al. Retinal image segmentation with a structure-texture demixing network // Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Lima, 2020: 765
[41]	Zheng S M, Zhang T Y, Zhuang J W, et al. A two-stream meticulous processing network for retinal vessel segmentation[J/OL]. arXiv preprint (2020-1-15) [2021-6-10]. https://arxiv.org/abs/2001.05829
[42]	Zou B J, Dai Y L, He Q, et al. Multi-label classification scheme based on local regression for retinal vessel segmentation. IEEE/ACM Trans Comput Biol Bioinform, 2020, PP(99): 1
[43]	Zhang T Y, Suen C Y. A fast parallel algorithm for thinning digital patterns. Commun ACM, 1984, 27(3): 236 doi: 10.1145/357994.358023
[44]	Hakim L, Yudistira N, Kavitha M, et al. U-net with graph based smoothing regularizer for small vessel segmentation on fundus image // Proceedings of the 26th International Conference on Neural Information Processing. Sydney, 2019: 515
[45]	Oktay O, Schlemper J, Folgoc L L, et al. Attention U-net: Learning where to look for the pancreas. 2018[J/OL]. arXiv preprint (2018-5-20) [2021-6-10]. https://arxiv.org/abs/1804.03999
[46]	Li L Z, Verma M, Nakashima Y, et al. IterNet: retinal image segmentation utilizing structural redundancy in vessel networks // 2020 IEEE Winter Conference on Applications of Computer Vision (WACV). Snowmass, 2020: 3645
[47]	Jin Q G, Meng Z P, Pham T D, et al. DUNet: A deformable network for retinal vessel segmentation. Knowl Based Syst, 2019, 178: 149 doi: 10.1016/j.knosys.2019.04.025
[48]	Wang B, Qiu S, He H G. Dual encoding U-net for retinal vessel segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 84
[49]	Wang X H, Jiang X D, Ren J F. Blood vessel segmentation from fundus image by a cascade classification framework. Pattern Recognit, 2019, 88: 331 doi: 10.1016/j.patcog.2018.11.030
[50]	Niemeijer M, Staal J, van Ginneken B, et al. Comparative study of retinal vessel segmentation methods on a new publicly available database // Proceedings of SPIE‒The International Society for Optical Engineering, 2004, 5370 I: 648
[51]	Hoover A D, Kouznetsova V, Goldbaum M. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans Med Imaging, 2000, 19(3): 203 doi: 10.1109/42.845178
[52]	Owen C G, Rudnicka A R, Mullen R, et al. Measuring retinal vessel tortuosity in 10-year-old children: Validation of the computer-assisted image analysis of the retina (CAIAR) program. Invest Ophthalmol Vis Sci, 2009, 50(5): 2004 doi: 10.1167/iovs.08-3018