Image segmentation metric and its application in the analysis of microscopic image

MA Bo-yuan; JIANG Shu-fang; YIN Dou; SHEN Hao-kai; BAN Xiao-juan; HUANG Hai-you; WANG Hao; XUE Wei-hua; FENG Hua

doi:10.13374/j.issn2095-9389.2020.05.28.002

Volume 43 Issue 1

Jan. 2021

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Engineering > 2021 > 43(1): 137-149

MA Bo-yuan, JIANG Shu-fang, YIN Dou, SHEN Hao-kai, BAN Xiao-juan, HUANG Hai-you, WANG Hao, XUE Wei-hua, FENG Hua. Image segmentation metric and its application in the analysis of microscopic image[J]. Chinese Journal of Engineering, 2021, 43(1): 137-149. doi: 10.13374/j.issn2095-9389.2020.05.28.002

Citation:

MA Bo-yuan, JIANG Shu-fang, YIN Dou, SHEN Hao-kai, BAN Xiao-juan, HUANG Hai-you, WANG Hao, XUE Wei-hua, FENG Hua. Image segmentation metric and its application in the analysis of microscopic image[J]. Chinese Journal of Engineering, 2021, 43(1): 137-149. doi: 10.13374/j.issn2095-9389.2020.05.28.002

Citation:

MA Bo-yuan, JIANG Shu-fang, YIN Dou, SHEN Hao-kai, BAN Xiao-juan, HUANG Hai-you, WANG Hao, XUE Wei-hua, FENG Hua. Image segmentation metric and its application in the analysis of microscopic image[J]. Chinese Journal of Engineering, 2021, 43(1): 137-149. doi: 10.13374/j.issn2095-9389.2020.05.28.002

PDF( 2502 KB)

Image segmentation metric and its application in the analysis of microscopic image

doi: 10.13374/j.issn2095-9389.2020.05.28.002

MA Bo-yuan^{1, 2, 3, 4},
JIANG Shu-fang⁵,
YIN Dou³,
SHEN Hao-kai⁶,
BAN Xiao-juan^{1, 2, 3, 4},
HUANG Hai-you^{1, 7, 8
,
,},
WANG Hao^{1, 9},
XUE Wei-hua^{9, 10},
FENG Hua³

1.
Beijing Advanced Innovation Center for Materials Genome Engineering, University of Science and Technology Beijing, Beijing 100083, China
2.
Beijing Key Laboratory of Knowledge Engineering for Materials Science, University of Science and Technology Beijing, Beijing 100083, China
3.
School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China
4.
Institute of Artificial Intelligence, University of Science and Technology Beijing, Beijing 100083, China
5.
Department of Obstetrics and Gynecology, General Hospital of PLA, Beijing 100853, China
6.
College of Information Science and Engineering, China University of Petroleum Beijing, Beijing 102249, China
7.
Institute for Advanced Materials and Technology, University of Science and Technology Beijing, Beijing 100083, China
8.
Shunde Graduate School, University of Science and Technology Beijing, Foshan 528300, China
9.
School of Materials Science and Engineering, University of Science and Technology Beijing, Beijing 100083, China
10.
School of Materials Science and Engineering, Liaoning Technical University, Fuxin 123099, China

More Information

Corresponding author: E-mail: huanghy@mater.ustb.edu.cn
Received Date: 2020-05-28
Publish Date: 2021-01-25

Abstract

Abstract

Material microstructure data are an important type of data in building intrinsic relationships between compositions, structures, processes, and properties, which are fundamental to material design. Therefore, the quantitative analysis of microstructures is essential for effective control of the material properties and performances of metals or alloys in various industrial applications. Microscopic images are often used to understand the important structures of a material, which are related to certain properties of interest. One of the key steps during material design process is the extraction of useful information from images through microscopic image processing using computational algorithms and tools. For example, image segmentation, which is a task that divides the image into several specific and unique regions, can detect and separate each microstructure to quantitatively analyze its size and shape distribution. This technique is commonly used in extracting significant information from microscopic images in material structure characterization field. With great improvement in computing power and methods, a large number of image segmentation methods based on different theories have made great progress, especially deep learning-based image segmentation method. Therefore selecting an appropriate evaluation method to assess the accuracy and applicability of segmentation results to properly select the optimal segmentation methods and their indications on the direction of future improvement is necessary. In this work, 14 evaluation metrics of image segmentation were summarized and discussed. The metrics were divided into five categories: pixel, intra class coincidence, edge, clustering, and instance based. In the application of material microscopic image analysis, we collected two classical datasets (Al–La alloy and polycrystalline images) to conduct quantitative experiment. The performance of different segmentation methods and different typical noises in different evaluation metrics were then compared and discussed. Finally, we discussed the advantages and applicability of various evaluation metrics in the field of microscopic image processing.
- computer vision,
- image segmentation,
- image processing,
- segmentation evaluation metrics,
- material microscopic image

FullText(HTML)

References(38)

References

[1]	Szeliski R. Computer Vision: Algorithms and Applications. Springer Science & Business Media, 2011
[2]	章毓晉. 圖像工程. 4版. 北京: 清華大學出版社, 2018 Zhang Y Z. Image Engineering. 4th Ed. Beijing: Tsinghua University Press, 2018
[3]	徐瑞. 圖像分割方法及性能評價綜述. 寧波工程學院學報, 2011, 23(3):76 doi: 10.3969/j.issn.1008-7109.2011.03.019 Xu R. An overview of image segmentation technique and performance evaluation. J Ningbo Univ Technol, 2011, 23(3): 76 doi: 10.3969/j.issn.1008-7109.2011.03.019
[4]	Gonzalez R C, Woods R E. Digital Image Processing. 3rd Ed. Upper Saddle River: Prentice Hall Press, 2008
[5]	王靜靜, 徐小亮, 梁凱彥, 等. 多孔基定形復合相變材料傳熱性能提升研究進展. 工程科學學報, 2020, 42(1):26 Wang J, Xu X L, Liang K Y, et al. Thermal conductivity enhancement of porous shape-stabilized composite phase change material for thermal energy storage applications: a review. Chin J Eng, 2020, 42(1): 26
[6]	叢明, 吳童, 劉冬, 等. 基于監督學習的前列腺MR/TRUS圖像分割和配準方法. 工程科學學報, 2020, 42(10):1362 Cong M, Wu T, Liu D, et al. Prostate MR/TRUS image segmentation and registration methods based on supervised learning. Chin J Eng, 2020, 42(10): 1362
[7]	宋曉艷. 體視學, 圖像分析與計算材料學之間的關系及進展. 中國體視學與圖像分析, 2008, 13(4):280 doi: 10.3969/j.issn.1007-1482.2008.04.013 Song X Y. Progress on the multi-disciplinary relationship of stereology, image analysis and computational materials science. Chin J Stereol Image Anal, 2008, 13(4): 280 doi: 10.3969/j.issn.1007-1482.2008.04.013
[8]	Rajan K. Materials informatics: The materials “gene” and big data. Ann Rev Mater Res, 2015, 45: 153
[9]	Butler K T, Davies D W, Cartwright H, et al. Machine learning for molecular and materials science. Nature, 2018, 559(7715): 547
[10]	LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521(7553): 436
[11]	Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern, 1979, 9(1): 62
[12]	Lakshmi S, Sankaranarayanan D V. A study of edge detection techniques for segmentation computing approaches. Int J Comput Appl, 2010, 1(1): 35
[13]	Roerdink J B T M, Meijster A. The watershed transform: definitions, algorithms and parallelization strategies. Fundam Inform, 2000, 41(1-2): 187
[14]	Adams R, Bischof L. Seeded region growing. IEEE Trans Pattern Anal Mach Intell, 1994, 16(6): 641
[15]	Jain A K. Data clustering: 50 years beyond K-means. Pattern Recognit Lett, 2010, 31(8): 651
[16]	Ma B Y, Ban X J, Su Y, et al. Fast-FineCut: Grain boundary detection in microscopic images considering 3D information. Micron, 2019, 116: 5
[17]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation // Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, 2015: 3431
[18]	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation // International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2015: 234
[19]	Li W, Field K G, Morgan D. Automated defect analysis in electron microscopic images. npj Comput Mater, 2018, 4: 36
[20]	Azimi S M, Britz D, Engstler M, et al. Advanced steel microstructural classification by deep learning methods. Sci Rep, 2018, 8: 2128
[21]	Ma B Y, Ban X J, Huang H Y, et al. Deep learning-based image segmentation for Al?La alloy microscopic images. Symmetry, 2018, 10(4): 107
[22]	Powers D M. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. J Mach Learn Technol, 2011, 2(1): 37
[23]	Dice L R. Measures of the amount of ecologic association between species. Ecology, 1945, 26(3): 297
[24]	Abdou I E, Pratt W K. Quantitative design and evaluation of enhancement/thresholding edge detectors. Proc IEEE, 1979, 67(5): 753
[25]	Mosinska A, Marquez-Neila P, Koziński M, et al. Beyond the pixel-wise loss for topology-aware delineation // Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018: 3136
[26]	Yeung K Y, Ruzzo W L. Details of the adjusted rand index and clustering algorithms, supplement to the paper an empirical study on principal component analysis for clustering gene expression data. Bioinformatics, 2001, 17(9): 763
[27]	Rand W M. Objective criteria for the evaluation of clustering methods. J Am Stat Assoc, 1971, 66(336): 846
[28]	Hubert L, Arabie P. Comparing partitions. J Classific, 1985, 2(1): 193
[29]	Meil? M. Comparing clusterings — an information based distance. J Multivar Anal, 2007, 98(5): 873
[30]	Nunez-Iglesias J, Kennedy R, Parag T, et al. Machine learning of hierarchical clustering to segment 2D and 3D images. PLoS ONE, 2013, 8(8): e71715
[31]	Booz Allen Hamilton. Data Science Bowl[J/OL]. Kaggle (2018-05-21)[2020-12-02]. https://www.kaggle.com/c/data-science-bowl-2018/overview/evaluation
[32]	Lin T Y, Maire M, Belongie S, et al. Microsoft COCO: Common Objects in Context// European Conference on Computer Vision. Zurich, 2014: 740
[33]	Waggoner J, Zhou Y J, Simmons J, et al. 3D materials image segmentation by 2D propagation: a graph-cut approach considering homomorphism. IEEE Trans Image Process, 2013, 22(12): 5282
[34]	Canny J. A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell, 1986, 8(6): 679
[35]	Vincent L, Soille P. Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans Pattern Anal Mach Intell, 1991, 13(6): 583
[36]	Hartigan J, Wong M. Algorithm AS 136: A K?means clustering algorithm. Appl Stat, 1979, 28(1): 100
[37]	Grady L. Random walks for image segmentation. IEEE Trans Pattern Anal Mach Intell, 2006, 28(11): 1768
[38]	薛維華. 晶粒組織的三維模型構建與定量表征研究[學位論文]. 北京: 北京科技大學, 2017 Xue W H. Three?Dimensional Modeling and Quantitative Characterization of Grain Structure[Dissertation]. Beijing: University of Science & Technology Beijing, 2017