Generic placeholder image

Current Computer-Aided Drug Design


ISSN (Print): 1573-4099
ISSN (Online): 1875-6697

Research Article

An Evaluation of Computational Learning-based Methods for the Segmentation of Nuclei in Cervical Cancer Cells from Microscopic Images

Author(s): Tarek Maylaa*, Feryal Windal, Halim Benhabiles, Gregory Maubon, Nathalie Maubon, Elodie Vandenhaute and Dominique Collard

Volume 18, Issue 2, 2022

Published on: 17 May, 2022

Page: [81 - 94] Pages: 14

DOI: 10.2174/1573409918666220208120756

Price: $65


Background: The manual segmentation of cellular structures on Z-stack microscopic images is time-consuming and often inaccurate, highlighting the need to develop auto-segmentation tools to facilitate this process.

Objective: This study aimed to compare the performance of three different machine learning architectures, including random forest (RF), AdaBoost, and multi-layer perceptron (MLP), for the autosegmentation of nuclei in proliferating cervical cancer cells on Z-Stack cellular microscopy proliferation images provided by the HCS Pharma. The impact of using post-processing techniques, such as the StarDist plugin and majority voting, was also evaluated.

Methods: The RF, AdaBoost, and MLP algorithms were used to auto-segment the nuclei of cervical cancer cells on microscopic images at different Z-stack positions. Post-processing techniques were then applied to each algorithm. The performance of all algorithms was compared by an expert to globally generated ground truth by calculating the accuracy detection rate, the Dice coefficient, and the Jaccard index.

Results: RF achieved the best accuracy, followed by the AdaBoost and then the MLP. All algorithms achieved good pixel classifications except in regions whereby the nuclei overlapped. The majority voting and StarDist plugin improved the accuracy of the segmentation but did not resolve the nuclei overlap issue. The Z-Stack analysis revealed similar segmentation results to the Z-stack layer used to train the image. However, a worse performance was noted for segmentations performed on different Z-stack positions, which were not used to train the algorithms.

Conclusion: All machine learning architectures provided a good segmentation of nuclei in cervical cancer cells but did not resolve the problem of overlapping nuclei and Z-stack segmentation. Further research should therefore evaluate the combined segmentation techniques and deep learning architectures to resolve these issues.

Keywords: High content screening, BIOMIMESYS, segmentation, machine learning, metrics, majority voting, Z-Stack.

« Previous
Graphical Abstract
Arbyn, M.; Weiderpass, E.; Bruni, L.; de Sanjosé, S.; Saraiya, M.; Ferlay, J.; Bray, F. Estimates of incidence and mortality of cervical can-cer in 2018: A worldwide analysis. Lancet Glob. Health, 2020, 8(2), e191-e203.
[] [PMID: 31812369]
Schirrmacher, V. From chemotherapy to biological therapy: A review of novel concepts to reduce the side effects of systemic cancer treatment. (Review). Int. J. Oncol., 2019, 54(2), 407-419 [Review]..
[] [PMID: 30570109]
Morgan, S.; Grootendorst, P.; Lexchin, J.; Cunningham, C.; Greyson, D. The cost of drug development: A systematic review. Health Policy, 2011, 100(1), 4-17.
[] [PMID: 21256615]
U.S. Food And Drug Administration. Drug Development Process.Available from, (accessed Jan 18, 2021).
Van Norman, G.A. Drugs, devices, and the FDA: Part 1: An overview of approval processes for drugs. JACC Basic Transl. Sci., 2016, 1(3), 170-179.
[] [PMID: 30167510]
Pampaloni, F.; Reynaud, E.G.; Stelzer, E.H.K. The third dimension bridges the gap between cell culture and live tissue. Nat. Rev. Mol. Cell Biol., 2007, 8(10), 839-845.
[] [PMID: 17684528]
Pharma, H.C.S. BIOMIMESYS® hydroscaffold products. Available from:. (accessed Feb 9, 2021).
Bubba, F.; Pouchol, C.; Ferrand, N.; Vidal, G.; Almeida, L.; Perthame, B.; Sabbah, M. A chemotaxis-based explanation of spheroid for-mation in 3D cultures of breast cancer cells. J. Theor. Biol., 2019, 479, 73-80.
[] [PMID: 31283914]
Nichols, A. High content screening as a screening tool in drug discovery. Methods Mol. Biol., 2007, 356, 379-387.
[] [PMID: 16988417]
Lin, S.; Schorpp, K.; Rothenaigner, I.; Hadian, K. Image-based high-content screening in drug discovery. Drug Discov. Today, 2020, 25(8), 1348-1361.
[] [PMID: 32561299]
Lichtman, J.W.; Conchello, J.A. Fluorescence microscopy. Nat. Methods, 2005, 2(12), 910-919.
[] [PMID: 16299476]
Al-Kofahi, Y.; Zaltsman, A.; Graves, R.; Marshall, W.; Rusu, M. A deep learning-based algorithm for 2-D cell segmentation in microscopy images. BMC Bioinformatics, 2018, 19(1), 365.
[] [PMID: 30285608]
Song, Y.; Zhang, L.; Chen, S.; Ni, D.; Li, B.; Zhou, Y.; Lei, B.; Wang, T. A deep learning based framework for accurate segmentation of cervical cytoplasm and nuclei. 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, Illinois, August 26-30, EMBC 2014,2014 pp. 2903-2906..
Gençtav, A.; Aksoy, S.; Önder, S. Unsupervised segmentation and classification of cervical cell images. Pattern Recognit., 2012, 45(12), 4151-4168.
Sikpa, D.; Fouquet, J.P.; Lebel, R.; Diamandis, P.; Richer, M.; Lepage, M. Automated detection and quantification of breast cancer brain metastases in an animal model using democratized machine learning tools. Sci. Rep., 2019, 9(1), 17333.
[] [PMID: 31758004]
Baltissen, D.; Wollmann, T.; Gunkel, M.; Chung, I.; Erfle, H.; Rippe, K.; Rohr, K. Comparison of segmentation methods for tissue micros-copy images of glioblastoma cells. Proceedings - International Symposium on Biomedical Imaging, Washington, DC, USA, April 4-7, 2018, 2018, pp. 396-399.,
Wen, S.; Kurc, T.M.; Hou, L.; Saltz, J.H.; Gupta, R.R.; Batiste, R.; Zhao, T.; Nguyen, V.; Samaras, D.; Zhu, W. Comparison of different classifiers with active learning to support quality control in nucleus segmentation in pathology images. AMIA Jt. Summits Transl. Sci. proceedings. AMIA Jt. Summits Transl. Sci., 2018, 227.,
Naylor, P.; Lae, M.; Reyal, F.; Walter, T. Nuclei segmentation in histopathology images using deep neural networks.Proceedings - International Symposium on Biomedical Imaging, Melbourne, Australia, April 18-21, 2017, pp. 933-936.
Fishman, D.; Salumaa, S.O.; Majoral, D.; Peel, S.; Wildenhain, J.; Schreiner, A.; Palo, K.; Parts, L. Segmenting nuclei in brightfield images with neural networks. bioRxiv, 2019.764894
De Xie, Y. Multi-pixels classification for nuclei segmentation in digital pathology based on deep machine learning. J. Phys. Conf. Ser., 2018, 1087(6)062030
Burger, W.; Burge, M.J.; Image, J. Digital Image Processing; Springer: London, 2016, pp. 23-35.
Arganda-Carreras, I.; Kaynig, V.; Rueden, C.; Eliceiri, K.W.; Schindelin, J.; Cardona, A.; Sebastian Seung, H. Trainable Weka Segmenta-tion: A machine learning tool for microscopy pixel classification. Bioinformatics, 2017, 33(15), 2424-2426.
[] [PMID: 28369169]
Hall, M.; Frank, E.; Holmes, G.; Pfahringer, B.; Reutemann, P.; Witten, I.H. The WEKA data mining software. SIGKDD Explor., 2009, 11(1), 10-18.
Breiman, L. Random forests. Mach. Learn., 2001, 45(1), 5-32.
Oshiro, T.M.; Perez, P.S.; Baranauskas, J.A. How many trees in a random forest? [International workshop on machine learning and data mining in pattern recognition, Berlin, Germany 2012, pp. 154- 168.,
Schapire, R.E. Explaining adaboost.Empirical Inference: Festschrift in Honor of Vladimir N. Vapnik; Springer: Berlin, 2013, pp. 37-52.
Noriega, L. Multilayer perceptron tutorial; Sch. Comput. Staff. Univ., 2005.
Schmidt, U.; Weigert, M.; Broaddus, C.; Myers, G. Cell detection with star-convex polygons. International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain2018, pp. 265-273.
Caicedo, J.C.; Goodman, A.; Karhohs, K.W.; Cimini, B.A.; Ackerman, J.; Haghighi, M.; Heng, C.; Becker, T.; Doan, M.; McQuin, C.; Rohban, M.; Singh, S.; Carpenter, A.E. Nucleus segmentation across imaging experiments: The 2018 Data Science Bowl. Nat. Methods, 2019, 16(12), 1247-1253.
[] [PMID: 31636459]
Tustison, N.J.; Gee, J.C. Introducing dice, jaccard, and other label overlap measures to ITK. Insight J., 2009, 707.

Rights & Permissions Print Export Cite as
© 2023 Bentham Science Publishers | Privacy Policy