A Review on Deep Learning Architecture and Methods for MRI Brain Tumour Segmentation

Author(s): M. Angulakshmi*, M. Deepa

Journal Name: Current Medical Imaging
Formerly: Current Medical Imaging Reviews

Volume 17 , Issue 6 , 2021

Become EABM
Become Reviewer
Call for Editor

Graphical Abstract:


Background: The automatic segmentation of brain tumour from MRI medical images is mainly covered in this review. Recently, state-of-the-art performance is provided by deep learning- based approaches in the field of image classification, segmentation, object detection, and tracking tasks.

Introduction: The core feature deep learning approach is the hierarchical representation of features from images, thus avoiding domain-specific handcrafted features.

Methods: In this review paper, we have dealt with a review of Deep Learning Architecture and Methods for MRI Brain Tumour Segmentation. First, we have discussed the basic architecture and approaches for deep learning methods. Secondly, we have discussed the literature survey of MRI brain tumour segmentation using deep learning methods and its multimodality fusion. Then, the advantages and disadvantages of each method are analyzed and finally, it is concluded with a discussion on the merits and challenges of deep learning techniques.

Results: The review of brain tumour identification using deep learning.

Conclusion: Techniques may help the researchers to have a better focus on it.

Keywords: Deep learning, MRI, brain tumour, classification, architecture, challenges.

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521(7553): 436-44.
[http://dx.doi.org/10.1038/nature14539] [PMID: 26017442]
Dahl GE, Yu D, Deng L, Acero A. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Trans Audio Speech Lang Process 2011; 20(1): 30-42.
Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Commun ACM 2017; 60(6): 84-90.
Silver D, Huang A, Maddison CJ, et al. Mastering the game of Go with deep neural networks and tree search. Nature 2016; 529(7587): 484-9.
[http://dx.doi.org/10.1038/nature16961] [PMID: 26819042]
Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature 2015; 518(7540): 529-33.
[http://dx.doi.org/10.1038/nature14236] [PMID: 25719670]
Bajaj AS, Chouhan U. A Review of Various Machine Learning Techniques for Brain Tumor Detection from MRI Images. Curr Med Imaging 2020; 16(8): 937-45.
[http://dx.doi.org/10.2174/1573405615666190903144419] [PMID: 33081656]
Tan WR, Chan CS, Aguirre HE, Tanaka K. ArtGAN: Artwork synthesis with conditional categorical GANs. 2017 IEEE International Conference on Image Processing (ICIP). 3760-4.
Briot JP, Pachet F. Music generation by deep learning-challenges and directions. arXiv preprint 1712.
Briot JP, Hadjeres G, Pachet FD. Deep learningtechniques for music generation- A survey. arXiv preprint 1709.
Işın A, Direkoğlu C, Şah M. Review of MRI-based braintumor image segmentation using deep learning methods. Procedia Comput Sci 2016; 102: 317-24.
Pal A, Chaturvedi A, Garain U, Chandra A, Chatterjee R. Severity grading of psoriatic plaques using deep CNN based multi-task learning. International Conference on Pattern Recognition (ICPR). Beijing, China. 2016.
Wang G. A perspective on deep imaging. IEEE Access 2016; 4: 8914-24.
Moeskops P, Wolterink JM, van der Velden BH, Gilhuijs KG, Leiner T, Viergever MA. Išgum Deep learning for multi-task medical image segmentation in multiple modalities. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2016 Oct 17-21; Athens, Greece: Springer 2016.
Volkenandt T, Freitag S, Rauscher M. Machine learning powered image segmentation. Microsc Microanal 2018; 24(S1): 520-1.
Rathi VG, Palani S. Brain tumor detection and classification using deep learning classifier on MRI images. Res J Appl Sci Eng Technol 2015; 10(2): 177-87.
Millioni R, Sbrignadello S, Tura A, Iori E, Murphy E, Tessari P. The inter- and intra-operator variability in manual spot segmentation and its effect on spot quantitation in two-dimensional electrophoresis analysis. Electrophoresis 2010; 31(10): 1739-42.
[http://dx.doi.org/10.1002/elps.200900674] [PMID: 20408132]
Shen D, Wu G, Suk HI. Deep learning in medical image analysis. Annu Rev Biomed Eng 2017; 19: 221-48.
[http://dx.doi.org/10.1146/annurev-bioeng-071516-044442] [PMID: 28301734]
Suzuki K. Overview of deep learning in medical imaging. Radiological Phys Technol 2017; 10(3): 257-73.
[http://dx.doi.org/10.1007/s12194-017-0406-5] [PMID: 28689314]
Haque IR, Neubert J. Deep learning approaches to biomedical image segmentation. Informatics in Medicine Unlocked 2020; 18
Guo Y, Liu Y, Oerlemans A, Lao S, Wu S, Lew MS. Deep learning for visual understanding: A review. Neurocomputing 2016; 187: 27-48.
Yokoyama Y, Katsumata T, Yasuda M. Restricted Boltzmann Machine with Multivalued Hidden Variables. Review of Socionetwork Strategies 2019; 13(2): 253-66.
Liu G, Bao H, Han B. A stacked autoencoder-based deep neural network for achieving gearbox fault diagnosis. Math Probl Eng 2018; 2018.
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014; 15(1): 1929-58.
Sun X, Nasrabadi NM, Tran TD. Supervised Deep Sparse Coding Networks for Image Classification. IEEE Trans Image Process 2019; 29: 405-18.
[http://dx.doi.org/10.1109/TIP.2019.2928121] [PMID: 31331886]
Pan Z, Yu W, Yi X, Khan A, Yuan F, Zheng Y. Recent progress on generative adversarial networks (GANs): A survey. IEEE Access 2019; 7: 36322-33.
Sherstinsky A. Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) network. Physica D 2020; 404
Turaga SC, Murray JF, Jain V, et al. Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Comput 2010; 22(2): 511-38.
[http://dx.doi.org/10.1162/neco.2009.10-08-881] [PMID: 19922289]
Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Martinez-Gonzalez P, Garcia-Rodriguez J. A survey on deep learning techniques for image and video semantic segmentation. Appl Soft Comput 2018; 70: 41-65.
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention. 234-41.
Milletari F, Navab N, Ahmadi SA. V-net: Fully convolutional neural networks for volumetric medical image segmentation. Fourth international conference on 3D vision (3DV). Oct 25; IEEE 2016; pp. 565-71.
Zhang Z, Wu C, Coleman S, Kerr D. DENSE-INception U-net for medical image segmentation. Comput Methods Programs Biomed 2020; 192
[http://dx.doi.org/10.1016/j.cmpb.2020.105395] [PMID: 32163817]
Ding Y, Li C, Yang Q, Qin Z, Qin Z. How to Improve the Deep Residual Network to Segment Multi-Modal Brain Tumor Images. IEEE Access 2019; 7: 152821-31.
Kumar GA, Sridevi PV. 3D deep learning for automatic brain MR tumor segmentation with T-spline intensity inhomogeneity correction. Autom Control Comput Sci 2018; 52(5): 439-50.
Feng X, Tustison NJ, Patel SH, Meyer CH. Brain tumor segmentation using an ensemble of 3d u-nets and overall survival prediction using radiomic features. Front Comput Neurosci 2020; 14: 25.
[http://dx.doi.org/10.3389/fncom.2020.00025] [PMID: 32322196]
Mittal M, Goyal LM, Kaur S, Kaur I, Verma A, Hemanth DJ. Deep learning based enhanced tumor segmentation approach for MR brain images. Appl Soft Comput 2019; 78: 346-54.
Sharif MI, Li JP, Khan MA, Saleem MA. Active deep neural network features selection for segmentation and recognition of brain tumors using MRI images. Pattern Recognit Lett 2020; 129: 181-9.
Yang T, Song J, Li L. A deep learning model integrating SK-TPCNN and random forests for brain tumor segmentation in MRI. Biocybern Biomed Eng 2019; 39(3): 613-23.
Deng W, Shi Q, Wang M, Zheng B, Ning N. Deep Learning-Based HCNN and CRF-RRNN Model for Brain Tumor Segmentation. IEEE Access 2020; 8: 26665-75.
Hussain S, Anwar SM, Majid M. Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing 2018; 282: 248-61.
Rebsamen M, Knecht U, Reyes M, Wiest R, Meier R, McKinley R. Divide and Conquer: Stratifying training data by tumor grade improves deep learning-based brain tumor segmentation. Front Neurosci 2019; 13: 1182.
[http://dx.doi.org/10.3389/fnins.2019.01182] [PMID: 31749678]
Nalepa J, Ribalta Lorenzo P, Marcinkiewicz M, et al. Fully-automated deep learning-powered system for DCE-MRI analysis of brain tumors. Artif Intell Med 2020; 102: 101769.
[http://dx.doi.org/10.1016/j.artmed.2019.101769] [PMID: 31980106]
Wang G, Li W, Zuluaga MA, et al. Interactive medical image segmentation using deep learning with image-specific fine tuning. IEEE Trans Med Imaging 2018; 37(7): 1562-73.
[http://dx.doi.org/10.1109/TMI.2018.2791721] [PMID: 29969407]
Laukamp KR, Thiele F, Shakirin G, et al. Fully automated detection and segmentation of meningiomas using deep learning on routine multiparametric MRI. Eur Radiol 2019; 29(1): 124-32.
[http://dx.doi.org/10.1007/s00330-018-5595-8] [PMID: 29943184]
Saba T, Mohamed AS, El-Affendi M, Amin J, Sharif M. Brain tumor detection using fusion of hand crafted and deep learning features. Cogn Syst Res 2020; 59: 221-30.
Sajid S, Hussain S, Sarwar A. Brain tumor detection and segmentation in MR images using deep learning. Arab J Sci Eng 2019; 44(11): 9249-61.
Guo Z, Li X, Huang H, Guo N, Li Q. Deep learning-based image segmentation on multimodal medical imaging. IEEE Trans Radiat Plasma Med Sci 2019; 3(2): 162-9.
Zhou T, Ruan S, Canu S. A review: Deep learning for medical image segmentation using multi-modality fusion. Array 2019; 3-4: 100004.
Angulakshmi M, Lakshmi Priya GG. Automated brain tumour segmentation techniques—a review. Int J Imaging Syst Technol 2017; 27(1): 66-77.
Angulakshmi M, Lakshmi Priya GG. Walsh Hadamard kernel‐based texture feature for multimodal MRI brain tumour segmentation. Int J Imaging Syst Technol 2018; 28(4): 254-66.
Wang G, Li W, Ourselin S, Vercauteren T. Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. International MICCAI brainlesion workshop 2017; 178-90.
Zhou C, Ding C, Lu Z, Wang X, Tao D. One-pass multi-task convolutional neural networks for efficient brain tumor segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Sep 16-20; Granada, Spain: Springer 2018; pp. 637-45.
Sun L, Zhang S, Chen H, Luo L. Brain tumor segmentation and survival prediction using multimodal MRI scans with deep learning. Front Neurosci 2019; 13: 810.
[http://dx.doi.org/10.3389/fnins.2019.00810] [PMID: 31474816]
Dolz J, Desrosiers C, Ayed IB. IVD-Net: Intervertebral disc localization and segmentation in MRI with a multi-modal UNet. International Workshop and Challenge on Computational Methods and Clinical Applications for Spine Imaging. Sep 19-21; China: Springer 2018.
Kamnitsas K, Ledig C, Newcombe VFJ, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 2017; 36: 61-78.
[http://dx.doi.org/10.1016/j.media.2016.10.004] [PMID: 27865153]
Kamnitsas K, Bai W, Ferrante E, et al. Ensembles of multiple models and architectures for robust brain tumour segmentation. International MICCAI Brainlesion Workshop. 2017 Sep 14-16; 2017.
Myronenko A. 3D MRI brain tumor segmentation using autoencoder regularization. International MICCAI Brainlesion Workshop. 2017 Sep 16-17; China: Springer 2018.
Clèrigues A, Valverde S, Bernal J, Freixenet J, Oliver A, Lladó X. Acute and sub-acute stroke lesion segmentation from multimodal MRI. Comput Methods Programs Biomed 2020; 194
[http://dx.doi.org/10.1016/j.cmpb.2020.105521] [PMID: 32434099]
Bui TD, Shin J, Moon T. 3d densely convolutional networks for volumetric segmentation. arXiv preprint 1709.
Özyurt F, Sert E, Avci E, Dogantekin E. Brain tumor detection based on Convolutional Neural Network with neutrosophic expert maximum fuzzy sure entropy. Measurement 2019; 147: 106830.
Özyurt F, Sert E, Avcı D. An expert system for brain tumor detection: Fuzzy C-means with super resolution and convolutional neural network with extreme learning machine. Med Hypotheses 2020; 134: 109433.
[http://dx.doi.org/10.1016/j.mehy.2019.109433] [PMID: 31634769]
Sert E, Özyurt F, Doğantekin A. A new approach for brain tumor diagnosis system: Single image super resolution based maximum fuzzy entropy segmentation and convolutional neural network. Med Hypotheses 2019; 133: 109413.
[http://dx.doi.org/10.1016/j.mehy.2019.109413] [PMID: 31586812]
Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural networks. science 2006; 313(5786): 504-7.
Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA, Bottou L. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 2010; 11(12:)
Nair V, Hinton GE. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27 th International Conference on Machine Learning. Jan 1-3; Haifa, Israel. 2010.
Pitchai R, Supraja P, Victoria AH, Madhavi M. Brain tumor segmentation using deep learning and fuzzy K-Means clustering for magnetic resonance images. Neural Process Lett 2020; 1-4.
Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint 1502.
Collobert R, Weston J. A unified architecture for natural language processing: Deep neural networks with multitask learning. Proceedings of the 25th international conference on Machine learning. 2008; Jul 5-7; Helsinki, Finland: ACM 2008.
Sutskever I, Martens J, Hinton GE. Generating text with recurrent neural networks. Proceedings of the 28th International Conference on Machine Learning. 2011 Jan 1-2; Bellevue, WA, USA: Elsevier 2011.
Hinton G, Deng L, Yu D, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process Mag 2012; 29(6): 82-97.
Szegedy C, Toshev A, Erhan D. Deep neural networks for object detection. Adv Neural Inf Process Syst 2013; 2553-61.
Ravì D, Wong C, Deligianni F, et al. Deep learning for health informatics. IEEE J Biomed Health Inform 2017; 21(1): 4-21.
[http://dx.doi.org/10.1109/JBHI.2016.2636665] [PMID: 28055930]
Taigman Y, Yang M, Ranzato MA, Wolf L. Deepface: Closing the gap to human-level performance in face verification. Proceedings of the IEEE conference on computer vision and pattern recognition. NW Washington, DC. IEEE 2015.
Zhang J, Zong C. Deep neural networks in machine translation: An overview. IEEE Intell Syst 2015; 5: 16-25.
Karpathy A, Fei-Fei L. Deep visual-semantic alignments for generating image descriptions. Proceedings of the IEEE conference on computer vision and pattern recognition. 2015 June 8-10; Boston, Massachusetts: IEEE 2015.
Mohsen H, El-Dahshan ES, El-Horbaty ES, Salem AB. Classification using deep learning neural networks for brain tumors. Future computing and tnformatics journal 2018; 3(1): 68-71.
Russakovsky O, Deng J, Su H, et al. Imagenet large scale visual recognition challenge. Int J Comput Vis 2015; 115(3): 211-52.
Everingham M, Winn J. The pascal visual object classes challenge 2012 (voc2012) development kit. Pattern Analysis, Statistical Modelling and Computational Learning, Tech Rep 2012; 25: 8.
Zhang W, Li R, Deng H, et al. Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Neuroimage 2015; 108: 214-24.
[http://dx.doi.org/10.1016/j.neuroimage.2014.12.061] [PMID: 25562829]
Kleesiek J, Urban G, Hubert A, et al. Deep MRI brain extraction: A 3D convolutional neural network for skull stripping. Neuroimage 2016; 129: 460-9.
[http://dx.doi.org/10.1016/j.neuroimage.2016.01.024] [PMID: 26808333]
Wu G, Kim M, Wang Q, Munsell BC, Shen D. Scalable high- performance image registration framework by unsupervised deep feature representations learning. IEEE Trans Biomed Eng 2016; 63(7): 1505-16.
[http://dx.doi.org/10.1109/TBME.2015.2496253] [PMID: 26552069]
Suk HI, Lee SW, Shen D. Alzheimer’s Disease Neuroimaging Initiative. Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. Neuroimage 2014; 101: 569-82.
[http://dx.doi.org/10.1016/j.neuroimage.2014.06.077] [PMID: 25042445]
Amin J, Sharif M, Gul N, Yasmin M, Shad SA. Brain tumor classification based on DWT fusion of MRI sequences using convolutional neural network. Pattern Recognit Lett 2020; 129: 115-22.
Shin HC, Roberts K, Lu L, Demner-Fushman D, Yao J, Summers RM. Learning to read chest x-rays: Recurrent neural cascade model for automated image annotation. Proceedings of the IEEE conference on computer vision and pattern recognition. June 26-1; Caesars Palace, Las Vegas, Nevada, United States: IEEE 2016.
Suk HI, Lee SW, Shen D. Alzheimer’s Disease Neuroimaging Initiative. Latent feature representation with stacked auto-encoder for AD/MCI diagnosis. Brain Struct Funct 2015; 220(2): 841-59.
[http://dx.doi.org/10.1007/s00429-013-0687-3] [PMID: 24363140]
Suk HI. Alzheimer’s disease Neuroimaging Initiative.. Deep learning in diagnosis of brain disordersRecent Progress in Brain and Cognitive Engineering. 1st ed.. Dordrecht:Springer Netherlands 2015; pp. 203-13.
Suk HI, Wee CY, Lee SW, Shen D. State-space model with deep learning for functional dynamics estimation in resting-state fMRI. Neuroimage 2016; 129: 292-307.
[http://dx.doi.org/10.1016/j.neuroimage.2016.01.005] [PMID: 26774612]
Pereira S, Pinto A, Alves V, Silva CA. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 2016; 35(5): 1240-51.
[http://dx.doi.org/10.1109/TMI.2016.2538465] [PMID: 26960222]
van Tulder G, de Bruijne M. Combining generative and discriminative representation learning for lung CT analysis with convolutional restricted Boltzmann machines. IEEE Trans Med Imaging 2016; 35(5): 1262-72.
[http://dx.doi.org/10.1109/TMI.2016.2526687] [PMID: 26886968]
Qi Dou , Hao Chen , Lequan Yu , et al. Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks. IEEE Trans Med Imaging 2016; 35(5): 1182-95.
[http://dx.doi.org/10.1109/TMI.2016.2528129] [PMID: 26886975]
Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J. Mitosis detection in breast cancer histology images with deep neural networks. International conference on medical image computing and computer-assisted intervention.
Chen H, Dou Q, Wang X, Qin J, Heng PA. Mitosis detection in breast cancer histology images via deep cascaded networks. 13th AAAI conference on artificial intelligence. 2016 Feb 12-17; Phoenix, Arizon, USA. AAAI 2016.
Cheng JZ, Ni D, Chou YH, et al. Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci Rep 2016; 6(1): 24454.
[http://dx.doi.org/10.1038/srep24454] [PMID: 27079888]
Roth HR, Lu L, Liu J, et al. Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging 2016; 35(5): 1170-81.
[http://dx.doi.org/10.1109/TMI.2015.2482920] [PMID: 26441412]
Shen W, Zhou M, Yang F, Yang C, Tian J. Multi-scale convolutional neural networks for lung nodule classification. International Conference on Information Processing in Medical Imaging. 2015 Jun 28- Jul 3; 2015.
Setio AA, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging 2016; 35(5): 1160-9.
[http://dx.doi.org/10.1109/TMI.2016.2536809] [PMID: 26955024]
Ciompi F, et al. Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of the-box. Med Image Anal 2015; 26(1): 195-202.
[http://dx.doi.org/10.1016/j.media.2015.08.001] [PMID: 26458112]
Li R, Zhang W, Suk HI, et al. Deep learning based imaging data completion for improved brain disease diagnosis. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2014 Sep 14-18; Boston, USA: Springer 2014.
Shin HC, Roth HR, Gao M, et al. Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning. IEEE Trans Med Imaging 2016; 35(5): 1285-98.
[http://dx.doi.org/10.1109/TMI.2016.2528162] [PMID: 26886976]
Gupta A, Ayhan M, Maida A. Natural image bases to represent neuroimaging data. International conference on machine learning. June 16-21;; Atlanta, USA. ACM 2013.
Brosch T, Tam R. Alzheimer’s Disease Neuroimaging Initiative. Manifold learning of brain MRIs by deep learning. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2013 Sep 22; Berlin, Heidelberg: Springer 2013.
Nie D, Wang L, Gao Y, Shen D. Fully convolutional networks for multi-modality isointense infant brain image segmentation. 2016 IEEE 13Th international symposium on biomedical imaging (ISBI).
Csurka G, Larlus D, Perronnin F, Meylan F. What is a good evaluation measure for semantic segmentation?. BMVC 2013.
Taha AA, Hanbury A. Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging 2015; 15(1): 29.
[http://dx.doi.org/10.1186/s12880-015-0068-x] [PMID: 26263899]
Costa H, Foody GM, Boyd DS. Supervised methods of image segmentation accuracy assessment in land cover mapping. Remote Sens Environ 2018; 205: 338-51.
Mendrik AM, Vincken KL, Kuijf HJ, et al. MRBrainS challenge: online evaluation framework for brain image segmentation in 3T MRI scans. Comput Intell Neurosci 2015; 2015
[http://dx.doi.org/10.1155/2015/813696] [PMID: 26759553]

Rights & PermissionsPrintExport Cite as

Article Details

Year: 2021
Published on: 08 January, 2021
Page: [695 - 706]
Pages: 12
DOI: 10.2174/1573405616666210108122048
Price: $65

Article Metrics

PDF: 34