Generic placeholder image

Current Medical Imaging


ISSN (Print): 1573-4056
ISSN (Online): 1875-6603

Review Article

Recent Advancements in Multimodal Medical Image Fusion Techniques for Better Diagnosis: An Overview

Author(s): Maruturi Haribabu, Velmathi Guruviah* and Pratheepan Yogarajah

Volume 19, Issue 7, 2023

Published on: 21 September, 2022

Page: [673 - 694] Pages: 22

DOI: 10.2174/1573405618666220606161137

Price: $65


Medical imaging plays a vital role in medical diagnosis and clinical treatment. The biggest challenge in the medical field is the correct identification of disease and better treatment. Multi-modal Medical Image Fusion (MMIF) is the process of merging multiple medical images from different modalities into a single fused image. The main objective of the medical image fusion is to obtain a large amount of appropriate information (i.e., features) to improve the quality and make it more informative for increasing clinical therapy for better diagnosis and clear assessment of medical-related problems. The MMIF is generally considered with MRI (Magnetic Resonance Imaging), CT (Computed Tomography), PET (Positron Emission Tomography), SPECT (Single Photon Emission Computed Tomography), MRA (Magnetic Resonance Angiography), T1-weighted MR, T2-weighted MR, X-ray, and ultrasound imaging (Vibro-Acoustography). This review article presents a comprehensive survey of existing medical image fusion methods and has been characterized into six parts: (1) Multi-modality medical images, (2) Literature review process, (3) Image fusion rules, (4) Quality evaluation metrics for assessment of fused image, (5) Experimental results on registered datasets and (6) Conclusion. In addition, this review article provides scientific challenges faced in MMIF and future directions for better diagnosis. It is expected that this review will be useful in establishing a concrete foundation for developing more valuable fusion methods for medical diagnosis.

Keywords: Multi-modality, medical images, medical image fusion, diagnosis, image fusion rules, quality assessment metrics.

Du J, Li W, Lu K, Xiao B. An overview of multi-modal medical image fusion. Neurocomputing 2016; 215: 3-20.
Azam MA, Khan KB, Ahmad M, Mazzara M. Multimodal medical image registration and fusion for quality enhancement. Comput Mater Continua 2021; 68(2021): 821-40.
Kaur H, Koundal D, Kadyan V. Image fusion techniques: A survey. Arch Comput Methods Eng 2021; 28(7): 4425-47.
[] [PMID: 33519179]
Hermessi H, Mourali O, Zagrouba E. Multimodal medical image fusion review: Theoretical background and recent advances. Signal Processing 2021; 183: 108036.
Tawfik N, Elnemr HA, Fakhr M, Dessouky MI, El-Samie A, Fathi E. Survey study of multimodality medical image fusion methods. Multimedia Tools Appl 2021; 80(4): 6369-96.
Swathi PS, Sheethal MS, Paul V. Survey on multimodal medical image fusion techniques. Int J Sci Eng Comput Technol 2016; 6(1): 33.
Li Y, Zhao J, Lv Z, Li J. Medical image fusion method by deep learning. Int J Cogn Comput Eng 2021; 2: 21-9.
Azam MA, Khan KB, Salahuddin S, et al. A review on multimodal medical image fusion: Compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metrics. Comput Biol Med 2022; 144: 105253.
[] [PMID: 35245696]
Heba M, Rabaieb ES, Elrahmana WA, Faragallahc OS, El-Samieb FE. Medical image fusion: A literature review present solutions and future directions. Minufiya J Electron Eng Res 2017; 26(2): 1-62.
Ramandeep RK. Review on different aspects of image fusion for medical imaging. Int J Sci Res 2014; 3(5): 1887-9.
James AP, Dasarathy BV. Medical image fusion: A survey of the state of the art. Inf Fusion 2014; 19: 4-19.
Huang B, Yang F, Yin M, Mo X, Zhong C. A review of multimodal medical image fusion techniques. Comput Math Methods Med 2020; 2020: 8279342.
[] [PMID: 32377226]
El-Gamal FE, Elmogy M, Atwan A. Current trends in medical image registration and fusion. Egyptian Inform J 2016; 17(1): 99-124.
Tirupal T, Mohan BC, Kumar SS. Multimodal medical image fusion techniques–A review. Curr Signal Transduct Ther 2021; 16(2): 142-63.
Meher B, Agrawal S, Panda R, Abraham A. A survey on region based image fusion methods. Inf Fusion 2019; 48: 119-32.
Narsaiah MN, Vathsal S, Reddy DV. A survey on image fusion requirements, techniques, evaluation metrics, and its applications. Int J Eng Technol 2018; 7(2.20): 260-6.
Rockinger O. Image fusion toolbox for Matlab. Technical report, Metapix 1999. Available from:
Johnson KA, Becker JA. The Whole Brain Atlas. Available from:
Durga Prasad Bavirisetti. Medical Imaging datasets. Available from: (Accessed on: 18-08-2020).
He C, Liu Q, Li H, Wang H. Multimodal medical image fusion based on IHS and PCA. Proc Eng 2010; 7: 280-5.
Daneshvar S, Ghassemian H. MRI and PET image fusion by combining IHS and retina-inspired models. Inf Fusion 2010; 11(2): 114-23.
Haddadpour M, Daneshvar S, Seyedarabi H. PET and MRI image fusion based on combination of 2-D Hilbert transform and IHS method. Biomed J 2017; 40(4): 219-25.
[] [PMID: 28918910]
Du J, Li W, Xiao B, Nawaz Q. Union Laplacian pyramid with multiple features for medical image fusion. Neurocomputing 2016; 194: 326-39.
Krishnamoorthy S, Soman KP. Implementation and comparative study of image fusion algorithms. Int J Comput Appl 2010; 9(2): 25-35.
Yang Y, Park DS, Huang S, Rao N. Medical image fusion via an effective wavelet-based approach. EURASIP J Adv Signal Process 2010; 2010: 1-3.
Singh R, Khare A. Multiscale medical image fusion in wavelet domain. Sci World J 2013; 2013: 521034.
Suraj AA, Francis M, Kavya TS, Nirmal TM. Discrete wavelet transform based image fusion and de-noising in FPGA. J Electrical Sys Inform Technol 2014; 1(1): 72-81.
Chandra SJ, Babu AN, Rao GS, et al. Medical fusion image using wavelet transformation. Int J Innov Technol Explor Eng 2019; 8(8): 1864-6.
Gomathi PS, Kalaavathi B. Medical image fusion based on redundant wavelet transform and morphological processing. Int J Comput Inform Eng 2014; 8(6): 1018-22.
Yadav HN. Multimodal medical image fusion for computer aided diagnosis. Comput Trendz 2015; 5(1 & 2): 21-5.
Wang HH. A new multiwavelet-based approach to image fusion. J Math Imaging Vis 2004; 21(2): 177-92.
Wang X, Shen Y, Zhou Z, Fang L. An image fusion algorithm based on lifting wavelet transform. J Opt 2015; 17(5): 055702.
El-Hoseny HM, Abd El-Rahman W, El-Rabaie ES, Abd El-Samie FE, Faragallah OS. An efficient DT-CWT medical image fusion system based on modified central force optimization and histogram matching. Infrared Phys Technol 2018; 94: 223-31.
Alipour SHM, Houshyari M, Mostaar A. A novel algorithm for PET and MRI fusion based on digital curvelet transform via extracting lesions on both images. Electron Physician 2017; 9(7): 4872-9.
[] [PMID: 28894548]
Yang L, Guo BL, Ni W. Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 2008; 72(1-3): 203-11.
Huang H, Feng XA, Jiang J. Medical image fusion algorithm based on nonlinear approximation of contourlet transform and regional features. J Electr Comput Eng 2017; 2017: 6807473.
Bhatnagar G, Wu QJ, Liu Z. Directive contrast based multimodal medical image fusion in NSCT domain. IEEE Trans Multimed 2013; 15(5): 1014-24.
Yang G, Li M, Chen L, Yu J. The nonsubsampled contourlet transform based statistical medical image fusion using generalized Gaussian density. Comput Math Methods Med 2015; 2015: 262819.
[] [PMID: 26557871]
Bhatnagar G, Wu QJ, Liu Z. A new contrast based multimodal medical image fusion framework. Neurocomputing 2015; 157: 143-52.
Gomathi PS, Kalaavathi B. Multimodal medical image fusion in non-subsampled contourlet transform domain. Circuits Sys 2016; 7(08): 1598.
Zhu Z, Zheng M, Qi G, Wang D, Xiang Y. A phase congruency and local Laplacian energy based multi-modality medical image fusion method in NSCT domain. IEEE Access 2019; 7: 20811-24.
Miao QG, Shi C, Xu PF, Yang M, Shi YB. A novel algorithm of image fusion using shearlets. Opt Commun 2011; 284(6): 1540-7.
Ahmed N. Medical image fusion based on shearlets and human feature visibility. Int J Comput Appl 2015; 125(12): 7-12.
Biswas B, Sen BK. Color PET-MRI medical image fusion combining matching regional spectrum in shearlet domain. Int J Image Graph 2019; 19(01): 1950004.
Xiaoxue X, Fucheng C, Weiwei S, Fu L. Multi-modal medical image fusion based on non-subsampled Shearlet Transform. Int J Signal Process Image Process Pattern Recogn 2015; 8(2): 41-8.
Singh S, Anand RS. Multimodal neurological image fusion based on adaptive biological inspired neural model in nonsubsampled shearlet domain. Int J Imaging Syst Technol 2019; 29(1): 50-64.
Yin M, Liu X, Liu Y, Chen X. Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain. IEEE Trans Instrum Meas 2018; 68(1): 49-64.
Li S, Yin H. Multimodal image fusion with joint sparsity model. Opt Eng 2011; 50(6): 067007.
Shreyamsha Kumar BK. Image fusion based on pixel significance using cross bilateral filter. Signal Image Video Process 2015; 9(5): 1193-204.
Bavirisetti DP, Kollu V, Gang X, Dhuli R. Fusion of MRI and CT images using guided image filter and image statistics. Int J Imaging Syst Technol 2017; 27(3): 227-37.
Bavirisetti DP, Xiao G, Zhao J, Dhuli R, Liu G. Multi-scale guided image and video fusion: A fast and efficient approach. Circuits Syst Signal Process 2019; 38(12): 5576-605.
Jiang Y, Wang M. Image fusion using multiscale edge-preserving decomposition based on weighted least squares filter. IET Image Process 2014; 8(3): 183-90.
Jian L, Yang X, Zhou Z, Zhou K, Liu K. Multi-scale image fusion through rolling guidance filter. Future Gener Comput Syst 2018; 83: 310-25.
Zhang Y, Li D, Zhang R, Cui Y. Sparse features with fast guided filtering for medical image fusion. J Med Imaging Health Inform 2020; 10(5): 1195-204.
Yadav SS, Jadhav SM. Deep convolutional neural network based medical image classification for disease diagnosis. J Big Data 2019; 6(1): 1-8.
Kermany DS, Goldbaum M, Cai W, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 2018; 172(5): 1122-1131.e9.
[] [PMID: 29474911]
Hesamian MH, Jia W, He X, Kennedy P. Deep learning techniques for medical image segmentation: Achievements and challenges. J Digit Imaging 2019; 32(4): 582-96.
[] [PMID: 31144149]
Zhou T, Ruan S, Canu S. A review: Deep learning for medical image segmentation using multi-modality fusion. Array 2019; 3: 100004.
Chen C, Qin C, Qiu H, et al. Deep learning for cardiac image segmentation: A review. Front Cardiovasc Med 2020; 7: 25.
[] [PMID: 32195270]
Liu Y, Chen X, Peng H, Wang Z. Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 2017; 36: 191-207.
Hermessi H, Mourali O, Zagrouba E. Convolutional neural network-based multimodal image fusion via similarity learning in the shearlet domain. Neural Comput Appl 2018; 30(7): 2029-45.
Hou R, Zhou D, Nie R, Liu D, Ruan X. Brain CT and MRI medical image fusion using convolutional neural networks and a dual-channel spiking cortical model. Med Biol Eng Comput 2019; 57(4): 887-900.
[] [PMID: 30471068]
Wang M, Liu X, Jin H. A generative image fusion approach based on supervised deep convolution network driven by weighted gradient flow. Image Vis Comput 2019; 86: 1-6.
Haskins G, Kruger U, Yan P. Deep learning in medical image registration: A survey. Mach Vis Appl 2020; 31(1): 1-8.
Ismail WZ, Sim KS. Contrast enhancement dynamic histogram equalization for medical image processing application. Int J Imaging Syst Technol 2011; 21(3): 280-9.
Maini R, Aggarwal H. A comprehensive review of image enhancement techniques. arXiv 2010; 2010: 1003.4053.
Lotfi Zadeh A. Fuzzy set. Inf Control 1965; 8(3): 338-53.
Atanasov KT. Intuitionistic fuzzy sets. Fuzzy Sets Syst 1986; 20(1): 87-96.
Sanjay AR, Soundrapandiyan R, Karuppiah M, Ganapathy RCT. International Journal of Intelligent Engineering and Systems 2017; 10(3): 355-62.
Balasubramaniam P, Ananthi VP. Image fusion using intuitionistic fuzzy sets. Inf Fusion 2014; 20: 21-30.
Aysha S, Tirupal T. Image fusion of medical images based on Fuzzy set. Elixir Digital Processing 2016; 96: 41225-8.
Soundrapandiyan R, Haldar R, Purushotham S, Pillai A. Multimodality medical image fusion using block based intuitionistic fuzzy sets. IIOAB J 2016; 7(5): 85-94.
Soundrapandiyan R, Karuppiah M, Kumari S, Kumar Tyagi S, Wu F, Jung KH. An efficient DWT and intuitionistic fuzzy based multimodality medical image fusion. Int J Imaging Syst Technol 2017; 27(2): 118-32.
Kumar M, Kaur A. Amita. Improved image fusion of colored and grayscale medical images based on intuitionistic fuzzy sets. Fuzzy Inform Eng 2018; 10(2): 295-306.
Tirupal T, Mohan BC, Kumar SS. Multimodal medical image fusion based on Sugeno’s intuitionistic fuzzy sets. ETRI J 2017; 39(2): 173-80.
Tirupal T, Chandra Mohan B, Srinivas Kumar S. Multimodal medical image fusion based on yager’s intuitionistic fuzzy sets. Iran J Fuzzy Sys 2019; 16(1): 33-48.
Agarwal J, Bedi SS. Implementation of hybrid image fusion technique for feature enhancement in medical diagnosis. Human-centric Comput. Inf Sci 2015; 5(1): 1-7.
Dai Y, Zhou Z, Xu L. The application of multi-modality medical image fusion based method to cerebral infarction. EURASIP J Image Video Process 2017; 2017(1): 1-6.
Zong JJ, Qiu TS. Medical image fusion based on sparse representation of classified image patches. Biomed Signal Process Control 2017; 34: 195-205.
Aktar MN, Lambert AJ, Pickering M. An automatic fusion algorithm for multi-modal medical images. Comput Methods Biomech Biomed Eng Imaging Vis 2018; 6(5): 584-98.
Daniel E, Anitha J, Kamaleshwaran KK, Rani I. Optimum spectrum mask based medical image fusion using Gray Wolf Optimization. Biomed Signal Process Control 2017; 34: 36-43.
Xia J, Chen Y, Chen A, Chen Y. Medical image fusion based on sparse representation and PCNN in NSCT domain. Comput Math Methods Med 2018; 2018: 2806047.
[] [PMID: 29991960]
Daniel E. Optimum wavelet-based homomorphic medical image fusion using hybrid genetic–grey wolf optimization algorithm. IEEE Sens J 2018; 18(16): 6804-11.
Tan L, Yu X. Medical image fusion based on fast finite shearlet transform and sparse representation. Comput Math Methods Med 2019; 2019: 3503267.
[] [PMID: 30944576]
Hassan M, Murtza I, Zafar Khan MA, Tahir SF, Fahad LG. Neuro‐wavelet based intelligent medical image fusion. Int J Imaging Syst Technol 2019; 29(4): 633-44.
Ramlal SD, Sachdeva J, Ahuja CK, Khandelwal N. An improved multimodal medical image fusion scheme based on hybrid combination of nonsubsampled contourlet transform and stationary wavelet transform. Int J Imaging Syst Technol 2019; 29(2): 146-60.
Rajalingam B, Priya R, Bhavani R. Medical image fusion based on hybrid algorithms for neuro cysticercosis and neoplastic disease analysis. IMCMS 2019; 2019: 15.
Huang C, Tian G, Lan Y, et al. A new pulse coupled neural network (PCNN) for brain medical image fusion empowered by shuffled frog leaping algorithm. Front Neurosci 2019; 13: 210.
[] [PMID: 30949018]
Li J, Peng Y, Song M, Liu L. Image fusion based on guided filter and online robust dictionary learning. Infrared Phys Technol 2020; 105: 103171.
El-Hoseny HM, Abd El-Rahman W, El-Shafai W, et al. Efficient multi-scale non-sub-sampled shearlet fusion system based on modified central force optimization and contrast enhancement. Infrared Phys Technol 2019; 102: 102975.
Hu Q, Hu S, Zhang F. Multi-modality medical image fusion based on separable dictionary learning and Gabor filtering. Signal Process Image Commun 2020; 83: 115758.
Parvathy VS, Pothiraj S. Multi-modality medical image fusion using hybridization of binary crow search optimization. Health Care Manage Sci 2020; 23(4): 661-9.
[] [PMID: 31292844]
Xu L, Si Y, Jiang S, Sun Y, Ebrahimian H. Medical image fusion using a modified shark smell optimization algorithm and hybrid wavelet-homomorphic filter. Biomed Signal Process Control 2020; 59: 101885.
Liu Y, Zhou D, Nie R, et al. Robust spiking cortical model and total-variational decomposition for multimodal medical image fusion. Biomed Signal Process Control 2020; 61: 101996.
Maqsood S, Javed U. Multi-modal medical image fusion based on two-scale image decomposition and sparse representation. Biomed Signal Process Control 2020; 57: 101810.
Ding Z, Zhou D, Nie R, Hou R, Liu Y. Brain medical image fusion based on dual-branch CNNs in NSST domain. BioMed Res Int 2020; 2020: 6265708.
[] [PMID: 32352003]
Xia J, Lu Y, Tan L. Research of multimodal medical image fusion based on parameter-adaptive pulse-coupled neural network and convolutional sparse representation. Comput Math Methods Med 2020; 2020: 3290136.
[] [PMID: 32411280]
Padmavathi K, Asha CS, Maya VK. A novel medical image fusion by combining TV-L1 decomposed textures based on adaptive weighting scheme. Eng Sci Technol 2020; 23(1): 225-39.
Liu Y, Chen X, Ward RK, Wang ZJ. Medical image fusion via convolutional sparsity based morphological component analysis. IEEE Signal Process Lett 2019; 26(3): 485-9.
Kaur K, Budhiraja S, Sharma N. Multimodal Medical Image Fusion based on Gray Wolf Optimization and Hilbert Transform. Biomed Pharmacol J 2019; 12(4): 2091-8.
Singh S, Anand RS. Multimodal medical image fusion using hybrid layer decomposition with CNN-based feature mapping and structural clustering. IEEE Trans Instrum Meas 2019; 69(6): 3855-65.
Reena Benjamin J, Jayasree T. Improved medical image fusion based on cascaded PCA and shift invariant wavelet transforms. Int J CARS 2018; 13(2): 229-40.
[] [PMID: 29250750]
Yadav SP, Yadav S. Image fusion using hybrid methods in multimodality medical images. Med Biol Eng Comput 2020; 58(4): 669-87.
[] [PMID: 31993885]
Jany Shabu SL, Jayakumar C. Detection of brain tumour by image fusion using SVM classifier. Comput Eng Intell Sys 2017; 8(7): 18-22.
Saboori A, Birjandtalab J. PET–MRI image fusion using adaptive filter based on spectral and spatial discrepancy. Signal Image Video Process 2019; 13(1): 135-43.
Li Y, Jiang Y, Gao L, Fan Y. Fast mutual modulation fusion for multi-sensor images. Optik (Stuttg) 2015; 126(1): 107-11.
Xu Z. Medical image fusion using multi-level local extrema. Inf Fusion 2014; 19: 38-48.
Liu Z, Song Y, Sheng VS, et al. MRI and PET image fusion using the nonparametric density model and the theory of variable-weight. Comput Methods Programs Biomed 2019; 175: 73-82.
[] [PMID: 31104716]
Lu H, Zhang L, Serikawa S. Maximum local energy: An effective approach for multisensor image fusion in beyond wavelet transform domain. Comput Math Appl 2012; 64(5): 996-1003.
Li X, Zhang X, Ding M. A sum-modified-Laplacian and sparse representation based multimodal medical image fusion in Laplacian pyramid domain. Med Biol Eng Comput 2019; 57(10): 2265-75.
[] [PMID: 31410692]
Aishwarya N, Bennila Thangammal C. A novel multimodal medical image fusion using sparse representation and modified spatial frequency. Int J Imaging Syst Technol 2018; 28(3): 175-85.
Liu D, Chen X, Peng D. Cosine similarity measure between hybrid intuitionistic fuzzy sets and its application in medical diagnosis. Comput Math Methods Med 2018; 2018: 3146873.
[] [PMID: 30416537]
Gambhir D, Manchanda M. A novel fusion rule for medical image fusion in complex wavelet transform domain. Int J Image Graph 2016; 16(04): 1650022.
Yang Y, Tong S, Huang S, Lin P. Dual-tree complex wavelet transform and image block residual-based multi-focus image fusion in visual sensor networks. Sensors (Basel) 2014; 14(12): 22408-30.
[] [PMID: 25587878]
XZheng Y. Blasch E, Liu Z Multispectral image fusion and colorization. Bellingham, Washington: SPIE Press 2018; pp. 230-2.
Zhang L, Zhang L, Mou X, Zhang D. FSIM: A feature similarity index for image quality assessment. IEEE Trans Image Process 2011; 20(8): 2378-86.
[] [PMID: 21292594]
Zhan K, Li Q, Teng J, Wang M, Shi J. Multifocus image fusion using phase congruency. J Electron Imaging 2015; 24(3): 033014.
Naidu VP, Raol JR. Pixel-level image fusion using wavelets and principal component analysis. Def Sci J 2008; 58(3): 338.
Al-Wassai FA, Kalyankar NV, Al-Zaky AA. Studying satellite image quality based on the fusion techniques. arXiv 2011; 2011: 1110.4970.
Mhangara P, Mapurisa W, Mudau N. Comparison of image fusion techniques using satellite pour l’Observation de la Terre (SPOT) 6 satellite imagery. Appl Sci (Basel) 2020; 10(5): 1881.
Memon F, Unar MA, Memon S. Image quality assessment for performance evaluation of focus measure operators. Mehran Univ Res J Eng Technol 2015; 34(4): 379-86.
Thakur KV, Damodare OH, Sapkal AM. Identification of suited quality metrics for natural and medical images. Signal Image Process. Int J 2016; 7(3): 29-43.
Singh R, Khare A. Fusion of multimodal medical images using Daubechies complex wavelet transform–A multiresolution approach. Inf Fusion 2014; 19: 49-60.
Dammavalam SR, Maddala S, Prasad MH. Quality assessment of pixel-level imagefusion using fuzzy logic. Int J Soft Comput 2013; 3(1): 13-25.
Pistonesi S, Martinez J, Ojeda SM, Vallejos R. Structural similarity metrics for quality image fusion assessment: Algorithms. Image Process On Line 2018; 8: 345-68.
Aslantas V, Bendes E. A new image quality metric for image fusion: The sum of the correlations of differences. AEU Int J Electron Commun 2015; 69(12): 1890-6.
Yang Y, Zheng W, Huang S. Effective multifocus image fusion based on HVS and BP neural network. Sci World J 2014; 2014: 281073.
[] [PMID: 24683327]

Rights & Permissions Print Export Cite as
© 2023 Bentham Science Publishers | Privacy Policy