Review of image similarity measures for joint image reconstruction from multiple measurements. (English) Zbl 1487.65138
Kaltenbacher, Barbara (ed.) et al., Time-dependent problems in imaging and parameter identification. Cham: Springer. 267-286 (2021).
Summary: It is fundamental in image processing how to measure image similarity quantitatively for tasks such as image quality assessment, image registration, image reconstruction from multiple measurements, etc.. An image similarity measure (ISM) is both task-dependent and feature-dependent, and must be designed according to the characteristics of specific tasks and features. Simply applying distances from the mathematical metric theory or general divergences to spaces of images or spaces of image features usually does not provide appropriate ISMs. In this chapter, we review several ISMs for image reconstruction problems from multiple measurements of various types in recent work. The multiple measurements considered here include multi-modality, multi-spectral, and multi-temporal measurements, with multi-modality tomography, multi-spectral XCT, and dynamic tomography, as the imaging applications, respectively. We focus on motivations and constructions of the ISMs and avoid their general rigorous mathematical presentations to simplify notations for the readability for a general audience. ISMs under review are proposed for image structural similarity and have been successfully applied to image reconstruction from multiple measurements.
For the entire collection see [Zbl 1471.65006].
For the entire collection see [Zbl 1471.65006].
MSC:
| 65M32 | Numerical methods for inverse problems for initial value and initial-boundary value problems involving PDEs |
| 65K10 | Numerical optimization and variational techniques |
| 78A46 | Inverse problems (including inverse scattering) in optics and electromagnetic theory |
| 92C55 | Biomedical imaging and signal processing |
| 94A08 | Image processing (compression, reconstruction, etc.) in information and communication theory |
| 35R30 | Inverse problems for PDEs |
References:
| [1] | G. Wrinkler, Image Analysis, Random Fields and Dynamic Monte Carlo Methods(Springer, Berlin, 1995) · Zbl 0821.68125 |
| [2] | J.M. Bernardo, A.F.M. Smith, Statistical Decision Theory and Bayesian Analysis(Wiley, New York, 2000) · Zbl 0943.62009 |
| [3] | Y. Zhou, X. Bai, W.Y. Liu, L.J. Latecki, Similarity fusion for visual tracking. Int. J. Comput. Vis. 118(3), 337-363 (2016) |
| [4] | J.S. Yang, H.Y. Yu, M. Jiang, G. Wang, High-order total variation minimization for interior SPECT. Inverse Problems 28(1), 015001 (2012) · Zbl 1234.92050 |
| [5] | J.S. Yang, H.Y. Yu, M. Jiang, G. Wang, High-order total variation minimization for interior tomography. Inverse Problems 26(3), 035013 (2010) · Zbl 1195.65218 |
| [6] | Wikipedia contributors: Matrix norm — Wikipedia, the free encyclopedia (2019) |
| [7] | Y. Wang, X. Jiang, B. Yu, M. Jiang, A hierarchical Bayesian approach for aerosol retrieval using MISR data. J. Am. Stat. Assoc. 108(502), 483-493 (2013) · Zbl 06195954 |
| [8] | G. Wang, M. Kalra, V. Murugan, Y. Xi, L. Gjesteby, M. Getzin, Q.S. Yang, W.X. Cong, M. Vannier, Vision 20/20: simultaneous CT-MRI - next chapter of multimodality imaging. Med. Phys. 42(10), 5879-5889 (2015) |
| [9] | R.C. Veltkamp, M. Hagedoorn, Shape similarity measures, properties and constructions, in Advances in Visual Information Systems, Proceedings, ed. by R. Laurini. Lecture Notes in Computer Science (2000), pp. 467-476 |
| [10] | A. Tversky, Features of similarity. Psychol. Rev. 84(4), 327-352 (1977) |
| [11] | T. Thireou, G. Kontaxakis, L.G. Strauss, A. Dimitrakopoulou-Strauss, S. Pavlopoulos, A. Santos, Feasibility study of the use of similarity maps in the evaluation of oncological dynamic positron emission tomography images. Med. Biol. Eng. Comput. 43(1), 23-32 (2005) |
| [12] | R. Temam, G. Strang, Functions of bounded deformation. Arch. Ration. Mech. Anal. 75(1), 7-21 (1980) · Zbl 0472.73031 |
| [13] | A. Sotiras, C. Davatzikos, N. Paragios, Deformable medical image registration: a survey. IEEE Trans. Med. Imag. 32(7), 1153-1190 (2013) |
| [14] | S. Somayajula, C. Panagiotou, A. Rangarajan, Q.Z. Li, S.R. Arridge, R.M. Leahy, PET image reconstruction using information theoretic anatomical priors. IEEE Trans. Med. Imag. 30(3), 537-549 (2011) |
| [15] | R.N. Shepard, Toward a universal law of generalization for psychological science. Science 237(4820), 1317-1323 (1987) · Zbl 1226.91051 |
| [16] | G. Schramm, M. Holler, A. Rezaei, K. Vunckx, F. Knoll, K. Bredies, F. Boada, J. Nuyts, Evaluation of parallel level sets and Bowsher’s method as segmentation-free anatomical priors for time-of-flight PET reconstruction. IEEE Trans. Med. Imag. 37(2), 590-603 (2018) |
| [17] | U. Schmitt, A.K. Louis, C. Wolters, M. Vauhkonen, Efficient algorithms for the regularization of dynamic inverse problems: II. Applications. Inverse Problems 18(3), 659-676 (2002) · Zbl 1003.65050 |
| [18] | U. Schmitt, A.K. Louis, Efficient algorithms for the regularization of dynamic inverse problems: I. theory. Inverse Problems 18(3), 645-658 (2002) · Zbl 1003.65049 |
| [19] | J.P. Schlomka, E. Roessl, R. Dorscheid, S. Dill, G. Martens, T. Istel, C. Baumer, C. Herrmann, R. Steadman, G. Zeitler, A. Livne, R. Proksa, Experimental feasibility of multi-energy photon-counting K-edge imaging in pre-clinical computed tomography. Phys. Med. Biol. 53(15), 4031-4047 (2008) |
| [20] | M. Schloegl, M. Holler, A. Schwarzl, K. Bredies, R. Stollberger, Infimal convolution of total generalized variation functionals for dynamic MRI. Magn. Reson. Med. 78(1), 142-155 (2017) |
| [21] | S. Santini, R. Jain, Similarity measures. IEEE Trans. Pattern Anal. Mach. Intell. 21(9), 871-883 (1999) |
| [22] | E. Roessl, R. Proksa, K-edge imaging in x-ray computed tomography using multi-bin photon counting detectors. Phys. Med. Biol. 52(15), 4679-4696 (2007) |
| [23] | D.S. Rigie, P.J. La Riviere, Joint reconstruction of multi-channel, spectral CT data via constrained total nuclear variation minimization. Phys. Med. Biol. 60(5), 1741-1762 (2015) |
| [24] | J. Rasch, E.M. Brinkmann, M. Burger, Joint reconstruction via coupled Bregman iterations with applications to PET-MR imaging. Inverse Problems 34(1), 014001 (2018) · Zbl 1381.92057 |
| [25] | C. Pöschl, O. Scherzer, Distance measures and applications to multimodal variational imaging, in Handbook of Mathematical Methods in Imaging(Springer, New York, 2015), pp. 125-155 · Zbl 1331.65052 |
| [26] | J.P.W. Pluim, J.B.A. Maintz, M.A. Viergever, f-information measures in medical image registration. IEEE Trans. Med. Imag. 23(12), 1508-1516 (2004) |
| [27] | J.P.W. Pluim, J.B.A. Maintz, M.A. Viergever, Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imag. 22(8), 986-1004 (2003) |
| [28] | J.P.W. Pluim, J.B.A. Maintz, M.A. Viergever, Image registration by maximization of combined mutual information and gradient information. IEEE Transa. Med. Imag. 19(8), 809-814 (2000) |
| [29] | C. Panagiotou, S. Somayajula, A.P. Gibson, M. Schweiger, R.M. Leahy, S.R. Arridge, Information theoretic regularization in diffuse optical tomography. J. Opt. Soc. Am. A-Opt. Image Sci. Vis. 26(5), 1277-1290 (2009) |
| [30] | T.S. Page, Image reconstruction by Mumford-Shah regularization with a priori edge information. Ph.D. Thesis, Universität Bremen (2015) |
| [31] | M. Moeller, E.M. Brinkmann, M. Burger, T. Seybold, Color Bregman TV. SIAM J. Imag. Sci. 7(4), 2771-2806 (2014) · Zbl 1361.94017 |
| [32] | J. Liu, X. Zhang, X. Zhang, H. Zhao, Y. Gao, D. Thomas, D.A. Low, H. Gao, 5D respiratory motion model based image reconstruction algorithm for 4D cone-beam computed tomography. Inverse Problems 31(11), 115007 (2015) · Zbl 1344.68264 |
| [33] | Y. Li, L. Hu, K. Xia, J. Luo, Fast distributed video deduplication via locality-sensitive hashing with similarity ranking. EURASIP J. Image Video Process. 2019, 1-11 (2019) |
| [34] | D. Li, T. Jiang, M. Jiang, Recent advances and challenges in video quality assessment. ZTE Commun. 17(01), 3-11 (2019) |
| [35] | S. Lefkimmiatis, A. Roussos, M. Unser, P. Maragos, Convex generalizations of total variation based on the structure tensor with applications to inverse problems, in Scale Space and Variational Methods in Computer Vision: SSVM 2013. Lecture Notes in Computer Science (Springer, Berlin, 2013), pp. 48-60 |
| [36] | F. Knoll, M. Holler, T. Koesters, R. Otazo, K. Bredies, D.K. Sodickson, Joint MR-PET reconstruction using a multi-channel image regularizer. IEEE Trans. Med. Imag. 36(1), 1-16 (2017) |
| [37] | D. Kazantsev, G. Van Eyndhoven, W.R.B. Lionheart, P.J. Withers, K.J. Dobson, S.A. McDonald, R., Atwood, P.D. Lee, Employing temporal self-similarity across the entire time domain in computed tomography reconstruction. Philos. Trans. R. Soc. A-Math. Phys. Eng. Sci. 373(2043), 20140389 (2015) · Zbl 1353.68285 |
| [38] | D. Kazantsev, J.S. Jorgensen, M.S. Andersen, W.R.B. Lionheart, P.D. Lee, P.J. Withers, Joint image reconstruction method with correlative multi-channel prior for x-ray spectral computed tomography. Inverse Problems 34(6), 064001 (2018) · Zbl 1441.94015 |
| [39] | H. Kalinić, S. Lonĉarić, B. Bijnens, Absolute joint moments: a novel image similarity measure. EURASIP J. Image Video Process. 2013(1), 24 (2013) |
| [40] | B. Jin, Personal communications (2019) |
| [41] | M. Jiang, P. Maass, T. Page, Regularizing properties of the Mumford-Shah functional for imaging applications. Inverse Problems 30(3), 035007 (2014) · Zbl 1293.94016 |
| [42] | M. Jiang, Perspectives of similarity measures for joint multimodality image reconstruction, in Wokshop on Tomographic Inverse Problems: Theory and Applications(2019) |
| [43] | M. Jiang, Perspectives of similarity measures for joint multimodality image reconstruction, in Tomographic Inverse Problems: Theory and Applications, Mathematisches Forschungsinstitut Oberwolfach (2019), pp. 37-40. Report No. 4/2019 |
| [44] | R. Huber, G. Haberfehlner, M. Holler, G. Kothleitner, K. Bredies, Total generalized variation regularization for multi-modal electron tomography. Nanoscale 11(12), 5617-5632 (2019) |
| [45] | K.M. Holt, Total nuclear variation and Jacobian extensions of total variation for vector fields. IEEE Trans. Image Process. 23(9), 3975-3989 (2014) · Zbl 1374.94140 |
| [46] | M. Holler, K. Kunisch, On infimal convolution of TV-type functionals and applications to video and image reconstruction. SIAM J. Imag. Sci. 7(4), 2258-2300 (2014) · Zbl 1308.94019 |
| [47] | M. Holler, R. Huber, F. Knoll, Coupled regularization with multiple data discrepancies. Inverse Problems 34(8), 084003 (2018) · Zbl 1439.65072 |
| [48] | D. He, M. Jiang, A.K. Louis, P. Maass, T. Page, Joint bi-modal image reconstruction of DOT and XCT with an extended Mumford-Shah functional, in 2019 IEEE 16th International Symposium on Biomedical Imaging. (ISBI 2019) (2019), pp. 1463-1466 |
| [49] | P. Harremoës, Information topologies with applications, in Entropy, Search, Complexity. Bolyai Society Mathematical Studies (Springer, Berlin, 2007), pp. 113-150 |
| [50] | B.N. Hahn, M.L. Kienle Garrido, An efficient reconstruction approach for a class of dynamic imaging operators. Inverse Problems 35(9), 094005 (2019) · Zbl 1418.94011 |
| [51] | E. Haber, J. Modersitzki, Beyond mutual information: a simple and robust alternative, in Bildverarbeitung für die Medizin(Springer, Berlin, 2005), pp. 350-354 |
| [52] | E. Haber, M.H. Gazit, Model fusion and joint inversion. Surv. Geophys. 34(5), 675-695 (2013) |
| [53] | K. Gong, J.X. Cheng-Liao, G.B. Wang, K.T. Chen, C. Catana, J.Y. Qi, Direct Patlak reconstruction from dynamic PET data using the kernel method with MRI information based on structural similarity. IEEE Trans. Med. Imag. 37(4), 955-965 (2018) |
| [54] | H. Gao, H. Yu, S. Osher, G. Wang, Multi-energy CT based on a prior rank, intensity and sparsity model (PRISM). Inverse Problems 27(11), 115012 (2011) · Zbl 1230.92024 |
| [55] | L.A. Gallardo, M.A. Meju, Structure-coupled multiphysics imaging in geophysical sciences. Rev. Geophys. 49(1), RG1003 (2011) |
| [56] | L.A. Gallardo, M.A. Meju, Joint two-dimensional DC resistivity and seismic travel time inversion with cross-gradients constraints. J. Geophys. Res. Solid Earth 109(B3), B03311 (2004) |
| [57] | L.A. Gallardo, M.A. Meju, Characterization of heterogeneous near-surface materials by joint 2D inversion of DC resistivity and seismic data. Geophys. Res. Lett. 30(13), 1 (2003) |
| [58] | L.A. Gallardo, Multiple cross-gradient joint inversion for geospectral imaging. Geophys. Res. Lett. 34(19), L19301 (2007) |
| [59] | E. Ferrante, N. Paragios, Slice-to-volume medical image registration: a survey. Med. Image Anal. 39, 101-123 (2017) |
| [60] | Y.M. Fang, Z.J. Fang, F.N. Yuan, Y. Yang, S.Y. Yang, N.N. Xiong, Optimized multioperator image retargeting based on perceptual similarity measure. IEEE Trans. Syst. Man Cybern.-Syst. 47(11), 2956-2966 (2017) |
| [61] | M.J. Ehrhardt, K. Thielemans, L. Pizarro, D. Atkinson, S. Ourselin, B.F. Hutton, S.R. Arridge, Joint reconstruction of PET-MRI by exploiting structural similarity. Inverse Problems 31(1), 015001 (2015) · Zbl 1320.92057 |
| [62] | M.J. Ehrhardt, M.M. Betcke, Multi contrast MRI reconstruction with structure-guided total variation. SIAM J. Imag. Sci. 9(3), 1084-1106 (2016) · Zbl 1346.47006 |
| [63] | M.J. Ehrhardt, S.R. Arridge, Vector-valued image processing by parallel level sets. IEEE Trans. Image Process. 23(1), 9-18 (2014) · Zbl 1374.94094 |
| [64] | M.J. Ehrhardt, Joint reconstruction for multi-modality imaging with common structure. Ph.D. Thesis, University College London (2015) |
| [65] | M. Droske, M. Rumpf, A variational approach to nonrigid morphological image registration. SIAM J. Appl. Math. 64(2), 668-687 (2004) · Zbl 1063.49013 |
| [66] | Q.Q. Ding, M. Burger, X.Q. Zhang, Dynamic SPECT reconstruction with temporal edge correlation. Inverse Problems 34(1), 22 (2018) · Zbl 1437.92070 |
| [67] | L. Desbat, S. Roux, P. Grangeat, Compensation of some time dependent deformations in tomography. IEEE Trans. Med. Imag. 26(2), 261-269 (2007) · Zbl 1244.94010 |
| [68] | I. Csiszár, On topology properties of f-divergences. Stud. Sci. Math. Hungar. 2, 329-339 (1967) · Zbl 0157.25803 |
| [69] | I. Csiszár, Information-type measures of difference of probability distributions and indirect observations. Stud. Sci. Math. Hungar. 2, 299-318 (1967) · Zbl 0157.25802 |
| [70] | I. Csiszár, Über topologische und metrische Eigenschaften der relativen Information der Ordnung p, in Third Prague Conference Information Theory, Statistical Decision Functions, Random Processes (Liblice, 1962)(Trans.) (Publishing House Czech Academy of Sciences, Prague, 1964), pp. 63-73 · Zbl 0125.09302 |
| [71] | I. Csiszár, Eine informationstheoretische Ungleichung und Ihre Anwendung auf den Beweis der Ergodizität von Markoffschen Ketten. Magyar Tud. Akad. Mat. Kutató Int. Közl. 8, 85-108 (1963) · Zbl 0124.08703 |
| [72] | I. Csiszár, Informationstheoretische Konvergenzbegriffe im Raum der Wahrscheinlichkeitsverteilungen. Magyar Tud. Akad. Mat. Kutató Int. Közl. 7, 137-158 (1962) · Zbl 0239.60034 |
| [73] | B. Crestel, G. Stadler, O. Ghattas, A comparative study of structural similarity and regularization for joint inverse problems governed by PDEs. Inverse Problems 35(2), 024003 (2019) · Zbl 1412.65197 |
| [74] | S.F. Cotter, B.D. Rao, E. Kjersti, K. Kreutz-Delgado, Sparse solutions to linear inverse problems with multiple measurement vectors. IEEE Trans. Signal Process. 53(7), 2477-2488 (2005) · Zbl 1372.65123 |
| [75] | T. Chan, S. Esedoglu, F. Park, A. Yip, Total variation image restoration: overview and recent developments, in Handbook of Mathematical Models in Computer Vision, ed. by N. Paragios, Y. Chen, O. Faugeras (Springer, Boston, 2006), pp. 17-31 |
| [76] | M. Burger, H. Dirks, L. Frerking, A. Hauptmann, T. Helin, S. Siltanen, A variational reconstruction method for undersampled dynamic x-ray tomography based on physical motion models. Inverse Problems 33(12), 124008 (2017) · Zbl 1381.92041 |
| [77] | X. Bresson, T.F. Chan, Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Problems Imag. 2(4), 455-484 (2008) · Zbl 1188.68337 |
| [78] | K. Brehmer, B. Wacker, J. Modersitzki, A novel similarity measure for image sequences, in 8th International Workshop, Biomedical Image Registration. WBIR 2018, ed. by S. Klein, M. Staring, S. Durrleman, S. Sommer. Lecture Notes in Computer Science (Springer, Cham, 2018), pp. 47-56 |
| [79] | K. Bredies, K. Kunisch, T. Pock, Total generalized variation. SIAM J. Imag. Sci. 3(3), 492-526 (2010) · Zbl 1195.49025 |
| [80] | K. Bredies, M. Holler, Regularization of linear inverse problems with total generalized variation. J. Inverse Ill-Posed Problems 22(6), 871-913 (2014) · Zbl 1302.65167 |
| [81] | J.E. Bowsher, V.E. Johnson, T.G. Turkington, R.J. Jaszczak, C. Floyd, R.E. Coleman, Bayesian reconstruction and use of anatomical a priori information for emission tomography. IEEE Trans. Med. Imag. 15(5), 673-686 (1996) |
| [82] | P. Blomgren, T.F. Chan, Color TV: total variation methods for restoration of vector-valued images. IEEE Trans. Image Process. 7(3), 304-309 (1998) |
| [83] | J.O. Berger, Statistical Decision Theory and Bayesian Analysis, 2nd edn. (Springer, New York, 1985) · Zbl 0572.62008 |
| [84] | E.V.D. Berg, M.P. Friedlander, Theoretical and empirical results for recovery from multiple measurements. IEEE Trans. Inf. Theory 56(5), 2516-2527 (2010) · Zbl 1366.94138 |
| [85] | M. Fazel, Matrix rank minimization with applications. Ph.D. Thesis, Stanford University (2002) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
