×

Learning to match: deriving optimal template-matching algorithms from probabilistic image models. (English) Zbl 1477.68432

Summary: Finding correspondences between images by template matching is a common problem in image understanding. Although a variety of solutions have been proposed, most of them rely on the arbitrary choice of a template and a matching function. Often, different cost functions lead to different results, and the choice of a good cost for a specific application remains an art. Statistical models on the other hand, allow us to derive optimal learning and matching algorithms from modeling assumptions using likelihood maximization principles. The key contribution of this paper is the development of a statistical framework for learning what function to optimize from training examples. We present a family of statistical models for grayscale images, which allow us to derive optimal template-matching algorithms. The intensity at each pixel is described by a random variable whose distribution is encoded by a deformable template. Firstly, we assume the intensity distribution to be Gaussian and derive an intensity-matching algorithm, which is a generalization of the classical sum-of-squared differences. Then, we introduce a hidden segmentation variable in the probabilistic model and derive a segmentation-matching algorithm that can handle photometric variations. Both models are exemplified on the automatic detection of anatomical landmarks in brain Magnetic Resonance Images.

MSC:

68T45 Machine vision and scene understanding
62H35 Image analysis in multivariate analysis
68T05 Learning and adaptive systems in artificial intelligence
92C55 Biomedical imaging and signal processing

Software:

AFNI; SIFT
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Allassonniere, S., Kuhn, E., Trouvé, A., & Amit, Y. (2006). Generative model and consistent estimation algorithms for non-rigid deformable models. In Acoustics, speech and signal processing, 2006. ICASSP 2006 proceedings. 2006 IEEE international conference on 5, V–V.
[2] Allassonnière, S., Amit, Y., & Trouvé, A. (2007). Toward a coherent statistical framework for dense deformable template estimation. Journal of the Royal Statistical Society B, 69, 3–29.
[3] Arad, N., Dyn, N., Reispeld, D., & Yeshurun, Y. (1994). Image warping by radial basis functions: application to facial expressions. CVGIP: Graphical Models and Image Processing, 56, 161–172. · doi:10.1006/cgip.1994.1015
[4] Ashburner, J., &amp; Friston, K. J. (1999). Nonlinear spatial normalization using basis functions. Human Brain Mapping, 7, 254–266. · doi:10.1002/(SICI)1097-0193(1999)7:4<254::AID-HBM4>3.0.CO;2-G
[5] Ashburner, J., &amp; Friston, K. J. (2005). Unified segmentation. NeuroImage, 26, 839–851. · doi:10.1016/j.neuroimage.2005.02.018
[6] Bajcsy, R., Kovacic, S. (1989). Multiresolution elastic matching. Computer Vision, Graphics and Image Processing, 46, 1–21. · doi:10.1016/S0734-189X(89)80014-3
[7] Barnea, D. I., &amp; Silverman, H. F. (1972). A class of algorithms for fast digital image registration. IEEE Transactions on Computers, 21(2), 179–186. · Zbl 0225.68055 · doi:10.1109/TC.1972.5008923
[8] Bookstein, F. L. (1989). Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(6), 567–585. · Zbl 0691.65002 · doi:10.1109/34.24792
[9] Bookstein, F. L. (1992). Morphometric tools for landmark data: geometry and biology. Cambridge: Cambridge University Press. · Zbl 0770.92001
[10] Bro-Nielsen, M., &amp; Gramkow, C. (1996). Fast fluid registration of medical images. In Lecture notes in computer science : Vol. 1131. Proceeding of 4th international conference on visualization in biomedical computing (VBC’96) (pp. 267–276). Berlin: Springer.
[11] Collignon, A., Maes, F., Delaere, D., Vandermeulen, D., Suetens, P., &amp; Marshal, G. (1995). Automated multi-modality image registration based on information theory. In C. B. Y. Bizais &amp; R. D. Paola (Eds.), Information processing in medical imaging (pp. 263–274). Dordrecht: Kluwer Academic.
[12] Cox, R. (1996). Afni: Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research, 29, 162–173. · doi:10.1006/cbmr.1996.0014
[13] Dalal, N., &amp; Triggs, B. (2005). Histograms of oriented gradients for human detection (pp. 886–893).
[14] Davatzikos, C. (1997). Spatial transformation and registration of brain imaging using elastically deformable models. Computer Vision and Image Understanding, 2(66), 207–222. · doi:10.1006/cviu.1997.0605
[15] Dempster, A., Laird, N., &amp; Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistical Society, 39, 1–38. · Zbl 0364.62022
[16] Fischl, B., Salat, D. H., van der Kouwe, A. J., Makris, N., Ségonne, F., Quinn, B. T., &amp; Dale, A. M. (2004). Sequence-independent segmentation of magnetic resonance images. NeuroImage, 23, S69–S84. · doi:10.1016/j.neuroimage.2004.07.016
[17] Frantz, S., Rohr, K., &amp; Stiehl, H. (2000). Localization of 3D anatomical point landmarks in 3D tomographic images using deformable models. In Lecture notes in computer science : Vol. 1935. Proc. MICCAI (pp. 492–501). Berlin: Springer.
[18] Friston, K. J., Ashburner, J., Poline, J. B., Frith, C. D., Heather, J. D., &amp; Frackowiak, R. (1995). Spatial registration and normalisation of images. Human Brain Mapping, 2, 165–189. · doi:10.1002/hbm.460030303
[19] Glasbey, C., &amp; Mardia, K. (2001). A penalized likelihood approach to image warping (with discussion). Journal of the Royal Statistical Society B, 63, 465–514. · Zbl 1040.62054 · doi:10.1111/1467-9868.00295
[20] Goshtasby, A., Staib, L., Studholme, C., &amp; Terzopoulos, D. (2003). Non-rigid image registration: Guest editors’ introduction. Computer Vision and Image Understanding, 89(2/3), 109–113. · doi:10.1016/S1077-3142(03)00016-X
[21] Grenander, U., &amp; Miller, M. (1998). Computational anatomy: An emerging discipline. Quarterly of Applied Mathematics, 4, 617–694. LVI. · Zbl 0952.92016
[22] Hartkens, T., Rohr, K., &amp; Stiehl, H. (1999). Performance of 3D differential operators for the detection of anatomical landmarks in MR and CT images. In Medical imaging 1999: image processing. Proceedings of the SPIE international symposium (Vol. 5032, pp. 32–43). · Zbl 1010.68570
[23] Izard, C., Jedynak, B., &amp; Stark, C. (2006). Spline-based probabilistic model for anatomical landmark detection. In R. Larsen, M. Nielsen, &amp; J. Sporring (Eds.), Lecture notes in computer science : Vol. 4190. Medical imaging computing and computer assisted intervention (MICCAI) (pp. 849–856). Berlin: Springer.
[24] Joshi, S., &amp; Miller, M. (2000). Landmark matching via large deformation diffeomorphisms. IEEE Transactions on Image Processing, 9, 1357–1370. · Zbl 0965.37065 · doi:10.1109/83.855431
[25] Leemput, K. V. (2001). A statistical framework for partial volume segmentation. In W. Niessen &amp; M. Viergever (Eds.), Lecture notes in computer science : Vol. 2208. MICCAI (pp. 204–212). Berlin: Springer. · Zbl 1041.68767
[26] Lester, H., Arridge, S., Jansons, K., Lemieux, L., Hajnal, J., &amp; Oatridge, A. (1999). Non-linear registration with the variable viscosity fluid algorithm. In Information processing in medical imaging (IPMI’99) (pp. 238–251).
[27] Levin, A., &amp; Weiss, Y. (2006). Learning to combine bottom-up and top-down segmentation. In Lecture notes in computer science : Vol. 3954. ECCV (pp. 581–594). Berlin: Springer.
[28] Li, H., Manjunath, B. S., &amp; Mitra, S. K. (1995). A contour–based approach to multisensor image registration. IEEE Transactions on Image Processing, 4(3), 320–334. · doi:10.1109/83.366480
[29] Lowe, D. (2003). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 20, 91–110.
[30] Maes, F., Collignon, A., Vandermeulen, D., Marshal, G., &amp; Suetens, P. (1997). Multimodality image registration by maximization of mutual information. IEEE Transactions on Medical Imaging, 16, 187–198. · doi:10.1109/42.563664
[31] Pohl, K. M., Wells, W. M., Guimond, A., Kasai, K., Shenton, M. E., Kikinis, R., Grimson, W. E. L., &amp; Warfield, S. K. (2002). Incorporating non-rigid registration into expectation-maximization algorithm to segment mr images. In T. Dohi &amp; R. Kikinis (Eds.), Lecture notes in computer science : Vol. 2488. MICCAI (pp. 564–571). Berlin: Springer. · Zbl 1028.68871
[32] Pohl, K. M., Fisher, J., Grimson, W. E. L., Kikinis, R., &amp; Wells, W. M. (2006). A Bayesian model for joint segmentation and registration. NeuroImage, 31(1), 228–239. · doi:10.1016/j.neuroimage.2005.11.044
[33] Pratt, W. K. (1974). Correlation techniques for image registration. IEEE Transactions on Aerospace and Electronic Systems, 10(3), 353–358. · doi:10.1109/TAES.1974.307828
[34] Qiu, A., Younes, L., Wang, L., Ratnanather, J. T., Gillepsie, S. K., Kaplan, G., Csernansky, J., &amp; Miller, M. I. (2007). Combining anatomical manifold information via diffeomorphic metric mappings for studying cortical thinning of the cingulate gyrus in schizophrenia. NeuroImage, 37(3), 821–833. · doi:10.1016/j.neuroimage.2007.05.007
[35] Roche, A., Malandain, G., &amp; Ayache, N. (2000). Unifying maximum likelihood approaches in medical image registration. International Journal of Imaging Systems and Technology, 11(1), 71–80. · doi:10.1002/(SICI)1098-1098(2000)11:1<71::AID-IMA8>3.0.CO;2-5
[36] Rohr, K. (2001). Landmark-based image analysis using geometric and intensity models. Dordrecht: Kluwer Academic. · Zbl 1057.68722
[37] Rohr, K., Stiehl, H., Sprengel, R., Buzug, T., Weese, J., &amp; Kuhn, M. (2001). Landmark-based elastic registration using approximating thin-plate splines. IEEE Transactions on Medical Imaging, 20(6), 526–534. · doi:10.1109/42.929618
[38] Schmid, C., Mohr, R., &amp; Bauckhage, C. (2000). Evaluation of interest point detectors. International Journal of Computer Vision, 37(2), 151–172. · Zbl 0985.68625 · doi:10.1023/A:1008199403446
[39] Studholme, C., Hill, D. L. G., &amp; Hawkes, D. J. (1995). Multiresolution voxel similarity measures for MR–PET registration. In C. B. Y. Bizais &amp; R. D. Paola (Eds.), Information processing in medical imaging (pp. 287–298). Dordrecht: Kluwer Academic.
[40] Szeliski, R. (2006). Image alignment and stitching: A tutorial. Fundamental Trends in Computer Graphics and Vision, 2(1), 1–104. · Zbl 1143.68623 · doi:10.1561/0600000009
[41] Talairach, J., Tournoux, P. (1988) Co-planar stereotaxic atlas of the human brain. Stuttgart: Thieme Medical.
[42] Thirion, J. P. (1996). New feature points based on geometric invariants for 3D image registration. International Journal of Computer Vision, 18:2, 121–137. · Zbl 05475550 · doi:10.1007/BF00054999
[43] Twining, C., Marsland, S., &amp; Taylor, C. (2002). Measuring geodesic distances on the space of bounded diffeomorphisms.
[44] Viola, P. (1995). Alignment by maximization of mutual information. Ph.D. thesis, Massachusetts Institute of Technology.
[45] Wahba, G. (1990). Spline models for observational data. Philadelphia: Society for Industrial and Applied Mathematics. · Zbl 0813.62001
[46] Wang, F., Vemuri, B. C., &amp; Eisenschenk, S. J. (2006). Joint registration and segmentation of neuroanatomic structures from brain mri. Academic Radiology, 13(9), 1104–1111. · doi:10.1016/j.acra.2006.05.017
[47] Wells, W., Kikinis, R., Grimson, W., &amp; Jolesz, F. (1996). Adaptive segmentation of MRI data. IEEE Transactions on Medical Imaging, 15, 429–442. · doi:10.1109/42.511747
[48] Wörz, S., &amp; Rohr, K. (2006). Localization of anatomical point landmarks in 3D medical images by fitting 3d parametric intensity models. Medical Image Analysis, 10(1), 41–58. · doi:10.1016/j.media.2005.02.003
[49] Zitová, B., &amp; Flusser, J. (2003). Image registration methods: a survey. Image and Vision Computing, 21, 977–1000. · doi:10.1016/S0262-8856(03)00137-9
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.