A multiple attributes convolution kernel with reproducing property. (English) Zbl 1425.68357
Summary: Various kernel-based methods have been developed with great success in many fields, but very little research has been published that is concerned with a multiple attribute kernel in reproducing kernel Hilbert space (RKHS). In this paper, we propose a novel elastic kernel called a multiple attribute convolution kernel with reproducing property (MACKRP) and present improved classification results over conventional approaches in the RKHS rather than the more commonly used Hilbert space. The MACKRP consists of two major steps. First, we find the basic solution of a generalized differential operator by the delta function, and then we design a convolution function using this solution. This convolution function is proven to be a specific reproducing kernel called a convolution reproducing kernel (CRK) in \(H^3\)-space. Second, we prove that the CRK satisfies the condition of Mercer kernel. And the CRK is composed of three attributes \(L_1\)-norm, \(L_2\)-norm and Laplace kernel), and each attribute can capture a different feature, with all attributes generating a novel kernel which we call an MACKRP. The experimental results demonstrate that the MACKRP possesses approximation and regularization performance and that classification results are consistently comparable or superior to a number of other state-of-the-art kernel functions.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 46E22 | Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) |
References:
| [1] | Lu H et al (2011) On feature combination and multiple kernel learning for object tracking. In: Computer vision (ACCV 2010). Springer, Berlin, pp 511-522 |
| [2] | Kim J, Scott CD (2010) L2 kernel classification. IEEE Trans Pattern Anal Mach Intell 32:1822-1831 · doi:10.1109/TPAMI.2009.188 |
| [3] | He R et al (2011) A regularized correntropy framework for robust pattern recognition. Neural Comput 23:2074-2100 · Zbl 1218.68144 · doi:10.1162/NECO_a_00155 |
| [4] | Shen C et al (2007) Fast global kernel density mode seeking: applications to localization and tracking. IEEE Trans Image Process 16:1457-1469 · doi:10.1109/TIP.2007.894233 |
| [5] | Tzortzis GF, Likas C (2009) The global kernel-means algorithm for clustering in feature space. IEEE Trans Neural Netw 20:1181-1194 · doi:10.1109/TNN.2009.2019722 |
| [6] | Jorstad A et al (2011) A deformation and lighting insensitive metric for face recognition based on dense correspondences. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 2353-2360 |
| [7] | Gao S et al (2010) Kernel sparse representation for image classification and face recognition. In: Computer vision (ECCV 2010). Springer, Berlin, pp 1-14 |
| [8] | Jose C et al (2013) Local deep kernel learning for efficient non-linear SVM prediction. In: Proceedings of the 30th international conference on machine learning (ICML-13), pp 486-494 |
| [9] | Caputo B et al (2004) Object categorization via local kernels. In: Proceedings of the 17th international conference on pattern recognition, 2004 (ICPR 2004), pp 132-135 |
| [10] | Boughorbel S et al (2005) Conditionally positive definite kernels for SVM based image recognition. In: IEEE international conference on multimedia and expo, 2005 (ICME 2005), pp 113-116 |
| [11] | Shen C et al (2007) Fast global kernel density mode seeking: applications to localization and tracking. IEEE Trans Image Process 16:1457-1469 · doi:10.1109/TIP.2007.894233 |
| [12] | Tzortzis G, Likas A (2008) The global kernel k-means clustering algorithm. In: IEEE international joint conference on neural networks, 2008 (IJCNN 2008). IEEE World Congress on Computational Intelligence, pp 1977-1984 |
| [13] | Gönen M, Alpaydin E (2008) Localized multiple kernel learning. In: Proceedings of the 25th international conference on machine learning, pp 352-359 · Zbl 1254.68204 |
| [14] | Rakotomamonjy A et al (2008) SimpleMKL. J Mach Learn Res 9:2491-2521 · Zbl 1225.68208 |
| [15] | Kloft M et al (2011) Lp-norm multiple kernel learning. J Mach Learn Res 12:953-997 · Zbl 1280.68173 |
| [16] | Bai L et al (2015) A quantum Jensen-Shannon graph kernel for unattributed graphs. Pattern Recognit 48:344-355 · Zbl 1373.68345 · doi:10.1016/j.patcog.2014.03.028 |
| [17] | Bai L, Hancock ER (2013) Graph kernels from the Jensen-Shannon divergence. J Math Imaging Vis 47:60-69 · Zbl 1276.68152 · doi:10.1007/s10851-012-0383-6 |
| [18] | Tuytelaars T et al (2011) The NBNN kernel. In: 2011 IEEE international conference on computer vision (ICCV), pp 1824-1831 |
| [19] | Zhang D et al (2010) Gaussian ERP kernel classifier for pulse waveforms classification. In: 20th international conference on pattern recognition 2010 (ICPR 10), pp 2736-2739 |
| [20] | Liu Z et al (2013) Multi-fault classification based on wavelet SVM with PSO algorithm to analyze vibration signals from rolling element bearings. Neurocomputing 99:399-410 · doi:10.1016/j.neucom.2012.07.019 |
| [21] | Rossi L et al (2013) A continuous-time quantum walk kernel for unattributed graphs. In: Graph-based representations in pattern recognition. Springer, Berlin, pp 101-110 · Zbl 1374.68474 |
| [22] | Luo Y et al (2013) Multiview vector-valued manifold regularization for multilabel image classification. IEEE Trans Neural Netw Learn Syst 24:709-722 · doi:10.1109/TNNLS.2013.2238682 |
| [23] | Luo Y et al (2013) Manifold regularized multitask learning for semi-supervised multilabel image classification. IEEE Trans Image Process 22:523-536 · Zbl 1373.94269 · doi:10.1109/TIP.2012.2218825 |
| [24] | Yu J et al (2012) Semisupervised multiview distance metric learning for cartoon synthesis. IEEE Trans Image Process 21:4636-4648 · Zbl 1373.94472 · doi:10.1109/TIP.2012.2207395 |
| [25] | Xu L et al (2015) An efficient multiple kernel learning in reproducing kernel Hilbert spaces (RKHS). Int J Wavelets Multiresolut Inf Process · Zbl 1329.68225 |
| [26] | Xu L et al (2015) A local-global mixed kernel with reproducing property. Neurocomputing 168:190-199 · doi:10.1016/j.neucom.2015.05.107 |
| [27] | Yu J et al (2014) Click prediction for web image reranking using multimodal sparse coding · Zbl 1374.94435 |
| [28] | Xu C et al (2014) Large-margin multi-view information bottleneck. IEEE Trans Pattern Anal Mach Intell 36:1559-1572 · doi:10.1109/TPAMI.2013.2296528 |
| [29] | Yu J et al (2014) High-order distance-based multiview stochastic learning in image classification |
| [30] | Liu W, Tao D (2013) Multiview hessian regularization for image annotation. IEEE Trans Image Process 22:2676-2687 · Zbl 1373.68436 · doi:10.1109/TIP.2013.2255302 |
| [31] | Xu C et al (2015) Multi-view intact space learning. IEEE Trans Pattern Anal Mach Intell |
| [32] | Luo Y et al (2015) Multi-view matrix completion for multi-label image classification. IEEE Trans Image Process 24:2355-2368 · Zbl 1408.94451 |
| [33] | Vito ED et al (2010) Spectral regularization for support estimation. In: Advances in neural information processing systems, pp 487-495 |
| [34] | Chapelle O et al (1999) Support vector machines for histogram-based image classification. IEEE Trans Neural Netw 10:1055-1064 · doi:10.1109/72.788646 |
| [35] | Aronszajn N (1950) Theory of reproducing kernels. Trans Am Math Soc 68:337-404 · Zbl 0037.20701 · doi:10.1090/S0002-9947-1950-0051437-7 |
| [36] | Tao D et al (2006) Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Trans Pattern Anal Mach Intell 28:1088-1099 · doi:10.1109/TPAMI.2006.134 |
| [37] | Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, London · Zbl 0994.68074 |
| [38] | Lapidus L, Pinder GF (2011) Numerical solution of partial differential equations in science and engineering. Wiley, New York · Zbl 0584.65056 |
| [39] | Ding L (2009) L1-norm and L2-norm neuroimaging methods in reconstructing extended cortical sources from EEG. In: Annual international conference of the IEEE on engineering in medicine and biology society, 2009 (EMBC 2009), pp 1922-1925 |
| [40] | Bektaş S, Şişman Y (2010) The comparison of L1 and L2-norm minimization methods. Int J Phys Sci |
| [41] | Yi H et al (2013) Reconstruction algorithms based on l1-norm and l2-norm for two imaging models of fluorescence molecular tomography: a comparative study. J Biomed Opt 18:056013-056014 · doi:10.1117/1.JBO.18.5.056013 |
| [42] | Drucker H et al (1997) Support vector regression machines. Adv Neural Inf Process Syst 9:155-161 |
| [43] | Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2:27 |
| [44] | Cheng J et al (2013) Superpixel classification based optic disc and optic cup segmentation for glaucoma screening. IEEE Trans Med Imaging 32:1019-1032 · doi:10.1109/TMI.2013.2247770 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
