Variable selection in classification for multivariate functional data. (English) Zbl 1443.62179
Summary: When classification methods are applied to high-dimensional data, selecting a subset of the predictors may lead to an improvement in the predictive ability of the estimated model, in addition to reducing the model complexity. In Functional Data Analysis (FDA), i.e., when data are functions, selecting a subset of predictors corresponds to selecting a subset of individual time instants in the time interval in which the functional data are measured. In this paper, we address the problem of selecting the most informative time instants in multivariate functional data, a case much less studied than its single-variate counterpart. Our proposal allows one to use in a very simple way high-order information of the data, e.g. monotonicity or convexity by means of the functional data derivatives. The aforementioned problem is addressed with tools of Global Optimization in continuous variables: the time instants are selected to maximize the correlation between the class label and the Support Vector Machine score used for classification. The effectiveness of the proposal is shown in univariate and multivariate datasets.
MSC:
| 62H30 | Classification and discrimination; cluster analysis (statistical aspects) |
| 90C26 | Nonconvex programming, global optimization |
Software:
fda (R)References:
| [1] | Baíllo, A.; Cuevas, A.; Fraiman, R., Classification methods for functional data, The Oxford Handbook of Functional Data Analysis, 259-297 (2011), Oxford University Press · Zbl 1284.62001 |
| [2] | Benítez-Peña, S.; Blanquero, R.; Carrizosa, E.; Ramírez-Cobo, P., Cost-sensitive feature selection for support vector machines, Comput. Oper. Res. (2018) · Zbl 1458.68158 |
| [3] | Berrendero, J.; Justel, A.; Svarc, M., Principal components for multivariate functional data, Comput. Stat. Data Anal., 55, 9, 2619-2634 (2011) · Zbl 1464.62025 |
| [4] | Berrendero, J. R.; Cuevas, A.; Torrecilla, J. L., Variable selection in functional data classification: a maxima-hunting proposal, Stat. Sin., 26, 2, 619-638 (2016) · Zbl 1356.62079 |
| [5] | J.R. Berrendero, A. [Cuevas], & J.L. [Torrecilla] (2016). Variable selection in functional data classification: a maxima-hunting proposal. Statistica Sinica, 26, 619-638.; J.R. Berrendero, A. [Cuevas], & J.L. [Torrecilla] (2016). Variable selection in functional data classification: a maxima-hunting proposal. Statistica Sinica, 26, 619-638. · Zbl 1356.62079 |
| [6] | Berrendero, J. R.; Cuevas, A.; Torrecilla, J. L., On the use of reproducing kernel hilbert spaces in functional classification, J. Am. Stat. Assoc. (2017) |
| [7] | Blanquero, R.; Carrizosa, E.; Chis, O.; Esteban, N.; Jiménez-Cordero, A.; Rodríguez, J. F.; Sillero-Denamiel, M. R., On extreme concentrations in chemical reaction networks with incomplete measurements, Ind. Eng. Chem. Res., 55, 11417-11430 (2016) |
| [8] | Blanquero, R.; Carrizosa, E.; Jiménez-Cordero, A.; Martín-Barragán, B., Functional-bandwidth kernel for support vector machine with functional data: an alternating optimization algorithm, Eur. J. Oper. Res. (2018) · Zbl 1430.90450 |
| [9] | Blanquero, R.; Carrizosa, E.; Jiménez-Cordero, A.; Martín-Barragán, B., Variable selection with support vector regression for multivariate functional data, Technical Report (2018), University of Edinburgh - Universidad de Sevilla · Zbl 1430.90450 |
| [10] | Blanquero, R.; Carrizosa, E.; Jiménez-Cordero, A.; Rodríguez, J. F., A global optimization method for model selection in chemical reactions networks, Comput. Chem. Eng., 93, 52-62 (2016) |
| [11] | Carrizosa, E.; Martín-Barragán, B.; Romero Morales, D., A nested heuristic for parameter tuning in support vector machines, Comput. Oper. Res., 43, 328-334 (2014) · Zbl 1349.62260 |
| [12] | Colson, B.; Marcotte, P.; Savard, G., An overview of bilevel optimization, Ann. Oper. Res., 153, 1, 235-256 (2007) · Zbl 1159.90483 |
| [13] | Cortes, C.; Vapnik, V., Support-vector networks, Mach. Learn., 20, 3, 273-297 (1995) · Zbl 0831.68098 |
| [14] | Cristianini, N.; Shawe-Taylor, J., An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods (2000), Cambridge University Press |
| [15] | De Boor, C., A Practical Guide to Splines, Applied Mathematical Sciences, 27 (1978), Springer-Verlag New York · Zbl 0406.41003 |
| [16] | Ding, C.; Peng, H., Minimum redundancy feature selection from microarray gene expression data, J. Bioinform. Comput. Biol., 3, 185-205 (2005) |
| [17] | Ferraty, F.; Hall, P.; Vieu, P., Most-predictive design points for functional data predictors, Biometrika, 97, 4, 807-824 (2010) · Zbl 1204.62064 |
| [18] | Ferraty, F.; Vieu, P., Nonparametric Functional Data Analysis: Theory and Practice (2006), Springer Science & Business Media · Zbl 1119.62046 |
| [19] | Ferris, M. C.; Munson, T. S., Semismooth support vector machines, Math. Program, 101, 1, 185-204 (2004) · Zbl 1076.90041 |
| [20] | J. Friedman, T. Hastie, R. Tibshirani, Datasets for The Elements of Statistical Learninghttps://web.stanford.edu/ hastie/ElemStatLearn/data.html; J. Friedman, T. Hastie, R. Tibshirani, Datasets for The Elements of Statistical Learninghttps://web.stanford.edu/ hastie/ElemStatLearn/data.html |
| [21] | Friedman, J.; Hastie, T.; Tibshirani, R., The Elements of Statistical Learning, Springer Series in Statistics, 1 (2001), Springer, Berlin · Zbl 0973.62007 |
| [22] | Gómez-Verdejo, V.; Verleysen, M.; Fleury, J., Information-theoretic feature selection for functional data classification, Neurocomputing, 72, 16, 3580-3589 (2009) |
| [23] | Ieva, F.; Paganoni, A. M.; Pigoli, D.; Vitelli, V., Multivariate functional clustering for the morphological analysis of electrocardiograph curves, J. R. Stat. Soc. Ser. C (Appl. Stat.), 62, 3, 401-418 (2013) |
| [24] | Jacques, J.; Preda, C., Model-based clustering for multivariate functional data, Comput. Stat. Data Anal., 71, 92-106 (2014) · Zbl 1471.62096 |
| [25] | James, G. M.; Wang, J.; Zhu, J., Functional linear regression that’s interpretable, Ann. Stat., 37, 2083-2108 (2009) · Zbl 1171.62041 |
| [26] | Jiménez-Cordero, A.; Maldonado, S., Automatic feature scaling and selection for support vector machine classification with functional data, Technical Report (2018), Universidad de los Andes - Universidad de Sevilla |
| [27] | Khalid, S.; Khalil, T.; Nasreen, S., A survey of feature selection and feature extraction techniques in machine learning, Proceedings of the Science and Information Conference, 372-378 (2014) |
| [28] | Li, Y.; Dong, M.; Hua, J., Localized feature selection for clustering, Pattern Recognit. Lett., 29, 1, 10-18 (2008) |
| [29] | Lindquist, M. A.; McKeague, I. W., Logistic regression with Brownian-like predictors, J. Am. Stat. Assoc., 104, 488, 1575-1585 (2009) · Zbl 1205.62125 |
| [30] | Locantore, N.; Marron, J. S.; Simpson, D. G.; Tripoli, N.; Zhang, J. T.; Cohen, K. L.; Boente, G.; Fraiman, R.; Brumback, B.; Croux, C.; Fan, J.; Kneip, A.; Marden, J. I.; Peña, D.; Prieto, J.; Ramsay, J. O.; Valderrama, M. J.; Aguilera, A. M., Robust principal component analysis for functional data, Test, 8, 1, 1-73 (1999) · Zbl 0980.62049 |
| [31] | López, J.; Maldonado, S.; Carrasco, M., Double regularization methods for robust feature selection and SVM classification via dc programming, Inf. Sci. (Ny), 429, 377-389 (2018) · Zbl 1436.68310 |
| [32] | Martín-Barragán, B.; Lillo, R.; Romo, J., Interpretable support vector machines for functional data, Eur. J. Oper. Res., 232, 1, 146-155 (2014) |
| [33] | Mehmood, T.; Liland, K. H.; Snipen, L.; Sæbø, S., A review of variable selection methods in partial least squares regression, Chemom. Intell. Lab. Syst., 118, 62-69 (2012) |
| [34] | Muñoz, A.; González, J., Representing functional data using support vector machines, Pattern Recognit. Lett., 31, 6, 511-516 (2010) |
| [35] | Naftchali, R. E.; Abadeh, M. S., A multi-layered incremental feature selection algorithm for adjuvant chemotherapy effectiveness/futileness assessment in non-small cell lung cancer, Biocybern. Biomed. Eng., 37, 3, 477-488 (2017) |
| [36] | Peng, H.; Fan, Y., Feature selection by optimizing a lower bound of conditional mutual information, Inf. Sci. (Ny), 418-419, 652-667 (2017) |
| [37] | Preda, C.; Saporta, G.; Lévéder, C., PLS classification of functional data, Comput. Stat., 22, 2, 223-235 (2007) · Zbl 1196.62086 |
| [38] | Ramsay, J. O.; Silverman, B. W., Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 77 (2002), Springer-Verlag · Zbl 1011.62002 |
| [39] | Ramsay, J. O.; Silverman, B. W., Functional Data Analysis, Springer Series in Statistics (2005), Springer-Verlag · Zbl 1079.62006 |
| [40] | Rossi, F.; Villa, N., Support vector machine for functional data classification, Neurocomputing, 69, 7, 730-742 (2006) |
| [41] | Székely, G. J.; Rizzo, M. L.; Bakirov, N. K., Measuring and testing dependence by correlation of distances, Ann. Stat., 35, 6, 2769-2794 (2007) · Zbl 1129.62059 |
| [42] | R. Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2017. https://www.R-project.org/; R. Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2017. https://www.R-project.org/ |
| [43] | Torrecilla, J. L.; Suárez, A., Feature selection in functional data classification with recursive maxima hunting, (Lee, D. D.; Sugiyama, M.; Luxburg, U. V.; Guyon, I.; Garnett, R., Advances in Neural Information Processing Systems 29 (2016), Curran Associates, Inc.), 4835-4843 |
| [44] | Torrecilla Noguerales, J. L., On the theory and practice of variable selection for functional data (2015), Universidad Autónoma de Madrid, Ph.D. thesis |
| [45] | Vapnik, V., Statistical Learning Theory (1998), John Wiley and Sons · Zbl 0935.62007 |
| [46] | Wang, H.; Yao, M., Fault detection of batch processes based on multivariate functional kernel principal component analysis, Chemom. Intell. Lab. Syst., 149, 78-89 (2015) |
| [47] | Wang, X.; Ray, S.; Mallick, B. K., Bayesian curve classification using wavelets, J. Am. Stat. Assoc., 102, 479, 962-973 (2007) · Zbl 1469.62291 |
| [48] | Zhang, X.; Park, B. U.; Wang, J.-L., Time-varying additive models for longitudinal data, J. Am. Stat. Assoc., 108, 503, 983-998 (2013) · Zbl 06224981 |
| [49] | Zhao, Y.; Ogden, R. T.; Reiss, P. T., Wavelet-based lasso in functional linear regression, J. Comput. Gr. Stat., 21, 3, 600-617 (2012) |
| [50] | Zibakhsh, A.; Abadeh, M. S., Gene selection for cancer tumor detection using a novel memetic algorithm with a multi-view fitness function, Eng. Appl. Artif. Intell., 26, 4, 1274-1281 (2013) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
