Pliable lasso for the support vector machine. (English) Zbl 07862749
Summary: In this article, we study the support vector machine with interaction effects. The pliable lasso penalty, which allows for estimating the main effects of the covariates \(X\) and the interaction effects between the covariates and a set modifiers \(Z\) is implemented to handle the interaction effect. Interaction variables are included in a hierarchical manner by first considering whether their corresponding main effect variables have been included in the model to avoid over-fitting. The loss function employed is the squared hinge loss, with the pliable lasso penalty and then, the block-wise coordinate descent approach is employed. The results from the simulation and real data show the effectiveness of the pliable lasso in building support vector machine models in situations where interaction effects are involved.
MSC:
| 62-XX | Statistics |
References:
| [1] | Alon, U., Barkai, N., Notterman, D. A., Gish, K., Ybarra, S., Mack, D., and Levine, A. J.. 1999. Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences of the United States of America96 (12):6745-50. doi:. |
| [2] | Asenso, T. Q., Zhang, H., and Liang, Y.. 2020. Pliable lasso for the multinomial logistic regression. Communications in Statistics - Theory and Methods0 (0):1-16. doi:. · Zbl 07533649 |
| [3] | Chang, K.-W., Hsieh, C.-J., and Lin, C.-J.. 2008. Coordinate descent method for large-scale l2-loss linear support vector machines. Journal of Machine Learning Research9:1369-98. · Zbl 1225.68157 |
| [4] | Du, W., and Tibshirani, R.. 2018. A pliable lasso for the cox model. |
| [5] | Dua, D., and Casey, G.. 2019. UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. |
| [6] | Friedman, J., Hastie, T., and Tibshirani, R.. 2010. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software33 (1):1-22. doi:. |
| [7] | Glaab, E., Bacardit, J., Garibaldi, J. M., and Krasnogor, N.. 2012. Using rule-based machine learning for candidate disease gene prioritization and sample classification of cancer gene expression data. PLoS ONE7 (7):e39932. |
| [8] | Golub, T., Slonim, D., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J., Coller, H., Loh, M., Downing, J., Caligiuri, M., et al.1999. Molecular classification of cancer: Class discovery and class prediction by gene expression. Science (New York, N.Y.)286 (5439):531-7. doi:. |
| [9] | Hastie, T., Rosset, S., Tibshirani, R., and Zhu, J.. 2004. The entire regularization path for the support vector machine. Journal of Machine Learning Research5:1391-415. · Zbl 1222.68213 |
| [10] | Huo, Y., Xin, L., Kang, C., Wang, M., Ma, Q., and Yu, B.. 2020. Sgl-svm: A novel method for tumor classification via support vector machine with sparse group lasso. Journal of Theoretical Biology486 (110098):110098. · Zbl 1429.92057 |
| [11] | Lee, Y.-J., and Mangasarian, O. L.. 2001. Ssvm: A smooth support vector machine for classification. Computational Optimization and Applications20 (1):5-22. doi:. · Zbl 1017.90105 |
| [12] | Lim, M., and Hastie, T.. 2015. Learning interactions via hierarchical group-lasso regularization. Journal of Computational and Graphical Statistics: A Joint Publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America24 (3):627-54. PMID: 26759522. |
| [13] | Martinez, D. L.2017. Regularization approaches for support vector machines with applications to biomedical data. CoRR. abs/1710.10600. |
| [14] | Platt, J. C.1998. Sequential minimal optimization: A fast algorithm for training support vector machines. Technical report, Advances in kernel methods – support vector learning. |
| [15] | Shevade, S., and Keerthi, S.. 2003. A simple and efficient algorithm for gene selection using sparse logistic regression. Bioinformatics (Oxford, England)19 (17):2246-53. doi:. |
| [16] | Singh, D., Febbo, P. G., Ross, K., Jackson, D. G., Manola, J., Ladd, C., Tamayo, P., Renshaw, A. A., D’Amico, A. V., Richie, J. P., et al.2002. Gene expression correlates of clinical prostate cancer behavior. Cancer Cell1 (2):203-9. doi:. |
| [17] | Tibshirani, R., and Friedman, J.. 2020. A pliable lasso. Journal of Computational and Graphical Statistics29 (1):215-25. doi:. · Zbl 07499284 |
| [18] | Vapnik, V. N.1995. The nature of statistical learning theory. New York: Springer-Verlag. · Zbl 0833.62008 |
| [19] | Wang, L., Zhu, J., and Zou, H.. 2006. The doubly regularized support vector machine. Statistica Sinica16 (2):589-616. · Zbl 1126.68070 |
| [20] | Wolberg, W. H., and Mangasarian, O. L.. 1990. Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proceedings of the National Academy of Sciences of the United States of America87 (23):9193-6. doi:. · Zbl 0709.92537 |
| [21] | Wu, S., Zou, H., and Yuan, M.. 2008. Structured variable selection in support vector machines. Electronic Journal of Statistics2 (none):103-17. doi:. · Zbl 1320.62154 |
| [22] | Zhu, J., Rosset, S., Hastie, T., and Tibshirani, R.. 2003. 1-norm support vector machines. In Neural information processing systems, 16. Cambridge, MA: MIT Press. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
