Real order total variation with applications to the loss functions in learning schemes. (English) Zbl 07864530
Summary: Loss functions are an essential part in modern data-driven approaches, such as bi-level training scheme and machine learnings. In this paper, we propose a loss function consisting of a \(r\)-order (an)-isotropic total variation semi-norms \(\mathrm{TV}^r\), \(r\in\mathbb{R}^+\), defined via the Riemann-Liouville (RL) fractional derivative. We focus on studying key theoretical properties, such as the lower semi-continuity and compactness with respect to both the function and the order of derivative \(r\), of such loss functions.
MSC:
| 26B30 | Absolutely continuous real functions of several variables, functions of bounded variation |
| 94A08 | Image processing (compression, reconstruction, etc.) in information and communication theory |
| 47J20 | Variational and other types of inequalities involving nonlinear operators (general) |
References:
| [1] | Agrawal, O. P., Fractional variational calculus in terms of riesz fractional derivatives, J. Phys. A: Math. Theor.40 (2007) 6287. · Zbl 1125.26007 |
| [2] | Brezis, H., Functional Analysis, Sobolev Spaces and Partial Differential Equations (, Springer2011). · Zbl 1220.46002 |
| [3] | Chen, D., Sun, S., Zhang, C., Chen, Y. and Xue, D., Fractional-order \(\text{TV}- L^2\) model for image denoising, Central Eur. J. Phys.11 (2013) 1414-1422. |
| [4] | Evans, L. C. and Gariepy, R. F., Measure Theory and Fine Properties of Functions (CRC Press, 2015). · Zbl 1310.28001 |
| [5] | I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville and Y. Bengio, Generative adversarial networks, CoRR, abs/1406.2661 (2014). |
| [6] | He, K., Zhang, X., Ren, S. and Sun, J., Deep residual learning for image recognition, in 2016 IEEE Conf. Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, , June 27-30, 2016 (IEEE Computer Society, 2016), pp. 770-778. |
| [7] | Huang, G., Liu, Z., van der Maaten, L. and Weinberger, K. Q., Densely connected convolutional networks, in 2017 IEEE Conf. Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, , July 21-26, 2017 (IEEE Computer Society, 2017), pp. 2261-2269. |
| [8] | Idczak, D. and Walczak, S. A., Fractional Sobolev spaces via Riemann-Liouville derivatives, J. Funct. Spaces Appl.2013 (2013) 128043. · Zbl 1298.46033 |
| [9] | Jumarie, G., Modified Riemann-Liouville derivative and fractional Taylor series of nondifferentiable functions further results, Comput. Math. Appl.51 (2006) 1367-1376. · Zbl 1137.65001 |
| [10] | Krizhevsky, A., Sutskever, I. and Hinton, G. E., Imagenet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems 25: 26th Annual Conf. Neural Information Processing Systems 2012, eds. Bartlett, P. L., Pereira, F. C. N., Burges, C. J. C., Bottou, L., and Weinberger, K. Q., December 3-6, 2012, Lake Tahoe, Nevada, United States, , 2012, pp. 1106-1114. |
| [11] | H. Liu, K. Simonyan and Y. Yang, DARTS: Differentiable architecture search, CoRR, abs/1806.09055 (2018). |
| [12] | Liu, P., Li, C., and Schönlieb, C.-B., GANReDL: Medical image enhancement using a generative adversarial network with real-order derivative induced loss functions, in Medical Image Computing and Computer Assisted Intervention — MICCAI 2019, eds. Shen, D., Liu, T., Peters, T. M., Staib, L. H., Essert, C., Zhou, S., Yap, P.-T., and Khan, A. (Springer International Publishing, Cham, 2019), pp. 110-117. |
| [13] | Rudin, L. I., Osher, S. and Fatemi, E., Nonlinear total variation based noise removal algorithms, Phys. D60 (1992) 259-268. · Zbl 0780.49028 |
| [14] | Samko, S. G., Kilbas, A. A. and Marichev, O. I., Fractional Integrals and Derivatives (Gordon and Breach Science Publishers, 1993). · Zbl 0818.26003 |
| [15] | Simonyan, K. and Zisserman, A., Very deep convolutional networks for large-scale image recognition, in 3rd Int. Conf. Learning Representations, ICLR 2015, eds. Bengio, Y., LeCun, Y. and Diego, S., CA, USA, , May 7-9, 2015 (IEEE, 2015). |
| [16] | Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S. E., Anguelov, D., Erhan, D., Vanhoucke, V. and Rabinovich, A., Going deeper with convolutions, in IEEE Conf. Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, , June 7-12, 2015 (IEEE Computer Society, 2015), pp. 1-9. |
| [17] | Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. and Polosukhin, I., Attention is all you need, in Advances in Neural Information Processing Systems 30: Annual Conf. Neural Information Processing Systems 2017, eds. Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N. and Garnett, R., December 4-9, 2017, Long Beach, CA, USA, , 2017, pp. 5998-6008. |
| [18] | Zhang, J. and Chen, K., A total fractional-order variation model for image restoration with nonhomogeneous boundary conditions and its numerical solution, SIAM J. Imaging Sci.8 (2015) 2487-2518. · Zbl 1327.62388 |
| [19] | Zoph, B. and Le, Q. V., Neural architecture search with reinforcement learning, in 5th Int.Conf. Learning Representations, ICLR 2017, Toulon, France, , April 24-26, 2017, Conference Track Proceedings, OpenReview.net, 2017. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
