×

Complete set of translation invariant measurements with Lipschitz bounds. (English) Zbl 1448.94052

Summary: In image and audio signal classification, a major problem is to build stable representations that are invariant under rigid motions and, more generally, to small diffeomorphisms. Translation invariant representations of signals in \(\mathbb{C}^n\) are of particular importance. The existence of such representations is intimately related to classical invariant theory, inverse problems in compressed sensing and deep learning. Despite an impressive body of literature on the subject, most representations available are either not stable due to the presence of high frequencies or non discriminative. In the present paper, we construct low dimensional representations of signals in \(\mathbb{C}^n\) that are invariant under finite unitary group actions, as a special case we establish the existence of low-dimensional and complete \(\mathbb{Z}_m\)-invariant representations for any \(m \in \mathbb{N}\). Our construction yields a stable, discriminative transform with semi-explicit Lipschitz bounds on the dimension; this is particularly relevant for applications. Using some tools from Algebraic Geometry, we define a high dimensional homogeneous function that is injective. We then exploit the projective character of this embedding and see that the target space can be reduced significantly by using a generic linear transformation. Finally, we introduce the notion of non-parallel map, which is enjoyed by our function and employ this to construct a Lipschitz modification of it.

MSC:

94A12 Signal theory (characterization, reconstruction, filtering, etc.)
22F05 General theory of group and pseudogroup actions
42C40 Nontrigonometric harmonic analysis involving wavelets and other special systems
47B32 Linear operators in reproducing-kernel Hilbert spaces (including de Branges, de Branges-Rovnyak, and other structured spaces)

Software:

ckn_kernel

References:

[1] Ma, Yingyi; Ganapathiraman, Vignesh; Zhang, Xinhua, Learning invariant representations with kernel warping, (Chaudhuri, Kamalika; Sugiyama, Masashi, Proceedings of Machine Learning Research (2019)), 1003-1012
[2] Balan, Radu; Zou, Dongmian, On Lipschitz analysis and Lipschitz synthesis for the phase retrieval problem, Linear Algebra Appl., 496, 152-181 (2016) · Zbl 1332.15042
[3] Balan, Radu; Casazza, Pete; Edidin, Dan, On signal reconstruction without phase, Appl. Comput. Harmon. Anal., 20, 3, 345-356 (2006) · Zbl 1090.94006
[4] Bandeira, Afonso S.; Cahill, Jameson; Mixon, Dustin G.; Nelson, Aaron A., Saving phase: injectivity and stability for phase retrieval, Appl. Comput. Harmon. Anal., 37, 1, 106-125 (2014) · Zbl 1305.90330
[5] Bandeira, Afonso S.; Blum-Smith, Ben; Kileel, Joe; Perry, Amelia; Weed, Jonathan; Wein, Alexander S., Estimation under group actions: recovering orbits from invariants (2017), arXiv preprint
[6] Bengio, Yoshua, Deep learning of representations: looking forward, (Statistical Language and Speech Processing. Statistical Language and Speech Processing, SLSP 2013. Statistical Language and Speech Processing. Statistical Language and Speech Processing, SLSP 2013, Lecture Notes in Computer Science, vol. 7978 (2013)) · Zbl 1192.68503
[7] Bethge, M.; Gerwinn, S.; Macke, J. H., Unsupervised learning of a steerable basis for invariant image representations, (Proceedings of SPIE Human Vision and Electronic Imaging XII (EI105) (2007)), 64920C
[8] Bietti, Alberto; Mairal, Julien, Group invariance, stability to deformations, and complexity of deep convolutional representations, J. Mach. Learn. Res., 20, 1-49 (2019) · Zbl 1483.68335
[9] Bronstein, Michael M.; Bruna, Joan; LeCun, Yann; Szlam, Arthur; Vandergheynst, Pierre, Geometric deep learning: going beyond Euclidean data, IEEE Signal Process. Mag., 34, 4, 18-42 (2017)
[10] Bruna, Joan; Mallat, Stéphane, Invariant scattering convolution networks, IEEE Trans. Pattern Anal. Mach. Intell., 35, 8, 1872-1886 (2013)
[11] Bruna, Joan; Mallat, Stephane, Multiscale sparse microcanonical models, Math. Stat. Learn., 1, 3, 257-315 (2018) · Zbl 1426.62111
[12] Burago, Dmitri; Burago, Yuri; Ivanov, Sergei, A Course in Metric Geometry, vol. 33 (2001), American Mathematical Soc. · Zbl 0981.51016
[13] Cahill, Jameson; Casazza, Peter; Daubechies, Ingrid, Phase retrieval in infinite-dimensional Hilbert spaces, Trans. Am. Math. Soc., Ser. B, 3, 3, 63-76 (2016) · Zbl 1380.46015
[14] Candes, Emmanuel J.; Romberg, Justin K.; Tao, Terence, Stable signal recovery from incomplete and inaccurate measurements, Commun. Pure Appl. Math., 59, 8, 1207-1223 (2006) · Zbl 1098.94009
[15] Cireşan, Dan Claudiu; Meier, Ueli; Gambardella, Luca Maria; Schmidhuber, Jürgen, Deep, big, simple neural nets for handwritten digit recognition, Neural Comput., 22, 12, 3207-3220 (2010)
[16] Conca, Aldo; Edidin, Dan; Hering, Milena; Vinzant, Cynthia, An algebraic characterization of injectivity in phase retrieval, Appl. Comput. Harmon. Anal., 38, 2, 346-356 (2015) · Zbl 1354.42003
[17] Derksen, Harm; Kemper, Gregor, Computational Invariant Theory (2015), Springer · Zbl 1332.13001
[18] Donoho, David L., Compressed sensing, IEEE Trans. Inf. Theory, 52, 4, 1289-1306 (2006) · Zbl 1288.94016
[19] Dufresne, Emilie, Separating invariants (2008), PhD thesis · Zbl 1173.13004
[20] Eppenhof, Koen A. J.; Lafarge, Maxime W.; Moeskops, Pim; Veta, Mitko; Pluim, Josien P. W., Deformable Image Registration Using Convolutional Neural Networks, Medical Imaging 2018: Image Processing, vol. 10574, 105740S (2018), International Society for Optics and Photonics
[21] Gu, Jiuxiang; Wang, Zhenhua; Kuen, Jason; Ma, Lianyang; Shahroudy, Amir; Shuai, Bing; Liu, Ting; Wang, Xingxing; Wang, Gang; Cai, Jianfei, Recent advances in convolutional neural networks, Pattern Recognit., 77, 354-377 (2018)
[22] Harris, Joe, Algebraic Geometry: A First Course, vol. 133 (2013), Springer Science & Business Media
[23] Hirn, Matthew; Poilvert, Nicolas; Mallat, Stephane, Wavelet scattering regressions of quantum chemical energies, Multiscale Model. Simul., 15, 2, 827-867 (2017) · Zbl 1365.42030
[24] Liu, Mengyi; Li, Shaoxin; Shan, Shiguang; Wang, Ruiping; Chen, Xilin, Deeply learning deformable facial action parts model for dynamic expression analysis, (Asian Conference on Computer Vision (2014), Springer), 143-157
[25] Mallat, Stéphane, Group invariant scattering, Commun. Pure Appl. Math., 65, 10, 1331-1398 (2012) · Zbl 1282.47009
[26] Meer, Peter; Lenz, Reiner; Ramakrishna, Sudhir, Efficient invariant representations, Int. J. Comput. Vis., 26, 2, 137-152 (1998)
[27] Mroueh, Youssef; Voinea, Stephen; Poggio, Tomaso, Learning with group invariant features: a kernel perspective, (Advances in Neural Information Processing Systems (2015)), 1558-1566
[28] Reimers, Fabian, Separating invariants of finite groups, J. Algebra, 507, 19-46 (2018) · Zbl 1395.13002
[29] Rohé, Marc-Michel; Datar, Manasi; Heimann, Tobias; Sermesant, Maxime; Pennec, Xavier, Svf-net: learning deformable image registration using shape matching, (International Conference on Medical Image Computing and Computer-Assisted Intervention (2017), Springer), 266-274
[30] Rong, Yi; Wang, Yang; Xu, Zhiqiang, Almost everywhere matrix recovery, Appl. Comput. Harmon. Anal. (2019)
[31] Vedaldi, A.; Soatto, S., Viewpoint induced deformation statistics and the design of viewpoint invariant features, (Singularities and Occlusions. Computer Vision. Singularities and Occlusions. Computer Vision, ECCV 2006. Singularities and Occlusions. Computer Vision. Singularities and Occlusions. Computer Vision, ECCV 2006, Lecture Notes in Computer Science, vol. 3952 (2006)), 360-373
[32] Vinzant, Cynthia, A small frame and a certificate of its injectivity, (2015 International Conference on Sampling Theory and Applications. 2015 International Conference on Sampling Theory and Applications, SampTA (2015), IEEE), 197-200
[33] Wang, Yang; Xu, Zhiqiang, Generalized phase retrieval: measurement number, matrix recovery and beyond, Appl. Comput. Harmon. Anal. (2017) · Zbl 1423.42059
[34] Wood, Jeffrey, Invariant pattern recognition: a review, Pattern Recognit., 29, 1, 1-17 (1996)
[35] Xu, Weixia; Huang, Dingjiang; Zhou, Shuigeng, Statistical learning with group invariance: problem, method and consistency, Int. J. Mach. Learn. Cybern., 10, 6, 1503-1511 (2019)
[36] Zhang, X.; Lee, W. S.; Teh, Y. W., Learning with invariance via linear functionals on reproducing kernel Hilbert space, (NIPS (2013))
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.