×

Physics-informed neural networks with hard constraints for inverse design. (English) Zbl 1478.35242

Summary: Inverse design arises in a variety of areas in engineering such as acoustic, mechanics, thermal/electronic transport, electromagnetism, and optics. Topology optimization is an important form of inverse design, where one optimizes a designed geometry to achieve targeted properties parameterized by the materials at every point in a design region. This optimization is challenging, because it has a very high dimensionality and is usually constrained by partial differential equations (PDEs) and additional inequalities. Here, we propose a new deep learning method-physics-informed neural networks with hard constraints (hPINNs) – for solving topology optimization. hPINN leverages the recent development of PINNs for solving PDEs, and thus does not require a large dataset (generated by numerical PDE solvers) for training. However, all the constraints in PINNs are soft constraints, and hence we impose hard constraints by using the penalty method and the augmented Lagrangian method. We demonstrate the effectiveness of hPINN for a holography problem in optics and a fluid problem of Stokes flow. We achieve the same objective as conventional PDE-constrained optimization methods based on adjoint methods and numerical PDE solvers, but find that the design obtained from hPINN is often smoother for problems whose solution is not unique. Moreover, the implementation of inverse design with hPINN can be easier than that of conventional methods because it exploits the extensive deep-learning software infrastructure.

MSC:

35R30 Inverse problems for PDEs
65K10 Numerical optimization and variational techniques
68T20 Problem solving in the context of artificial intelligence (heuristics, search strategies, etc.)
92B20 Neural networks for/in biological studies, artificial life and related topics

References:

[1] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng, TensorFlow: A system for large-scale machine learning, in Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, 2016, pp. 265-283.
[2] S. Badia and F. Verdugo, Gridap: An extensible finite element toolbox in Julia, J. Open Source Softw., 5 (2020), 2520.
[3] N. Baker, F. Alexander, T. Bremer, A. Hagberg, Y. Kevrekidis, H. Najm, M. Parashar, A. Patra, J. Sethian, S. Wild, K. Wilcox, and S. Lee, Workshop Report on Basic Research Needs for Scientific Machine Learning: Core Technologies for Artificial Intelligence, Tech. report, U.S. DOE Office of Science, Washington, DC, 2019.
[4] E. Bayati, R. Pestourie, S. Colburn, Z. Lin, S. G. Johnson, and A. Majumdar, Inverse designed metalenses with extended depth of focus, ACS Photonics, 7 (2020), pp. 873-878.
[5] M. P. Bendsoe and O. Sigmund, Topology Optimization: Theory, Methods, and Applications, Springer-Verlag, Berlin, Heidelberg, 2013. · Zbl 1059.74001
[6] D. P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, London, 2014. · Zbl 0572.90067
[7] T. Borrvall and J. Petersson, Topology optimization of fluids in Stokes flow, Internat. J. Numer. Methods Fluids, 41 (2003), pp. 77-107. · Zbl 1025.76007
[8] R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu, A limited memory algorithm for bound constrained optimization, SIAM J. Sci. Comput., 16 (1995), pp. 1190-1208, https://doi.org/10.1137/0916069. · Zbl 0836.65080
[9] N. J. Champagne II, J. G. Berryman, and H. M. Buettner, FDFD: A \(3\) D finite-difference frequency-domain code for electromagnetic induction tomography, J. Comput. Phys., 170 (2001), pp. 830-848. · Zbl 0984.78012
[10] X. Chen, J. Duan, and G. E. Karniadakis, Learning and meta-learning of stochastic advection-diffusion-reaction systems from sparse measurements, Eur. J. Appl. Math., 32 (2021), pp. 397-420. · Zbl 1538.68040
[11] Y. Chen, L. Lu, G. E. Karniadakis, and L. Dal Negro, Physics-informed neural networks for inverse problems in nano-optics and metamaterials, Opt. Express, 28 (2020), pp. 11618-11633.
[12] A. Dener, M. A. Miller, R. M. Churchill, T. Munson, and C.-S. Chang, Training Neural Networks under Physical Constraints Using a Stochastic Augmented Lagrangian Approach, preprint, https://arxiv.org/abs/2009.07330, 2020.
[13] S. Dong and N. Ni, A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks, preprint, https://arxiv.org/abs/2007.07442, 2020.
[14] X. Duan, X. Qin, and F. Li, Topology optimization of Stokes flow using an implicit coupled level set method, Appl. Math. Model., 40 (2016), pp. 5431-5441. · Zbl 1465.65070
[15] J. W. Goodman, Introduction to Fourier Optics, Roberts and Company Publishers, Greenwood Village, CO, 2005.
[16] J. K. Guest and J. H. Prévost, Topology optimization of creeping fluid flows using a Darcy-Stokes finite element, Internat. J. Numer. Methods Engrg., 66 (2006), pp. 461-484. · Zbl 1110.76310
[17] K. Guo, Z. Yang, C. Yu, and M. J. Buehler, Artificial intelligence and machine learning in design of mechanical materials, Mater. Horiz., 8 (2021), pp. 1153-1172.
[18] X. He, K. Zhao, and X. Chu, AutoML: A survey of the state-of-the-art, Knowl. Based Syst., 212 (2021), 106622.
[19] J. Hicken and J. Alonso, Comparison of reduced-and full-space algorithms for PDE-constrained optimization, in Proceedings of the 51st AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition, 2013, 1043.
[20] J. S. Jensen and O. Sigmund, Topology optimization for nano-photonics, Laser Photonics Rev., 5 (2011), pp. 308-321.
[21] J. Jiang, M. Chen, and J. A. Fan, Deep neural networks for the evaluation and design of photonic devices, Nat. Rev. Mater., 6 (2021), pp. 679-700.
[22] J. Jiang and J. A. Fan, Simulator-based training of generative neural networks for the inverse design of metasurfaces, Nanophotonics, 1 (2019), pp. 1059-1069.
[23] P. Jin, L. Lu, Y. Tang, and G. E. Karniadakis, Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness, Neural Netw., 130 (2020), pp. 85-99. · Zbl 1475.68315
[24] S. G. Johnson, Notes on Perfectly Matched Layers (PMLs), preprint, https://arxiv.org/abs/2108.05348, 2021.
[25] M. Kadic, G. W. Milton, M. van Hecke, and M. Wegener, 3D metamaterials, Nat. Rev. Phys., 1 (2019), pp. 198-210.
[26] G. Karniadakis and S. J. Sherwin, Spectral/hp Element Methods for Computational Fluid Dynamics, 2nd ed., Numer. Math. Sci. Comput., Oxford University Press, New York, 2005. · Zbl 1116.76002
[27] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, Physics-informed machine learning, Nat. Rev. Phys., 3 (2021), pp. 422-440.
[28] D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, preprint, https://arxiv.org/abs/1412.6980, 2014.
[29] G. Kissas, Y. Yang, E. Hwuang, W. R. Witschey, J. A. Detre, and P. Perdikaris, Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks, Comput. Methods Appl. Mech. Eng., 358 (2020), 112623. · Zbl 1441.76149
[30] K. Kojima, B. Wang, U. Kamilov, T. Koike-Akino, and K. Parsons, Acceleration of FDTD-based inverse design using a neural network approach, in Integrated Photonics Research, Silicon and Nanophotonics, Optical Society of America, 2017, ITu1A.4.
[31] P. L. Lagari, L. H. Tsoukalas, S. Safarkhani, and I. E. Lagaris, Systematic construction of neural forms for solving partial differential equations inside rectangular domains, subject to initial, boundary and interface conditions, Int. J. Artif. Intell., 29 (2020), 2050009.
[32] I. E. Lagaris, A. Likas, and D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw. Learn. Syst., 9 (1998), pp. 987-1000.
[33] D. Liu, Y. Tan, E. Khoram, and Z. Yu, Training deep neural networks for the inverse design of nanophotonic structures, ACS Photonics, 5 (2018), pp. 1365-1369.
[34] Z. Liu, D. Zhu, S. P. Rodrigues, K. Lee, and W. Cai, Generative model for the inverse design of metasurfaces, Nano Lett., 18 (2018), pp. 6570-6576.
[35] L. Lu, X. Meng, Z. Mao, and G. E. Karniadakis, DeepXDE: A deep learning library for solving differential equations, SIAM Rev., 63 (2021), pp. 208-228, https://doi.org/10.1137/19M1274067. · Zbl 1459.65002
[36] X. Luo, M. R. Maxey, and G. E. Karniadakis, Smoothed profile method for particulate flows: Error analysis and simulations, J. Comput. Phys., 228 (2009), pp. 1750-1769. · Zbl 1409.76102
[37] K. Maute and O. Sigmund, Topology optimization approaches: A comparative review, Struct. Multidiscip. Optim., 48 (2013), pp. 1031-1055.
[38] S. Molesky, Z. Lin, A. Y. Piggott, W. Jin, J. Vucković, and A. W. Rodriguez, Inverse design in nanophotonics, Nat. Photonics, 12 (2018), pp. 659-670.
[39] V. Nagarajan and J. Z. Kolter, Generalization in Deep Networks: The Role of Distance from Initialization, preprint, https://arxiv.org/abs/1901.01672, 2019.
[40] Y. Nandwani, A. Pathak, and P. Singla, A primal dual formulation for deep learning with constraints, in Advances in Neural Information Processing Systems, 2019, pp. 12157-12168.
[41] J. Nocedal and S. Wright, Numerical Optimization, Springer, New York, 2006. · Zbl 1104.65059
[42] G. Pang, L. Lu, and G. E. Karniadakis, fPINNs: Fractional physics-informed neural networks, SIAM J. Sci. Comput., 41 (2019), pp. A2603-A2626, https://doi.org/10.1137/18M1229845. · Zbl 1420.35459
[43] R. Pestourie, Assume Your Neighbor is Your Equal: Inverse Design in Nanophotonics, Ph.D. thesis, Harvard University, Cambridge, MA, 2020.
[44] R. Pestourie, Y. Mroueh, T. V. Nguyen, P. Das, and S. G. Johnson, Active learning of deep surrogates for PDEs: Application to metasurface design, npj Comput. Mater., 6 (2020), 164.
[45] R. Pestourie, C. Pérez-Arancibia, Z. Lin, W. Shin, F. Capasso, and S. G. Johnson, Inverse design of large-area metasurfaces, Opt. Express, 26 (2018), pp. 33732-33747.
[46] J. Peurifoy, Y. Shen, L. Jing, Y. Yang, F. Cano-Renteria, B. G. DeLacy, J. D. Joannopoulos, M. Tegmark, and M. Soljačić, Nanophotonic particle simulation and inverse design using artificial neural networks, Sci. Adv., 4 (2018), eaar4206.
[47] T. Poggio, K. Kawaguchi, Q. Liao, B. Miranda, L. Rosasco, X. Boix, J. Hidary, and H. Mhaskar, Theory of Deep Learning III: Explaining the Non-overfitting Puzzle, preprint, https://arxiv.org/abs/1801.00173, 2017.
[48] M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378 (2019), pp. 686-707. · Zbl 1415.68175
[49] M. Raissi, A. Yazdani, and G. E. Karniadakis, Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations, Science, 367 (2020), pp. 1026-1030. · Zbl 1478.76057
[50] F. Sahli Costabal, Y. Yang, P. Perdikaris, D. E. Hurtado, and E. Kuhl, Physics-informed neural networks for cardiac activation mapping, Front. Phys., 8 (2020), 42.
[51] H. Sasaki and H. Igarashi, Topology optimization accelerated by deep learning, IEEE Trans. Magn., 55 (2019), pp. 1-5.
[52] V. Schulz and M. Siebenborn, Computational comparison of surface metrics for PDE constrained shape optimization, Comput. Methods Appl. Math., 16 (2016), pp. 485-496. · Zbl 1342.49065
[53] V. Sekar, M. Zhang, C. Shu, and B. C. Khoo, Inverse design of airfoil using a deep convolutional neural network, AIAA J., 57 (2019), pp. 993-1003.
[54] H. Sheng and C. Yang, PFNN: A Penalty-Free Neural Network Method for Solving a Class of Second-Order Boundary-Value Problems on Complex Geometries, preprint, https://arxiv.org/abs/2004.06490, 2020.
[55] Y. Shin, J. Darbon, and G. E. Karniadakis, On the Convergence of Physics Informed Neural Networks for Linear Second-order Elliptic and Parabolic Type PDEs, preprint, https://arxiv.org/abs/2004.01806, 2020. · Zbl 1473.65349
[56] K. Shukla, A. D. Jagtap, and G. E. Karniadakis, Parallel Physics-Informed Neural Networks via Domain Decomposition, preprint, https://arxiv.org/abs/2104.10013, 2021.
[57] S. So, T. Badloe, J. Noh, J. Rho, and J. Bravo-Abad, Deep learning enabled inverse design in nanophotonics, Nanophotonics, 9 (2020), pp. 1041-1057.
[58] M. H. Tahersima, K. Kojima, T. Koike-Akino, D. Jha, B. Wang, C. Lin, and K. Parsons, Deep neural network inverse design of integrated photonic power splitters, Sci. Rep., 9 (2019), pp. 1-9.
[59] A. M. Tartakovsky, C. O. Marrero, P. Perdikaris, G. D. Tartakovsky, and D. Barajas-Solano, Physics-informed deep neural networks for learning parameters and constitutive relationships in subsurface flow problems, Water Resour. Res., 56 (2020), e2019WR026731.
[60] M. Toussaint, Introduction to Optimization: Constrained Optimization, teaching lecture, 2014.
[61] I. Wang and J. C. Spall, Stochastic optimisation with inequality constraints using simultaneous perturbations and penalty functions, Int. J. Control, 81 (2008), pp. 1232-1238. · Zbl 1152.90549
[62] Z. Wang, M. S. Triantafyllou, Y. Constantinides, and G. E. Karniadakis, A spectral-element/Fourier smoothed profile method for large-eddy simulations of complex VIV problems, Comput. Fluids, 172 (2018), pp. 84-96. · Zbl 1410.76320
[63] D. A. White, W. J. Arrighi, J. Kudo, and S. E. Watts, Multiscale topology optimization using neural network surrogate models, Comput. Methods Appl. Mech. Engrg., 346 (2019), pp. 1118-1135. · Zbl 1440.74319
[64] A. Yazdani, L. Lu, M. Raissi, and G. E. Karniadakis, Systems biology informed deep learning for inferring parameters and hidden dynamics, PLoS Comput. Biol., 16 (2020), e1007575.
[65] D. Zhang, L. Guo, and G. E. Karniadakis, Learning in modal space: Solving time-dependent stochastic PDEs using physics-informed neural networks, SIAM J. Sci. Comput., 42 (2020), pp. A639-A665, https://doi.org/10.1137/19M1260141. · Zbl 1440.60067
[66] D. Zhang, L. Lu, L. Guo, and G. E. Karniadakis, Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems, J. Comput. Phys., 397 (2019), 108850. · Zbl 1454.65008
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.