Hostname: page-component-7b7567c8c-wxvwp Total loading time: 0 Render date: 2026-11-15T08:37:41.645Z Has data issue: false hasContentIssue false
  • English
  • Français

Deep learning models for global coordinate transformations that linearise PDEs

Part of:Partial differential equationsNumerical problems in dynamical systemsApproximation methods and numerical treatment of dynamical systemsConnections between Deep learning and Partial Differential EquationsArtificial intelligence (68Txx)

Published online by Cambridge University Press:  24 September 2020

CRAIG GINOpen the ORCID record for CRAIG GIN[Opens in a new window],
BETHANY LUSCHOpen the ORCID record for BETHANY LUSCH[Opens in a new window],
STEVEN L. BRUNTON and
J. NATHAN KUTZOpen the ORCID record for J. NATHAN KUTZ[Opens in a new window]
CRAIG GIN
Affiliation:
Department of Applied Mathematics, University of Washington, Seattle, WA 98195, USA emails: crgin@ncsu.edu; kutz@uw.edu
BETHANY LUSCH
Affiliation:
Argonne Leadership Computing Facility, Argonne National Laboratory, Lemont, IL 60439, USA email: blusch@anl.gov
STEVEN L. BRUNTON
Affiliation:
Department of Applied Mathematics, University of Washington, Seattle, WA 98195, USA emails: crgin@ncsu.edu; kutz@uw.eduDepartment of Mechanical Engineering, University of Washington, Seattle, WA 98195, USA email: sbrunton@uw.edu
J. NATHAN KUTZ
Affiliation:
Department of Applied Mathematics, University of Washington, Seattle, WA 98195, USA emails: crgin@ncsu.edu; kutz@uw.edu
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

We develop a deep autoencoder architecture that can be used to find a coordinate transformation which turns a non-linear partial differential equation (PDE) into a linear PDE. Our architecture is motivated by the linearising transformations provided by the Cole–Hopf transform for Burgers’ equation and the inverse scattering transform for completely integrable PDEs. By leveraging a residual network architecture, a near-identity transformation can be exploited to encode intrinsic coordinates in which the dynamics are linear. The resulting dynamics are given by a Koopman operator matrix K. The decoder allows us to transform back to the original coordinates as well. Multiple time step prediction can be performed by repeated multiplication by the matrix K in the intrinsic coordinates. We demonstrate our method on a number of examples, including the heat equation and Burgers’ equation, as well as the substantially more challenging Kuramoto–Sivashinsky equation, showing that our method provides a robust architecture for discovering linearising transforms for non-linear PDEs.

Keywords

Koopman theorydeep neural networksresidual networkslinearising transformsCole–Hopf transform

MSC classification

Primary:35A22: Transform methods (e.g. integral transforms)
Secondary:35A35: Theoretical approximation to solutions37M99: None of the above, but in this section65P99: None of the above, but in this section68T99: None of the above, but in this section

Information

Type
Papers
Information
European Journal of Applied Mathematics , Volume 32 , Special Issue 3: Connections between Deep learning and Partial Differential Equations , June 2021, pp. 515 - 539
Check for updates
Copyright
© The Author(s), 2020. Published by Cambridge University Press

References

Ablowitz, M. J. & Segur, H. (1981) Solitons and the Inverse Scattering Transform, Vol. 4, SIAM, Philadelphia, PA.CrossRefGoogle Scholar
Bishop, C. (2006) Pattern Recognition and Machine Learning, Springer, New York, NY.Google Scholar
Boyce, W. E. & DiPrima, R. C. (2008) Elementary Differential Equations, 9th ed., Wiley, Hoboken, NJ.Google Scholar
Brunton, S. L., Brunton, B. W., Proctor, J. L. & Kutz, J. N. (2016) Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control. PLOS ONE 11, 1–19.CrossRefGoogle ScholarPubMed
Brunton, S. L. & Kutz, J. N. (2019) Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control, Cambridge University Press, Cambridge.CrossRefGoogle Scholar
Budišić, M. & Mezić, I. (2012) Geometry of the ergodic quotient reveals coherent structures in flows. Physica D Nonlinear Phenomena 241, 1255–1269.CrossRefGoogle Scholar
Champion, K., Lusch, B., Kutz, J. N. & Brunton, S. L. (2019) Data-driven discovery of coordinates and governing equations. Proc. Nat. Acad. Sci. 116, 22445–22451.Google Scholar
Cole, J. D. (1951) On a quasi-linear parabolic equation occurring in aerodynamics. Quart. Appl. Math. 9, 225–236.CrossRefGoogle Scholar
Cybenko, G. (1989) Approximation by superpositions of a sigmoidal function. Math. Control Sig. Syst. (MCSS) 2, 303–314.CrossRefGoogle Scholar
Dsilva, C. J., Talmon, R., Coifman, R. R. & Kevrekidis, I. G. (2018) Parsimonious representation of nonlinear dynamical systems through manifold learning: a chemotaxis case study. Appl. Comput. Harmonic Anal. 44, 759–773.CrossRefGoogle Scholar
Foias, C., Jolly, M., Kevrekidis, I. & Titi, E. (1994) On some dissipative fully discrete nonlinear Galerkin schemes for the Kuramoto-Sivashinsky equation. Phys. Lett. A 186, 87–96.CrossRefGoogle Scholar
Gonzalez-Garcia, R., Rico-Martinez, R. & Kevrekidis, I. (1998) Identification of distributed parameter systems: a neural net based approach. Comput. Chem. Eng. 22, S965–S968.CrossRefGoogle Scholar
Goodfellow, I., Bengio, Y. & Courville, A. (2016) Deep Learning, MIT Press, Cambridge, MA. http://www.deeplearningbook.org.Google Scholar
Haberman, R. (2004) Applied Partial Differential Equations: with Fourier Series and Boundary Value Problems, Pearson Prentice Hall, Upper Saddle River, NJ.Google Scholar
He, K., Zhang, X., Ren, S. & Sun, J. (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Ccomputer Vision and Pattern Recognition, pp. 770–778.CrossRefGoogle Scholar
Hopf, E. (1950) The partial differential equation u t + uu x = μu xx . Comm. Pure App. Math. 3, 201–230.CrossRefGoogle Scholar
Hornik, K., Stinchcombe, M. & White, H. (1990) Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Networks 3, 551–560.CrossRefGoogle Scholar
Kassam, A. & Trefethen, L. (2005) Fourth-order time stepping for stiff PDEs. SIAM J. Sci. Comput. 26, 1214–1233.CrossRefGoogle Scholar
Klus, S., Nüske, F., Koltai, P., Wu, H., Kevrekidis, I., Schütte, C. & Noé, F. (2018) Data-driven model reduction and transfer operator approximation. J. Nonlinear Sci. 28, 985–1010.CrossRefGoogle Scholar
Koopman, B. O. (1931) Hamiltonian systems and transformation in Hilbert space. Proc. Nat. Acad. Sci. 17, 315–318.Google Scholar
Kutz, J. N. (2013) Data-Driven Modeling & Scientific Computation: Methods for Complex Systems & Big Data, Oxford University Press, Oxford.Google Scholar
Kutz, J. N., Brunton, S. L., Brunton, B. W. & Proctor, J. L. (2016) Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, Philadelphia, PA.CrossRefGoogle Scholar
Kutz, J. N., Proctor, J. L. & Brunton, S. L. (2018) Applied Koopman theory for partial differential equations and data-driven modeling of spatio-temporal systems. Complexity 2018, 1–16.CrossRefGoogle Scholar
Li, Q., Dietrich, F., Bollt, E. M. & Kevrekidis, I. G. (2017) Extended dynamic mode decomposition with dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator. Chaos Interdiscip. J. Nonlinear Sci. 27, 103111.CrossRefGoogle ScholarPubMed
Lu, L., Shin, S., Su, Y. & Karniadakis, G. (2019) Dying ReLU and initialization: theory and numerical examples. arXiv:1903.06733.Google Scholar
Lu, L., Su, Y. & Karniadakis, G. (2018) Collapse of deep and narrow neural nets. arXiv:1808.04947.Google Scholar
Lusch, B., Kutz, J. N. & Brunton, S. L. (2018) Deep learning for universal linear embeddings of nonlinear dynamics. Nat. Commun. 9, 4950.CrossRefGoogle ScholarPubMed
Mallat, S. (2016) Understanding deep convolutional networks. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 374, 20150203.CrossRefGoogle ScholarPubMed
Mardt, A., Pasquali, L., Wu, H. & Noé, F. (2018) VAMPnets: deep learning of molecular kinetics. Nat. Commun. 9, 5.CrossRefGoogle ScholarPubMed
Mezić, I. (2005) Spectral properties of dynamical systems, model reduction and decompositions. Nonlinear Dyn. 41, 309–325.CrossRefGoogle Scholar
Mezić, I. (2013) Analysis of fluid flows via spectral properties of the Koopman operator. Ann. Rev. Fluid Mech. 45, 357–378.CrossRefGoogle Scholar
Mezić, I. & Banaszuk, A. (2004) Comparison of systems with complex behavior. Physica D Nonlinear Phenomena 197, 101–133.CrossRefGoogle Scholar
Neu, J. C. (1980) The method of near-identity transformations and its applications. SIAM J. Appl. Math. 38, 189–208.CrossRefGoogle Scholar
Noé, F. & Nüske, F. (2013) A variational approach to modeling slow processes in stochastic dynamical systems. Multiscale Model. Simul. 11, 635–655.CrossRefGoogle Scholar
Nüske, F., Keller, B. G., Pérez-Hernández, G., Mey, A. S. & Noé, F. (2014) Variational approach to molecular kinetics. J. Chem. Theory Comput. 10, 1739–1752.CrossRefGoogle ScholarPubMed
Otto, S. E. & Rowley, C. W. (2019) Linearly-recurrent autoencoder networks for learning dynamics. SIAM J. Appl. Dyn. Syst. 18, 558–593.CrossRefGoogle Scholar
Page, J. & Kerswell, R. R. (2018) Koopman analysis of burgers equation. Phys. Rev. Fluids 3, 071901.CrossRefGoogle Scholar
Pan, S. & Duraisamy, K. (2020) Physics-informed probabilistic learning of linear embeddings of nonlinear dynamics with guaranteed stability. SIAM J. Appl. Dyn. Syst. 19, 480–509.CrossRefGoogle Scholar
Rico-Martinez, R., Kevrekidis, I. & Krischer, K. (1995) Nonlinear system identification using neural networks: dynamics and instabilities. In: Neural Networks for Chemical Engineers, pp. 409–442.Google Scholar
Rowley, C. W., Mezić, I., Bagheri, S., Schlatter, P. & Henningson, D. (2009) Spectral analysis of nonlinear flows. J. Fluid Mech. 645, 115–127.CrossRefGoogle Scholar
Schmid, P. J. (2010) Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech. 656, 5–28.CrossRefGoogle Scholar
Takeishi, N., Kawahara, Y. & Yairi, T. (2017) Learning Koopman invariant subspaces for dynamic mode decomposition. In: Advances in Neural Information Processing Systems, pp. 1130–1140.Google Scholar
Wehmeyer, C. & Noé, F. (2017) Time-lagged autoencoders: deep learning of slow collective variables for molecular kinetics. J. Chem. Phys. 148, 241703.CrossRefGoogle Scholar
Wiggins, S. (2003) Introduction to Applied Nonlinear Dynamical Systems and Chaos, Vol. 2, Springer, New York, NY.Google Scholar
Williams, M. O., Kevrekidis, I. G. & Rowley, C. W. (2015) A data-driven approximation of the Koopman operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25, 1307–1346.CrossRefGoogle Scholar
Williams, M. O., Rowley, C. W. & Kevrekidis, I. G. (2015) A kernel-based method for data-driven Koopman spectral analysis. J. Comput. Dyn. 2, 247–265.CrossRefGoogle Scholar
Yeung, E., Kundu, S. & Hodas, N. (2019) Learning deep neural network representations for Koopman operators of nonlinear dynamical systems. In: 2019 American Control Conference (ACC), pp. 4832–4839.CrossRefGoogle Scholar