Data-driven time-scale separation of ODE right-hand sides using dynamic mode decomposition and time delay embedding
Abstract
Multi-physics simulation often involve multiple different scales. The ARKODE ODE solver package in the SUNDIALS library addresses multi-scale problems with a multi-rate time-integrator that can work with a right-hand side that has fast scale and slow scale components. In this report, we use dynamic mode decomposition and time delay embedding to extract the fast and and slow components of the right-hand sides of a simple ODE from data. We then use the extracted components to solve the ODE with ARKODE. Finally, to move towards a real-world use case, we attempt to extract fast and slow scale dynamics from synthetic seismic modeling data.
1 Introduction and Overview
As computing power increases, multi-physics simulations which involve the coupling of different physical phenomena are becoming more common and more ambitious. The different physics in these simulations often occur at different scales. As a result, there are inefficiencies when solving the ODE systems that arise because the time-step of the integrator is restricted by the fast scale. The ARKODE [6] ODE solver package in the SUNDIALS library [1] addresses these problems with a multirate time-integrator that is defined by a right-hand side composed of a fast-scale and slow-scale components as shown in Equation (1). Naturally, this leads to the question of how to split the right-hand side appropriately. In this report, we will take a data-driven approach based on the dynamic mode decomposition (DMD), a powerful tool for approximating dynamics from data [4, 7].
(1) |
In [2], Grosek et al., introduced a method for extracting the low-rank and sparse components of a DMD approximation in the context of background and foreground separation in videos, however, we will apply the concept to other dynamical systems where there exists a slow-scale (background) that is much slower than a fast-scale (foreground). In order to successfully reconstruct the dynamics from systems with only one partial state information, we will use time delay embeddings to augment the state. This method was presented by Kamb et al. as a Koopman observable in [3].
After extracting the the fast and slow components of an simple ODE right-hand side, we will solve a multi-rate ODE by feeding the approximated components into ARKODE. Finally, we will attempt to extract fast and slow-scale dynamics from synthetic tsunami modeling data produced with a code developed by Vogl and Leveque based on the Clawpack solver package [8]. The idea is that with some additional work, the extracted components could be used as the fast and slow right hand sides in ARKODE to solve ODE system(s) that stem from the wave propagation equations discussed.
The rest of this report is organized as follows. Section 2 discusses the theory behind the methods used to form our separation algorithm. Section 3 explains the algorithm and its implementation in MATLAB. Section 4 discussed the computational results for the toy problem and the synthetic tsunami data. Finally, we provide our conclusions in Section 5.
2 Theoretical Background
In this section we will discuss the theory behind the different methods used in our separation algorithm. Consider a dynamical system
(2) |
where is a vector representing the state of our dynamical system at time , contains the parameters of the system, and represents the dynamics. By sampling (2) with a uniform sampling rate we get the discrete-time flow-map
(3) |
2.1 Koopman Theory
The Koopman operator provides an operator perspective on dynamical systems. The insight behind the operator is that the finite-dimensional, nonlinear dynamics of (2) can be transformed to an infinite-dimensional, linear dynamical system by considering a new space of scalar observable functions on the state instead of the state directly [3]. The Kooopman operator acts on the observable:
where maps the measurement function to the next set of value it will take after time .
2.2 Dynamic Mode Decomposition
By sampling (2) with a uniform sampling rate, the DMD approximates the low-dimensional modes of the linear, time-independent Koopman operator in order to estimate the dynamics of the system. Consider data points with a total of samplings in time, then we can form the two matrices:
(4) |
(5) |
The Koopman operator provides a mapping that takes the system, modeled by the data, from time to , i.e. . We can use the singular value decomposition (SVD) of the matrix to reduce the rank as well as to obtain the benefits of unitary matrices. Accordingly, , where is unitary, is diagonal, and is unitary. Thus, the approximate eigendecomposition of the Koopman operator is:
the eigenvalues are given by , and the DMD mode (eigenvector of the Koopman operator) corresponding to eigenvalue is .
Using the approximate eigendecomposition of the Koopman operator we obtain the DMD reconstruction of the dynamics:
(6) |
where
If we consider components of a dynamical system that changes very slowly with time with respect to the fast components, it becomes clear that they will have an associated mode such that
(7) |
For this reason the DMD can be used to separate the dynamics into a fast and slow component. Assume that , where , satisfies Equation (7), and that is bounded away from zero. Then we can redefine Equation (6) for a multi-scale dynamical system as:
(8) |
If we consider the low-rank reconstruction (9) of the DMD and the fact that (10) should be true, it follows that the sparse reconstruction given can be calculated using the real values of elements only.
(9) |
(10) |
(11) |
(12) |
2.3 Time Delay Embedding
Delay embedding is a classic technique for overcoming partial state information. In the context of the DMD and Koopman operator theory, we can use time delay embedding to construct a new observable: with lag time . This is the delay embedding of the trajectory . We can use the embedding to form the Hankel matrix:
(13) |
By examining the singular values of the Hankel matrix we can find the best rank- approximation of the trajectory space. Then, we can form the matrices and for the DMD from and set for the best DMD approximation using the embedding [3].
3 Algorithm Implementation and Development
We develop the algorithm with the assumption we are given measurements and snapshots of a dynamical system with fast and slow scales, but only partial state information. This data forms the matrix . Since we only have partial state information, the first step in the algorithm is to employ time delay embeddings to form a Hankel matrix with embeddings. We choose an that minimizes the approximation error in the second stage of the algorithm.
The second step in the algorithm is to form the matrices and from , and then apply the DMD method with a rank-reduction that is chosen based on the singular values of . This produces the DMD approximation to the dynamical system as given in Equation (6).
The third step in the algorithm is to extract the fast and slow components by applying Equations (8) – (12). These fast and slow components can then be used to form the fast and slow right-hand side functions provided to the ARKODE multirate integrator.

4 Computational Results
We applied the time-scale separation algorithm in two different problems. The first problem is a simple, contrived toy problem. The second scenario uses synthetic seismic data for tsunami modeling that is closer to real-world data.
4.1 Toy Problem
The toy problem uses is defined as the simple IVP:
It is clear the right hand side has a fast component, and a slow component . Using MATLAB we solve the ODE with a uniform time step size of 0.01. The solution is used as our dynamics data that we construct the Hankel matrix with and then perform the DMD. The derivative of the fast and slow components extracted is shown in Figure 4. These components were used in the ARKODE solver by reading the different components in from a data file instead of computing the values of the functions at some time instance.





4.2 Synthetic Seismic Data
The second problem we apply our scale separation algorithm to is a seismic problem where waves, or primary waves, are fast and S waves, or secondary waves are slow. In [8], Vogl et al. present a method for a high-resolution seismic model of a fault slippage under the ocean. In the work they present a linear hyperbolic system of equations that model the 2D plane-strain case of isotropic linear elasticity:
(14) |
A general Riemann solution to produces the eigenvectors of , where for the specific data we will examine. The eigenvectors correspond to P and S waves traveling from the source.
As part of their work, Vogl et al. developed a simulation code for the problem based on the Clawpack solver package. The simulation code takes measurements at varying location from the fault with ”gauges”. The data recorded corresponds to the vector which contains key parameters of the plane-strain equations such as the stress, density and velocity. We build a data matrix from 10 sequential gauges and the vertical velocity. This data matrix is then fed into our time scale separation algorithm.





5 Summary and Conclusions
In summary, in this report, we used dynamic mode decomposition and time delay embedding to extract the fast and and slow components of the right-hand sides of a simple ODE from data. We then used the extracted components to solve the ODE with ARKODE. Finally, to move towards a real-world use case, we attempted to extract fast and slow scale dynamics from synthetic seismic modeling data. We found our algorithm to be adequate with the simple dynamics in a toy problem. In the more difficult case with the synthetic seismic data it showed promise, there seemed to be a split of fast and slow scale components, but more work needs to be done to verify the accuracy of the components. In the future, we would like to explore using the multi-resolution DMD presented in [5] to compare the performance and fit for the use case.
6 Acknowledgements
We would like to thank Christopher Vogl for valuable insight into the seismic model used for the synthetic data experiment and with assistance in collecting the data from the simulation code.
This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. LLNL-TR-848209.
References
- [1] D. J. Gardner, D. R. Reynolds, C. S. Woodward, and C. J. Balos, Enabling new flexibility in the sundials suite of nonlinear and differential/algebraic equation solvers, ACM Trans. Math. Softw., 48 (2022).
- [2] J. Grosek and J. N. Kutz, Dynamic Mode Decomposition for Real-Time Background/Foreground Separation in Video, arXiv e-prints, (2014), p. arXiv:1404.7592.
- [3] M. Kamb, E. Kaiser, S. L. Brunton, and J. N. Kutz, Time-delay observables for koopman: Theory and applications, SIAM Journal on Applied Dynamical Systems, 19 (2020), pp. 886–917.
- [4] J. N. Kutz, S. L. Brunton, B. W. Brunton, and J. L. Proctor, Chapter 1: Dynamic Mode Decomposition: An Introduction, 2016, pp. 1–24.
- [5] J. N. Kutz, X. Fu, and S. L. Brunton, Multiresolution dynamic mode decomposition, SIAM Journal on Applied Dynamical Systems, 15 (2016), pp. 713–735.
- [6] D. R. Reynolds, D. J. Gardner, C. S. Woodward, and R. Chinomona, Arkode: A flexible ivp solver infrastructure for one-step methods, ACM Trans. Math. Softw., (2023). Just Accepted.
- [7] J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton, and J. N. Kutz, On Dynamic Mode Decomposition: Theory and Applications, arXiv e-prints, (2013), p. arXiv:1312.0041.
- [8] C. J. Vogl and R. J. LeVeque, A high-resolution finite volume seismic model to generate seafloor deformation for tsunami modeling, Journal of Scientific Computing, 73 (2017), pp. 1204–1215.