Deep Neural Networks with Koopman Operators for Modeling and Control of Autonomous Vehicles

Autonomous vehicles and driving technologies have received notable attention in the past several decades. Autonomous vehicles are promising to free the hands of humans from tedious long-distance drivings and have huge potential to reduce traffic congestion and accidents. A classic autonomous driving system usually consists of key modules of perception, localization, decision-making, trajectory planning and control. In trajectory planning and control, the information of vehicle dynamics is usually required for realizing agile and safe maneuver, especially in complex and unstructured road scenarios.

In order to realize high-performance trajectory tracking of autonomous vehicles, many control methods, such as Linear Quadratic Regulator (LQR) [1] and Model Predictive Control (MPC) [2], rely on vehicle model information with different levels and structures. However, to obtain a precise model of vehicle dynamics is difficult for the following reasons: i) the vehicle dynamics in the longitudinal and lateral directions are highly coupled; ii) strong non-linearity and model uncertainty become dominant factors especially when the vehicle reaches the limit of tire-road friction [3, 4, 5]. Among the existing approaches, classic physics-based modeling methods rely on Newton’s second law of motion, where multiple physical parameters are required to be determined, see [6, 7]. Indeed, some model parameters, such as cornering stiffness coefficient, are not measurable and difficult to be estimated [8, 9, 10].

As an alternative to analytic modeling of vehicle dynamics, machine learning techniques have been used in recent years for building data-driven models of vehicles, see [11, 12]. However, deep neural networks commonly lack interpretability, which has recently been noted as a challenge for applications with safety requirements and remains as a cutting-edge research topic. As a result, the models established by deep neural networks might have unknown sensitive modes and be fragile to model uncertainties. Also, due to the nonlinear activation functions, the obtained dynamics model is not easy to be used for designing a well-posed controller such as MPC, LQR, and so forth. Recently, the Koopman operator, being an invariant linear operator (probably of infinite dimension), has been regarded as a powerful tool for capturing the intrinsic characteristics of a nonlinear system via linear evolution in the lifted observable space. To obtain a realistic model description, usually, dimensionality reduction methods such as Dynamic Mode Decomposition (DMD) in [13, 14] and Extended Dynamic Mode Decomposition (EDMD) in [15] can be used to approximate the Koopman operator with a finite dimension matrix. In [16], a data-driven identification method using Koopman operator was proposed for vehicle dynamics but the basis functions were either manually designed or determined by the knowledge of the nonlinear dynamics, which is a nontrivial task due to the dynamics in the longitudinal and lateral directions are strongly coupled and highly nonlinear. It is necessary to study learning-based construction methods for basis functions of the Koopman operator. Furthermore, the data-driven vehicle models studied in previous works have not been verified in practical controller design process.

To solve the above problems, this paper proposes a novel data-driven vehicle modeling approach based on deep neural networks with an interpretable Koopman operator. In the proposed approach, a deep learning-based extended dynamic mode decomposition (Deep EDMD) algorithm is presented to learn a finite basis function set of the Koopman operator. Furthermore, a data-driven model predictive controller is designed for path tracking control of autonomous vehicles by making use of the dynamic model learned by Deep EDMD. In the proposed algorithm, deep neural networks serve as the encoder and decoder in the framework of EDMD for learning the Koopman operator. The simulation studies in a high-fidelity CarSim environment are performed for performance validation. The results show that the proposed Deep EDMD method outperforms EDMD, deep neural network (DNN), and extreme learning machine-based EDMD (ELM-EDMD, designed using the ELM described in [17]) algorithms in terms of modeling performance. Also, the model predictive controller designed with Deep EDMD can realize satisfactory trajectory tracking performance and meet real-time implementation requirements.

The contributions of the paper are two-fold:

1) An EDMD-based deep learning approach (Deep EDMD) is proposed for identifying an integrated vehicle dynamical model in both longitudinal and lateral directions. Different from other machine learning-based approaches, deep neural networks play the role of learning feature representations for EDMD in the framework of the Koopman operator. Also, the deep learning network is utilized for automatically learning suitable observable basis functions, which is in contrast to that being manually selected or based on prior model information as in [16]. The resulting model is a linear global model in the lifted space with a nonlinear mapping from the original state space.

2) A novel model predictive controller was designed with the learned model using Deep EDMD (called DE-MPC) for time-varying reference tracking of autonomous vehicles. In DE-MPC, the linear part of the model is used as the predictor in the prediction horizon of the MPC, while the nonlinear mapping function is used for resetting the initial lifted state at each time instant with the real state.

The rest of this paper is structured as follows. In section II, the most related works are reviewed, while the dynamics of four-wheel vehicles are shown in Section III. Section IV presents the main idea of the Deep EDMD method for data-driven model learning. And Section V gives a linear MPC with Deep EDMD for reference tracking. The simulation results are reported in SectionVI, while some conclusions are drawn in Section VII.

In the past decades, neural networks were widely studied to approximate dynamical systems due to their powerful representation capability. Earlier works in [18, 19] have resorted to recurrent neural networks (RNNs) for the approximation of dynamics. It was verified that any continuous curve can be represented by the output of RNNs. In [20], an RNN-based modeling approach was presented to model the dynamical steering behavior of an autonomous vehicle. The obtained nonlinear model was used to design a nonlinear MPC for realizing steering control. In addition to RNNs, fruitful contributions have resorted to artificial neural networks (ANNs) with multiple hidden layers for the approximation of vehicle dynamics. Among them, [11] used fully-connected layers to model the vehicle dynamics in the lateral direction. The physics model was considered for determining the inputs of the networks to guarantee the reliability of the modeling results. It is noteworthy that most of the models with neural networks are nonlinear, and the identification process may suffer from the vanishing of gradients. In recent years, the reliability in identification with neural networks has received notable attention. In [21], different levels of convolution-like neural network structures were proposed for modeling the longitudinal velocity of vehicles. During the construction of neural networks, it integrated the physics principles to improve the robustness and interpretability of the modeling. Note that, most of the above approaches assumed the decoupling of dynamics and approximate in the sole longitudinal direction or lateral direction. As a fact, couplings between both directions can be naturally dealt with when it comes to use multi-variable control techniques, such as MPC and LQR.

As an alternative, the Koopman operator has been recently realized to be a powerful tool for representing nonlinear dynamics. The main idea of the Koopman operator was initially introduced by Koopman [22], with the goal of describing the nonlinear dynamics using a linear model with the state space constructed by observable functions. It has been proven that the linear model with the Koopman operator can ideally capture all the characteristics of the nonlinear system as long as the state space adopted is invariant. That is to say, the structure with the Koopman operator is interpretable. A sufficient condition for the convergence of the Koopman operator is that an infinite number of observable functions are provided.

For practical reasons, dimensionality reduction methods, such as DMDs, have been used to obtain a linear model of finite order, see [13, 23] and the references therein. The order of the resultant model corresponds to the number of observable functions used. In general, the observable functions are directly measured with multiple sensors. To avoid redundant sensors, efforts have been contributed to extending the idea of DMDs for constructing observable functions, see [24, 25, 15]. In EDMDs, the observable functions are designed using basis functions probably of multiple types, such as Gaussian functions, polynomial functions, and so forth. The convergence has been proven in [26] under the assumption that the state space constructed by basis functions is invariant or of infinite dimension.

From a practical viewpoint, the approximation performance might be sensitive to basis functions, hence the selection of basis functions requires specialist experience. Motivated by the above problems, recent works have resorted to design deep neural networks to generate observable functions automatically. In [27], an autoencoder is utilized to learn dictionary functions and Koopman modes for unforced dynamics. The finite-horizon approximation of the Koopman operator was computed via solving a least-squares problem. In [28], a deep variational Koopman (DVK) model was proposed with a Long Short Term Memory (LSTM) network for learning the distribution of the lifted functions. A deep learning network was used in [29] to learn Koopman modes in the framework of DMD. From the application aspect, a deep DMD method has been proposed for the modal analysis of fluid flows in [30]. In [31] and [32], the Koopman operator has been extended to the modeling of power systems. In [33], a highway traffic prediction model was built based on the Koopman operator. Other applications can be founded for representing neurodynamics in [34] and molecular dynamics in [35, 36], and model reduction in [37]. In the aspect of robot modeling and control, a DMD-based modeling and controlling approach was proposed for soft robotics in [38] and the Koopman operator with EDMD was applied to approximate vehicle dynamics in [16]. Different from the above works, we propose a deep neural network to learn the basis functions of the Koopman operator in the framework of EDMD, with the goal of data-driven modeling of vehicles in both longitudinal and lateral directions. The deep neural networks serve as the encoder and decoder whose weights are automatically updated in the learning process, avoiding the manual selection problem of observable functions as in [16].

In this paper, we consider the type of autonomous vehicles consisting of four wheels with front-wheel-steering functionality. A sketch of the vehicle dynamics is depicted in Figure 1.

As shown in Figure 1, according to [3, 39], the longitudinal and lateral forces can be represented as

where $F_{xw_{i}}$, $F_{yw_{i}}$ denote the longitudinal and lateral forces of tire $i$, $\delta_{i}$ denotes the wheel angle of tire $i$, and $\delta_{3}=\delta_{4}=0$. The term $-\frac{1}{2}\rho AV^{2}\left[\begin{array}[]{c}c_{fx}\\ c_{fy}\\ \end{array}\right]$ denotes the longitudinal and lateral air-resistance forces applied to the vehicle body. The parameter $\rho$, $A$, and $V$ denote the air mass density, the frontal area to the airflow, and the relative air speed respectively, and where $c_{fx}$ and $c_{fy}$ are drag coefficients defined as configurable functions with respect to the yaw angle. $F_{x}$ and $F_{y}$ can be calculated by the Pacejka model in [4, 39], which is an empirical formula given as

where $H=\varLambda_{\ast}F_{z}$, and where $F_{z}$ is the tire load while $\varLambda$ is a function with respect to the longitudinal slip $s$ and sideslip angle $\alpha$. Besides, $s$ and $\alpha$ are functions with respect to the steering wheel angle $\zeta$ and the engine $\eta$ (stands for the throttle if $\eta\geq 0$ and for the brake otherwise). That is to say, $F_{x}$ is a function with respect to $\zeta$ and $\eta$, i.e. $s=\vartheta\left(\zeta,\eta\right)$. $E$ and $D$ are shape factors and $G=\frac{C_{\alpha}}{HE}$, where $C_{\alpha}$ is the cornering stiffness. According to the Newton-Euler equation, we can obtain equations of motion as follows:

where $v_{x}$ and $v_{y}$ are the longitudinal and lateral velocities respectively, $m$ is the mass of vehicle, $I_{z}$ is the yaw moment of inertia, $\psi$ is the yaw angle, $\beta$ is the slip angle, $\delta=\frac{\delta_{1}+\delta_{2}}{2}$, $a_{1}$ and $a_{2}$ are the distances between the front and rear axles to the center of gravity, $C_{\alpha f}=C_{\alpha fL}+C_{\alpha fR}$ and $C_{\alpha r}=C_{\alpha rL}+C_{\alpha rR}$.

Combining (1)-(3g), it is possible to write the vehicle dynamics with interactions in the longitudinal and lateral directions, that is

where the state $x=(v_{x},v_{y},\dot{\psi})$, $u=(\zeta,\eta)$, where the steering wheel angle $\zeta$ establishes a relation with $\delta_{i}$, i.e., $\delta=\varUpsilon(\zeta)$. The choice of dynamics constructed as (4) avoids to measure $F_{x}$, $F_{y}$, and $F_{z}$ with expensive sensors, but it is challenging from the modeling viewpoint due to the strong non-linearity being hidden in $f_{c}$. For this reason, we propose a deep neural network method based on the Koopman operator to capture the global dynamics of (4) with a linear model using lifted abstract state space.

In this section, we first give the formulation with the Koopman operator and its numerical approximation with EDMD for approximating vehicle dynamics. Then, we present the main idea of the Deep EDMD algorithm for modeling the vehicle dynamics, where a deep neural network is utilized to construct an observable subspace of the Koopman operator automatically.

In this section, a novel model predictive controller based on Deep EDMD, i.e., DE-MPC, is proposed to show how the learned dynamical model can be utilized for controlling the nonlinear system with good performance. To this end, the learned model (27) is firstly used to define an augmented version with the input treated as the extended state, i.e.,

where $\Delta u_{t}=u_{t}-u_{t-1}$,

$\mathscr{C}=\left[\begin{matrix}I_{L\times L}&0_{L\times m}\\ \end{matrix}\right],\xi_{t}=\left[\begin{array}[]{c}\chi_{t}\\ u_{t-1}\\ \end{array}\right]$. And the state initialization at any time instant is given as

It is now ready to state a model predictive controller with (28) and (29). The objective of DE-MPC is to steer the time-varying reference $y_{ref}$, which can be computed with the lifted observable function using the reference trajectory. To obtain satisfactory tracking performance, we also include the original state $x$ as the abstract one in the lifted space. Therefore, at any time instant $t$, the finite horizon optimization problem can be stated as

where $N_{p}$ is the prediction horizon, $N_{c}$ is the control horizon, $Q$, $R$ are positive-definite matrices for penalizing the tracking errors and the control increment; while $\rho>0$ is the penalty matrix of the slack variable $\varepsilon$ and $\boldsymbol{1}_{m}\in\mathbb{R}^{m}$ is a vector with all the entries being $1$. Note that we use $N_{p}>N_{c}$ and the control increment $\Delta u_{t}=0$ is assumed for all $N_{c}\leq t\leq N_{p}$. In the following we will reformulate  (30c) in a more straightforward manner. First, let $\mathcal{Y}_{t}=\left[y_{t+1}^{\top}\,\,y_{t+2}^{\top}\,\,\cdots\,\,y_{t+N_{p}}^{\top}\right]^{\top}$, one can compute

where

$G_{t}=\left[g_{t}^{\top}\,\,g_{t+1}^{\top}\,\,\cdots\,\,g_{t+N_{c}-1}^{\top}\right]^{\top}$, for $G=U$, $g=u$ and $G=\Delta U$, $g=\Delta u$. Therefore, we can reformulate optimization problem (30c) as

where $\mathcal{H}=\text{diag}\{\varTheta^{\top}Q_{\mathcal{Y}}\varTheta+R_{\mathcal{Y}},\rho\}$, $\mathcal{G}_{t}=\left[\begin{matrix}2E_{t}^{\top}Q_{\mathcal{Y}}\varTheta&0\\ \end{matrix}\right]$, $\mathcal{P}_{t}=E_{t}^{\top}Q_{\mathcal{Y}}E_{t}$ and $E_{t}=\varGamma\xi_{t}-\mathcal{Y}_{ref,t}$, and $\mathcal{Y}_{ref,t}$ is defined as the sequence ${y}_{ref,t+1},{y}_{ref,t+2},\cdots,{y}_{ref,t+N_{p}}$. The matrices $Q_{\mathcal{Y}}=\text{diag}\{Q,\cdots,Q\}\in\mathbb{R}^{NN_{p}\times NN_{p}}$ and $R_{\mathcal{Y}}=\text{diag}\{R,\cdots,R\}\in\mathbb{R}^{mN_{c}\times mN_{c}}$ .

Let $\Delta U_{t|t}=[\Delta u_{t|t}^{\top}\ \cdots\ \Delta u_{t+N_{c}-1|t}^{\top}]^{\top}$ be the optimal solution to (32) at time $t$, the control applied to the system is

Then the optimization problem is solved again in the time $t+1$ according to the moving horizon strategy. To clearly illustrate the algorithm of DE-MPC, the implementing steps are summarized in Algorithm 2.

In this section, the proposed Deep EDMD method was validated in a high-fidelity CarSim simulation environment. The comparisons with EDMD, ELM-EDMD, and DNN were considered to show the effectiveness of our approach.

In this paper, we propose a novel data-driven vehicle modeling and control approach based on deep neural networks with an interpretable Koopman operator. In the proposed approach, a deep learning-based extended dynamic mode decomposition (Deep EDMD) algorithm is presented to learn a finite basis function set of the Koopman operator. Based on the dynamic model learned by Deep EDMD, a novel model predictive controller called DE-MPC is presented for path tracking control of autonomous vehicles. In the proposed algorithm, deep neural networks serve as the encoder and decoder in the framework of EDMD for learning the Koopman operator. The differences of our approach with the classic DNN for modeling vehicle dynamics can be summarized as follows: i) in our approach, the deep learning networks are designed as the encoder and decoder to learn the Koopman eigenfunctions and Koopman modes, which is different from classic machine learning-based modeling methods; ii) the resulting model is a global predictor with a linear dynamic evolution and a nonlinear static mapping function, hence it is friendly for real-time implementation with optimization-based methods, such as MPC.

Simulation studies with data sets obtained from a high-fidelity CarSim environment have been performed, including the comparisons with EDMD, ELM-EDMD, and DNN. The results show the effectiveness of our approach and the advantageous point in terms of modeling accuracy than EDMD, ELM-EDMD, and DNN. Also, our approach is more robust than DNN in the scenario which has the random hidden and bias layer in the training process. We also tested the performance of DE-MPC for realizing trajectory tracking in the CarSim environment. Satisfactory tracking performance further verifies the effectiveness of the proposed Deep EDMD and DE-MPC methods. Future research will utilize the learned model for trajectory tracking in a real-world experimental platform of autonomous vehicles.