Recursive Geman-McClure Estimator for Implementing Second-Order Volterra Filter

Adaptive Volterra filters have been thoroughly studied in diverse applications [1, 2, 3, 4]. However, such filters become computationally expensive when a large number of multidimensional coefficients are required. To overcome this problem, many methods were developed [5, 6, 7, 8]. Among these, the second-order Volterra (SOV) filter was widely applied to identify nonlinear system with acceptable error level [9].

The impulsive noise is a great challenge for nonlinear system identification. It has been shown that the impulsive noise could be better modeled by $\alpha$-stable distribution [10]. A symmetric $\alpha$-stable distribution probability density function (PDF) is defined by means of its characteristic function [10] $\psi(\omega)=\exp\left\{-\gamma|\omega|^{\alpha}\right\}$, where $0<\alpha\leq 2$ is the characteristic exponent, controlling the heaviness of the PDF tails, and $\gamma>0$ is the dispersion, which plays an analogous role to the variance. Such $\alpha$-stable noise tends to produce “outliers”. The recursive least square (RLS), based on the second-order moment, is not robust against outliers [11]. To address stability problem in impulsive noise environments, several RLS-based algorithms were proposed [12, 13, 14]. In [13], a recursive least $p$-norm (RL$p$N) algorithm was proposed. However, this algorithm only achieves improved performance when $p$ closes to $\alpha$, where $p$ is the order of moments [13]. Another strategy is named as recursive least M-estimate (RLM) algorithm which exploits the M-estimate function to suppress the outliers [14]. Although it is superior to several existing outlier-resistant methods, it suffers from performance degradation in highly impulsive noise environments.

Motivated by these considerations, we employ another M-estimator, named Geman-McClure estimate [15], for nonlinear system identification. Like the $\mathcal{L}_{p}$ estimator, the Geman-McClure estimator is a non-convex M-estimator, which is more efficient for learning system [16]. To the best of our knowledge, no adaptive algorithms can achieve improved performance in both Gaussian and $\alpha$-stable noise environments. By integrating the Geman-McClure estimator in the SOV filter structure, the proposed SOV recursive Geman-McClure algorithm achieves smaller steady-state kernel error as compared with the state-of-art algorithms. Note that a significant reduction is in fact vital from an engineering application perspective. Such a gain would likely justify the increase in computational complexity required to run the proposed algorithm. The fact that a considerable amount of mathematics is needed to derive the algorithms is of no consequence for practical applications, where the cost of hardware principally matters. In this paper, by proper application of mathematical concepts (even if at times cumbersome), we showed that key information about the signal environment could be extracted can be from their observation. Particularly, our main contributions are listed as follows:

Fig. 1 shows the nonlinear system identification model based on the SOV filter, where $x(n)$ and $y(n)$ denote the input and output data, $e(n)=d(n)-y(n)$ denotes the error signal, $d(n)$ is the desired signal, and $\xi(n)$ is the noise signal. Given desired signal $d(n)$ that satisfies a model of the form

we want to estimate an $L\times 1$ unknown vector $\bm{h}_{o}$, where $L$ denotes the length of the SOV filter. The expanded input vector $\bm{x}(n)$ and the corresponding expanded coefficient vector ${\bm{\hat{h}}}(n)$ of the SOV system at time $n$ are expressed as

where $M$ denotes the length of the linear kernel, and $h_{r}$ stands for the $r$th-order Volterra kernel. Thus, the output of the SOV filter is expressed as

In this case, $L=M(M+3)/2$. In practice, the noise signal $\xi(n)$ may be either Gaussian or non-Gaussian. Hence, it is very clear that efforts make sense in pursuing a more effective SOV-algorithm that satisfies faster convergence and smaller misalignment.

The conventional Geman-McClure estimator has the following form [15]:

where $\sigma$ is a constant that modulates the shape of the loss function. Fig. 2 shows the score functions $\phi(e)$ of the M-estimator and the Geman-McClure estimator, where $\phi(e)=\partial\Theta(e)/\partial e$. It can be observed that for larger values of $e$, the weight updation is small and thus the algorithm is stable in the presence of impulsive noise. For performance improvement, the type of recursive algorithms is usually preferred. Inspired by these merits, the cost function of the proposed algorithm is defined as follows:

where $0\ll\lambda<1$ is the forgetting factor. The error signal $e(k,n)$ can be expressed as

Taking the gradient of $J(n)$ with respect to the weight vector ${\bm{\hat{h}}}(n)$, and letting the equation be zero, we have

where $\rho(k,n)=\frac{\sigma^{2}}{(\sigma^{2}+e^{2}(k,n))^{2}}$ is the weighting factor. Then, the expression of ${\bm{\hat{h}}}(n)$ can be rewritten as:

where ${\bm{P}}(n)={\bm{\Phi}}^{-1}(n)$, ${\bm{\Phi}}(n)=\sum\limits_{k=1}^{n}\lambda^{n-k}\rho(k,n)\bm{x}(k){\bm{x}^{\mathrm{T}}}(i)$ and ${\bm{\theta}}(n)=\sum\limits_{k=1}^{n}\lambda^{n-i}\rho(k,n)d(k)\bm{x}(k)$. If $\rho(k,n)=1$, the above update equation becomes the conventional RLS algorithm. If $\rho(k,n)\neq 1$, $\bm{P}(n)$ and ${\bm{\theta}}(n)$ are the weighted autocorrelation matrix and the weighted cross-correlation vector of the optimal weights via $\rho(k,n)$. We have to recalculate (9) at each iteration. In our previous studies, an online recursive method is considered to overcome this limitation [11]. By using this approach, $\bm{P}(n)$ and ${\bm{\theta}}(n)$ can be adapted by

Using the matrix inversion lemma [17], the adaptation equation of ${\bm{P}}(n)$ can be obtained as

where ${\bm{P}}(0)=\zeta^{-1}{\bf I}$, ${\bf I}$ is an identity matrix, $\zeta=0.01$ is a small positive number, and the gain vector is

Rewriting (9) in a recursive way, we can obtain the following update equation for ${\bm{\hat{h}}}(n)$

Remark 1: In the expression (14), one can see an implicit relationship between ${\bm{\hat{h}}}(n)$ and $\rho(n,n)$. The algorithm requires an iterative approximation to the optimal solution, where $\rho(n,n)$ is calculated by using ${\bm{\hat{h}}}(n-1)$, and the new value for ${\bm{\hat{h}}}(n)$ is obtained with the compute the value of $\rho(n,n)$.

Remark 2: The proposed algorithm is easy to implement in the SOV filter, since it does not require any priori information on the noise characteristics. It requires two parameters ($\sigma$ and $\lambda$) to improve the overall nonlinear filtering performance.

In this section, given the Assumptions, we theoretically study the performance of the proposed SOV algorithm. Because the output of the Volterra-based algorithms linearly depend on the coefficients of the filter itself, we can follow the approach in [17, 18] for analyzing the mean performance and steady-state excess mean square error (EMSE) of SOV recursive Gemen-McClure algorithm.

To begin with, we define the weight deviation vector as $\widetilde{{\bm{h}}}(n)\triangleq{\bm{h}_{o}}-{\bm{\hat{h}}}(n)$. In steady-state, the a priori error $e_{a}(n)$ and the a posterior error $e_{p}(n)$ are defined as $e_{a}(n)\triangleq{\bm{x}^{\mathrm{T}}}(n)\widetilde{{\bm{h}}}(n-1),\;e_{p}(n)\triangleq{\bm{x}^{\mathrm{T}}}(n)\widetilde{{\bm{h}}}(n)$. The mathematical analysis needs the following assumptions.
•  Assumptions
a) The input signal $\bm{x}(n)$ is independent and identically distributed (i.i.d.) with zero-mean, and $\left\lVert\bm{x}(n)\right\rVert_{\bm{\Omega}}^{2}$ is approximately independent of the a priori error $e_{a}(n)$ at steady-state, where $\left\lVert\bm{x}\right\rVert_{\bm{\Omega}}^{2}={\bm{x}}^{\mathrm{T}}{\bm{\Omega}}{\bm{x}}$ stands for the squared-weighted Euclidean norm of a vector.

To verify the theoretical findings and to illustrate the effectiveness of the proposed algorithm, we present simulations when implemented in Matlab R2013a running on a 2.1GHz AMD processor with 4GB of RAM. The EMSE and normalized mean square deviation (NMSD) $=20{\mathrm{log}}_{10}{\mathrm{E}}\{{||{\bm{\hat{h}}}(n)-\bm{h}_{o}||_{2}}\}/{||\bm{h}_{o}||_{2}}$ are employed to evaluate the performance. The results are averaged over 300 independent simulations.

In this paper, we proposed a recursive Geman-McClure algorithm based on SOV filter, which was derived by minimizing the Geman-McClure estimator of the error signal. Detailed steady-state analysis was presented. One of the advantages of the proposed algorithm is that it has only two parameters, the constant $\sigma$ and the forgetting factor $\lambda$, which have quite wide ranges for choice to achieve excellent performance. We carried out computer simulations that support the analytical findings and confirm the effectiveness of the proposed algorithm. Note that the variance of $\alpha$-stable noise is infinite, so it is very difficult, if not impossible, to conduct a steady-state performance analysis of the proposed algorithm. Similarly, theoretical analysis of global stability and convergence is also very hard and rare for the Volterra filters in the presence of $\alpha$-stable noise. Rigorous mathematical analysis is lacking in a long period. We leave these investigations for future work.