Design and Implementation of a Neural Network Based Predistorter for Enhanced Mobile Broadband

We use a feed-forward NN that is fully-connected with $K$ hidden layers, and $N$ neurons per hidden layer. The nonlinear activation applied in hidden layers is chosen to be a rectified linear unit (ReLU), shown in (1), which can easily be implemented with a single multiplexer in hardware.

$\displaystyle\text{ReLU}(x)=\max(0,x)$

(1)

The input and output data to the predistorter is complex-valued, while NNs typically operate on real-valued data. To accommodate this, we split the real and imaginary parts of each time-domain input sample, $x(n)$, on to separate neurons.

Although PA-induced nonlinearities are present in the transmitted signal, the relationship between the input and output data is still mostly linear. Although in principle, a NN can learn this relationship given training data, this turns out to be difficult in practice [15]. As such, we implement a linear bypass in our NN that directly passes the inputs to the output neurons where they are added in with the output from the final hidden layer, as can be seen in Fig. 2. This way, the NN entirely focuses on the nonlinear portion of the signal.

This work primarily focuses on the implementation and running complexity of the DPD application, which consists of inference on a pre-trained NN. The training is assumed to be able to run offline and, once the model is learned, significant updates will not be necessary and occasional offline re-training to account for long-term variations would be sufficient.

In [17], we first use input/output data of the PA to train a NN to model the PA behavior. We then connect a second DPD NN to the PA NN model. We treat the combined DPD NN and PA NN as one large NN. However, during the second training phase, we only update the weights corresponding to the DPD NN. We then connect the DPD NN to the real PA and use it to predistort for the actual device.

The process of predistorting can excite a different region of the PA than when predistortion is not used. To account for this, it is not uncommon in other DPD methods to have multiple training iterations. A similar idea is adopted in [17] and in this work. Once training of the PA and the DPD is performed, we then retransmit through the actual PA while using the DPD NN. Using the new batch of input/output data, we then can update the PA NN model and in turn refine the DPD NN. An example of the iterative training procedure is shown in Fig. 3, where the MSE training loss is shown for the PA NN model and the combined DPD-PA is shown for two training iterations.

An extension of a memory polynomial from [4] is shown in (2). This form of memory polynomial predistorts the complex baseband PA input $x(n)$ to be $\hat{x}(n)$ by computing nonlinearities of the form $x(n)|x(n)|^{p}$ and convolving them with an FIR filter for both $x(n)$ and its conjugate, $x^{*}(n)$. This conjugate processing gives the model the expressive power to combat PA nonlinearities and any IQ imbalance in the system. $P$ and $M$ are the highest nonlinearity order and memory depth in the main branch, while $Q$ and $L$ are the highest order and memory in the conjugate branch. The complex-valued coefficients $\alpha_{p,m}$ and $\beta_{q,l}$ represent the DPD coefficients that need to be learned for nonlinearity orders $p$ and $q$ and memory tap $m$ and $l$. Finally, the DC term $c$ accounts for any local oscillator leakage in the system.

The total number of complex-valued parameters in (2) is given as

$\displaystyle n_{\text{PAR, poly}}=M\left(\frac{P+1}{2}\right)+L\left(\frac{Q+1}{2}\right)+1.$

(3)

Assuming three real multiplications per complex multiplication, we get the following number of multiplications in the system

$\displaystyle n_{\text{MUL, poly}}$

$\displaystyle=3n_{\text{PAR, poly}}+\sum_{\begin{subarray}{c}p=3,\\ p\text{ odd}\end{subarray}}^{P}\frac{1}{2}\left(p+5\right)+\sum_{\begin{subarray}{c}q=3,\\ q\text{ odd}\end{subarray}}^{Q}\frac{1}{2}\left(q+5\right)$

(4)

Here, each complex coefficient accounts for three multiplication. The expression, $x(n)|x(n)|^{p-1}$ is computed once for each $n$ over a given $p$ and delayed in the design to generate the appropriate value for each $m$. We note that $|x(n)|^{p-1}$ can always be simplified to $(\Re{(x(n))^{2}+\Im{(x(n)}}^{2})^{\frac{p-1}{2}}$ since $p$ is odd. This accounts for $(\frac{p-1}{2}+1)$ multiplications before being multiplied by the complex-valued $x(n)$ which adds 2 more multiplications. The same is true for the conjugate processing.

The output of a densely connected NN is given by

$\displaystyle{\mathbf{h}_{1}}(n)=f\left(\mathbf{W}_{1}\begin{bmatrix}\Re(x(n))\\ \Im(x(n))\end{bmatrix}+\mathbf{b}_{1}\right),$

(5)

$\displaystyle{\mathbf{h}_{i}}(n)=f\left(\mathbf{W}_{i}\mathbf{h}_{i-1}(n)+\mathbf{b}_{i}\right),\quad i=2,\ldots,K,$

(6)

$\displaystyle\mathbf{z}(n)=\mathbf{W}_{K+1}\mathbf{h}_{K}(n)+\mathbf{b}_{K+1}+\mathbf{W}_{\text{linear}}\begin{bmatrix}\Re(x(n))\\ \Im(x(n))\end{bmatrix},$

(7)

$\displaystyle\hat{x}(n)=z_{1}(n)+1j\cdot z_{2}(n),$

(8)

where $f$ is a nonlinear activation function (such as the ReLU from (1)), $\mathbf{W}_{i}$ and $\mathbf{b}_{i}$ are weight matrices and bias vectors corresponding to the $i$th layer in the NN, and $j$ is the imaginary unit. The final output of the network after hidden layer $K$ is given by (7) where the first element represents the real part of the signal, and the second element represents the imaginary part. In (7), $W_{\text{linear}}$ is a 2$\times$2 matrix of the weights corresponding to the linear bypass. In practice, we fix it to be the identity matrix, $\mathbf{I}_{2}$, to reduce complexity though these weights could also be learned in systems with significant IQ imbalance.

Assuming $N$ neurons per hidden layer and $K$ hidden layers, the number of multiplications is given by

$\displaystyle n_{\text{MUL, NN}}=4N+(K-1)N^{2}.$

(9)

The performance results for each predistorter as a function of the number of required multiplications are shown in Figs. 4–6. These results were obtained using the RFWebLab platform [21]. RFWebLab is a web-connected PA at Chalmers University. This system uses a Cree CGH40006-TB GaN PA with a peak output power of 6 W. The precision is 14 bits for the feedback on the ADC and 16 bits for the DAC. Using their Matlab API, we test the NN predistorter using a 10 MHz OFDM signal. This signal has random data on 600 subcarriers spaced apart by 15 kHz and is similar to LTE signals commonly used in cellular deployments. It provides an interesting test scenario in that it has a sufficiently high peak-to-average power ratio (PAPR) to make predistortion challenging. We train on 10 symbols then validate on 10 different symbols. The Adam optimizer is used with an MSE loss function. ReLU activation functions are used in the hidden layer neurons.

Specifically, we tested the following DPDs: 1) a NN DPD with $K=1$ with $N=\{1,...,20,25,31\}$ (dark green). 2) a NN DPD with $K=2$ with $N=\{1,...,8\}$ (light green). 3) a polynomial DPD without memory and with $P=1$ to $P=13$ (dark blue), 2) a polynomial DPD with $M=2$ memory taps and with $P=1$ to $P=13$ (light blue), 3) a polynomial DPD with $M=4$ memory taps and with $P=1$ to $P=13$ (pink), All DPDs were evaluated in terms of the adjacent channel leakage ratio (ACLR), the error vector magnitude (EVM), and the spectra of the post-PA pre-distorted signals. A predistorter with $M=4$ and $Q=P$ was also evaluated. However, the system did not have significant IQ imbalance, so the addition of the conjugate processing to the memory polynomial only had the effect of significantly increasing complexity.

We implement the NN-DPD on FPGA with the goal of realizing high throughput via maximum parallelization and pipeling. The top-level overview of the design is shown in Fig. 7. Here, each wire corresponds to a 16-bit bus. The real and imaginary parts of the PA input signal stream in each clock cycle. Weights are stored in a RAM which can be written to from outside the FPGA design. After the RAM is loaded, the weights and biases are written to individual registers in the neuron processing elements which cache them for fast access during inference. A chain of pipeline registers pass the inputs to the output to be added to the output of the final layer.

After the weights are loaded into RAM, the RAM controller loads each of the weights into a weights cache in each PE. To do this, a counter increments through each address in the RAM. The current address and the value at that address are broadcast to all neurons. Each address corresponds with a specific weight or bias. Whenever the weights cache in a neuron reads addresses corresponding to the weights and biases for its neuron, it saves the data into a register dedicated to that parameter. These registers output to the corresponding multiplier or adder.

An example neuron PE is shown in Fig. 8. Each PE is implemented with a sufficient number of multipliers for performing the multiplication of the weights by the inputs in parallel. The results from each multiplier are added together, along with the bias and passed to the ReLU activation function, which is implemented with a single multiplexer.

The memory polynomial is also implemented using 16 bits throughout the design. We target the design for maximum throughput by fully parallelizing and pipelining it so that a new time-domain input sample can streamed in each clock cycle. The main overall structure of the design is shown in Fig. 9. Each polynomial “branch” of the memory polynomial corresponding to nonlinear order $p$ computes $x(n)|x(n)|^{p-1}$ and there is a branch for each $p$ in the design. This computation from each branch is passed to an FIR filter with complex taps. Three multiplications are used for each complex multiplication in each filter. A RAM is implemented to interface with some outside controller for receiving updated weights. Once the coefficients $\alpha$ and $\beta$ are loaded into the design, they can be moved from the RAM to registers near each multiply similarly to the cache implemented in the NN design.

The Xilinx Vivado post-place-and-route utilization results are shown in Table I. Overall, the NN-based design offers numerous advantages over the memory polynomial. Specifically, for the target of an ACLR less than -32 dB, the NN requires 48% of the lookup tables, 42% of the flip-flops, and 15% reduction in the number of digital signal processors. In terms of timing, there is a 9.6% increase in throughput with a 46% decrease in latency. These reductions in utilization occur while also seeing improved ACLR.