Low-Complexity Near-ML Decoding of Large Non-Orthogonal STBCs using Reactive Tabu Search

MIMO systems that employ non-orthogonal space-time block codes (STBC) from cyclic division algebras (CDA) for arbitrary number of transmit antennas, $N_{t}$, are attractive because they can simultaneously provide both full-rate (i.e., $N_{t}$ complex symbols per channel use, which is same as in V-BLAST) as well as full transmit diversity [1],[2]. The $2\times 2$ Golden code is a well known non-orthogonal STBC from CDA for 2 transmit antennas [3]. High spectral efficiencies of the order of tens of bps/Hz can be achieved using large non-orthogonal STBCs. For e.g., a $16\times 16$ STBC from CDA has 256 complex symbols in it with 512 real dimensions; with 16-QAM and rate-3/4 turbo code, this system offers a high spectral efficiency of 48 bps/Hz. Decoding of non-orthogonal STBCs with such large dimensions, however, has been a challenge. Sphere decoder and its low-complexity variants are prohibitively complex for decoding such STBCs with hundreds of dimensions. Recently, we proposed a low-complexity near-ML achieving algorithm to decode large non-orthogonal STBCs from CDA; this algorithm, which is based on bit-flipping approach, is termed as likelihood ascent search (LAS) algorithm [4]-[6]. In this paper, we present a reactive tabu search (RTS) based approach to near-ML decoding of non-orthogonal STBCs with large dimensions.

Key attractive features of the proposed RTS based decoding are its low-complexity and near-ML performance in systems with large dimensions (e.g., hundreds of dimensions). While creating hundreds of dimensions in space alone (e.g., V-BLAST) requires hundreds of antennas, use of non-orthogonal STBCs from CDA can create hundreds of dimensions with just tens of antennas (space) and tens of channel uses (time). Given that 802.11 smart WiFi products with 12 transmit antennas¹¹112 antennas in these products are now used only for beamforming. Single-beam multi-antenna approaches can offer range increase and interference avoidance, but not spectral efficiency increase. at 2.5 GHz are now commercially available [7] (which establishes that issues related to placement of many antennas and RF/IF chains can be solved in large aperture communication terminals like set-top boxes/laptops), large non-orthogonal STBCs (e.g., $16\hskip-1.42262pt\times\hskip-1.42262pt16$ STBC from CDA) in combination with large dimension near-ML decoding using RTS can enable communications at increased spectral efficiencies of the order of tens of bps/Hz (note that current standards achieve only $<10$ bps/Hz using only up to 4 tx antennas).

Tabu search (TS), a heuristic originally designed to obtain approximate solutions to combinatorial optimization problems [8]-[11], is increasingly applied in communication problems [12]-[14]. For e.g., in [12], design of constellation label maps to maximize asymptotic coding gain is formulated as a quadratic assignment problem (QAP), which is solved using RTS [11]. RTS approach is shown to be effective in terms of BER performance and efficient in terms of computational complexity in CDMA multiuser detection [13]. In [14], a fixed TS based detection in V-BLAST is presented. In this paper, we establish that RTS based decoding of non-orthogonal STBCs can achieve excellent BER performance (near-ML and near-capacity performance) in large dimensions at practically affordable low-complexities. We also present a stopping-criterion for the RTS algorithm. RTS for large dimension non-orthogonal STBC decoding has not been reported so far. Our results in this paper can be summarized as follows:

• •

  Under i.i.d fading and perfect channel state information at the receiver (CSIR), our simulation results show that RTS based decoding of $12\times 12$ STBC from CDA and 4-QAM (288 real dimensions) achieves $i)$ $10^{-3}$ uncoded BER at an SNR of just 0.5 dB away from SISO AWGN performance, and $ii)$ a coded BER performance close to within about 5 dB of the theoretical capacity using rate-3/4 turbo code at a spectral efficiency of 18 bps/Hz.

• •

  Compared to the LAS algorithm we reported recently in [4]-[6], RTS achieves near-SISO AWGN performance with less number of dimensions than with LAS; this is achieved at some extra complexity compared to LAS.

• •

  We report good BER performance when i.i.d fading and perfect CSIR assumptions are relaxed by adopting a spatially correlated MIMO channel model, and a training based iterative RTS decoding/channel estimation scheme.

Consider a STBC MIMO system with multiple transmit and receive antennas. An $(n,p,k)$ STBC is represented by a matrix ${\bf X}_{c}\in{\mathbb{C}}^{n\times p}$, where $n$ and $p$ denote the number of transmit antennas and number of time slots, respectively, and $k$ denotes the number of complex data symbols sent in one STBC matrix. The $(i,j)$th entry in ${\bf X}_{c}$ represents the complex number transmitted from the $i$th transmit antenna in the $j$th time slot. The rate of an STBC is $\frac{k}{p}$. Let $N_{r}$ and $N_{t}=n$ denote the number of receive and transmit antennas, respectively. Let ${\bf H}_{c}\in{\mathbb{C}}^{N_{r}\times N_{t}}$ denote the channel gain matrix, where the $(i,j)$th entry in ${\bf H}_{c}$ is the complex channel gain from the $j$th transmit antenna to the $i$th receive antenna. We assume that the channel gains remain constant over one STBC matrix and vary (i.i.d) from one STBC matrix to the other. Assuming rich scattering, we model the entries of ${\bf H}_{c}$ as i.i.d $\mathcal{C}\mathcal{N}(0,1)$. The received space-time signal matrix, ${\bf Y}_{c}\in{\mathbb{C}}^{N_{r}\times p}$, can be written as

$${\bf Y}_{c}={\bf H}_{c}{\bf X}_{c}+{\bf N}_{c},\vskip-5.69054pt$$

(1)

where ${\bf N}_{c}\in{\mathbb{C}}^{N_{r}\times p}$ is the noise matrix at the receiver and its entries are modeled as i.i.d $\mathcal{C}\mathcal{N}\big{(}0,\sigma^{2}=\frac{N_{t}E_{s}}{\gamma}\big{)}$, where $E_{s}$ is the average energy of the transmitted symbols, and $\gamma$ is the average received SNR per receive antenna [15], and the $(i,j)$th entry in ${\bf Y}_{c}$ is the received signal at the $i$th receive antenna in the $j$th time-slot. Consider linear dispersion STBCs, where ${\bf X}_{c}$ can be written in the form [15]

$\displaystyle{\bf X}_{c}$

$\displaystyle=$

$\displaystyle\sum_{i=1}^{k}x_{c}^{(i)}{\bf A}_{c}^{(i)},$

(2)

where $x_{c}^{(i)}$ is the $i$th complex data symbol, and ${\bf A}_{c}^{(i)}\in{\mathbb{C}}^{N_{t}\times p}$ is its corresponding weight matrix. The received signal model in (1) can be written in an equivalent V-BLAST form as

$${\bf y}_{c}\,\,=\,\,\sum_{i=1}^{k}x_{c}^{(i)}\,(\widehat{{\bf H}}_{c}\,{\bf a}_{c}^{(i)})+{\bf n}_{c}\,\,=\,\,\widetilde{{\bf H}}_{c}{\bf x}_{c}+{\bf n}_{c},$$

(3)

where ${\bf y}_{c}\in{\mathbb{C}}^{N_{r}p\times 1}=vec\,({\bf Y}_{c})$, $\widehat{{\bf H}}_{c}\in{\mathbb{C}}^{N_{r}p\times N_{t}p}=({\bf I}\otimes{\bf H}_{c})$, ${\bf a}_{c}^{(i)}\in{\mathbb{C}}^{N_{t}p\times 1}=vec\,({\bf A}_{c}^{(i)})$, ${\bf n}_{c}\in{\mathbb{C}}^{N_{r}p\times 1}=vec\,({\bf N}_{c})$, ${\bf x}_{c}\in{\mathbb{C}}^{k\times 1}$ whose $i$th entry is the data symbol $x_{c}^{(i)}$, and $\widetilde{{\bf H}}_{c}\in{\mathbb{C}}^{N_{r}p\times k}$ whose $i$th column is $\widehat{{\bf H}}_{c}\,{\bf a}_{c}^{(i)}$, $i=1,2,\cdots,k$. Each element of ${\bf x}_{c}$ is an $M$-PAM/$M$-QAM symbol. Let ${\bf y}_{c}$, $\widetilde{{\bf H}}_{c}$, ${\bf x}_{c}$, ${\bf n}_{c}$ be decomposed into real and imaginary parts as:

	$\displaystyle{\bf y}_{c}={\bf y}_{I}+j{\bf y}_{Q},$	$\displaystyle{\bf x}_{c}={\bf x}_{I}+j{\bf x}_{Q},$
	$\displaystyle{\bf n}_{c}={\bf n}_{I}+j{\bf n}_{Q},$	$\displaystyle\widetilde{{\bf H}}_{c}={\bf H}_{I}+j{\bf H}_{Q}.$			(4)

Further, we define ${\bf H}_{r}\in{\mathbb{R}}^{2N_{r}p\times 2k}$, ${\bf y}_{r}\in{\mathbb{R}}^{2N_{r}p\times 1}$, ${\bf x}_{r}\in{\mathbb{R}}^{2k\times 1}$, and ${\bf n}_{r}\in{\mathbb{R}}^{2N_{r}p\times 1}$ as

	$\displaystyle{\bf H}_{r}=\left(\begin{array}[]{cc}{\bf H}_{I}\hskip 5.69054pt-{\bf H}_{Q}\\ {\bf H}_{Q}\hskip 14.22636pt{\bf H}_{I}\end{array}\right),\hskip 11.38109pt{\bf y}_{r}=[{\bf y}_{I}^{T}\hskip 5.69054pt{\bf y}_{Q}^{T}]^{T},$		(7)
	$\displaystyle\hskip 11.38109pt{\bf x}_{r}=[{\bf x}_{I}^{T}\hskip 5.69054pt{\bf x}_{Q}^{T}]^{T},\hskip 11.38109pt{\bf n}_{r}=[{\bf n}_{I}^{T}\hskip 5.69054pt{\bf n}_{Q}^{T}]^{T}.$		(8)

Now, (3) can be written as

$\displaystyle{\bf y}_{r}$

$\displaystyle=$

$\displaystyle{\bf H}_{r}{\bf x}_{r}+{\bf n}_{r}.$

(9)

Henceforth, we work with the real-valued system in (9). For notational simplicity, we drop subscripts $r$ in (9) and write

$\displaystyle{\bf y}$

$\displaystyle=$

$\displaystyle{\bf H}{\bf x}+{\bf n},\vskip-1.42262pt$

(10)

where ${\bf H}={\bf H}_{r}\in{\mathbb{R}}^{2N_{r}p\times 2k}$, ${\bf y}={\bf y}_{r}\in{\mathbb{R}}^{2N_{r}p\times 1}$, ${\bf x}={\bf x}_{r}\in{\mathbb{R}}^{2k\times 1}$, and ${\bf n}={\bf n}_{r}\in{\mathbb{R}}^{2N_{r}p\times 1}$. We assume that the channel coefficients are known at the receiver but not at the transmitter. Let ${\mathbb{A}}\stackrel{{\scriptstyle\triangle}}{{=}}\{a_{q},q=1,2,\cdots,M\},$ where $a_{q}=2q-1-M$ denote the $M$-PAM signal set from which $x_{i}$ ($i$th entry of ${\bf x}$) takes values, $i=0,\cdots,2k-1$. The ML solution is given by

$\displaystyle{\bf d}_{ML}$

$\displaystyle=$

$\displaystyle{\mbox{arg min}\atop{{\bf d}\in{\mathbb{A}}^{2k}}}\thinspace{\bf d}^{T}{\bf H}^{T}{\bf H}{\bf d}-2{\bf y}^{T}{\bf H}{\bf d},$

(11)

whose complexity is exponential in $k$.

In this section, we present the RTS algorithm for decoding non-orthogonal STBCs. The goal is to get $\widehat{{\bf x}}$, an estimate of ${\bf x}$, given ${\bf y}$ and ${\bf H}$.

Neighborhood Definitions: For each vector in the solution space, define the neighborhood structure as follows. Symbol neighborhood of a signal point $a_{q}\in{\mathbb{A}}$, $q=1,2,\cdots,M$, is defined as the set ${\cal N}(a_{q})\subset{\mathbb{A}}-\{a_{q}\}$; e.g., for 4-PAM, ${\mathbb{A}}=\{-3,-1,1,3\}$, and ${\cal N}(-3)=\{-1,1\}$, ${\cal N}(1)=\{-1,3\}$, and so on. Then, $N\stackrel{{\scriptstyle\triangle}}{{=}}|{\cal N}(a_{q})|,\forall q\in\{1\cdots M\}$ is the number of symbol neighbors of $a_{q}$. Note that the maximum and minimum value $N$ can take is $M-1$ and 1, respectively. Let ${\bf x}^{(m)}=\small{[x_{0}^{(m)}\thinspace x_{1}^{(m)}\cdots x_{2k-1}^{(m)}]}$ denote the data vector in the $m$th iteration. We refer to the vector

$\displaystyle\mathbf{z}^{(m)}(u,v)\,=\,\big{[}z^{(m)}_{0}(u,v)\,\,\,\,z^{(m)}_{1}(u,v)\,\cdots\,z^{(m)}_{2k-1}(u,v)\big{]},$

(12)

as the $(u,v)$th vector neighbor of $\mathbf{x}^{(m)}$, $u=0,\cdots,2k-1$, $v=0,\cdots,N-1$, if $1)$ $\mathbf{x}^{(m)}$ differs from $\mathbf{z}^{(m)}(u,v)$ in the $u$th coordinate, and $2)$ the $u$th element of $\mathbf{z}^{(m)}(u,v)$ is the $v$th symbol neighbor of $x_{u}^{(m)}$. That is,

$$z_{i}^{(m)}(u,v)\,\,=\,\,\left\{\begin{array}[]{ll}x^{(m)}_{i}&\mbox{for}\,\,\,i\neq u\\ w_{v}(x_{u}^{(m)})&\mbox{for}\,\,\,i=u,\end{array}\right.$$

(13)

where $w_{v}(a)$, $v=0,1,\cdots,N-1$ is the $v$th element in ${\cal N}(a)$. So we will have $2kN$ vectors which differ from a given vector in the solution space in only one coordinate. These $2kN$ vectors form the neighborhood of the given vector. It is noted that bit-flipping is a special case with $N=1$ and $M=2$.

$$\hskip-11.38109pt\left[\hskip-2.84526pt\begin{array}[]{cccc}\sum_{i=0}^{n-1}d_{0,i}\,t^{i}&\delta\sum_{i=0}^{n-1}d_{n-1,i}\,\omega_{n}^{i}\,t^{i}&\cdots&\delta\sum_{i=0}^{n-1}d_{1,i}\,\omega_{n}^{(n-1)i}\,t^{i}\\ \sum_{i=0}^{n-1}d_{1,i}\,t^{i}&\sum_{i=0}^{n-1}d_{0,i}\,\omega_{n}^{i}\,t^{i}&\cdots&\delta\sum_{i=0}^{n-1}d_{2,i}\,\omega_{n}^{(n-1)i}\,t^{i}\\ \sum_{i=0}^{n-1}d_{2,i}\,t^{i}&\sum_{i=0}^{n-1}d_{1,i}\,\omega_{n}^{i}\,t^{i}&\cdots&\delta\sum_{i=0}^{n-1}d_{3,i}\,\omega_{n}^{(n-1)i}\,t^{i}\\ \vdots&\vdots&\vdots&\vdots\\ \sum_{i=0}^{n-1}d_{n-2,i}\,t^{i}&\sum_{i=0}^{n-1}d_{n-3,i}\,\omega_{n}^{i}\,t^{i}&\cdots&\delta\sum_{i=0}^{n-1}d_{n-1,i}\,\omega_{n}^{(n-1)i}t^{i}\\ \sum_{i=0}^{n-1}d_{n-1,i}\,t^{i}&\sum_{i=0}^{n-1}d_{n-2,i}\,\omega_{n}^{i}\,t^{i}&\cdots&\sum_{i=0}^{n-1}d_{0,i}\,\omega_{n}^{(n-1)i}\,t^{i}\end{array}\hskip-2.84526pt\right]\hskip 0.0pt(\mbox{9.a})$$

Tabu Matrix: A tabu_matrix of size $2kM\times N$ with non-negative integer entries is created; this matrix will contain the tabu information for all the moves in the search. A non-zero entry in the tabu_matrix means that the corresponding move is a tabu.

RTS Algorithm: Let $\mathbf{g}^{(m)}$ be the vector which has the least ML cost found till the $m$th iteration of the algorithm. Let $l_{rep}$ be the average length (in number of iterations) between two successive occurrences of the same solution vector (repetitions), at the end of an iteration. Tabu period, $P$, a dynamic non-negative integer parameter, is defined. If a move is marked as tabu in an iteration, it will remain as tabu for $P$ subsequent iterations. The algorithm starts with an initial solution vector ${\bf x}^{(0)}$, which, for e.g., could be the MMSE or MF output vector. Set $\mathbf{g}^{(0)}={\bf x}^{(0)}$, $l_{rep}=0$, and $P=P_{0}$. All the entries of the tabu_matrix are set to zero. The following steps 1) to 3) are performed in each iteration. Consider $m$th iteration in the algorithm, $m\geq 0$.

Step 1): Define $\thinspace\mathbf{y}_{mf}\stackrel{{\scriptstyle\triangle}}{{=}}\mathbf{H}^{T}\mathbf{y}$, $\thinspace\mathbf{R}\stackrel{{\scriptstyle\triangle}}{{=}}\mathbf{H}^{T}\mathbf{H}$, and $\thinspace\mathbf{f}^{(m)}\stackrel{{\scriptstyle\triangle}}{{=}}\mathbf{R}\mathbf{x}^{(m)}-\mathbf{y}_{mf}$. Let $\mathbf{e}^{(m)}(u,v)=\mathbf{z}^{(m)}(u,v)-\mathbf{x}^{(m)}$. The ML costs of the $2kN$ neighbors of $\mathbf{x}^{(m)}$, namely, $\mathbf{z}^{(m)}(u,v)$, $u=0,\cdots,2k-1$, $v=0,\cdots,N-1$, are computed as

	$\displaystyle\phi(\mathbf{z}^{(m)}(u,v))$	$\displaystyle\hskip-5.69054pt=$	$\displaystyle\hskip-5.69054pt\big{(}\mathbf{x}^{(m)}+\mathbf{e}^{(m)}(u,v)\big{)}^{T}\mathbf{R}\thinspace\big{(}\mathbf{x}^{(m)}+\mathbf{e}^{(m)}(u,v)\big{)}$		(14)
			$\displaystyle\hskip-5.69054pt-2\big{(}\mathbf{x}^{(m)}+\mathbf{e}^{(m)}(u,v)\big{)}^{T}\mathbf{y}_{mf}$
		$\displaystyle\hskip-99.58464pt=$	$\displaystyle\hskip-51.21495pt\phi(\mathbf{x}^{(m)})+2\big{(}\mathbf{e}^{(m)}(u,v)\big{)}^{T}\mathbf{R}\thinspace\mathbf{x}^{(m)}$
			$\displaystyle\hskip-51.21495pt+\thinspace\big{(}\mathbf{e}^{(m)}(u,v)\big{)}^{T}\mathbf{R}\thinspace\mathbf{e}^{(m)}(u,v)-2\big{(}\mathbf{e}^{(m)}(u,v)\big{)}^{T}\mathbf{y}_{mf}$
		$\displaystyle\hskip-99.58464pt=$	$\displaystyle\hskip-51.21495pt\phi(\mathbf{x}^{(m)})+\underbrace{2\thinspace e_{u}^{(m)}(u,v)\thinspace{f}_{u}^{(m)}+\thinspace\big{(}e_{u}^{(m)}(u,v)\big{)}^{2}\thinspace\mathbf{R}_{u,u}}_{\stackrel{{\scriptstyle\triangle}}{{=}}\thinspace C\big{(}e_{u}^{(m)}(u,v)\big{)}}\,,$

where $e_{u}^{(m)}(u,v)$ is the $u$th element of $\mathbf{e}^{(m)}(u,v)$, $f_{u}^{(m)}$ is $u$th element of $\mathbf{f}^{(m)}$, and $\mathbf{R}_{u,u}$ is the $(u,u)$th element of $\mathbf{R}$. $\phi(\mathbf{x}^{(m)})$ on the RHS in (14) can be dropped since it will not affect the cost minimization. Let

$\displaystyle\hskip-19.91692pt(u_{1},v_{1})$

$\displaystyle\hskip-5.69054pt=$

$\displaystyle\hskip-5.69054pt{\mbox{arg min}\atop{u,v}}\thinspace\thinspace\thinspace C\big{(}e_{u}^{(m)}(u,v)\big{)}.$

(15)

The move $(u_{1},v_{1})$ is accepted if any one of the following two conditions is satisfied:

$i)$ $\phi(\mathbf{z}^{(m)}(u,v))<\phi(\mathbf{g}^{(m)})$

$ii)$ $\textit{tabu\_matrix}((u_{1}-1)M+q,v_{1})=0$ where $q:x_{u}^{(m)}=a_{q}\in\mathbb{A}$.

If move $(u_{1},v_{1})$ is accepted, then make

$\displaystyle\mathbf{x}^{(m+1)}$

$\displaystyle=$

$\displaystyle\mathbf{x}^{(m)}+\mathbf{e}(u_{1},v_{1}).$

(16)

If move $(u_{1},v_{1})$ is not accepted (i.e., neither of conditions $i)$ and $ii)$ is satisfied), find $(u_{2},v_{2})$ such that

$\displaystyle\hskip-19.91692pt(u_{2},v_{2})$

$\displaystyle\hskip-5.69054pt=$

$\displaystyle\hskip-5.69054pt{\mbox{arg min}\atop{u,v\thinspace\mbox{:}\thinspace u\neq u_{1},v\neq v_{1}}}\thinspace\thinspace\thinspace C\big{(}e_{u}^{(m)}(u,v)\big{)},$

(17)

and check for acceptance of the $(u_{2},v_{2})$ move. If this also cannot be accepted, repeat the procedure for $(u_{3},v_{3})$, and so on. If all the $2kN$ moves are tabu, then all the tabu_matrix entries are decremented by the minimum value in the tabu_matrix; this goes on till one of the moves becomes permissible. Let $(u^{\prime},v^{\prime})$ be the index of the neighbor with the minimum cost for which the move is permitted. Let $x_{u^{\prime}}^{(m)}=a_{q^{\prime}}=w_{v^{\prime\prime}}(x_{u^{\prime}}^{(m+1)})$, and $x_{u^{\prime}}^{(m+1)}=a_{q^{\prime\prime}}$, where $a_{q^{\prime}},a_{q^{\prime\prime}}\in\mathbb{A}$.

Step 2: After a move is done, the new solution vector is checked for repetition. For the channel model in (10), repetition can be checked by comparing the ML costs of the solutions in the previous iterations. If there is a repetition, the length of the repetition from the previous occurrence is found, and the average length, $l_{rep}$, is updated. The tabu period $P$ is modified as $P=P+1$. If the number of iterations elapsed since the last change of the value of $P$ exceeds $\beta l_{rep}$, for a fixed $\beta>0$, make $P=P-1$. The minimum value of $P$, however, will be 1. Note that this step, if executed, also qualifies as the one which changed $P$. After a move $(u^{\prime},v^{\prime})$ is accepted, make

	$\displaystyle\textit{tabu\_matrix}\thinspace((u^{\prime}-1)M+q^{\prime},v^{\prime})\,\,=\,\,P+1,$
	$\displaystyle\textit{tabu\_matrix}\thinspace((u^{\prime}-1)M+q^{\prime\prime},v^{\prime\prime})\,\,=\,\,P+1,$		(18)

and $\mathbf{g}^{(m+1)}=\mathbf{g}^{(m)}$. However, if $\phi(\mathbf{x}^{(m+1)})<\phi(\mathbf{g}^{(m)})$, then make

	$\displaystyle\textit{tabu\_matrix}\thinspace((u^{\prime}-1)M+q^{\prime},v^{\prime})\,\,=\,\,0,$
	$\displaystyle\textit{tabu\_matrix}\thinspace((u^{\prime}-1)M+q^{\prime\prime},v^{\prime\prime})\,\,=\,\,0,$		(19)

and $\mathbf{g}^{(m+1)}=\mathbf{x}^{(m+1)}$.

Step 3): Update the entries of the tabu_matrix as

$\displaystyle\hskip-17.07164pt\textit{tabu\_matrix}\thinspace(r,s)$

$\displaystyle\hskip-2.84526pt=$

$\displaystyle\hskip-2.84526pt\max\{\textit{tabu\_matrix}\thinspace(r,s)-1,0\},$

(20)

for $r=0,\cdots,2kM-1$, $s=0,\cdots,N-1.$ $\mathbf{f}^{(m)}$ is updated as

$\displaystyle\mathbf{f}^{(m+1)}\,\,\,=\,\,\,\mathbf{f}^{(m)}+e_{u^{\prime}}^{(m)}(u^{\prime},v^{\prime})\mathbf{R}_{u^{\prime}},$

(21)

where $\mathbf{R}_{u^{\prime}}$ is the ${u^{\prime}}$th column of $\mathbf{R}$.

Stopping criterion: The algorithm can be stopped based on a fixed number of iterations. Though convergence can be slow at low SNRs (typ. hundreds of iterations), it can be fast (typ. tens of iterations) at moderate to high SNRs. So rather than fixing a large number of iterations to stop the algorithm irrespective of the SNR, we use an efficient stopping criterion which makes use of the knowledge of the best ML cost in a given iteration, as follows.

Since the ML criterion is to minimize ${\|\mathbf{Hx}-\mathbf{y}\|}^{2}$, the minimum value of the objective function $\mathbf{x}^{T}\mathbf{H}^{T}\mathbf{Hx}-2\mathbf{x}^{T}\mathbf{H}^{T}\mathbf{y}$, $\mathbf{x}\in\mathbb{R}^{2k}$, is equal to $-\mathbf{y}^{T}\mathbf{y}$. We stop the algorithm when the least ML cost achieved in an iteration is within certain range of the global minimum, which is $-\mathbf{y}^{T}\mathbf{y}$. We stop the algorithm in the $m$th iteration, if the condition

$\displaystyle\frac{|\phi(\mathbf{g}^{(m)})-(-\mathbf{y}^{T}\mathbf{y})|}{|-\mathbf{y}^{T}\mathbf{y}|}\,\,\,<\,\,\,\alpha_{1},$

(22)

is met with at least min_iter iterations being completed to make sure the search algorithm has ‘settled.’ The bound is gradually relaxed as the number of iterations increase and the algorithm is terminated when

$\displaystyle\frac{|\phi(\mathbf{g}^{(m)})-(-\mathbf{y}^{T}\mathbf{y})|}{|-\mathbf{y}^{T}\mathbf{y}|}\,\,\,<\,\,\,m\alpha_{2}.$

(23)

In addition, we terminate the algorithm whenever the number of repetitions of solutions exceeds max_rep. Also, the maximum number of iterations is set to max_iter. We have found that use of the following stopping criterion parameters results in low complexity without compromising much on the performance (compared to a fixed number of iterations of 300) for 4-QAM: $\textit{min\_iter}=20$, $\textit{max\_iter}=300$, $\textit{max\_rep}=75$, $\alpha_{1}=0.05$, and $\alpha_{2}=0.0005$.

In this section, we present the uncoded/coded BER performance of the RTS algorithm in decoding non-orthogonal STBCs with $\delta=t=1$ (i.e., ILL) and $\delta=e^{\sqrt{5}{\bf j}}$, $t=e^{\bf j}$ (i.e., FD-ILL²²2Our simulation results show that the BER performance of FD-ILL and ILL STBCs with RTS decoding are almost the same.). The following RTS parameters are used in all the simulations: MMSE initial vector, $P_{0}=2,\beta=1,0.1,\alpha_{1}=5\%,\alpha_{2}=0.05\%,\textit{max\_rep=75},\textit{max\_iter}=300,\textit{min\_iter}=20$.