Error Exponents of Optimum Decoding for the Interference Channel

The $M$-user interference channel (IFC) models the communication between $M$ transmitter-receiver pairs, wherein each receiver must decode its corresponding transmitter’s message from a signal that is corrupted by interference from the other transmitters, in addition to channel noise. The information theoretic analysis of the IFC was initiated over 30 year ago and has recently witnessed a resurgence of interest, motivated by new potential applications, such as wireless communication over unregulated spectrum.

Previous work on the IFC has focused on obtaining inner and outer bounds to the capacity region for memoryless interference and noise, with a precise characterization of the capacity region remaining elusive for most channels, even for $M=2$ users. The best known inner bound for the IFC is the Han-Kobayashi (HK) region, established in [1]. It has been found to be tight in certain special cases ([1, 2]), and recently was found to be tight to within 1 bit for the two user Gaussian IFC [3]. No achievable rates that lie outside the HK region are known for any IFC with $M=2$ users.

Our aim in this paper is to extend the study of achievable schemes to the analysis of error exponents, or exponential rates of decay of error probabilities, that are attainable as a function of user rates. To our knowledge, there has been no prior treatment of error exponents for the IFC. In particular, the error bounds underlying the achievability results in [1] yield vanishing error exponents (though still decaying error probability) at all rates.

The notion of an error exponent region, or a set of achievable exponential rates of decay in the error probabilities for different users at a given operating rate-tuple in a multi-user communication network, was formalized recently in [4], and studied therein for Gaussian multiple access and broadcast channels. Our main result, presented in Section IV, is a single letter characterization of an achievable error exponent region, as a function of user rates, for the $M=2$ user finite alphabet, memoryless interference channel. The region is derived by bounding the average error probability of random codebooks comprised of i.i.d. codewords uniformly distributed over a type class, under maximum likelihood (ML) decoding at each user. Unlike the single user setting, in this case, the effective channel determining each receiver’s ML decoding rule is induced both by the noise and the interfering user’s codebook. Our focus on optimal decoding is a departure from the conventional achievability arguments in [1] and elsewhere, which are based on joint-typicality decoding, with restrictions on the decoder to “treat interference as noise” or to “decode the interference” in part or in whole. However, in this work, we confine our analysis to codebook ensembles that are simpler than the superposition codebooks of [1].

The analysis of the probability of decoding error under optimal decoding is complicated due to correlations induced by the interfering signal. Usual methods for bounding the probability of error based on Jensen’s inequality and other related inequalities (see, e.g., (8) below) fail to give good results. Our bounding approach combines some of the classical information theoretic approaches of [5] and [6] with an analytical technique from statistical physics that was applied recently to the study of single user channels in [7, 8]. More specifically, as in [5], we use auxiliary parameters $\rho$ and $\lambda$ to get an upper bound on the average probability of decoding error under ML decoding, which we then bound using the method of types [6]. Key in our derivation is the use of distance enumerators in the spirit of [7] and [8], which allows us to avoid using Jensen’s inequality in some steps, and allows us to maintain exponential tightness in other inequalities by applying them to only polynomially few terms (as opposed to exponentially many) in certain sums that bound the probability of decoding error. It should be emphasized, in this context, that the use of this technique was pivotal to our results. Our earlier attempts, that were based on more ‘traditional’ tools, failed to provide meaningful results. In fact, they all turned out to be inferior to some trivial bounds.

The paper is organized as follows. The notation, various definitions, and the channel model assumed throughout the paper are detailed in Section II. In Section III, we derive an “easy” set of attainable error exponents which we shall treat as a benchmark for the exponents of the main section, Section IV. The “easy” exponents are obtained by simple extensions to the interference channel of existing error exponent results for single user and multiple access channels, based on random constant composition codebooks and suboptimal decoders. Then, in Section IV, we derive another set of attainable exponents by analyzing ML decoding for the channel induced by the interfering codebook. In Section V, we show that the minimizations required to evaluate the new error exponents can be written as convex optimization problems, and, as a result, can be solved efficiently. We follow this up in Section VI with a numerical comparison of the new exponents with the baseline exponents of Section III for a simple IFC. These numerical results demonstrate that the new exponents are never worse (at least for the chosen channel and parameters) and, for most rates, strictly improve over the baseline exponents.

An earlier version of this work was presented in [9].

Unless otherwise stated, we use lowercase and uppercase letters for scalars, boldface lowercase letters for vectors, uppercase (boldface) letters for random variables (vectors), and calligraphic letters for sets. For example, $a$ is a scalar, $v$ is a vector, $X$ is a random variable, $X$ is a random vector, and ${\cal S}$ is a set. For a real number $a$ we shall, on occasion, let $\overline{a}$ denote $1-a$. Also, we use $\log(\cdot)$ to denote natural logarithm, $E$ to denote expectation, and Pr to denote probability. For independent random variables $X$ and $Y$ distributed according to $P_{X,Y}(x,y)=P_{X}(x)P_{Y}(y)$, $(x,y)\in{\cal X}\times{\cal Y}$, we denote the conditional expectation operator $\mbox{\boldmath$E$}_{X}(\cdot)$ as $\mbox{\boldmath$E$}_{X}(f(X,Y))\stackrel{{\scriptstyle\triangle}}{{=}}\sum_{x\in{\cal X}}f(x,Y)P_{X}(x)$ for any function $f(\cdot,\cdot)$. All information quantities (entropy, mutual information, etc.) and rates are in nats. Finally, we use $\doteq$, $\stackrel{{\scriptstyle.}}{{\leq}}$, etc., to denote equality or inequality to the first order in the exponent, i.e. $a_{n}\doteq b_{n}\Leftrightarrow\lim_{n\to\infty}\frac{1}{n}\log\frac{a_{n}}{b_{n}}=0$; $a_{n}\stackrel{{\scriptstyle.}}{{\leq}}b_{n}\Leftrightarrow\lim\sup_{n\to\infty}\frac{1}{n}\log\frac{a_{n}}{b_{n}}\leq 0$.

The empirical probability mass function of the finite alphabet sequence $\mbox{\boldmath$v$}=(v(1),\ldots,v(n))$ with alphabet ${\cal V}$ is denoted as the vector $\{P_{\mbox{\boldmath$v$}}(v),~v\in{\cal V}\}$, where each $P_{\mbox{\boldmath$v$}}(v)$ is the relative frequency of $v(i)=v$ along $v$. The type class associated with an empirical probability mass function $P$, which will be denoted by ${\cal T}_{P}$, is the set of all $n$–vectors $\{\mbox{\boldmath$v$}\}$ whose empirical probability mass function is equal to $P$. Similar conventions will apply to pairs and triples of vectors of length $n$, which are defined over the corresponding product alphabets. Information measures pertaining to empirical distributions will be denoted using the standard notational conventions, except that we use “$\hat{\quad}$” as well as subscripts that indicate the sequences from which these empirical distributions were extracted. For example, we write $\hat{H}_{\mbox{\boldmath$x$}\mbox{\boldmath$y$}\mbox{\boldmath$z$}}(X,Y|Z)$ and $\hat{I}_{\mbox{\boldmath$x$}\mbox{\boldmath$y$}\mbox{\boldmath$z$}}(X,Y;Z)$ to denote the conditional entropy of $(X,Y)$ given $Z$ and the mutual information between $(X,Y)$ and $Z$, respectively, computed with respect to the empirical distribution $P_{\mbox{\boldmath$x$}\mbox{\boldmath$y$}\mbox{\boldmath$z$}}(x,y,z)$. We denote the relative entropy or Kullback-Leibler divergence between distributions $P_{X}$ and $P_{Y}$ as $D(P_{X}||P_{Y})\stackrel{{\scriptstyle\triangle}}{{=}}\sum_{x}P_{X}(x)\log(P_{X}(x)/P_{Y}(x))$, and we write $D(P_{X|Z}||P_{Y|Z}|P_{Z})$ for the conditional relative entropy between conditional distributions $P_{X|Z}$ and $P_{Y|Z}$ conditioned on $P_{Z}$, which is defined as $D(P_{X|Z}||P_{Y|Z}|P_{Z})\stackrel{{\scriptstyle\triangle}}{{=}}\sum_{x,z}P_{Z}(z)P_{X|Z}(x|z)\log(P_{X|Z}(x|z)/P_{Y|Z}(x|z))$ .

We continue with a formal description of the two–user IFC setting. Let $\mbox{\boldmath$x$}_{i}=(x_{i}(1),\ldots,x_{i}(n))\in{\cal X}^{n}_{i}$, $i=1,2$, denote the channel input signals of the two transmitters, and let $\mbox{\boldmath$y$}_{i}=(y_{i}(1),\ldots,y_{i}(n))\in{\cal Y}^{n}_{i}$ be the corresponding channel outputs received by decoders 1 and 2, where ${\cal X}_{i}$ and ${\cal Y}_{i}$ denote the input and output alphabets, and which we assume to be finite. Each (random) output symbol pair $(Y_{1}(j),Y_{2}(j))$ is assumed to be conditionally independent of all other outputs, and all input symbols, given the two corresponding (random) input symbols $(X_{1}(j),X_{2}(j))$, and the corresponding conditional probability is assumed to be constant from symbol to symbol. An $(n,R_{1},R_{2})$ code for the IFC consists of pairs of encoding and decoding functions, $(f_{1},f_{2})$ and $(g_{1},g_{2})$, respectively, where $f_{i}:\{1,\ldots,M_{i}\}\rightarrow{\cal X}^{n}_{i}$, $M_{i}=\lceil e^{nR_{i}}\rceil$, and $g_{i}:{\cal Y}^{n}_{i}\rightarrow\{1,\ldots,M_{i}\}$, $i=1,2$. The performance of the code is characterized by a pair of error probabilities $P_{e,i}=\mbox{Pr}(\hat{W}_{i}\neq W_{i})$, $i=1,2$, where $\hat{W}_{i}=g_{i}(\mbox{\boldmath$Y$}_{i})$ and $\mbox{\boldmath$Y$}_{i}$ is the random output when user $i$ transmits $\mbox{\boldmath$X$}_{i}=f_{i}(W_{i})$, assuming the messages $W_{i}$ are uniformly distributed on the sets of indices $\{1,2,\ldots,M_{i}\}$, $i=1,2$. The per user error probabilities depend on the channel only through the marginal conditional distributions of the channel outputs given the corresponding channel input pairs. We shall denote these conditional distributions as $q_{i}(y|x_{1},x_{2})\stackrel{{\scriptstyle\triangle}}{{=}}\mbox{Pr}(Y_{i}(j)=y|(X_{1}(j),X_{2}(j))=(x_{1},x_{2}))$.

A pair of error exponents $(E_{1},E_{2})$ is attainable at a rate pair $(R_{1},R_{2})$ if there is a sequence of $(n,R_{1},R_{2})$ codes satisfying $E_{i}\leq\liminf_{n\to\infty}-(1/n)\log P_{e,i}$ for $i=1,2$. The set of all attainable error exponents at $(R_{1},R_{2})$ comprises the error exponent region at $(R_{1},R_{2})$ and we shall denote it as ${\cal E}(R_{1},R_{2})$. The main result of this paper is a single letter characterization of a non–trivial subset of ${\cal E}(R_{1},R_{2})$ for each $R_{1},R_{2}$.

In this section, we present achievable error exponents for the interference channel which are based on known results of error exponents for single user and multiple access channels (MAC) for fixed composition codebooks [12, 13, 11]. These exponents will be used as a baseline for comparing the performance of the error exponents that we derive in Section IV.

In the following, we will focus on the error performance of user 1, and as a result, all explanations and expressions will be specialized to receiver 1. Similar expressions also hold for user 2 with the exchange of indices $1\leftrightarrow 2$.

A possibly suboptimal decoder for the interference channel can be obtained from a given multiple access channel decoder by simply ignoring the decoded message of the interfering transmitter. For example, following [13], we can use a minimum entropy decoder that for a given received vector $\mbox{\boldmath$y$}_{1}$ at receiver $1$ computes $(\hat{\mbox{\boldmath$x$}}_{1},\hat{\mbox{\boldmath$x$}}_{2})$

and throws away $\hat{\mbox{\boldmath$x$}}_{2}$.

It follows from [13] that for random codebooks of fixed composition $Q_{1},Q_{2}$, the average probability of decoding both messages in error, where the averaging is done over the random choice of codebooks, satisfies:

where

with $|\cdot|^{+}=\max\{\cdot,0\}$.

In addition, the average probability of decoding the message of the interfering transmitter correctly but the message of the desired transmitter incorrectly satisfies:

where

Therefore, the overall average error performance of this MAC decoder in the IFC satisfies:

A second suboptimal decoder that leads to tractable error performance bounds is the single user maximum mutual information decoder (which in this case coincides with the minimum entropy decoder):

In this case, standard application of the method of types [11] leads to the following bound on the average error probability under random fixed composition codebooks of types $Q_{1},Q_{2}$:

where

We can choose the better decoder between these two, that leads to the better error performance. Therefore, we obtain that

is an achievable error exponent at receiver 1, with an analogous exponent following for receiver 2.

Our main contribution is stated in the following theorem, which presents a new error exponent region for the discrete memoryless two–user IFC. While the full proof appears in Appendix A, we also provide a proof outline below, to give an idea of the main steps.

and

where ${\cal S}$ is the probability simplex in $\mathcal{X}_{1}\times\mathcal{X}_{2}\times\mathcal{Y}_{1}$. In the bound (2), $(\rho,\lambda)\in[0,1]^{2}$ can be chosen to maximize the error exponent $E_{R,1}$.

In eqs. (2), (3), (6), and (7), $Q_{1}$ and $Q_{2}$ are probability distributions defined over the alphabets $\mathcal{X}_{1}$ and $\mathcal{X}_{2}$ respectively.

Expressions for the error probability $P_{e,2}$ and error exponent $E_{R,2}$ equivalent to (2) and (3) can be stated for the receiver of user 2 by replacing $X_{1}\leftrightarrow X_{2}$, $Y_{1}\rightarrow Y_{2}$, and $q_{1}\rightarrow q_{2}$ in all the expressions. By varying $Q_{1}$ and $Q_{2}$ over all probability distributions in $\mathcal{X}_{1}$ and $\mathcal{X}_{2}$ respectively, we obtain the error exponent region for fixed rates $R_{1}$ and $R_{2}$.

For a given channel output $\mbox{\boldmath$y$}_{1}\in\mathcal{Y}_{1}^{n}$, the best decoding rule to minimize the probability of error in decoding the message of user 1 is ML decoding, which consists of picking the message $m$ which maximizes $P(\mbox{\boldmath$y$}_{1}|\mbox{\boldmath$x$}_{1,m})=\sum_{i=1}^{M_{2}}q_{1}^{(n)}(\mbox{\boldmath$y$}_{1}|\mbox{\boldmath$x$}_{1,m},\mbox{\boldmath$x$}_{2,i})/M_{2}$. Letting

be the “average” channel observed at receiver 1, where the averaging is done over the codewords of user 2 in ${\cal C}_{2}$, the decoding error probability at receiver 1 for transmitted codeword $\mbox{\boldmath$x$}_{1,m}$ and codebooks ${\cal C}_{1}$ and ${\cal C}_{2}$ is given by:

With the introduction of the average channel (9), and the use of two auxiliary parameters $(\rho,\lambda)\in[0,1]^{2}$, we can follow the approach of [5] to bound the conditional probability of decoding error $P_{e,1}(\mbox{\boldmath$x$}_{m},{\cal C}_{1},{\cal C}_{2}|\mbox{\boldmath$y$}_{1})$. Taking expectation over the random choice of codebooks ${\cal C}_{1}$ and ${\cal C}_{2}$ we obtain an average error probability:

where we used Jensen’s inequality to move the second expectation inside $(\cdot)^{\rho}$.

Equation (11) is hard to handle, mainly due to the correlation introduced by ${\cal C}_{2}$ between the two factors inside the outer expectation. Furthermore, the evaluation of the inner expectations over $\mbox{\boldmath$X$}_{1}$ are complicated due to the powers $\overline{\rho\lambda}$ and $\lambda$ affecting $q_{1,{\cal C}_{2}}^{(n)}(\mbox{\boldmath$y$}_{1}|\mbox{\boldmath$X$}_{1})$. Bounding methods based on Jensen’s inequality and (8) fail to give good results due to the loss of exponential tightness.

We proceed with a refined bounding technique based on the method of types inspired by [7]. While in this approach we still use (8), we use it to bound sums with a number of terms that only grows polynomially with $n$, and as a result, exponential tightness is preserved.

Since the channel is memoryless,

where we used $N_{\mbox{\boldmath$x$}_{1},\mbox{\boldmath$y$}_{1}}(P_{\hat{X}_{1}\hat{X}_{2}\hat{Y}_{1}})$ to denote the number of codewords $\mbox{\boldmath$x$}_{2}$ in ${\cal C}_{2}$ such that $(\mbox{\boldmath$x$}_{1},\mbox{\boldmath$x$}_{2},\mbox{\boldmath$y$}_{1})$ have empirical distribution $P_{\hat{X}_{1}\hat{X}_{2}\hat{Y}_{1}}$. We also used $\mbox{\boldmath$E$}_{\hat{X}_{1}\hat{X}_{2}\hat{Y}_{1}}(\cdot)$ to denote expectation with respect to the distribution $P_{\hat{X}_{1}\hat{X}_{2}\hat{Y}_{1}}$.

Replacing (12) in (11) and using (8) three times, we obtain:

where we used $\hat{P}=P_{\hat{X}_{1}\hat{X}_{2}\hat{Y}_{1}}$ and $\hat{P}^{\prime}=P_{\hat{X}_{1}^{\prime}\hat{X}_{2}^{\prime}\hat{Y}_{1}^{\prime}}$ to shorten the expression.

We next consider the bounding of

and note that $N_{\mbox{\boldmath$X$}_{1},\mbox{\boldmath$y$}_{1}}(\hat{P})$ and $N_{\mbox{\boldmath$X$}_{1},\mbox{\boldmath$y$}_{1}}(\hat{P}^{\prime})$ are formed by sums of an exponentially large number of indicator functions, each of which takes value 1 with exponentially small probability. These sums concentrate around their means, which show different behavior depending on how the number of terms in the sum ($e^{nR_{2}}$) compares to the probability of each of the indicator functions taking value 1 (depending on the case considered, these probabilities take the form $e^{-nI(\hat{X}_{2};\hat{X}_{1},\hat{Y}_{1})}$, $e^{-nI(\hat{X}_{2}^{\prime};\hat{X}_{1}^{\prime},\hat{Y}_{1}^{\prime})}$, or $e^{-nI(\hat{X}_{2}^{\prime};\hat{Y}_{1}^{\prime})}$). Whenever one of the factors in (14) concentrates around its mean it behaves as a constant, and hence is uncorrelated with the remaining factor. As a result, the correlation between the two factors of (14), which complicates the analysis, can be circumvented. We give the details of this part of the derivation in Appendix A, but note here that the resulting bound on $E(\mbox{\boldmath$y$}_{1},\hat{P},\hat{P}^{\prime})$ depends on $\mbox{\boldmath$y$}_{1}$ only through a factor $1(\mbox{\boldmath$y$}_{1}\in P_{\hat{Y}_{1}},P_{\hat{Y}_{1}^{\prime}};P_{\hat{X}_{1}}=P_{\hat{X}_{1}^{\prime}}=Q_{1};P_{\hat{X}_{2}}=P_{\hat{X}_{2}^{\prime}}=Q_{2})$. Therefore, the innermost sum in (13) can be evaluated by counting the number of vectors $\mbox{\boldmath$y$}_{1}\in\mathcal{Y}_{1}^{n}$ that have empirical types $P_{\hat{Y}_{1}}$ and $P_{\hat{Y}_{1}^{\prime}}$. Note that this count can only be positive for $P_{\hat{Y}_{1}}=P_{\hat{Y}_{1}^{\prime}}$. This count is approximately equal to $e^{nH(\hat{Y}_{1})}$ to first order in the exponent. Furthermore, the sums over $\hat{P}$ and $\hat{P}^{\prime}$ in (13) have a number of terms that only grows polynomially with $n$. Therefore, to first order, the exponential growth rate of (13) equals the maximum exponential growth rate of the argument of the outer two sums, where the maximization is performed over the distributions $\hat{P}$ and $\hat{P}^{\prime}$ which are rational, with denominator $n$. We can further upper bound the probability of error by enlarging the optimization region, maximizing over any probability distributions $\hat{P},\hat{P}^{\prime}$.

In order to get a valid evaluation of $E_{R,1}(R_{1},R_{2},Q_{1},$ $Q_{2},\rho,\lambda)$, for any given $Q_{1},Q_{2},\rho,\lambda$ satisfying the constraints of the outer maximization, we need to accurately solve the inner minimization problems. A brute force search may not give accurate enough results in reasonable time. As will be shown below, the first minimization problem in (3) is a convex problem, and as a result, it that can be solved efficiently. In addition, convexity allows to lower bound the objective function by its supporting hyperplane, which in turn, allows to get a reliable¹¹1In our implementation we solve the original convex optimization problem using the MATLAB function fmincon. lower bound through the solution of a linear program.

The second minimization problem is not convex due to the non–convex constraint $R_{2}\leq I(\hat{X}_{2};\hat{Y}_{1})$. If we remove this constraint, it will be later shown that we obtain a convex problem that can be solved efficiently. There are two possible situations:

The first situation occurs when the optimal solution to the modified problem satisfies $R_{2}\leq I(\hat{X}_{2};\hat{Y}_{1})$: in this case, the solution to the modified problem is also a solution to the original problem.

The second situation is when the optimal solution to the modified problem satisfies $R_{2}>I(\hat{X}_{2};\hat{Y}_{1})$: in this case, a solution to the original problem must satisfy $R_{2}=I(\hat{X}_{2};\hat{Y}_{1})$. We prove this statement by contradiction. Let $P^{*}_{1}$ be the optimal solution to the modified problem, and $P^{*}_{2}$ be an optimal solution to the original problem. Now assume conversely, that there is no $P^{*}_{2}$ that satisfies $R_{2}=I(\hat{X}_{2};\hat{Y}_{1})$. With this assumption, we have that at $P^{*}_{2}$, $R_{2}<I(\hat{X}_{2};\hat{Y}_{1})$. Let $\mathcal{D}\triangleq\{P=(P_{\hat{X}_{1}\hat{X}_{2}\hat{Y}_{1}},P_{\hat{X}_{1}^{\prime}\hat{X}_{2}^{\prime}\hat{Y}_{1}^{\prime}}):P_{\hat{X}_{1}}=P_{\hat{X}_{1}^{\prime}}=Q_{1},P_{\hat{X}_{2}}=P_{\hat{X}_{2}^{\prime}}=Q_{2}\}$. Note that $\mathcal{D}$ is a convex set and $P_{1}^{*},P_{2}^{*}\in\mathcal{D}$. Due to the continuity of $I(\hat{X}_{2};\hat{Y}_{1})$, the straight line in $\mathcal{D}$ that joins $P^{*}_{1}$ and $P^{*}_{2}$ must pass through an intermediate point $\overline{P}=\alpha P^{*}_{1}+(1-\alpha)P^{*}_{2}$, $\alpha\in(0,1)$, that satisfies $I(\hat{X}_{2};\hat{Y}_{1})=R_{2}$. Let $f_{2}(\cdot)$ be the objective function of the second minimization problem in (3), restricted to $\mathcal{D}$. It will be shown later that $f_{2}(\cdot)$, restricted to this domain, is a convex function. By hypothesis, $f_{2}(\overline{P})>f_{2}(P_{2}^{*})$ and we have $f_{2}(P_{1}^{*})\leq f_{2}(P_{2}^{*})<f_{2}(\overline{P})$. On the other hand, from the convexity of $f_{2}(\cdot)$, restricted to $\mathcal{D}$, we have $f_{2}(\overline{P})\leq\alpha f_{2}(P_{1}^{*})+(1-\alpha)f_{2}(P_{2}^{*})\leq f_{2}(P_{2}^{*})$ and we get a contradiction. Therefore, it follows that there is a solution $P^{*}_{2}$ to the original problem that satisfies $R_{2}=I(\hat{X}_{2};\hat{Y}_{1})$.

Let $f_{1}(\cdot)$ be the objective function of the first minimization problem in (3). First, we note that $P_{2}^{*}$ satisfies the constraints of the first minimization problem since they are less restrictive than the constraints of the second minimization problem in (3). We next prove that $f_{1}(P_{2}^{*})=f_{2}(P_{2}^{*})$. As a result, the optimal solution $P^{*}$ of the first minimization problem satisfies $f_{1}(P^{*})\leq f_{1}(P_{2}^{*})=f_{2}(P^{*}_{2})$, and we do not need to know $f_{2}(P_{2}^{*})$ to evaluate the argument of the maximization in (3). Using the fact that at $P_{2}^{*}$, $I(\hat{X}_{2};\hat{Y}_{1})=I(\hat{X}_{2}^{\prime};\hat{Y}_{1}^{\prime})=R_{2}$, we have:

where we used the identity $I(\hat{X}_{1}^{\prime};\hat{X}_{2}^{\prime},\hat{Y}_{1}^{\prime})-I(\hat{X}_{1}^{\prime};\hat{Y}_{1}^{\prime})=I(\hat{X}_{2}^{\prime};\hat{X}_{1}^{\prime},\hat{Y}_{1}^{\prime})-I(\hat{X}_{2}^{\prime};\hat{Y}_{1}^{\prime})$ in the second equality.

In summary, if the solution to the second minimization problem in (3), without the constraint on $R_{2}$, satisfies $R_{2}>I(\hat{X}_{2};\hat{Y}_{1})$, then the first minimization problem in (3) dominates the expression. Otherwise, the solution to the second minimization problem in (3) without the constraint $R_{2}\leq I(\hat{X}_{2};\hat{Y}_{1})$, equals the solution to the second minimization problem with this constraint.

It remains to show that the objective functions of the minimization problems in (3), $f_{1}(P_{\hat{X}_{1}\hat{X}_{2}\hat{Y}_{1}},P_{\hat{X}_{1}^{\prime}\hat{X}_{2}^{\prime}\hat{Y}_{1}^{\prime}})$, $f_{2}(P_{\hat{X}_{1}\hat{X}_{2}\hat{Y}_{1}},P_{\hat{X}_{1}^{\prime}\hat{X}_{2}^{\prime}\hat{Y}_{1}^{\prime}})$, restricted to the domain $\mathcal{D}$, are convex functions. Since the sum of convex functions is convex, to prove the convexity of $f_{1}(\cdot)$ on $\mathcal{D}$, we only need to prove that the different terms of