A short tutorial on Wirtinger Calculus with applications in quantum information

The objects of interest in this discussion are real-valued functions with (generally complex) matrix inputs. We begin with a simple example that is not entirely trivial. Consider the function

	$\displaystyle f:M_{2}(\mathbb{R})$	$\displaystyle\longrightarrow\mathbb{R}$		(1.1)
	$\displaystyle\mathbf{X}$	$\displaystyle\mapsto\operatorname{Tr}(\mathbf{X}^{2}).$

How do we optimize this function? The obvious way to do so would be to observe that for the purposes of optimization, the matrix structure of $\mathbf{X}$ can be disregarded. What matters is that $\mathbf{X}$ essentially comprises a set of parameters, and $f$ is a multivariable function. If we write $\mathbf{X}=\begin{bmatrix}a&b\\ c&d\end{bmatrix}$, then after identifying $\mathbf{X}$ with the vector $(a,b,c,d)$ we can express²²2Strictly speaking, the functions in Equations 1.1 and 1.2 are not the same, simply because they have different domains ($M_{2}(\mathbb{R})$ versus $\mathbb{R}^{4}$). $f$ as

$\displaystyle f(a,b,c,d)=a^{2}+d^{2}+2bc.$

(1.2)

Conventional optimization methods can then be implemented accordingly. (Note that constraints may have to be further imposed on $\mathbf{X}$ to ensure the existence of solutions. For example in Equation 1.1 above we note that simply by rescaling $\mathbf{X}$ we can send $f(\mathbf{X})$ to $\pm\infty$, thus no optimal solution exists. To remedy this we could impose, say, $\operatorname{Tr}\mathbf{X}=1$.)

One quickly observes the impracticality of this approach except in the simplest of cases. Even if the form of $f$ is relatively simple, like the one above, a few issues persist. For instance, if $\mathbf{X}\in M_{n}(\mathbb{R})$ for general $n$, then as $n$ increases the corresponding analog to $f$ in Equation 1.2 becomes considerably messier. Likewise, had the field been $\mathbb{C}$ instead of $\mathbb{R}$, or if there were constraints among the elements of $\mathbf{X}$ (if $\mathbf{X}\in M_{n}(\mathbb{R})$ is symmetric, say, then the number of Lagrange multipliers required goes as $\Theta(n^{2})$). But perhaps the most salient deficiency lies beyond the mathematics: that $\mathbf{X}$ is a matrix (and not just a bunch of numbers) often holds significant physical meaning, and one would like to preserve this structure to gain further insight into the problem. This is perhaps best illustrated with an example. Suppose the optimal solution to some problem (similar in flavour to Problem 1.1 above) is given by $\mathbf{X^{\star}}=\begin{bmatrix}a^{\star}&b^{\star}\\ c^{\star}&d^{\star}\end{bmatrix}$ with

	$\displaystyle a^{\star}$	$\displaystyle=p^{3}-p^{2}+2pqr+qrs-qr$
	$\displaystyle b^{\star}$	$\displaystyle=p^{2}q+q^{2}r+pqs+qs^{2}-pq-qs$
	$\displaystyle c^{\star}$	$\displaystyle=p^{2}r+qr^{2}+prs+rs^{2}-pr-ps$
	$\displaystyle d^{\star}$	$\displaystyle=s^{3}-s^{2}+2qrs+pqr-qr$

where $\mathbf{Y}=\begin{bmatrix}p&q\\ r&s\end{bmatrix}$ is a matrix involved in the specification of the problem. One suspects it might be possible to express $\mathbf{X^{\star}}$ in terms of $\mathbf{Y}$, but it is not immediately obvious how³³3Here, $\mathbf{X^{\star}}=\mathbf{Y}^{3}-\mathbf{Y}^{2}$. The solution form is admittedly rather artificial. For a ‘realistic’ example see Example 4.3 below.. By using the ‘obvious’ method above, we have obscured potentially important information in the form of an explicit connection between $\mathbf{X^{\star}}$ and $\mathbf{Y}$.

Wouldn’t it be nice if we could somehow differentiate $f(\mathbf{X})$ with respect to $\mathbf{X}$ as a whole? That is, we obtain something like

$\displaystyle\frac{df(\mathbf{X})}{d\mathbf{X}}=f^{\prime}(\mathbf{X})$

for some matrix function $f^{\prime}$, which we can call the matrix derivative of $f$. Then by setting $f^{\prime}(\mathbf{X})=0$ we could hopefully extract the optimal $\mathbf{X}$, in a similar flavour to the procedure taught in elementary calculus. Showing that this could be done for a large class of interesting and relevant functions, and how to do it, is the focus of this article.

We define the following notations. Let $\mathbb{N}=\{1,2,\dots\}$ be the set of positive natural numbers. For $n\in\mathbb{N}$, $[n]=\{1,2,\dots,n\}$. If $z\in\mathbb{C}$, $z^{*}\in\mathbb{C}$ denotes its complex conjugate. $M_{n}(\mathbb{F})/\mathbb{F}^{n\times n}$ is the set of $n\times n$ dimensional matrices over the field $\mathbb{F}$, where $\mathbb{F}=\mathbb{R}/\mathbb{C}$. We denote a Hilbert space by $\mathcal{H}$ ($\mathcal{H}_{N}$ if its dimension is to be explicitly specified), the set of linear operators on $\mathcal{H}$ by $\mathcal{L}(\mathcal{H})$, and the set of density operators on $\mathcal{H}$ by $\mathcal{D}(\mathcal{H})$. $\|\cdot\|_{F}$ is the Frobenius norm. The symbol $\odot$ denotes component-wise product, e.g. for vectors $(v\odot w)_{i}=v_{i}w_{i}$, for matrices $(A\odot B)_{ij}=A_{ij}B_{ij}$. Throughout most of this text, we adopt the convention whereby scalar quantities are denoted by lowercase symbols ($x$), vector quantities by lowercase boldface $(\mathbf{x})$ and matrix quantities by capital boldface $(\mathbf{X})$.

We will not be pedantic with domain/codomain issues. While we generally write $f:\mathbb{C}\longrightarrow\mathbb{C}$ or $f:\mathbb{R}^{n}\longrightarrow\mathbb{R}^{m}$, points where $f$ is not well-defined are implicitly excluded from the domain. E.g. if $f(z)=1/z$ then the domain of $f$ is understood to be $\mathbb{C}\setminus\{0\}$. Similarly, a function is complex-differentiable at a point if its complex derivative exists at that point, and it is holomorphic on an open set if it is complex-differentiable at every point in the open set. This is strictly speaking a stronger condition than complex-differentiability, but we shall simply use the two terms interchangeably.

As will be shown below in Proposition 2.6, non-constant real-valued functions of complex numbers cannot be complex-differentiable, in the sense that the quantity

$\displaystyle\frac{df(z)}{dz}=\lim_{h\rightarrow 0}\frac{f(z+h)-f(z)}{h}$

is not well-defined. However, the optimization of $f$ entails taking the derivatives $\frac{\partial f}{\partial x}$ and $\frac{\partial f}{\partial y}$, which we assume a priori are well-defined. That $\frac{\partial f}{\partial x}$ and $\frac{\partial f}{\partial y}$ exist, but not $\frac{df}{dz}$, might cause some confusion since $z=(x,y)$. In this section, we clarify the difference between real- and complex- differentiability of a function $f:\mathbb{C}\longrightarrow\mathbb{C}$. The material presented here is standard fare in real and complex analysis, see for example [Rud53, Rem91, GK06].

In single-variable calculus, the derivative of $f:\mathbb{R}\longrightarrow\mathbb{R}$ at $x$ is

$\displaystyle f^{\prime}(x)=\lim_{h\rightarrow 0}\frac{f(x+h)-f(x)}{h},$

(1.3)

with the usual interpretation being the ‘slope of the tangent of $f$ at $x$’. For multivariable functions the generalization of Equation 1.3 is as follows. We say $f:\mathbb{R}^{n}\longrightarrow\mathbb{R}^{m}$ is real-differentiable at $x$ if there exists $T\in\mathcal{L}(\mathbb{R}^{n},\mathbb{R}^{m})$ such that

$\displaystyle\lim_{h\rightarrow 0}\frac{\|f(x+h)-f(x)-T(h)\|}{\|h\|}=0,$

(1.4)

where $\|\cdot\|$ denotes the Euclidean norm. We write $f^{\prime}(x)=T$ and call it the derivative/differential of $f$ at $x$. Thus, this generalized derivative is a linear operator, not just a number (when $n=m=1$, a linear operator is just a scaling, so it can be identified with the scale factor, a real number). The matrix representation of $f^{\prime}(x)$ with respect to the standard Euclidean bases is the Jacobian matrix $\left[\frac{\partial f_{i}}{\partial x_{j}}(x)\right]_{\begin{subarray}{c}1\leq i\leq m\\ 1\leq j\leq n\end{subarray}}$. While the geometric picture of a slope no longer holds, the interpretation of $f^{\prime}(x)$ as best linear approximation of $f$ at $x$ still applies. The differential also subsumes as special cases the directional derivative (when $n=1$) and the gradient vector (when $m=1$). It is a standard result that for an open set $O\subseteq\mathbb{R}^{n}$, $f$ is real-differentiable on $O$ if and only if all its partial derivatives $\frac{\partial f_{i}}{\partial x_{j}}$, $1\leq i\leq m,1\leq j\leq n$ exist and are continuous on $O$ (Theorem 9.21, [Rud53]). Thus for functions of our interest, we can assume they are real-differentiable since we almost always assume $\frac{\partial f_{i}}{\partial x_{j}}$ are all smooth.

The complex derivative is obtained by directly extending the concept of infinitesimal difference quotients in calculus (Eq. 1.3) to the complex domain. A function $f:\mathbb{C}\longrightarrow\mathbb{C}$ is complex-differentiable at $z$ if the derivative $f^{\prime}(z)=\lim_{h\rightarrow 0}\frac{f(z+h)-f(z)}{h}$ exists, or equivalently,

$\displaystyle\lim_{h\rightarrow 0}\frac{f(z+h)-f(z)-f^{\prime}(z)h}{h}=0.$

(1.5)

Since $f:\mathbb{C}\longrightarrow\mathbb{C}$ is also $f:\mathbb{R}^{2}\longrightarrow\mathbb{R}^{2}$, the notion of real-differentiability also applies to $f$. How do Equations 1.4 and 1.5 relate to each other? First, recall that the Cauchy-Riemann equations

$\displaystyle\frac{\partial u}{\partial x}=\frac{\partial v}{\partial y},\qquad\frac{\partial u}{\partial y}=-\frac{\partial v}{\partial x}$

(1.6)

holding on $\mathcal{O}$ form a necessary and sufficient condition for $f$ to be complex-differentiable on $\mathcal{O}$. In this case, $f^{\prime}(z)=u_{x}+iv_{x}=v_{y}-iu_{y}$. We next recast 1.5 into a form more similar to 1.4. The multiplication of two complex numbers can be viewed as a matrix acting on a vector in $\mathbb{R}^{2}$: for $z,w\in\mathbb{C}$,

	$\displaystyle z\cdot w=(\operatorname{Re}z\operatorname{Re}w-\operatorname{Im}z\operatorname{Im}w)+i(\operatorname{Re}z\operatorname{Im}w+\operatorname{Im}z\operatorname{Re}w)$	$\displaystyle=\begin{bmatrix}\operatorname{Re}z\operatorname{Re}w-\operatorname{Im}z\operatorname{Im}w\\ \operatorname{Re}z\operatorname{Im}w+\operatorname{Im}z\operatorname{Re}w\end{bmatrix}$
		$\displaystyle=\begin{bmatrix}\operatorname{Re}z&-\operatorname{Im}z\\ \operatorname{Im}z&\operatorname{Re}z\end{bmatrix}\begin{bmatrix}\operatorname{Re}w\\ \operatorname{Im}w\end{bmatrix}.$

Thus we have that if $f^{\prime}(z)$ exists, then

		$\displaystyle\lim_{h\rightarrow 0}\frac{f(z+h)-f(z)-f^{\prime}(z)h}{h}=0$
	$\displaystyle\implies$	$\displaystyle\lim_{h\rightarrow 0}\frac{f(z+h)-f(z)-\begin{bmatrix}u_{x}&-v_{x}\\ v_{x}&u_{x}\end{bmatrix}h}{h}=0$
	$\displaystyle\implies$	$\displaystyle\lim_{h\rightarrow 0}\frac{f(z+h)-f(z)-\begin{bmatrix}u_{x}&u_{y}\\ v_{x}&v_{y}\end{bmatrix}h}{h}=0\qquad$	by Cauchy-Riemann
	$\displaystyle\implies$	$\displaystyle\lim_{h\rightarrow 0}\frac{f((x,y)+h)-f(x,y)-f^{\prime}(x,y)h}{h}=0\qquad$	since $f^{\prime}(x,y)=\begin{bmatrix}u_{x}&u_{y}\\ v_{x}&v_{y}\end{bmatrix}$
	$\displaystyle\iff$	$\displaystyle\lim_{h\rightarrow 0}\frac{\|f((x,y)+h)-f(x,y)-f^{\prime}(x,y)h\|}{\|h\|}=0,$

5 i.e. $f$ is real-differentiable, and the complex derivative $f^{\prime}(z)$ (a complex number) is identified with the differential/real derivative $f^{\prime}(x,y)$ (a linear operator). Furthermore, note that

$\displaystyle f^{\prime}(z)=f^{\prime}(x,y)=\begin{bmatrix}u_{x}&u_{y}\\ v_{x}&v_{y}\end{bmatrix}=\begin{bmatrix}u_{x}&-v_{x}\\ v_{x}&u_{x}\end{bmatrix}=rA,$

where $0\leq r\in\mathbb{R}$ and $A\in SO_{2}(\mathbb{R})$. This operator is a composition of scaling and an orientation-preserving rotation (the scaling factor and rotation angle generally differ from point to point). Such maps are widely called ‘conformal’ in the literature. In short, we have the following fact.

The converse does not hold: if a map $f$ is real-differentiable it need not be complex-differentiable, the classic example being complex conjugation: $f(z)=z^{\star}$ or equivalently $f(x,y)=(x,-y)$. The conformality of the differential makes complex-differentiability a much more rigid property compared to real-differentiability. This rigidity however underlies many remarkable properties of holomorphic functions and thus the power of complex analysis, broadly defined as the study of these functions.

We begin by discussing general complex-valued functions, $f:\mathbb{C}\longrightarrow\mathbb{C}$. Since $\mathbb{C}$ is just $\mathbb{R}^{2}$ endowed with the multiplication operation $(a,b)\times(c,d)\mapsto(ac-bd,ad+bc)$, we can view $f$ as

	$\displaystyle f:\;$	$\displaystyle\mathbb{R}^{2}\longrightarrow\mathbb{R}^{2}$
		$\displaystyle(x,y)\mapsto(u(x,y),v(x,y)).$

Here we assume $f$ is real-differentiable, but $f$ need not be complex-differentiable (refer to Section 1.3 above for the difference). Often, however, $f$ is expressed in terms of $z$ and $z^{*}$. More precisely, viewing $z,z^{*}$ as functions from $\mathbb{R}\times\mathbb{R}$ to $\mathbb{C}$, one has a function $\tilde{f}:\mathbb{C}\times\mathbb{C}\longrightarrow\mathbb{C}$ such that

$\displaystyle f(x,y)=:\tilde{f}\circ(z,z^{*})(x,y)=\tilde{f}(z(x,y),z^{*}(x,y))=\tilde{f}(x+iy,x-iy).$

(2.1)

Conversely, we also have

$\displaystyle\tilde{f}(z,z^{*})=f\circ(x,y)(z,z^{*})=f(x(z,z^{*}),y(z,z^{*}))=f\left(\frac{z+z^{*}}{2},\frac{z-z^{*}}{2}\right).$

(2.2)

Partial differentiating $f$ and applying the chain rule gives

	$\displaystyle\frac{\partial f}{\partial x}(x,y)$	$\displaystyle=\frac{\partial\tilde{f}}{\partial z}\frac{\partial z}{\partial x}+\frac{\partial\tilde{f}}{\partial z^{*}}\frac{\partial z^{*}}{\partial x}=\frac{\partial\tilde{f}}{\partial z}(z(x,y),z^{*}(x,y))+\frac{\partial\tilde{f}}{\partial z^{*}}(z(x,y),z^{*}(x,y))$
	$\displaystyle\frac{\partial f}{\partial y}(x,y)$	$\displaystyle=\frac{\partial\tilde{f}}{\partial z}\frac{\partial z}{\partial y}+\frac{\partial\tilde{f}}{\partial z^{*}}\frac{\partial z^{*}}{\partial y}=i\cdot\frac{\partial\tilde{f}}{\partial z}(z(x,y),z^{*}(x,y))-i\cdot\frac{\partial\tilde{f}}{\partial z^{*}}(z(x,y),z^{*}(x,y)).$

After rearranging terms,

	$\displaystyle\frac{\partial\tilde{f}}{\partial z}(z(x,y),z^{*}(x,y))$	$\displaystyle=\frac{1}{2}\left(\frac{\partial f}{\partial x}-i\frac{\partial f}{\partial y}\right)(x,y)$
	$\displaystyle\frac{\partial\tilde{f}}{\partial z^{*}}(z(x,y),z^{*}(x,y))$	$\displaystyle=\frac{1}{2}\left(\frac{\partial f}{\partial x}+i\frac{\partial f}{\partial y}\right)(x,y).$

The notations $z,z^{*}$ may raise questions on independence. This is irrelevant—one may simply write $z_{1},z_{2}$ if one wishes. We emphasize that the fundamental input variables are the two real numbers $x$ and $y$.

The purpose of the discussion above is as follows. We have that $f$ and $\tilde{f}$ are two different expressions of the complex function $f:\mathbb{C}\longrightarrow\mathbb{C}$, $f$ being expressed in terms of $x$ and $y$, while $\tilde{f}$ being expressed in terms of $z$ and $z^{*}$. Often, it is the ‘tilde-form’ of $f$ that is given. In such cases, certain procedures become a hassle. Examples include verifying the Cauchy-Riemann conditions, or finding the optimal points for $f$ (in the case $f$ is real-valued). The common feature of these operations is that they involve taking the partial derivatives $\frac{\partial}{\partial x},\frac{\partial}{\partial y}$ of $f$, which requires $f$ to be expressed in terms of $x$ and $y$ beforehand. This conversion (from $\tilde{f}(z,z^{*})$ to $f(x,y)$, using Eq. 2.1) is generally tedious, and the resulting form of $f(x,y)$ cumbersome. For instance, consider $\tilde{f}(z,z^{*})=z^{m}z^{*n}$ for large $m,n\in\mathbb{Z}$. As we shall see below, the operators $\frac{\partial}{\partial z},\frac{\partial}{\partial z^{*}}$ allow us to circumvent this by retaining the form $\tilde{f}(z,z^{*})$ and partially differentiate with respect to $z$ and $z^{*}$ instead. In a nutshell, we can view $\tilde{f}(z,z^{*})$ as a black box encapsulating the complexity of $f(x,y)$ and $\frac{\partial}{\partial z},\frac{\partial}{\partial z^{*}}$ as ‘higher-level’ operators acting on the black box.

We state without proof the following:

We emphasize that $f$ need not be complex-differentiable—so long as $f(x,y)$ is real-differentiable, or equivalently, the partial derivatives $u_{x},u_{y},v_{x},v_{y}$ exist (see Section 1.3), the Wirtinger derivatives of $f$ exist. If $f$ is complex-differentiable, however, one Wirtinger derivative becomes zero and the other reduces to the normal complex derivative.

The concise equation $\frac{\partial f}{\partial z^{*}}=0$ thus merges the two Cauchy-Riemann equations into one. This also agrees with conventional wisdom whereby complex-differentiable functions ‘have no $z^{*}$ terms in their expressions’. The next result enables us to optimize for real-valued $f$ using Wirtinger derivatives.

As in Proposition 2.5, we see the role of the Wirtinger derivatives as bookkeeping devices. Here $\partial f/\partial z=0$ effectively merges the optimality conditions $\partial f/\partial x=0$ and $\partial f/\partial y=0$ into a single equation. The nature of the stationary point (i.e. whether it is a minimum/maximum/saddle point) has to be checked by inspecting higher-order derivatives or via additional considerations.

The extension of this formalism to functions $f:\mathbb{C}^{n}\longrightarrow\mathbb{C}$ is straightforward. We have

	$\displaystyle f:\;$	$\displaystyle\mathbb{R}^{2n}\longrightarrow\mathbb{R}^{2}$
		$\displaystyle(x_{1},y_{1},\dots,x_{n},y_{n})=(\mathbf{x},\mathbf{y})\mapsto(u(\mathbf{x},\mathbf{y}),v(\mathbf{x},\mathbf{y})).$

Now for $i=1,\dots,n$ regard $z_{i},z_{i}^{*}$ as functions from $\mathbb{R}^{n}\times\mathbb{R}^{n}$ to $\mathbb{C}$, where $z_{i}(\mathbf{x},\mathbf{y})=x_{i}+iy_{i}$ and $z_{i}^{*}(\mathbf{x},\mathbf{y})=x_{i}-iy_{i}$. Then we have a function $\tilde{f}:\mathbb{C}^{n}\times\mathbb{C}^{n}\longrightarrow\mathbb{C}$ such that

$\displaystyle f(\mathbf{x},\mathbf{y})=:\tilde{f}\circ(\mathbf{z},\mathbf{z^{*}})(\mathbf{x},\mathbf{y})=\tilde{f}(\mathbf{z}(\mathbf{x},\mathbf{y}),\mathbf{z^{*}}(\mathbf{x},\mathbf{y}))=\tilde{f}(\mathbf{x+iy},\mathbf{x-iy}).$

(2.6)

Partial differentiating $f$ with respect to each $x_{i}$ and $y_{i}$ gives

	$\displaystyle\frac{\partial f}{\partial x_{i}}(\mathbf{x},\mathbf{y})$	$\displaystyle=\sum_{k=1}^{n}\frac{\partial\tilde{f}}{\partial z_{k}}\frac{\partial z_{k}}{\partial x_{i}}+\sum_{k=1}^{n}\frac{\partial\tilde{f}}{\partial z_{k}^{*}}\frac{\partial z_{k}^{*}}{\partial x_{i}}=\frac{\partial\tilde{f}}{\partial z_{i}}(\mathbf{z}(\mathbf{x},\mathbf{y}),\mathbf{z^{*}}(\mathbf{x},\mathbf{y}))+\frac{\partial\tilde{f}}{\partial z_{i}^{*}}(\mathbf{z}(\mathbf{x},\mathbf{y}),\mathbf{z^{*}}(\mathbf{x},\mathbf{y}))$
	$\displaystyle\frac{\partial f}{\partial y_{i}}(\mathbf{x},\mathbf{y})$	$\displaystyle=\sum_{k=1}^{n}\frac{\partial\tilde{f}}{\partial z_{k}}\frac{\partial z_{k}}{\partial y_{i}}+\sum_{k=1}^{n}\frac{\partial\tilde{f}}{\partial z_{k}^{*}}\frac{\partial z_{k}^{*}}{\partial y_{i}}=i\cdot\frac{\partial\tilde{f}}{\partial z_{i}}(\mathbf{z}(\mathbf{x},\mathbf{y}),\mathbf{z^{*}}(\mathbf{x},\mathbf{y}))-i\cdot\frac{\partial\tilde{f}}{\partial z_{i}^{*}}(\mathbf{z}(\mathbf{x},\mathbf{y}),\mathbf{z^{*}}(\mathbf{x},\mathbf{y}))$

so after rearranging terms we obtain the Wirtinger derivatives

	$\displaystyle\frac{\partial\tilde{f}}{\partial\mathbf{z}}(\mathbf{z}(\mathbf{x},\mathbf{y}),\mathbf{z^{*}}(\mathbf{x},\mathbf{y}))$	$\displaystyle=\frac{1}{2}\left(\frac{\partial f}{\partial\mathbf{x}}-i\frac{\partial f}{\partial\mathbf{y}}\right)(\mathbf{x},\mathbf{y})$
	$\displaystyle\frac{\partial\tilde{f}}{\partial\mathbf{z^{*}}}(\mathbf{z}(\mathbf{x},\mathbf{y}),\mathbf{z^{*}}(\mathbf{x},\mathbf{y}))$	$\displaystyle=\frac{1}{2}\left(\frac{\partial f}{\partial\mathbf{x}}+i\frac{\partial f}{\partial\mathbf{y}}\right)(\mathbf{x},\mathbf{y}).$

Here $\frac{\partial}{\partial\mathbf{z}}=\left(\frac{\partial}{\partial z_{n}},\dots,\frac{\partial}{\partial z_{1}}\right)$ and similarly for $\frac{\partial}{\partial\mathbf{z^{*}}}$, $\frac{\partial}{\partial\mathbf{x}}$ and $\frac{\partial}{\partial\mathbf{y}}$. As before, we abuse notation and write both $f(\mathbf{x},\mathbf{y})$ and $f(\mathbf{z},\mathbf{z^{*}})$, so we can write

$\displaystyle\frac{\partial}{\partial\mathbf{z}}=\frac{1}{2}\left(\frac{\partial}{\partial\mathbf{x}}-i\frac{\partial}{\partial\mathbf{y}}\right),\qquad\frac{\partial}{\partial\mathbf{z^{*}}}=\frac{1}{2}\left(\frac{\partial}{\partial\mathbf{x}}+i\frac{\partial}{\partial\mathbf{y}}\right).$

(2.7)

The extension of Proposition 2.6 also holds, namely if $f:\mathbb{C}^{n}\longrightarrow\mathbb{R}$ is real-valued then $f$ has a stationary point at $\mathbf{z}=(\mathbf{x},\mathbf{y})$ if and only if $\frac{\partial f}{\partial\mathbf{z/z^{*}}}(\mathbf{z})=\mathbf{0}$.

Now we consider functions of the form $f:\mathbb{C}^{n\times n}\longrightarrow\mathbb{C}$.⁵⁵5We could more generally let the domain be $\mathbb{C}^{m\times n}$. Indeed, many of the results developed below hold for such matrices as well. But for simplicity and because our matrices of interest in applications are square, we let $m=n$ here. The following development, which we include for pedagogical purposes, is virtually the same as that in Section 2.2, just with different indexing. We have

	$\displaystyle f:\;$	$\displaystyle\mathbb{R}^{2(n\times n)}\longrightarrow\mathbb{R}^{2}$
		$\displaystyle(x_{ij},y_{ij})_{i,j\in[n]}=(\mathbf{X},\mathbf{Y})\mapsto(u(\mathbf{X},\mathbf{Y}),v(\mathbf{X},\mathbf{Y})).$

For $i=1,\dots,n$ regard $z_{ij},z_{ij}^{*}$ as functions from $\mathbb{R}^{n\times n}\times\mathbb{R}^{n\times n}$ to $\mathbb{C}$, where $z_{ij}(\mathbf{X},\mathbf{Y})=x_{ij}+iy_{ij}$ and $z_{ij}^{*}(\mathbf{X},\mathbf{Y})=x_{ij}-iy_{ij}$. Then we have a function $\tilde{f}:\mathbb{C}^{n\times n}\times\mathbb{C}^{n\times n}\longrightarrow\mathbb{C}$ such that

$\displaystyle f(\mathbf{X},\mathbf{Y})=:\tilde{f}\circ(\mathbf{Z},\mathbf{Z^{*}})(\mathbf{X},\mathbf{Y})=\tilde{f}(\mathbf{Z}(\mathbf{X},\mathbf{Y}),\mathbf{Z^{*}}(\mathbf{X},\mathbf{Y}))=\tilde{f}(\mathbf{X+iY},\mathbf{X-iY}).$

(3.1)

Again partial differentiating $f$ with respect to each $x_{ij}$ and $y_{ij}$ gives

	$\displaystyle\frac{\partial f}{\partial x_{ij}}(\mathbf{X},\mathbf{Y})$	$\displaystyle=\sum_{k,l=1}^{n}\frac{\partial\tilde{f}}{\partial z_{kl}}\frac{\partial z_{kl}}{\partial x_{ij}}+\sum_{k,l=1}^{n}\frac{\partial\tilde{f}}{\partial z_{kl}^{*}}\frac{\partial z_{kl}^{*}}{\partial x_{ij}}$
		$\displaystyle=\sum_{k,l=1}^{n}\frac{\partial\tilde{f}}{\partial z_{kl}}\delta_{ki}\delta_{lj}+\sum_{k,l=1}^{n}\frac{\partial\tilde{f}}{\partial z_{kl}^{*}}\delta_{ki}\delta_{lj}$
		$\displaystyle=\frac{\partial\tilde{f}}{\partial z_{ij}}(\mathbf{Z}(\mathbf{X},\mathbf{Y}),\mathbf{Z^{*}}(\mathbf{X},\mathbf{Y}))+\frac{\partial\tilde{f}}{\partial z_{ij}^{*}}(\mathbf{Z}(\mathbf{X},\mathbf{Y}),\mathbf{Z^{*}}(\mathbf{X},\mathbf{Y}))$

and

	$\displaystyle\frac{\partial f}{\partial y_{ij}}(\mathbf{X},\mathbf{Y})$	$\displaystyle=\sum_{k,l=1}^{n}\frac{\partial\tilde{f}}{\partial z_{kl}}\frac{\partial z_{kl}}{\partial y_{ij}}+\sum_{k,l=1}^{n}\frac{\partial\tilde{f}}{\partial z_{kl}^{*}}\frac{\partial z_{kl}^{*}}{\partial y_{ij}}$
		$\displaystyle=i\cdot\frac{\partial\tilde{f}}{\partial z_{ij}}(\mathbf{Z}(\mathbf{X},\mathbf{Y}),\mathbf{Z^{*}}(\mathbf{X},\mathbf{Y}))-i\cdot\frac{\partial\tilde{f}}{\partial z_{ij}^{*}}(\mathbf{Z}(\mathbf{X},\mathbf{Y}),\mathbf{Z^{*}}(\mathbf{X},\mathbf{Y})).$

Rearranging terms as before gives us for $1\leq i,j\leq n$

	$\displaystyle\frac{\partial\tilde{f}}{\partial z_{ij}}(\mathbf{Z}(\mathbf{X},\mathbf{Y}),\mathbf{Z^{*}}(\mathbf{X},\mathbf{Y}))$	$\displaystyle=\frac{1}{2}\left(\frac{\partial f}{\partial x_{ij}}-i\frac{\partial f}{\partial y_{ij}}\right)(\mathbf{X},\mathbf{Y})$		(3.2)
	$\displaystyle\frac{\partial\tilde{f}}{\partial z_{ij}^{*}}(\mathbf{Z}(\mathbf{X},\mathbf{Y}),\mathbf{Z^{*}}(\mathbf{X},\mathbf{Y}))$	$\displaystyle=\frac{1}{2}\left(\frac{\partial f}{\partial x_{ij}}+i\frac{\partial f}{\partial y_{ij}}\right)(\mathbf{X},\mathbf{Y}).$

To preserve the matrix structure of the parameters $z_{ij}$ and $z_{ij}^{*}$ we use the standard notation

$\displaystyle\frac{\partial}{\partial\mathbf{Z}}:=\begin{bmatrix}\frac{\partial}{\partial z_{11}}&\dots&\frac{\partial}{\partial z_{1n}}\\ \vdots&\ddots&\vdots\\ \frac{\partial}{\partial z_{n1}}&\dots&\frac{\partial}{\partial z_{nn}}\end{bmatrix}\qquad\frac{\partial}{\partial\mathbf{Z^{*}}}:=\begin{bmatrix}\frac{\partial}{\partial z_{11}^{*}}&\dots&\frac{\partial}{\partial z_{1n}^{*}}\\ \vdots&\ddots&\vdots\\ \frac{\partial}{\partial z_{n1}^{*}}&\dots&\frac{\partial}{\partial z_{nn}^{*}}\end{bmatrix}$

(3.3)

and similarly for $\frac{\partial}{\partial\mathbf{X}}$ and $\frac{\partial}{\partial\mathbf{Y}}$. Then Equation 3.2 is concisely stated as

	$\displaystyle\frac{\partial\tilde{f}}{\partial\mathbf{Z}}(\mathbf{Z}(\mathbf{X},\mathbf{Y}),\mathbf{Z^{*}}(\mathbf{X},\mathbf{Y}))$	$\displaystyle=\frac{1}{2}\left(\frac{\partial f}{\partial\mathbf{X}}-i\frac{\partial f}{\partial\mathbf{Y}}\right)(\mathbf{X},\mathbf{Y})$		(3.4)
	$\displaystyle\frac{\partial\tilde{f}}{\partial\mathbf{Z^{*}}}(\mathbf{Z}(\mathbf{X},\mathbf{Y}),\mathbf{Z^{*}}(\mathbf{X},\mathbf{Y}))$	$\displaystyle=\frac{1}{2}\left(\frac{\partial f}{\partial\mathbf{X}}+i\frac{\partial f}{\partial\mathbf{Y}}\right)(\mathbf{X},\mathbf{Y}).$

$\frac{\partial}{\partial\mathbf{Z}}$ and $\frac{\partial}{\partial\mathbf{Z^{*}}}$ are the matrix Wirtinger derivatives of $f$ and they are the protagonists of this article. As before, we abuse notation and write both $f(\mathbf{X},\mathbf{Y})$ and $f(\mathbf{Z},\mathbf{Z^{*}})$, so we can write

$\displaystyle\frac{\partial}{\partial\mathbf{Z}}=\frac{1}{2}\left(\frac{\partial}{\partial\mathbf{X}}-i\frac{\partial}{\partial\mathbf{Y}}\right),\qquad\frac{\partial}{\partial\mathbf{Z^{*}}}=\frac{1}{2}\left(\frac{\partial}{\partial\mathbf{X}}+i\frac{\partial}{\partial\mathbf{Y}}\right).$

(3.5)

The following matrix version of Proposition 2.6 will be invoked frequently in optimization problems. The proof is omitted since it is a simple generalization of the proof in Proposition 2.6.