Towards a conjecture on a special class of matrices over commutative rings of characteristic 2

At Eurocrypt 2021, Keller and Rosemarin posed the following conjecture in [8] (an initial version appeared on ePrint in Feb. 2020 ¹¹1See https://eprint.iacr.org/eprint-bin/versions.pl?entry=2020/179), in their study of the resistance of the HADES design against invariant subspace attacks.

In Conjecture 1, a $2^{k}\times 2^{k}$ special matrix over a commutative ring²²2All rings considered in this paper are assumed to be unital ones. $R$ is defined recursively in the manner that

where $A$ and $B$ are both $2^{k-1}\times 2^{k-1}$ special matrices over $R$ (see [8, Definition 1]). Note that when $R=\mathbb{F}_{2^{n}}$ is a finite field, a special matrix is just the so-called Finite-Field-Hadamard (FFHadamard) matrix definied in [9]. Since when $\hbox{\rm{Char}}(R)=2$ such special matrices share similar properties with the classical $\{\pm 1\}$-valued Hadamard matrices, in the following we also call them Hadamard matrices over $R$. See Section 2 for some known properties of Hadamard matrices and their applications.

Hadamard matrices over a commutative ring $R$ have many nice properties. For example, the set of all $2^{k}\times 2^{k}$ Hadamard matrices, $\mathcal{H}_{k}(R)$, forms a commutative ring (see [8, Proposition 1]), and since it is naturally an $R$-module, it forms a commutative $R$-algebra. We further characterize structure of this algebra in Section 5 to help understanding properties of Hadamard matrices deeper.

It is easy to observe that any $H\in\mathcal{H}_{k}(R)$ is determined by its first row, say, $(a_{0},a_{1},\ldots,a_{2^{k}-1})\in R^{2^{k}}$, from the recursive definition of a Hadamard matrix. By induction on $k$, one can prove that each element of $H$ can be determined by

Note here that we index the rows and columns of $H$ starting from 0, and $\oplus$ is the exclusive-or operation of integers, in the sense of distinguishing them with binary vectors in $\mathbb{F}^{k}_{2}$ through 2-adic expansions. From this explicit representation of Hadamard matrices, one can derive all properties of them presented in [8] in the case $\hbox{\rm{Char}}(R)=2$, in a slightly different but more direct manner. We summarize some of them in the following proposition.

From an algebraic point of view, Proposition 2.1 says that the two maps

are both homomorphisms of rings.

Let $R$ be a commutative ring with characteristic 2 and denote by $\mathscr{M}_{s\times s}(\mathcal{H}_{k}(R))$ and $\mathscr{M}_{s\times s}(R)$ the $\mathcal{H}_{k}(R)$- and $R$-algebra of $s\times s$ matrices over $\mathcal{H}_{k}(R)$ and $R$, respectively. The homomorphism $\lambda:\;\mathcal{H}_{k}(R)\longrightarrow R$ extends naturally to an $R$-algebraic homomorphism

Similarly, the homomorphism det extends to $\overline{\hbox{\rm{det}}}$ over matrix algebras in this manner. For the purpose of clearity, let Det denote the classical determinalt map for $s\times s$ matrices over commutative rings, which is a homomorphism between multiplicative monoids of rings. Then Proposition 2.1 also implies that the diagrams for multiplicative monoids of algebras (rings)

and

are both commutative (see also [8, Proposition 8]).

In fact, when $R=\mathbb{F}_{2^{n}}$ is a finite field, properties of Hadmard matrices over $R$, namely, the FFHadmard matrices, appear in the literature before [8]. They behave as a good source of involutory MDS matrices which are friendly in the design of linear diffusion layers for classical symmetric ciphers. We refer to [10, 1, 9, 6] for some previous work related to them. In addition, when $R=\mathbb{R}$ is the real field, we recall another interesting source of Hadmard matrices over $R$ in cryptography. Let $X$ be a random variable over $\mathbb{F}_{2}^{k}$ with probability distribution $(x_{0},x_{1},\ldots,x_{2^{k}-1})$, that is, $\Pr(X=i)=x_{i}$ for any $0\leq i\leq 2^{k}-1$ distinguished with a binary vector in $\mathbb{F}_{2}^{k}$ by 2-adic expansion. Let $X^{\prime}$ be a random variable over $\mathbb{F}_{2}^{k}$ which is independent with $X$ with probability distribution $(x_{0}^{\prime},x_{1}^{\prime},\ldots,x_{2^{k}-1}^{\prime})$. Then the probability distribution $(y_{0},y_{1},\ldots,y_{2^{k}-1})$ of the random variable $Y=X\oplus X^{\prime}$ can be determined by

where $T_{\oplus}$ is just the Hadamard matrix determined by the first row $(x_{0},x_{1},\ldots,$ $x_{2^{k}-1})$, namely, $(T_{\oplus})_{i,j}=x_{i\oplus j}$, $0\leq i,j\leq 2^{k}-1$. This result is useful in truncated differential attacks of symmetric ciphers; see [7].

In this part we explain how to prove Conjecture 1. It turns out that the main argument leads to the proof is very simple.

For a generic $s\times s$ matrix $A=(a_{ij})$ over a commutative ring, it is clear from the definition of determinant that $\hbox{\rm{Det}}(A)$ is a multivariate polynomial in the entries $\{a_{ij}\}$. Assume $f_{A}(x)=\hbox{\rm{det}}(xI_{s}-A)=x^{s}+\sum_{i=1}^{s}f_{i}x^{s-i}$ is the characteristic polynomial of $A$. We are then clear that $f_{k}$ is a multivariate polynomial in the entries $\{a_{ij}\}$ for any $1\leq k\leq s$. In fact, it is well known $f_{s}=(-1)^{s}\hbox{\rm{Det}}(A)$. A not-so-well-known result is that for $1\leq k\leq s$,

where $\hbox{\rm{tr}}(\wedge^{k}A)$ is the trace of the $k$-th exterior power of the endmorphism induced by $A$, which can be computed as the sum of all principle minors of $A$ of size $k$. Since each minor of $A$ is the determinant of a sub-matrix, of course a multivariate polynomial in the entries of $A$, hence $f_{k}$ is also a multivariate polynomial in the entries of $A$ for any $1\leq k\leq s$.

Let $R$ be a commutative ring with characteristic 2, and let $M$ and $M^{\prime\prime}$ be the matrices in Conjecture 1. Instead of a block matrix, we view $M$ as a matrix over the commutative ring $\mathcal{H}_{k}(R)$. Assume the characteristic polynomial of $M$ and $M^{\prime\prime}$ are $Q(x)=\sum_{i=0}^{s}Q_{i}x^{i}$ and $q(x)=\sum_{i=0}^{s}q_{i}x^{i}$, respectively. Note that $Q_{i}\in\mathcal{H}_{k}(R)$ while $q_{i}\in R$, $0\leq i\leq s$. From the above discussion, $Q_{i}$ and $q_{i}$ can be computed by evaluating the same multivariate polynomial in the corresponding entries of $M$ and $M^{\prime\prime}$, respectively. Since $\lambda:\;\mathcal{H}_{k}(R)\longrightarrow R$ is a homomorphism, we are clear that $\lambda(Q_{i})=q_{i}$. From Proposition 2.1 (2), we also have $Q_{i}^{2}=\lambda(Q_{i})^{2}\cdot id=q_{i}^{2}\cdot id$, where $id$ is the identity element of $\mathcal{H}_{k}(R)$, namely, $I_{2^{k}}$.

By Cayley–Hamilton theorem for matrices over commutative rings, we know that $Q(M)=0$, and of course $Q(M)^{2}=0$. Since the ring $\mathcal{H}_{k}(R)$ also has characteristic 2, we have

This completes the proof.

The proof of Conjecture 1 answers the second open problem in [8], that is, the lower bound of dimension of the invariant subspace $U$ defined for the $t\times t$ Cauchy-type MDS matrix $M$ used in the design of the Starkad cipher can be improved to $t-2s$ where $t=2^{k}\cdot s$.

A natural following question is whether this bound can be further improved. We should note first that the bound $t-2s$ is a general one, not depending on the ring $R$ and the shapes of these Hadamard blocks of $M$. Of course when these blocks are of certain special types, e.g., scalar matrices, the bound $t-2s$ can be improved to, e.g., $t-s$. However, this is not the case for the Cauchy matrix used in Starkad.

Another natural question is whether the characteristic polynomial $q(x)$ in Conjecture 1 can be replaced by minimal polynomial, which may has a degree less than $s$. More precisely, if the minimal polynomials of $M^{\prime\prime}$ is $\phi(x)$, shall we have $\phi(M)^{2}=0$? First, it should be noted that in this case the method for proving Conjecture 1 in Section 3 will not work, since coefficients of minimal polynomial of a matrix have no direct and explicit relations with its entries. Second, when $R$ is a generic commutative ring, the minimal polynomial of a matrix over $R$ may not be unique. Actually, minimal polynomial of a matrix $A$ over $R$ is defined as the least degree polynomials in the annihilating ideal of $A$ in $R[x]$, which may not be a principle ideal. Even the minimal polynomial is unique, $\phi(M)^{2}=0$ does not always hold. Indeed, one can quickly observe that, for example, when $M^{\prime\prime}=0$, its minimal polynomial is $\phi(x)=x$, however, one cannot obtain $M^{2}=0$ for any $M$ whose blocks all have eigenvalue 0.

But on the contrary, if we can find the minimal polynomial $\Phi(x)=\sum_{i=0}^{s}\Phi_{i}x^{i}\in\mathcal{H}_{k}(R)[x]$ of $M$, that is $\Phi(M)=0$, then we can obtain $\phi(M)^{2}=0$ where $\phi(x)=\sum_{i=0}^{s}\phi_{i}x^{i}$ with $\phi_{i}=\lambda(\Phi_{i})$. This will improve the lower bound of $\dim U$ to $t-2\cdot\deg\Phi(x)$. However, for generic $s\times s$ matrices over commutative rings, the best general upper bound for the degrees of their minimal polynomials one can get is $s$. So in this sense, the bound $t-2s$ for a generic $M$ is optimal. When $M$ is considered in some special classes of matrices over $\mathcal{H}_{k}(R)$, e.g., circulant matrices, Vandermonde matrices, or Hadamard matrices we consider, this bound can possibly be improved.

As for Conjecture 1, it seems hard to directly prove it through evaluating $q(M)^{2}$. It has already been observed in [8] that $q(M)$ lies in the kernel of the homomorphism $\bar{\lambda}$, that is, all blocks of $q(M)$ have eigenvalue 0. However, as mentioned above, we cannot obtain $\tilde{M}^{2}=0$ for any $\tilde{M}\in\hbox{\rm{ker}}\bar{\lambda}$ in general. Besides, we can see that if $q(M)^{2}=0$, then for any $\tilde{M}\in\hbox{\rm{ker}}\bar{\lambda}$, we have

But this does not mean if $g(M)=0$ for certain $g(x)\in R[x]$, then we have $g(M+\tilde{M})=0$ for any $\tilde{M}\in\hbox{\rm{ker}}\bar{\lambda}$. Indeed, any $M\in\mathscr{M}_{s\times s}(\mathcal{H}_{k}(R))$ can be factorized into

for a unique $\tilde{M}\in\hbox{\rm{ker}}\bar{\lambda}$. Obviously, for any $g(x)\in R[x]$ with $g(M^{\prime\prime})=0$ (e.g., $g(x)=q(x)$, the characteristic polynomial of $M^{\prime\prime}$), we have

But this does not promise $g(M)=0$ for any $\tilde{M}\in\hbox{\rm{ker}}\bar{\lambda}$.

Another interesting corollary of the proved Conjecture 1 is, if ${M}\in\hbox{\rm{ker}}\bar{\lambda}$, then we have $M^{2s}=0$ (not depending on $k$ which determines the size of each block) since the characteristic polynomial of $M^{\prime\prime}=0$ is $x^{s}$. Recall that in [8] it was proved $M^{k+1}=0$, an equality depending on $k$. The power $k+1$ comes from [8, Proposition 7], namely, any $k+1$ elements of $\mathcal{H}_{k}(R)$ all having eigenvalue 0 will multiply to 0. The result $M^{2s}=0$ implies more complicated relations between elements of $\mathcal{H}_{k}(R)$ having eigenvalue 0, which seems not easy to directly reveal. In the next section we will further discuss the set of all such elements.

To further understand properties of Hadamard matrices over a commutative ring $R$ (not necessarily with characteristic 2), in this part, we give characterizations of the structure of the algebra formed by them, namely, the $R$-algebra $\mathcal{H}_{k}(R)$.

Let $G=(\mathbb{F}_{2}^{k},\oplus)$, the additive group of the vector space $\mathbb{F}_{2}^{k}$. We denote the identity of $G$ by $e$, i.e., $e=(0,0,\ldots,0)$. Let $R[G]$ be the group ring (algebra) generated by $G$ over $R$. Elements of $R[G]$ are all of the form $a=\sum_{g\in G}a_{g}g$ where $a_{g}\in R$ for any $g\in G$, that is, formal linear combinations of elements of $G$ over $R$. Multiplication of two elements $a$ and $b$ are defined in a convolutional manner, that is,

We have the following theorem.

$R[G]$ is an algebra over $R$ with dimension $2^{k}$, and a basis is $\{g\mid g\in G\}$. Note that all these basis elements are idempotent in $R[G]$. Elements of $R[G]$ can also be distinguished with functions from $G$ to $R$. In this sense, $R[G]$ is isomorphic to the $R$-representation of $G$. Since $G=\mathbb{F}_{2}^{\oplus k}$, the $k$-fold direct sum of $\mathbb{F}_{2}$, we also have

Here $\otimes k$ denotes $k$-fold tensor product of an $R$-algebra. This tensor decomposition of $\mathcal{H}_{k}(R)$ can also be made explicit. Let $\{e_{i}\mid 0\leq i\leq k-1\}$ be the standard basis of $G$ over $\mathbb{F}_{2}$, i.e., $e_{i}$ has $i$-th component 1 and all other components 0 ($0\leq i\leq k-1$). Then any $g\in G\backslash\{e\}$ can be represented as $g=e_{i_{1}}\oplus e_{i_{2}}\oplus\cdots\oplus e_{i_{s}}$ for certain $0\leq i_{1}<i_{2}<\cdots<i_{s}\leq k-1$. It is easy to check that the Hadamard matrix corresponding to this $g$ under the isomorphism in Theorem 5.2, which is actually a permutation matrix, can be decomposed into

where

Besides, the Hadamard matrix corresponding to $e$ is obviously $I_{2^{k}}=I_{2}^{\otimes k}$. Note that $I_{2}$ and $J_{2}$ form the basis of $\mathcal{H}_{1}(R)$ and $J_{2}^{2}=I_{2}$. Therefore, under the conversion that $J_{2}^{0}=I_{2}$, the isomorphism (4) implies that any $2^{k}\times 2^{k}$ Hadamard matrix over $R$ can be decomposed into a polynomial-like form, that is,

where

From properties of Kronecker products of matrices, we have

Hence this polynomial-like representation for Hadamard matrices indeed induces an isomorphism between $\mathcal{H}_{k}(R)$ and a polynomial algebra. In this sense, $\mathcal{H}_{k}(R)$ is also a Clifford algebra over $R$.

Recall that for any group ring, we can define the augmentation map, that is,

The kernel $I$ of $\epsilon$ is called the augmentation ideal of $R[G]$. It is easy to prove that, as a sub-algebra of $R[G]$, $I$ has dimension $2^{k}-1$ with a basis $\{g-e\mid g\in G\backslash\{e\}\}$.

In the following, we assume $\hbox{\rm{Char}}(R)=2$. From Proposition 2.1 we know that, by distinguishing a Hadamard matrix over $R$ with an element in $R[G]$, its image under $\epsilon$ is just the eigenvalue. Therefore, all elements in $I$ are nilpotent. When the ring $R$ has no nilpotent elements, the ideal $I$ is just the nilradical of $R[G]$, i.e., intersection of all prime ideals of $R[G]$. Specifically, when $R$ is a field, we know that $R[G]$ is an Artinian algebra (finite dimensional algebras over fields are Artinian), so $I$ is simultaneously the nilradical and Jacobson radical of $R[G]$. In fact, $I$ is the unique maximal ideal of $R[G]$ since it has a dimension $2^{k}-1$ and thus $R[G]$ is a local ring.

The nilpotency degree of an ideal is defined to be the smallest power that will make it vanish. For the ideal $I$ we talk about, its nilpotency degree can be determined.

Theorem 5.4 indicates that any $k+1$ elements of $I$ multiply to 0, which coincides with [8, Proposition 7]. Considering Theorem 5.3, the proof of Theorem 5.4 can become more simpler or in some sense obvious. In fact, it is easy to observe that, under the isomorphisms in Theorem 5.2 and Theorem 5.3, the augmentation ideal $I$ of $R[G]$ is isomorphic to the ideal

of the polynomial algebra $R[x_{1},x_{2},\ldots,x_{k}]/(x_{1}^{2}-1,\ldots,x_{k}^{2}-1)$. Note that $I^{\prime k+1}$ is generated by

which are all 0. This is because any $(k+1)$-term multiplication of $k$ elements must contain duplicate terms, killing itself modulo $x_{i}^{2}-1$ for some $1\leq i\leq k$. Therefore, after establishing the isomorphisms in Theorem 5.2 and Theorem 5.3, one can obtain a one-sentence proof of [8, Proposition 7].

Let $R$ be any unital ring (not necessarily commutative). We can also define Hadamard matrices over $R$ like in (2) for a given vector of lenth $2^{k}$ over $R$. Then it is direct to check the set of all such matrices, also denoted by $\mathcal{H}_{k}(R)$, forms a ring. Note that $\mathcal{H}_{k}(R)$ is not commutative if $R$ is not. Specially, if $R$ is the matrix ring, we call a Hadamard matrix over $R$ a block-Hadamard matrix. Indeed, it can be viewed as a block matrix of the Hadamard type. On the other hand, like in Conjecture 1 we can also consider block matrices whose blocks are all of Hadamard type, and we call them Hadamard-block matrices.

Now assume $R$ is a commutative ring. Then the sets of all $2^{k}s\times 2^{k}s$ block-Hadamard matrices and Hadamard-block matrices over $R$ admit the algebras $\mathcal{H}_{k}(\mathscr{M}_{s\times s}(R))$ and $\mathscr{M}_{s\times s}(\mathcal{H}_{k}(R))$, respectively. Note here that neither of the two algebras is commutative. We have the following theorem.

From Theorem 6.5 and the proved Conjecture 1, we have the following interesting result.

In this paper, we prove the conjecture posed by Keller and Rosemarin at Eurocrypt 2021 1 and thus give an affirmative answer to their open problem on an improved lower bound for dimension of invariant subspace of the Starkad cipher. We further study the set formed by all Hadamard matrices over commutative rings and reveal its algebraic structure from the perspectives of group algebra and polynomial algebra. It turns out that these characterizations can help us to understand properties of Hadamard matrices deeper and easier. In particular, the group algebra approach can promise generalizations of Hadamard matrices by considering other Abelian groups $G$ instead of $(\mathbb{F}_{2}^{k},\oplus)$, which have potential applications in the designs of linear diffusion layers for classical and arithmetization-oriented symmetric ciphers. We will study this topic in a further work.