Random Restrictions of High-Rank Tensors and Polynomial Maps

It is instructive to first consider the case of matrices, which is simpler and illustrates the spirit of our main results. For a matrix $A\in\mathbb{F}^{n\times n}$ and subsets $I,J\subseteq[n]$, denote by $A_{|I\times J}$ the sub-matrix of $A$ induced by the rows in $I$ and columns in $J$. Given $\sigma\in(0,1)$, consider a random set $I\subseteq[n]$ containing each element independently with probability $\sigma$; we write $I\sim[n]_{\sigma}$ when $I$ is distributed as such. Note that, if $I\sim[n]_{\rho}$ and $J\sim[n]_{\sigma}$ are independent, then $I\cup J\sim[n]_{\eta}$ with $\eta=1-(1-\rho)(1-\sigma)$.

• Proof:

  Write $\rho=1-\sqrt{1-\sigma}$ and let $J,J^{\prime}\sim[n]_{\rho}$ be independent random sets; note that $J\cup J^{\prime}\sim[n]_{\sigma}$. Let $r=\operatorname{rk}(A)$, and fix a set $S\subseteq[n]$ of $r$ linearly independent rows of $A$. By the Chernoff bound [17], the probability that the set $J$ satisfies $|J\cap S|<\rho r/2$ is at most $e^{-\rho r/8}$.

  Now let $B:=A_{|(J\cap S)\times[n]}$ be the (random) sub-matrix of $A$ formed by the rows in $J\cap S$. Since its rows are linearly independent, the rank of $B$ is precisely $|J\cap S|$; let $T\subseteq[n]$ be a set of $|J\cap S|$ linearly independent columns of $B$. Then the probability that $|J^{\prime}\cap T|<\rho|T|/2$ is at most $e^{-\rho|T|/8}$, and the rank of $B_{|(J\cap S)\times(J^{\prime}\cap T)}=A_{|(J\cap S)\times(J^{\prime}\cap T)}$ is equal to $|J^{\prime}\cap T|$. It follows from the union bound and monotonicity of rank under restrictions that, with probability at least $1-2e^{-\rho^{2}r/16}$, the principal sub-matrix of $A$ induced by $J\cup J^{\prime}$ has rank at least $\rho^{2}r/4$. The result now follows since $J\cup J^{\prime}\sim[n]_{\sigma}$. $\Box$

Here we generalize Proposition 1.1 to tensors and polynomial maps for rank functions that satisfy a few natural properties, namely “sub-additivity”, “monotonicity”, a “Lipschitz condition” and, in the case of polynomial maps, “symmetry” (see Section 2 and Section 3 for the precise definitions). Those functions which satisfy these properties are called natural rank functions; we note that all notions of rank mentioned above are natural rank functions.

Since our results are independent of the field considered (which can be finite or infinite), we will always denote it by $\mathbb{F}$ and suppress statements of the form “let $\mathbb{F}$ be a field” or “for every field $\mathbb{F}$”. We begin by considering the case of tensors.

We define $(\mathbb{F}^{\infty})^{\otimes d}$ to be the set of $d$-tensors over $\mathbb{F}$ with finite support. Note that the tensors defined on finite sets naturally embed into this set, and that the rank functions for tensors discussed above are invariant under this embedding. Our main result regarding tensors is then as follows:

As noted before, the union of independent Bernoulli-random subsets of $[n]$ is again a Bernoulli-random subset. As a consequence of this and monotonicity under restrictions, in the standard case of “cubic” tensors where every row is indexed by the same set, one also obtains the following symmetric version of the last theorem:

Whereas the proof of the matrix case (Proposition 1.1) uses the fact that a rank-$r$ matrix contains a full-rank $r\times r$ submatrix, the proof of the general case of Theorem 1.4 proceeds differently and instead uses ideas from probability theory, in particular concerning concentration inequalities on product spaces. It will be presented in Section 2.

Recently, the topic of high-rank restrictions of tensors was explored in detail by Karam [20], and Gowers provided an example of a 3-tensor of slice rank $4$ which does not have a full-rank $4\times 4\times 4$ subtensor (see [20, Proposition 3.1]). There are some interesting parallels between our results and those of Karam, which will be discussed later on in Section 4; to the best of our knowledge, his results do not suffice to establish our main theorem above, nor the other way around.

Next we consider the setting of polynomial maps, which are formally defined as follows:

We denote the space of all polynomial maps $\phi:\mathbb{F}^{n}\to\mathbb{F}^{k}$ of degree at most $d$ by $\operatorname{Pol}_{\leq d}(\mathbb{F}^{n},\mathbb{F}^{k})$, and write

Our main result in this setting is the following:

The proof of this theorem will be given in Section 3. For reasons that will be made clear in that section, this proof will be (at least superficially) quite different from that of the tensor case, Theorem 1.4; it relies instead on results from analysis of Boolean functions taken together with elementary combinatorial arguments.

In Section 4 we discuss the relationship between our work and other works present in the literature. In particular, we will explain the specific problem which led to the study of rank under random coordinate restrictions. We will also give a simple conditional proof of our main theorems mimicking the case of matrices given in Proposition 1.1, as long as we assume the existence of a so-called “linear core” which would generalize (in a somewhat weak sense) the existence of a full-rank $r\times r$ submatrix inside matrices of rank $r$. Finally, we propose some open problems to further our understanding of rank functions.

Here we collect some notation that will be used throughout the paper. Given sets $I_{1},\dots,I_{d}$, we write $I_{[d]}=I_{1}\times\cdots\times I_{d}$. For a set $J$ and some $\sigma\in[0,1]$, we denote by $\pi_{\sigma}^{J}$ the product distribution on $\{0,1\}^{J}$ where each coordinate is independently set to $1$ with probability $\sigma$, or to $0$ with probability $1-\sigma$; when $J=[n]$, we write simply $\pi_{\sigma}^{n}$. We write $J_{\sigma}$ for the probability distribution over subsets of $J$ where each element is present independently with probability $\sigma$. To denote that a random variable $X$ is distributed according to a distribution $\mu$, we write $X\sim\mu$.

The main step in our proof of Theorem 1.4 is a type of concentration inequality for monotone sub-additive functions on the hypercube.

In order to obtain such a result we will consider a notion of two-point distance from sets $A\subseteq\{0,1\}^{n}$, which intuitively measures how many coordinates of some given point $x$ cannot be captured by any two elements of $A$.

Note that this “distance” $h_{2}(x;A)$ can be zero even if $x\notin A$; for instance, $h_{2}\big{(}00;\,\{01,10\}\big{)}=0$. The reason for considering this notion is that one gets much better concentration for $h_{2}$ than one does for the usual Hamming distance. This is shown by the following result, which is a special case of an inequality of Talagrand [33, Theorem 3.1.1]:

The next lemma is the key result needed for proving our main theorem for tensors, and it deals more abstractly with monotone, sub-additive Lipschitz functions on the hypercube. We endow $\{0,1\}^{n}$ with the usual partial order, where $x\leq y$ if the support of $x$ is contained in the support of $y$. A function $f:\{0,1\}^{n}\to\mathbb{R}$ is monotone if $f(x)\leq f(y)$ whenever $x\leq y$, and it is sub-additive if $f(x+y)\leq f(x)+f(y)$ for all $x,y\in\{0,1\}^{n}$ with disjoint supports. Finally, $f$ is $1$-Lipschitz if

For a string $x\in\{0,1\}^{n}$ and set $I\subseteq[n]$, we let $x_{I}\in\{0,1\}^{n}$ be the string that equals $x$ on the indices in $I$ and is zero elsewhere. The lemma that follows and its proof are inspired by an argument of Schechtman [31, Corollary 12].

• Proof:

  We first prove the first inequality above. Let $\sigma\in(0,1/2)$, and consider the set

  $$A=\big{\{}x\in\{0,1\}^{n}:\,f(x)\leq\sigma r/4\big{\}}.$$

  By Talagrand’s Inequality (Theorem 2.4) we have

  $$\mbox{\rm Pr}_{x\sim\pi_{\sigma}^{n}}\big{[}h_{2}(x;A)\geq\sigma r/6\big{]}\leq 2^{-\sigma r/6}\,\mbox{\rm Pr}_{x\sim\pi_{\sigma}^{n}}\big{[}f(x)\leq\sigma r/4\big{]}^{-2},$$

  so to finish the proof it suffices to show that

  $$\mbox{\rm Pr}_{x\sim\pi_{\sigma}^{n}}\big{[}h_{2}(x;A)\geq\sigma r/6\big{]}\geq 2\sigma/3.$$

  We claim that

  $$\big{\{}x\in\{0,1\}^{n}:\,f(x)\geq 2\sigma r/3\big{\}}\subseteq\big{\{}x\in\{0,1\}^{n}:\,h_{2}(x;A)\geq\sigma r/6\big{\}}.$$

  (1)

  Indeed, if $h_{2}(x;A)<\sigma r/6$ then there are $y,z\in A$ such that

  $$\big{|}\big{\{}i\in[n]:\,x_{i}\neq y_{i}\And x_{i}\neq z_{i}\big{\}}\big{|}<\sigma r/6.$$

  Denote $I=\big{\{}i\in[n]:\,x_{i}\neq y_{i}\And x_{i}\neq z_{i}\big{\}}$, $J=\big{\{}i\in[n]:\,x_{i}=y_{i}\big{\}}$ and $J^{\prime}=\big{\{}i\in[n]:\,x_{i}\neq y_{i}\And x_{i}=z_{i}\big{\}}$; note that these sets partition $[n]$, and by assumption $|I|<\sigma r/6$. We then have

  	$\displaystyle f(x)$	$\displaystyle\leq f(x_{I}+x_{J})+f(x_{J^{\prime}})$
  		$\displaystyle=f(x_{I}+y_{J})+f(z_{J^{\prime}})$
  		$\displaystyle\leq|I|+f(y_{J})+f(z_{J^{\prime}})$
  		$\displaystyle\leq|I|+f(y)+f(z)$
  		$\displaystyle<2\sigma r/3,$

  so $\big{\{}h_{2}(x;A)<\sigma r/6\big{\}}\subseteq\big{\{}f(x)<2\sigma r/3\big{\}}$ and inclusion (1) follows. It thus suffices to show that $\mbox{\rm Pr}_{x\sim\pi_{\sigma}^{n}}\big{[}f(x)\geq 2\sigma r/3\big{]}\geq 2\sigma/3$.

  We will next prove that, for any integer $k\geq 1$, we have

  $$\mbox{\rm Pr}_{x\sim\pi_{1/k}^{n}}\big{[}f(x)\geq r/k\big{]}\geq 1/k.$$

  (2)

  This is done by a simple coupling argument, which will be important for us again later on. Consider a uniformly random ordered $k$-partition $(I_{1},\dots,I_{k})$ of $[n]$; thus the $k$ sets $I_{i}$ are pairwise disjoint and have union $[n]$, with each of the $k^{n}$ possible such $k$-tuples having the same probability. Denoting by $\mathbf{1}_{I}$ the indicator function of set $I$, we have

  $$r=f(\mathbf{1}_{[n]})=f(\mathbf{1}_{I_{1}}+\dots+\mathbf{1}_{I_{k}})\leq\sum_{i=1}^{k}f(\mathbf{1}_{I_{i}}),$$

  so $f(\mathbf{1}_{I_{i}})\geq r/k$ holds for at least one $i\in[k]$ in every ordered partition $(I_{1},\dots,I_{k})$. By symmetry, it follows that $\mbox{\rm Pr}\big{[}f(\mathbf{1}_{I_{1}})\geq r/k\big{]}\geq 1/k$. Since the marginal distribution of $\mathbf{1}_{I_{1}}$ (and every other $\mathbf{1}_{I_{i}}$) is precisely $\pi_{1/k}^{n}$, we conclude that

  $$\mbox{\rm Pr}_{x\sim\pi_{1/k}^{n}}\big{[}f(x)\geq r/k\big{]}=\mbox{\rm Pr}\big{[}f(\mathbf{1}_{I_{1}})\geq r/k\big{]}\geq 1/k,$$

  as wished.

  Now let $k=\lceil 1/\sigma\rceil$. Since $0<\sigma<1/2$, we have $2\sigma/3\leq 1/k\leq\sigma$, and so by monotonicity of $f$

  	$\displaystyle\mbox{\rm Pr}_{x\sim\pi_{\sigma}^{n}}\big{[}f(x)\geq 2\sigma r/3\big{]}$	$\displaystyle\geq\mbox{\rm Pr}_{x\sim\pi_{1/k}^{n}}\big{[}f(x)\geq 2\sigma r/3\big{]}$
  		$\displaystyle\geq\mbox{\rm Pr}_{x\sim\pi_{1/k}^{n}}\big{[}f(x)\geq r/k\big{]}.$

  From inequality (2) we then conclude that

  $$\mbox{\rm Pr}_{x\sim\pi_{\sigma}^{n}}\big{[}f(x)\geq 2\sigma r/3\big{]}\geq 1/k\geq 2\sigma/3,$$

  finishing the proof of the first inequality in the statement of the lemma.

  The second inequality is proven using the same arguments, now applied to the parameter $\sigma=1/2$ and the set $A=\big{\{}x\in\{0,1\}^{n}:\,f(x)\leq r/6\big{\}}$. By the same argument as above,

  $$\big{\{}x\in\{0,1\}^{n}:\,f(x)\geq r/2\big{\}}\subseteq\big{\{}x\in\{0,1\}^{n}:\,h_{2}(x;A)\geq r/6\big{\}}.$$

  Talagrand’s inequality and coupling imply that

  	$\displaystyle 2^{-r/6}\mbox{\rm Pr}_{x\sim\pi_{1/2}^{n}}\big{[}f(x)\leq r/6\big{]}^{-2}$	$\displaystyle\geq\mbox{\rm Pr}_{x\sim\pi_{1/2}^{n}}\big{[}h_{2}(x;A)\geq r/6\big{]}$
  		$\displaystyle\geq\mbox{\rm Pr}_{x\sim\pi_{1/2}^{n}}\big{[}f(x)\geq r/2\big{]}$
  		$\displaystyle\geq\frac{1}{2}.$

  Rearranging then gives the result for $\sigma=1/2$; we obtain slightly better bounds since in this case $1/\sigma=2$ is an integer, and so no rounding errors occur. By monotonicity of $f$, the same bound continues to hold for all $\sigma>1/2$. $\Box$

It is now a simple matter to use Lemma 2.5 to prove the Random Restriction Theorem for tensors.

• Proof of Theorem 1.4:

  Let $\operatorname{rk}:(\mathbb{F}^{\infty})^{\otimes d}\to\mathbb{R}$ be a natural rank function, and $T\in\bigotimes_{i=1}^{d}\mathbb{F}^{[n_{i}]}$ be a tensor. We wish to show that

  $$\mbox{\rm Pr}_{I_{1}\sim[n_{1}]_{\sigma},\dots,I_{d}\sim[n_{d}]_{\sigma}}\big{[}\operatorname{rk}\big{(}T_{|I_{[d]}}\big{)}\geq\kappa(\sigma)\cdot\operatorname{rk}(T)\big{]}\geq 1-C(\sigma)e^{-\kappa(\sigma)\operatorname{rk}(T)}$$

  for some well-chosen constants $C(\sigma),\kappa(\sigma)>0$ depending only on the value of $\sigma>0$ and the order $d$ of the tensor. We consider here the case where $\sigma<1/2$, as the case where $\sigma\geq 1/2$ is analogous.²²2This second case is also an immediate consequence of first case together with monotonicity of $\operatorname{rk}$ under restrictions.

  Define the function $f_{1}:\{0,1\}^{n_{1}}\to\mathbb{R}_{+}$ as follows: for $x\in\{0,1\}^{n_{1}}$ with support $J$, $f_{1}(x)$ is the rank of the subtensor of $T$ whose indices of the first row belong to $J$. In formula:

  $$f_{1}(\mathbf{1}_{J})=\operatorname{rk}\big{(}T_{|J\times\prod_{j=2}^{d}[n_{j}]}\big{)}\quad\text{for all }J\subseteq[n_{1}].$$

  By the definition of natural rank, $f_{1}$ is a monotone, sub-additive $1$-Lipschitz function with maximum value $\operatorname{rk}(T)$. Since $\mathbf{1}_{J}\sim\pi_{\sigma}^{n_{1}}$ when $J\sim[n_{1}]_{\sigma}$, it follows from Lemma 2.5 that

  $$\mbox{\rm Pr}_{I_{1}\sim[n_{1}]_{\sigma}}\bigg{[}\operatorname{rk}\big{(}T_{|I_{1}\times\prod_{j=2}^{d}[n_{j}]}\big{)}\geq\frac{\sigma\operatorname{rk}(T)}{4}\bigg{]}\geq 1-\sqrt{\frac{3}{2\sigma}}\,2^{-\frac{\sigma}{12}\operatorname{rk}(T)}.$$

  (3)

  For any fixed $I_{1}\subseteq[n_{1}]$ satisfying $f_{1}(\mathbf{1}_{I_{1}})\geq\sigma\operatorname{rk}(T)/4$, we define the function $f_{2}:\{0,1\}^{n_{2}}\to\mathbb{R}_{+}$ by

  $$f_{2}(\mathbf{1}_{J})=\operatorname{rk}\big{(}T_{|I_{1}\times J\times\prod_{j=3}^{d}[n_{j}]}\big{)}\quad\text{for all }J\subseteq[n_{2}].$$

  This function is again monotone, sub-additive and $1$-Lipschitz, and it has maximum value at least $\sigma\operatorname{rk}(T)/4$. By Lemma 2.5 we have

  $$\mbox{\rm Pr}_{I_{2}\sim[n_{2}]_{\sigma}}\bigg{[}\operatorname{rk}\big{(}T_{|I_{1}\times I_{2}\times\prod_{j=3}^{d}[n_{j}]}\big{)}\geq\Big{(}\frac{\sigma}{4}\Big{)}^{2}\operatorname{rk}(T)\bigg{]}\geq 1-\sqrt{\frac{3}{2\sigma}}\,2^{-\frac{\sigma}{12}\frac{\sigma}{4}\operatorname{rk}(T)}.$$

  Since this holds whenever $f_{1}(\mathbf{1}_{I_{1}})\geq\sigma\operatorname{rk}(T)/4$, taking the union bound together with inequality (3) we obtain

  $$\mbox{\rm Pr}_{I_{1}\sim[n_{1}]_{\sigma},I_{2}\sim[n_{2}]_{\sigma}}\bigg{[}\operatorname{rk}\big{(}T_{|I_{1}\times I_{2}\times\prod_{j=3}^{d}[n_{j}]}\big{)}\geq\Big{(}\frac{\sigma}{4}\Big{)}^{2}\operatorname{rk}(T)\bigg{]}\geq 1-2\sqrt{\frac{3}{2\sigma}}\,2^{-\frac{\sigma}{12}\frac{\sigma}{4}\operatorname{rk}(T)}.$$

  Proceeding in this same way for each row of the tensor, and always taking the union bound, we eventually conclude that

  $$\mbox{\rm Pr}_{I_{1}\sim[n_{1}]_{\sigma},\dots,I_{d}\sim[n_{d}]_{\sigma}}\bigg{[}\operatorname{rk}\big{(}T_{|I_{[d]}}\big{)}\geq\Big{(}\frac{\sigma}{4}\Big{)}^{d}\operatorname{rk}(T)\bigg{]}\geq 1-d\sqrt{\frac{3}{2\sigma}}\,2^{-\frac{\sigma}{12}(\frac{\sigma}{4})^{d-1}\operatorname{rk}(T)},$$

  which finishes the proof. $\Box$