Lower Bounds and Improved Algorithms for Asymmetric Streaming Edit Distance and Longest Common Subsequence

We start with edit distance. One difficulty here is to handle substitutions, as with substitutions edit distance becomes similar to Hamming distance, and this is exactly one of the reasons why strong complexity results separating edit distance and Hamming distance are rare. Indeed, if we define $\mathsf{ED}(x,y)$ to be the smallest number of insertions and deletions (without substitutions) to transform $x$ into $y$, then $\mathsf{ED}(x,y)=2n-2\mathsf{LCS}(x,y)$ and thus a lower bound for exactly computing $\mathsf{LCS}$ (e.g., those implied from [GG10, EJ08]) would translate directly into the same bound for exactly computing $\mathsf{ED}$. On the other hand, with substitutions things become more complicated: if $\mathsf{LCS}(x,y)$ is small (e.g., $\mathsf{LCS}(x,y)\leq n/2$) then in many cases (such as examples obtained by reducing from [GG10, EJ08]) the best option to transform $x$ into $y$ is just replacing each symbol in $x$ by the corresponding symbol in $y$ if they are different, which makes $\mathsf{ED}(x,y)$ exactly the same as their Hamming distance.

To get around this, we need to ensure that $\mathsf{LCS}(x,y)$ is large. We demonstrate our ideas by first describing an $\Omega(n)$ lower bound for the deterministic two party communication complexity of $\mathsf{ED}(x,y)$, using a reduction from the equality function which is well known to have an $\Omega(n)$ communication complexity bound. Towards this, fix $\Sigma=[3n]\cup\{a\}$ where $a$ is a special symbol, and fix $y=1\circ 2\circ\cdots\circ 3n$. We divide $x$ into two parts $x=(x_{1},x_{2})$ such that $x_{1}$ is obtained from the string $(1,2,4,5,\cdots,3i-2,3i-1,\cdots,3n-2,3n-1)$ by replacing some symbols of the form $3j-1$ by $a$, while $x_{2}$ is obtained from the string $(2,3,5,6,\cdots,3i-1,3i,\cdots,3n-1,3n)$ by replacing some symbols of the form $3j-1$ by $a$. Note that the way we choose $(x_{1},x_{2})$ ensures that $\mathsf{LCS}(x,y)\geq 2n$ before replacing any symbol by $a$.

Intuitively, we want to argue that the best way to transform $x$ into $y$, is to delete a substring at the end of $x_{1}$ and a substring at the beginning of $x_{2}$, so that the resulted string becomes an increasing subsequence as long as possible. Then, we insert symbols into this string to make it match $y$ except for those $a$ symbols. Finally, we replace the $a$ symbols by substitutions. If this is true then we can finish the argument as follows. Let $T_{1},T_{2}\subset[n]$ be two subsets with size $t=\Omega(n)$, where for any $i\in\{1,2\}$, all symbols of the form $3j-1$ in $x_{i}$ with $j\in T_{i}$ are replaced by $a$. Now if $T_{1}=T_{2}$ then it doesn’t matter where we choose to delete the substrings in $x_{1}$ and $x_{2}$, the number of edit operations is always $3n-2+t$ by a direct calculation. On the other hand if $T_{1}\neq T_{2}$ and assume for simplicity that the smallest element they differ is an element in $T_{2}$, then there is a way to save one substitution, and the the number of edit operations becomes $3n-3+t$.

The key part is now proving our intuition. For this, we consider all possible $r\in[3n]$ such that $x_{1}$ is transformed into $y[1:r]$ and $x_{2}$ is transformed into $y[r+1:3n]$, and compute the two edit distances respectively. To analyze the edit distance, we first show by a greedy argument that without loss of generality, we can assume that we apply deletions first, followed by insertions, and substitutions at last. This reduces the edit distance problem to the following problem: for a fixed number of deletions and insertions, what is the best way to minimize the Hamming distance (or maximize the number of agreements of symbols at the same indices) in the end. Now we break the analysis of $\mathsf{ED}(x_{1},y[1:r])$ into two cases. Case 1 is where the number of deletions (say $d_{d}$) is large. In this case, the number of insertions (say $d_{i}$) must also be large, and we argue that the number of agreements is at most $\mathsf{LCS}(x_{1},y[1:r])+d_{i}$. Case 2 is where $d_{d}$ is small. In this case, $d_{i}$ must also be small. Now we crucially use the structure of $x_{1}$ and $y$, and argue that symbols in $x_{1}$ larger than $3d_{i}$ (or original index beyond $2d_{i}$) are guaranteed to be out of agreement. Thus the number of agreements is at most $\mathsf{LCS}(x_{1}[1:2d_{i}],y[1:r])+d_{i}$. In each case combining the bounds gives us a lower bound on the total number of operations. The situation for $x_{2}$ and $y[r+1:3n]$ is completely symmetric and this proves our intuition.

In the above construction, $x$ and $y$ have different lengths ($|x|=4n$ while $|y|=3n$). We can fix this by adding a long enough string $z$ with distinct symbols than those in $\{x,y\}$ to the end of both $x$ and $y$, and then add $n$ symbols of $a$ at the end of $z$ for $y$. We argue that the best way to do the transformation is to transform $x$ into $y$, and then insert $n$ symbols of $a$. To show this, we first argue that at least one symbol in $z$ must be kept, for otherwise the number of operations is already larger than the previous transformation. Then, using a greedy argument we show that the entire $z$ must be kept, and thus the natural transformation is the optimal.

To extend the bound to randomized algorithms, we modify the above construction and reduce from Set Disjointness ($\mathsf{DIS}$), which is known to have randomized communication complexity $\Omega(n)$. Given two strings $\alpha,\beta\in\{0,1\}^{n}$ representing the characteristic vectors of two sets $A,B\subseteq[n]$, $\mathsf{DIS}(\alpha,\beta)=0$ if and only if $A\cap B\neq\emptyset$, or equivalently, $\exists j\in[n],\alpha_{j}=\beta_{j}=1$. For the reduction, we first create two new strings $\alpha^{\prime},\beta^{\prime}\in\{0,1\}^{2n}$ which are “balanced” versions of $\alpha,\beta$. Formally, $\forall j\in[n],\alpha^{\prime}_{2j-1}=\alpha_{j}$ and $\alpha^{\prime}_{2j}=1-\alpha_{j}$. We create $\beta^{\prime}$ slightly differently, i.e., $\forall j\in[n],\beta^{\prime}_{2j-1}=1-\beta_{j}$ and $\beta^{\prime}_{2j}=\beta_{j}$. Now both $\alpha^{\prime}$ and $\beta^{\prime}$ have $n$ $1$’s, we can use them as the characteristic vectors of the two sets $T_{1},T_{2}$ in the previous construction. A similar argument now leads to the bound for randomized algorithms.

Our lower bounds for randomized algorithms computing $\mathsf{LCS}$ exactly are obtained by a similar and simpler reduction from $\mathsf{DIS}$: we still fix $y$ to be an increasing sequence of length $8n$ and divide $y$ evenly into $4n$ blocks of constant size. Now $x_{1}$ consists of the blocks with an odd index, while $x_{2}$ consists of the blocks with an even index. Thus $x$ is a permutation of $y$. Next, from $\alpha,\beta\in\{0,1\}^{n}$ we create $\alpha^{\prime},\beta^{\prime}\in\{0,1\}^{2n}$ in a slightly different way and use $\alpha^{\prime},\beta^{\prime}$ to modify the $2n$ blocks in $x_{1}$ and $x_{2}$ respectively. If a bit is $1$ then we arrange the corresponding block in the increasing order, otherwise we arrange the corresponding block in the decreasing order. A similar argument as before now gives the desired $\Omega(n)$ bound. We note that [SW07] has similar results for LIS by reducing from $\mathsf{DIS}$. However, our reduction and analysis are different from theirs. Thus we can handle $\mathsf{LCS}$, and even the harder case where $x$ is a permutation of $y$.

We now turn to $\mathsf{LCS}$ over a small alphabet. To illustrate our ideas, let’s first consider $\Sigma=\{0,1\}$ and choose $y=0^{n/2}1^{n/2}$. It is easy to see that $\mathsf{LCS}(x,y)=\mathsf{LNST}(x,n/2)$. We now represent each string $x\in\{0,1\}^{n}$ as follows: at any index $i\in[n]\cup\{0\}$, we record a pair $(p,q)$ where $p=\mathsf{min}(\text{the number of 0's in }x[1:i],n/2)$ and $q=\mathsf{min}(\text{the number of 1's in }x[i+1:n],n/2)$. Thus, if we read $x$ from left to right, then upon reading a $0$, $p$ may increase by $1$ and $q$ does not change; while upon reading a $1$, $p$ does not change and $q$ may decrease by $1$. Hence if we use the horizontal axis to stand for $p$ and the vertical axis to stand for $q$, then these points $(p,q)$ form a polygonal chain. We call $p+q$ the value at point $(p,q)$ and it is easy to see that $\mathsf{LCS}(x,y)$ must be the value of an endpoint of some chain segment.

Using the above representation, we now fix $\Sigma=\{0,1,2\}$ and choose $y=0^{n/3}1^{n/3}2^{n/3}$, so $\mathsf{LCS}(x,y)=\mathsf{LNST}(x,n/3)$. We let $x=(x_{1},x_{2})$ such that $x_{1}\in\{0,1\}^{n/2}$ and $x_{2}\in\{1,2\}^{n/2}$. Since any common subsequence between $x$ and $y$ must be of the form $0^{a}1^{b}2^{c}$ it suffices to consider common subsequence between $x_{1}$ and $0^{n/3}1^{n/3}$, and that between $x_{2}$ and $1^{n/3}2^{n/3}$, and combine them together. Towards that, we impose the following properties on $x_{1},x_{2}$: (1) The number of $0$’s, $1$’s, and $2$’s in each string is at most $n/3$; (2) In the polygonal chain representation of each string, the values of the endpoints strictly increase when the number of $1$’s increases; and (3) For any endpoint in $x_{1}$ where the number of $1$’s is some $r$, there is a corresponding endpoint in $x_{2}$ where the number of $1$’s is $n/3-r$, and the values of these two endpoints sum up to a fixed number $t=\Omega(n)$. Note that property (2) implies that $\mathsf{LCS}(x,y)$ must be the sum of the values of an endpoint in $x_{1}$ where the number of $1$’s is some $r$, and an endpoint in $x_{2}$ where the number of $1$’s is $n/3-r$, while property (3) implies that for any string $x_{1}$, there is a unique corresponding string $x_{2}$, and $\mathsf{LCS}(x,y)=t$ (regardless of the choice of $r$).

We show that under these properties, all possible strings $x=(x_{1},x_{2})$ form a set $S$ with $|S|=2^{\Omega(n)}$, and this set gives a fooling set for the two party communication problem of computing $\mathsf{LCS}(x,y)$. Indeed, for any $x=(x_{1},x_{2})\in S$, we have $\mathsf{LCS}(x,y)=t$. On the other hand, for any $(x_{1},x_{2})\neq(x^{\prime}_{1},x^{\prime}_{2})\in S$, the values must differ at some point for $x_{1}$ and $x^{\prime}_{1}$. Hence by switching, either $(x_{1},x^{\prime}_{2})$ or $(x^{\prime}_{1},x_{2})$ will have a $\mathsf{LCS}$ with $y$ that has length at least $t+1$. Standard arguments now imply an $\Omega(n)$ communication complexity lower bound. A more careful analysis shows that we can even replace the symbol $2$ by $0$, thus resulting in a binary alphabet.

The above argument can be easily modified to give a $\Omega(1/\varepsilon)$ bound for $1+\varepsilon$ approximation of $\mathsf{LCS}$ when $\varepsilon<1$, by taking the string length to be some $n^{\prime}=\Theta(1/\varepsilon)$. To get a better bound, we combine our technique with the technique in [EJ08] and consider the following direct sum problem: we create $r$ copies of strings $\{x^{i},i\in[r]\}$ and $\{y^{i},i\in[r]\}$ where each copy uses distinct alphabets with size $2$. Assume for $x^{i}$ and $y^{i}$ the alphabet is $\{a_{i},b_{i}\}$, now $x^{i}$ again consists of $r$ copies of $(x^{i}_{j1},x^{i}_{j2}),j\in[r]$, where each $x^{i}_{j\ell}\in\{a_{i},b_{i}\}^{n^{\prime}/2}$ for $\ell\in[2]$; while $y^{i}$ consists of $r$ copies $y^{i}_{j}=a_{i}^{n^{\prime}/3}b_{i}^{n^{\prime}/3}a_{i}^{n^{\prime}/3},j\in[r]$. The direct sum problem is to decide between the following two cases for some $t=\Omega(n^{\prime})$: (1) $\exists i$ such that there are $\Omega(r)$ copies $(x^{i}_{j1},x^{i}_{j2})$ in $x^{i}$ with $\mathsf{LCS}((x^{i}_{j1}\circ x^{i}_{j2}),y^{i}_{j})\geq t+1$, and (2) $\forall i$ and $\forall j$, $\mathsf{LCS}((x^{i}_{j1}\circ x^{i}_{j2}),y^{i}_{j})\leq t$. We do this by arranging the $x^{i}$’s row by row into an $r\times 2r$ matrix (each entry is a length $n^{\prime}/2$ string) and letting $x$ be the concatenation of the columns. We call these strings the contents of the matrix, and let $y$ be the concatenation of the $y^{i}$’s. Now intuitively, case (1) and case (2) correspond to deciding whether $\mathsf{LCS}(x,y)\geq 2rt+\Omega(r)$ or $\mathsf{LCS}(x,y)\leq 2rt$, which implies a $1+\Omega(1/t)=1+\varepsilon$ approximation. The lower bound follows by analyzing the $2r$-party communication complexity of this problem, where each party holds a column of the matrix.

However, unlike the constructions in [GG10, EJ08] which are relatively easy to analyze because all symbols in $x$ (respectively $y$) are distinct, the repeated symbols in our construction make the analysis of $\mathsf{LCS}$ much more complicated (we can also use distinct symbols but that will only give us a bound of $\frac{\sqrt{|\Sigma|}}{\varepsilon}$ instead of $\frac{|\Sigma|}{\varepsilon}$). To ensure that the $\mathsf{LCS}$ is to match each $(x^{i}_{j1},x^{i}_{j2})$ to the corresponding $y^{i}_{j}$, we use another $r$ symbols $\{c_{i},i\in[r]\}$ and add buffers of large size (e.g., size $n^{\prime}$) between adjacent copies of $(x^{i}_{j1},x^{i}_{j2})$. We do the same thing for $y^{i}_{j}$ correspondingly. Moreover, it turns out we need to arrange the buffers carefully to avoid unwanted issues: in each row $x^{i}$, between each copy of $(x^{i}_{j1},x^{i}_{j2})$ we use a buffer of new symbol. Thus the buffers added to each row $x^{i}$ are $c_{1}^{n^{\prime}},c_{2}^{n^{\prime}},\cdots,c_{r}^{n^{\prime}}$ sequentially and this is the same for every row. That is, in each row the contents use the same alphabet $\{a_{i},b_{i}\}$ but the buffers use different alphabets $\{c_{i},i\in[r]\}$. Now we have a $r\times 3r$ matrix and we again let $x$ be the concatenation of the columns while let $y$ be the concatenation of the $y^{i}$’s. Note that we are using an alphabet of size $|\Sigma|=3r$. We use a careful analysis to argue that case (1) and case (2) now correspond to deciding whether $\mathsf{LCS}(x,y)\geq 2rn^{\prime}+rt+\Omega(r)$ or $\mathsf{LCS}(x,y)\leq 2rn^{\prime}+rt$, which implies a $1+\varepsilon$ approximation. The lower bound follows by analyzing the $3r$-party communication complexity of this problem, and we show a lower bound of $\Omega(r/\varepsilon)=\Omega(|\Sigma|/\varepsilon)$ by generalizing our previous fooling set construction to the multi-party case, where we use a good error correcting code to create the $\Omega(r)$ gap.

The above technique works for $\varepsilon<1$. For the case of $\varepsilon\geq 1$ our bound for $\mathsf{LCS}$ can be derived directly from our bound for $\mathsf{LIS}$, which we describe next.

Our $\Omega(|\Sigma|)$ lower bound over small alphabet is achieved by modifying the construction in [EJ08] and providing a better analysis. Similar as before, we consider a matrix $B\in\{0,1\}^{\frac{r}{c}\times r}$ where $c$ is a large constant and $r=|\Sigma|$. We now consider the $r$-party communication problem where each party holds one column of $B$, and the problem is to decide between the following two cases for a large enough constant $l$: (1) for each row in $B$, there are at least $l$ $0$’s between any two $1$’s, and (2) there exists a row in $B$ which has more than $\alpha r$ $1$’s, where $\alpha\in(1/2,1)$ is a constant. We can use a similar argument as in [EJ08] to show that the total communication complexity of this problem is $\Omega(r^{2})$ and hence at least one party needs $\Omega(r)$. The difference is that [EJ08] sets $l=1$ while we need to pick $l$ to be a larger constant to handle the case $\varepsilon\geq 1$. For this we use the Lovász Local Lemma with a probabilistic argument to show the existence of a large fooling set. To reduce to $\mathsf{LIS}$, we define another matrix $\tilde{B}$ such that $\tilde{B}_{i,j}=(i-1)\frac{r}{c}+j$ if $B_{i,j}=1$ and $\tilde{B}_{i,j}=0$ otherwise. Now let $x$ be the concatenation of all columns of $\tilde{B}$. We show that case (2) implies $\mathsf{LIS}(x)\geq\alpha r$ and case (1) implies $\mathsf{LIS}(x)\leq(1/c+1/l)r$. This implies a $1+\varepsilon$ approximation for any constant $\varepsilon>0$ by setting $c$ and $l$ appropriately.

The construction is slightly different for $\mathsf{LNS}$. This is because if we keep the $0$’s in $\tilde{B}$, they will already form a very long non-decreasing subsequence and we will not get any gap. Thus, we now let the matrix $B$ have size $r\times cr$ where $c$ can be any constant. We replace all $0$’s in column $i$ with a symbol $b_{i}$ for $i\in[cr]$, such that $b_{1}>b_{2}>\cdots>b_{cr}$. Similarly we replace all $1$’s in row $j$ with a symbol $a_{j}$ for $j\in[r]$, such that $a_{1}<a_{2}<\cdots<a_{r}$. Also, we let $a_{1}>b_{1}$. We can show that the two cases now correspond to $\mathsf{LNS}(x)>\alpha cr$ and $\mathsf{LNS}(x)\leq(2+c/l)r$.

We further prove an $\Omega(|\Sigma|\log(1/\varepsilon))$ lower bound for $1+\varepsilon$ approximation of $\mathsf{LNS}$ when $\varepsilon<1$. This is similar to our previous construction for $\mathsf{LCS}$, except we don’t need buffers here, and we only need to record the number of some symbols. More specifically, let $l=\Theta(1/\varepsilon)$ and $S$ be the set of all strings $x=(x_{1},x_{2})$ over alphabet $\{a,b\}$ with length $2l$ such that $x_{1}=a^{\frac{3}{4}l+t}b^{\frac{1}{4}l-t}$ and $x_{2}=a^{\frac{3}{4}l-t}b^{\frac{1}{4}l+t}$ for any $t\in[\frac{l}{4}]$. Thus $S$ has size $\frac{l}{4}=\Omega(1/\varepsilon)$ and $\forall x\in S$, the number of $a$’s in $x$ is exactly $\frac{3}{2}l$. Further, for any $(x_{1},x_{2})\neq(x^{\prime}_{1},x^{\prime}_{2})\in S$, either $(x_{1},x^{\prime}_{2})$ or $(x^{\prime}_{1},x_{2})$ has more than $\frac{3}{2}l$ $a$’s. We now consider the $r\times 2r$ matrix where each row $i$ consists of $\{(x^{i}_{j1},x^{i}_{j2}),j\in[r]\}$ such that each $x^{i}_{j{\ell}}$ has length $l$ for $\ell\in[2]$, and for the same row $i$ all $\{(x^{i}_{j1},x^{i}_{j2})\}$ use the same alphabet $\{a_{i},b_{i}\}$ while for different rows the alphabets are disjoint. To make sure the $\mathsf{LNS}$ of the concatenation of the columns is roughly the sum of the number of $a_{i}$’s, we require that $b_{r}<b_{r-1}<\cdots<b_{1}<a_{1}<a_{2}<\cdots<a_{r}$. Now we analyze the $2r$ party communication problem of deciding whether the concatenation of the columns has $\mathsf{LNS}\geq crl+\Omega(r)$ or $\mathsf{LNS}\leq crl$ for some constant $c$, which implies a $1+\varepsilon$ approximation. The lower bound is again achieved by generalizing the set $S$ to a fooling set for the $2r$ party communication problem using an error correcting code based approach.

In Theorem 4, we give three lower bounds for $\mathsf{LNST}$. The first two lower bounds are adapted from our lower bounds for $\mathsf{LIS}$ and $\mathsf{LNS}$, while the last lower bound is adapted from our lower bound for $\mathsf{LCS}$ by ensuring all symbols in different rows or columns of the matrix there are different.

Our algorithm for edit distance builds on and improves the algorithm in [FHRS20, CJLZ20]. The key idea of that algorithm is to use triangle inequality. Given a constant $\delta$, the algorithm first divides $x$ evenly into $b=n^{\delta}$ blocks. Then for each block $x^{i}$ of $x$, the algorithm recursively finds an $\alpha$-approximation of the closest substring to $x^{i}$ in $y$. That is, the algorithm finds a substring $y[l_{i}:r_{i}]$ and a value $d_{i}$ such that for any substring $y[l:r]$ of $y$, $\mathsf{ED}(x^{i},y{[l_{i}:r_{i}]})\leq d_{i}\leq\alpha\mathsf{ED}(x^{i},y{[l:r]})$. Let $\tilde{y}$ be the concatenation of $y[l_{i}:r_{i}]$ from $i=1$ to $b$. Then using triangle inequality, [FHRS20] showed that $\mathsf{ED}(y,\tilde{y})+\sum_{i=1}^{b}d_{i}$ is a $2\alpha+1$ approximation of $\mathsf{ED}(x,y)$. The $\tilde{O}(n^{\delta})$ space is achieved by recursively applying this idea, which results in a $O(2^{1/\delta})$ approximation.

To further reduce the space complexity, our key observation is that, instead of dividing $x$ into blocks of equal length, we can divide it according to the positions of the edit operations that transform $x$ to $y$. More specifically, assume we are given a value $k$ with $\mathsf{ED}(x,y)\leq k\leq c\mathsf{ED}(x,y)$ for some constant $c$, we show how to design an approximation algorithm using space $\tilde{O}(\sqrt{k})$. Towards this, we can divide $x$ and $y$ each into $\sqrt{k}$ blocks $x=x^{1}\circ\cdots\circ x^{\sqrt{k}}$ and $y=y^{1}\circ\cdots\circ y^{\sqrt{k}}$ such that $\mathsf{ED}(x^{i},y^{i})\leq\frac{\mathsf{ED}(x,y)}{\sqrt{k}}\leq\sqrt{k}$ for any $i\in[\sqrt{k}]$. However, such a partition of $x$ and $y$ is not known to us. Instead, we start from the first position of $x$ and find the largest index $l_{1}$ such that $\mathsf{ED}(x[1:l_{1}],y[p_{1},q_{1}])\leq\sqrt{k}$ for some substring $y[p_{1}:q_{1}]$ of $y$. To do this, we start with $l=\sqrt{k}$ and try all substrings of $y$ with length in $[l-\sqrt{k},l+\sqrt{k}]$. If there is some substring of $y$ within edit distance $\sqrt{k}$ to $x[1:l]$, we set $l_{1}=l$ and store all the edit operations that transform $y[p_{1}:q_{1}]$ to $x[1:l_{1}]$ where $y[p_{1}:q_{1}]$ is the substring closest to $x[1:l_{1}]$ in edit distance. We continue doing this with $l=l+1$ until we can not find a substring of $y$ within edit distance $\sqrt{k}$ to $x[1:l]$.

One problem here is that $l$ can be much larger than $\sqrt{k}$ and we cannot store $x[1:l]$ with $\tilde{O}(\sqrt{k})$ space. However, since we have stored some substring $y[p_{1}:q_{1}]$ (we only need to store the two indices $p_{1},q_{1}$) and the at most $\sqrt{k}$ edit operations that transform $y[p_{1}:q_{1}]$ to $x[1:l-1]$, we can still query every bit of $x[1:l]$ using $\tilde{O}(\sqrt{k})$ space.