Online and Offline Algorithms for Counting Distinct Closed Factors via Sliding Suffix Trees

Let $\Sigma$ be an ordered alphabet. An element of $\Sigma$ is called a character. An element of $\Sigma^{\star}$ is called a string. The empty string, denoted by $\varepsilon$, is the string of length $0$. For a string $T\in\Sigma^{\star}$, the length of $T$ is denoted by $|T|$. If $T=xyz$ for some strings $x,y,z\in\Sigma^{\star}$, we call $x$, $y$, and $z$ a prefix, a factor, and a suffix of $T$, respectively. A string $b\neq T$ is said to be a border of $T$ if $b$ is both a prefix of $T$ and a suffix of $T$. We denote by $\mathit{bord}(T)$ the longest border of $T$. Note that the longest border always exists since $\varepsilon$ is a border of any string. For each $i$ with $1\leq i\leq|T|$, we denote by $T[i]$ the $i$th character of $T$. For each $i,j$ with $1\leq i\leq j\leq|T|$, we denote by $T[i..j]$ the factor of $T$ that starts at position $i$ and ends at position $j$. For convenience, we define $T[i..j]=\varepsilon$ for any $i>j$. For a strings $T$ and $S$, the set $\mathit{occ}_{T}(S)=\{i\mid T[i..i+|S|-1]=S\}$ of integers is said to be the occurrences of $S$ in $T$. If $\mathit{occ}_{T}(S)\neq\emptyset$, we say that $S$ occurs in $T$ as a factor. A string $T$ is said to be closed if there is a border of $T$ that occurs exactly twice in $T$. If $|\mathit{occ}_{T}(S)|=1$, we say that $S$ is a unique factor of $T$. Also, if $|\mathit{occ}_{T}(S)|\geq 2$, we say that $S$ is a repeating factor of $T$. We denote by $\mathit{lrs}(T)$ the longest repeating suffix of string $T$. Further we denote $\mathit{lrs}^{2}(T)=\mathit{lrs}(\mathit{lrs}(T))$.

In what follows, we fix a non-empty string $T$ of arbitrary length $n$ over an alphabet $\Sigma$ of size $\sigma$. This paper assumes the standard word RAM model with word size $\Omega(\log n)$.

The most important tool of this paper is a suffix tree of string $T$ [38]. A suffix tree of $T$, denoted by $\mathsf{STree}(T)$, is a compact trie for the set of suffixes of $T$. Below, we summarize some known properties of suffix trees and define related notations:

• •

  Each edge of $\mathsf{STree}(T)$ is labeled with a factor of $T$ of length one or more.

• •

  Each internal node $u$, including the root, has at least two children, and the first characters of the labels of outgoing edges from $u$ are mutually distinct (unless $T$ is a unary string).

• •

  For each node $v$ of $\mathsf{STree}(T)$, we denote by $\mathsf{str}_{T}(v)$ the string spelled out from the root to $v$. We also define $\mathsf{strlen}_{T}(u)=|\mathsf{str}_{T}(u)|$ for a node $u$.

• •

  There is a one-to-one correspondence between leaves of $\mathsf{STree}(T)$ and unique suffixes of $T$. More precisely, for each leaf $\ell$, the string $\mathsf{str}_{T}(\ell)$ equals some unique suffix of $T$, and vice versa. Note that repeating suffixes of $T$ are not always represented by a node in $\mathsf{STree}(T)$.

Throughout this paper, we assume for convenience that the last character of $T$ is unique. For each $1\leq i\leq n$, we denote by $\mathsf{leaf}_{T}(i)$ the leaf $\ell$ of $\mathsf{STree}(T)$ such that $\mathsf{str}_{T}(\ell)=T[i..n]$.

It is known that $\mathsf{STree}(T)$ for given string $T$ can be constructed in $O(n)$ time [16] if $\Sigma$ is linearly sortable²²2A typical example of linearly sortable alphabets is an integer alphabet $\{1,2,\ldots,n^{c}\}$ for some constant $c$. Any $n$ characters from the alphabet can be sorted in $O(n)$ time using radix sort., i.e., any $n$ characters from $\Sigma$ can be sorted in $O(n)$ time. Additionally, when the input string $T$ is given in an online manner, $\mathsf{STree}(T)$ can be constructed in $O(n\log\sigma)$ time using Ukkonen’s algorithm [36]. In both algorithms, the resulting suffix trees are edge-sorted, meaning that the first characters of the labels of outgoing edges from each node are (lexicographically) sorted. Thus we assume that all suffix trees in this paper are edge-sorted.

In other words, for every $j=1,2,\ldots,n-1$, we can update $\mathsf{STree}(T[1..j])$ to $\mathsf{STree}(T[1..j+1])$ in amortized $O(\log\sigma)$ time. Ukkonen’s algorithm maintains the active point in the suffix tree, which is the locus of the longest repeating suffix of the string. To maintain the active point efficiently, Ukkonen’s algorithm uses auxiliary data structures called suffix links: Each internal node $u$ of the suffix tree has a suffix link that points to the node $v$, such that $\mathsf{str}_{T}(v)$ is the suffix of $\mathsf{str}_{T}(u)$ of length $|\mathsf{str}_{T}(u)|-1$. See Fig. 1 for examples of a suffix tree and related notations.

Furthermore, based on Ukkonen’s algorithm, we can maintain suffix trees for a sliding window over $T$ in a total of $O(n\log\sigma)$ time, using space proportional to the window size:

In other words, for every incremental pair of indices $i,j$ with $i\leq j$, we can update $\mathsf{STree}(T[i..j])$ to either $\mathsf{STree}(T[i..j+1])$ or $\mathsf{STree}(T[i+1..j])$ in amortized $O(\log\sigma)$ time. Later, we use the above algorithmic results as black boxes.

For a node-weighted tree $\mathcal{T}$, where the weight of each non-root node is greater than the weight of its parent, a weighted ancestor query (WAQ) is defined as follows: given a node $u$ of the tree and an integer $t$, the query returns the farthest (highest) ancestor of $u$ whose weight is at least $t$. We denote by $\mathsf{WAQ}_{\mathcal{T}}(u,t)$ the returned node for weighted ancestor query $(u,t)$ on tree $\mathcal{T}$. The subscript $\mathcal{T}$ will be omitted when clear from the context. In general, a WAQ on a tree of size $N$ can be answered in $O(\log\log N)$ time, which is worst-case optimal with linear space. However, if the input is the suffix tree of some string of length $n$, and the weight function is $\mathsf{strlen}_{T}$, as defined in the previous subsection, any WAQ can be answered in $O(1)$ time after $O(n)$-time preprocessing [10].

First we consider the changes in the number of distinct closed factors when a character is appended. Let $\mathcal{C}(T)$ be the set of distinct closed factors occurring in $T$. Let $d_{j}=|\mathcal{C}(T[1..j])|-|\mathcal{C}(T[1..j-1])|$ for $2\leq j\leq n$. For convenience let $d_{1}=1$.

Based on this observation, we count the distinct borders of closed factors instead of the closed factors themselves. We show the following:

We note that the upper bound $d_{j}\leq|\mathit{lrs}(T[1..j])|-|\mathit{lrs}^{2}(T[1..j]))|$ of $d_{j}$ is already claimed in [31], however, the equality was not shown. By definition of $d_{j}$ and Lemma 1, the next corollary immediately follows:

By Theorem 1, we can compute $\sum_{j\in J}|\mathit{lrs}(T[1..j])|$ in $O(n\log\sigma)$ time online. Thus, by Corollary 1, our remaining task is to compute $\sum_{j\in J}|\mathit{lrs}^{2}(T[1..j])|$ online, equivalently, to compute the sequence $\mathcal{Z}=(|z_{1}|,|z_{2}|$ $\ldots,|z_{n}|)$ where $z_{j}=\mathit{lrs}^{2}(T[1..j])$ for each $j$. To compute the sequence, we use sliding suffix trees of Theorem 2. It is known that the starting position of $\mathit{lrs}(T[1..j])$ is not smaller than that of $\mathit{lrs}(T[1..j-1])$. Namely, starting from $t_{1}=\mathit{lrs}(T[1..1])=\varepsilon$, the sequence $\langle t_{1},t_{2},\ldots,t_{n}\rangle$ of factors of $T$ can be obtained by $O(n)$ sliding-operations on $T$ that consists of (1) appending a character to the right or (2) deleting the leftmost character from the factor, where $t_{j}=\mathit{lrs}(T[1..j])$ for each $j$. Thus, by Theorem 2, we can obtain every $|z_{i}|\in\mathcal{Z}$ in amortized $O(\log\sigma)$ time by appropriately applying sliding-operations to reach the windows $t_{j}$ for all $j$ with $j\in J$ in order. Finally, we obtain the following: