A Complexity Theory for Hard Enumeration Problems

While decision problems often ask for the existence of a solution to some problem instance, enumeration problems aim at outputting all solutions. In many domains, enumeration problems are thus the most natural kind of problems. Just take the database area (usually the user is interested in all answer tuples and not just in a yes/no answer) or diagnosis (where the user wants to retrieve possible explanations, and not only whether one exists) as two examples. Nevertheless, the complexity of enumeration problems is far less studied than the complexity of decision problems.

It should be noted that even simple enumeration problems may produce big output. To capture the intuition of easy to enumerate problems – despite a possibly exponential number of output values – various notions of tractable enumeration classes have been proposed in [1]. The class DelayP (“polynomial delay”) contains all enumeration problems where, for given instance $x$, (1) the time to compute the first solution, (2) the time between outputting any two consecutive solutions, and (3) the time to detect that no further solution exists, are all polynomially bounded in the size of $x$. The class IncP (“incremental polynomial time”) contains those enumeration problems where, for given instance $x$, the time to compute the next solution and for detecting that no further solution exists is polynomially bounded in the size of both $x$ and of the already computed solutions. Obviously, the relationship $\text{\sf DelayP}\subseteq\text{\sf IncP}$ holds. In [2], the proper inclusion $\text{\sf DelayP}\subsetneq\text{\sf IncP}$ is mentioned. For these tractable enumeration classes, a variety of membership results exist, a few examples are given in [3, 4, 5, 6, 7, 8]

There has also been work on intractable enumeration problems. Intractability of enumeration is typically proved by showing intractability of a related decision problem rather than directly proving lower bounds by relating one enumeration problem to the other. Tools for a more fine-grained analysis of intractable enumeration problems are missing to date. For instance, up to now we are not able to make a differentiated analysis of the complexity of the following typical enumeration problems:

$\Pi_{k}\mathrm{SAT}^{e}$ / $\Sigma_{k}\mathrm{SAT}^{e}$ Instance: $\psi=\forall y_{1}\exists y_{2}\ldots Q_{k}y_{k}\phi(\vec{x},\vec{y})$ / $\psi=\exists y_{1}\forall y_{2}\ldots Q_{k}y_{k}\phi(\vec{x},\vec{y})$ Output: All assignments for $\vec{x}$ such that $\psi$ is true.

This is in sharp contrast to decision problems, where the polynomial hierarchy is crucial for a detailed complexity analysis. As a matter of fact, it makes a big difference, if an NP-hard problem is in NP or not. Indeed, NP-complete problems have an efficient transformation into SAT and can therefore be solved by making use of powerful SAT-solvers. Similarly, problems in ${\Sigma}_{2}^{P}$ can be solved by using ASP-solvers. Finally, also for problems on higher levels of the polynomial hierarchy, the number of quantifier alternations in the QBF-encoding matters when using QBF-solvers. For counting problems, an analogue of the polynomial hierarchy has been defined in form of the $\#\cdot{\cal C}$–classes with ${\cal C}\in\{\text{\sf P},\text{\sf coNP},{\Pi}_{2}^{P},\dots\}$ [9, 10]. For enumeration problems, no such analogue has been studied.

• •

  Enumeration complexity classes. In Section 3, we introduce a hierarchy of complexity classes of intractable enumeration via oracles and prove that it is strict unless the polynomial hierarchy collapses.

• •

  Declarative-style reductions. In Section 4, we introduce a declarative-style notion of reductions. While they enjoy some desirable properties, we do not succeed in exhibiting complete problems under this type of reductions.

• •

  Procedural-style reductions. In Section 5, we introduce a procedural-style notion of reductions, show that they have the desirable properties any reduction should possess, and show that the enumeration problems associated with the typical quantified Boolean satisfiability problems are complete in our hierarchy of enumeration problems.

• •

  Completeness results for natural problems. Starting with these completeness results, we establish a chain of reductions among several natural enumeration problems from areas such as generalized satisfiability, circumscription, model-based diagnosis, abduction, and repairs of inconsistent databases in Section 6, thus proving completeness of these problems in classes of our hierarchy.

This work is an extension of the conference article [11].

In the following, $\Sigma$ denotes a finite alphabet and $R$ denotes a polynomially bounded, binary relation $R\subseteq\Sigma^{*}\times\Sigma^{*}$, i.e., there is a polynomial $p$ such that for all $(x,y)\in R$, $|y|\leq p(|x|)$ holds. For every string $x$, $R(x)=\{y\in\Sigma^{*}\mid(x,y)\in R\}$. A string $y\in R(x)$ is called a solution for $x$. With such a relation $R$, we can associate several natural computational problems:

$\text{\sc Exist}{\_}R$ Instance: $x\in\Sigma^{*}$ Question: Is $R(x)\neq\emptyset$?

$\text{\sc Exist-AnotherSol}{\_}R$ Instance: $x\in\Sigma^{*},Y\subseteq R(x)$ Output: Is $(R(x)\setminus Y)\neq\emptyset$?

$\text{\sc AnotherSol}{\_}R$ Instance: $x\in\Sigma^{*},Y\subseteq R(x)$ Output: $y\in R(x)\!\setminus\!Y$ or declare that no such $y$ exists.

$\text{\sc Check}{\_}R$ Instance: $(x,y)\in\Sigma^{*}\times\Sigma^{*}$ Question: Is $(x,y)\in R$?

$\text{\sc ExtSol}{\_}R$ Instance: $(x,y)\in\Sigma^{*}\times\Sigma^{*}$ Question: Is there some $y^{\prime}\in\Sigma^{*}$ such that $(x,yy^{\prime})\in R$?

$\text{\sc Count}{\_}R$ Instance: $x\in\Sigma^{*}$ Output: $\bigl{|}\{y\in\Sigma^{*}\mid(x,y)\in R\}\bigr{|}$.

$\text{\sc Enum}{\_}R$ Instance: $x\in\Sigma^{*}$ Output: $R(x)=\{y\in\Sigma^{*}\mid(x,y)\in R\}$.

We assume the reader to be familiar with the polynomial hierarchy – the complexity classes P, NP, coNP and, more generally, ${\Delta}_{k}^{P}$, ${\Sigma}_{k}^{P}$, and ${\Pi}_{k}^{P}$ for $k\in\{0,1,\dots\}$.

As we usually deal with exponential runtime of algorithms in the course of an enumeration process, we will also make use of the weak EXP hierarchy (see [12]) in the following. It is an exponential time analogue of the polynomial hierarchy defined as follows:

A hierarchy of counting complexity classes similar to the polynomial hierarchy was defined [9, 10]. Let ${\cal C}$ be a complexity class of decision problems. Then $\#\cdot{\cal C}$ denotes the class of all counting problems ${\cal P}$, such that there exists a relation $R$ with $\text{\sc Check}{\_}R\in{\cal C}$ and ${\cal P}=\text{\sc Count}{\_}R$. For $\mathsf{\#P}$ – the best known counting complexity class – we have $\mathsf{\#P}=\#\cdot\mathsf{P}$. Moreover, the inclusions $\mathsf{\#P}\subseteq\#\cdot\mathsf{NP}\subseteq\#\cdot\mathsf{coNP}\subseteq\#\cdot\Sigma_{2}^{P}\subseteq\#\cdot\Pi_{2}^{P}\subseteq\dots$ hold.

A complexity class ${\mathcal{C}}$ is closed under a reduction $\leq_{r}$ if, for any two binary relations $R_{1}$ and $R_{2}$ we have that $R_{2}\in{\mathcal{C}}$ and $R_{1}\leq_{r}R_{2}$ imply $R_{1}\in{\mathcal{C}}$. Furthermore, a reduction $\leq_{r}$ is transitive if for any three binary relations $R_{1},R_{2},R_{3}$, it is the case that $R_{1}\leq_{r}R_{2}$ and $R_{2}\leq_{r}R_{3}$ implies $R_{1}\leq_{r}R_{3}$.

In Section 1, we have already recalled two important tractable enumeration complexity classes, DelayP and IncP from [1]. Note that in [2, 13], these classes are defined slightly differently by restricting DelayP and IncP to those problems $\text{\sc Enum}{\_}R$ such that the corresponding problem $\text{\sc Check}{\_}R$ is in P — in addition to the constraints on the allowed delays. We adhere to the definition of tractable enumeration classes from [1].

In contrast to counting complexity, defining a hierarchy of enumeration problems via the $\text{\sc Check}{\_}R$ problem of binary relations $R$ does not seem appropriate. Note that while the same counting problem ${\mathcal{P}}$ can be defined by different relations, i.e., ${\cal P}=\text{\sc Count}{\_}R_{1}$ and ${\cal P}=\text{\sc Count}{\_}R_{2}$ for $R_{1}\neq R_{2}$, the relation associated with an enumeration problem is fixed.

Thus, we need an alternative approach for defining meaningful enumeration complexity classes. To this end, we first fix our computation model: Observe that there is a subtle difference between how Turing Machines and random access machines (RAMs) can access data in their “memory”. Due to the linear nature of the tapes of Turing Machines, accessing an exponential size data structure requires exponential time, even if just a small portion of its data is actually read. This is usually not a problem when studying decision or search problems, since writing an exponentially sized data structure onto the tape already requires exponential time, and as a result the additional exponential time to read the data structure has no more effect on the overall runtime.

This situation, however, changes for enumeration problems and polynomial delay, since it is now possible that, while computing an exponential number of solutions, an exponential size data structure accumulates. Thus computing another solution with only polynomial delay is not possible with Turing Machines if this requires some information from such a data structure. However, by maintaining suitable index structures, it might be possible to gather the necessary information in polynomial time from such a data structure on a random access machine (RAM) that allows to access its memory directly and not first needs to move the read/write head over the correct tape position. In fact, due to this property that allows to access (polynomially sized) parts of exponential size data in polynomial time, it is common to use the RAM model as a computational model for the study of enumeration problems (cf. [2]). We restrict ourselves here to polynomially bounded RAMs, i.e., throughout the computation of such a machine, the size of the content of each register is polynomially bounded in the size of the input.

For enumeration, we will also make use of RAMs with an $\mathsf{output}$-instruction, as defined in [2]. This model can be extended further by introducing decision oracles. The input to the oracle is stored in special registers and the oracle takes consecutive non-empty registers as input. Moreover, following [14], we use a computational model that does not delete the input of an oracle call once such a call is made. For a detailed definition, refer to [2]. It is important to note that due to the exponential runtime of an enumeration algorithm and the fact that the input to an oracle is not deleted when the oracle is executed, the input to an oracle call may eventually become exponential as well. Clearly, this can only happen if exponentially many consecutive special registers are non-empty, since we assume also each special register to be polynomially bounded.

Using this we define a collection of enumeration complexity classes via oracles:

In fact, given that the classes ${\Delta}_{k}^{P}$ describe problems that can be solved in polynomial time by a deterministic machine with a ${\Sigma}_{k-1}^{P}$-oracle, it is not very surprising that for enumeration algorithms, ${\Delta}_{k}^{P}$-oracles can be simulated by ${\Sigma}_{k-1}^{P}$-oracles, as long as one accounts for the potentially accumulating input to the ${\Delta}_{k}^{P}$-oracles.

This property is not restricted to the case shown in Theorem 1, but applies to all ${\Delta}_{k}^{P}$-oracles. As a result, in the remainder of this work, we omit any result regarding the classes $\text{\sf DelayP}_{p}^{{\Delta}_{k}^{P}},\text{\sf DelayP}^{{\Delta}_{k}^{P}}$ or $\text{\sf IncP}^{{\Delta}_{k}^{P}}$, as they are given implicitly by a result on $\text{\sf DelayP}_{p}^{{\Sigma}_{k}^{P}}$ or $\text{\sf IncP}^{{\Sigma}_{k}^{P}}$ respectively. Indeed, Theorem 1 shows the equality of $\text{\sf DelayP}^{{\Delta}_{k}^{P}}$ and $\text{\sf IncP}^{{\Sigma}_{k-1}^{P}}$. Concerning the remaining classes, observe that $\text{\sf DelayP}_{p}^{\text{\sf P}^{{\mathcal{C}}}}=\text{\sf DelayP}_{p}^{{\mathcal{C}}}$ for any decision complexity class ${\mathcal{C}}$; in particular we obtain $\text{\sf DelayP}_{p}^{{\Delta}_{k}^{P}}=\text{\sf DelayP}_{p}^{{\Sigma}_{k-1}^{P}}$. A similar results holds for the $\text{\sf IncP}^{{\mathcal{C}}}$ classes even without the bound on the input size to the decision oracle. This is due to the fact that the size of any input $\alpha$ to the decision oracle is at most polynomial in the combined size of the instance and number of outputs. Thus an incremental delay gives us enough time to make $\mathsf{poly}(|\alpha|)$ many oracles calls, meaning that $\text{\sf IncP}^{{\Delta}_{k}^{P}}=\text{\sf IncP}^{{\Sigma}_{k-1}^{P}}$.

As far as we know, only a few kinds of reductions between enumeration problems have been investigated so far. One such reduction is implicitly described in [15]. It establishes a bijection between sets of solutions. Different approaches introduced in [16] and in [17] relax this condition and allow for non-bijective reduction relations. We go further in that direction in proposing a declarative style reduction relaxing the isomorphism requirement while closing the relevant enumeration classes.

Intuitively, $\tau$ establishes a relationship between instances $x$, solutions $y\in{R_{2}}(\sigma(x))$ and solutions $z\in{R_{1}}(x)$. We can thus use $\tau$ to design an enumeration algorithm for ${R_{1}}(x)$ via an enumeration algorithm for ${R_{2}}(\sigma(x))$. The conditions imposed on $\tau$ have the following meaning: By condition 1, the solutions $z\in{R_{1}}(x)$ can be computed by iterating through the solutions $y\in{R_{2}}(\sigma(x))$ and computing $\tau(x,y,-)\subseteq{R_{1}}(x)$. Conditions 2 and 3 make sure that the delay of enumerating ${R_{1}}(x)$ only differs by a polynomial from the delay of enumerating ${R_{2}}(\sigma(x))$: Condition 2 ensures that, for every $y$, the set $\tau(x,y,-)$ can be enumerated with polynomial delay and that we do not encounter more than a polynomial number of “useless” $y$ (i.e., a solution $y\in{R_{2}}(\sigma(x))$ which is associated with no solution $z\in{R_{1}}(x)$).

In principle, we may thus get duplicates $z$ associated with different values of $y$. However, Condition 3 ensures that each $z$ can be associated with at most polynomially many values $y$. Using a priority queue storing all $z$ that are output, we can avoid duplicates, cf. the proof of Proposition 8 or [2]. Figure 2 illustrates the idea of $\tau$, and a concrete example of an e-reduction is provided next.

The $e$-reductions have two desirable and important properties, as stated next.

Note that, as an additional benefit, $\text{\sc Enum}{\_}R_{1}\leq_{e}\text{\sc Enum}{\_}R_{2}$ does not imply $\text{\sc Exist}{\_}R_{1}\leq_{m}\text{\sc Exist}{\_}R_{2}$. An example for this behavior can be found in Theorem 14 below.

Thus, besides being a natural extension of existing reductions, Proposition 8 also shows that $e$-reductions have important desirable properties. As a result, they are very well suited for reasoning about enumeration algorithms and complexities. For example, they allow to compare different enumeration problems, thus supporting to make statements on one problem “relative” to some other problem, and can be used to derive enumeration algorithms – hence complexity upper bounds.

In fact, they have already been implicitly used previously for these purposes. For example, an important result from [18] compares the complexity of two well-known problems by implicitly using an $e$-reduction:

Note that $\text{\sc Dom-Enum}\leq_{e}\text{\sc Trans-Enum}$ holds as well. The original question studied in [18] was whether any of these problems can be enumerated in a time that is polynomially in both, the size of the input and the size of the output. For the two problems Dom-Enum and Trans-Enum, this is a longstanding open problem. The conclusion in [18] was that if such an enumeration algorithm exists for one of the problems, then this is also possible for the other one. It can be shown that the existence of the $e$-reductions between Dom-Enum and Trans-Enum also induces the same result. Moreover, they are also equivalent for all of the classes that are closed under $e$-reductions as given in Proposition 8, i.e., if one of the two problems is in any of these classes, then the other problem is as well.

Another example that implicitly uses $e$-reductions is [19]. In later sections of this paper, we will use them to retrieve reductions among enumeration problems as well. And even though we do not show any completeness results under $e$-reduction, we will actually use them to show hardness under alternative reductions that will be introduced in the next section.

Although Turing reductions are too strong to show completeness results for classes in the polynomial hierarchy, because the classes ${\Sigma}_{k}^{P}$ and ${\Delta}_{k+1}^{P}$ have the same closure, Turing style reductions turn out to be meaningful in our case. In this section we introduce two types of reductions that are motivated by Turing reductions. Both of them are able to reduce between enumeration problems for which the $e$-reductions seem to be too weak.

Towards this goal, we first have to define the concept of RAMs with an oracle for enumeration problems (such oracles were already introduced in [2]). The intuition behind the definition of such enumeration oracle machines is the following: For algorithms (i.e., Turing machines or RAMs in the case of enumeration) using a decision oracle for the language $L$, we usually have a special instruction that given an input $x$ decides in one step whether $x\in L$, and then executes the next step of the algorithm accordingly. For an algorithm ${\mathcal{A}}$ using an enumeration oracle, an input $x$ to some $\text{\sc Enum}{\_}R$-oracle returns in a single step (using the instruction $\mathsf{NOO}$, see the definition below) a single element of $R(x)$, and then ${\mathcal{A}}$ can proceed according to this output.

When executing $\mathsf{NOO}$, the machine writes – in one step – some $y_{i}\in R(x)$ to register $A_{e}$, where $x$ is the word stored in $O^{e}(0),O^{e}(1),\ldots$ and $y_{i}$ is defined as follows: