Structural Optimal Jacobian Accumulation and Minimum Edge Count are NP-Complete Under Vertex Elimination
Abstract
We study graph-theoretic formulations of two fundamental problems in algorithmic differentiation. The first (Structural Optimal Jacobian Accumulation) is that of computing a Jacobian while minimizing multiplications. The second (Minimum Edge Count) is to find a minimum-size computational graph. For both problems, we consider the vertex elimination operation. Our main contribution is to show that both problems are NP-complete, thus resolving longstanding open questions. In contrast to prior work, our reduction for Structural Optimal Jacobian Accumulation does not rely on any assumptions about the algebraic relationships between local partial derivatives; we allow these values to be mutually independent. We also provide -time exact algorithms for both problems, and show that under the exponential time hypothesis these running times are essentially tight. Finally, we provide a data reduction rule for Structural Optimal Jacobian Accumulation by showing that false twins may always be eliminated consecutively.
1 Introduction
A core subroutine in numerous scientific computing applications is the efficient computation of derivatives. If inexactness is acceptable, finite difference methods [22] may be applicable. Meanwhile, computer algebra packages allow for exact symbolic computation, though these methods suffer from the requirement of a closed-form expression of the function to be differentiated as well as poor running time in practice. A third approach is algorithmic differentiation, also sometimes called automatic differentiation. Algorithmic differentiation provides almost exact computations, incurring only rounding errors (in contrast to finite difference methods which also incur truncation errors). Furthermore, the running time required by algorithmic differentiation is bounded by the time taken to compute (in contrast to symbolic methods). We refer to the works by Griewank and Walther [11] and by Naumann [20] for an introduction and numerous applications.
In algorithmic differentiation, we assume that the function is implemented as a numerical program. The key insight is that such programs consist of compositions of elementary functions, e.g., multiplication, sin, cos, etc., for which the derivatives are known or easily computable. Then, derivative computations for the function follow by application of the chain rule. The relevant numerical program can be modeled as a directed acyclic graph (DAG) , referred to as the computational graph of [10]. The source and sink vertices model the inputs and outputs of , respectively. The internal vertices model elementary function calls. Arcs (directed edges) model data dependencies. Let denote the set of all paths from source to sink . If we associate to each arc the known (or easily computable) local partial derivative , then the chain rule allows us to compute the derivative of with respect to by
as shown by Bauer [2]. This weighted DAG is the linearized computational graph.
Using the linearized computational graph, we may model the computation of a Jacobian as follows. An elimination of an internal vertex is the deletion of and the creation of all arcs from each in-neighbor to each out-neighbor of which are not already present. We write for the resulting DAG. An elimination sequence of length is a tuple of internal vertices, and we denote by the result of eliminating these vertices in the order given by . We call a total elimination sequence if . If is a total elimination sequence, then with appropriate local partial derivative computations or updates during the sequence, can be thought of as a bipartite DAG (with sources on one side and sinks on the other) representing the Jacobian matrix of the associated numerical program. To reflect the number of multiplications needed to maintain correctness of Equation (1), we say that the cost of eliminating an internal vertex is the Markowitz degree of , that is, the in-degree of times the out-degree of . The cost of an elimination sequence is the sum of the costs of the involved eliminations. We can now phrase the problem of computing a Jacobian with few multiplications in purely graph-theoretic terms:
Optimal Jacobian Accumulation is known to be NP-complete only under certain assumptions regarding algebraic dependencies between local partial derivatives [19]. Despite this, heuristics used in practice are based on the purely structural formulation presented here [4, 6, 9, 21] and understanding the complexity of this formulation, open since at least 1993 [8], has recently been highlighted as an important problem in applied combinatorics [1].
A solution to Structural Optimal Jacobian Accumulation is always a total elimination sequence, resulting in a bipartite DAG representing the Jacobian. A related problem, first posed by Griewank and Vogel in 2005 [10], is to identify a (not necessarily total) elimination sequence which results in a computational graph with a minimum number of arcs:
The motivation to solve this problem is twofold. First, suppose that our function is a map from to and we wish to multiply its Jacobian (a matrix in ) by a matrix . We may model this computation by augmenting the linearized computational graph with new source vertices. For each new source and each original source , we add the arc . By labeling the new arcs with entries from the matrix , we may obtain the result of the matrix multiplication via application of Equation (1). We refer to the work by Mosenkis and Naumann [17] for a formal presentation. The number of multiplications required to use Equation (1) in this way grows with the number of arcs in , thus motivating the computation of a small (linearized) computational graph. A second motivation is that can sometimes reveal useful information not evident in the Jacobian matrix. This situation, known as scarcity, is described by Griewank and Vogel [10]. Thus, it is desirable to store , rather than the Jacobian matrix, and consequently it is also desirable for to be as small as possible.
Despite these motivations and several algorithmic studies [10, 16, 17], the computational complexity of Minimum Edge Count has remained open since its introduction [10, 17]. Like Structural Optimal Jacobian Accumulation, resolving this question has recently been highlighted as an important open problem [1].
Our Results.
We show that both Structural Optimal Jacobian Accumulation and Minimum Edge Count are NP-complete, resolving the key complexity questions which have stood open since 1993 [8] and 2005 [10], respectively. Furthermore, we prove that unless the exponential time hypothesis (ETH)111The ETH is a popular complexity assumption that states that 3-Sat cannot be solved in subexponential time. See Section 2 for more details. fails, neither problem admits a subexponential algorithm, i.e., an algorithm with running time . We complement our lower bounds by providing -time algorithms for both problems.
2 Preliminaries and Basic Observations
In this section, we define the notation we use throughout the paper, introduce relevant concepts from the existing literature, and show two useful basic propositions.
Notation.
For a positive integer , we use to denote the set . We use standard graph terminology. In particular, a graph or is a pair where denotes the set of vertices and and denote the set of (undirected) edges or (directed) arcs, respectively. We use to indicate the number of vertices in a graph and to indicate the number of edges or arcs. For an (undirected) edge between two vertices and we write , and for an arc (a directed edge) from to we write . Given a vertex , we denote by and the open in- and out-neighborhoods of , respectively. The Markowitz degree is defined to be . If the graph is clear from context, we omit the subscript in the above notation. We say that two vertices and are false twins in a directed graph if and . Given sequences and , we write for the combined sequence , and we generalize this notation to more than two sequences.
Reductions and the ETH.
We assume the reader to be familiar with basic concepts in complexity theory like big-O (Bachmann–Landau) notation, NP-completeness, and polynomial-time many-one reductions (also known as Karp reductions). We refer to the standard textbook by Garey and Johnson [7] for an introduction. We use to hide factors that are polynomial in the input size and call a polynomial-time many-one reduction a linear reduction when the size of the constructed instance is linear in the size of the original instance , that is, . The exponential time hypothesis (ETH) [12] states that there is some such that each algorithm solving 3-Sat takes at least time, where is the number of variables in the input instance. Assuming the ETH, 3-Sat and many other problems cannot be solved in subexponential () time [13]. It is known that if there is a linear reduction from a problem to a problem and cannot be solved in subexponential time, then also cannot be solved in subexponential time [13].
Fundamental Observations.
We next show two useful observations. The first one states that the order in which vertices are eliminated does not affect the resulting graph (note that it may still affect the cost of the elimination sequence). This is a folklore result, but to our knowledge no proof is known. Our argument can be seen as an adaptation of one used by Rose and Tarjan to prove a closely related result [23].
Proposition 1.
Let be a DAG, let be a set of vertices, and let and be two permutations of the vertices in . Then, .
Proof.
We first show for any DAG and any three vertices that there is a directed path from to in if and only if there is a directed path from to in . To this end, first assume that there is a directed path from to in . If does not contain , then is also a directed path in . Otherwise, let be the vertices before and after in , respectively. Since the elimination of adds an arc from to , there is also a directed path from to in this case. Now assume that there is a directed path in . We assume without loss of generality that is a shortest path in . There are again two cases. Either is also a directed path in or there is at least one arc in that is not present in . In the first case, shows that there is a directed path from to in by definition. In the second case, we consider the first arc in that is not contained in . Note that by construction is only added to if is an in-neighbor of and is an out-neighbor of in . Moreover, since is a shortest path, there is no other vertex in that is also a out-neighbor of as otherwise the arc exists in contradicting that is a shortest path. We can now replace the arc in by a subpath to get a directed path from to in .
Let be a sequence. By induction on (using the above argument), it holds for any two vertices that there is a directed path from to in if and only if there one in .
Now assume towards a contradiction that . Note that both contain the same set of vertices. We assume without loss of generality that there is an arc that exists in but not in . By the above argument, since appears in there is a directed path from to in . However, since does not appear in , there is no directed path from to in , a contradiction. This concludes the proof. ∎
Let be a sequence of internal vertices, and let be the set of vertices appearing in . In the rest of this paper, we may use to denote the graph . By Proposition 1, this notation is well-defined.
To conclude this section, we show that false twins can be handled uniformly in Structural Optimal Jacobian Accumulation. Let be a set of false twins, i.e., and are false twins for every . Then we may assume, without loss of generality, that eliminating any also entails eliminating the rest of the vertices in immediately afterward.
Proposition 2.
Let be a DAG and let be a set of false twins. Then, there exists an optimal elimination sequence (for Structural Optimal Jacobian Accumulation) that eliminates the vertices of consecutively.
Proof.
Let be a set of false twins in . We first prove the result when . Let , and let be an optimal solution. We may assume that and are eliminated non-consecutively in , as otherwise the proof is complete. We further assume without loss of generality that is eliminated before in . Let be the subsequence of before , be the subsequence between and and be the subsequence after . Let , and be the sets of vertices appearing in , and , respectively. Note that and might be empty, but contains at least one vertex. Let and . Let , and be the costs of , and , respectively. Since is optimal, it holds that and . Now, we claim that both and are optimal (it suffices to show that either or is optimal, but it will be useful later to prove that both are). Assume otherwise, so at least one of the inequalities or is strict. This implies . We will show that this inequality leads to a contradiction.
It holds that the cost of in and the cost of in are accounted for identically in the total costs of , and . Thus, any difference in the values of and is attributable entirely to differing costs of eliminating and the vertices in . Let and be the costs of in , and , respectively. Note that these terms are well-defined by Proposition 1. This implies
(5) |
Moreover, since and are false twins and the elimination of does not change the cost of eliminating , it holds that and . Substituting this into Inequality (5) yields . Next, let and let for each . Notice that it holds that , , and . To conclude the proof, we will show that for each ,
Note that this implies , yielding the desired contradiction. To show the above claim, we consider three cases: (i) is an out-neighbor of in , (ii) is an in-neighbor of in , and (iii) is neither an in- nor an out-neighbor of in . Note that since is a DAG, cannot be both an in-neighbor and an out-neighbor of . Moreover, since and are false twins, is an in-/out-neighbor of if and only if it is an in-/out-neighbor of . In the first case, note that
-
•
,
-
•
, and
-
•
.
The first holds as the out-degree of does not change if and/or are eliminated. To see the second, note that eliminating and can reduce the in-degree of by at most two. Finally, if is already eliminated, then eliminating does not add any new in-neighbors of since and are false twins (and this property remains true even if other vertices are eliminated). Thus, we get
The second case is analogous with the roles of in- and out-neighbors swapped, that is,
-
•
,
-
•
, and
-
•
.
This yields
Finally, in the third case it holds that
-
•
, and
-
•
.
Thus, we get
Since holds in all cases and we showed before that this contradicts Inequality (5), this concludes the proof when .
To generalize the result to larger sets , we use induction on the size of . As shown above, the base case where holds true. Next, assume the proposition is true for . Let , and let be an optimal elimination sequence. Let . By the inductive hypothesis (on ) and Proposition 1, we may assume that or for some sequences , and . By applying the argument from the case where to and or to and , we observe optimal solutions or . This completes the proof. ∎
3 Structural Optimal Jacobian Accumulation is NP-complete
In this section, we show that Structural Optimal Jacobian Accumulation is NP-complete. We reduce from Vertex Cover, which is defined as follows.
Theorem 1.
Structural Optimal Jacobian Accumulation is NP-complete. Assuming the ETH, it cannot be solved in time.
Proof.
We first show containment in NP. Note that a total elimination sequence is a permutation of and therefore can be encoded in polynomial space. Moreover, given such a sequence, we can compute its cost in polynomial time by computing the vertex eliminations one after another.
To show hardness, we reduce from Vertex Cover. It is well known that Vertex Cover is NP-hard and cannot be solved in time unless the ETH fails [13, 14]. We will provide a linear reduction from Vertex Cover to Structural Optimal Jacobian Accumulation thereby proving the theorem. To this end, let be an input instance of Vertex Cover. Let and . We create an equivalent instance of Structural Optimal Jacobian Accumulation as follows. For each vertex , we create five vertices . The vertices and are contained in for all . Vertices and are contained in . Next, we add the set of arcs to for each . Finally, for each edge , we add the arcs and to . Notice that every arc goes from a lower-indexed vertex to a higher-indexed vertex. Hence, the constructed digraph is a DAG. To finish the construction, we set An illustration of the construction is depicted in Figure 1.
Since the reduction can clearly be computed in polynomial time, it only remains to show correctness. We proceed to show that is a yes-instance of Vertex Cover if and only if the constructed instance is a yes-instance of Structural Optimal Jacobian Accumulation.
First, assume that contains a vertex cover of size at most . We show that eliminating all vertices with first, then for all , followed by for all , and finally all vertices for each results in a total cost of . To see this, note that the cost of eliminating for any vertex is as has a single in-neighbor , three out-neighbors , and , and out-neighbors (one for each ). The cost of eliminating for any afterwards is as by construction has in-neighbors , two out-neighbors and no in-neighbors in , as each is by definition in and hence the corresponding has been eliminated before. The cost of eliminating for is afterwards as has the single in-neighbor , two out-neighbors and and out-neighbors (note that cannot have neighbors in since is a vertex cover and ). Finally, the cost of eliminating for any is since has in-neighbors and two out-neighbors and . Summing these costs over all vertices and applying the handshake lemma222The handshake lemma states that the sum of vertex degrees is twice the number of edges [5]. gives a total cost of
This shows that the constructed instance is a yes-instance of Structural Optimal Jacobian Accumulation.
In the other direction, assume that there is an ordering of the vertices in resulting in a total cost of at most . Let be the set of vertices such that for each it holds that is eliminated before or is eliminated before for any by . We will show that is a vertex cover of size at most in . To this end, we first provide a lower bound for the cost of eliminating any vertex, regardless of which vertices have been eliminated previously. Note that for any vertex has two out-neighbors and in , in-neighbors in , and one additional in-neighbor which is either if was not eliminated before or if was eliminated before. Hence, the cost for eliminating is at least . Moreover, the cost of eliminating for any vertex is at least as has the in-neighbor , two out-neighbors , and for each at least one additional out-neighbor ( if was not eliminated before or and if was eliminated before). Summing these costs over all vertices (and again applying the handshake lemma) gives a lower bound of .
The next step is to prove that contains at most vertices. To this end, note that for each vertex , the cost increases by at least one over the analyzed lower bound. If is eliminated after for some , then the cost of eliminating increases by one as has the additional out-neighbor . If is eliminated before for some and , then the cost of eliminating increases by one as the out-neighbor is replaced by the two out-neighbors and . This immediately implies that .
Finally, we show that is a vertex cover. Assume towards a contradiction that this is not the case. Then, there is some edge with and . By definition of , it holds that is eliminated before , is eliminated after , is eliminated before , and is eliminated after by . Note that this implies that eliminates before , before , before , before , a contradiction. Thus, is a vertex cover of size at most and the initial instance of Vertex Cover is therefore a yes-instance. This concludes the proof. ∎
4 Minimum Edge Count is NP-complete
In this section we show that Minimum Edge Count is NP-complete and, assuming the ETH, it cannot be solved in subexponential time. To this end, we reduce from Independent Set, which is defined as follows.
Theorem 2.
Minimum Edge Count is NP-complete. Assuming the ETH, it cannot be solved in time.
Proof.
We again start by showing containment in NP. We can encode a (not necessarily total) elimination sequence in polynomial space. Moreover, given such a sequence, we can compute the resulting DAG in polynomial time and verify that it contains at most edges.
To show hardness, we reduce from Independent Set in 2-degenerate subcubic graphs of minimum girth five, that is, graphs in which each vertex has between two and three neighbors and each cycle has length at least five. This problem is known to be NP-hard and cannot be solved in time assuming the ETH [15]. We will provide a linear reduction from that problem to Minimum Edge Count to show the theorem. To this end, let be an instance of Independent Set where each vertex has between two and three neighbors and no cycles of length three or four are present in . We will construct an instance of Minimum Edge Count. We begin by imposing an arbitrary total order on the vertex set . We partition the vertices into four types based on their degree and the order as follows. Vertices of type 1 have degree 3 and either all neighbors come earlier with respect to or all neighbors come later. Vertices of type 2 have degree 3 and at least one earlier and at least one later neighbor with respect to . Vertices of type 3 have degree 2 and either both neighbors come earlier or both neighbors come later with respect to . Finally, vertices of type 4 have degree 2 and one of the neighbors comes earlier with respect to while the other neighbor comes later.
We now describe the construction of , depicted in Figure 2. We begin by creating a set of sink vertices. Next, for each , we create a vertex as well as two vertex sets and both of size . We add arcs from each vertex in to every vertex in to . We also add arcs from to each vertex in to . Finally, we add arcs from to based on the type of as follows. Note that since , there are up to possible arcs from vertices in to vertices in . For type-1 vertices, we add 14 of the possible arcs (it does not matter which arcs we add). For vertices of type 2,3, and 4, we add and arcs to , respectively. After completing this procedure for every vertex in , we add an arc for any edge where comes before in the order . This concludes the construction of the graph . Let and be the number of vertices and arcs in . To conclude the construction, we set . Observe that is a DAG and the number of vertices and arcs is linear in .
Since the reduction can be computed in polynomial time and the constructed instance has linear size in the input instance, it only remains to show correctness. To this end, first assume that there exists an independent set of size in . We eliminate the vertices . Note that Proposition 1 ensures that the order in which we eliminate the vertices does not matter. We will show that the resulting graph contains at most arcs.
First, note that new arcs between vertices and might be created while eliminating vertices in . However, since is an independent set, such arcs are only created between vertices where . Since has girth at least 5, if the elimination of a vertex in could create the arc , then this arc was not there initially and was not added by the elimination of a different vertex in . We next show that the elimination of each vertex in reduces the number of arcs in the graph by exactly one. Let be an arbitrary vertex. If has degree three, then has exactly incident arcs, where 12 are to or from and and three are to vertices . The number of new arcs created in this case is as shown next. For each , four new arcs are created from vertices in to if comes before with respect to and four new arcs from to vertices in are created otherwise. Hence, in any case, 12 new arcs are created. If is of type 1, then 2 arcs are created from vertices in to vertices in . If is of type 2, then two additional arcs are created between the vertices , and . Hence, if has degree 3, then the elimination of vertex removes arcs and creates 14 new ones, that is, the number of arcs decreases by one. If has degree 2, then is incident to exactly arcs (12 to and from vertices in , and and two additional arcs to or from vertices and ). The number of new arcs created in this case is as shown next. For each , four new arcs are created from vertices in to if comes before with respect to and four new arcs from to vertices in are created otherwise. In any case, 8 new arcs are created this way. If is of type 3, then five arcs are created from vertices in to vertices in . If is of type 4, then four additional arcs are created from vertices in to vertices in and one additional arc is added between and . Hence, if has degree 2, then the elimination of vertex removes arcs and creates 13 new ones, that is, the number of arcs also decreases by one in this case. Since and each of the removals decreases the number of arcs by one, the resulting graph contains at most arcs, showing that the constructed instance is a yes-instance.
For the other direction, assume that is a solution to , that is, has at most arcs. Further, assume that is minimal in the sense that, for each , contains more arcs than . Note that this notion is well-defined due to Proposition 1. Moreover, given any solution, a minimal one can be computed in polynomial time. Let . We will show that induces an independent set in and that , that is, is an independent set of size at least in and the original instance is therefore a yes-instance.
Assume toward a contradiction that is not an independent set in , that is, there exist vertices such that . We claim that is not minimal in this case. To prove this claim, note that eliminating a vertex does not decrease the in-degree or out-degree of any vertex (at any stage during a elimination sequence) and if , then one of the degrees of increases. If is neither an in-neighbor nor an out-neighbor of , then eliminating does not change either degree of . If is an in-neighbor, then the out-degree of remains unchanged and the in-degree increases as the vertices in become in-neighbors of (and they cannot be in-neighbors of while is not eliminated). If is an out-neighbor of , then the in-degree of remains unchanged and the out-degree increases as the vertices in become new out-neighbors of . Let be the number of vertices such that or is an arc in and . Note that since and . Eliminating in removes arcs. If has degree 3, then eliminating in creates at least arcs since (i) implies and (ii) eliminating creates four arcs between vertices in and each other (in- or out-)neighbor of except for vertices in . If has degree 2, then eliminating creates at least four arcs from vertices in to vertices in plus at least arcs, that is, at least arcs since implies . Hence, in any case the number of newly created arcs is at least as large as the number of removed arcs. That is, the number of arcs does not decrease, showing that is not a minimal solution.
It only remains to show that . As analyzed in the forward direction, the elimination of any vertex reduces the number of arcs by exactly one if no vertex with was eliminated before. Since , this shows that , concluding the proof. ∎
5 Algorithms
In this section, we give two simple algorithms that show that Structural Optimal Jacobian Accumulation and Minimum Edge Count can be solved in time. We begin with Minimum Edge Count.
Proposition 3.
Minimum Edge Count can be solved in time and with polynomial space.
Proof.
By Proposition 1, the order in which vertices in an optimal solution are eliminated is irrelevant. Hence, we can simply test for each subset of vertices, how many arcs remain if the vertices in are eliminated. Since there are possible subsets and each of the at most eliminations for each subset can be computed in time, all subsets can be tested in time. ∎
We continue with Structural Optimal Jacobian Accumulation, where we use an algorithmic framework due to Bodlaender et al. [3].
Proposition 4.
Structural Optimal Jacobian Accumulation can be solved in time. It can also be solved in time using polynomial space.
Proof.
As shown by Bodlaender et al. [3], any vertex ordering problem can be solved in time and in time using polynomial space if it can be reformulated as , where is a permutation of the vertices, is the set of all vertices that appear before in , and can be computed in time. We show that Structural Optimal Jacobian Accumulation fits into this framework (with ). We only consider vertices in , that is, non-terminal vertices as these are all the vertices that should be eliminated. We use the function
Note that we can compute (and therefore ) in time. Note that given a permutation , the cost of eliminating all vertices in exactly corresponds to as the cost of eliminating a vertex in a solution sequence following is exactly This concludes the proof. ∎
6 Conclusion
We have resolved a pair of longstanding open questions by showing that Structural Optimal Jacobian Accumulation and Minimum Edge Count are both NP-complete. Our progress opens the door to many interesting questions. On the theoretical side, a key next step is to understand the complexities of both problems under the more expressive edge elimination operation [18]. There are also promising opportunities to develop approximation algorithms and/or establish lower bounds.
Acknowledgments
The authors would like to sincerely thank Paul Hovland for drawing their attention to the studied problems at Dagstuhl Seminar 24201, for insightful discussions, and for generously reviewing a preliminary version of this manuscript, providing valuable feedback and comments.
MB was supported by the European Research Council (ERC) project LOPRE (819416) under the Horizon 2020 research and innovation program. AC, YM, and BDS were partially supported by the Gordon & Betty Moore Foundation under grant GBMF4560 to BDS.
References
- [1] S. G. Aksoy, R. Bennink, Y. Chen, J. Frías, Y. R. Gel, B. Kay, U. Naumann, C. O. Marrero, A. V. Petyuk, S. Roy, I. Segovia-Dominguez, N. Veldt, and S. J. Young. Seven open problems in applied combinatorics. Journal of Combinatorics, 14(4):559–601, 2023.
- [2] F. L. Bauer. Computational graphs and rounding error. SIAM Journal on Numerical Analysis, 11(1):87–96, 1974.
- [3] H. L. Bodlaender, F. V. Fomin, A. M. C. A. Koster, D. Kratsch, and D. M. Thilikos. A note on exact algorithms for vertex ordering problems on graphs. Theory of Computing Systems, 50(3):420–432, 2012.
- [4] J. Chen, P. Hovland, T. Munson, and J. Utke. An integer programming approach to optimal derivative accumulation. In Proceedings of the 6th International Conference on Automatic Differentiation, pages 221–231, Berlin Heidelberg, 2012. Springer.
- [5] R. Diestel. Graph Theory. Springer, Berlin Heidelberg, 2012.
- [6] S. A. Forth, M. Tadjouddine, J. D. Pryce, and J. K. Reid. Jacobian code generated by source transformation and vertex elimination can be as efficient as hand-coding. ACM Transactions on Mathematical Software, 30(3):266–299, 2004.
- [7] M. R. Garey and D. S. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman, USA, 1979.
- [8] A. Griewank. Some bounds on the complexity of gradients, Jacobians, and Hessians. In Complexity in numerical optimization, pages 128–162. World Scientific, Berlin Heidelberg, 1993.
- [9] A. Griewank and U. Naumann. Accumulating Jacobians as chained sparse matrix products. Mathematical Programming, 95:555–571, 2003.
- [10] A. Griewank and O. Vogel. Analysis and exploitation of Jacobian scarcity. In Proceedings of the 2nd International Conference on High Performance Scientific Computing, pages 149–164, Berlin Heidelberg, 2003. Springer.
- [11] A. Griewank and A. Walther. Evaluating derivatives: principles and techniques of algorithmic differentiation. SIAM, Philadelphia, 2008.
- [12] R. Impagliazzo and R. Paturi. On the complexity of -Sat. Journal on Computer and Syststem Sciences, 62(2):367–375, 2001.
- [13] R. Impagliazzo, R. Paturi, and F. Zane. Which problems have strongly exponential complexity? Journal of Computer and System Sciences, 63(4):512–530, 2001.
- [14] R. M. Karp. Reducibility among Combinatorial Problems, pages 85–103. Springer US, Boston, MA, 1972.
- [15] C. Komusiewicz. Tight running time lower bounds for vertex deletion problems. ACM Transactions on Computation Theory, 10(2):6:1–6:18, 2018.
- [16] A. Lyons and J. Utke. On the practical exploitation of scarsity. In Proceedings of the 5th International Conference on Automatic Differentiation, pages 103–114, Berlin Heidelberg, 2008. Springer.
- [17] V. Mosenkis and U. Naumann. On optimality preserving eliminations for the minimum edge count and optimal Jacobian accumulation problems in linearized DAGs. Optimization Methods and Software, 27(2):337–358, 2012.
- [18] U. Naumann. Elimination Techniques for Cheap Jacobians, pages 247–253. Springer, Berlin Heidelberg, 2002.
- [19] U. Naumann. Optimal Jacobian accumulation is NP-complete. Mathematical Programming, 112:427–441, 2008.
- [20] U. Naumann. The art of differentiating computer programs: An introduction to algorithmic differentiation. SIAM, Philadelphia, 2011.
- [21] J. D. Pryce and E. M. Tadjouddine. Fast automatic differentiation Jacobians by compact lu factorization. SIAM Journal on Scientific Computing, 30(4):1659–1677, 2008.
- [22] A. Quarteroni, R. Sacco, and F. Saleri. Numerical mathematics. Springer Science & Business Media, Berlin Heidelberg, 2006.
- [23] D. J. Rose and R. E. Tarjan. Algorithmic aspects of vertex elimination on directed graphs. SIAM Journal on Applied Mathematics, 34(1):176–197, 1978.