The Theory of Gene Family Histories

The set $L(T)$ of all $\prec_{T}$ minimal vertices form the leaves of $T$. The subtree $T(x)$ rooted at $x\in V(T)$ is defined as the subgraph induced by the vertex set $\{y\in V(T)\mid y\preceq_{T}x\}$. For any subset $A\subseteq V(T)$, the least common ancestor $\operatorname{lca}(A)$ is the $\preceq_{T}$-minimal vertex $w$ that is an ancestor of all $y\in A$. In particular, we have $\operatorname{lca}(\{x\})=x$ for all $x\in V(T)$. For simplicity, we write $\operatorname{lca}(x,y)$ instead of $\operatorname{lca}(\{x,y\})$. In addition, we may use the subscript “_T” to indicate that $\operatorname{lca}_{T}$ is take w.r.t. the tree $T$. Moreover, we set $\rho_{T}\coloneqq\operatorname{lca}(L(T))$ and note that $\rho_{T}=0_{T}$ if and only if the root $0_{T}$ has at least two children. The set of inner vertices $V^{0}(T)\coloneqq(V(T)\setminus(L(T)\cup\{O_{T}\}))\cup\{\rho_{T}\}$ of $T$ comprise $V(T)\setminus L(T)$ and excludes the root if $0_{T}\neq\rho_{T}$. A rooted tree is phylogenetic if all its inner vertices have at least two children, and binary if all its inner vertices have exactly two children. We will assume throughout that $T$ is phylogenetic but not necessarily binary.

A refinement of a phylogenetic tree $T$ is a phylogenetic tree $T^{\prime}$ on the same leaf set such that $T$ can be obtained from $T^{\prime}$ by contracting edges. The restriction $T|_{L^{\prime}}$ of $T$ to $L^{\prime}$ is the phylogenetic tree with leaf set $L^{\prime}$ obtained from the tree $T$ by deleting all vertices in $L\setminus L^{\prime}$ and their incident edges and by additionally suppressing all inner vertices of degree two except the root. A phylogenetic tree $T^{\prime}$ on some subset $L^{\prime}\subseteq L$ is said to be displayed by $T$ (or equivalently that $T$ displays $T^{\prime}$) if $T^{\prime}$ coincides with $T|_{L^{\prime}}$, see Fig. 2.

Throughout this chapter we assume that all trees are planted and phylogenetic, unless explicitly stated otherwise.

Genes evolve within genomes, i.e., species. In order to understand the relationship between genes and species, we need to describe how the gene tree $T$ “fits together” with the species tree $S$. This idea is formalized by the notion of a reconciliation map that locates the vertices of the gene tree, i.e., the evolutionary events in the history of the genes, in the species tree. We model evolutionary events as precise point in time. This is, of course, an approximation. In reality, even punctual events such as gene duplications require time to spread through a population and become fixed. Similarly, speciation is also a population-based process that takes time Marques:19 and may also involve additional effects such as incomplete lineage sorting Zheng:14 ; Chan:17 . Nevertheless, it is justified to view events as points-in-time in the context of macro-evolutions, where population effects are neglected altogether.

In order to study reconciliation from a formal point of view, we start out with only minimal requirements that are straightforward to argue:

While Condition (R0) is used to identify the planted roots, Condition (R1) ensures that leaves of $T$, i.e., extant genes, are found in leaves of $S$, i.e., in extant species. Condition (R2) ensures that temporally distinct speciation events cannot be mapped to the same vertex in $S$, i.e., the same speciation event. Condition (R3), finally ensures a weak form of temporal consistency by forbidding that a descendant of $x$ in $T$ can be mapped to an ancestor of $\mu(x)$ in $S$. Thus, if $y\prec_{T}x$, then either $\mu(y)\preceq_{S}\mu(x)$ or $\mu(x)$ and $\mu(y)$ are $\preceq_{S}$-incomparable. For completeness we note that many authors prefer reconciliation maps of the form $\gamma:V(T)\to V(S)$, i.e., mappings from vertices to vertices Tofigh:11 ; Bansal:12 ; Stolzer:12 . This can be easily translated, however: If $\mu(x)=uv$, then $\gamma(x)=v$, i.e., everything is mapped to the lower end of the edge in $S$. The other direction is a bit more intricate. In essence, however, it suffices to set $\mu(x)=uv$ if $\gamma(x)=v$ and there is a $x^{\prime}\prec_{T}x$ with $\gamma(x^{\prime})=\gamma(x)=v$.

An important distinction in evolution is the difference between vertical and horizontal inheritance. Vertical inheritance implies that the the descendants of a gene are harbored by descendant species. In contrast, horizontal inheritance, i.e., horizontal transfer consists in the relocation of a gene to a different lineage. Formally, this situation can be captured by distinguishing vertical and Horizontal Gene Transfer (HGT) edges in the gene tree $T$:

As a direct consequence of (R3), a reconciliation $(S,T,\mu)$ is HGT-free if and only if it satisfies

• (R4)

  If $y\prec_{T}x$ then $\mu(y)\preceq_{S}\mu(x)$.

In fact, (R4) states that all edges of $T$ correspond to vertical inheritance.

Usually, it is known a priori which gene is found in which species. That is, a map $\sigma:L(T)\to L(S)$ is given that assigns to each extant gene the extant species in which it occurs.

An early important observation in this field is that every gene tree can be reconciled with any species tree even without the need to consider horizontal gene transfer Guigo:96 ; Page:97 :

To see this, it suffices to consider the reconciliation map $\mu_{0}$ defined by setting

$$\begin{split}&\mu_{0}(0_{T})\coloneqq 0_{S};\\ &\mu_{0}(x)\coloneqq\sigma(x)\text{ for all }x\in L(T);\text{ and }\\ &\mu_{0}(x)\coloneqq 0_{S}\rho_{S}\text{ for all inner vertices }x\in V^{0}(T).\end{split}$$

(1)

Every reconciliation map $\mu$ can be used to imply gene loss events. To see this, consider an edge $xy\in V(T)$ such that $\mu(y)\prec_{S}\mu(x)$ for a given reconciliation map $\mu$ and for which there is a vertex $u\in V(S)$ with $\mu(y)\preceq_{S}u\preceq_{S}\mu(x)$, and thus $u\notin L(S)$. During the speciation $u\in V^{0}(S)$, a descendant of the each gene present in the genome is transmitted to every descendant lineage, i.e., to every edge $uw\in E(S)$. If no descendant of the gene is found in a species descending from $w$, then the gene must have died out in the entire species subtree $S(w)$. This is explained most parsimoniously by a single loss event occurring already along the edge $uw\in E(S)$. Fig. 3 shows that for the case of the reconciliation $\mu_{0}$ this reasoning implies an unrealistically large number of gene loss events: for example, the five paralogs present at the earliest speciation give rise to a total of 10 genes, of which half are subsequently lost again. This reconciliation $\mu_{0}$ will in general not be a plausible biological explanation because it implies unreasonably large number of gene losses. This observation led to the development of a number of alternative scoring functions $F(\mu)\coloneqq F(T,S,\mu,\sigma)$ for reconciliation maps with given trees $T$ and $S$ and a given leaf assignment $\sigma$ that count the number of duplication and/or gene losses. We refer to Zhang:97 and the references therein for detailed discussion.

We briefly consider this optimization problem for the HGT-free case here. Assume that we are given a leaf-assignment $\sigma$. The task is then to find a $\sigma$-reconciliation that optimizes a given scoring function. By Thm. 2.1 we know that a solution always exists. Since any vertex $x\in V(T)$ has to be mapped at or above the last common ancestor of all the species in which the descendants of $x$ are found, every solution satisfies

$$\operatorname{lca}_{S}(\sigma(L(T(x))))\preceq_{S}\mu(x)$$

(2)

The so-called LCA reconciliation $\mu^{*}$ attains equality whenever possible. More precisely, we define $\mu^{*}$ as follows. For each $x\in V(T)$, let $w(x)\coloneqq\operatorname{lca}_{S}(\sigma(L(T(x))))$ and $u$ be the parent of $w(x)$ in $S$ which always exists in planted trees.

$$\mu^{*}(x)\coloneqq\begin{cases}w(x)&\text{whenever there is no $y$ with $y\prec_{T}x$ and $w(x)=w(y)$}\\ uw(x)&\text{otherwise.}\end{cases}$$

The LCA-reconciliation $\mu^{*}$ is the most parsimonious reconciliation w.r.t. to several cost measures including the number of duplications and the number duplications and gene losses Chen:00 ; Zhang:97 ; Zmasek:01 . Corresponding maximum likelihood and Bayesian methods is described in Gorecki:11 and Arvestad:03 , respectively. Most parsimonious reconciliations can still be computed in polynomial time if HGT events are included and penalized Bansal:12 ; Tofigh:11 . However, as we shall see in the following section, these reconciliations are not always biologically feasible, see also Menet:22 .

It was noted in Bansal:12 ; Tofigh:11 that the definition of reconciliations in the previous section is not sufficient to ensure that $(T,S,\mu)$ can be interpreted as series of events along a linear time axis. To this end we consider evolutionary scenarios as reconciliations of dated trees:

Axioms (S0) and (S1) coincide with (R0) and (R1), respectively, in the definition of a reconciliation. The remaining two axioms, (S2) and (S3), specify time consistency. As for reconciliations in the previous section, we do not assume that a leaf-map $\sigma:L(T)\to L(S)$ is given, although this typically will be the case in practical applications.

Relaxed scenarios can be seen as reconciliations for which a time axis is given explicitly. It is not difficult to show that for a given a relaxed scenario $(T,S,\mu,\tau_{T},\tau_{S})$, the triple $(T,S,\mu)$ is indeed a reconciliation (cf. Lemma 2 and 3 in SHL+23 ). The converse, however, is not true in general, i.e., it is not always possible to find time stamps that turn a reconciliation into a relaxed scenario cf. e.g. Fig. 2 in Nojgaard:18a . This motivates the following

A variety of different auxiliary graphs on $V(S)$ Tofigh:11 ; Stolzer:12 or $V(S)\cup V(T)$ Nojgaard:18a ; Lafond:20 were introduced to summarize the temporal constraints. In each case, it was shown that a reconciliation $(T,S,\mu)$ is time consistent if and only if the corresponding auxiliary graph is acyclic. It can be shown, furthermore, that these auxiliary graphs are acyclic in the special case of HGT-free reconciliations, i.e., every HGT-free reconciliation is automatically time-consistent and can be extended to a relaxed scenario.

In the general case, the additional requirement of time-consistency also makes the computation of maximum parsimony reconciliations difficult. As shown in Tofigh:11 , the problem then becomes NP hard.

From a biological point of view, the inner vertices $V^{0}(T)$ of the gene tree $T$ model evolutionary events. In the case of HGT-free scenarios, i.e., reconciliations $(T,S,\mu)$ that satisfy (R4), there are only two fundamentally distinct events: gene duplications and speciation events. Since speciation events are also modeled by the inner vertices $V^{0}(S)$, it is natural to distinguish duplications and speciation by their image under the reconciliation map $\mu$. This naturally leads to the following definition Geiss:20b :

The first two cases, $\circledcirc$ and $\odot$, distinguish the planted root and the leaves of $T$. The remaining two cases identify speciation events with the $u\in V(T)$ that are mapped to inner vertices in the species tree. In contrast, if $\mu(u)\in E(T)$, it does not correspond to a speciation and thus corresponds to a gene duplication, denoted by $\square$. The conditions on the reconciliation $(T,S,\mu)$ considered so far, however, are not sufficient to ensure that $x\in V^{0}(T)$ with $\ell_{\mu}(x)=\newmoon$ represents a biologically meaningful speciation event. Additional constraint on the evolutionary scenario need to be introduced.

This is possibly best explained by assuming to have a binary gene tree $T$. In this case, if $\ell_{\mu}(x)=\newmoon$, the two children $v^{\prime}$ and $v^{\prime\prime}$ of $x$ should be mapped via $\mu$ into two distinct lineages. Otherwise, if $v^{\prime}$ and $v^{\prime\prime}$ are mapped into the same lineage there is no clear historical trace that justifies $x$ to be a speciation vertex. In other words, $\ell_{\mu}(x)=\newmoon$ is biologically plausible, if for the two children $v^{\prime}$ and $v^{\prime\prime}$ of $x$ the images $\mu(v^{\prime})$ and $\mu(v^{\prime\prime})$ are $\preceq_{S}$-incomparable. Moreover, similar to Eq. 2 and to accommodate most parsimonious reconciliation (LCA-reconciliation) one may assume that $\mu(x)=\operatorname{lca}_{S}(\mu(v^{\prime}),\mu(v^{\prime\prime}))$. The latter discussion naturally translates to the case of general not necessarily binary gene trees as follows:

• (R5)

  Suppose $\mu(x)\in V(S)^{0}$ for some $x\in V$. Then

  • (i)

    $\mu(x)=\operatorname{lca}_{S}(\mu(v^{\prime}),\mu(v^{\prime\prime}))$ for at least two distinct children $v^{\prime},v^{\prime\prime}$ of $x$ in $T$.

  • (ii)

    $\mu(v^{\prime})$ and $\mu(v^{\prime\prime})$ are incomparable in $S$ for any two distinct children $v^{\prime}$ and $v^{\prime\prime}$ of $x$ in $T$.

As shown in Geiss:20b ; Nojgaard:18a , the axioms (R0)-(R5) are equivalent to axioms that are commonly used in the literature Gorecki:06 ; Vernot:08 ; Doyon:11 ; Rusin:14 ; Hellmuth:17 . In particular, for any HGT-free $\sigma$-reconciliation $(T,S,\mu,\sigma)$, there is a LCA-reconciliation for $T$ and $S$ Hellmuth:17 . The latter observation can easily be extended for $\sigma$-reconciliations involving HGTs and by considering LCA-maps restricted to the HGT-free subtrees of $T$ Hellmuth:17 . Thus, the axiom set used here naturally corresponds to LCA-mappings and hence, to most parsimonious reconciliations. In addition, (Geiss:20b, , Lem.2) shows that $L(T(v^{\prime}))\cap L(T(v^{\prime\prime}))\neq\emptyset$ for a pair of distinct children $v^{\prime}$ and $v^{\prime\prime}$ of $x$ implies $\ell_{\mu}(x)=\square$ for all HGT-free $\sigma$-reconciliations that satisfy (R5). This is turn motivates to consider the extremal event labeling $\hat{\ell}$, which assume that these are the only duplications and thus assigns $\hat{\ell}(x)=\newmoon$ whenever $\mu(L(T(v^{\prime})))\cap\mu(L(T(v^{\prime\prime}))=\emptyset$ for all pairs of children of $x$. It is important to note that the extremal labeling $\hat{\ell}$ is defined solely on the information of $T$ and does not depend on the existence of a species tree $S$ or a reconciliation map $\mu$.There is no a priori guarantee, therefore, that the extremal event labeling can be realized by an actual biological scenario.

Many of the combinatorial methods for determining orthology start from reciprocal best (blast) hits. Here, we consider best matches as a basic evolutionary concept that is approximated on sequence data by “best hits”. We therefore consider best matches relative to an underlying phylogenetic tree, albeit this tree is usually unknown. Our starting point is therefore a gene tree $T$ together with a map $\sigma:V(T)\to Y$, where $Y$ is a set of species. It will be convenient to treat $\sigma$ as a coloring of the leaves of tree $T$ by the species in which the extant genes reside.

Note that a gene $x$ may have two or more (reciprocal) best matches of the same color $r\neq\sigma(x)$. Some orthology detection tools, such as ProteinOrtho Lechner:11a , explicitly attempt to extract all reciprocal best matches from the sequence data.

A directed, vertex-labeled graph $(G,\sigma)$ is a best match graph (BMG) if there is a leaf-labeled tree $(T,\sigma)$ such that $(x,y)$ is a directed edge in $(G,\sigma)$ if and only if $y$ is best match of $x$ in $(T,\sigma)$. Given $(T,\sigma)$ we write $\mathfrak{B}(T,\sigma)$ for its best match graph.

A key observation towards characterizing best match graphs is that some subsets of vertices on two colors (species) yield constraints on triples displayed by any leaf-colored tree that might explain a BMG.

The sets $\mathscr{R}(G,\sigma)$ and $\mathscr{F}(G,\sigma)$ denotes the set of all informative and forbidden triples, respectively. It is shown in Geiss:20b that, if $(G,\sigma)=\mathfrak{B}(T,\sigma)$, then $T$ displays all triples in $\mathscr{R}(G,\sigma)$ and none of the triples in $\mathscr{F}(G,\sigma)$. The sets $\mathscr{R}(G,\sigma)$ and $\mathscr{F}(G,\sigma)$ also give rise to convenient characterizations of BMGs.

Intriguingly, every BMG $(G,\sigma)$ is associated with a unique least-resolved tree $(T^{*},\sigma)$. That is, (i) $\mathfrak{B}(T^{*},\sigma)=(G,\sigma)$ and (ii) if $\mathfrak{B}(T,\sigma)=(G,\sigma)$ then $T$ displays $T^{*}$. The least resolved tree $T^{*}$ therefore captures the reliable phylogenetic information about the gene tree $T$ that is provided by the best matches. The least-resolved tree for a BMG $(G,\sigma)$ is precisely the tree $\mathop{Aho}(\mathscr{R}(G,\sigma))$ and can be constructed in polynomial time.

While least-resolved trees serve as a scaffold to cover phylogenetic information without making more assumptions on the evolutionary history than actually provided by the data, one is in many cases additionally interested to find binary trees which can be considered as the most “highly” resolved histories. Binary trees are of particular interest because true multifurcations are most likely rare, i.e., most polytomies are a consequence of insufficient resolution of the available data Maddison:89 ; DeSalle:94 ; Walsh:99 . However, not every BMG can be explained by a binary tree (cf. (Schaller:21a, , Fig.6A)). Binary explainable BMGs are characterized as those BMGs that do not contain a certain colored graph on four vertices, termed hourglass, as induced subgraph Schaller:21a . Note that binary trees explaining a BMG are not necessarily unique, however, they all display $\mathop{Aho}(\mathscr{R}(G,\sigma))$ and can be constructed in polynomial time Schaller:21e .

Binary explainable BMGs also have a convenient characterization in terms of triple sets. Consider the following extension of the set of informative triples:

$$\mathscr{R}^{B}(G,\sigma)\coloneqq\mathscr{R}(G,\sigma)\cup\left\{bb^{\prime}|a:ab|b^{\prime}\in\mathscr{F}(G,\sigma)\text{ and }\sigma(b)=\sigma(b^{\prime})\right\}$$

(3)

As shown in Schaller:21e , replacing $\mathscr{R}(G,\sigma)$ by $\mathscr{R}^{B}(G,\sigma)$ in Thm. 1 yields a characterization of binary explainable BMGs. Moreover, if $(G,\sigma)$ is a binary explainable BMG, then $T^{B}\coloneqq\mathop{Aho}(\mathscr{R}^{B}(G,\sigma))$ has the property that a binary tree $(T,\sigma)$ satisfies $\mathfrak{B}(T,\sigma)=\mathfrak{B}(T^{B},\sigma)=(G,\sigma)$ if and only if $T$ is a refinement of $T^{B}$. We summarize these results in the following

In practice, $(G,\sigma)$ is obtained empirically by comparing similarities of gene sequences. Most likely, $(G,\sigma)$ thus will not be a BMG but differ from the true best match graph by both false positive and false negative edges. The arc modification problems for BMGs aim at correcting such errors. Like most graph editing problems, it is NP-complete Schaller:21b ; Schaller:21e . However, efficient heuristics can be devised that solve the problem with acceptable accuracy Schaller:21g

Walter Fitch Fitch:70 defined orthology as homology deriving from a speciation event. While later discussions qualified this simple concept in the presence of HGT (see below), the notion is clear in a HGT-free setting. We assume throughout this section that scenarios and reconciliations are HGT-free.

We write $\Theta(T,t)$ for the orthology graph with vertex set $L(T)$ and edges $xy\in E(\Theta(T,t)$ whenever $x$ and $y$ are orthologs. Note that the corresponding “paralogy graph” is simply the complement graph of $\Theta(T,t)$. Moreover, $\Theta(T,t)$ is symmetric but not necessarily transitive (edges $xy$ and $yz$ do not imply that $xz$ is an edge). Orthology graphs feature a simple structure:

A classical characterization of cographs is the following: $G$ is a cograph if and only if it does not contain a $P_{4}$, i.e, a path on four vertices, as an induced subgraph Corneil:81 .

Given a reconciliation $(T,S,\mu)$, orthology is therefore implied by the reconciliation maps, since, by Def. 7 above, $\mu$ specifies the event labeling $\ell_{\mu}$. The dependence of orthology on $\mu$ is crucial. In the extreme case of $\mu_{0}$, we have $\mu_{0}(V(T^{0}))\subseteq E(S)$, and thus all events are classified as duplications. Conversely, the LCA reconciliation maps as many $u\in V^{0}(T)$ to inner vertices of $S$ as possible, and thus can be expected to result in a large number orthologous pairs.

The starting point for many algorithmic approaches to orthology detection is the observation that two orthologs $x$ and $y$ are also reciprocal best matches. We are therefore in particular interested in obtaining information on the orthology relation starting from best match data only.

Writing $\mathfrak{B}_{sym}(T,\sigma)$ for the subgraph of the best match $\mathfrak{B}(T,\sigma)$ comprising the reciprocal, i.e., bi-directional edges, and denoting by $\hat{\ell}_{T}$ the extremal event labeling for $T$ introduced in the previous section, we obtain the following key result:

Theorem 4, in particular, shows that using the reciprocal best match graph $\mathfrak{B}_{sym}(T,\sigma)$ as an approximation for the orthology relation does not yield false negative orthology assignments. In general, however, $\Theta(T,\ell_{\mu})$ is a proper subgraph of $\mathfrak{B}_{sym}(T,\sigma)$, i.e., reciprocal best match graphs may contain false-positive orthology assignments. In particular, $\mathfrak{B}_{sym}(T,\sigma)$ does not need to be a cograph and thus may contain induced $P_{4}$s.

If $L(T(v^{\prime}))\cap L(T(v^{\prime\prime}))\neq\emptyset$ for some children $v^{\prime}$ and $v^{\prime\prime}$ of $x$ then $\ell_{\mu}(x)=\square$ for all HGT-free $\sigma$-reconciliations that satisfy (R5) (cf. (Geiss:20b, , Lem.2)). (Schaller:21a, , Lemma 10) characterizes false orthology assignments in best matches graphs and provides an even stronger results. In particular, it shows that $\hat{\ell}_{\mu}(x)=\square$ precisely if $L(T(v^{\prime}))\cap L(T(v^{\prime\prime}))\neq\emptyset$ for some children $v^{\prime}$ and $v^{\prime\prime}$ of $x$. Using the fact that $\Theta(T,\ell_{\mu})\subseteq\Theta(T,\hat{\ell}_{T})$ allows us to rephrase this result in following, more convenient form:

Note that Thm. 5 depends on the structure of the underlying tree $(T,\sigma)$ and trees that explain a given BMG are not necessarily unique. Hence, there might be a tree $(T^{\prime},\sigma)$ such that $\mathfrak{B}(T,\sigma)=\mathfrak{B}(T^{\prime},\sigma)$ but for which an edge $xy$ is a false-positive orthology assignment w.r.t. $(T,\sigma)$ but not w.r.t. $(T^{\prime},\sigma)$. Hence, we are interested, in particular, in those bi-directional edges $xy$ of a BMG $(G,\sigma)$ that are false orthology assignment for every tree $(T,\sigma)$ that satisfies $\mathfrak{B}(T,\sigma)=(G,\sigma)$. Such edges are called unambiguously false orthology assignment in $(G,\sigma)$ and can be identified in chains of overlapping hourglasses. For the details we refer to Def.14 and Def.16 in Schaller:21a for the detailed specification of all “hug”-pairs and we obtain

The no-hug graph $\mathfrak{N}(G,\sigma)$ is the subgraph of $(G_{sym},\sigma)\subseteq(G,\sigma)$ from which all edges $xy$ that are hug pairs have been removed. In particular, $\mathfrak{N}(\mathfrak{B}(T,\sigma))$ contains the orthology graph for every HGT-free $\sigma$-reconciliation satisfying (R5) $\mu$ as a subgraph (cf. (Schaller:21a, , Cor. 5)):

$$\Theta(T,\ell_{\mu})\subseteq\Theta(T,\hat{\ell}_{T}))\subseteq\mathfrak{N}(\mathfrak{B}(T,\sigma))\subseteq\mathfrak{B}_{sym}(T,\sigma)$$

(4)

As shown in Schaller:21a , $\mathfrak{N}(\mathfrak{B}(T,\sigma))$ is in fact an orthology graph. However, since only unambiguously false orthology have been removed, $\mathfrak{N}(\mathfrak{B}(T,\sigma))$ may still contain false orthologs. That is, reciprocal best matches $xy$ might be paralogs in the true scenario $(T,\ell)$ but appear as orthologs in alternative scenarios that are consistent with the best match data. In fact, it is always possible to “move up” a speciation event and to replace it by a duplication followed by losses. In this manner it is always possible to explain best match without orthologs. The no-hug graphs $\mathfrak{N}(\mathfrak{B}(T,\sigma))$ thus provides the “most parsimonious” explanation in the sense that it predicts an orthology relationship whenever this is consistent with the best match data instead of an alternative explanation, which would comprise a duplication event accompanied by complementary losses. Thus, the reconciliation underlying $\mathfrak{N}(\mathfrak{B}(T,\sigma))$ is a least conceptually related to LCA reconciliations. A closer inspection of this connection, however, is a topic for future research.

In practice, one could obtain an accurate orthology assignment by estimating an initial, species labeled graph $(\tilde{G},\sigma)$ representing best (blast) hit data and then edit these initial data to a BMG, i.e., a graph $(G,\sigma)$ that conforms to Def. 8. This step is, in general, NP-hard HGS:20 ; Schaller:21b and thus, will require an efficient heuristic. From the the BMG $(G,\sigma)$ we can then remove, in polynomial-time, all hug-pairs to obtain the no-hug graph $\mathfrak{N}(G,\sigma)$ that is free of unambiguously false orthologs and also an orthology graph.

Since orthology graphs are cographs, there is also a more direct, albeit less accurate approach towards estimating the orthology graph. To this end, one extracts the reciprocal best hits $(\tilde{G}_{sym},\sigma)$ from the initial estimated best (blast) hits $(\tilde{G},\sigma)$ and finds the cograph $H$ that differs by the fewest edge-insertions and edge-deletions from $\tilde{G}_{sym}$. This cograph editing problem is also NP-hard Liu:12 . However, it remains tractable if $\tilde{G}_{sym}$ is not too dissimilar from a cograph. Fast heuristics have become available for this task in last years White:18 ; Hellmuth:20b ; Crespelle:21 .

By Theorem 1, BMGs can be recognized in polynomial-time and the tree $(T_{\mathop{Aho}},\sigma)\coloneqq(\mathop{Aho}(\mathscr{R}(G,\sigma)),\sigma)$ is the unique least resolved tree that explains a BMG $(G,\sigma)$. However, it is not possible in general to equip $T_{\mathop{Aho}}$ with an event labeling $\ell$ such that $\Theta(T_{\mathop{Aho}},\ell)=\mathfrak{N}(G,\sigma)$. Instead, $T_{\mathop{Aho}}$ can be augmented to a uniquely defined tree $\hat{\ell}_{T^{*}}$ that is equipped with the extremal event labeling $\hat{\ell}_{T^{*}}$. The event labeled tree $(T^{*},\hat{\ell}_{T^{*}})$ satisfied

$$\Theta(T^{*},\hat{\ell}_{T^{*}})=\mathfrak{N}(\mathfrak{B}(T,\sigma))$$

(5)

and can be obtained in polynomial time (cf. (Schaller:21a, , Thm.10)). The extremal labeling $\hat{\ell}_{T^{*}}$ is defined solely on the information of $T^{*}$ and does not depend on the existence of a species tree $S$ or a reconciliation map $\mu$.

In summary, therefore, it is possible to use the information of a BMG $(G,\sigma)$ to get mathematically sound estimates $G^{\prime}$ of orthology relations and a resulting gene tree $(T,\ell)$ such that $G^{\prime}=\Theta(T,\ell)$ in polynomial time.

An intriguing observation is that the event labeled gene trees $(T,\ell)$ have implications for the species tree irrespective of the details of the reconciliation map. To this end, let $\mathscr{S}(T,\ell,\sigma)$ denote the set of all triples $\sigma(x)\sigma(y)|\sigma(z)$ that satisfy (i) $\sigma(x)$, $\sigma(y)$, and $\sigma(z)$ are pairwise distinct species and (ii) $\ell(\operatorname{lca}(x,z))=\newmoon$, i.e., the root of $xy|z$ is a speciation event.

Note that the set of species triples in Prop. 1 depends on the event-labeling $\ell$ of $T$, but not on the details of the reconciliation map $\mu$ as long as $\mu$ gives rise to correct event-labeling. Several types of problems that are concerned with optimally editing a given undirected graph to an orthology relation that, at the same time, satisfies additional constraints (e.g. that the resulting event-labeled gene tree can be reconciled with some (unknown) species tree) have been considered e.g. in Lafond:16 . While most of the latter problems are NP-complete, certain types of problems that are related to the correction of homology relations that provide only partial information about orthologs and non-orthologs can be solved in polynomial-time Lafond:14 ; Nojgaard:18b .

Although Prop. 1 is rather technical, it has significant practical importance. An estimate of $(T,\ell)$, e.g. obtained directly from reciprocal best match data by means of cograph editing provides a collection of species triples. Pooling these data over a large number of gene families indeed yields sufficient information to infer fully resolved species trees Hellmuth:15a .

Orthologs are often summarized as clusters of orthologous groups (COGs) Tatusov:97 . We have seen, however, that orthologous genes form cographs, and hence orthology is in general not a transitive relation. COGs are only an approximation that is particularly useful if the the gene family history contains only a small number of duplications. A special case is one-to-one orthology, where each gene has a unique ortholog in every other species. In this case, the orthology relation becomes transitive and the the orthology graph $\Theta$ reduces to a disjoint union of cliques Roth:08 . These clusters, e.g. computed by OMA Roth:08 , are induced complete subgraphs, i.e., cliques in the full orthology graph.

Most other approaches to computing COGs allow co-orthologs, i.e., clusters are not restricted to cliques of the orthology graph and thus, may include orthologs and paralogs Tatusov:97 . A wide variety of clustering algorithms have been used to extract COGs from sequence similarity data. The definition of such COGs necessarily depends on stringency parameters that gauge the trade-off between size and stringency of COGs. From a theoretical point of view transitivity clustering Rahmann:07 is interesting because of its conceptual similarity to co-graph editing: here the initial orthology estimate is edited by insertion end deletion of edges to a transitive graph, i.e., a partitioning into COGs. In Falls:08 , maximal Túran (complete multipartite) graphs are computed. These form a special class of co-graphs and accommodate so-called in-paralogs TremblaySavard:12 ; Altenhoff:19 , i.e., duplicate gene that originated after the most recent speciation event in each lineage. Complementarily, it is of interest to partition a gene set such that out-paralogs (i.e., pairs of genes arising from duplications that pre-date all speciation events) are placed in different clusters. These correspond to the connected components of the orthology graph $\Theta$.

Fitch Fitch:00 defined “two genes $x$ and $y$ as xenologs if their history, since their common ancestor, involves an interspecies (horizontal) transfer of the genetic material for at least one of them.” Two leaves in $x,y\in L(T)$ in a tree $T$ are thus xenologs whenever the unique path connecting $x$ and $y$ in $T$ contains an HGT-edge. By Def. 2, the subset of HGT-edges $H_{\mu}\subseteq E(T)$, and thus xenology, depends explicitly on the reconciliation $(T,S,\mu)$. As in the previous sections, we are interested in inferring xenology without first computing a reconciliation $(T,S,\mu)$. Again, we approach this problem by studying properties of graphs that are implied by reconciliation or a relaxed scenario.

Let use first consider the parts of a scenario that are unaffected by HGT events. Deleting from $T$ all HGT-edges yields a forest and induces a partition of $L(T)=L_{1}\mathbin{\mathchoice{\leavevmode\vtop{\halign{\hfil$\m@th\displaystyle#$\hfil\cr\cup\cr\cdot\crcr}}}{\leavevmode\vtop{\halign{\hfil$\m@th\textstyle#$\hfil\cr\cup\cr\cdot\crcr}}}{\leavevmode\vtop{\halign{\hfil$\m@th\scriptstyle#$\hfil\cr\cup\cr\cdot\crcr}}}{\leavevmode\vtop{\halign{\hfil$\m@th\scriptscriptstyle#$\hfil\cr\cup\cr\cdot\crcr}}}}L_{2}\mathbin{\mathchoice{\leavevmode\vtop{\halign{\hfil$\m@th\displaystyle#$\hfil\cr\cup\cr\cdot\crcr}}}{\leavevmode\vtop{\halign{\hfil$\m@th\textstyle#$\hfil\cr\cup\cr\cdot\crcr}}}{\leavevmode\vtop{\halign{\hfil$\m@th\scriptstyle#$\hfil\cr\cup\cr\cdot\crcr}}}{\leavevmode\vtop{\halign{\hfil$\m@th\scriptscriptstyle#$\hfil\cr\cup\cr\cdot\crcr}}}}\dots\mathbin{\mathchoice{\leavevmode\vtop{\halign{\hfil$\m@th\displaystyle#$\hfil\cr\cup\cr\cdot\crcr}}}{\leavevmode\vtop{\halign{\hfil$\m@th\textstyle#$\hfil\cr\cup\cr\cdot\crcr}}}{\leavevmode\vtop{\halign{\hfil$\m@th\scriptstyle#$\hfil\cr\cup\cr\cdot\crcr}}}{\leavevmode\vtop{\halign{\hfil$\m@th\scriptscriptstyle#$\hfil\cr\cup\cr\cdot\crcr}}}}L_{k}$ of the leaf set such that the restriction $T_{|L_{i}}$ contains no HGT-edges. Consequently, one can perform the analysis of HGT-free systems outlined above for each of the gene sets $L_{i}$ separately. Note that in general, this does not amount to simply considering the subgraph $(G[L_{i}],\sigma_{|L_{i}})$ of empirical BMGs $(G,\sigma)$ induced by $L_{i}$. In general the empirical BMGs $(G_{i},\sigma)$ with $V(G_{i})=L_{i}$ will feature best matches that might be worse than those best matches in $(G[L_{i}],\sigma_{|L_{i}})$ that are implied by $T$ and the additional knowledge of HGT-edges. We shall return to the inference of HGT-edges and the partitioning of the gene set into maximal HGT-free subsets below.

Mechanistically HGT is a inherently directional event. There is a clear distinction between the horizontally transferred ”copy” and the “original” that continues to be transferred vertically. It is significant, therefore, whether HGT-edges are found along the path from $\operatorname{lca}(x,y)$ to $y$, the path from $\operatorname{lca}(x,y)$ to $x$, or along both paths. Mathematically, this can be captured in the following manner.

A graph $G$ is a (directed) Fitch graph if $G=\mathfrak{F}(T,H)$ for some tree $T$ and edge set $H\subseteq E(T)$. In Geiss:18 a characterization of Fitch graphs in terms of eight forbidden subgraphs on three vertices is given.

Fitch graphs also have a surprisingly simple characterization in terms of their “closed complementary neighborhoods”. For a directed graph consider

$$\overline{N}[x]\coloneqq\{y\in L\mid(x,y)\notin E(G)\},$$

(6)

Since all graphs considered here are loop-free, we have $x\in\overline{N}[x]$. We write $\overline{\mathcal{N}}(G)\coloneqq\{\overline{N}[y]\mid y\in L\}$.