Complex event recognition meets hierarchical conjunctive queries

The technical contributions and outline of the paper are the following.

In Section 2, we provide some basic definitions plus recalling the definition of CCEA.

In Section 3, we introduce the concept of parallelization for standard non-deterministic NFA, called PFA, and study their properties. We show that PFA can be determinized in exponential time (similar to NFA) (Proposition 3.2). We then apply this notion to CER and define the class of PCEA, showing that it is strictly more expressive than CCEA (Proposition 3.4).

Section 4 compares PCEA with HCQ under bag semantics. Given that PCEA runs over streams and HCQ over relational databases, we must revisit the semantics of HCQ and formalize in which sense an HCQ and a PCEA define the same query. We show that under such comparison, every HCQ $Q$ under bag semantics can be expressed by a PCEA with equality predicates of exponential size in $|Q|$ and of quadratic size if $Q$ does not have self joins (Theorem 4.1). Furthermore, if $Q$ is acyclic but not hierarchical, then $Q$ cannot be defined by any PCEA (Theorem 4.2).

In Section 5, we study the evaluation of PCEA in a streaming scenario. Specifically, we present a streaming evaluation algorithm under a sliding window with logarithmic update time and output-linear delay for the class of unambiguous PCEA with equality predicates (Theorem 5.1).

Dynamic query evaluation of HCQ and acyclic CQ has been studied in [5, 18, 28, 20]. This research line did not study HCQ or acyclic CQ in the presence of order predicates. [29, 26] studied CQ under comparisons (i.e., $\theta$-joins) but in a static setting (i.e., no updates). The closest work is [19], which studied dynamic query evaluation of CQ with comparisons; however, this work did not study well-behaved classes of HCQ with comparisons, and, further, their algorithms have update time linear in the data.

Complex event recognition and, more generally, data stream processing have studied the evaluation of joins over streams (see, e.g., [31, 30, 21]). To the best of our knowledge, no work in this research line optimizes queries focused on HCQ or provides guarantees regarding update time or enumeration delay in this setting. We base our work on [16], which we will discuss extensively.

A string is a sequence of elements $\bar{s}=a_{0}\ldots a_{n-1}$. For presentation purposes, we make no distinction between a sequence or a string and, thus, we also write $\bar{s}=a_{0},\ldots,a_{n-1}$ for denoting a string. We will denote strings using a bar and its $i$-th element by $\bar{s}[i]=a_{i}$. We use $|\bar{s}|=n$ for the length of $\bar{s}$ and $\{\bar{s}\}=\{a_{0},\ldots,a_{n-1}\}$ to consider $\bar{s}$ as a set. Given two strings $\bar{s}$ and $\bar{s}^{\prime}$, we write $\bar{s}\bar{s}^{\prime}$ for the concatenation of $\bar{s}$ followed by $\bar{s}^{\prime}$. Further, we say that $\bar{s}^{\prime}$ is a prefix of $\bar{s}$, written as $\bar{s^{\prime}}\preceq_{p}\bar{s}$, if $|\bar{s}^{\prime}|\leq|\bar{s}|$ and $\bar{s}^{\prime}[i]=\bar{s}[i]$ for all $i<|\bar{s}^{\prime}|$. Given a non-empty set $\Sigma$ we denote by $\Sigma^{*}$ the set of all strings from elements in $\Sigma$, where $\epsilon\in\Sigma^{*}$ denotes the $0$-length string. For a function $f:\Sigma\rightarrow\Omega$ and $\bar{s}\in\Sigma^{*}$, we write $f(\bar{s})=f(a_{0})\ldots f(a_{n-1})$ to denote the point-wise application of $f$ over $\bar{s}$.

A Non-deterministic Finite Automaton (NFA) is a tuple $\pazocal{A}=(Q,\Sigma,\Delta,I,F)$ such that $Q$ is a finite set of states, $\Sigma$ is a finite alphabet, $\Delta\subseteq Q\times\Sigma\times Q$ is the transition relation, and $I$ and $F$ are the set of initial and final states, respectively. A run of $\pazocal{A}$ over a string $\bar{s}=a_{0}\ldots a_{n-1}\in\Sigma^{*}$ is a non-empty sequence $p_{0}\ldots p_{n}$ such that $p_{0}\in I$, and $(p_{i},a_{i},p_{i+1})\in\Delta$ for every $i<n$. We say that $\pazocal{A}$ accepts a string $\bar{s}\in\Sigma^{*}$ iff there exists such a run of $\pazocal{A}$ over $\bar{s}$ such that $p_{n}\in F$. We define the language $\pazocal{L}(\pazocal{A})\subseteq\Sigma^{*}$ of all strings accepted by $\pazocal{A}$. Finally we say that $\pazocal{A}$ is a Deterministic Finite Automaton (DFA) iff $\Delta$ is given as a partial function $\Delta:Q\times\Sigma\rightarrow Q$ and $|I|=1$.

Fix a set D of data values. A relational schema $\sigma$ (or just schema) is a pair $(\textbf{T},\operatorname{arity})$ where T are the relation names and $\operatorname{arity}:\textbf{T}\rightarrow\mathbb{N}$ maps each name to a number, that is, its arity. An $R$-tuple of $\sigma$ (or just a tuple) is an object $R(a_{0},\ldots,a_{k-1})$ such that $R\in\textbf{T}$, each $a_{i}\in\textbf{D}$, and $k=\operatorname{arity}(R)$. We will write $R(\bar{a})$ to denote a tuple with values $\bar{a}$. We denote by $\operatorname{Tuples}[\sigma]$ the set of all $R$-tuples of all $R\in\textbf{T}$. We define the size of a tuple $R(\bar{a})$ as $|R(\bar{a})|=\sum_{i=0}^{k-1}|\bar{a}[i]|$ with $k=\operatorname{arity}(R)$ where $|\bar{a}[i]|$ is the size of the data value $\bar{a}[i]\in\textbf{D}$, which depends on the domain.

A stream $\pazocal{S}$ over $\sigma$ is an infinite sequence of tuples $\pazocal{S}=t_{0}t_{1}t_{2}\ldots$ such that $t_{i}\in\operatorname{Tuples}[\sigma]$ for every $i\geq 0$. For a running example, consider the schema $\sigma_{0}$ with relation names $\textbf{T}=\{R,S,T\}$, $\operatorname{arity}(R)=\operatorname{arity}(S)=2$ and $\operatorname{arity}(T)=1$. A stream $\pazocal{S}_{0}$ over $\sigma_{0}$ could be the following:

$$\pazocal{S}_{0}\ :=\ \underbrace{S(2,11)}_{\texttt{\scriptsize{0}}}\ \underbrace{T(2)}_{\texttt{\scriptsize{1}}}\ \underbrace{R(1,10)}_{\texttt{\scriptsize{2}}}\ \underbrace{S(2,11)}_{\texttt{\scriptsize{3}}}\ \underbrace{T(1)}_{\texttt{\scriptsize{4}}}\ \underbrace{R(2,11)}_{\texttt{\scriptsize{5}}}\ \underbrace{S(4,13)}_{\texttt{\scriptsize{6}}}\ \underbrace{T(1)}_{\texttt{\scriptsize{7}}}\ \ldots$$

where we add an index (i.e., the position) below each tuple (for simplification, we use $\textbf{D}=\mathbb{N}$).

For a fix $k$, a $k$-predicate $P$ is a subset of $\operatorname{Tuples}[\sigma]^{k}$. Further, we say that $\bar{t}=(t_{1},\ldots,t_{k})$ satisfies $P$ iff $\bar{t}\in P$. We say that $P$ is unary if $k=1$ and binary if $k=2$. In the following, we denote any class of unary or binary predicates by U or B, respectively.

Although we define our automata models for any class of unary and binary predicates, the following two predicate classes will be relevant for algorithmic purposes (see Section 4 and 5). Let $\sigma$ be a schema. We denote by $\textbf{U}_{\operatorname{lin}}$ the class of all unary predicates $U$ such that, for every $t\in\operatorname{Tuples}[\sigma]$, one can decide in linear time over $|t|$ whether $t$ satisfies $U$ or not. In addition, we denote by $\textbf{B}_{\operatorname{eq}}$ the class of all equality predicates defined as follow: a binary predicate $B$ is an equality predicate iff there exist partial functions $\vec{\reflectbox{$B$}}$ and $\vec{B}$ over $\operatorname{Tuples}[\sigma]$ such that, for every $t_{1},t_{2}\in\operatorname{Tuples}[\sigma]$, $(t_{1},t_{2})\in B$ iff $\reflectbox{$\vec{\reflectbox{$B$}}$}(t_{1})$ and $\vec{B}(t_{2})$ are defined and $\reflectbox{$\vec{\reflectbox{$B$}}$}(t_{1})=\vec{B}(t_{2})$. Further, we require that one can compute $\reflectbox{$\vec{\reflectbox{$B$}}$}(t_{1})$ and $\vec{B}(t_{2})$ in linear time over $|t_{1}|$ and $|t_{2}|$, respectively. For example, recall our schema $\sigma_{0}$ and consider the binary predicate $(Tx,Sxy)=\{(T(a),S(a,b))\mid a,b\in\textbf{D}\}$. Then by using the functions $\reflectbox{$\vec{\reflectbox{$B$}}$}(T(a))=a$ and $\vec{B}(S(a,b))=a$, one can check that $(Tx,Sxy)$ is an equality predicate.

Note that $\textbf{B}_{\operatorname{eq}}$ is a more general class of equality predicates compared with the ones used in [16], that will serve in our automata models for comparing tuples by “equality” in different subsets of attributes. We take here a more semantic presentation, where the equality comparison between tuples is directly given by the functions $\vec{\reflectbox{$B$}}$ and $\vec{B}$ and not symbolically by some formula.

A Chain Complex Event Automaton (CCEA) [16] is a tuple $\pazocal{C}\ =\ (Q,\textbf{U},\textbf{B},\Omega,\Delta,I,F)$ where $Q$ is a finite set of states, U is a set of unary predicates, B is a set of binary predicates, $\Omega$ is a finite set of labels, $I:Q\rightarrow\textbf{U}\times(2^{\Omega}\setminus\{\emptyset\})$ is a partial initial function, $F\subseteq Q$ is the set of final states, and $\Delta$ is a finite transition relation of the form: $\Delta\subseteq Q\times\textbf{U}\times\textbf{B}\times(2^{\Omega}\setminus\{\emptyset\})\times Q.$ Let $\pazocal{S}=t_{0}t_{1}\ldots$ be a stream. A configuration of $\pazocal{C}$ over $\pazocal{S}$ is a tuple $(p,i,L)\in Q\times\mathbb{N}\times(2^{\Omega}\setminus\{\emptyset\})$, representing that the automaton $\pazocal{C}$, is at state $p$ after having read and marked $t_{i}$ with the set of labels $L$. For $\ell\in\Omega$, we say that $(p,i,L)$ marked position $i$ with $\ell$ iff $\ell\in L$. Given a position $n\in\mathbb{N}$, we say that a configuration is accepting at position $n$ iff it is of the form $(p,n,L)$ and $p\in F$. Then a run $\rho$ of $\pazocal{C}$ over $\pazocal{S}$ is a sequence of configurations:

$$\rho\ :=\ (p_{0},i_{0},L_{0}),(p_{1},i_{1},L_{1}),\ldots,(p_{n},i_{n},L_{n})$$

such that $i_{0}<i_{1}<\ldots<i_{n}$, $I(p_{0})=(U,L_{0})$ is defined and $t_{i_{0}}\in U$, and there exists a transition $(p_{j-1},U_{j},B_{j},L_{j},p_{j})\in\Delta$ such that $t_{i_{j}}\in U_{j}$ and $(t_{i_{j-1}},t_{i_{j}})\in B_{j}$ for every $j\in[1,n]$. Intuitively, a run of a CCEA is a subsequence of the stream that can follow a path of transitions, where each transition checks a local condition (i.e., the unary predicate $U_{j}$) and a join condition (i.e., the binary predicate $B_{j}$) with the previous tuple. For the first tuple, a CCEA can only check a local condition (i.e., there is no previous tuple).

Given a run $\rho$ like above, we define its valuation $\nu_{\rho}:\Omega\rightarrow 2^{\mathbb{N}}$ such that $\nu_{\rho}(\ell)$ is the set consisting of all positions in $\rho$ marked by $\ell$, formally, $\nu_{\rho}(\ell)=\{i_{j}\mid\,j\leq n\wedge\ell\in L_{j}\}$. Further, given a position $n\in\mathbb{N}$, we say that $\rho$ is an accepting run at position $n$ iff $(p_{n},i_{n},L_{n})$ is an accepting configuration at $n$. Then the output of $\pazocal{C}$ over $\pazocal{S}$ at position $n$ is defined as:

$${\lsem{}{\pazocal{C}}\rsem}_{n}(\pazocal{S})\ =\ \{\nu_{\rho}\mid\text{$\rho$ is an accepting run at position $n$ of $\pazocal{C}$ over $S$}\}.$$

Note that the definition of CCEA above differs from [16] to fit our purpose better. Specifically, we use a set of labels $\Omega$ to annotate positions in the streams and define valuations in the same spirit as the model of annotated automata used in [3, 23]. One can see this extension as a generalization to the model in [16], where $|\Omega|=1$. This extension will be helpful to enrich the outputs of our models for comparing them with hierarchical conjunctive queries with self-joins (see Section 4).

For our algorithms, we assume the computational model of Random Access Machines (RAM) with uniform cost measure, and addition as it basic operation [1, 15]. This RAM has read-only registers for the input, read-writes registers for the work, and write-only registers for the output. This computation model is a standard assumption in the literature [5, 6].

As it is common in the area [24], we define (unordered) trees as a finite set of strings $t\subseteq\mathbb{N}^{\ast}$ that satisfies two conditions: (1) $t$ contains the empty string, (i.e., $\varepsilon\in t$), and (2) $t$ is a prefix-closed set, namely, if $a_{1}...a_{n}\in t$, then $a_{1}...a_{j}\in t$ for every $j<n$. We will refer to the string of $t$ as nodes, and the root of a tree, $\texttt{root}(t)$, will be the empty string $\varepsilon$.

Let $\bar{u},\bar{v}\in t$ be nodes. The depth of $\bar{u}$ will be given by its length $0pt{t}{\bar{u}}=|\bar{u}|$. We say that $\bar{u}$ is the parent of $\bar{v}$ and write $\texttt{parent}_{t}(\bar{v})=\bar{u}$ if $\bar{v}=\bar{u}\cdot n$ for some $n\in\mathbb{N}$. Likewise, we say that $\bar{v}$ is a child of $\bar{u}$ if $\bar{u}$ is the parent of $\bar{v}$ and define $\texttt{children}_{t}(\bar{u})=\{\bar{v}\in t\mid\texttt{parent}_{t}(\bar{v})=\bar{u}\}$. Similarly, we define the descendants of $\bar{u}$ as $\texttt{desc}_{t}(\bar{u})=\{\bar{v}\in t\mid\bar{u}\preceq_{p}\bar{v}\}$ and the ancestors as $\texttt{ancst}_{t}(\bar{u})=\{\bar{v}\in t\mid\bar{v}\preceq_{p}\bar{u}\}$; note that $\bar{u}\in\texttt{desc}_{t}(\bar{u})$ and $\bar{u}\in\texttt{ancst}_{t}(\bar{u})$. A node $\bar{u}$ is a leaf of $t$ if $\texttt{desc}_{t}(\bar{u})=\{\bar{u}\}$, and an inner node if it is not a leaf node. We define the set of leaves of $\bar{u}$ as $\texttt{leaves}_{t}(\bar{u})=\{\bar{v}\in\texttt{desc}_{t}(\bar{u})\mid\bar{v}\text{ is a leaf node}\}$.

A labeled tree $\tau$ is a function $\tau\colon t\rightarrow L$ where $t$ is a tree and $L$ is any finite set of labels. We use $\operatorname{dom}(\tau)$ to denote the underlying tree structure $t$ of $\tau$. Given that $\tau$ is a function, we can write $\tau(\bar{u})$ to denote the label of node $\bar{u}\in\operatorname{dom}(\tau)$. To simplify the notation, we extend all the definitions above for a tree $t$ to labeled tree $\tau$, changing $t$ by $\operatorname{dom}(\tau)$. For example, we write $\bar{u}\in\tau$ to refer to $\bar{u}\in\operatorname{dom}(\tau)$, or $\texttt{parent}_{\tau}(\bar{u})$ to refer to $\texttt{parent}_{\operatorname{dom}(\tau)}(\bar{u})$. Finally, we say that two labeled trees $\tau$ and $\tau^{\prime}$ are isomorphic if there exists a bijection $f\colon\operatorname{dom}(\tau)\rightarrow\operatorname{dom}(\tau^{\prime})$ such that $\bar{u}\preceq_{p}\bar{v}$ iff $f(\bar{u})\preceq_{p}f(\bar{v})$ and $\tau(\bar{u})=\tau^{\prime}(f(\bar{u}))$ for every $\bar{u},\bar{v}\in\operatorname{dom}(\tau)$. We will usually say that $\tau$ and $\tau^{\prime}$ are equal, meaning they are isomorphic.

A Parallelized Finite Automaton (PFA) is a tuple $\mathcal{P}=(Q,\Sigma,\Delta,I,F)$ where $Q$ is a finite set of states, $\Sigma$ is a finite alphabet, $I,F\subseteq Q$ are the sets of initial and accepting states, respectively, and $\Delta\subseteq 2^{Q}\times\Sigma\times Q$ is the transition relation. We define the size of $\mathcal{P}$ as $|\mathcal{P}|=|Q|+\sum_{(P,a,q)}(|P|+1)$, namely, the number of states plus the size of encoding the transitions.

A run tree of a PFA $\mathcal{P}$ over a string $\bar{s}=a_{1}\ldots a_{n}\in\Sigma^{\ast}$ is a labeled tree $\tau:t\rightarrow\operatorname{dom}(\tau)$ such that $0pt{\tau}{\bar{u}}=n$ for every leaf $\bar{u}\in\tau$; in other words, every node of $\tau$ is labeled by a state of $\mathcal{P}$ and all branches have the same length $n$. In addition, $\tau$ must satisfy the following two conditions: (1) every leaf node $\bar{u}$ of $t$ is labeled by an initial state (i.e., $\tau(\bar{u})\in I$) and (2) for every inner node $\bar{v}$ at depth $i$ (i.e., $0pt{\tau}{\bar{v}}=i$) there must be a transition $(P,a_{n-i},q)\in\Delta$ such that $\tau(\bar{v})=q$, $|\texttt{children}_{\tau}(\bar{v})|=|P|$ and $P=\{\tau(\bar{u})\mid\bar{u}\in\texttt{children}_{\tau}(\bar{v})\}$, that is, children have different labels and $P$ is the set of labels in the children of $\bar{v}$. We say that $\tau$ is an accepting run of $\mathcal{P}$ over $\bar{s}$ iff $\tau$ is a run of $\mathcal{P}$ over $\bar{s}$ and $\tau(\varepsilon)\in F$ (recall that $\varepsilon=\texttt{root}(\tau)$). We say that $\mathcal{P}$ accepts a string $\bar{s}\in\Sigma^{\ast}$ if there is an accepting run of $\mathcal{P}$ over $\bar{s}$ and we define the language recognized by $\mathcal{P}$, $\pazocal{L}(\mathcal{P})$, as the set of strings that $\mathcal{P}$ accepts.

One can see that PFA is a generalization of an NFA. Indeed, NFA is a special case of an PFA where each run tree $\tau$ is a line. Nevertheless, PFA do not add expressive power to NFA, given that PFA is another model for recognizing regular languages, as the next result shows.

Intuitively, one could interpret a PFA as an Alternating Finite Automaton (AFA) [7] that runs backwards over the string (however, they still process the string in a forward direction). It was shown in [7, Theorem 5.2 and 5.3] that for every AFA that defines a language $L$ with $n$ states, there exists an equivalent DFA with $2^{2^{n}}$ states in the worst case that recognizes $L$. Nevertheless, they argued that the reverse language $L^{R}=\{a_{1}a_{2}\ldots a_{n}\in\Sigma^{\ast}\mid a_{n}\ldots a_{2}a_{1}\in L\}$ can always be accepted by a DFA with at most $2^{n}$ states. Then, one can see Proposition 3.2 as a consequence of reversing an alternating automaton. Despite this connection, we use here PFA as a proper automata model, which was not studied or used in [7]. Another related proposal is the parallel finite automata model presented in [25]. Indeed, one can consider PFA as a restricted case of this model, although it was not studied in [25]. For this reason, we decided to name the PFA model with the same acronym but a slightly different name as in [25].

A Parallelized Complex Event Automaton (PCEA) is the extension of CCEA with the idea of parallelization as in PFA. Specifically, a PCEA is a tuple $\pazocal{P}=(Q,\textbf{U},\textbf{B},\Omega,\Delta,F)$, where $Q$, U, B, $\Omega$, and $F$ are the same as for CCEA, and $\Delta$ is a finite transition relation of the form:

$$\Delta\subseteq 2^{Q}\times\textbf{U}\times\textbf{B}^{Q}\times(2^{\Omega}\setminus\{\emptyset\})\times Q.$$

where $\textbf{B}^{Q}$ are all partial functions $\mathcal{B}:Q\rightarrow\textbf{B}$, that associate a state $q$ to a binary predicate $\mathcal{B}(q)$. We define the size of $\pazocal{P}$ as $|\pazocal{P}|=|Q|+\sum_{(P,U,\mathcal{B},L,q)\in\Delta}(|P|+|L|)$. Note that $\pazocal{P}$ does not define the initial function explicitly. As we will see, transitions of the form $(\emptyset,U,\mathcal{B},L,q)$ will play the role of the initial function on a run of $\pazocal{P}$.

Next, we extend the notion of a run from CCEA to its parallelized version. Let $\pazocal{S}=t_{0}t_{1}\ldots$ be a stream. A run tree of $\pazocal{P}$ over $\pazocal{S}$ is now a labeled tree $\tau:t\rightarrow(Q\times\mathbb{N}\times(2^{\Omega}\setminus\{\emptyset\}))$ where each node $\bar{u}\in\tau$ is labeled with a configuration $\tau(\bar{u})=(q,i,L)$ such that, for every child $\bar{v}\in\texttt{children}_{\tau}(\bar{u})$ with $\tau(\bar{v})=(p,j,M)$, it holds that $j<i$. In other words, the positions of $\tau$-configurations increase towards the root of $\tau$, similar to the runs of a CCEA. In addition, $\bar{u}$ must satisfy the transition relation $\Delta$, that is, there must exist a transition $(P,U,\mathcal{B},L,q)\in\Delta$ such that (1) $t_{i}\in U$, (2) $|\texttt{children}_{\tau}(\bar{u})|=|P|$ and $P=\{p\mid\exists\bar{v}\in\texttt{children}_{\tau}(\bar{u}).\,\tau(\bar{v})=(p,j,M)\}$, and (3) for every $\bar{v}\in\texttt{children}_{\tau}(\bar{u})$ with $\tau(\bar{v})=(p,j,M)$, $(t_{j},t_{i})\in\mathcal{B}(p)$. Similar to PFA, condition (2) forces that there exists a bijection between $P$ and the states at the children of $\bar{u}$. Instead, condition (3) forces that two consecutive configurations $(p,j,M)$ and $(q,i,L)$ must satisfy the binary predicate in $\mathcal{B}(p)$ associated with $p$. Notice that, if $\bar{u}$ is a leaf node in $\tau$, then it must hold that $P=\emptyset$ and condition (3) is trivially satisfied. Also, note that we do not assume that all leaves are at the same depth.

Given a position $n\in\mathbb{N}$, we say that $\tau$ is an accepting run at position $n$ iff the root configuration $\tau(\varepsilon)$ is accepting at position $n$. Further, we define the output of a run $\tau$ as the valuation $\nu_{\tau}:\Omega\rightarrow 2^{\mathbb{N}}$ such that $\nu_{\tau}(\ell)=\{i\mid\,\exists\bar{u}\in\tau.\ \tau(\bar{u})=(q,i,L)\wedge\ell\in L\}$ for every label $\ell\in\Omega$. Finally, the output of a PCEA $\pazocal{P}$ over $\pazocal{S}$ at the position $n$ is defined as:

$${\lsem{}{\pazocal{P}}\rsem}_{n}(\pazocal{S})\ =\ \{\nu_{\tau}\mid\text{$\tau$ is an accepting run at position $n$ of $\pazocal{P}$ over $S$}\}.$$

It is easy to see that every CCEA is a PCEA where every transition $(P,U,\mathcal{B},L,q)\in\Delta$ satisfies that $|P|\leq 1$. Additionally, the previous example gives evidence that PCEA is a strict generalization of CCEA, namely, there exists no CCEA that can define $\pazocal{P}_{0}$. Intuitively, since a CCEA can only compare the current tuple to the last tuple, for a stream like $\pazocal{S}=R(a,b),T(a),S(a,b)$ it would be impossible to check conditions over the second attribute of tuples $R(a,b)$ and $S(a,b)$.

We end this section by introducing a subclass of PCEA relevant to our algorithmic results. Let $\pazocal{P}$ be a PCEA and $\tau$ a run of $\pazocal{P}$ over some stream. We say that $\tau$ is simple iff for every two different nodes $\bar{u},\bar{u}^{\prime}\in\tau$ with $\tau(\bar{u})=(q,i,L)$ and $\tau(\bar{u}^{\prime})=(q^{\prime},i^{\prime},L^{\prime})$, if $i=i^{\prime}$, then $L\cap L^{\prime}=\emptyset$. In other words, $\tau$ is simple if all positions of the valuation $\nu_{\tau}$ are uniquely represented in $\tau$. We say that $\pazocal{P}$ is unambiguous if (1) every accepting run of $\pazocal{P}$ is simple and (2) for every stream $\pazocal{S}$ and accepting run $\tau^{\prime}$ of $\pazocal{P}$ over $\pazocal{S}$ with valuation $\nu_{\tau}$, there is no other run $\tau$ of $\pazocal{P}$ with valuation $\nu_{\tau^{\prime}}$ such that $\nu_{\tau}=\nu_{\tau^{\prime}}$. For example, the reader can check that $\pazocal{P}_{0}$ is unambiguous.

Condition (2) of unambiguous PCEA ensures that each output is witnessed by exactly one run. This condition is common in MSO enumeration [2, 22] for a one-to-one correspondence between outputs and runs. Condition (1) forces a correspondence between the size of the run and the size of the output it represents. As we will see, both conditions will be helpful for our evaluation algorithm, and satisfied by our translation of hierarchical conjunctive queries into PCEA in the next section.

A bag (also called a multiset) is usually defined in the literature as a function that maps each element to its multiplicity (i.e., the number of times it appears). In this work, we use a different but equivalent representation of a bag where each element has its own identity. This representation will be helpful in our context to deal with duplicates in the stream and define the semantics of hierarchical CQ in the case of self joins.

We define a bag (with own identity) $B$ as a surjective function $B:I\rightarrow U$ where $I$ is a finite set of identifiers (i.e., the identity of each element) and $U$ is the underlying set of the bag. Given any bag $B$, we refer to these components as $I(B)$ and $U(B)$, respectively. For example, a bag $B=\{\!\!\{a,a,b\}\!\!\}$ (where $a$ is repeated twice) can be represented with a surjective function $B_{0}=\{0\mapsto a,1\mapsto a,2\mapsto b\}$ where $I(B_{0})=\{0,1,2\}$ and $U(B_{0})=\{a,b\}$. In general, we will use the standard notation for bags $\{\!\!\{a_{0},\ldots,a_{n-1}\}\!\!\}$ to denote the bag $B$ whose identifiers are $I(B)=\{0,\ldots,n-1\}$ and $B(i)=a_{i}$ for each $i\in I(B)$. Note that if $B:I\rightarrow U$ is injective, then $B$ encodes a set (i.e., no repetitions). We write $a\in B$ if $B(i)=a$ for some $i\in I(B)$ and define the empty bag $\emptyset$ such that $I(\emptyset)=\emptyset$ and $U(\emptyset)=\emptyset$.

For a bag $B$ and an element $a$, we define the multiplicity of $a$ in $B$ as $\operatorname{mult}_{B}(a)=|\{i\mid B(i)=a\}|$. Then, we say that a bag $B^{\prime}$ is contained in $B$, denoted as $B^{\prime}\subseteq B$, iff $\operatorname{mult}_{B^{\prime}}(a)\leq\operatorname{mult}_{B}(a)$ for every $a$. We also say that two bags $B^{\prime}$ and $B$ are equal, and write $B=B^{\prime}$, if $B^{\prime}\subseteq B$ and $B\subseteq B^{\prime}$. Note that two bags can be equal although the set of identifiers can be different (i.e., they are equal up to a renaming of the identifiers). Given a set $A$, we say that $B$ is a bag from elements of $A$ (or just a bag of $A$) if $U(B)\subseteq A$.

Recall that D is our set of data values and let $\sigma=(\textbf{T},\operatorname{arity})$ be a schema. A relational database $D$ (with duplicates) over $\sigma$ is a bag of $\operatorname{Tuples}[\sigma]$. Given a relation name $R\in\textbf{T}$, we write $R^{D}$ as the bag of $D$ containing only the $R$-tuples of $D$, formally, $I(R^{D})=\{i\in I(D)\mid D(i)=R(\bar{a})\text{ for some $\bar{a}$}\}$ and $R^{D}(i)=D(i)$ for every $i\in I(R^{D})$. For example, consider again the schema $\sigma_{0}$. Then a database $D_{0}$ over $\sigma_{0}$ is the bag:

$$D_{0}\ :=\ \{\!\!\{\,S(2,11),T(2),R(1,10),S(2,11),T(1),R(2,11)\,\}\!\!\}.$$

Here, one can check that $T^{D_{0}}=\{\!\!\{T(2),T(1)\}\!\!\}$ and $S^{D_{0}}=\{\!\!\{S(2,11),S(2,11)\}\!\!\}$.

Fix a schema $\sigma=(\textbf{T},\operatorname{arity})$ and a set of variables X disjoint from D (i.e., $\textbf{X}\cap\textbf{D}=\emptyset$). A Conjunctive Query (CQ) over relational schema $\sigma$ is a syntactic structure of the form:

$$Q(\bar{x})\ \leftarrow\ R_{0}(\bar{x}_{0}),\ldots,R_{m-1}(\bar{x}_{m-1})$$

such that $Q$ is a relational name not in T, $R_{i}\in\textbf{T}$, $\bar{x}_{i}$ is a sequence of variables in X and data values in D, and $|\bar{x}_{i}|=\operatorname{arity}(R_{i})$ for every $i<m$. Further, $\bar{x}$ is a sequence of variables in $\bar{x}_{0},\ldots,\bar{x}_{m-1}$. We will denote a CQ like (4) by $Q$, where $Q(\bar{x})$ and $R_{0}(\bar{x}_{0}),\ldots,R_{m-1}(\bar{x}_{m-1})$ are called the head and the body of $Q$, respectively. Furthermore, we call each $R_{i}(\bar{x}_{i})$ an atom of $Q$. For example, the following are two conjunctive queries $Q_{0}$ and $Q_{1}$ over the schema $\sigma_{0}$:

$$Q_{0}(x,y)\leftarrow T(x),\,S(x,y),\,R(x,y)\ \ \ \ \ \ \ \ \ \ \ \ Q_{1}(x,y)\leftarrow T(x),\,R(x,y),\,S(2,y),\,\,T(x)$$

Note that a query can repeat atoms. For this reason, we will regularly consider $Q$ as a bag of atoms, where $I(Q)$ are the positions of $Q$ and $U(Q)$ is the set of distinct atoms. For instance, we can consider $Q_{1}$ above as a bag of atoms, where $I(Q_{1})=\{0,1,2,3\}$ (i.e., the position of the atoms) and $Q_{1}(0)=T(x)$, $Q_{1}(1)=R(x,y)$, $Q_{1}(2)=S(2,y)$, $Q_{1}(3)=T(x)$. We say that a CQ $Q$ has self joins if there are two atoms with the same relation name. We can see in the previous example that $Q_{1}$ has self joins, while $Q_{0}$ does not.

Let $Q$ be a CQ, and $D$ be a database over the same schema $\sigma$. A homomorphism is any function $h:\textbf{X}\cup\textbf{D}\rightarrow\textbf{D}$ such that $h(a)=a$ for every $a\in\textbf{D}$. We extend $h$ as a function from atoms to tuples such that $h(R(\bar{x})):=R(h(\bar{x}))$ for every atom $R(\bar{x})$. We say that $h$ is a homomorphism from $Q$ to $D$ if $h$ is a homomorphism and $h(R(\bar{x}))\in D$ for every atom $R(\bar{x})$ in $Q$. We denote by $\operatorname{Hom}(Q,D)$ the set of all homomorphisms from $Q$ to $D$.

To define the bag semantics of CQ, we need a more refined notion of homomorphism that specifies the correspondence between atoms in $Q$ and tuples in $D$. Formally, a tuple-homomorphism from $Q$ to $D$ (or t-homomorphism for short) is a function $\eta:I(Q)\rightarrow I(D)$ such that there exists a homomorphism $h_{\eta}$ from $Q$ to $D$ satisfying that $h_{\eta}(Q(i))=D(\eta(i))$ for every $i\in I(Q)$. For example, consider again $Q_{0}$ and $D_{0}$ above, then $\eta_{0}=\{0\mapsto 1,1\mapsto 3,2,\mapsto 5\}$ and $\eta_{1}=\{0\mapsto 1,1\mapsto 0,2\mapsto 5\}$ are two t-homomorphism from $Q_{0}$ to $D_{0}$.

Intuitively, a t-homomorphism is like a homomorphism, but it additionally specifies the correspondence between atoms (i.e., $I(Q)$) and tuples (i.e., $I(D)$) in the underlying bags. One can easily check that if $\eta$ is a t-homomorphism, then $h_{\eta}$ (restricted to the variables of $Q$) is unique. For this reason, we usually say that $h_{\eta}$ is the homomorphism associated to $\eta$. Note that the converse does not hold: for $h$ from $Q$ to $D$, there can be several t-homomorphisms $\eta$ such that $h=h_{\eta}$.

Let $Q(\bar{x})$ be the head of $Q$. We define the output of a CQ $Q$ over a database $D$ as:

$${\lsem{}{Q}\rsem}(D)\ =\ \{\!\!\{Q(h_{\eta}(\bar{x}))\,\mid\,\text{$\eta$ is a t-homomorphism from $Q$ to $D$}\}\!\!\}.$$

Note that the result is another relation where each $Q(h_{\eta}(\bar{x}))$ is witnessed by a t-homomorphism from $Q$ to $D$. In other words, there is a one-to-one correspondence between tuples in ${\lsem{}{Q}\rsem}(D)$ and t-homomorphisms from $Q$ to $D$.

In the literature, homomorphisms are usually used to define the set semantics of a CQ $Q$ over a database $D$. They are helpful for set semantics but “inconvenient” for bag semantics since it does not specify the correspondence between atoms and tuples; namely, they only witness the existence of such correspondence. In [8], Chaudhuri and Vardi introduced the bag semantics of CQ by using homomorphisms, which we recall next. Let $Q$ be a CQ like (4) and $D$ a database over the same schema $\sigma$, and let $h\in\operatorname{Hom}(Q,D)$. We define the multiplicity of $h$ with respect to $Q$ and $D$ by:

$$\operatorname{mult}_{Q,D}(h)\ =\ \prod_{i=0}^{m-1}\operatorname{mult}_{D}(h(R_{i}(\bar{x}_{i})))$$

Chaudhuri and Vardi defined the bag semantics $\lceil\lceil{Q}\rfloor\rfloor$ of $Q$ over $D$ as the bag $\lceil\lceil{Q}\rfloor\rfloor(D)$ such that each tuple $Q(\bar{a})$ has multiplicity equal to:

$$\operatorname{mult}_{\lceil\lceil{Q}\rfloor\rfloor(D)}(Q(\bar{a}))\ =\ \sum_{h\in\operatorname{Hom}(Q,D)\,\colon\,h(\bar{x})=\bar{a}}\operatorname{mult}_{Q,D}(h)$$

In Appendix B, we prove that for every CQ $Q$ and database $D$ it holds that ${\lsem{}{Q}\rsem}(D)=\lceil\lceil{Q}\rfloor\rfloor(D)$, namely, the bag semantics introduced here (i.e., with t-homomorphisms) is equivalent to the standard bag semantics of CQ. The main difference is that the standard bag semantics of CQ are defined in terms of homomorphisms and multiplicities, and there is no direct correspondence between outputs and homomorphisms. For this reason, we redefine the bag semantics of CQ in terms of t-homomorphism that will connect the outputs of CQ with the outputs of PCEA over streams.

Now, we define the semantics of CQ over streams, formalizing its comparison with queries in complex event recognition. For this purpose, we must show how to interpret streams as databases and encode CQ’s outputs as valuations. Fix a schema $\sigma$ and a stream $\pazocal{S}=t_{0}t_{1}\cdots$ over $\sigma$. Given a position $n\in\mathbb{N}$, we define the database of $\pazocal{S}$ at position $n$ as the $\sigma$-database $D_{n}[\pazocal{S}]=\{\!\!\{t_{0},t_{1},\ldots,t_{n}\}\!\!\}$. For example, $D_{5}[\pazocal{S}_{0}]=D_{0}$. One can interpret here that $\pazocal{S}$ is a sequence of inserts, and then $D_{n}[\pazocal{S}]$ is the database version at position $n$. Since $D_{n}[\pazocal{S}]$ is a bag, the identifiers $I(D_{n}[\pazocal{S}])$ coincide with the positions of the sequence $t_{0}\ldots t_{n}$.

Let $Q$ be a CQ over $\sigma$, and let $\eta:I(Q)\rightarrow I(D_{n}[\pazocal{S}])$ be a t-homomorphism from $Q$ to $D_{n}[\pazocal{S}]$. If we consider $\Omega=I(Q)$, we can interpret $\eta$ as a valuation $\hat{\eta}:\Omega\rightarrow 2^{\mathbb{N}}$ that maps each atom of $Q$ to a set with a single position; formally, $\hat{\eta}(i)=\{\eta(i)\}$ for every $i\in I(Q)$. Then, we define the semantics of $Q$ over stream $\pazocal{S}$ at position $n$ as:

$${\lsem{}{Q}\rsem}_{n}(\pazocal{S})\ =\ \{\hat{\eta}\mid\text{$\eta$ is a t-homomorphism from $Q$ to $D_{n}[\pazocal{S}]$}\}$$

Note that ${\lsem{}{Q}\rsem}_{n}(\pazocal{S})$ is equivalent to evaluating $Q$ over $D_{n}[\pazocal{S}]$ where instead of outputting a bag of tuples ${\lsem{}{Q}\rsem}(D_{n}[\pazocal{S}])$, we output the t-homomorphisms (i.e., as valuations) that are in a one-to-one correspondence with the tuples in ${\lsem{}{Q}\rsem}(D_{n}[\pazocal{S}])$.