VERCEL: Verification and Rectification of Configuration Errors with Least Squares

We now illustrate the functionality of Vercel with an example. Figure 1(a) is a toy network of 4 routers. For simplicity, we consider 3-bit headers though generalization to more bits is trivial. In this example, Vercel checks for reachability between source router $Y$ and destination $R$. The forwarding table at router $U$ shows that packets with header 000/3 and 01/2 are forwarded along output ports $0$ and $1$, respectively. Vercel collects rules from the forwarding tables and utilizes a binary tree to arrange packet headers (as in Figure 1(a)). While doing so, Vercel also keeps track of (router, port) pairs for each rule. The 4 leaf nodes of the binary tree $\{a1,\;q2,\;a3,\;a4\}$ denote non-overlapping headers $\{000/3,\;001/3,\;01/2,\;1/1\}$. In the figure, the node labels, i.e., $r$ (supernet), $a$ (atomic) and $q$ (iatomic) denote a specific class of headers defined later in Section III-B.

Now assume a network administrator inserts a new rule at router $Q$ (as shown in Figure 1(a)), which forwards packets with header 0/1 on its output port $0$ (shown by dashed block in the forwarding table at router $Q$). Vercel detects the rule update and inserts the corresponding header 0/1 in the binary tree. Post insertion, Vercel now traverses the headers in its subtrees to check if any of them were impacted for reachability during the update. There are three such (impacted) headers i.e., 000/3, 001/3, 01/2, which are added to a set, $S\textsuperscript{affected}$. Vercel creates 3-dimensional orthogonal vectors [1, 0, 0], [0, 1, 0] and [0, 0, 1] corresponding to impacted headers 001/3, 000/3 and 01/2. For simplicity, we have selected orthogonal vectors with binary values. Next, Vercel creates a vector $b_{init}=$ [1, 1, 1] that corresponds to three non-overlapping headers $\{q2,\;a1,\;a3\}$ for which we want to evaluate reachability. After creating $b_{init}$, Vercel creates subspaces for port 0 at routers $Y$, $U$ and $Q$. This is because along port 0, these routers forward some packets whose headers are in the set $S\textsuperscript{affected}$. Figure 1(b) shows $x$-$y$ and $y$-$z$ planes corresponding to the $(router,port)$ pairs $\{(Y,0),\;(U,0)\}$, forming two subspaces. Since router $Y$ forwards packets with header 001/3 and 000/3 on its port $0$, therefore its subspace ($x$-$y$ plane) is defined by a matrix $\mathcal{A}_{Y}^{0}$. Similarly, router $U$ forwards packets with header 000/3 and 01/2 on its port $0$, therefore its subspace ($y$-$z$ plane) is defined by a matrix $\mathcal{A}_{U}^{0}$.

After creating the subspaces for different ports, Vercel evaluates reachability by modelling packet forwarding at router $Y$. This is done by solving $\mathcal{A}_{Y}^{0}x=b_{init}$. At router $Y$, vector $b_{init}$ neither lies in the column space ($C(\mathcal{A}_{Y}^{0})$) nor in the null space of $(\mathcal{A}_{Y}^{0})^{T}$ ($Y$ in superscript denotes the matrix transpose). Equivalently, $b_{init}$ lies at the union of the column and null space of $\mathcal{A}_{Y}^{0}$ and $(\mathcal{A}_{Y}^{0})^{T}$, respectively. The above scenario implies that no solution is possible for $\mathcal{A}_{Y}^{0}x=b_{init}$. In this case, least squares finds an approximate solution resulting in a projection point $b_{Y}=$ [1, 1, 0] in $C(\mathcal{A}_{Y}^{0})$, i.e., $x$-$y$ plane. The point $b_{Y}$ implies that router $Y$ forwards packets with headers 001/3 and 000/3 through port 0, which are then received by router $U$ on path $Y-U$. Since packets with header 01/2 are not forwarded from router $Y$, therefore the third index (corresponding to header 01/2) of vector $b_{Y}$ contains a 0 entry. Similarly, Vercel models the packet forwarding at router $U$ at port $0$ by solving $\mathcal{A}_{U}^{0}x=b_{Y}$ and resulting in a projection point $b_{U}=$ [0, 1, 0] in the $y$-$z$ plane, indicating that router $U$ only forwards packets with header 000/3 through port $0$. Since router $R$ receives a non-zero vector $b_{U}$ from router $U$, therefore Vercel confirms reachability between router $Y$ and $R$ along the path $Y-U-R$. Finally, at the destination (router $R$), Vercel determines the set of reachable packet headers by computing the dot product between $b_{U}$ and vector representation of headers in the set $S\textsuperscript{affected}$. For this example, Vercel obtains a non-zero dot product between $b_{U}$ and vector [0, 1, 0] (corresponding to the header 000/3), implying packets with header 000/3 are reachable from router $Y$ to $R$.

Using least squares, we check for reachability and express intent to establish a service between two nodes in the network. If reachability between these two nodes does not exist, then Vercel recommends a path (Section V-E) and rectifies the situation by automatically populating tables at interim routers leading to reachability. The fact that misconfigurations are not just found, but also rectified and that the rectification is a direct by-product of $\mathcal{A}x=b$ using least squares, is a significant value addition to operators. An operator only has to express a high level intent for a bunch of services and not worry about how these are correctly provisioned. The correctness is taken care of by Vercel.

Let us assume we performed least squares on $\mathcal{A}x=b$ for a port and identified that our target vector $b$ is in the null space of $\mathcal{A}^{T}$. Which means “all” the headers represented by $b$ are not forwarded along that port. Now, if we were to ensure reachability for any of the headers in $b$, then $b$ must be in the column space of $\mathcal{A}$. The simplest way of doing this is to replace $\mathcal{A}$ with an augmented matrix $[\mathcal{A}|b]$.

We now describe rectification via the following example resulting from a users’ intent. Consider a four-node network consisting of routers $Y$, $U$, $Q$ and $R$, shown on the right side of Fig. 2. Forwarding tables at each of the routers and corresponding vector notations for their non-overlapping headers are also shown in the figure. Assume an intent by the user whose goal is to set up a service between nodes $Y$ and $R$. Vercel is responsible for parsing the intent statement to extract information such as source ($Y$), destination ($R$), and action (service provisioning). Vercel then examines the configuration of routers $Y$ and $U$, generates forwarding matrices for both, and initializes $b_{init}$ (all prefixes) for reachability check. Upon analysis, Vercel determines that destination $R$ is currently unreachable from source $Y$ (as there are no forwarding rules installed at routers $U$ and $Q$ to forward packets received from $Y$ to $R$). In response, Vercel enables its rectification engine and proposes a solution based on linear algebraic operations. The activated rectification engine creates a new rule (01/2, Port 1) at router $Q$ to establish reachability. The verification and rectification engines of Vercel work together for this intent, and the flow of operations is shown in Fig. 2. A detailed explanation of the linear algebraic operations performed for rectification is provided below.

In Fig. 3 is an example demonstrating Vercel’s rectification system for the intent expressed in Fig. 2. We apply Vercel to verify reachability, for which matrices $\mathcal{A}$ and vector $b$ are created as shown in Fig. 3. After applying $\mathcal{A}x=b$ at router $Y$ and $U$, we get $b_{U}$, which is a set of prefixes reachable from $Y$ to $Q$ (through path $Y-U-Q$). Now at router $Q$, while applying $\mathcal{A}_{Q}x=b_{U}$, Vercel finds that $b_{U}$ is in the null space of $\mathcal{A}_{Q}^{T}$, which implies none of the prefixes in $b_{U}$ will be forwarded towards $R$. At this point, our recommendation system kicks in and suggests a “fix” to address the fault at router $Q$, such that reachability between $Y$ and $R$ is established. As part of the fix, Vercel’s recommendation system adds $b_{U}$ as a new column of matrix $\mathcal{A}_{Q}$ (and updates the corresponding forwarding rules at router $Q$). Now, the updated matrix $\mathcal{A}_{Q}$ has three columns. This solves the problem as when we evaluate $\mathcal{A}_{Q}x=b_{U}$, $b_{U}$ will be in the column space of $\mathcal{A}_{Q}$. As a result of $\mathcal{A}_{Q}x=b_{U}$, we get $b_{Q}=[0,1,0]$, which is a vector that represents the prefixes reachable till destination $R$. Since this vector is now non-zero, a few prefixes are now reachable at the destination after applying the fix.

Nevertheless, employing rectification to address issues may inadvertently result in unintended consequences, such as rendering some destinations unreachable. To mitigate this, our rectification system operates based on the following principle: when implementing a fix (by adding a rule), the reachability of any current header should remain unaffected. For this, identifying the appropriate header is crucial whenever a rule update is conducted as part of rectification. In this regard, Vercel opts for a header that was previously unreachable between the source and destination pairs of interest. In accordance with this principle, the following procedure is followed during the rectification process.

Assume the rectification system suggests adding a rule with header $z$ in one of the routers’ forwarding tables. Now, Vercel calculates if there exist any reachable headers $z’$ (for some other source-destination pairs) in the same table that overlap with $z$. Vercel then performs $z-z’$ to check if the resultant set is non-empty. The non-emptiness here implies the existence of the non-reachable headers between source and destination. If so, it configures the corresponding non-empty set as a separate rule in the forwarding table. In case the result ($z-z’$) is empty, then Vercel attempts to find a header (other than $z$) and repeats the process. If we exhaust all possible ‘z’s through computation and still cannot find a non-empty $z-z’$, then Vercel notifies that rectification is not possible. By computing such $z-z’$, Vercel ensures that while adding a rule during the rectification process, it does not impact the reachability of any existing header.

Initially, Vercel queries routers or an SDN controller to obtain forwarding tables and compute the network topology as shown in Fig. 4. Thereafter, Vercel arranges all the existing packet headers (extracted from rules in the forwarding tables) in the form of a binary tree. For adding a new header, Vercel first identifies its correct position in the tree by traversing the tree based on $L$-header bits. These bits form a path from the root to a prospective node, where the new header is to be inserted. At every level (in the tree) the corresponding header bit indicates the traversal direction (0 = left and 1 = right branch). A path from the root to an intermediate/leaf node corresponds to a specific packet header. The role of the binary tree is in efficiently dividing the packet header space into non-overlapping partitions. Each packet belonging to a partition observes similar processing at all the routers in the network. The binary-tree enabled succinct representation provides Vercel a way in dividing the packet header space into three categories: supernet, atomic and iatomic. Out of these, atomic and iatomic categories together represent all the non-overlapping headers extracted from the forwarding tables across the network, while supernet represents the union of one or more atomic and iatomic headers.

Vercel labels a packet header as a supernet, if the corresponding path snippet from the root terminates at a non-leaf node. The remaining packet headers extracted from the rules (whose corresponding paths from the root terminate at a leaf node) are denoted as atomic. Due to the hierarchy in the header representation, multiple headers (e.g., supernet and atomic) may overlap. Therefore, Vercel further fragments supernets to create a new category of headers (called induced-atomic headers (iatomic) that do not overlap with the atomic headers. For example, in Figure 1(a) we observe 3 atomic headers (labeled as $a$), 1 iatomic header (labeled as $q$) and 3 supernets (labeled as $r$) in the binary tree. Note that the atomic and iatomic headers represent the leaf nodes of a binary tree.

In addition to the attribute of label types ($r/a/q$), each node of the binary tree is also characterized by a list containing tuples. A tuple $(i,\;p)$ at node $k$ denotes that router $i$ contains a rule that maps a packet header (represented by a path from the root to node $k$) to an output port $p$. In addition, Vercel assigns an integer identifier to each leaf node that distinguishes headers within the set of atomic and iatomic headers.

So far, we have defined atomic and iatomic headers; however, a crucial aspect is to analyze the task of identifying and segregating them once the tree is constructed. The following Lemma and Theorem provide a theoretical understanding of the procedure and complexities involved in determining the atomic and iatomic headers (the reader may skip the proofs for continuity).

Lemma 1: The worst-case space and time complexity of finding all atomic headers in a network containing $m$ forwarding rules is $\mathcal{O}(m)$.

Proof of Lemma 1: In the worst case, $m$ forwarding rules are non-overlapping with each other, and therefore, corresponding headers represent leaf nodes of the binary tree. In this case, Vercel labels all leaf nodes ($m$) as atomic by performing a post-order traversal of the tree, implying that the space complexity of the atomic headers is $\mathcal{O}(m)$. A post-order traversal of the binary tree with $m$ leaf nodes can be performed in $\mathcal{O}(m)$ time. Therefore, the time complexity of identifying atomic headers is also $\mathcal{O}(m)$.

Theorem 1: Given $m$ forwarding rules and a set of atomic headers having space complexity of $\mathcal{O}(m)$, the space complexity of the smallest size induced-atomic set is $\mathcal{O}(m)$ and it can be computed with the time complexity of $\mathcal{O}(m)$.

Proof of Theorem 1: Space complexity of iatomic nodes: Consider a general case, in which multiple (say $u$) leaf nodes (atomic headers) have an ancestor supernet ($r$) with a maximum path length of $v$ between a supernet and an atomic header. To represent all headers of the supernet ($r$) using non-overlapping partitions, Vercel introduces a maximum of $uv$ number of new iatomic headers in the tree along the path between $r$ and $v$. In the extreme case, when a supernet is a root node of the binary tree and all other headers are atomic, the number of iatomic nodes cannot be greater than $\mathcal{O}(mw)$, where $w$ is the maximum path length between the root node and a leaf node and $m$ is the number of rules in the forwarding tables across the network. A similar argument holds for multiple supernets having a common ancestor (as a supernet) in the binary tree. Since $w$ is dependent on the header length (e.g., an IP prefix has a maximum value of 32 in case of IPv4) and $w\ll m$, therefore $w$ is treated as a constant and space complexity of iatomic headers becomes $\mathcal{O}(m)$.

Time complexity of adding iatomic nodes: Vercel creates all iatomic headers (new leaf nodes) by performing the post-order traversal of the binary tree. While doing so, Vercel checks whether the current node of the tree is labeled as a supernet. If the current node is labeled as a supernet, then for all descendant nodes having a single child, Vercel creates a second child and labels it as iatomic (constant time operation). Since the binary tree has the space complexity $\mathcal{O}(m)$ (with $m$ rules and $\mathcal{O}(m)$ atomic headers); therefore, the time complexity of post-order traversal along with the addition of new leaf nodes (iatomic headers) is $\mathcal{O}(m)$.

Vercel is always tuned into the network to intercept any rule update at the routers. Once Vercel identifies an update, it computes whether reachability between different source-destination pairs is intact. For this, Vercel determines corresponding atomic+iatomic headers from the created binary tree, whose reachability may be affected. Vercel denotes all such headers in a set $S\textsuperscript{affected}$. To create $S\textsuperscript{affected}$, Vercel traverses a path of length $L$ from the root to a node $o_{j}$, directed by the $L$ bits of the header (specified by the rule being inserted/deleted). Thereafter, Vercel traverses subtrees of node $o_{j}$. During this traversal, if Vercel observes an atomic/iatomic node, then it appends the node’s identifier in the set $S\textsuperscript{affected}$. Besides the set $S\textsuperscript{affected}$, Vercel creates another set $P\textsuperscript{affected}$, which contains those ports on which a router forwards the packet with the header in the set $S\textsuperscript{affected}$. Since nodes of the binary tree store information of the router’s identifier ($i$) and corresponding port ($p$) in the form of a tuple ($i,\;p$); therefore, during the tree traversal (as described for $S\textsuperscript{affected}$), Vercel collects tuples from the nodes labeled as atomic/supernet and includes those in the set $P\textsuperscript{affected}$.

The following theorem shows that it is possible to identify sets $S\textsuperscript{affected}$ and $P\textsuperscript{affected}$ in linear time, which justifies the fast verification time of Vercel (Section VI).

Theorem 2: The worst case time complexity to identify both the sets $S\textsuperscript{affected}$ and $P\textsuperscript{affected}$ is $\mathcal{O}(m)$, where $m$ is the number of atomic+iatomic headers in the network.

Proof of Theorem 2: Insertion of a rule requires finding affected headers and relevant router ports (Section III.B). Insertion of a header (corresponding to a new rule) in the binary tree requires $\mathcal{O}(m)$ time for traversing a path length of $L$ (for $L$ bits header) from the root to a node $o_{j}$ and visiting all child nodes of $o_{j}$ in order to create iatomic nodes in the tree (worst case scenario). While traversing the tree, Vercel finds all overlapping rules in a router, creates the set $S\textsuperscript{affected}$ by appending the identifier of the leaves and identifies relevant router ports $P\textsuperscript{affected}$ by collecting the $(i,\;p)$ tuples. This tree traversal a) from the root to node $o_{j}$ and b) subtrees of $o_{j}$ can be done with the time complexity of $\mathcal{O}(m)$. Therefore, after a rule insertion, the time complexity to create the sets $S\textsuperscript{affected}$ and $P\textsuperscript{affected}$ is $\mathcal{O}(m)$. A similar argument holds for the rule deletion that has a time complexity of $\mathcal{O}(m)$.

After obtaining sets $S\textsuperscript{affected}$ and $P\textsuperscript{affected}$, Vercel represents atomic+iatomic headers from the set $S\textsuperscript{affected}$ in a vector space of $m$-dimensions. For that, Vercel computes the number of headers in the set $S\textsuperscript{affected}$ i.e., $m=|S\textsuperscript{affected}|$. Thereafter, Vercel defines an $m$-dimensional space by creating $m$ orthogonal vectors and uniquely maps these vectors to $m$ atomic+iatomic headers in the set $S\textsuperscript{affected}$. Orthogonality is essential to model the packet forwarding at routers with least squares (Section III-E). The set of orthogonal vectors can be generated using the Gram-Schmidt [28] process (in which we take any vector, and then using that as a reference, make all other vectors orthogonal, one vector at a time). Later, we shall do away with requirement of orthogonal vectors while achieving linear time reachability computations (Section IV).

After making an $m$-dimensional space, Vercel makes subspaces for each port. A port’s subspace includes packets with atomic and iatomic headers that a router sends to that port. To make these subspaces, Vercel goes through each ($i,\;p$) tuple in the set $P\textsuperscript{affected}$. For each tuple, it picks packets with atomic and iatomic headers from the set $S\textsuperscript{affected}$ that router $i$ sends through output port $p$. If router $i$ sends packets with a unique set of $1\leq n\leq m$ headers through port $p$, Vercel creates a subspace for the ($i,\;p$) tuple. To do so, Vercel picks $n$ orthogonal vectors and stores them in a matrix $\mathcal{A}_{i}^{p}$ of size $(m,\;n)$. In addition to $\mathcal{A}_{i}^{p}$, Vercel makes a vector $b_{init}$ with $m$ dimensions. This vector shows the packet headers being evaluated for reachability between a source and destination.

After creating subspaces, Vercel utilizes ports in the set $P\textsuperscript{affected}$ to traverse different paths between the source and destination for evaluating reachability. Assume source $S$, destination $D$ and an interim router $i$, denoted by path: $S,...,i-1,i,i+1,...,D$. Also, assume that Vercel has computed packets (with headers in $S\textsuperscript{affected}$) that are reachable up to router $i$ from $S$. These packet headers are represented by vector $b_{i-1}$. Now, Vercel aims to identify those headers in $b_{i-1}$ that router $i$ forwards to $i+1$ through one of its output ports $p$. Algebraically, this situation can be achieved by first solving $\mathcal{A}_{i}^{p}x_{i}^{p}=b_{i-1}$ to obtain $\hat{x}_{i}^{p}$. Thereafter, with $\hat{x}_{i}^{p}$, Vercel obtains an orthogonal projection of $b_{i-1}$ onto the column space of $\mathcal{A}_{i}^{p}$ (denoted as $C(\mathcal{A}_{i}^{p})$). The column space of a matrix denotes those points that can be obtained through a linear combination of columns of the matrix. The efficient way to solve $\mathcal{A}_{i}^{p}x_{i}^{p}=b_{i-1}$ is least squares, which always guarantees to provide a solution in real-time. While obtaining the projection of $b_{i-1}$ in $C(\mathcal{A}_{i}^{p})$ using least squares, there can be three possible outcomes:

Case 1: This is the dominant case in which router $i$ forwards some of the received packets along an output port $p$. This is because the forwarding table has some of the matching rules for all incoming packets. Mathematically, $b_{i-1}$ is partially both in the column space of $\mathcal{A}_{i}^{p}$ and the null space of $(\mathcal{A}_{i}^{p})^{T}$. The null space of a matrix denotes points that are orthogonal to all rows of the matrix. In other words, $b_{i-1}$ is a linear combination of basis vectors from $C(\mathcal{A}_{i}^{p})$ and $N((\mathcal{A}_{i}^{p})^{T}))$. As a consequence, there exists no solution to the linear equations $\mathcal{A}_{i}^{p}x_{i}^{p}=b_{i-1}$. Therefore, we utilize least squares to obtain the best approximate solution ($\hat{x}_{i}^{p}$). Subsequently, $\hat{x}$ is used to obtain a projection point $b_{i}=\mathcal{A}_{i}^{p}\hat{x}_{i}^{p}$. Now, the projected point $b_{i}$ (present in $C(\mathcal{A}_{i}^{p})$), represents a subset of packet headers received at router $i$ (from the previous router $i-1$) which are being forwarded along output port $p$. For an example of this case, refer to Fig. 1(b), where nodes $Y$ and $U$ are forwarding some of the prefixes in $b_{i}$.

Case 2: In this case, router $i$ forwards all the received packets along its output port $p$. Equivalently, $b_{i-1}$ is present in the column space of $\mathcal{A}_{i}^{p}$. In this case, the method of least squares returns an exact solution $x_{i}^{p}$. The projected point $b_{i}=\mathcal{A}_{i}^{p}x_{i}^{p}$ contains information on all atomic+iatomic headers present in $b_{i-1}$. Therefore, with least squares, Vercel models packet forwarding for all headers in $b_{i-1}$ to the next router $i+1$ along the port $p$ at router $i$.

Case 3: In this case, router $i$ blocks all the received packets along its output port $p$, implying $b_{i-1}$ lies in the null space of $(\mathcal{A}_{i}^{p})^{T}$; therefore, least squares returns a solution $\hat{x}_{i}^{p}=0$. The projection point is $b_{i}=\mathcal{A}_{i}^{p}\hat{x}_{i}^{p}=0$. The projection point $b_{i}=0$ specifies that router $i$ blocks all packets with atomic+iatomic headers represented by $b_{i-1}$ at output port $p$.

If there exist multiple paths between a source and destination, then Vercel would compute reachability along each path and store the reachable set of packet headers at the destination as $b_{reachable}=\sum_{d=1}^{e}b_{d}$, where $b_{d}$ is the set of reachable headers at the destination node along path $d$ and $e$ is the total number of paths created with ports in $P\textsuperscript{affected}$. After traversing all the paths, Vercel labels an atomic+iatomic header (in the set $S\textsuperscript{affected}$) as reachable if it obtains a non-zero dot product between the vector representation of the header and $b_{reachable}$.

At some middleboxes in the network, forwarding tables also contain rules that transform packet headers, thereby making reachability computation complex. In general, header transformation (denoted by function $f^{tr}$) is a set of rules containing the header “match” and “transformed” fields. To model header transformations, Vercel extracts transformation rules from routers and inserts the headers (specified in the “match” and “transformed” fields) in the binary tree (as discussed in Section III-A, III-B). Thereafter, Vercel creates a transformation matrix $T_{i}\in\mathbb{R}^{m\times m}$, for each router $i$ performing header transformations. Here, $m$ denotes the total number of atomic+iatomic headers in the network. Vercel initializes all entries of matrix $T$ to zero, representing the absence of header transformation. Thereafter, a few entries in the matrix $T$ are updated to 1, based on the transformation rules. An entry $T^{(j,k)}=1$ of the matrix specifies that an atomic/iatomic header with identifier $k$ is transformed to another atomic/iatomic header with identifier $j$. While solving for reachability, Vercel first computes $t_{i}=H(T_{i}b_{i-1})$ to transform the headers and then solves $\mathcal{A}_{i}^{p}x_{i}^{p}=t_{i}$ to model forwarding at router $i$ (where $H$ represents the unit step function). The vector $t_{i}$ represents atomic+iatomic headers reachable up to the router $i$ and then transformed using $T_{i}$.

Example of Transformation Matrix: Consider the headers in the forwarding table of router $U$ (Figure 1(a)) and a transformation function (denoted as $f^{tr}_{U}$) containing a single rule $f^{tr}_{U}:01/2\to 00/2$. To create the transformation matrix at router $U$, i.e., $T_{U}$, Vercel requires information of atomic+iatomic headers ahead in time. In this example, the atomic and iatomic headers are $\{a1,q2,a3,a4\}$. The matched header 01/2 ($a3$) is itself atomic while transformed header 00/2 can be represented by a combination of an atomic 000/3 ($a1$) and an iatomic 001/3 ($q2$) headers. Now, $T_{U}$ is a (4, 4) matrix populated as follows.

The transformation matrix $T_{U}$ is initialized with “1”s along its diagonal, except for the index (3, 3). To represent the transformation of header $a3$ to ($a1,\;q2$), $T_{U}$ is initialized with “1”s at indices (1, 3) and (2, 3).

In a network, devices such as firewalls and routers contain an access control list (ACL) to restrict the access of packets. A rule in an ACL is a filtering function $f^{acl}$ that maps a set of packet headers to a set of {permit, deny} actions. To incorporate packet filtering at a router/firewall $i$, Vercel extracts ACL rules from $i$ and arranges the header fields from ACL rules in the existing binary tree. The process of adding ACL rules to the binary tree is similar to that of adding forwarding rules (Section III-A), though with different actions. Thereafter, to model packet filtering for all the atomic+iatomic headers in the affected set (Section III-C), Vercel creates an $m$-dimensional filtering vector $g_{i}$ with binary values (0/1). A non-zero entry at the $j$^th index of the vector $g_{i}$ implies that an ACL rule allows packets with atomic/iatomic header $s_{j}$ to pass through router $i$. After obtaining the filtering vector $g_{i}$, Vercel performs filtering at router $i$ as $f_{i}=g_{i}\otimes b_{i-1}$, where $\otimes$ represents the Hadamard product between vectors. An index $j$ with the non-zero entry in the filtered vector $f_{i}$ implies that received packets with atomic/iatomic header $s_{j}$ are permitted to pass through router $i$. After filtering, Vercel models packet forwarding along port $p$ at router $i$ by solving $\mathcal{A}_{i}^{p}x_{i}=f_{i}$ with least squares.

To detect a forwarding loop, consider an intermediate router $i$, along the path $S$,…,$i$-$1$, $i$, $i$+$1$,…,$D$. Router $i$ receives packets with headers in vector $b_{i-1}$ from router $i$-$1$. Thereafter, Vercel models packet forwarding at router $i$ along a port $p$ using $\mathcal{A}_{i}^{p}x_{i}^{p}=b_{i-1}$ and obtains a projection point $b_{i}\neq 0$. After obtaining $b_{i}$, Vercel checks whether the next router $i+1$ (connected through port $p$ at router $i$) has already been traversed by the received packets. If the identifier of the next router $i+1$ is already traversed, then Vercel confirms that the rule update will trigger a loop. To compute which packet headers are in a loop, Vercel searches for the indices with non-zero entries in $b_{i}$.

Blackholes represent a network state in which a router receives a packet, but its forwarding table does not contain a corresponding rule. To identify a blackhole at router $i$, Vercel implements the following steps. (1) Vercel models the forwarding behavior of router $i$ based on the rules in its forwarding table. Since we have $v_{i}^{p}$ for all the affected ports of $i$ (Section III-C, IV), Vercel performs element-wise logical OR between all $v_{i}^{p}$ (denoting atomic+iatomic headers that router $i$ forwards to its output port $p$) to get a unified forwarding vector $v_{i}$ corresponding to router $i$. Intuitively, $v_{i}$ represents packets with atomic+iatomic headers that router $i$ forwards to its neighbors. (2) Vercel computes $v_{i}\otimes b_{i-1}$ to obtain a projection point $b_{i}$. Vector $b_{i}$ represents those atomic+iatomic headers in $b_{i-1}$ that router $i$ forwards to its neighbors. (3) To detect a blackhole at router $i$, Vercel computes $c_{i}=b_{i-1}\oplus b_{i}$ (where $\oplus$ denotes the element-wise logical XOR). The indices with a non-zero entry in vector $c_{i}$ represent packets (with atomic+iatomic headers) received at router $i$ that do not get any match in the forwarding table.

The approximate solution $\hat{x}$ obtained with least squares provides a projection point $\mathcal{A}\hat{x}$. We can measure the difference between two points $\mathcal{A}\hat{x}$ and $b$ to obtain the projection error, whose absolute value for computation of reachability has no implication. However, the projection error ($\mathcal{A}\hat{x}-b$) is a useful metric to provide intuition into packet processing at a router as well as network-wide metric computations. While the quantum of projection error ($\mathcal{A}\hat{x}-b$) has no bearing on reachability, it turns out that the cumulative error vector could be useful in identifying path correctness. In short, the cumulative $l2$ norm of projection errors across a path can reflect the misconfigurations along the path.

When a router receives a packet at a port, then there must exist a rule that forwards the packet to another port. However, if no rule exists that matches the address field of this packet, then taking $\mathcal{A}\hat{x}-b$ contributes towards a higher projection error. If the configuration is not along the shortest path, then even if the projection error at a router is low, the accumulated projection error along the path adds up and becomes high.

Hence, one use of projection error computation is to find if the configuration is on the shortest path, which gives a direct measure of the data plane correctness. To achieve this, the controller computes the $l2$ norm of the error at each node and adds the errors along all possible paths obtained through router configurations. If the configurations are correct, the shortest path should have the least total error. If this is not the case, Vercel adjusts the configurations based on the errors at intermediate nodes to ensure they result in the shortest path.

For more insight into this, suppose a provider wants to ensure that all the flows belonging to say a VR application must be configured always on the shortest path. To this end, Vercel initializes vector $b_{0}$ with the IP addresses corresponding to these flows and runs a reachability check. While doing so, it computes the cumulative error along each path to the destination. In case the minimum cumulative error is not obtained from the shortest path, then Vercel identifies that most of the configuration is on longer paths. Now, by observing each intermediate node’s projection error, Vercel identifies which flows do not have a matching rule along the shortest path. Finally, these missing rules are configured at the identified nodes. To ensure that the non-shortest path is not chosen, the corresponding rules are first identified and then deleted from the nodes along the non-shortest paths by using projection errors.

Example: To understand how projection errors can be used prolifically, revert back to Figure 1, we also observe that router $U$ drops some packets due to the absence of a rule that will process header $001/3$. This is observed by computing the projection error between vectors $b_{Y}=$ [1, 1, 0] (headers received at $U$) and $b_{U}=$ [0, 1, 0] (headers forwarded by $U$ along port 0). The projection error ($b_{Y}-b_{U}$) along port 0 at router $U$ is [1, 0, 0], and this indicates that packets with header 001/3 are not forwarded along the output port 0 at router $U$. However, as per the shortest path configuration, router $U$ should forward packets with header 001/3 along port 0 (in which case the projection error is zero). Therefore, the error vector helps in identifying possible misconfigurations.

A second use of the projection error is for automatic table population amidst no reachability. If reachability does not exist between two routers, then the cumulative projection error will be very large along all the paths. A path with the lowest cumulative l2 norm indicates that there are already a large number of relevant rules installed at the routers, and we just have to add a few more to establish reachability. On that path, for the nodes which have high projection error (compared to other nodes), Vercel recommends the addition of table entries such that the projection error at those nodes reduces. The table entries (prefix, port), now facilitate the router at which the entry is added to route the packet to the destination, leading to reachability. Through this iterative process, the entries can be updated to result in the shortest path. In this way, by using a simple script, Vercel can automatically establish reachability while achieving the shortest path routing for the source-destination pairs of interest.

Vercel is implemented as $\sim$2000 lines of single-threaded Python 3.7 code with dependency on Numpy [29] (for matrix computations) and Networkx [30] (for creating a network graph). We evaluate the performance of Vercel on a single core of an Intel Core i7 CPU at 3.6GHz clock and 64GB RAM.

To extensively evaluate the performance of Vercel, we have considered network topologies ranging from campus networks to large enterprise/provider networks with millions of rules (up to 124.7 million). Shown in Table II are the datasets that we consider for Vercel’s evaluation. The Stanford and Berkeley datasets serve as benchmarks for evaluating network invariants [26, 31]. Further, we also consider datasets RF 1755, RF 6461, RF 3257, and INET consisting of rules from the autonomous systems derived from the Rocketfuel Project [32]. These datasets represent dense network topologies and contain millions of forwarding rules, suitable for evaluatng scalability of a verification technique. To evaluate Vercel’s performance on a service provider network, we consider datasets Airtel 1, Airtel 2 used by [15], which are available at [31]. To expand to very large providers, we created two synthetic datasets – Simnet1 and Simnet2, containing 1000 and 2000 nodes with 100K edges in each network. We assign IP addresses on these two networks based on the mask length distribution extracted from the Rocketfuel project. Finally, we populate forwarding rules on Simnet1 and Simnet2 with shortest path routing. We also point out that Vercel has been implemented in conjunction with controllers and switches such as ONOS and OVS, thus showing industry-centric compatibility. Table II provides the specifics of the datasets such as nodes, edges, number of rules in the network, number of updates (insertion/deletion). To compute the verification time of Vercel, we first load 90% of the forwarding rules from the dataset, then populate the binary tree and then perform real-time updates by randomly selecting the remaining rules for insertion and deletion at router forwarding tables. We consider dynamic updates by first performing rule insertion in the forwarding tables (which comprises half of the rule updates shown in the sixth column of Table II), followed by rule deletion.

The most important metric to evaluate a network verification scheme is verification time [16, 15, 17, 24]. Figure 5 shows the cumulative distribution function (CDF) of verification time achieved by Vercel applied to the datasets [31, 26]. From Figure 5, we observe that Vercel processes at least 70% of the updates within $100\mu s$ on all the datasets. Even if we consider 90% of the updates, verification time of Vercel remains under $500\mu s$. These results imply that Vercel achieves sub-millisecond verification time, even on large networks.

Table III provides verification time of Vercel on 10 datasets. The first row of Table III shows the total number of atomic+ iatomic headers in each network. Note that the ratio of atomic+iatomic headers to the number of forwarding rules on the INET dataset (containing the largest number of rules across all datasets) is as small as $\sim$0.005, hence Vercel exploits the overlap among forwarding rules and represents the header space with a small number of non-overlapping atomic+iatomic headers, thereby reducing the number of headers to be processed. The second and third rows of Table III show the median and mean of the verification time for Vercel on different datasets. These results show that the median and mean verification time of Vercel remains bounded to within $77\mu s$ and $162\mu s$ on all but the two synthetic datasets. In the case of the INET dataset, the mean increases to $162\mu s$ as the average number of neighbors per router is $\sim$260 (which is significantly higher than all other datasets). This increase in nodal degree results in the traversal of more paths in the network, increasing verification time. We also observe the mean and median of verification time of Vercel on two synthetic networks is up to $428\mu s$ and $194\mu s$. Since these networks contain a large number of (2000) nodes and (100K) edges, therefore reachability analysis requires traversing longer paths between source and destination nodes. Accordingly, an increase in the path length increases verification time.

Another metric of interest (benchmarked by [15, 17]) is – how many updates are verified in less than $250\mu s$ (third row in Table III). Vercel verifies $\sim$74% of the updates within $250\mu s$, on a large network with $\sim$124.7M rules. For a network with $2000$ nodes, Vercel manages to verify $\sim$55% of the rules within $250\mu s$. The primary reason for this fast verification time is the quick identification of the affected atomic+iatomic headers and verifying multiple headers together in vector space using least squares.

Shown in Table IV (rows labeled with RL and RB) is the time for determining reachability in the presence of loops and in another case with blackholes. In these two cases, the mean verification time of Vercel is less than 90 and 98 $\mu$s on all the datasets. Verifying loops require a slightly longer time because Vercel performs extra checks for re-traversal along a path and stores a non-zero vector $b_{i}$ to identify the headers in the loop (Section V). For blackhole detection, Vercel performs some extra steps (element-wise logical OR and XOR, see Section V) along with a reachability check, which slightly increases the time for verification.

Since none of the existing datasets provide ACL rules, we created ACL rules by mapping the destination IP addresses (in the existing datasets) to filtering actions (permit/deny). Table IV shows that the mean and median verification time in the presence of ACL rules (represented as rows labeled with RF for Vercel) is less than 91 and 87 $\mu$s on all the datasets that we considered.

We define a function to transform an IP prefix into another randomly chosen IP prefix and store the transformation function in a matrix $T_{i}$, for each router $i$ (Section V). In Table IV (with the rows labeled as RT), we observe that the mean verification time of Vercel is less than 1 ms on all the datasets. Due to the matrix-vector product $T_{i}b_{i-1}$, the mean verification time increases in the presence of transformation functions. Note that the verification time of Vercel is comparable to APKeep [17], in which the verification time is also bounded within 1 ms. Note, Veriflow [16] and Deltanet [15] do not model packet transformations.

Policy-based routing (PBR) [33] is used by network administrators to define the routing behavior of packets in a network by overriding the underlying routing protocol. Vercel assigns a PBR label to those nodes of the binary tree whose corresponding headers are generated through PBR. Thereafter, Vercel does not allow any non-PBR protocol to update the forwarding action (e.g., output port) at the labeled node in the binary tree. Subsequently, after a rule update, Vercel determines the affected headers and ports in the binary tree. Finally, Vercel evaluates reachability and checks for the violation of routing policies. In our evaluation, Vercel identifies whether, after a rule update, the traffic: a) violates a path length constraint; or, b) passes through a predetermined set of routers. Table IV (rows labeled with RP) shows that the median and mean verification time of Vercel for checking these two policies together is less than 88 and 96 $\mu s$. With an additional consideration of routing policy, the verification time increases slightly. The reason is that after evaluating reachability at the destination, Vercel has to iterate over the reachable paths to check for policy violations.

Through Vercel we determine the quality of different paths (for example when deploying ECMP) by utilizing the projection error accumulated over nodes along each path. The cumulative $L2$-norm of the projection error (across all nodes along a path) quantifies the quality of a path in terms of the number of misconfigured packet headers at routers on the path. Figure 6 shows the cumulative $L2$ norm of the projection error as a function of path length. Since protocols configure most of the services on shortest paths, therefore we observe low projection error on such paths. Generally, fewer services are configured on longer paths, which is reflected by an increase in the cumulative L2 norm of the projection error. Therefore, we can use projection error as a metric to recommend the paths for service configurations.

As discussed in Section II-B, error rectification is done by including $b$ in the column space of matrix $\mathcal{A}$ for all nodes along the path, where $b$ is in the null space of $\mathcal{A}^{T}$. We created a synthetic network of 50 nodes and 200 edges to evaluate this approach and configured corresponding forwarding tables. Now, Vercel is applied to identify reachability between an arbitrary source and destination node. Assume, Vercel recognizes that there is no reachability, indicating that a fix is required. Computing this fix must be done fast. To show rectification computation, we varied the number of atomic/iatomic headers in the network and measured the time to rectify across differing path lengths. Results shown in Figure 7 allude to the time required to establish reachability. A path length of five required more fixes than a shorter path length of three. However, even if we increase the network-wide number of atomic/iatomic headers, the fixing time does not grow linearly. This is because Vercel leverages vectorization, where the same algebraic operation on different headers can be performed simultaneously. From Figure 7, it is evident that even on a longer path, Vercel can establish reachability within 50$\mu s$ irrespective of the number of atomic/iatomic headers.

We compare Vercel with the state-of-art real-time data plane verification tools such as AP Verifier [22], Veriflow [16], NetPlumber [24], Deltanet [15] and APKeep [17]. Table V compares the verification time of the various tools on the Stanford (campus) and Airtel (service provider) datasets. Note that on a large dataset such as Stanford, the performance of Vercel is better than all other verification techniques except Deltanet. Note Deltanet is designed only to model forwarding rules. In contrast, Vercel, APKeep, NetPlumber, and AP Verifier support data plane verification with a larger set of network functions such as filtering and transformation. Table V shows that, on the Stanford dataset, Vercel achieves mean verification time of 53 $\mu$s and its performance is summarized as: $8\times$ over Veriflow, $164\times$ over NetPlumber, $1.7\times$ over APKeep, $36\times$ over AP Verifier. On the Stanford dataset, Vercel processes 99.7% of the rule updates within 250 $\mu$s. Whereas, existing solutions such as NetPlumber, APVerifier, Veriflow and APKeep process 23.6%, 13.3%, 96.1% and 96.4% of the rule updates within 250 $\mu$s, respectively. Similarly, on a small dataset of Airtel 1, Vercel achieves a significant improvement of $2.5\times$ over AP Verifier, $1.84\times$ over Veriflow, and $118\times$ over NetPlumber. The reason for Vercel’s speedup is the simultaneous processing of multiple atomic+iatomic headers at different ports. In contrast, other approaches do not have the in-built capability of simultaneous processing and are comparatively slower.

A network administrator can use “what-if” queries to determine the fate of packets when a link goes down or model other scenarios. After a link failure, the SDN controller/ routing protocol recomputes paths between different source-destination pairs. Subsequently, the controller deletes existing rules corresponding to the failed links and inserts new rules (based on the updated paths) in the forwarding table of routers. To simulate a link failure, we select a random link $(l)$ in the network and delete rules from the both the connected routers that forward packets over $l$. As shown in the first row of Table VI, a link failure can trigger deletion of 1000s of rules, therefore Vercel computes reachability by considering batches of rules. Table VI shows that the mean verification time for checking reachability along with loops on the INET dataset for Vercel is around 534 ms when a link failure triggers $\sim$3060 rule deletions in the network. In contrast, Deltanet and Veriflow require 2888 and 29117 ms. This result shows that Vercel is up to 5$\times$ and 54$\times$ faster than Deltanet and Veriflow on INET dataset. However, Deltanet performs better in 3 of the 5 datasets; though, the performance of Deltanet is dependent on network size and the number of rules.

We now show the memory utilization of Vercel on the various datasets and compared to Veriflow and Deltanet. Table VI shows that Vercel is upto 7.8$\times$ memory efficient than Deltanet. Deltanet is memory intensive due to the implementation of two data structures: 1) binary search tree to create atoms and 2) edge labeled graph to model the forwarding behavior of packets in a network. As compared to Veriflow, Vercel consumes more memory because of the extra space required to store: a) (node, port) tuples at different nodes of the tree; b) node label in string format and; c) integer identifier at leaf nodes of the tree, but also leads to more expressive power and path recommendation.