Generation Matrix: An Embeddable Matrix Representation for Hierarchical Trees

Considering the challenges above, we propose an embeddable matrix representation called Generation Matrix. Generation Matrix is a lower triangular matrix containing only $2n-1$ non-zero elements (i.e., the weights of nodes and edges). Applying sparse storage technologies[12], we only need to store non-zero elements, satisfying the succinctness. Compared with others [5, 6, 7, 8], Generation Matrix emphasizes the application in the hierarchical tree algorithms. Our analysis of properties shows that many calculations on Generation Matrix have specific mathematical meanings. We can explain them and combine them to design complex hierarchical tree algorithms. More importantly, we demonstrate that the inverse of the Generation Matrix contains the inherent logic of recursion. Therefore, we can use Generation Matrix to simulate the top-down and bottom-up recursions without accessing the local structures. Besides, we study the relationship between Generation Matrix and some existing matrix representations and find it can be easily converted to others. It implies that Generation Matrix can be combined with the theories from other matrix representations to solve hierarchical tree problems.

To demonstrate the practicability of Generation Matrix, we introduce an application on the differentially private hierarchical tree release above. Considering the consistency problem[13, 14], we design a Generation Matrix-based optimally consistent release algorithm (GMC) for differentially private hierarchical trees. To our knowledge, GMC is the first solution to the problem by using matrix theory. It has an exceptionally concise process description so that just a simple matrix expression can summarize the core process. GMC embodies the advantages of Generation Matrix in solving problems in cross-domain. Therefore, Generation Matrix has positive significance for promoting the development of hierarchical tree-related research and applying matrix theories to solve hierarchical tree problems.

The rest of the paper is organized as follows. Section 2 reviews the related works of existing hierarchical tree representations and differentially private hierarchical tree release. Section 3 introduces the preliminaries of hierarchical trees and the optimally consistent release of differentially private hierarchical trees. Section 4 defines Generation Matrix, then analyzes its mathematical properties and the conversion relationship with other matrix representations. In Section 5, we show the application of Generation Matrix on differentially private hierarchical tree release and design GMC. Finally, Section 6 compares GMC with the existing technology through experiments and demonstrates its efficiency.

We first recall the definition of the hierarchical tree.

In the hierarchical tree, we denote the relationships between the nodes as $i\to j$, indicating node $j$ is the parent of node $i$. Fig. 1 shows a weighted hierarchical tree with $10$ nodes. The node weights and edge weights are denoted as ${{w}_{i}}$ and ${{w}_{i\to j}}$, respectively. By the height of each node, we can obtain a good quasi-ranking, which is defined as follows.

By the height of the nodes, we define a kind of induced subtree of hierarchical trees, called $k-$Order Subtree.

As a bottom-up induced subtree, ${{\mathcal{T}}^{\left(k\right)}}$ satisfies transitive, i.e., ${{\mathcal{T}}^{\left(a\right)\left(b\right)}}={{\mathcal{T}}^{\left(a+b\right)}}$. It can help us determine whether the subtrees obtained in different ways are equivalent. In subsequent applications, we use the concept of $k-$Order Subtree to simplify the description of the tree structure. In Fig. 1, we use dotted circles to mark the subtrees of $\mathcal{T}$ from $0$ to $2$ orders. It can be seen that the $0-$Order Subtree is $\mathcal{T}$ itself actually; ${{\mathcal{T}}^{\left(1\right)}}$ is a subtree composed of non-leaf nodes of $\mathcal{T}$. Let ${{n}_{k}}$ denote the number of nodes contained in ${{\mathcal{T}}^{\left(k\right)}}$, then there are ${{n}_{0}}\equiv n$ and ${{n}_{1}}$ equal to the number of non-leaf nodes of $\mathcal{T}$.

Before describing the optimally consistent releasing of the differentially private hierarchical tree, we first recall the hierarchical tree releasing model. Consider a set of unit counts ${{x}_{i}}:\mathcal{D}\to\mathbb{N}$$\left(1\leq i\leq m\right)$ for private dataset $\mathcal{D}$, where ${{x}_{i}}$ indicates the number of records in $\mathcal{D}$ that satisfies the mutually exclusive unit condition ${{\varphi}_{i}}$. The unit count ${{x}_{i}}$ satisfies

Since ${{\varphi}_{i}}$ is mutually exclusive, any $t\in\mathcal{D}$ satisfies and only satisfies one ${{\varphi}_{i}}$. Therefore, organizing ${{x}_{i}}$ into the form of a vector, we will get $\mathbf{x}={{\left[{{x}_{1}},{{x}_{2}},\ldots,{{x}_{m}}\right]}^{T}}$.

As shown in Fig. 2, each leaf corresponds to a ${{x}_{i}}$. The non-leaf node’s value equals the sum of the leaves’ value of the subtree rooted at that node. Therefore, the results of the hierarchical tree meet the consistency, i.e., “the sum of the values of the child nodes is equal to the value of the parent node”. In Fig. 2, we denote the value of the $i$-th node as ${{v}_{i}}$. Then, organizing ${{v}_{i}}$ into the form of $\mathbf{v}={{\left[{{v}_{1}},{{v}_{2}},\ldots,{{v}_{n}}\right]}^{\operatorname{T}}}$ in turn, we can get the to-be-released data $\mathbf{v}$.

However, several works[13, 19, 20, 21, 14] have demonstrated that releasing an unprotected hierarchical tree will result in privacy disclosure. To protect individual privacy, DWork et al. [18] proposed differential privacy, defined as follows.

Under differential privacy, the process of hierarchical tree releasing can be described as

where $\widetilde{\mathbf{v}}$ is the $\mathbf{v}$ after noise addition, which satisfies differential privacy. $\mathbf{\xi}$ is the random vector for the noise addition. Each element ${{\xi}_{i}}$ is i.i.d and satisfies ${{\xi}_{i}}\sim\operatorname{Lap}\left({\Delta}/{\varepsilon}\;\right)$, where $\operatorname{Lap}$ represents a Laplacian distribution and $\Delta$ is data sensitivity. In hierarchical tree releasing, $\Delta$ equals the height of $\mathcal{T}$[13].

To keep the consistency of the hierarchical tree after adding noise, we can get the optimally consistent release $\overline{\mathbf{v}}$ by following the optimization equation according to maximum likelihood post-processing proposed by Lee et al. [22].

where ${{\mathbf{M}}_{\mathcal{T}}}$ is the consistency constraint matrix of a hierarchical tree, defined as follows.

The optimization equation (6) has the following closed-form expression:

Since Formula (8) involves the inner product and inverse operations of the matrix, the time complexity of the direct solution is as high as $O\left({{n}^{3}}\right)$. The amount of calculation is too large to obtain an efficient enough algorithm directly by the expressions. On the surface, Formula (8) is not a good choice for solving optimally consistent releases, but under the theories of Generation Matrix, we can convert it into another form and apply the properties of Generation Matrix to obtain an efficient algorithm.

Before defining Generation Matrix, we number the nodes of $\mathcal{T}$ by descending order of height firstly.

Under the descending order of height, we define Generation Matrix for $\mathcal{T}$.

As shown in Fig. 3, since $\mathcal{T}$ satisfies Descending Order of Height, the number of any non-root node $i$ in $\mathcal{T}$ is always bigger than its parent. It ensures Generation Matrix is always a lower triangular matrix. According to Def. 10, there is a one-to-one mapping between arbitrary non-zero weighted hierarchical tree and $\mathbf{G}_{\mathcal{T}}^{\left({{\mathbf{w}}_{node}},{{\mathbf{w}}_{edge}}\right)}\in{{\mathbb{R}}^{n\times n}}$, i.e., the matrix representation of the non-zero weighted hierarchical tree is unique. When we only need to describe the structure of the hierarchical tree, we can use a Generation Matrix with all weights of $1$ to represent it, i.e., $\mathbf{G}_{\mathcal{T}}^{\left(\mathbf{1},\mathbf{1}\right)}$. We call $\mathbf{G}_{\mathcal{T}}^{\left(\mathbf{1},\mathbf{1}\right)}$ Structure Matrix, which is abbreviated as ${{\mathbf{G}}_{\mathcal{T}}}$. If two hierarchical trees have the same structure and arrangement, the positions of non-zero elements in their Generation Matrices are always the same, which we call Similar Generation Matrices.

Every Generation Matrix from the same hierarchical tree is always similar. We can use $\mathbf{G}\sim{{\mathbf{G}}_{\mathcal{T}}}$ as sufficient to judge whether the hierarchical tree represented by $\mathbf{G}$ has the same structure as $\mathcal{T}$.

According to Def. 2, ${{\mathcal{T}}^{\left(k\right)}}$ is an induced subtree composed of nodes $i$ with ${{h}_{i}}\geq k$ in $\mathcal{T}$. Under Descending Order of Height, these nodes are always ranked first. In Fig. 3, $k-$Order Submatrix that represents the $k-$Order Subtree is denoted as $\mathbf{G}_{{{\mathcal{T}}^{\left(k\right)}}}^{\left({{\mathbf{w}}_{node}},{{\mathbf{w}}_{edge}}\right)}$. We can obtain it by taking the ${{n}_{k}}-$order leading principal minor of $\mathbf{G}_{\mathcal{T}}^{\left({{\mathbf{w}}_{node}},{{\mathbf{w}}_{edge}}\right)}$ (i.e., the elements in rows and columns from $1$ to ${{n}_{k}}$). In particular, $\mathbf{G}_{{{\mathcal{T}}^{\left(1\right)}}}^{\left({{\mathbf{w}}_{node}},{{\mathbf{w}}_{edge}}\right)}$ represents the sub-tree composed of non-leaf nodes of $\mathcal{T}$.

Considering a specific application, one problem we may encounter is that the nodes of the hierarchical tree are numbered but not in Descending Order of Height. Under the matrix representation, the problem is elementary to solve. We can adopt a sparse mapping matrix to convert the original number into Descending Order of Height.

By the definition of the mapping matrix, $\mathcal{S}$ always represents an injection and satisfies ${{\mathbf{H}}_{\mathcal{S}}}{{\mathbf{H}}_{\mathcal{S}}}^{T}=\mathbf{I}$. If $\mathcal{S}$ represents a bijection, ${{\mathbf{H}}_{\mathcal{S}}}$ will be a permutation matrix. When the basis vector ${{\mathbf{e}}_{i}}$ acts on ${{\mathbf{H}}_{\mathcal{S}}}$, the following equations holds.

where $s_{i}^{-1}$ represents the inverse mapping of ${{s}_{i}}$, which satisfies $s_{{{s}_{i}}}^{-1}=i$.

To describe the mapping relationship between the nodes before and after sorting by Descending Order of Height, we only need to define the ordered set $\mathcal{S}=\left\langle{{s}_{1}},{{s}_{2}},\ldots,{{s}_{n}}\right\rangle$, where ${{s}_{i}}$ represents the sorted number of the node initially numbered $i$. Then, we can use ${{\mathbf{H}}_{\mathcal{S}}}\mathbf{v}$ to get the vector before sorting from $\mathbf{v}$ after sorting.

In addition, the mapping matrix can be used to represent other mapping relationships, such as the mapping of ${{v}_{i}}$ to ${{x}_{i}}$. For example, in Fig. 1, if we use an ordered set $\mathcal{H}=\left\langle{{\hbar}_{1}},{{\hbar}_{2}},\ldots,{{\hbar}_{m}}\right\rangle$ to represent the mapping relationship between ${{v}_{i}}$ and ${{x}_{i}}$, then ${{\hbar}_{i}}=i+5$, and we have the mapping matrix ${{\mathbf{H}}_{\hbar}}$ to represent their mapping relationship.

Our research shows that Generation Matrix has many mathematical properties that deserve attention. These properties can help us solve various problems about the analysis and calculation of hierarchical trees. According to Def. 10, it is not difficult to find that Generation Matrix satisfies sparsity.

Due to the sparsity of Generation Matrix, we can apply various sparse matrix technologies such as COO (Coordinate Format) and CSR (Compressed Sparse Row) to calculate hierarchical tree models efficiently. Under the sparse representations, the storage and calculation of Generation Matrix are both only $O\left(n\right)$. Therefore, the application based on Generation Matrix does not cause more computational overhead. Currently, the computing technologies of sparse matrices are very mature and widely used in various high-performance platforms [29, 30, 31].

One of the fundamental properties of Generation Matrices is invertibility. By solving the equations $\mathbf{G}{{{}_{\mathcal{T}}^{\left({{\mathbf{w}}_{node}},{{\mathbf{w}}_{edge}}\right)}}^{T}}\mathbf{z}=\mathbf{v}$ and $\mathbf{G}_{\mathcal{T}}^{\left({{\mathbf{w}}_{node}},{{\mathbf{w}}_{edge}}\right)}\mathbf{z}=\mathbf{v}$ about $\mathbf{z}$, we find two interesting and important mathematical properties of Generation Matrices. We collectively call them the propagation of Generation Matrix.

Prop. 14 and Prop. 15 show that Generation Matrix can simulate multiple recursive operations of hierarchical trees. According them, Cor. 1 analyzes the elements affected by the propagations of Generation Matrix.

Combining the propagations, we further study a variety of matrix operations of Generation Matrices. They have strong interpretability and provide crucial theoretical support for the research of hierarchical trees.

The properties above indicate that Generation Matrix is an effective and easy-to-use tool for various hierarchical tree analyses. In addition to the conclusions about vectors discussed above, there are some conclusions about matrices as follows. Compared with conclusions about vectors, they focus on describing the characteristics between nodes.

Prop. 16 shows that ${{\mathbf{G}}_{\mathcal{T}}}^{-1}$ is a matrix indicating the relationship between ancestors and descendants. Although ${{\mathbf{G}}_{\mathcal{T}}}^{-1}$ is denser than ${{\mathbf{G}}_{\mathcal{T}}}$, in most cases, ${{\mathbf{G}}_{\mathcal{T}}}^{-1}$ is still sparse.

Similar to Prop. 16, Prop. 8 can also be used as an indicator matrix to describe the relationship between nodes. Prop. 9 is an important property, which describes an effective method of calculating common ancestors. As an essential feature to describe the correlation between nodes, the number of common ancestors has an important application value for hierarchical tree analyses[8].

In the study of spectral graph theory, feature analysis is usually indispensable. Our research shows that the eigenvalues and eigenvectors of a Generation Matrix satisfy the following properties.

Prop. 10 shows that the eigenvalues and eigenvectors of the Generation Matrix have many interesting properties. For example, the eigenvalue of Generation Matrix is the weights of the nodes, which is much easier to solve than other matrix representations; the eigenvectors also satisfy some propagation properties similar to Prop. 14 and 15. Notably, the eigenvectors are conditional, which means that Generation Matrix is not always diagonalizable. Some Generation Matrices, especially the eigenvectors of ${{\mathbf{G}}_{\mathcal{T}}}$, still have many problems waiting to be studied. Although feature analysis is not the main focus in this paper, Prop. 10 still provides some valuable references for the subsequent research works.

Considering the relationship between $\mathbf{G}_{\mathcal{T}}^{\left({{\mathbf{w}}_{node}},{{\mathbf{w}}_{edge}}\right)}$ and corresponding ${{\mathbf{G}}_{\mathcal{T}}}$, we find that $\mathbf{G}_{\mathcal{T}}^{\left({{\mathbf{w}}_{node}},{{\mathbf{w}}_{edge}}\right)}$ satisfies a particular decomposition form, which we call the diagonal decomposition of Generation Matrix.

By the diagonal decomposition of Generation Matrix, we can express arbitrary $\mathbf{G}_{\mathcal{T}}^{\left({{\mathbf{w}}_{node}},{{\mathbf{w}}_{edge}}\right)}$ as an expression with ${{\mathbf{G}}_{\mathcal{T}}}$. Using Prop. 11, we can extend some mathematical properties of ${{\mathbf{G}}_{\mathcal{T}}}$ to $\mathbf{G}_{\mathcal{T}}^{\left({{\mathbf{w}}_{node}},{{\mathbf{w}}_{edge}}\right)}$ to solve more problems effectively.

Our research shows that Generation Matrix is not an isolated matrix representation from others. Through proper operations, we can convert Generation Matrix into other matrix representations. Fig. 4 shows the four matrix representations that can be transformed by the Generation Matrix constructed by the hierarchical tree in Fig. 1, including Adjacency Matrix, Laplacian Matrix, Distance Matrix, and Ancestor Matrix.

It can be seen from the theorems above that we can convert Generation Matrix into other matrix representations by simple matrix expressions. However, the reverse is not easy. Except for Adjacency Matrix, other matrix representations cannot be directly converted back to Generation Matrix. Therefore, we can use Generation Matrix to construct other matrix representations. Besides, it also implies that the theories of Generation Matrix have a particular internal connection with the matrix represented. We can combine the theories of Generation Matrix and other matrix representations to solve more problems about hierarchical trees.

In this section, we introduce how to apply Generation Matrix to efficiently and concisely achieve an optimally consistent release on differentially private hierarchical tree release. Since each leaf of ${\mathcal{T}}$ corresponds to a ${{x}_{i}}$, we use the mapping matrix ${{\mathbf{H}}_{\mathcal{H}}}$ to represent the mapping relationship between leaf nodes and hierarchical tree nodes. Using Prop. 14, the hierarchical tree building process ${{\operatorname{BuildTree}}_{\mathcal{T}}}$ can be described as

At the same time, we also define the inverse tree-building process ${{\operatorname{BuildTree}}_{\mathcal{T}}}^{-1}$ as the following expression, which is taking the leaves of $\mathcal{T}$ and restoring them to $\mathbf{x}$.

Although most works[13, 19, 20, 21, 14] do not take matrix analysis as the theoretical basis for optimally consistent release, there are many advantages to applying matrix analysis. One of them is error analysis. Using matrix analysis, we can quickly calculate the overall mean square error of the “Node Query” after the post-processing for optimally consistent release.

According to Thm. 28, the overall mean square error depends on the number of leaves after post-processing. Generally, $n$ is much less than the number of leaves $m$, so post-processing will significantly reduce the error. As the proof shown in Appendix A.7, We applied the property of the trace of the matrix to obtain a concrete and concise demonstration, which embodies matrix analysis’s great potential for solving problems.

Next, we will introduce how to apply Generation Matrix to achieve an efficient enough algorithm.

To obtain an efficient release algorithm, we apply QR decomposition to analyze Formula (8). Unfortunately, the traditional QR decomposition[23] cannot meet our analysis requirements, so we propose another QR decomposition form, namely the $\text{``LO''}-$QR decomposition.

Correspondingly, we call the traditional QR decomposition the $\text{``UO''}-$QR decomposition, which decomposes a matrix into an upper triangular matrix. Although the two have different forms, they both achieve decomposition through a series of basic orthogonal transformations. Householder transformation is the most widely used among various orthogonal transformation techniques because of its high efficiency, easy implementation, and applicability to sparse matrices. Completing QR decomposition once requires $m$ householder transformations. To describe the QR decomposition process in more detail, we define $\left(\mathcal{S},j\right)-$Householder transformation to describe each transformation.

For $\mathbf{M}$, we denote $\left(\mathcal{S},j\right)-$Householder transformation as $\mathbf{Y}={{\operatorname{House}}_{\mathcal{S},j}}\left(\mathbf{M}\right)$. The QR decomposition based on the Householder transformation can be expressed as

where $\mathbf{R}$ is the result of QR decomposition and “$\centerdot$” represents the operation of function composition such as ${{f}_{2}}\centerdot{{f}_{1}}\left(x\right)={{f}_{2}}\left({{f}_{1}}\left(x\right)\right)$. For $\text{``UO''}-$QR decomposition, the parameter ${{\varsigma}_{i}}=\left(\left\langle i,\cdots,n\right\rangle,i\right)$; for $\text{``LO''}-$QR decomposition, the parameter ${{\varsigma}_{i}}=\left(\left\langle m-i+1,1,\cdots,m-i,m+1,\cdots,n\right\rangle,m-i+1\right)$.

Next, we apply the $\text{``LO''}-$QR decomposition on ${{\mathbf{M}}_{\mathcal{T}}}$. To understand how $\text{``LO''}-$QR decomposition affects ${{\mathbf{M}}_{\mathcal{T}}}$, we divide ${{\mathbf{M}}_{\mathcal{T}}}$ into the following forms,

The upper half ${{\mathbf{M}}_{\uparrow}}\in{{\mathbb{R}}^{{{n}_{1}}\times{{n}_{1}}}}$ of ${{\mathbf{M}}_{\mathcal{T}}}$ consists of the first ${{n}_{1}}$ rows; The second half ${{\mathbf{M}}_{\downarrow}}\in{{\mathbb{R}}^{m\times{{n}_{1}}}}$ consists of the remaining $m$ rows. According to Def. 5, we have ${{\mathbf{M}}_{\uparrow}}={{\mathbf{G}}_{{{\mathcal{T}}^{\left(1\right)}}}}$ and ${{\mathbf{M}}_{\downarrow}}=-{{\mathbf{H}}_{{{\mathcal{S}}^{\left(f\right)}}}}$, where the $i$-th element of ordered set ${{\mathcal{S}}^{\left(f\right)}}$ satisfies $s_{i}^{\left(f\right)}=\text{ }{{f}_{i+{{n}_{1}}}}$. Considering the effects of Householder transformation on ${{\mathbf{M}}_{\mathcal{T}}}$, we denote the matrix obtained after the $k$-th Householder transformation as $\mathbf{M}_{\mathcal{T}}^{\left(k\right)}\left(0\leq k\leq{{n}_{1}}\right)$, whose upper half and second half are denoted as $\mathbf{M}_{\uparrow}^{\left(k\right)}$ and $\mathbf{M}_{\downarrow}^{\left(k\right)}$. $\mathbf{M}_{\mathcal{T}}^{\left(0\right)}$ is the form before Householder transformation, satisfying $\mathbf{M}_{\mathcal{T}}^{\left(0\right)}={{\mathbf{M}}_{\mathcal{T}}}$; $\mathbf{M}_{\mathcal{T}}^{\left({{n}_{1}}\right)}$ is the result after $\text{``LO''}-$QR decomposition. Thm. 6 demonstrates that, in the process of Householder transformation, $\mathbf{M}_{\mathcal{T}}^{\left(k\right)}$ satisfies some invariant properties.

As shown in Fig. 5, with the Householder transformation proceeds, the non-zero elements in $\mathbf{M}_{\downarrow}^{\left(k\right)}$ shift from right to left, and the position of the non-zero elements of $\mathbf{M}_{\uparrow}^{\left(k\right)}$ remains unchanged. Specifically, in the $k$-th Householder transformation, all non-zero elements in the $\left({{n}_{1}}-k+1\right)$-th column of $\mathbf{M}_{\downarrow}^{\left(k\right)}$ are transferred to the ${{f}_{{{n}_{1}}-k+1}}$-th column. Combining Thm. 6, we can infer the specific form of $\mathbf{M}_{\mathcal{T}}^{\left({{n}_{1}}-1\right)}$ after ${{n}_{1}}-1$ times of Householder transformation. After the last householder transformation, the result satisfies the following theorem.

Thm. 33 implies that the calculation ${{\left({{\mathbf{M}}_{\mathcal{T}}}^{T}{{\mathbf{M}}_{\mathcal{T}}}\right)}^{-1}}$ about ${{\mathbf{M}}_{\mathcal{T}}}$ in expression (8) can be replaced by ${{\mathcal{T}}^{\left(1\right)}}$. Since it is related to $\mathcal{T}$ and ${{\mathcal{T}}^{\left(1\right)}}$ simultaneously, we denote it as ${{\mathbf{G}}_{{{\mathcal{T}}^{\left(1\right)}}\leftarrow\mathcal{T}}}$. Thm. 8 shows that ${{\mathbf{G}}_{{{\mathcal{T}}^{\left(1\right)}}\leftarrow\mathcal{T}}}$ and ${{\mathbf{M}}_{\mathcal{T}}}$ are inner-product-equivalent.

The property of inner product equivalence provides us with a vital optimization idea for an optimal release, as shown in Cor. 35. Using matrix analysis, we convert the expression (8) into an expression about ${{\mathbf{G}}_{{{\mathcal{T}}^{\left(1\right)}}\leftarrow\mathcal{T}}}$ and then use the mathematical properties of Generation Matrix to improve the efficiency of optimal release.

According to Cor. 35, we get another optimal release form as follows.