HDPView: Differentially Private Materialized View for Exploring High Dimensional Relational Data

Let $X$ be the input database with $n$ records consisting of an attribute set $A$ that has $d$ attributes $A=\{a_{1},\dots,a_{d}\}$. The domain $dom(a)$ of an attribute $a$ has a finite ordered set of discrete values, and the size of the domain is denoted as $|dom(a)|$. The overall domain size of $A$ is $|dom(A)|=\prod_{i\in[d]}|dom(a_{i})|$, where $[d]=\{1,\dots,d\}$. In the case where attribute $a$ is continuous, we transform the domain into a discrete domain by binning, and in the case where attribute $a$ is categorical, we transform it into an ordered domain. Then, $dom(a)$ can be represented as a range $r[s_{a},e_{a}]$ where for all $p_{a}\in dom(a)$, $s_{a}\leq p_{a}\leq e_{a}$. For ranges $r_{1},r_{2}$, $|r_{1}\cap r_{2}|$ means the number of value $p_{a}$ satisfies $s_{a}\leq p_{a}\leq e_{a}$.

We consider transforming the database $X$ into the $d$-mode count tensor $\mathcal{X}$, where given $d$ ranges $r_{1},\dots,r_{d}$, $\mathcal{X}[r_{1},\dots,r_{d}]$ represents the number of records where $(a_{1}(\in r_{1}),\dots,a_{d}(\in r_{d}))\in X$. We utilize $x$ $(\in\mathcal{X})$ as a count value in $\mathcal{X}$; this corresponds to a cell of the count tensor. We denote a subtensor of $\mathcal{X}$ as block $\mathcal{B}\subseteq\mathcal{X}$. $\mathcal{B}$ is also a $d$-mode count tensor, but its domain in each dimension is smaller than or equal to that of the original count tensor $\mathcal{X}$; i.e., each attribute $a_{i}$ $(i\in[d])$, $r[s_{a_{i}},e_{a_{i}}]$ of $\mathcal{B}$ and $r[s^{\prime}_{a_{i}},e^{\prime}_{a_{i}}]$ of $\mathcal{X}$ satisfy $s^{\prime}_{a_{i}}\leq s_{a_{i}}$ and $e_{a_{i}}\leq e^{\prime}_{a_{i}}$. We denote the domain size of $\mathcal{B}$ as $|\mathcal{B}|$.

Last, we denote $q$ as a counting query and $\mathbf{W}$ as a workload. $\mathbf{W}$ is a set of $|\mathbf{W}|$ counting queries, where $\mathbf{W}=\{q_{1},...,q_{|\mathbf{W}|}\}$, and $q(\mathcal{X})$ returns the counting query results for count tensor $\mathcal{X}$.

DP (Dwork, 2006) is a rigorous mathematical privacy definition that quantitatively evaluates the degree of privacy protection when we publish outputs. DP is used in broad domains and applications (Bindschaedler et al., 2017; Chaudhuri et al., 2019; Papernot et al., 2016). The importance of DP is supported by the fact that the US census announced ’2020 Census results will be protected using “differential privacy”, the new gold standard in data privacy protection’ (Abowd, 2018).

Practically, we employ a randomized mechanism $\mathcal{M}$ that ensures DP for a function $f$. The mechanism $\mathcal{M}$ perturbs the output of $f$ to cover $f$’s sensitivity, which is the maximum degree of change over any pair of datasets $D$ and $D^{\prime}$.

The Laplace mechanism can be used for randomizing numerical data. Releasing the histogram is a typical use case of this mechanism.

The exponential mechanism is the random selection algorithm. The selection probability is weighted based on a score in a quality metric for each item.

Quantifying the privacy of differentially private mechanisms is essential for releasing multiple outputs. Sequential composition and parallel composition are standard privacy accounting methods.

This section describes the foundation of multidimensional data-aware segmentation that seeks a solution for the differentially private view $\tilde{\mathcal{X}}$ from the input count tensor $\mathcal{X}$. Every count $\tilde{x}\in\tilde{\mathcal{X}}$ is sanitized to satisfy DP. We formulate multidimensional block segmentation as an optimization problem.

For any count $x$ in the block $\mathcal{B}_{i}$, we have two types of errors: Perturbation Error (PE) and Aggregation Error (AE). Assuming that we replace any count $x\in\mathcal{B}_{i}$ with $\bar{x}_{i}=(S_{i}+z_{i})/|\mathcal{B}_{i}|$, the absolute error between $x$ and $\bar{x}_{i}$ can be computed as

Therefore, the total error over block $\mathcal{B}_{i}$, namely, the segmentation error (SE), can be given by:

where

(5) and (6) represent the AE and the PE, respectively.

Our proposed p-view has a simple structure. The p-view consists of a set of blocks, each of which has a range for each attribute and an appropriately randomized count value, as shown in Figure 1. Formally, we define the p-view as follows.

Thus, each block $\mathcal{B}_{i}$ has this $d$-dimensional domain and the sanitized sum of count values $\tilde{S_{i}}$.

In the range counting query processing, a counting query $q$ needs to have the range condition $c_{q}=\{r[s^{(q)}_{a_{1}},e^{(q)}_{a_{1}}],...,r[s^{(q)}_{a_{d}},e^{(q)}_{a_{d}}]\}$. Let the ranges of $\mathcal{B}_{i}$ be $\{r[s^{(i)}_{a_{1}},e^{(i)}_{a_{1}}],...,r[s^{(i)}_{a_{d}},e^{(i)}_{a_{d}}]\}$, and we calculate the intersection of $c_{q}$ and the block and add the count value according to the size of the intersection. Hence, the result can be calculated as follows.

The number of intersection calculations is proportional to the number of blocks, and the complexity of the query processing is $\mathcal{O}(md)$.

Our challenge is to devise a simple yet effective algorithm that enables us to efficiently search a block partitioning with small total errors and DP guarantees. As a realization of the algorithm, we propose HDPView.

Figure 2 illustrates an overview of our proposed algorithm. First, HDPView creates the initial block $\mathcal{B}^{(0)}$ that covers the whole count tensor $\mathcal{X}$. Second, we recursively bisect a block $\mathcal{B}$ (initially $\mathcal{B}=\mathcal{B}^{(0)}$) into two disjoint blocks $\mathcal{B}_{L}$ and $\mathcal{B}_{R}$. Before bisecting $\mathcal{B}$, we check whether the AE over $\mathcal{B}$ is sufficiently small. If the result of the check is positive, we stop the recursive bisection for $\mathcal{B}$. Otherwise, we continue to split $\mathcal{B}$. We pick a splitting point $p\in dom(a)$ ($a\in A$) for splitting $\mathcal{B}$ into $\mathcal{B}_{L}$ and $\mathcal{B}_{R}$ which have smaller AEs. Although splitting does not always result in smaller total AEs, proper cut point obviously makes AEs much smaller. Third, HDPView recursively executes these steps separately for $\mathcal{B}_{L}$ and $\mathcal{B}_{R}$. After convergence is met for all blocks, HDPView generates a randomized aggregate by $S_{i}+z_{i}$ where $z_{i}\sim Lap(1/\epsilon)$ for each block $\mathcal{B}_{i}$. Finally, for all $x\in\mathcal{B}_{i}$, we obtain the randomized count $\tilde{x}=(S_{i}+z_{i})/|\mathcal{B}_{i}|$.

The abovementioned algorithm can discover blocks that heuristically reduce the AEs, and is efficient due to its simplicity. However, the question is how can we make the above algorithm differentially private?. To solve this question, we introduce two mechanisms, random converge (Section 5.2) and random cut (Section 5.3). Random converge determines the convergence of the recursive bisection, and random cut determines the effective single cutting point. These provide reasonable partitioning strategy to reduce the total errors with small privacy budget consumption.

The overall algorithm of HDPView are described in Algorithm 1. Let $\epsilon_{b}=\epsilon_{r}+\epsilon_{p}$ be the total privacy budget for HDPView, where $\epsilon_{r}$ is the budget for the recursive bisection and $\epsilon_{p}$ is the budget for the perturbation. HDPView utilizes $\gamma\epsilon_{r}$ for random converge and $(1-\gamma)\epsilon_{r}$ for random cut ($0\leq\gamma\leq 1$). $\alpha$ is a hyperparameter that determines the size of $\lambda$ and $\delta$, where $\lambda$ corresponds to the Laplace noise scale of random converge (Lines 10, 18) and $\delta$ is a bias term for AE (Lines 11, 17). These are, sketchily, tricks for performing random converge with depth-independent scales, which are explained in Section 5.2 and a detailed proof of DP is given in Section 5.4. The algorithm runs recursively (Lines 12, 30, 31), alternating between random converge (Lines 16-20) and random cut (Lines 21-28). The random converge stops when the AE becomes small enough, consuming a total budget of $\gamma\epsilon_{r}$ independent of the number of depth. The random cut consumes a budget of $(1-\gamma)\epsilon_{r}/\kappa$ for each cutting point selection until the depth exceeds $\kappa$. $\kappa$ is set as $\kappa=\beta\log_{2}{\bar{n}}$, where $\beta>0$ is hyperparameter and $\bar{n}$ is the total domain size of the data. As we see later in Theorem 2, AE is not increased by splitting, so if the depth is greater than $\kappa$, we split randomly without any privacy consumption until convergence. After the recursive bisection converges, HDPView perturbs the count by adding the Laplace noise while consuming $\epsilon_{p}$.

AE decreases by properly splitting the blocks, however unnecessary block splitting leads to an increase in PE as mentioned above. To stop the recursive bisection at the appropriate depth, we need to obtain the exact AE of the block, which is a data-dependent output, therefore we need to provide a DP guarantee. One approach is to publish differential private AE so that making the decision for the stop is also DP by the post-processing property. In other words, the stop is determined by $\text{AE}(\mathcal{B})+Lap(\lambda)\leq\theta$ where $\theta$ is a threshold indicating AE is small enough. However, this method consumes privacy budget every time the AE is published, and the budget cannot be allocated unless the depth of the partition is decided in advance. Therefore, we utilize the observation for the privacy loss of Laplace mechanism-based threshold query (Zhang et al., 2016) and design the biased AE (BAE) of the block $\mathcal{B}$ instead of $\text{AE}(\mathcal{B})$: $\text{BAE}(\mathcal{B})=\mbox{max}(\theta+2-\delta,\,\text{AE}(\mathcal{B})-k\delta)$, where $k$ is the current depth of bisection, $\delta$ $(>0)$ is a bias parameter, i.e., we determine the convergence by $\text{BAE}(\mathcal{B})+Lap(\lambda)\leq\theta$. Intuitively, the BAE is designed to tightly bound the privacy loss of the any number of Laplace mechanism-based threshold queries with constant noise scale $\lambda$. When the value is sufficiently larger than the threshold, this privacy loss decreases exponentially (Zhang et al., 2016). Then, it can be easily bounded by an infinite series regardless of the number of queries. Conversely, when the value is small compared to the threshold, each threshold query consumes a constant budget. To limit the number of such budget consumptions, a bias $\delta$ is used to force a decrease in the value for each threshold query (i.e., each depth) because BAE has a minimum and if the value is guaranteed to be less than the minimum for adjacent databases, the privacy loss is zero. The design of our BAE allows for two constant budget consumptions at most, with the remainder being bounded by an infinite series. We give a detailed proof in Section 5.4. As a whole, since BAE is basically close to AE, AEs are expected to become sufficiently small overall.

Then, we consider about $\theta$ where if $\theta$ is too large, block partitioning will not sufficiently proceed, causing large AEs, and if it is too small, more blocks will be generated, leading to increase in total PEs. To prevent unwanted splitting, it is appropriate to stop when the increase in PE is greater than the decrease in AE. We design the threshold $\theta$ as $1/\epsilon_{p}$ which is the standard deviation of the Laplace noise to be perturbed. Considering the each bisection increases the total PE by $1/\epsilon_{p}$, when the AE becomes less than the PE, the division will increase the error at least. Hence, it is reasonable to stop under this condition.

Here, the primary question is how to pick a reasonable cutting point from all attribute values in a block $\mathcal{B}$ under DP. Our intuition is that a good cutting point results in smaller AEs in the two split blocks. We design random cut by combining an exponential mechanism with scoring based on the total AE after splitting.

Let $\mathcal{B}^{(p)}_{L}$ and $\mathcal{B}^{(p)}_{R}$ be the blocks split from $\mathcal{B}$ by the cutting point $p$, and the quality function $Q$ of $p$ in $\mathcal{B}$ is defined as follows:

Then, we compute the score for all attribute values $p\in dom(a)$, $a\in A$ , and satisfies $|\mathcal{B}^{(p)}_{L}|\geq 1$ and $|\mathcal{B}^{(p)}_{R}|\geq 1$. Note that the number of candidates for $p$ is proportional to the sum of the domains for each attribute $\sum_{i\in[d]}|dom(a_{i})|$, not to the total domains $\prod_{i\in[d]}|dom(a_{i})|$. We employ weighted sampling via an exponential mechanism to choose one cutting point $p*$. The sampling probability of $p$ is proportional to

where $\Delta_{Q}$ is the L1-sensitivity of the quality metric $Q$. We denote the L1-sensitivity of AE as $\Delta_{AE}$, and we can easily find $\Delta_{Q}=2\Delta_{AE}$ because $Q$ is the sum of two AEs. Thus, each time a cut point $p$ is published according to such weighted sampling, a privacy budget of $\epsilon$ is consumed. We set $\epsilon$ as the budget allocated to random cut (i.e., $(1-\gamma)\epsilon_{r}$) divided by $\kappa$. If the cutting depth exceeds $\kappa$, we switch to random sampling (Line 27 in Algorithm 1). Hence, cutting will not stop regardless of the depth or budget.

Compared to Privtree (Zhang et al., 2016), for a $d$-dimensional block, at each cut, HDPView generates just 2 blocks with this random cut while Privtree generates $2^{d}$ blocks with fixed cutting points. Privtree’s heuristics prioritizes finer partitioning, which sufficiently works in low-dimensional data because AEs become very small and the total PEs is not so large. In high-dimensional data, however, it causes unnecessary block splitting resulting in too much PEs. HDPView carefully splits blocks one by one, thus suppressing unnecessary block partitioning and reducing the number of blocks i.e., smaller PEs. It also enables flexibly shaped multidimensional block partitioning. Moreover, while whole design of HDPView including convergence decision logic and cutting strategy are based on an error optimization problem as described in Section 4, Privtree has no such background. This allows HDPView to provide effective block partitioning rather than simply fewer blocks, which we empirically confirm in Section 6.2.

For privacy consumption accounting, since HDPView recursively splits a block into two disjoint blocks, we only have to trace a path toward convergence. In other words, because HDPView manipulates all the blocks separately, we can track the total privacy consumption by the parallel composition for each converged block. The information published by the recursive bisection is the result of segmentation; however, note that since there is a constraint on the cutting method for the block, it must be divided into two parts; in the worst case, the published blocks may expose all the cutting points. For a given converged block $\mathcal{B}$, we denote the series of cutting points by $S_{\mathcal{B}}=[p_{1},....p_{k}]$, and $\mathcal{B}_{p_{i}}$ as the block after being divided into two parts at cutting point $p_{i}$. To show the DP guarantee, let $D$ and $D^{\prime}$ be the neighboring databases, and let $\Pr[S_{\mathcal{B}}|D]$ be the probability that $S_{\mathcal{B}}$ is generated from $D$. We need to show that for any $D$, $D^{\prime}$, and $S_{\mathcal{B}}$ that

to show that the recursive bisection satisfies $\epsilon$-DP.

The block with the largest $\left|\frac{\Pr[S_{\mathcal{B}}|D]}{\Pr[S_{\mathcal{B}}|D^{\prime}]}\right|$ of the converged disjoint blocks is $\mathcal{B}^{*}$, which has the longest $S_{\mathcal{B}^{*}}$ and contains different data between $D$ and $D^{\prime}$. Random converge and random cut are represented as follows.

where $\mathcal{B}_{p_{0}}$ is the initial count tensor and for all $i$, $\mathcal{B}^{\prime}_{p_{i}}$ indicates a neighboring block for $\mathcal{B}_{p_{i}}$. Taking the logarithm,

and let the first item of the right-hand of Eq.(15) be $(*1)$, and the other items be $(*2)$.

$(*1)$ corresponds to the privacy of the random cut, with each probability following Eq.(12). Given $\epsilon=\epsilon_{cut}$, for any $k$, the following holds from sequential composition.

The following are privacy guarantees for the other part, $(*2)$, based on the observations presented in (Zhang et al., 2016). First, we consider the sensitivity of AE $\Delta_{AE}$.

Thus, we also obtain $|\text{BAE}(\mathcal{B})-\text{BAE}(\mathcal{B}^{\prime})|\leq 2$, and

Furthermore, from the proof in the Appendix in (Zhang et al., 2016), when we have $f(x)=\mathrm{ln}\left(\frac{\Pr[x+Lap(\lambda)>\theta]}{\Pr[x-2+Lap(\lambda)>\theta]}\right)$, then

Next, we show the monotonic decreasing property of AE for block partitioning.

Considering $\text{BAE}(\mathcal{B})$, there exists a natural number $m$ ($1\leq m\leq k$) where if $i<m$, $\text{BAE}(\mathcal{B}_{p_{i}}))\geq\text{BAE}(\mathcal{B}_{p_{i+1}})+\delta\geq\theta+2-\delta$, if $i=m$, $\theta+2\geq\text{BAE}(\mathcal{B}_{p_{i}})\geq\theta+2-\delta$, and if $m<i$, $\text{BAE}(\mathcal{B}_{p_{i}})=\theta+2-\delta$. Therefore, using Eqs.(17, 18),

Thus, to make $(*2)$ satisfy $\gamma\epsilon_{r}$-DP, $\frac{2}{\lambda}\cdot\frac{3\exp\left(\frac{\delta}{\lambda}\right)-2}{\exp\left(\frac{\delta}{\lambda}\right)-1}\leq\gamma\epsilon_{r}$ should holds. Since the $\lambda$ and $\delta$ that satisfy these conditions are not uniquely determined, these values are determined by giving $\exp(\delta/\lambda)$ as a hyperparameter $\alpha$. Then, we can always calculate $\lambda=(\frac{3\alpha-2}{\alpha-1})\cdot(\frac{2}{\gamma\epsilon_{r}})$ and $\delta=\lambda\log{\alpha}$, in turn, which satisfies $(*2)\leq\gamma\epsilon_{r}$. $\alpha$ is valid for $\alpha>1$. If $\alpha$ is extremely close to 1, $\lambda$ diverges and random convergence is too inaccurate. As $\alpha$ increases, $\lambda$ decreases, but $\delta$ increases. Thus $\lambda$ and $\delta$ are trade-offs, and independently of the dataset, there exists a point at which both values are reasonably small. Around $\alpha=1.4\sim 1.8$ works well empirically. Please see Appendix A for a specific analytical result.

Finally, together with $(*1)$, the recursive bisection by random converge and random cut satisfies $\epsilon_{r}$-DP. In addition, the perturbation consumes $\epsilon_{p}$ for each block to add Laplace noise, so together with this, HDPView satisfies $\epsilon_{p}+\epsilon_{r}=\epsilon_{b}$-DP.

When a p-view created by HDPView publishes a counting query answer, we can dynamically estimate an upper bound distribution of the error included in the noisy answer. The upper bound of the error can be computed from the number of blocks used to answer the query and the distribution of the perturbation. Note that this can be computed without consuming any extra privacy budget because, as shown in 5.4, in addition to the count values, block partitioning results are released in a DP manner.

As a count query on the p-view is processed as Eq. (10), the answer consists of the sum of the query results for each block, and from Eq. (4), each block contains two types of errors: AE and PE. Let the error of a counting query $q$ be $Error(q,\mathcal{X},\mathcal{X^{\prime}}):=||q(\mathcal{X})-q(\mathcal{X^{\prime}})||_{1}$ where $\mathcal{X}$ and $\mathcal{X^{\prime}}$ are the original and noisy data, respectively, and we define the error by the L1-norm. First, since the AE depends on the concrete count values of each block involved in each query condition, we characterise the block distribution by defining an $\xi$-uniformly scattered block.

Then, we have the following theorem for the error.