Multiple Private Key Generation for Continuous Memoryless Sources with A Helper

Our main contributions are summarized as follows.

Firstly, we derive the output statistics of applying hash functions to multiple random sequences under the Rényi divergence measure in Lemma 1. Lemma 1 is an extension of [11, Theorem 1] to a multi-terminal case and a strict generalization of [30, Theorem 1] where the output statistics of random binning under the total variational distance measure was derived. Furthermore, Lemma 1 turns out to be an important tool in analyzing secrecy constraints in key generation problems, especially when the key needs to be protected from continuous observations correlated to observations at legitimate users.

Secondly, to illustrate the power of Lemma 1, we derive the capacity region for the multiple private key generation problem with a helper for CMS. To be specific, we revisit the model in [18] and derive the capacity region for CMS under a symmetric security requirement which did not appear in [18]. The converse proof follows by judiciously adapting the techniques in [22, Theorem 1] to our setting. In the achievability proof, we use Lemma 1 to analyze the secrecy constraints on generated keys which need to be protected from correlated continuous observations of illegitimate users. Furthermore, we use the quantization techniques in [22], the large deviations analysis for distributed source coding in [31], the Fourier Motzkin Elimination and the techniques to bound the difference between the differential entropy of CMS and the discrete entropy of the quantized random variables. We remark that the techniques used in our paper can also apply to strengthen all the four cases in [18] with strong secrecy and for CMS. Furthermore, we also extend our result to a cellular model involving more than four terminals and derive inner and outer bounds for the capacity region.

The rest of the paper is organized as follows. In Section I, we set up the notation. In Section II, we formulate the problem of output statistics of hash functions and present our main result under the Rényi divergence measure in Lemma 1. Subsequently in Section III, invoking Lemma 1, we derive the capacity region for the multiple private key generation problem with a helper. Furthermore, we generalize our result to a cellular model and derive bounds on the capacity region. The proofs of the capacity region for the multiple private key generation with a helper are given Sections IV and V. Finally, we conclude our paper and discuss future research directions in Section VI. For the smooth presentation of our main results, the proofs of all supporting lemmas are deferred to the appendices.

Throughout the paper, random variables and their realizations are in capital (e.g., $X$) and lower case (e.g., $x$) respectively. All sets are denoted in calligraphic font (e.g., $\mathcal{X}$). We use $\mathcal{X}^{\mathrm{c}}$ to denote the complement of $\mathcal{X}$ and use $U_{\mathcal{X}}$ to denote the uniform distribution over $\mathcal{X}$. Given any two integers $a,b$, we use $[a:b]$ to denote the set of all integers between $a$ and $b$ and we use $[a]$ to denote $[1:a]$ for any integer $a\geq 1$. Let $\mathcal{T}:=\{1,\ldots,T\}$. Given a sequence of random variables $X_{1},X_{2},\ldots,X_{T}$ and any subset $\mathcal{S}\subseteq\mathcal{T}$, we use $X_{\mathcal{S}}$ and $\{X_{t}\}_{t\in\mathcal{S}}$ interchangeably. Furthermore, let $X^{n}:=(X_{1},\ldots,X_{n})$ be a random vector of length $n$. For any $(a,b)\in[1:n]^{2}$, we use $X_{a}^{b}$ and $(X_{a},\ldots,X_{b})$ interchangeably. For information theoretical quantities, we follow [32].

Before presenting the main result, we first introduce some definitions. Given two distributions $(P_{A_{1}},Q_{A_{1}})$ defined an alphabet $\mathcal{A}_{1}$, the KL divergence is defines as

$\displaystyle D(P_{A_{1}}\|Q_{A_{1}})$

$\displaystyle:=\sum_{a_{1}\in\mathcal{A}_{1}}P_{A_{1}}(a_{1})\log\frac{P_{A_{1}}(a_{1})}{Q_{A_{1}}(a_{1})}.$

(1)

Furthermore, given $s\in[-1,\infty)$, the Rényi divergence or order $1+s$ is defined as

$\displaystyle D_{1+s}(P_{A_{1}}\|Q_{A_{1}})$

$\displaystyle:=\left\{\begin{array}[]{ll}\frac{1}{s}\log\sum_{a_{1}\in\mathcal{A}_{1}}P_{A_{1}}^{1+s}(a_{1})Q_{A_{1}}^{-s}(a_{1})&s\neq 0,\\ D(P_{A_{1}}\|Q_{A_{1}})&s=0.\end{array}\right.$

(4)

It is well known that $D_{1+s}(P_{A_{1}}\|Q_{A_{1}})$ is non-decreasing in $s$ (cf. [29]) and thus $D(P_{A_{1}}\|Q_{A_{1}})\leq D_{1+s}(P_{A_{1}}\|Q_{A_{1}})$ for all $s\geq 0$.

Given a joint distribution $P_{A_{1}E}$ on the alphabet $\mathcal{A}_{1}\times\mathcal{E}$, the conditional entropy is defined as

$\displaystyle H(A_{1}|E)$

$\displaystyle:=-\sum_{e\in\mathcal{E}}P_{E}(e)\sum_{a_{1}\in\mathcal{A}_{1}}P_{A_{1}|E}(a_{1}|e)\log P_{A_{1}|E}(a_{1}|e).$

(5)

Furthermore, given $s\in[-1,\infty)$, the conditional Rényi entropy of order $1+s$ is defined as

$\displaystyle H_{1+s}(A_{1}|E)$

$\displaystyle:=\left\{\begin{array}[]{ll}-\frac{1}{s}\log\sum_{e}P_{E}(e)\sum_{a_{1}}P_{A_{1}|E}^{1+s}(a_{1}|e)&s\neq 0,\\ H(A_{1}|E)&s=0,\end{array}\right.$

(8)

and the Gallager’s conditional Rényi entropy of order $s$ is defined as

$\displaystyle H_{1+s}^{\uparrow}(A_{1}|E)$

$\displaystyle:=-\frac{1+s}{s}\log\sum_{e}\Big{(}\sum_{a_{1}}P_{A_{1}E}^{1+s}(a_{1},e)\Big{)}^{\frac{1}{1+s}}.$

(9)

We remark that for continuous random variables, the summations in (4), (8), (9) should be replaced by integrals.

We then recall the formal definition of a hash function [33, Eq. (1)] (see also [34, 35]).

We remark that random binning in source coding problems (e.g., [32, Chapter 15.4.1]) is a universal₂ hash function.

In this subsection, we study the output statistic of applying hash functions to multiple random sequences under the Rényi divergence measure. For simplicity, we use $\mathcal{T}$ to denote the set $\{1,\ldots,T\}$. Consider a sequence of random variables $(A_{\mathcal{T}},E)=(A_{1},\ldots,A_{T},E)$ with a joint distribution $P_{A_{\mathcal{T}}E}$ defined on an alphabet $\prod_{t\in\mathcal{T}}\mathcal{A}_{t}\times\mathcal{E}$ where all $t\in\mathcal{T}$, the alphabet $\mathcal{A}_{t}$ is finite. Let $(A_{\mathcal{T}}^{n},E^{n})$ be an i.i.d sequence generated according to the distribution $P_{A_{\mathcal{T}}E}$.

For each $t\in\mathcal{T}$, let $f_{X_{t}}^{(n)}$ be an $\varepsilon$-almost universal₂ hash function mapping from $\mathcal{A}_{t}^{n}$ to $\mathcal{M}_{t}:=\{1,\ldots,N_{t}\}$ where $X_{t}$ describes the stochastic behavior of the hash function. Furthermore, the rate of the hash function $f_{X_{t}}$ is defined as $R_{t}:=\frac{1}{n}\log N_{t}$. We are interested in the output statistics of applying hash functions to the random sequences $A_{\mathcal{T}}^{n}$, i.e., $\{f_{X_{t}}^{(n)}(A_{t}^{n})\}_{t\in\mathcal{T}}$.

For ease of notation, let $M_{t}:=f_{X_{t}}^{(n)}(A_{t}^{n})$ for each $t\in\mathcal{T}$. Furthermore, for each $t\in\mathcal{T}$, let $U_{\mathcal{M}_{t}}$ denote the uniform distribution over $\mathcal{M}_{t}$ and let $P_{M_{t}}$ denotes the induced output distribution by $P_{A_{t}}^{n}$ and the hash function $f_{X_{t}}$, i.e., for all $m_{t}\in\mathcal{M}_{t}$,

$\displaystyle P_{M_{t}}(m_{t})=\sum_{a_{t}^{n}\in\mathcal{A}_{t}^{n}}P_{A_{t}}^{n}(a_{t}^{n})1\{f_{X_{t}}(a_{t}^{n})=m_{t}\}.$

(11)

To quantify the output statistics of the hash functions, it is common to use the KL divergence $D(P_{M_{\mathcal{T}}E^{n}}\|\prod_{t\in\mathcal{T}}U_{\mathcal{M}_{t}}\times P_{E}^{n})$ as a measure where

	$\displaystyle D(P_{M_{\mathcal{T}}E^{n}}\|\prod_{t\in\mathcal{T}}U_{\mathcal{M}_{t}}\times P_{E}^{n})$	$\displaystyle=D(P_{M_{\mathcal{T}}}\|\prod_{t\in\mathcal{T}}U_{\mathcal{M}_{t}})+I(M_{\mathcal{T}};E^{n})$		(12)
		$\displaystyle=\sum_{t\in\mathcal{T}:t\geq 2}I(M_{t};M_{[1:t-1]})+\sum_{t\in\mathcal{T}}D(P_{M_{t}}\|U_{\mathcal{M}_{t}})+I(M_{\mathcal{T}};E^{n}).$		(13)

Note that if $D(P_{M_{\mathcal{T}}E^{n}}\|\prod_{t\in\mathcal{T}}U_{\mathcal{M}_{t}}\times P_{E}^{n})<\delta$ for some $\delta>0$, then we have the following results

1. (i)

   $\sum_{t\in\mathcal{T}:t\geq 2}^{T}I(M_{t};M_{[1:t-1]})$ is small, indicating that the output of hash functions $M_{t_{1}}$ and $M_{t_{2}}^{n}$ are almost independent for all distinct pairs $(t_{1},t_{2})\in\mathcal{T}^{2}$;

2. (ii)

   $\sum_{t\in\mathcal{T}}D(P_{M_{t}}\|U_{\mathcal{M}_{t}})$ is small, indicating that the output of each hash function $M_{t}$ is almost uniform over $\mathcal{M}_{t}$ for all $t\in\mathcal{M}_{t}$;

3. (iii)

   $I(M_{\mathcal{T}};E^{n})$ is small, indicating that the collection of outputs of hash functions $M_{\mathcal{T}}=(M_{1},\ldots,M_{T})$ is almost independent of the side information $E^{n}$.

In this subsection, instead of using (13), we make use of the Rényi divergence of order $1+s$ (cf. (4)) as the measure of output statistics of hash functions, i.e.,

	$\displaystyle C_{1+s}(M_{\mathcal{T}}|E^{n})$	$\displaystyle:=D_{1+s}(P_{M_{\mathcal{T}}E^{n}}\|\prod_{t\in\mathcal{T}}U_{\mathcal{M}_{t}}\times P_{E}^{n})$		(14)
		$\displaystyle=\sum_{t\in\mathcal{T}}\log M_{t}-H_{1+s}(M_{\mathcal{T}}|E^{n}),$		(15)

where $s\in(0,1]$ (15) follows from (8). Note that the measure in (15) is a strict generalization of that in (13).

Our results in the following lemma concern the output statistics of $\varepsilon$-almost universal₂ hash functions for any $\varepsilon\in\mathbb{R}_{+}$ unless otherwise stated.

Note that the asymptotic performance in Claim (iii) is achieved only by ($1$-almost) universal₂ hash functions since we put the additional constraint of $\varepsilon=1$. This condition could potentially be relaxed with techniques in [36].

The proof of Lemma 1 is inspired by [11, Theorem 1], [28, Lemma 1 and Theorem 2] and provided in Appendix -A. A few remarks are in order.

Firstly, Lemma 1 is a generalization of [11, Theorem 1] to multi-terminal. In the proof of Lemma 1, we also borrow an idea from [30, Theorem 1] which studied the output statistics of universal₂ hash functions under the total variation distance measure instead of the Rényi divergence considered here. Invoking (21) and Pinsker’s inequality, it is easy to see that that [30, Theorem 1] is indeed a corollary of Lemma 1.

Secondly, since the Rényi divergence $D_{1+s}(\cdot)$ is a non-decreasing $s$ and thus for all $s\geq 0$,

	$\displaystyle D(P_{M_{\mathcal{T}}E^{n}}\|\prod_{t\in\mathcal{T}}U_{\mathcal{M}_{t}}\times P_{E}^{n})$	$\displaystyle=C_{1}(M_{\mathcal{T}}|E^{n})$		(20)
		$\displaystyle\leq C_{1+s}(M_{\mathcal{T}}|E^{n}).$		(21)

Thus, our results in Lemma 1 can be used to analyze the secrecy constraints in key generation problems if the constraints are expressed in terms of KL divergences or mutual information terms as in existing literature.

It is not apparent how one can use Lemma 1 for this purpose. To illustrate this, in the following, we briefly discuss the case in a private key generation problem involving three terminals: two legitimate users Alice and Bob, observing sequences $A_{1}^{n}$ and $A_{2}^{n}$ respectively, and one illegitimate user Eve who has access to side information $A^{n}$. Let $\mathbf{F}$ denote the public communication between Alice and Bob and let $K$ denote the secret key generated by them. To make sure the generated key is secure, we need $I(K;A^{n},\mathbf{F})$ to be small and to make sure the generated key is uniform, we need to make $D(P_{K}\|\mathrm{U}_{K})$ to be small where $\mathrm{U}_{K}$ is the uniform distribution over the alphabet of the secret key.

Note that in key generation problems, in the achievability part, the public communication $\mathbf{F}$ is usually the random binning $(M_{1},M_{2})$ of observations $(A_{1}^{n},A_{2}^{n})$ at legitimate users and the secret key $K$ is usually obtained by applying a hash function on a commonly agreed binning sequence (which is correlated with $(A_{1}^{n},A_{2}^{n})$). Thus, we have that

	$\displaystyle D(P_{KA^{n}\mathbf{F}}\|P_{\mathrm{U}}\times P_{A^{n}}P_{\mathbf{F}})$	$\displaystyle=D(P_{K}\|\mathrm{U}_{K})+I(K;A^{n},M_{1},M_{2})$		(22)
		$\displaystyle\leq D(P_{KM_{1}M_{2}A^{n}}\|\mathrm{U}_{K}\times\mathrm{U}_{M_{1}}\mathrm{U}_{M_{2}}\times P_{A}^{n}),$		(23)

where $U_{M_{i}},~{}i\in[2]$ is the uniform distribution over the alphabet of random binning.

Therefore, using Lemma 1 and (21), we have

• •

  if (17) is satisfied, then the secrecy constraint satisfies $\frac{1}{n}D(P_{KA^{n}\mathbf{F}}\|\mathrm{U}_{K}\times P_{A^{n}\mathbf{F}})$ vanishes to zero as $n\to\infty$ and thus ensures weak secrecy;

• •

  if the right hand side of (19) is always positive, then the secrecy constraint $D(P_{KA^{n}\mathbf{F}}\|\mathrm{U}_{K}\times P_{A^{n}\mathbf{F}})$ decays exponentially fast and thus ensures strong secrecy.

In the remaining of this paper, to illustrate in detail how the result in Lemma 1 can be used in analyses of secrecy constraints, we consider a multiple private key generation problem and derive the capacity region for the problem under mild conditions.

Let $P_{A_{0}A_{1}A_{2}A_{3}}$ be a joint probability density function (pdf) of random variables $(A_{0},A_{1},A_{2},A_{3})$ defined on a continuous alphabet $\mathcal{A}_{0}\times\mathcal{A}_{1}\times\mathcal{A}_{2}\times\mathcal{A}_{3}$. We assume that the pdf $P_{A_{0}A_{1}A_{2}A_{3}}$ satisfies that for any non-empty set $\mathcal{S}\subseteq\{0,1,2,3\}$, the (joint) differential entropy $h(A_{\mathcal{S}})$ is finite, i.e.,

$\displaystyle|h(A_{\mathcal{S}})|=|h(\{A_{t}\}_{t\in\mathcal{S}})|<\infty.$

(24)

Let $(A_{0}^{n},A_{1}^{n},A_{2}^{n},A_{3}^{n})$ be a sequence of continuous random variables generated i.i.d. according to a pdf $P_{A_{0}A_{1}A_{2}A_{3}}$.

In this subsection, we revisit the multiple key generation model [18] by studied Zhang et al. as shown in Figure 1. In this model, there are four terminals: Alice has access to $A_{1}^{n}$, Bob has access to $A_{0}^{n}$, Charlie has access to $A_{2}^{n}$ and Helen has access to $A_{3}^{n}$. It is assumed that there is a noiseless public channel and all terminals talk interactively in $r$ rounds. Let the overall messages transmitted over the public channel be $\mathbf{F}:=(F_{1},\ldots,F_{4r})$. For $j=\{1,\ldots,4r\}$, $F_{j}$ is a function of $A_{t}^{n}$ and previous messages $F^{j-1}$ where $t=j\mod 4$.

Let the alphabet of secret keys be $\mathcal{K}_{t}:=\{1,\ldots,K_{t}\}$ for $t\in[2]$. Using the public messages $\mathbf{F}$ and the source sequence $A_{1}^{n}$, Alice generates a private key $K_{\mathrm{A}}\in\mathcal{K}_{1}$. Using $(\mathbf{F},A_{0}^{n})$, Bob generates private keys $(K_{\rm{BA}},K_{\rm{BC}})\in\mathcal{K}_{1}\times\mathcal{K}_{2}$. Using $(\mathbf{F},A_{2}^{n})$, Charlie generates $K_{\mathrm{C}}\in\mathcal{K}_{2}$. We require that Alice and Bob agree on a private private key while Charlie and Bob agree on another private key, i.e. $K_{\mathrm{A}}=K_{\rm{BA}}$ and $K_{\mathrm{C}}=K_{\rm{BC}}$. A private key generation protocol consists of the public communication $\mathbf{F}$. Note that in the above model, Helen is an untrusted helper who helps other terminals by transmitting messages over the public channel so that other terminals can obtain common sequences for subsequent key generations.

In [18], the authors considered four models with different secrecy requirements for discrete memoryless sources, depending on whether $(K_{\mathrm{A}},K_{\mathrm{C}})$ is known by Helen and whether $K_{\mathrm{C}}$ is known by Alice. Our setting differs from [18] in the following two aspects:

1. (i)

   We consider different secrecy requirements on generated keys. To be specific, we require the private key $K_{\mathrm{A}}$ is only known by Alice and Bob and the private key $K_{\mathrm{C}}$ is only known by Bob and Charlie.

2. (ii)

   We consider continuous memoryless sources, which requires different techniques in the analyses and derivation of fundamental limits concerning the performance of optimal protocols.

We then give a formal definition of the capacity region of multiple private key generation with a helper, which concerns the asymptotic fundamental limits of optimal protocols.

Note that (26), (27) imply that i) the generated key $K_{\mathrm{A}}$ is almost uniform over $\mathcal{K}_{1}$ and independent of $(\mathbf{F},A_{2}^{n},A_{3}^{n})$ and ii) $K_{\mathrm{C}}$ is almost uniform over $\mathcal{K}_{2}$ and independent of $(\mathbf{F},A_{1}^{n},A_{3}^{n})$. Furthermore, the secrecy requirements in (26), (27) are strong in contrast to the weak ones in [18].

To present our result, we need the following definition. Let $\mathcal{R}^{*}$ be the set of pairs $(R_{1},R_{2})$ such that

	$\displaystyle R_{1}$	$\displaystyle\leq I(A_{1};A_{0}|A_{2},A_{3}),$		(29)
	$\displaystyle R_{2}$	$\displaystyle\leq I(A_{2};A_{0}|A_{1},A_{3}),$		(30)
	$\displaystyle R_{1}+R_{2}$	$\displaystyle\leq I(A_{1},A_{2};A_{0}|A_{3})-I(A_{1};A_{2}|A_{3}),$		(31)

where the mutual information is calculated with respect to the pdf $P_{A_{0}A_{1}A_{2}A_{3}}$ or its induced pdfs.

The proof of Theorem 2 is given in Section IV. In the achievability proof, we first quantize the continuous source sequence similarly as in the proof of [22, Theorem 1]. Then, the terminals communicate over the public channel so that the quantized version of $(A_{0}^{n},A_{1}^{n})$ are decoded almost surely by Bob who observes $A_{0}^{n}$. The reliability analysis (cf. (25)) for key agreement proceeds similarly as the error exponent analysis for Slepian-Wolf coding introduced in [31]. The secrecy analysis (cf. (26), (26)) follows by invoking (19) in Lemma 1. Subsequently, we need to apply Fourier Motzkin Elimination to obtain the conditions on achievable rate pairs. Finally, as the quantization level goes to infinity, we show that any rate pair inside $\mathcal{R}^{*}$ is achievable by exploring the relationship between the differential entropy of continuous random variables and the discrete entropy of the quantized random variables.

We remark that Theorem 2 holds also for DMS, as can be gleaned from the proof. Furthermore, we can derive the achievable reliability-secrecy exponent which is positive for rate pairs inside $\mathcal{R}^{*}$. We remark that Lemma 1, especially Eq. (19), is critical to derive secrecy exponents [11, 12] for key generation problems of DMS. This means that, we can not only show that Eq. (26) and Eq. (27) hold, but also derive a lower bound on the speed at which the secrecy constraints in Eq. (26) and Eq. (27) vanish to zero exponentially as the length of observed sequences tends to infinity. This is yet another advantage of our method beyond quantization based techniques in [15] which could only be used to show that secrecy constraints vanish to zero but not the manner or the rate of decay.

The proof of Corollary 3 is given in Section V. When the Markov chain $A_{1}-A_{3}-A_{2}$ holds, we have $I(A_{1};A_{2}|A_{3})=0$. The achievability part follows from Theorem 2 and the converse part follows by judiciously adapting the converse techniques in [22] to our setting. We remark that the proof techniques used to prove Theorem 2 and Corollary 3 can also be applied to all the four models in [18] and thus show that the capacity results in [18] also hold for CMS with strong secrecy.

Recall that $\mathcal{T}=\{1,\ldots,T\}$. For each $t\in\mathcal{T}$, define an alphabet of keys as $\mathcal{K}_{t}:=\{1,\ldots,K_{t}\}$. Let $(A_{\mathcal{T}},A_{0})$ be distributed according to a joint pdf $P_{A_{0}A_{\mathcal{T}}}$ with zero mean vector and covariance matrix $\Sigma$. In this subsection, we consider a cellular model where there is a base station and $T$ terminals. This model is a generalization of our setting in Section III-A in the spirit of [4] and has potential applications in internet of things where multiple terminals need to generate private keys with the help of other (potentially untrusted) terminals.

The base station observes the source sequence $A_{0}^{n}$ and for $t\in\mathcal{T}$, terminal $t$ observes the source sequence $A_{t}^{n}$. Fix arbitrary subset $\mathcal{S}$ of $\mathcal{T}$. For each $t\in\mathcal{S}$, terminal $t$ aims to generate a private key with the base station, concealed from all other terminals. We assume that the public communication is done in $r$ rounds over a noiseless public channel which is accessed by all terminals. Let $\mathbf{F}$ denote the overall communication over the public channel. For each $t\in\mathcal{S}$, given $\mathbf{F}$ and $A_{t}^{n}$, terminal $t$ generates a private key $K_{\mathrm{P},t}\in\mathcal{K}_{t}$. Furthermore, given $A_{0}^{n}$ and $\mathbf{F}$, the base station generates a sequence of private keys $\{K_{\mathrm{B},t}\}_{t\in\mathcal{S}}$. The goal of a good protocol is to enable the base station and each terminal $t\in\mathcal{S}$ to generate an agreed private key, which is concealed from all other terminals.

The capacity region for this cellular model is defined as follows.

Before presenting the main results, we need the following definitions. Note that for $\mathcal{U}\subseteq\mathcal{S}$, $\mathcal{U}\bigcup\mathcal{U}^{\mathrm{c}}=\mathcal{T}$.

	$\displaystyle\mathcal{R}_{\rm{in}}$	$\displaystyle:=\Big{\{}R_{\mathcal{S}}:~{}\forall~{}\emptyset\neq\mathcal{U}\subseteq\mathcal{S}:~{}\sum_{t\in\mathcal{U}}R_{t}\leq\sum_{t\in\mathcal{U}}h(A_{t}|A_{\mathcal{T}\bigcap\{t\}^{\mathrm{c}}})-h(A_{\mathcal{U}}|A_{\mathcal{U}^{\mathrm{c}}},A_{0})\Big{\}}$		(36)
	$\displaystyle\mathcal{R}_{\rm{out}}$	$\displaystyle:=\Big{\{}R_{\mathcal{S}}:~{}\forall~{}\emptyset\neq\mathcal{U}\subseteq\mathcal{S}:~{}\sum_{t\in\mathcal{U}}R_{t}\leq h(A_{\mathcal{U}}|A_{\mathcal{U}^{\mathrm{c}}})-h(A_{\mathcal{U}}|A_{\mathcal{U}^{\mathrm{c}}},A_{0})\Big{\}}.$		(37)

The proof of Theorem 4 is omitted since it is a generalization of the proofs of Theorem 2 and Corollary 3. In fact, we can recover the result in 2 and Corollary 3 by letting $\mathcal{T}=\{1,2,3\}$ and $\mathcal{S}=\{1,2\}$.

Here we provide only the proof sketch. In the achievability proof, we need to first quantize the source sequence at each terminal $t\in\mathcal{T}$ and thus obtain $B_{t}^{n}$. Then, for $t\in\mathcal{S}$, terminal $t$ sends a message $M_{t}\in\mathcal{M}_{t}$ over the public channel and generates a private key $K_{\mathrm{P},t}$ using $B_{t}^{n}$. For $t\in\mathcal{S}^{\mathrm{c}}$, terminal $t$ sends the complete quantized source sequence. Thus, the public message $\mathbf{F}=(M_{\mathcal{S}},B_{\mathcal{S}^{\mathrm{c}}}^{n})$. Given $A_{0}^{n}$ and $\mathbf{F}$, the base station estimates $B_{\mathcal{S}}^{n}$ and generate private key $K_{\mathrm{B},t}$ using $\mathbf{F}$ and the estimated sequences $\hat{B}_{t}^{n}$ for all $t\in\mathcal{S}$. The error probability in key agreement is derived by using the distributed source coding idea and the secrecy analysis is done by invoking (19) in Lemma 1. Let $\tilde{R}_{t}$ be the rate of the message at terminal $t$ and let the quantization interval go to zero. To satisfy (33) and (34), we conclude that the rates should satisfy that for any positive $\delta$,

	$\displaystyle R_{t}+\tilde{R}_{t}$	$\displaystyle\leq H(A_{t}|A_{\mathcal{T}\bigcap\{t\}^{\mathrm{c}}})-\delta,$		(39)
	$\displaystyle\sum_{j\in\mathcal{U}}\tilde{R}_{j}$	$\displaystyle\geq H(A_{\mathcal{U}}|A_{\mathcal{U}^{\mathrm{c}}},A_{0})+\delta.$		(40)

for each $t\in\mathcal{S}$ and for each non-empty subset $\mathcal{U}$ of $\mathcal{S}$. Without loss of generality, we can assume that $\mathcal{S}=\{1,\ldots,|\mathcal{S}|\}$. By applying the Fourier Motzkin Elimination successively to eliminate $\tilde{R}_{t}$ for all $t\in\mathcal{S}$, we conclude that any $R_{\mathcal{S}}\in\mathcal{R}_{\rm{in}}$ is achievable.

The converse proof proceeds similarly as Corollary 3 by assuming that there exists a super terminal observing $A_{\mathcal{U}}^{n}$ and generating secret keys $K_{\mathrm{P},\mathcal{U}}:=\{K_{\mathrm{P},t}\}_{t\in\mathcal{U}}$ for each non-empty subset $\mathcal{U}$ of $\mathcal{S}$. This is possible since $T$ is finite and (34) implies that for any non-empty subset $\mathcal{U}$ of $\mathcal{S}$, we have that for any positive $\delta$ and sufficiently large $n$,

	$\displaystyle D(P_{K_{\mathrm{P},\mathcal{U}}\mathbf{F}A_{\mathcal{U}^{\mathrm{c}}}^{n}}\|\prod_{t\in\mathcal{U}}U_{\mathcal{M}_{t}}\times P_{\mathbf{F}A_{\mathcal{U}^{\mathrm{c}}}^{n}})$	$\displaystyle\leq\sum_{t\in\mathcal{U}}\sum_{t\in\mathcal{U}}D(P_{K_{\mathrm{P},t}\mathbf{F}A^{n}_{\mathcal{T}\bigcap\{t\}^{\mathrm{c}}}}\|U_{\mathcal{K}_{t}}\times P_{\mathbf{F}A^{n}_{\mathcal{T}\bigcap\{t\}^{\mathrm{c}}}})$		(41)
		$\displaystyle\leq|\mathcal{U}|\delta\leq T\delta.$		(42)

Fix an integer $q$. Let $g_{q}:\mathcal{R}\to[0,1,\ldots,2q^{2}]$ be a quantization function with quantization level $\Delta=\frac{1}{q}$ such that $g_{q}(a)=0$ if $a\leq-q$ or $a>q$ and $g_{q}(a)=\lceil q(q+a)\rceil$ if $a\in(-q,q]$. Note that the quantized random variable has a finite alphabet $\mathcal{B}=[0,1,\ldots,2q^{2}]$ with the size $2q^{2}+1$. Applying the quantization function $Q$ on all $\{A_{t}\}_{t\in\mathcal{T}}$ to obtain quantized version $\{B_{t}\}_{t\in\mathcal{T}}$. We first quantize the sequences $A_{t}^{n}$ using the function $g_{q}$ and obtain corresponding quantized sequences $B_{t}^{n}=g_{q}(A_{t}^{n})$ for $t=0,1,2,3$.

Let $X^{5}=(X_{1},X_{2},X_{3},X_{4},X_{5})$ be a sequence of independent random variables. Let $f_{X_{t}}$ be an universal₂ random hash function mapping from $\mathcal{B}_{t}^{n}$ to $\mathcal{M}_{t}=\{1,\ldots,N_{t}\}$ for $t=1,2,3$ where $X_{t}$ describes the stochastic behavior of the hash function. Similarly, let $f_{X_{t+3}}$ be random hash function mapping from $\mathcal{B}_{t}^{n}$ to $\mathcal{K}_{t}=\{1,\ldots,K_{t}\}$ for $t=1,2$. Furthermore, for any positive $\delta$, let $\log N_{t}=n\tilde{R}_{t}$ for $t=1,2,3$ and $\log K_{t}=nR_{t}$ for $t=1,2$.

Codebook Generation: The code book generated by Alice is $\mathcal{C}_{\mathrm{A}}:=\bigcup_{a_{1}^{n}\mathcal{A}_{1}^{n}}(f_{X_{1}}(g_{q}(a_{1}^{n})),f_{X_{4}}(g_{q}(a_{1}^{n})))$. The codebook generated by Charlie is $\mathcal{C}_{\mathrm{C}}:=\bigcup_{a_{2}^{n}\mathcal{A}_{2}^{n}}(f_{X_{2}}(g_{q}(a_{2}^{n})),f_{X_{5}}(g_{q}(a_{2}^{n})))$. The codebook generated by Helen is $\mathcal{C}_{\mathrm{H}}:=\bigcup_{a_{3}^{n}\mathcal{A}_{3}^{n}}f_{X_{3}}(g_{q}(a_{3}^{n}))$. The random codebook $\mathcal{C}_{X^{5}}:=\{\mathcal{C}_{\mathrm{A}},\mathcal{C}_{\mathrm{C}},\mathcal{C}_{\mathrm{H}}\}$ controlled by random variables $X^{5}$ is assumed to be known by all users Alice, Bob, Charlie and Helen.

Encoding: Recall that $B_{t}^{n}=g_{q}(A_{t}^{n})$ for $t=0,1,2,3$. Given $A_{1}^{n}$, Alice sends $m_{1}:=f_{X_{1}}(B_{1}^{n})$ over the public channel and takes $f_{X_{4}}(B_{1}^{n})$ as the private key $K_{\mathrm{A}}$. Given $A_{2}^{n}$, Charlie sends $m_{2}:=f_{X_{2}}(B_{2}^{n})$ and takes $f_{X_{5}}(B_{2}^{n})$ as the private key $K_{\mathrm{C}}$. Given $A_{3}^{n}$, Helen sends $m_{3}:=f_{X_{3}}(B_{3}^{n})$ over the public channel.

Decoding: Let $P_{B_{0}B_{1}B_{1}B_{3}}$ be induced by $P_{A_{0}A_{1}A_{2}A_{3}}$ and the quantization function $g_{q}$. Given the messages $\mathbf{F}=(m_{1},m_{2},m_{3})$ transmitted over the public discussion channel and the source sequence $A_{0}^{n}$, Bob uses maximum likelihood decoding to obtain $(\hat{B}_{1}^{n},\hat{B}_{2}^{n},\hat{B}_{3}^{n})$, i.e.,

$\displaystyle(\hat{B}_{1}^{n},\hat{B}_{2}^{n},\hat{B}_{3}^{n})$

$\displaystyle:=\operatorname*{arg\,max}_{\begin{subarray}{c}(\tilde{b}_{1}^{n},\tilde{b}_{2}^{n},\tilde{b}_{3}^{n}):\\ f_{X_{t}}(\tilde{b}_{t}^{n})=m_{t},~{}t=1,2,3\end{subarray}}P_{B_{1}B_{2}B_{3}|B_{0}}^{n}(\tilde{b}_{1}^{n},\tilde{b}_{2}^{n},\tilde{b}_{3}^{n}|B_{0}^{n}).$

(43)

Then, Bob claims that $K_{\rm{BA}}=f_{X_{4}}(\hat{B}_{1}^{n})$ and $K_{\rm{BC}}=f_{X_{5}}(\hat{B}_{3}^{n})$.

Given the above coding strategy, we obtain that

$\displaystyle\max\big{\{}\Pr\{K_{\mathrm{A}}\neq K_{\rm{BA}}\},\Pr\{K_{\mathrm{C}}\neq K_{\rm{BC}}\}\big{\}}$

$\displaystyle\leq\Pr\Big{\{}(\hat{B}_{1}^{n},\hat{B}_{2}^{n},\hat{B}_{3}^{n})\neq(B_{1}^{n},B_{2}^{n},B_{3}^{n})\Big{\}}.$

(44)

Note that the average is not only over all possible realizations of source sequences but also over all possible random universal₂ hash functions. Recall that in this section $\mathcal{T}=\{1,2,3\}$ and all the quantized random variable have the same alphabet $\mathcal{B}$. Given $\emptyset\neq\mathcal{S}\subseteq\mathcal{T}$ and $a_{0}^{n}\in\mathcal{R}^{n}$, define the error events:

	$\displaystyle\mathcal{E}_{\mathcal{S}}$	$\displaystyle:=\Big{\{}\tilde{b}_{\mathcal{T}}^{n}\in\mathcal{B}^{3n}:~{}\forall~{}t\in\mathcal{T},~{}f_{X_{t}}(\tilde{b}_{t}^{n})=m_{t},~{}\forall~{}t\in\mathcal{S},~{}\tilde{b}_{t}^{n}\neq B_{t}^{n},$
		$\displaystyle\qquad\quad\forall~{}t\in\mathcal{T}\bigcap\mathcal{S}^{\mathrm{c}},~{}\tilde{b}_{t}^{n}=B_{t}^{n},~{}P_{B_{\mathcal{T}}|B_{0}}^{n}(\tilde{b}_{\mathcal{T}}^{n}|B_{0}^{n})\geq P_{B_{\mathcal{T}}|B_{0}}^{n}(B_{\mathcal{T}}^{n}|B_{0}^{n})\Big{\}}.$		(45)

Then, similarly as [31], we have that for any $\emptyset\neq\mathcal{S}\subseteq\mathcal{T}$ and arbitrary $s\in[0,1]$,

	$\displaystyle\Pr\big{\{}\exists\tilde{b}_{\mathcal{T}}^{n}\in\mathcal{E}_{\mathcal{S}}\big{\}}$
	$\displaystyle=\mathbb{E}_{X_{\mathcal{S}}}\Big{[}\sum_{b_{\mathcal{T}}^{n}}P_{B_{\mathcal{T}}}^{n}(b_{\mathcal{T}}^{n})\Pr\big{\{}\exists\tilde{b}_{\mathcal{T}}^{n}\in\mathcal{E}_{\mathcal{S}}|b_{\mathcal{T}}^{n}\big{\}}\Big{]}$		(46)
	$\displaystyle\leq\sum_{b_{0}^{n},b_{\mathcal{T}}^{n}}P_{B_{0},B_{\mathcal{T}}}^{n}(b_{0}^{n},b_{\mathcal{T}}^{n})\Bigg{(}\sum_{\begin{subarray}{c}\tilde{b}_{\mathcal{S}}^{n}\end{subarray}}1\{P_{B_{\mathcal{T}}|B_{0}}^{n}(\tilde{b}_{\mathcal{S}}^{n},b_{\mathcal{T}\bigcap\mathcal{S}^{\mathrm{c}}}^{n}|b_{0}^{n})\geq P_{B_{\mathcal{T}}|B_{0}}^{n}(b_{\mathcal{T}}^{n}|b_{0}^{n})\}\mathbb{E}_{X_{\mathcal{S}}}\Big{[}1\{f_{X_{t}}(\tilde{b}_{t}^{n})=f_{X_{t}}^{n}(b_{t}^{n}),~{}t\in\mathcal{S}\}\Big{]}\Bigg{)}^{s}$		(47)
	$\displaystyle\leq\sum_{b_{0}^{n},b_{\mathcal{T}}^{n}}P_{B_{0},B_{\mathcal{T}}}^{n}(b_{0}^{n},b_{\mathcal{T}}^{n})\Bigg{(}\sum_{\begin{subarray}{c}\tilde{b}_{\mathcal{S}}^{n}\end{subarray}}\frac{P_{B_{\mathcal{S}}|B_{0}B_{\mathcal{T}\bigcap\mathcal{S}^{\mathrm{c}}}}^{n}(\tilde{b}_{\mathcal{S}}^{n}|b_{0}^{n},b_{\mathcal{T}\bigcap\mathcal{S}^{\mathrm{c}}}^{n})}{P_{B_{\mathcal{S}}|B_{0}B_{\mathcal{T}\bigcap\mathcal{S}^{\mathrm{c}}}}^{n}(b_{\mathcal{K}}^{n}|b_{0}^{n}b_{\mathcal{T}\bigcap\mathcal{S}^{\mathrm{c}}}^{n})}\times\prod_{t\in\mathcal{S}}\frac{1}{M_{t}}\Bigg{)}^{s}$		(48)
	$\displaystyle\leq\exp\Big{(}-ns\Big{(}\sum_{t\in\mathcal{S}}\tilde{R}_{t}-H_{1+s}^{\uparrow}(B_{\mathcal{S}}|B_{0},B_{\mathcal{T}\bigcap\mathcal{K}^{\mathrm{c}}})\Big{)},$		(49)

where (49) follows from the definition in (9). Thus, invoking (44) and (49), we obtain that

	$\displaystyle\max\big{\{}\Pr\{K_{\mathrm{A}}\neq K_{\rm{BA}}\},\Pr\{K_{\mathrm{C}}\neq K_{\rm{BC}}\}\big{\}}$
	$\displaystyle\leq\sum_{\emptyset\neq\mathcal{S}\subseteq\mathcal{T}}\Pr\big{\{}\exists\tilde{b}_{\mathcal{T}}^{n}\in\mathcal{E}_{\mathcal{S}}\big{\}}$		(50)
	$\displaystyle\leq\sum_{\emptyset\neq\mathcal{S}\subseteq\mathcal{T}}\exp\Big{\{}-n\max_{s\in[0,1]}s\Big{(}\sum_{t\in\mathcal{S}}\tilde{R}_{t}-H_{1+s}^{\uparrow}(B_{\mathcal{S}}|B_{0},B_{\mathcal{S}^{\mathrm{c}}})\Big{)}\Big{\}}$		(51)
	$\displaystyle\leq(2^{T}-1)\times\exp\Big{\{}-n\min_{\emptyset\neq\mathcal{S}\subseteq\mathcal{T}}\max_{s\in[0,1]}s\Big{(}\sum_{t\in\mathcal{S}}\tilde{R}_{t}-H_{1+s}^{\uparrow}(B_{\mathcal{S}}|B_{0},B_{\mathcal{S}^{\mathrm{c}}})\Big{)}\Big{\}}.$		(52)