Improved bounds for noisy group testing with constant tests per item

Oliver Gebhard, Oliver Johnson, Philipp Loick, Maurice Rolvien {Gebhard, Loick, Rolvien}@math.uni-frankfurt.de, Goethe University, Mathematics Institute,
10 Robert Mayer St, Frankfurt 60325, Germany. O.Johnson@bristol.ac.uk, University of Bristol, School of Mathematics,
Woodland Road, Bristol, BS8 1UG, United Kingdom

Abstract.

The group testing problem is concerned with identifying a small set of infected individuals in a large population. At our disposal is a testing procedure that allows us to test several individuals together. In an idealized setting, a test is positive if and only if at least one infected individual is included and negative otherwise. Significant progress was made in recent years towards understanding the information-theoretic and algorithmic properties in this noiseless setting. In this paper, we consider a noisy variant of group testing where test results are flipped with certain probability, including the realistic scenario where sensitivity and specificity can take arbitrary values. Using a test design where each individual is assigned to a fixed number of tests, we derive explicit algorithmic bounds for two commonly considered inference algorithms and thereby naturally extend the results of Scarlett & Cevher (2016) and Scarlett & Johnson (2020). We provide improved performance guarantees for the efficient algorithms in these noisy group testing models – indeed, for a large set of parameter choices the bounds provided in the paper are the strongest currently proved.

1. Introduction

1.1. Motivation and background

Suppose we have a large collection of $n$ people, a small number $k$ of whom are infected by some disease, and where only $m\ll n$ tests are available. In a landmark paper [16] from 1943, Dorfman introduced the idea of group testing. The basic idea is as follows: rather than screen one person using one test, we could mix samples from individuals in one pool, and use a single test for this whole pool. The task is to recover the infection status of all individuals using the pooled test results. Dorfman’s original work was motivated by a biological application, namely identifying individuals with syphilis. Subsequently, group testing has found a number of related applications, including detection of HIV [51], DNA sequencing [29, 37] and protein interaction experiments [35, 49]. More recently, it has been recognised as an essential tool to moderate pandemic spread [12], where identifiying infected individuals fast and at a low cost is indispensable [32]. In particular, group testing has been identified as a testing scheme for the detection of COVID-19 [2, 17, 21]. From a mathematical perspective, group testing is a prime example of an inference problem where one wants to learn a ground truth from (possibly noisy) measurements [1, 8, 15]. Over the last decade, it has regained popularity and a significant body of research was dedicated to understand its information-theoretic and algorithmic properties [9, 13, 14, 44, 45, 46]. In this paper, we provide improved upper bounds on the number of tests that guarantee successful inference for the noisy variant of group testing.

1.2. Related Work

1.2.1. Noiseless Group Testing

In the simplest version of group testing, we suppose that a test is positive if and only if the pool contains at least one infected individual. We refer to this as the noiseless case. In this setting, each negative test guarantees that every member of the corresponding pool is not infected, so they can be removed from further consideration. However, a positive test only tells us that at least one item in the test is defective (but not which one), and so requires further investigation. Dorfman’s original work [16] proposed a simple adaptive strategy where a small pool of individuals is tested, and where each positive test is followed up by testing every individual in the corresponding pool individually. Since then it has been an important problem to find the optimal way to recover the whole population’s infection status in the noiseless case (see [7] for a detailed survey). A simple counting argument (see for example [7, Section 1.4]) shows that to ensure recovery with zero error probability, since every possible defective set must give different test outcomes, the following must hold in the noiseless setting:

(1.1)

\displaystyle 2^{m}\geq\binom{n}{k}\qquad\Rightarrow\qquad m\geq m^{0}_{\inf}:=\frac{1}{\log 2}k\log(n/k)

This can be extended to the case of recovery with small error probability, for example with the bound (see [7, Eq. (1.7)]) that the success probability

(1.2)

\displaystyle\mathbb{P}(\rm{suc})\leq\frac{2^{m}}{\binom{n}{k}},

meaning that the success probability must decay exponentially with the number of tests below $m^{0}_{\inf}$ . Hwang [24] provided an algorithm based on repeated binary search, which is essentially optimal in terms of the number of tests required in that it requires $m^{0}_{\inf}+O(k)$ tests, but may require many stages of testing. The question of whether non-adaptive algorithms (or even adaptive algorithms with a limited number of stages) can attain the bound (1.1) remained open until recently. [4, 14] showed that the answer depends on the prevalence of the disease, for example on the value of $\theta\in(0,1)$ in a parameterisation¹¹1The result of [14] is two-fold. On the one hand, it provides a method to recover infected individuals w.h.p.as well as attaining (1.1) for a certain range of $\theta<\theta^{*}$ . On the other hand they show that (1.1) cannot be attained by any testing procedure for larger $\theta>\theta^{*}$ . One finds $\theta^{*}=\log(2)\cdot(1+\log(2))^{-1}$ . where the number of infected individuals $k\sim n^{\theta}$ . Non-adaptive testing schemes can be represented through a binary $(m\times n)$ -matrix that indicates which individual participates in which test. Significant research was dedicated to see which design attains the optimal performance, although much of the recent research analysed the performance of randomized designs. Initial research focused on the case where the matrix entries are i.i.d. [3, 5, 46], which we will refer to as Bernoulli pooling. Later work considered a constant column design where each individual is assigned to a (near-)constant number of tests [6, 13, 14, 26]. Indeed [14] showed that such a design is information-theoretically optimal in the noiseless setting and it is to be expected that this remains true for the noisy case. To recover the ground truth from the test results and the pooling scheme, this paper focuses on two non-adaptive algorithms, COMP and DD, which are relatively simple to perform and interpret in the noiseless case. We describe them in more detail below, but in brief COMP [10] simply builds a list of all the individuals who ever appear in a negative test and are hence certainly healthy, and assumes that the other individuals are infected. DD [5] uses COMP as a first stage and builds on it by looking for individuals who appear in a positive test that only otherwise contains individuals known to be healthy. While the noiseless case provides an interesting mathematical abstraction, it is clear that it may not be realistic in practice [40].

1.2.2. Noisy Group Testing

In medical applications [42] the two occurring types of noise in a testing procedure are related to sensitivity (the probability that a test containing an infected individual is indeed positive) and specificity (the probability that a test with only healthy individuals is indeed negative), and in that language we cannot assume the gold standard of tests with unit specificity and sensitivity. Thus, research attention in recent years has shifted towards the noisy version of group testing [10, 43, 44, 46, 47, 48]. On the one hand, the adaptive noisy case was considered in [43, 44]. On the other hand [10, 27, 28, 33, 46, 47, 48] looked at the non-adaptive noise case from different angles (for instance linear programming, belief propagation, and Markov Chain Monte Carlo). In [46, 47, 48] the algorithmic performance guarantees within noisy group testing under Bernoulli pooling are discussed. First of all [46] obtained a converse as well as a theoretical achievability bound, but stated the practical recovery as an direction for further research. In the following [47, 48] shed light on this question by using Bernoulli pooling.²²2[47] introduced an approach based on separate decoding of items for symmetric noise models. While this approach works well for small $\theta$ (in particular $\theta\rightarrow 0$ ), the performance drops dramatically for larger $\theta$ . For most $\theta$ this approach is worse off than the noisy DD discussed in [48]. Note there exist some noise levels with the very strong restriction assuming $p=q$ where [47] improve over our results in the $\theta$ very close to 0 regime. Due to the generality of our model we will from now on focus on [48] as benchmark for our results. In this paper we focus on the COMP and DD algorithms, since it is possible to deduce explicit performance guarantees for them. The original COMP and DD were designed for the noiseless case and do not automatically carry over to general noisy models. However, recent work of Scarlett and Johnson [48] showed that noisy versions of these algorithms can perform well under certain noise models using i.i.d. (Bernoulli pooling) test designs, particularly focusing on $Z$ channel and reverse $Z$ channel noise. As common medical tests have different values for sensitivity and specificity [31] the analysis of a generalized noise model beyond the $Z$ and reverse $Z$ channel is warranted.

1.2.3. Model Justification

As described for example in pandemic plans developed by the EU, US and WHO [19, 38, 39], and in COVID-specific work [36], adaptive strategies may not be suitable for pandemic prevention. For example, if a test takes one day to prepare and for the results to be known, then each stage will require an extra day to perform, meaning that adaptive group testing information can be received too late to be useful. Hence the need to perform large-scale testing to identify infected individuals fast relative to the doubling time [12, 32, 36] can make adaptive group testing unsuitable to prevent an infectious disease from spreading. Furthermore it may be difficult to preserve virus samples in a usable state for long enough to perform multi-round testing [22]. Due to its automation potential and the fact that tests can be completed in parallel (for example by the use of 96-well PCR plates [18]), the main applications of group testing such as DNA screening [11, 29, 37], HIV testing [51] and protein interaction analysis [35, 49] are non-adaptive, where all tests are specified upfront and performed in parallel. For example, while group testing strategies appear to be useful to identify individuals infected with COVID-19 (see for example [17, 21]), testing for the presence of the SARS-CoV-19 virus is not perfect [52], and so we need to understand the effect of both false positive and false negative errors in this context, with non-identical error probabilities. For this reason, we consider a general $p-q$ noise model in this paper. Under this model, a truly negative test is flipped with probability $p$ to display a positive test result, while a truly positive test is flipped to negative with probability $q$ (Figure 1). Its formulation is sufficiently general to accommodate the recovery of the noiseless results ( $p=q=0$ ), Z channel ( $p=0$ ), reverse Z channel ( $q=0$ ) and the Binary Symmetric Channel ( $p=q$ ). However, our results include the case of non-zero $p$ and $q$ without having to make the somewhat artificial assumption that false negative and false positive errors are equally likely. We note that it may be unrealistic to assume that the noise parameters are known exactly, and more sophisticated models may be needed to understand the real world. Nevertheless our analysis of a generalised noise model serves as a starting point towards a full understanding of the difficulties occurring while implementing group testing algorithms in laboratories.

Figure 1. The

p-q

-noise model: the result of each standard noiseless group test is transmitted independently through the given noisy communication channel.

1.3. Contribution

This paper provides a simultaneous extension of [13] and [26, 48], by analysing noisy versions of COMP and DD under more general noise models for constant-column weight designs. In contrast to prior work [5, 26] assuming sampling with replacement, in this paper we use sampling without replacement, meaning that our designs have exactly the same number of tests for each item, rather than approximately the same as in those previous works. This makes little difference in practice, but may be closer to the spirit of LDPC codes for example.

We provide explicit bounds on the performance of these algorithms in a generalized noise model. We will prove that (noisy versions of) COMP as well as DD succeed with $\Theta(k\log(n/k))$ tests. Our analysis reveals the exact constants to ensure the recovery with these two inference algorithms. The main results will be stated formally in Theorems 2.1 and 2.2, but we would like to give the reader a first insight of what will follow. We analyze Algorithms 1 and 2 for the constant degree model, where there are $m=ck\log(n/k)$ tests performed and each individual chooses $\Delta=cd\log(n/k)$ tests uniformly at random. Let $p,q\geq 0,p+q<1$ and $\epsilon>0$ .

We start with the performance of COMP (Algorithm 1), as stated in Theorem 2.1:

For any $\Delta:=\Delta(c,d)$ we find a threshold $\alpha:=\alpha(d,p,q)$ such that COMP succeeds in inferring the infected individuals if the number of tests

m\geq(1+\varepsilon)m_{COMP}=\min_{\alpha,d}\max\left\{{b_{1}(\alpha,d),b_{2}(\alpha,d)}\right\}k\log(n/k)

The next step on our agenda is the performance of DD (Algorithm 2), as stated in Theorem 2.1:

For any $\Delta:=\Delta(c,d)$ we find thresholds $\alpha:=\alpha(d,p,q)$ and $\beta:=\beta(d,q)$ such that DD succeeds in inferring the infected individuals if the number of tests

m\geq(1+\varepsilon)m_{\text{DD}}(n,\theta,p,q)=\min_{\alpha,\beta,d}\max\left\{{c_{1}(\alpha,d),c_{2}(\alpha,d),c_{3}(\beta,d),c_{4}(\alpha,\beta,d)}\right\}k\log(n/k)

For all typical noise channels (Z, reverse Z and BSC) we compare the constant-column and Bernoulli design and find for all such instances that the required number of tests in the former is lower than the number needed in the latter thereby improving on results from [48], and providing the strongest performance guarantees currently proved for efficient algorithms in noisy group testing.

As group testing offers an essential tool for pandemic prevention [32] and as the the accuracy of medical testing is limited [31, 40] this paper provides the natural next step in the group testing literature.

1.4. Test design and notation

To formalize our notation, we write $n$ for the number of individuals in the population, $\bm{\sigma}$ for a binary vector representing the infection status of each individual, $k$ (the Hamming weight of $\bm{\sigma}$ ) for the number of infected individuals and $m$ for the number of tests performed. We assume that $k$ is known for the purposes of matrix design, though in practice (see [7, Remark 2.3]) it is generally enough to know $k$ up to a constant factor to design a matrix with good properties. In this paper, in line with other work such as [5], we consider a scaling $k\sim n^{\theta}$ for some fixed $\theta\in(0,1)$ , referred to in [7, Remark 1.1] as the sparse regime³³3Note that the analysis directly extends to $k=\Theta(n^{\theta})$ as a constant factor in front does not influence the analysis.. In addition to the interesting phase transitions observed using this scaling, this sparse regime is particularly relevant as it was found suitable to model the early state of a pandemic [50].

Let us next introduce the test design. With $V=(x_{i})_{i\in[n]}$ denoting the set of $n$ individuals⁴⁴4 $[n]$ will be used as an abbreviated notation for the set $\left\{{1,\dots,n}\right\}$ . and $F=(a_{i})_{i\in[m]}$ the set of $m$ tests, the test design can be envisioned as a bipartite factor graph with $n$ variable nodes "on the left" and $m$ factor nodes "on the right". We draw a configuration $\bm{\sigma}\in\left\{{0,1}\right\}^{V}$ , encoding the infection status of each individual, uniformly at random from vectors of Hamming weight $k$ . The set of healthy individuals will be denoted by $V_{0}$ and the set of infected individuals by $V_{1}$ . In symbols,

\displaystyle V_{0}=\left\{{x\in V:\bm{\sigma}(x)=0}\right\}\qquad\text{and}\qquad V_{1}=V\setminus V_{0}=\left\{{x\in V:\bm{\sigma}(x)=1}\right\}

The lower bound from (1.1) suggests that in the noisy group testing setting it is natural to compare the performance of algorithms and matrix designs in terms of the prefactor of $k\log(n/k)$ in the number of tests required. To be precise, we carry out $m$ tests, and each item is assigned to exactly $\Delta$ tests chosen uniformly at random without replacement. We parameterize $m$ and $\Delta$ as

(1.3)

\displaystyle m=ck\log(n/k)\qquad\text{and}\qquad\Delta=cd\log(n/k)

for some suitably chosen constants $c,d\geq 0$ .

Let $\partial x$ denote the set of tests that individual $x$ appears in and $\partial a$ the set of individuals assigned to test $a$ . The resulting (non-constant) collection of test degrees will be denoted by the vector $\bm{\Gamma}=(\bm{\Gamma}_{a})_{a\in[m]}$ . Further, let

(1.4)

\displaystyle\Gamma_{\min}=\min_{a\in[m]}\Gamma_{a}\qquad\text{and}\qquad\Gamma_{\max}=\max_{a\in[m]}\Gamma_{a}.

Throughout, $\bm{G}=\bm{G}(n,m,\Delta)$ describes the random bipartite factor graph from this construction.

Now consider the outcome of the tests. Recall from above that a standard noiseless group test $a$ gives a positive result if and only if there is at least one defective item contained in the pool, or equivalently if $\sum_{x\in\partial a}\bm{\sigma}(x)\geq 1$ . Even in the noisy case, this sum is a useful object to consider. Writing $\bm{1}$ for the indicator function, we define

(1.5)

\bm{\sigma}^{*}(a)=\bm{1}\left\{{\sum_{x\in\partial a}\bm{\sigma}(x)\geq 1}\right\}

to be the outcome we would observe in the noiseless case using the test matrix corresponding to $\bm{G}$ . We will say that test $a$ is truly positive if $\bm{\sigma}^{*}(a)=1$ and truly negative otherwise.

However, we do not observe the values of $\bm{\sigma}^{*}(a)$ directly, but rather see what we will refer to as the displayed test outcomes $\hat{\bm{\sigma}}(a)$ – the outcomes of sending the true outcomes $\bm{\sigma}^{*}(a)$ independently through the $p-q$ channel of Figure 1. Since in this model a truly positive test remains positive with probability $1-q$ and a truly negative test is displayed as positive with probability $p$ we can write

(1.6)

\displaystyle\hat{\bm{\sigma}}(a)

\displaystyle=\bm{1}\left\{{{\rm Be}(p)=1}\right\}\left(1-\bm{\sigma}^{*}(a)\right)+\bm{1}\left\{{{\rm Be}(1-q)=1}\right\}\bm{\sigma}^{*}(a)

where ${\rm Be}(r)$ denotes a Bernoulli random variable with parameter $r$ independent of all other randomness in the model. For models with binary outputs, this is the most general channel satisfying the noisy defective channel property of [7, Definition 3.3], though more general models are possible under the only defects matter property [7, Definition 3.2], where the probability of a test being positive depends on the number of infected individuals it contains.

Note that if $p+q>1$ , we can preprocess the outputs from (1.6) by flipping them, i.e. setting ${\widetilde{p}}=1-p$ and ${\widetilde{q}}=1-q$ , where ${\widetilde{p}}+{\widetilde{q}}<1$ . Hence without loss of generality we will assume throughout that $p+q<1$ . In the case $p+q=1$ , the test outcomes are independent of the inputs, and we cannot hope to find the infected individuals – see Corollary 2.3.

With $\bm{m}_{0}$ being the number of truly negative tests, let $\bm{m}_{0}^{f}$ be the number of truly negative tests that are flipped to display a positive test result and $\bm{m}_{0}^{u}$ be the number of truly negative tests that are unflipped. Similarly, define $\bm{m}_{1}$ as the number of truly positive tests, of which $\bm{m}_{1}^{f}$ are flipped to a negative test result and of which $\bm{m}_{1}^{u}$ are unflipped. For reference, for $t\in\left\{{0,1}\right\}$ we write

	$\displaystyle\bm{m}_{t}=$	$\displaystyle\left\|{\left\{{a:\bm{\sigma}^{*}(a)=t}\right\}}\right\|$
	$\displaystyle\bm{m}_{t}^{f}=\left\|{\left\{{a:\bm{\sigma}^{*}(a)=t,\hat{\bm{\sigma}}(a)\neq t}\right\}}\right\|$	$\displaystyle\quad\text{and}\quad\bm{m}_{t}^{u}=\left\|{\left\{{a:\bm{\sigma}^{*}(a)=t,\hat{\bm{\sigma}}(a)=t}\right\}}\right\|$

Here we use bold letters to indicate random variables. Throughout the paper, we use the standard Landau notation $o(\cdot),O(\cdot),\Theta(\cdot),\Omega(\cdot),\omega(\cdot)$ and define $0\log 0=0$ . Furthermore we say that a property $\mathcal{P}$ holds with high probability ( w.h.p.), if $\mathbb{P}\left({\mathcal{P}}\right)=1$ as $n\to\infty$ . In order to quantify the performance of our algorithms, for any $0<r\neq s<1$ , we write

(1.7)

\displaystyle D_{\mathrm{KL}}\left({{{r}\|{s}}}\right):=r\log\left(\frac{r}{s}\right)+(1-r)\log\left(\frac{1-r}{1-s}\right),

for the relative entropy of a Bernoulli random variable with parameter $r$ to a Bernoulli random variable with parameter $s$ , commonly referred to as the Kullback–Leibler divergence. Here and throughout the paper we use $\log$ to denote the natural logarithm. For $r$ or $s$ equal to $0$ or $1$ we define the value of $D_{\mathrm{KL}}\left({{{\cdot}\|{\cdot}}}\right)$ (possibly infinite) on grounds of continuity, so for example $D_{\mathrm{KL}}\left({{{0}\|{s}}}\right)=-\log(1-s)$ .

2. Main results

With the test design and notation in place, we are now in a position to state our main results. Theorems 2.1, 2.2 are the centerpiece of this paper, featuring improved bounds for the noisy group testing problem for the general $p-q$ model. We follow up in Section 2.2 with a discussion of the combinatorics underlying both algorithms, and provide a converse bound in Section 2.3. Subsequently, in Section 2.4 we show how the bounds simplify when we consider the special cases of the Z, the reverse Z and Binary Symmetric Channel. Finally, in Section 2.5 we derive sufficient conditions under which DD requires fewer tests than the COMP algorithm and compare the bounds of our constant-column design against the Bernoulli design employed in prior literature.

2.1. Bounds for Noisy Group Testing

We will consider two well-known algorithms from the noiseless setting to identify infected individuals in this paper. First, we study a noisy variant of the COMP algorithm, originally introduced in [10].

1 Declare every individual that appears in

\alpha\Delta

or more displayed negative tests as healthy.

Declare all remaining individuals as infected.

Algorithm 1 The noisy COMP algorithm

Note that for $\alpha\Delta=1$ the formulation of Algorithm 1 coincides with the standard ${\tt COMP}$ algorithm where an individual is classified as healthy if it appears in at least one displayed negative test which constitutes a sufficient condition in the noiseless case. We now state the first main result of this paper.

Theorem 2.1 (Noisy COMP).

Let $p,q\geq 0$ , $p+q<1,d\in(0,\infty),\alpha\in(q,e^{-d}(1-p)+\left({1-e^{-d}}\right)q)$ . Suppose that $0<\theta<1$ and let

	$\displaystyle m_{\text{COMP}}$	$\displaystyle=m_{\text{COMP}}(n,\theta,p,q)=\min_{\alpha,d}\max\left\{{b_{1}(\alpha,d),b_{2}(\alpha,d)}\right\}k\log(n/k)$
		$\displaystyle\text{where}\qquad b_{1}(\alpha,d)=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{q}}}\right)}$
		$\displaystyle\text{and}\qquad b_{2}(\alpha,d)=\frac{1}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{e^{-d}(1-p)+\left({1-e^{-d}}\right)q}}}\right)}$

If $m\geq(1+\varepsilon)m_{\text{COMP}}$ for some $\varepsilon>0$ , noisy COMP will recover $\bm{\sigma}$ w.h.p. given test design $\bm{G}$ and test results $\hat{\bm{\sigma}}$ .

The noisy variant of the DD algorithm of [5] was introduced in [48] and reads as follows:

1 Declare every individual that appears in

\alpha\Delta

or more displayed negative tests as healthy and remove such individual from every assigned test.

2 Declare every yet unclassified individual who is now the only unclassified individual in

\beta\Delta

or more displayed positive tests as infected.

Declare all remaining individuals as healthy.

Algorithm 2 The noisy DD algorithm [48]

Note that the formulation of Algorithm 2 reduces to the noiseless version of DD introduced in [5] by taking $\alpha\Delta=\beta\Delta=1$ . This is because in the noiseless setting a single negative test or a single positive test with just individuals already classified as uninfected is sufficient in the noiseless case. Furthermore note that for $\beta=0$ noisy DD and noisy COMP are the same. From now on we assume $\beta>0$ . The proof of Theorem 2.1 can be found in Appendix B. We now state the second main result of the paper.

Theorem 2.2 (Noisy DD).

Let $p,q\geq 0$ , $p+q<1,d\in(0,\infty),\alpha\in(q,e^{-d}(1-p)+\left({1-e^{-d}}\right)q)$ and $\beta\in(0,e^{-d}(1-q))$ and define $w=e^{-d}p+(1-e^{-d})(1-q)$ . Suppose that $0<\theta<1$ and let

	$\displaystyle m_{\text{DD}}$	$\displaystyle=m_{\text{DD}}(n,\theta,p,q)=\min_{\alpha,\beta,d}\max\left\{{c_{1}(\alpha,d),c_{2}(\alpha,d),c_{3}(\beta,d),c_{4}(\alpha,\beta,d)}\right\}k\log(n/k)$
		$\displaystyle\text{where}\qquad c_{1}(\alpha,d)=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{q}}}\right)}$
		$\displaystyle\text{and}\qquad c_{2}(\alpha,d)=\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{1-w}}}\right)}$
		$\displaystyle\text{and}\qquad c_{3}(\beta,d)=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\beta}\\|{(1-q)e^{-d}}}}\right)}$
		$\displaystyle\text{and}\qquad c_{4}(\alpha,\beta,d)=\max_{1-\alpha\leq z\leq 1}\left\{{\frac{1}{1-\theta}\frac{1}{d\left({D_{\mathrm{KL}}\left({{{z}\\|{w}}}\right)+\bm{1}\left\{{\beta>\frac{ze^{-d}p}{w}}\right\}zD_{\mathrm{KL}}\left({{{\frac{\beta}{z}}\\|{\frac{e^{-d}p}{w}}}}\right)}\right)}}\right\}$

If $m\geq(1+\varepsilon)m_{\text{DD}}$ for some $\varepsilon>0$ , then noisy DD will recover $\bm{\sigma}$ w.h.p. given test design $\bm{G}$ and test results $\hat{\bm{\sigma}}$ .

The proof of Theorem 2.2 can be found in Appendix C. While the bounds appear cumbersome at first glance, the optimization is of finite dimension and for every specific value of $p$ and $q$ can be efficiently solved to arbitrary precision yielding explicit values for $m_{\text{COMP}}$ and $m_{\text{DD}}$ . For illustration purposes, we will calculate those bounds for several values of $p,q$ and $\theta$ .

2.2. The combinatorics of the noisy group testing algorithms

In the following, we outline the combinatorial structures that Algorithm 1 and 2 take advantage of.
We start with defining the three types of tests that are relevant for the classification of an individual $x_{i}$ while using COMP and DD. In the first stage we find

•

Type DN: Displayed negative tests
•

Type DP: Displayed positive tests

Note that the only available information during the first stage of the algorithms is the test result and the pooling structure – no information about the individuals’ infection status is available. We give an illustration on the left hand side of Figure 2. After this step COMP terminates by declaring all remaining individuals as infected.

The DD algorithm continues with a second step which considers just the displayed positive tests. From the first step of the algorithm one receives the estimate of the set of non-infected individuals obtained in the first round. Now distinguish the following two types, illustrated on the right hand side in Figure 2:

•

Type Displayed-Positive-Single (DP-S): Displayed positive tests in which all other individuals are already declared as uninfected.
•

Type Displayed-Positive-Multiple (DP-M): Displayed positive tests with at least one other individual that is not contained in the estimated set of uninfected individuals.

2.2.1. The noisy COMP algorithm

To get started, let us shed light on the combinatorics of noisy COMP (Algorithm 1). For the noiseless case, the COMP algorithm classifies each individual that appears in at least one negative test as healthy and all other individuals as infected, since the participation in a negative test is a sufficient condition for the individual to be healthy.

For the noisy case, the situation is not as straightforward, since an infected individual might appear in displayed negative tests that were flipped when sent through the noisy channel. Thus, a single negative test is not definitive evidence that an individual is healthy. Yet, we can use the number of negative tests to tell the infected individuals apart from the healthy individuals.

Clearly, noisy COMP (Algorithm 1) using a threshold $\alpha\Delta$ succeeds if no healthy individual appears in fewer than $\alpha\Delta$ displayed negative tests and no infected individual appears in more than $\alpha\Delta$ displayed negative tests. To this end, we define

(2.1)

\displaystyle\bm{N}_{x}=\left|{\left\{{a\in\partial x:\hat{\bm{\sigma}}(a)=0}\right\}}\right|

for the number of displayed negative tests that item $x$ appears in. In terms of Figure 2, the algorithm determines the infection status by counting the number of tests of Type DN.

Figure 2. The relevant neighborhood structures for the analysis of the algorithms, on the left for the first stage and on the right for the second step. Rectangles represent tests (displayed positive in red, displayed negative in blue). Blue circles represent individuals that have been classified as healthy in the first step of DD (or by COMP). White circles represent individuals that are unclassified in the current stage. We refer to displayed negative tests as Type DN, displayed positive tests as Type DP, displayed positive with a single unclassified individual as Type DP-S and displayed positive with a multiple unclassified individual as Type DP-M

2.2.2. The noisy DD algorithm

As in the prior section, let us first consider the noiseless DD algorithm. The first step is identical to COMP classifying all individuals that are contained in at least one negative test as healthy. In a second step, the algorithm checks each individual to see if it is contained in a positive test as the only remaining unclassified individual after the first step of the algorithm and thus must be infected.

Again, the situation is more intricate when we add noise, since neither a single negative test gives us confidence that an individual is healthy nor does a positive test where the individual is the single remaining unclassified individual after the first step of the algorithm inform us that this individual must be infected. Instead we count and compare the number of such tests. The first step of the noisy DD algorithm is identical to noisy COMP, but we are not required to identify all healthy individuals in the first step (we are able to keep some unclassified for the second round). Thus, after the first step, we are left with all infected individuals $V_{1}$ (as the algorithm did not try to classify any individual as infected in the first step) and a set of yet unclassified healthy individuals (as some of them might exhibit a first neighbourhood that is not sufficient for a clear first round classification) which we will denote by $V_{0,\text{PD}}$ .These are healthy individuals who did not appear in sufficiently many displayed negative tests to be declared healthy with confidence in the first step⁵⁵5Note that the bounds are taken in a way such that no infected individual is classified as uninfected in the first round.. In symbols, for some $\alpha\in(0,1)$

\displaystyle V_{0,\text{PD}}=\left\{{x\in V_{0}:\bm{N}_{x}<\alpha\Delta}\right\}

To tell $V_{1}$ and $V_{0,\text{PD}}$ apart, we consider the number of displayed positive tests $\bm{P}_{x}$ where the individual $x$ appears on its own after removing the individuals , which were declared healthy already, $V_{0}\setminus V_{0,\text{PD}}$ from the first step, i.e.

(2.2)

\displaystyle\bm{P}_{x}=\left|{\left\{{a\in\partial x:\hat{\bm{\sigma}}(a)=1\text{ and }\partial a\setminus\left\{{x}\right\}\subset V_{0}\setminus V_{0,\text{PD}}}\right\}}\right|

Referring to Figure 2, the second step of the algorithm is based on counting tests of Type DP-S. Tests of Type DP-M contain another remaining unclassified individual after the first step of the algorithm from $V_{0,PD}\cup V_{1}$ . The noisy DD algorithm takes advantage of the fact that it is less likely for an individual $x\in V_{0,\text{PD}}$ to appear as the only yet unclassified individual in a displayed positive test than it is for an individual in $x\in V_{1}$ . For $x\in V_{0,\text{PD}}$ such a test would be truly negative and would have been flipped (which occurs with probability $p$ ) to display a positive test result. Conversely, an individual $x\in V_{1}$ renders any of its tests truly positive and thus the only requirement is that the test otherwise contains only individuals which were declared healthy already, and is not flipped (which occurs with probability $1-q$ ). For this reason, we will see that the distribution of $\bm{P}_{x}$ differs between $x\in V_{1}$ and $x\in V_{0,\text{PD}}$ , and the difference $(1-q)-p>0$ helps determine the size of this difference. The second step of DD exploits this observation by counting tests of Type DP-S.

2.3. The Channel Perspective of noisy group testing

Motivated by (1.1), we can describe the bounds in terms of rate, in a Shannon-theoretic sense. That is, we follow the common notion to define the rate (bits learned per test) of an algorithm in this setting (for instance as in [9]) to be

\displaystyle R:=\frac{\log\binom{n}{k}}{m\log 2}\sim\frac{k\log(n/k)}{m\log 2}.

(Recall that we take logarithms to base $e$ throughout this paper). For example the fact that Theorems 2.1 and 2.2 show that noisy COMP and DD respectively can succeed w.h.p. ; with $m\geq(1+\epsilon)ck\log(n/k)$ tests for some $c$ is equivalent to the fact that $R=1/(c\log 2)$ is an achievable rate in a Shannon-theoretic sense.

We now give a counterpart to these two theorems by stating a universal converse for the $p-q$ channel below, improving on the universal counting bound from (1.1). The starting observation (see [7, Theorem 3.1]) is that no group testing algorithm can succeed w.h.p. with rate greater than $C_{\text{Chan}}$ , the Shannon capacity of the corresponding noisy communication channel. Thus, we cannot hope to succeed w.h.p. with $m<(1-\epsilon)ck\log(n/k)$ tests where $c=1/(C_{\text{Chan}}\log 2)$ . Hence as a direct consequence of the value of the channel capacity of the $p-q$ channel, we deduce the following statement.

Corollary 2.3.

Let $p,q\geq 0$ , $p+q<1$ and $\epsilon>0$ , write $h(\cdot)$ for the binary entropy in nats (logarithms taken to base $e$ ) and $\phi=\phi(p,q)=(h(p)-h(q))/(1-p-q)$ . If we define

m_{\text{COUNT}}=\left(\frac{1}{D_{\mathrm{KL}}\left({{{q}\|{1/(1+e^{\phi})}}}\right)}\right)k\log(n/k),

then for $m\leq(1-\epsilon)m_{\text{COUNT}}$ no algorithm can recover $\bm{\sigma}$ w.h.p. for any matrix design.

Remark 2.4.

This result follows from Lemma F.1 derived in Appendix F below. As discussed there, this derivation (combined with the fact that each test is negative with probability $e^{-d}$ ) suggests a choice of density for the matrix:

d=d^{*}_{{\rm ch}}=\log(1-p-q)-\log\left(\frac{1}{1+e^{\phi}}-q\right).

While a choice of $\Delta=c\cdot d^{*}_{{\rm ch}}\cdot\log(n/k)$ is not necessarily optimal, it may be regarded as a sensible heuristic that provides good rates for a range of $p$ and $q$ values.

2.4. Applying the results to standard channels

With Theorem 2.1 and Theorem 2.2 we derived achievable rates for the generalized p-q-model (see Figure 1). Prior research considered the Z channel where $p=0$ and $q>0$ , the Reverse Z channel where $p>0$ and $q=0$ and the Binary Symmetric Channel with $p=q>0$ . These channels are common models in coding theory [41], but are also often considered in medical applications [30, 31] concerned with taking imperfect sensitivity ( $q>0$ ), specificity ( $p>0$ ) or both ( $p>0$ and $q>0$ ) into account. As a consequence we also compare our results with the most recent results of Johnson and Scarlett [48]. In the following section we will demonstrate how performance guarantees on these channels can directly be obtained from our main theorems.

2.4.1. Recovery of the noiseless model

Note that the bounds Corollary 2.5 and Corollary 2.6 are already known [10, 26]. We would like to give the reader an idea of how one can see that our cumbersome looking bounds relate to the more accessible bounds given for the noiseless case. First, we show the noiseless bounds can be simply recovered by letting $p,q\rightarrow 0$ . In the noiseless setting, it is sufficient, by definition of the algorithm, to set both $\alpha\Delta=1$ and $\beta\Delta=1$ . To see why, observe that in the absence of noise a single negative test is sufficient evidence that an individual is healthy. Conversely, a single positive test where the individual only appears with individuals , which were declared healthy already, implies that particular individual must surely be infected. As shown in [13] the optimal parameter choice for the density parameter $d$ in the constant-column design in the noiseless setting is $\log(2)$ . Applying these values to Theorem 2.1 we recover the noiseless bound for COMP.These bounds were first stated in [10].

Corollary 2.5 (COMP in the noiseless setting).

Let $p,q\rightarrow 0$ , $0<\theta<1$ and $\varepsilon>0$ . Further, let

\displaystyle m_{{\tt COMP},\text{noiseless}}=\frac{1}{(1-\theta)\log^{2}2}k\log(n/k).

Furthermore let $m_{\tt COMP}(n,\theta,p,q)$ be defined as in Theorem 2.1 Then we find

m_{\tt COMP}(n,\theta,p,q)\underset{p,q\rightarrow 0}{\longrightarrow}m_{{\tt COMP},\text{noiseless}}

Proof.

We start by taking the bounds $b_{1}(\alpha,d)$ and $b_{2}(\alpha,d)$ . To see how this boils down to $m_{{\tt COMP},\text{noiseless}}$ , we start with using the well-known fact that within the near constant column design $d=\log(2)$ is the optimal choice [13]. Now by taking both $p,q\rightarrow 0$ one realizes that $b_{1}(\alpha,\log(2))$ vanishes as $\log(p)\rightarrow-\infty$ as $p\rightarrow 0$ . Turning our focus to the second bound we see that it boils down to

b_{2}(\alpha,\log(2)))=\frac{1}{(1-\theta)\log(2)}\frac{1}{\log(2)+\alpha\log(\alpha)+(1-\alpha)\log(1-\alpha)}

On the one hand we realize that $\alpha\log(\alpha)+(1-\alpha)\log(1-\alpha)$ is negative for all $\alpha\in(0,1)$ . This leads to

b_{2}(\alpha,\log(2))>b_{2}(0,\log(2))

On the other hand we realize that in the noiseless case a single negative test is sufficient for a classification as uninfected. Therefore we may choose $\alpha>0$ sufficiently small. One indeed realizes that for each $\alpha$ we can choose $\varepsilon:=\varepsilon(\alpha)>0$ appropriately, such that the bounds given in Theorem 2.1 recover the noiseless case. ∎

We also recover the noiseless bounds for the DD algorithm as stated in [26].

Corollary 2.6 (DD in the noiseless setting).

Let $p,q\rightarrow 0,0<\theta<1$ and $\varepsilon>0$ . Further, let

\displaystyle m_{{\tt DD},\text{noiseless}}=\max\left\{{1,\frac{\theta}{1-\theta}}\right\}\frac{1}{\log^{2}2}k\log(n/k).

Furthermore let $m_{\tt DD}(n,\theta,p,q)$ be defined as in Theorem 2.2 Then we find

m_{\tt DD}(n,\theta,p,q)\underset{p,q\rightarrow 0}{\longrightarrow}m_{{\tt DD},\text{noiseless}}

Proof.

We start with taking $c_{1}(\alpha,d),c_{2}(\alpha,d),c_{3}(\beta,d)$ and $c_{4}(\alpha,\beta,d)$ as defined in Theorem 2.2. First of all we take $c_{4}(\alpha,\beta,d)$ . By assumption we find $\beta>0$ and therefore the indicator is 1 as soon as we let $p\rightarrow 0$ . Furthermore for $p\rightarrow 0$ we get $-\log(p)\rightarrow\infty$ and find $c_{4}\rightarrow 0$ . Second of all we take $c_{1}(\alpha,d)$ . With a similar argument as before we see that $c_{1}(\alpha,d)\rightarrow 0$ for $q\rightarrow 0$ as in this case we find $-\log(q)\rightarrow\infty$ . Therefore we are left with $c_{2}(\beta,d)$ and $c_{3}(\alpha,\beta,d)$ . Again, we use the well known fact that in the noiseless case $d=\log(2)$ is the optimal choice. Therefore with $p,q\rightarrow 0$ the two remaining bounds read as follows:

	$\displaystyle c_{2}(\alpha,\log(2))=\frac{1}{\log(2)\left({\log(2)+\alpha\log(\alpha)+(1-\alpha)\log(1-\alpha)}\right)}$
	$\displaystyle c_{3}(\alpha,\beta,\log(2))=\frac{\theta}{(1-\theta)}\frac{1}{\log(2)\left({\log(2)+\beta\log(\beta)+(1-\beta)\log(1-\beta)}\right)}$

Again we see that $x\log(x)+(1-x)\log(1-x)$ is negative for $x\in(0,1)$ . Therefore we find

	$\displaystyle c_{2}(\alpha,\log(2))>c_{2}(0,\log(2))$
	$\displaystyle c_{3}(\alpha,\log(2))>c_{3}(0,\log(2))$

Now as as before in this case again a single negative test as well as a single test with only already classified uninfected individuals is sufficient. Therefore we can choose $\alpha,\beta>0$ sufficiently small. One indeed realizes that for each $\alpha,\beta>0$ one can choose $\varepsilon:=\varepsilon(\alpha,\beta)$ appropriately such that the bounds of Theorem 2.2 recover the noiseless case. ∎

2.4.2. The Z channel

In the Z channel, we have $p=0$ and $q>0$ , i.e. no truly negative test displays a positive test result. Thus, in this case finding one positive test with only one unclassified individual is a clear indication, therefore we again can choose $\beta>0$ sufficiently small and remain agnostic about $\alpha$ and $d$ . The bounds for COMP and DD thus read as follows.

Corollary 2.7 (Noisy COMP for the Z channel).

Let $p{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\rightarrow}0,0<q<1,0<\theta<1$ and $\varepsilon>0$ . Further, let

	$\displaystyle m_{{\tt COMP},Z}$	$\displaystyle=\min_{\alpha,d}\max\left\{{b_{1}(\alpha,d),b_{2}(\alpha,d)}\right\}k\log(n/k)$
	$\displaystyle\text{ with }\quad b_{1}(\alpha,d)$	$\displaystyle=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{q}}}\right)}\quad\text{ and }\quad b_{2}(\alpha,d)=\frac{1}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{e^{-d}+\left({1-e^{-d}}\right)q}}}\right)}.$

If $m>(1+\varepsilon)m_{{\tt COMP},Z}$ , noisy COMP will recover $\bm{\sigma}$ w.h.p. given $\bm{G},\hat{\bm{\sigma}}$ .

Corollary 2.8 (Noisy DD for the Z channel).

Let $p{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\rightarrow}0,0<q<1,0<\theta<1$ and $\varepsilon>0$ . Further, let

	$\displaystyle m_{{\tt DD},Z}$	$\displaystyle=\min_{\alpha,d}\max\left\{{c_{1}(\alpha,d),c_{2}(\alpha,d),c_{3}(d)}\right\}k\log(n/k)$
	with	$\displaystyle c_{1}(\alpha,d)=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{q}}}\right)}\quad\text{ and }\quad c_{2}(\alpha,d)=\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{e^{-d}+\left({1-e^{-d}}\right)q}}}\right)}$
	and	$\displaystyle c_{3}(d)=\frac{\theta}{1-\theta}\frac{1}{-d\log\left({1-e^{-d}(1-q)}\right)}.$

If $m>(1+\varepsilon)m_{{\tt DD},Z}$ , noisy DD will recover $\bm{\sigma}$ w.h.p. given $\bm{G},\hat{\bm{\sigma}}$ .

Proof.

The bounds $c_{1}$ and $c_{2}$ follow directly from Theorem 2.2 by letting $p\rightarrow 0$ . An immediate consequence of $p\rightarrow 0$ is that due to the fact that $-\log(p)\rightarrow\infty$ and one finds that $c_{4}\rightarrow 0$ , thus being trivial in this case. For $c_{3}$ we use the fact that we can choose $\beta>0$ sufficiently small we find $D_{\mathrm{KL}}\left({{{\alpha}\|{e^{-d}(1-q)}}}\right)=-\log\left({1-e^{-d}(1-q)}\right)-\delta(\beta)$ for $\delta(\beta)>0$ . Note that by definition of the noise model, we may choose an arbitrary $\beta_{\min}$ very close to zero and as a consequence $\beta=\beta_{\min}$ leading to $\delta(\beta)\rightarrow\delta_{\min}$ . The assertion follows as for each $\beta$ we may choose $\varepsilon:=\varepsilon(\beta)>0$ such that $(1+\varepsilon)>\left({1+\varepsilon\left({\beta_{\min}}\right)}\right)$ . ∎

An illustration of the bounds from Corollary 2.7 and 2.8 for sample values of $q$ is shown in Figure 5.

2.4.3. Reverse Z channel

In the reverse Z channel, we have $q=0$ and $p>0$ , i.e. no truly positive test displays a negative test result. Thus, we may choose $\alpha>0$ sufficiently small and remain agnostic about $\beta$ and $d$ . The bounds for the noisy COMP and DD thus read as follows.

Corollary 2.9 (Noisy COMP for the Reverse Z channel).

Let $0<p<1,q\rightarrow 0,0<\theta<1$ and $\varepsilon>0$ . Further, let

\displaystyle m_{{\tt COMP},\text{rev Z}}

\displaystyle=\frac{1}{1-\theta}\min_{d}\left\{{\frac{1}{-d\log\left({1-e^{-d}(1-p)}\right)}}\right\}k\log(n/k).

If $m>(1+\varepsilon)m_{{\tt COMP},\text{rev Z}}$ , noisy COMP will recover $\bm{\sigma}$ w.h.p. given $\bm{G},\hat{\bm{\sigma}}$ .

Proof.

The corollary follows from Theorem 2.1 and the fact that for $q\rightarrow 0$ one finds that $D_{\mathrm{KL}}\left({{{\alpha}\|{0}}}\right)$ diverges, Thereby $b_{1}\rightarrow 0$ just gives a trivial bound in this case. Furthermore for sufficiently small $\alpha>0$ we get $D_{\mathrm{KL}}\left({{{\alpha}\|{e^{-d}(1-p)}}}\right)\rightarrow-\log\left({1-e^{-d}(1-p)}\right)-\delta(\alpha)$ . Due to the noise assumption, we may choose an arbitrary $\alpha_{\min}$ very close to zero and $\alpha=\alpha_{\min}$ which leads to $\delta(\alpha)\rightarrow\delta\left({\alpha_{\min}}\right)$ . The assertion follows by choosing $\varepsilon:=\varepsilon(\alpha)>0$ such that $(1+\varepsilon)>\left({1+\varepsilon\left({\alpha_{\min}}\right)}\right)$ . ∎

Note that Corollary 2.9 does not yield an immediate closed form expression for the optimal value of $d$ .

Corollary 2.10 (Noisy DD in the Reverse Z channel).

Let $0<p<1,q\rightarrow 0,0<\theta<1$ and $\varepsilon>0$ . Further, let

	$\displaystyle m_{{\tt DD},\text{rev Z}}$	$\displaystyle=\min_{\beta,d}\max\left\{{c_{2}(d),c_{3}(\beta,d),c_{4}(\beta,d)}\right\}k\log(n/k)$
	with	$\displaystyle c_{2}(d)=\frac{1}{-d\log\left({1-e^{-d}(1-p)}\right)}\quad\text{ and }\quad c_{3}(\beta,d)=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\beta}\\|{e^{-d}}}}\right)}$
	and	$\displaystyle c_{4}(\beta,d)=\frac{1}{1-\theta}\frac{1}{d\left({-\log\left({1-e^{-d}(1-p)}\right)+D_{\mathrm{KL}}\left({{{\beta}\\|{\frac{e^{-d}p}{e^{-d}p+\left({1-e^{-d}}\right)}}}}\right)}\right)}$

If $m>(1+\varepsilon)m_{{\tt DD},\text{rev Z}}$ , noisy DD will recover $\bm{\sigma}$ w.h.p. given $\bm{G},\hat{\bm{\sigma}}$ .

Proof.

First of all we assume $q\rightarrow 0$ . Therefore we find $c_{1}\rightarrow 0$ as $-\log(q)\rightarrow\infty$ . The bounds $c_{2},c_{3}$ follow from Theorem 2.2 and the same manipulations as above. For $c_{4}$ , we again see that by definition of the noise model we may choose $\alpha>0$ as close to zero as we like. Therefore we get $(1-\alpha)$ close to 1, which leads to $z\rightarrow 1$ . The assertion follows as for each $\alpha$ we can choose $\varepsilon:=\varepsilon(\alpha)>0$ such that $(1+\varepsilon)>\left({1+\varepsilon\left({\alpha_{\min}}\right)}\right)$ .∎

An illustration of the bounds of Corollary 2.9 and 2.10 for sample values of $p$ is shown in Figure 6.

2.4.4. Binary Symmetric Channel

In the Binary Symmetric Channel (BSC), we set $p=q>0$ . Even though information-theoretic arguments would suggest setting $d=\log 2$ , we formulate the expression below with general $d$ . We also keep the threshold parameters $\alpha$ and $\beta$ . The bounds for the noisy DD and COMP only simplify slightly.

Corollary 2.11 (Noisy COMP in the Binary Symmetric Channel).

Let $0<p=q<1/2,0<\theta<1$ and $\varepsilon>0$ . Further, let

		$\displaystyle m_{{\tt COMP},\text{BSC}}=\min_{\alpha,d}\max\left\{{b_{1}(\alpha,d),b_{2}(\alpha,d)}\right\}k\log(n/k)$
	with	$\displaystyle b_{1}(\alpha,d)=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{p}}}\right)}\quad\text{ and }\quad b_{2}(\alpha,d)=\frac{1}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{e^{-d}+p-2e^{-d}p}}}\right)}.$

If $m>(1+\varepsilon)m_{{\tt COMP},\text{BSC}}$ , noisy COMP will recover $\bm{\sigma}$ w.h.p. given $\bm{G},\hat{\bm{\sigma}}$ .

Corollary 2.12 (Noisy DD in the Binary Symmetric Channel).

Let $0<p=q<1/2,0<\theta<1$ and $\varepsilon>0$ and define $v=1-e^{-d}-p+2e^{-d}p$ . Further, let

		$\displaystyle m_{{\tt DD},\text{BSC}}=\min_{\alpha,\beta,d}\max\left\{{c_{1}(\alpha,d),c_{2}(\alpha,d),c_{3}(\beta,d),c_{4}(\alpha,\beta,d)}\right\}k\log(n/k)$
	with	$\displaystyle c_{1}(\alpha,d)=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{p}}}\right)}\quad\text{ and }\quad c_{2}(\alpha,d)=\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{e^{-d}+p-2e^{-d}p}}}\right)}$
	and	$\displaystyle c_{3}(\beta,d)=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\beta}\\|{(1-p)e^{-d}}}}\right)}$
	and	$\displaystyle c_{4}(\alpha,\beta,d)=\max_{1-\alpha\leq z\leq 1}\left\{{\frac{1}{1-\theta}\frac{1}{d\left({D_{\mathrm{KL}}\left({{{z}\\|{v}}}\right)+\bm{1}\left\{{\beta>\frac{ze^{-d}p}{v}}\right\}zD_{\mathrm{KL}}\left({{{\frac{\beta}{z}}\\|{\frac{e^{-d}p}{v}}}}\right)}\right)}}\right\}.$

If $m>(1+\varepsilon)m_{{\tt DD},\text{BSC}}$ , noisy DD will recover $\bm{\sigma}$ w.h.p. given $\bm{G},\hat{\bm{\sigma}}$ .

An illustration of the bounds of Corollary 2.11 and 2.12 is shown in Figure 7.

2.5. Comparison of noisy COMP and DD

An obvious next question is to find conditions under which the noisy DD algorithm requires fewer tests than the noisy COMP. For the noiseless setting, it can be easily shown that DD provably outperforms COMP for all $\theta\in(0,1)$ . For the noisy case, matters are slightly more complicated.

Recall that noisy COMP classifies all individuals appearing in less than $\alpha\Delta$ displayed negative tests as infected while noisy DD additionally requires such individuals to appear in more than $\beta\Delta$ displayed positive tests as the only yet unclassified individual. Thus, it might well be that an infected individual is classified correctly by noisy COMP, while it is missed by the noisy DD algorithm.

That being said, our simulations indicate that noisy DD generally requires fewer tests than noisy COMP, but for the reason mentioned above we can only prove that for the reverse Z channel while remaining agnostic about the Z channel and the Binary Symmetric Channel, as the next proposition evinces.

Proposition 2.13.

For all $p,q\geq 0$ with $p+q<1$ there exists a $d^{*}\in(0,\infty)$ such that $m_{\text{COMP}}\geq m_{\text{DD}}$ as long as $e^{-d^{*}}p\geq q$ .

In terms of the common noise channels Proposition 2.13 gives the following corollary.

Corollary 2.14.

In the reverse Z channel, $m_{\text{COMP}}\geq m_{\text{DD}}$ .

The proof can be found in Appendix D. Our simulations suggest that this superior performance of noisy DD holds as well for the Z channel and Binary Symmetric Channel. Please refer to Figure 3 for an illustration.

Refer to caption — Figure 3. Comparison of the bound for noisy DD and noisy COMP in the Z-channel and the Binary Symmetric Channel for different noise level. (Note for black and white prints: The lines in the diagram are in the same order as given in the legend from top to bottom)

2.6. Relation to Bernoulli testing

In [48] sufficient bounds for noisy group testing and a Bernoulli test design where each individual joins every test independently with some fixed probability were derived. Thus, the variable degrees fluctuate and we end up with some individuals assigned only to few tests. In contrast, we work under a model in this paper where each individual joins an equal number of tests $\Delta$ chosen uniformly at random without replacement. For the noiseless case, it is by now clear that the near-constant-column design better facilitates inference than the Bernoulli test design [13, 26]. We find that the same holds true for the noisy variant of the COMP algorithm. Let us denote by $m_{\text{COMP}}^{\text{Ber}}$ the number of tests required for the noisy COMP to succeed under a Bernoulli test design.

Proposition 2.15.

For all $p+q<1$ , we have

\displaystyle m_{\text{COMP}}^{\text{Ber}}\geq m_{\text{COMP}}

We see the same effect for the noisy variant of the DD algorithm for all simulations, but for technical reasons only prove it for the Z channel.

Proposition 2.16.

For the Z channel where $p=0$ and $0<q<1$ , we have

\displaystyle m_{\text{DD}}^{\text{Ber}}>m_{\text{DD}}

For an illustration on the magnitude of the difference, we refer to Figure 4 and Figure 8.

Appendix

The core of the technical sections is the proof of Theorems 2.1 and Theorem 2.2. Some groundwork with standard concentration bounds and group testing properties can be found in Section A. We continue with the proof of Theorems 2.1 and 2.2 in Sections B and C, respectively. The structure of the proofs follows a similar logic. First, we derive the distributions for the number of displayed positive and negative tests for infected and healthy individuals. Second, we threshold these distributions using sharp Chernoff concentration bounds to deduce the bounds stated in Theorem 2.1 and Theorem 2.2. Thereafter, we proceed to the proof of Proposition 2.13 in Section D, while the proofs of Propositions 2.15 and 2.16 follow in Section E. The proof of Corollary 2.3 can be found in Section F. Additional illustrations of our results for the different channels can be found in Section G.

Appendix A Groundwork

For starters, let us recall the Chernoff bound for binomial and hypergeometric distributions.

Lemma A.1 (Chernoff bound for the binomial distribution [25]).

Let $p<q<r\in(0,1)$ and $\bm{X}\sim{\rm Bin}(n,q)$ be a binomially distributed random variable. Then

	$\displaystyle\mathbb{P}\left({\bm{X}\leq\lceil pn\rceil}\right)$	$\displaystyle=\exp\left({-\left({1+n^{-\Omega(1)}}\right)nD_{\mathrm{KL}}\left({{{p}\\|{q}}}\right)}\right)$
	$\displaystyle\mathbb{P}\left({\bm{X}\geq\lceil rn\rceil}\right)$	$\displaystyle=\exp\left({-\left({1+n^{-\Omega(1)}}\right)nD_{\mathrm{KL}}\left({{{r}\\|{q}}}\right)}\right)$

Lemma A.2 (Chernoff bound for the hypergeometric distribution [23]).

Let $p<q<r\in(0,1)$ and $\bm{Y}\sim H(N,Q,n)$ be a hypergeometrically distributed random variable. Further, let $q=Q/N$ . Then

	$\displaystyle\mathbb{P}\left({\bm{Y}\leq\lceil pn\rceil}\right)$	$\displaystyle=\exp\left({-\left({1+n^{-\Omega(1)}}\right)nD_{\mathrm{KL}}\left({{{p}\\|{q}}}\right)}\right)$
	$\displaystyle\mathbb{P}\left({\bm{Y}\geq\lceil rn\rceil}\right)$	$\displaystyle=\exp\left({-\left({1+n^{-\Omega(1)}}\right)nD_{\mathrm{KL}}\left({{{r}\\|{q}}}\right)}\right)$

The next lemma provides that the test degrees, as defined in (1.4) above, are tightly concentrated. Recall from (1.3) that the number of tests $m=ck\log(n/k)$ and each item appears in $\Delta=cd\log(n/k)$ tests.

Lemma A.3.

With probability $1-o(n^{-2})$ we have

\displaystyle dn/k-\sqrt{dn/k}\log n\leq\bm{\Gamma}_{\min}\leq\bm{\Gamma}_{\max}\leq dn/k{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}+}\sqrt{dn/k}\log n

Proof.

The probability that an individual $x$ is assigned to test $a$ is given by

(A.1)

\displaystyle\mathbb{P}\left({x\in\partial a}\right)=1-\mathbb{P}\left({x\notin\partial a}\right)=1-\binom{m-1}{\Delta}\binom{m}{\Delta}^{-1}=\Delta/m=d/k

Since each individual is assigned to tests independently, the total number of individuals in a given test follows the binomial distribution ${\rm Bin}\left({n,d/k}\right)$ . The assertion now follows from applying the Chernoff bound for this binomial distribution at the expectation (Lemma A.1). ∎

Next, we show that the number of truly negative tests $\bm{m}_{0}$ (and thus the number of truly positive tests $\bm{m}_{1}$ ) is tightly concentrated.

Lemma A.4.

With probability $1-o(n^{-2})$ we have $\bm{m}_{0}=e^{-d}m+O(\sqrt{m}\log^{3}n)$ .

Proof.

Recall from (A.1) that

\displaystyle\mathbb{P}\left({x\in\partial a}\right)=d/k

Since infected individuals are assigned to tests mutually independently, we find for a test $a$ that

\displaystyle\mathbb{P}\left({V_{1}\cap\partial a=\emptyset}\right)=\mathbb{P}\left({{\rm Bin}\left({k,d/k}\right)=0}\right)=\left({1-d/k}\right)^{k}=\left({1+n^{-\Omega(1)}}\right)e^{-d}.

Consequently, $\mathbb{E}\left[{\bm{m}_{0}}\right]=\left({1+n^{-\Omega(1)}}\right)e^{-d}m$ . Finally, changing the set of tests for a specific infected individual shifts the total number of negative tests by at most $\Delta$ . Therefore, the McDiarmid inequality (Lemma 1.2 in [34]) yields

\displaystyle\mathbb{P}\left({\left|{\bm{m}_{0}-\mathbb{E}\left[{\bm{m}_{0}}\right]}\right|\geq t}\right)\leq 2\exp\left({-\frac{t^{2}}{4k\Delta^{2}}}\right).

The lemma follows from setting $t=O\left({\sqrt{m}\log^{3}n}\right)$ . ∎

With the concentration of $\bm{m}_{0}$ and $\bm{m}_{1}$ at hand, we readily obtain estimates for $\bm{m}_{0}^{f},\bm{m}_{0}^{u},\bm{m}_{1}^{f}$ and $\bm{m}_{1}^{u}$ . We remind ourselves that these are the number of flipped, unflipped negative tests and the number of flipped, unflipped positive tests as defined in Sec. 1.4.

Corollary A.5.

With probability $1-o(n^{-2})$ we have

(i)

$\bm{m}_{0}^{f}=e^{-d}pm+O\left({\sqrt{m}\log^{3}n}\right)$
(ii)

$\bm{m}_{0}^{u}=e^{-d}(1-p)m+O\left({\sqrt{m}\log^{3}n}\right)$
(iii)

$\bm{m}_{1}^{f}=(1-e^{-d})qm+O\left({\sqrt{m}\log^{3}n}\right)$
(iv)

$\bm{m}_{1}^{u}=(1-e^{-d})(1-q)m+O\left({\sqrt{m}\log^{3}n}\right)$

Proof.

Since each test is flipped with probability $p$ and $q$ independently, the claims follow from Lemma A.4 and the Chernoff bound for the binomial distribution (Lemma A.1). ∎

In the following, let $\mathcal{E}$ be the event that the bounds from Lemma A.4 and A.5 hold. Note that $\mathcal{E}$ holds with high probability.

Appendix B Proof of COMP bound, Theorem 2.1

Recall from (2.1) that we write $\bm{N}_{x}$ for the number of displayed negative tests that item $x$ appears in (as illustrated by the right branch of Fig. 2). The proof of Theorem 2.1 is based on two pillars. First, Lemmas B.1 and B.2 provide the distribution of $\bm{N}_{x}$ for healthy and infected individuals, respectively. We will see that these distributions differ according to the infection status of the individual. Second, we will derive a suitable threshold $\alpha\Delta$ via Lemma B.3 and B.4 to tell healthy and infected individuals apart w.h.p. We start by analysing individuals in the infected set $V_{1}$ . Throughout the section, we assume $\alpha\in(q,e^{-d}(1-p)+\left({1-e^{-d}}\right)q)$ .

Lemma B.1.

Given $x\in V_{1}$ , its number of displayed negative tests $\bm{N}_{x}$ is distributed as ${\rm Bin}(\Delta,q)$ .

Proof.

Any test containing an infected individual is truly positive because of the presence of the infected individual. Since an infected individual is assigned to $\Delta$ different tests and each such test is flipped with probability $q$ independently, the lemma follows immediately. ∎

Next, we consider the distribution for healthy individuals. Recall that $\mathcal{E}$ denotes the event that the bounds from Lemma A.4 and Corollary A.5 hold.

Lemma B.2.

Given $x\in V_{0}$ and conditioned on $\mathcal{E}$ , the total variation distance of the distribution of $\bm{N}_{x}$ and $\bm{T}_{h}$ that is distributed as $H\left({m,m\left({e^{-d}(1-p)+\left({1-e^{-d}}\right)q}\right),\Delta}\right)$ tends to zero with $n$ , that is

d_{TV}(\bm{N}_{x},\bm{T}_{h})=n^{-\Omega(1)}

Proof.

Since $x$ is healthy, the outcome of all the tests remains the same if it is removed from consideration (if we perform group testing with $n-1$ items and the corresponding reduced matrix).

Thus, given $\mathcal{E}$ , we find that with $x$ removed the $\bm{m}_{0}^{f},\bm{m}_{0}^{u},\bm{m}_{1}^{f},\bm{m}_{1}^{u}$ still satisfy the bounds from Corollary A.5. As a result the number of displayed negative tests (which consist of unflipped truly negative tests and flipped truly positive tests) is given by

(B.1)

\bm{m}_{0}^{u}+\bm{m}_{1}^{f}=\left(e^{-d}(1-p)+(1-e^{-d})q\right)m+O\left({\sqrt{m}\log^{3}n}\right)

Now, adding $x$ back into consideration: $x\in V_{0}$ chooses $\Delta$ tests without replacement independently of this. Hence, given that the random quantity $\bm{m}_{0}^{u}+\bm{m}_{1}^{f}=\ell$ , the $\bm{N}_{x}$ (the number of displayed negative tests that item $x$ appears in) is distributed as $H(m,\ell,\Delta)$ . Hence, a conditioning argument shows that the linear combination of distribution functions

\sum_{\ell}\mathbb{P}\left({\bm{m}_{0}^{u}+\bm{m}_{1}^{f}=\ell}\right)\mathbb{P}(H\left({m,\ell,\Delta}\right)\leq x)

tends to the distribution function of $H\left({m,m\left({e^{-d}(1-p)+\left({1-e^{-d}}\right)q}\right),\Delta}\right)$ in total variation distance, due to the concentration of $\bm{m}_{0}^{u}+\bm{m}_{1}^{f}$ as obtained in Corollary A.5. ∎

Moving to the second pillar of the proof, we need to demonstrate that no infected individual is assigned to more than $\alpha\Delta$ displayed negative tests as shown by the following lemma.

Lemma B.3.

If $c>(1+\eta)\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\|{q}}}\right)}$ for some small $\eta>0$ , $\bm{N}_{x}<\alpha\Delta$ for all $x\in V_{1}$ w.h.p.

Proof.

We have to ensure that $\mathbb{P}(\exists x\in V_{1}:\bm{N}_{x}\geq\alpha\Delta)=o(1)$ . By Lemma B.1 and the union bound, we thus need to have

\displaystyle o(1)=k\cdot\mathbb{P}\left({\bm{N}_{x}\geq\alpha\Delta:x\in V_{1}}\right)=k\cdot\mathbb{P}\left({{\rm Bin}(\Delta,q)\geq\alpha\Delta}\right)=k\cdot\exp\left(-\left(1+\Delta^{-\Omega(1)}\right)\Delta D_{\mathrm{KL}}\left({{{\alpha}\|{q}}}\right)\right),

by the Chernoff bound for the binomial distribution (Lemma A.1). Since $k\sim n^{\theta}$ and $\Delta=cd(1-\theta)\log n$ the following must hold

\displaystyle\theta-cd(1-\theta)D_{\mathrm{KL}}\left({{{\alpha}\|{q}}}\right)<0

The lemma follows from rearranging terms and the fact that if we choose the number of tests slightly above the required number of tests (larger by a factor of $1+\eta$ for $\eta>0$ ), the assertion holds w.h.p. as $n\rightarrow\infty$ . ∎

We proceed to show that no healthy individual is assigned to less than $\alpha\Delta$ displayed negative tests.

Lemma B.4.

If $c>(1+\eta)\frac{1}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\|{e^{-d}(1-p)+\left({1-e^{-d}}\right)q}}}\right)}$ for some small $\eta>0$ , $\bm{N}_{x}>\alpha\Delta$ for all $x\in V_{0}$ w.h.p.

Proof.

We need to ensure that $\mathbb{P}(\exists x\in V_{0}:\bm{N}_{x}<\alpha\Delta)=o(1)$ . Since $\mathcal{E}$ occurs w.h.p. by Lemma A.4 and Corollary A.5, we need to have by Lemma B.2 and the union bound that

(B.2)

\displaystyle(n-k)\cdot\mathbb{P}\left({\bm{N}_{x}\leq\alpha\Delta|x\in V_{0},\mathcal{E}}\right)\leq n\cdot\mathbb{P}\left({{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\bm{T}_{h}}\leq\alpha\Delta}\right)=o(1).

We remind ourselves that $\bm{T}_{h}\sim H\left({m,m\left({e^{-d}(1-p)+\left({1-e^{-d}}\right)q}\right),\Delta}\right)$ and together with the Chernoff bound for the hypergeometric distribution (Lemma A.2) this leads to the following condition⁶⁶6Note that the additive rule of the logarithm allows us to move the error term from inside the KL-divergence to outside

\displaystyle 1-cd(1-\theta)D_{\mathrm{KL}}\left({{{\alpha}\|{(1-p)e^{-d}+(1-e^{-d})q}}}\right)<0

in a similar way to the proof of Lemma B.3. The lemma follows from rearranging terms and the fact that if we choose the number of tests slightly above the required number of tests (larger by a factor of $1+\eta$ for $\eta>0$ ), the assertion holds w.h.p. as $n\rightarrow\infty$ . ∎

Proof of Theorem 2.1.

The theorem is now an immediate consequence of Lemma B.3 and B.4 which guarantee that w.h.p. classifying individuals according to the threshold $\alpha\Delta$ for negative displayed tests recovers $\bm{\sigma}$ , and the fact that the choice of $\alpha$ and $d$ is at our disposal. ∎

Appendix C Proof of DD bound, Theorem 2.2

The proof of Theorem 2.2 follows a similar two-step approach as the proof of Theorem 2.1 by first finding the distribution of $\bm{P}_{x}$ (the number of displayed positive tests where individual $x$ appears on its own after removing the individuals, which were declared healthy already, $V_{0}\setminus V_{0,\text{PD}}$ , illustrated by DP-S in Fig. 2). We then threshold the distributions for healthy and infected individuals. To get started, we revise the second bound from Theorem 2.1 to allow $kn^{-\Omega(1)}$ healthy individuals to not be classified yet after the first step of DD. Recall that, we assume $\alpha\in(q,e^{-d}(1-p)+\left({1-e^{-d}}\right)q)$ and $\beta\in(0,e^{-d}(1-q))$ .

Lemma C.1.

\displaystyle c>(1+\eta)\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\|{e^{-d}(1-p)+\left({1-e^{-d}}\right)q}}}\right)}

for some small $\eta>0$ , we have ${\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\left|{\bm{V_{0,\textbf{PD}}}}\right|}=kn^{-\Omega(1)}$ w.h.p.

Proof.

The lemma follows immediately by replacing the r.h.s. of (B.2) with $kn^{-\delta}$ for some small $\delta=\delta(\eta)$ , rearranging terms and applying Markov’s inequality. ∎

For the next lemmas, we need an auxiliary notation denoting the number of tests $\bm{m}_{0,\text{nd}}$ that only contain individuals from $V_{0}\setminus V_{0,\text{PD}}$ . In symbols,

\displaystyle\bm{m}_{0,\text{nd}}=\left|{\left\{{a\in F:\partial a\subset V_{0}\setminus V_{0,\text{PD}}}\right\}}\right|.

Lemma C.2.

\displaystyle c>(1+\eta)\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\|{e^{-d}(1-p)+\left({1-e^{-d}}\right)q}}}\right)}

for some small $\eta>0$ , we have $\bm{m}_{0,\text{nd}}=\left({1-n^{-\Omega(1)}}\right)e^{-d}m$ with probability $1-o(n^{-2})$ .

Proof.

As in the proof of Lemma B.2 above, we consider the graph in two rounds: in the first round we consider the tests containing infected individuals. Since each healthy individual $x\in V_{0}$ does not impact the number of positive and negative tests, we know by Lemma A.4 that with probability $1-o(n^{-2})$ we find that the number of truly negative tests $\bm{m}_{0}=e^{-d}m+O\left({\sqrt{m}\log^{4}n}\right)$ after the first round. Furthermore the presence of a healthy individual has no impact on the number of displayed negative tests, as unflipped negative tests remain unflipped and flipped positive tests remain flipped. In the second round, we consider the effect of adding healthy individuals into the tests. Knowing the number of negative tests w.h.p. we can think of the participation of individuals $x\in V_{0,\text{PD}}$ in these tests as a balls into bins experiment. Starting with the number of truly negative tests $\bm{m}_{0}$ (given by the first round) we conduct a worst case analysis to see how many of those tests may include one of the $x\in V_{0,\text{PD}}$ . Consider some particular truly negative test $a$ . We are interested in the probability that none of the elements of $V_{0,\text{PD}}$ is contained. The probability that a given individual $x\in V_{0,\text{PD}}$ (knowing that it participates in $N_{x}\leq\alpha\Delta$ displayed negative tests, which is of lower order than $m$ ) is assigned to this test is given by⁷⁷7We refer the reader to [20] for two results we use while obtaining (C.3) (apply Claim 7.3 to the binomial coefficients) as well as (C.4)(apply Claim 7.4 as error corrected version of Bernoulli’s inequality).Please note that these bounds in particular hold for $\Delta=\Theta(\log(n))$ and $k\sim n^{\theta}$ .

(C.1)	$\displaystyle\mathbb{P}\left({x\in\partial a{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\|x\in V_{0,\text{PD}}}}\right)$	$\displaystyle=1-\mathbb{P}\left({x\notin\partial a{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\|x\in V_{0,\text{PD}}}}\right)$
(C.2)		$\displaystyle=1-\sum_{i=0}^{\alpha\Delta}\mathbb{P}\left({\bm{N}_{x}=i\|x\in V_{0,PD}}\right)\binom{m-1}{\Delta-i}\binom{m}{\Delta-i}^{-1}$
(C.3)		$\displaystyle\leq 1-\big{(}1+n^{-\Omega(1)}\big{)}\sum_{i=0}^{\alpha\Delta}\mathbb{P}\left({\bm{N}_{x}=i\|x\in V_{0,PD}}\right)\left({1-\frac{1}{m}}\right)^{\Delta-i}$
(C.4)		$\displaystyle\leq 1-\big{(}1+n^{-\Omega(1)}\big{)}\sum_{i=0}^{\alpha\Delta}\mathbb{P}\left({\bm{N}_{x}=i\|x\in V_{0,PD}}\right)\left({1-\frac{1}{m}}\right)^{\Delta}=\big{(}1+n^{-\Omega(1)}\big{)}\left({\frac{\Delta}{m}+O(k^{-2})}\right)=\frac{d}{k}+O(k^{-2})$

We can now calculate the probability that no individual $x\in V_{0,\text{PD}}$ is assigned to $a$ , bearing in mind that the size of $V_{0,\text{PD}}$ is random, and that each such individual is assigned to tests mutually independently. Using (C.4), and decomposing the sum into two parts, this is given by (for a given $V$ )

	$\displaystyle\mathbb{P}\left({\big{\{}V_{0,\text{PD}}\cap\partial a\big{\}}=\emptyset}\right)$	$\displaystyle=\sum_{j=0}^{n}\mathbb{P}\left({\left\|{\bm{V_{0,\textbf{PD}}}}\right\|=j}\right)\mathbb{P}\left({\big{\{}V_{0,\text{PD}}\cap\partial a\big{\}}=\emptyset\Big{\|}\left\|{\bm{V_{0,\textbf{PD}}}}\right\|=j}\right)$
		$\displaystyle=\sum_{j=0}^{V}\mathbb{P}\left({\left\|{\bm{V_{0,\textbf{PD}}}}\right\|=j}\right)\left({1-\frac{d}{k}+O\left({k^{-2}}\right)}\right)^{j}+\sum_{j=V+1}^{n}\mathbb{P}\left({\left\|{\bm{V_{0,\textbf{PD}}}}\right\|=j}\right)\left({1-\frac{d}{k}+O\left({k^{-2}}\right)}\right)^{j}$
		$\displaystyle\geq\sum_{j=0}^{V}\mathbb{P}\left({\left\|{\bm{V_{0,\textbf{PD}}}}\right\|=j}\right)\left({1-\frac{d}{k}+O\left({k^{-2}}\right)}\right)^{V}=\mathbb{P}\left({\left\|{\bm{V_{0,\textbf{PD}}}}\right\|\leq V}\right)\left({1-\frac{d}{k}+O\left({k^{-2}}\right)}\right)^{V}$

By Lemma C.1, we can choose $V=kn^{-\Omega(1)}$ such that $\mathbb{P}\left({\left|{\bm{V_{0,\textbf{PD}}}}\right|\leq V}\right)$ is arbitrarily close to 1, and knowing that $\left({1-\frac{d}{k}+O\left({k^{-2}}\right)}\right)^{V}\simeq\exp(-dV/k)=\exp(-dn^{-\Omega(1)})$ we find

\mathbb{P}\left({\big{\{}V_{0,\text{PD}}\cap\partial a\big{\}}=\emptyset}\right)=1-n^{-\Omega(1)}.

By combining this with the findings of Lemma A.4 we find $\mathbb{E}\left[{\bm{m}_{0,\text{nd}}}\right]=\left({1-n^{-\Omega(1)}}\right)e^{-d}m$ . The lemma follows by a similar application of the McDiarmid inequality as used in the proof of Lemma A.4.

∎

Note that, changing the set of tests for a specific individual $x\in V_{1}\cup V_{0,\text{PD}}$ shifts $\bm{m}_{0,\text{nd}}$ by at most $\Delta$ . Thus, such an individual choosing from this set is not affecting the order of $\bm{m}_{0,\text{nd}}$ .
Let $\mathcal{F}$ be the event that $\bm{m}_{0,\text{nd}}=\left({1-n^{-\Omega(1)}}\right)e^{-d}m$ . By Lemma C.2, $\mathbb{P}\left({\mathcal{F}}\right)=1-o(n^{-2})$ if

\displaystyle c>(1+\eta)\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\|{e^{-d}(1-p)+\left({1-e^{-d}}\right)q}}}\right)}

for some small $\eta>0$ . With Lemma C.2 at hand, we are in a position to describe the distribution of $\bm{P}_{x}$ for healthy and infected individuals (recall the definition of $\bm{P}_{x}$ in (2.2)). Let us start with infected individuals.

Lemma C.3.

Given $x\in V_{1}$ and conditioned on $\mathcal{F}$ , the total variation distance between $\bm{P}_{x}$ and $\bm{Q}_{H}$ , a random variable with hypergeometric distribution $H\left({m,me^{-d}(1-q),\Delta}\right)$ , tends to zero with $n$ , that is

d_{TV}\left(\bm{P}_{x},\bm{Q}_{H}\right)=n^{-\Omega{(1)}}.

Proof.

We are interested in the neighborhood structure of one given infected individual $x\in V_{1}$ , and we check how the remaining individuals influence the test types. In particular we are interested in the number of tests $a\in F$ such that $\partial a\subset V_{0}\setminus V_{0,\text{PD}}$ are contained in the neighborhood of an infected individual $x$ . Knowing the total number of tests $m$ and fixed degree $\Delta$ , for a given value of the random quantity $\bm{m}_{0,\text{nd}}=\ell$ , we find that this quantity of interest follows a $H\left({m,\ell,\Delta}\right)$ -distribution. Given $\mathcal{F}$ , Lemma C.2 gives that $\bm{m}_{0,\text{nd}}$ is highly concentrated,

\bm{m}_{0,\text{nd}}=\left({1-n^{-\Omega(1)}}\right)e^{-d}m

with high probability. Hence a conditioning argument, similar to Lemma B.2, shows that the linear combination of distribution functions

\sum_{\ell}\mathbb{P}(\bm{m}_{0,\text{nd}}=\ell)\mathbb{P}(H\left({m,\ell,\Delta}\right)\leq x)

tends to the distribution function of $H\left({m,me^{-d},\Delta}\right)$ in total variation distance, due to the concentration result obtained in Lemma C.2. Since each test featuring $x$ will truly be positive (as we assume $x$ to be infected) and will be displayed positive with probability $1-q$ independently, the lemma follows immediately. ∎

To describe the distribution of $\bm{P}_{x}$ for healthy individuals, let us introduce the random variable $\bm{P}_{x}(P)$ , which is $\bm{P}_{x}$ conditioned on the individual appearing in $P$ displayed positive tests, as follows:

\displaystyle\mathbb{P}\left({\bm{P}_{x}(P)=t}\right)=\mathbb{P}\left({\bm{P}_{x}=t|\bm{N}_{x}=\Delta-P}\right)

Then, we find for healthy individuals the following conditional distribution.

Lemma C.4.

Given $x\in V_{0}$ ,conditioned on $\mathcal{E}$ and $\mathcal{F}$ , the total variation distance between $\bm{P}_{x}(P)$ and
$\bm{B}_{h}\sim H\left({m\left({e^{-d}p+(1-e^{-d})(1-q)}\right),m\left({e^{-d}p}\right),P}\right)$ tends to zero with $n$ . That is

d_{TV}(\bm{P}_{x}(P),\bm{B}_{h})=n^{-\Omega(1)}.

Proof.

We proceed with the same exposition and reasoning as in the proof of Lemma C.3. Due to the fact that $x$ is healthy we can remove it without affecting the test result. Therefore we can analyse its neighborhood structure induced by the pooling graph while excluding it. Since by assumption individual $x\in V_{0}$ is assigned to exactly $P$ displayed positive and the total number of displayed positive test is given by $\bm{m}_{0}^{f}+\bm{m}_{1}^{u}$ , we see that $\bm{P}_{x}(P)$ is $H\left({\bm{m}_{0}^{f}+\bm{m}_{1}^{u},\bm{m}_{0,\text{nd}},P}\right)$ -distributed. Due to the fact that the event $\mathcal{E}$ pinpoints the amount of displayed positive and negative tests we can derive the distribution of neighbors the individual may choose from. Recalling the results of Corollary A.5, we see that w.h.p.

	$\displaystyle\bm{m}_{0}^{f}$	$\displaystyle=e^{-d}pm+O\left({\sqrt{m}\log^{3}n}\right),$
	$\displaystyle\text{and }\bm{m}_{1}^{u}$	$\displaystyle=(1-e^{-d})(1-q)m+O\left({\sqrt{m}\log^{3}n}\right).$

Furthermore we get from Lemma C.2 that w.h.p.

\bm{m}_{0,\text{nd}}=\left({1-n^{-\Omega(1)}}\right)e^{-d}m.

Now we apply the concentration results obtained in Corollary A.5 and Lemma C.2 to obtain a linear combination of distribution functions

\sum_{\ell,v}\mathbb{P}(\bm{m}_{0,\text{nd}}=\ell,\bm{m}_{0}^{f}+\bm{m}_{1}^{u}=v)\cdot\mathbb{P}(H\left({v,\ell,\Delta}\right)\leq x)

that tends to $H\left({m\left({e^{-d}p+(1-e^{-d})(1-q)}\right),me^{-d},P}\right)$ . The lemma follows since truly negative tests get flipped independently with probability $p$ . ∎

Having derived the distributions for $\bm{P}_{x}$ for $x\in V_{1}$ and $\bm{P}_{x}(P)$ for $x\in V_{0}$ we can now determine a threshold $\beta\Delta$ of displayed positive tests where the individual appears only with individuals from the set $V_{0}\setminus V_{0,\text{PD}}$ such that we can tell $V_{1}$ and $V_{0,\text{PD}}$ apart and thus recover $\bm{\sigma}$ . Let us start with infected individuals.

Lemma C.5.

As long as

\displaystyle c>(1+\eta)\max\left\{{\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\|{e^{-d}(1-p)+\left({1-e^{-d}}\right)q}}}\right)},\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\beta}\|{(1-q)e^{-d}}}}\right)}}\right\}

for some small $\eta>0$ , we have $\bm{P}_{x}>\beta\Delta$ for all $x\in V_{1}$ w.h.p.

Proof.

We need to ensure that $\mathbb{P}(\exists x\in V_{1}:\bm{P}_{x}<\beta\Delta)=o(1)$ . For the bound on $c$ from the lemma, we know that $\mathcal{F}$ occurs w.h.p. by Lemma C.2. In combination with Lemma C.3 and the union bound we need to ensure that

(C.5)

\displaystyle k\cdot\mathbb{P}\left({\bm{P}_{x}\leq\beta\Delta|x\in V_{1},\mathcal{F}}\right)=k\cdot\mathbb{P}({\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\bm{Q}_{H}}\leq\beta\Delta)+kn^{-\Omega(1)}=o(1),

where as before $\bm{Q}_{H}$ is a random variable with hypergeometric distribution $H\left({m,me^{-d}(1-q),\Delta}\right)$ . Using the Chernoff bound for the hypergeometric distribution (Lemma A.2), the following condition for (C.5) to hold arises

(C.6)

\displaystyle\theta-cd(1-\theta)D_{\mathrm{KL}}\left({{{\beta}\|{(1-q)e^{-d}}}}\right)<0

The lemma follows from rearranging terms in (C.6) and the fact that if we choose the number of tests slightly above the required number of tests (larger by a factor of $1+\eta$ for $\eta>0$ ), the assertion holds w.h.p. as $n\rightarrow\infty$ . ∎

We proceed with the set of individuals $V_{0,\text{PD}}$ .

Lemma C.6.

As long as

	$\displaystyle c$	$\displaystyle>(1+\eta)\max\Bigg{\{}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{e^{-d}(1-p)+\left({1-e^{-d}}\right)q}}}\right)},$
		$\displaystyle\qquad\max_{1-\alpha\leq z\leq 1}\left\{{\frac{1}{1-\theta}\frac{1}{d\left({D_{\mathrm{KL}}\left({{{z}\\|{e^{-d}p+(1-e^{-d})(1-q)}}}\right)+zD_{\mathrm{KL}}\left({{{\frac{\beta}{z}}\\|{\frac{e^{-d}p}{e^{-d}p+(1-e^{-d})(1-q)}}}}\right)}\right)}}\right\}\Bigg{\}}$

for some small $\eta>0$ , we have $\bm{P}_{x}<\beta\Delta$ for all $x\in V_{0,\text{PD}}$ w.h.p.

Proof.

We need to ensure that $\mathbb{P}(\exists x\in V_{0,PD}:\bm{P}_{x}>\beta\Delta)=o(1)$ . For the bound on $c$ from the lemma, we know that $\mathcal{F}$ occurs w.h.p. by Lemma C.2. Moreover, $\mathcal{E}$ occurs w.h.p. by Lemma A.4 and Corollary A.5. We write $w=e^{-d}p+\left({1-e^{-d}(1-q)}\right)$ for brevity. Combining this fact with Lemma B.2 and C.4 we need to ensure

(C.7)			$\displaystyle(n-k)\sum_{P=(1-\alpha)\Delta}^{\Delta}\mathbb{P}\left({\bm{N}_{x}=\Delta-P\|x\in V_{0},\mathcal{E}}\right)\mathbb{P}\left({\bm{P}_{x}(P)\geq\beta\Delta\|x\in V_{0},\mathcal{F}}\right)$
(C.8)			$\displaystyle=\left({1-n^{-\Omega(1)}}\right)n\sum_{P=(1-\alpha)\Delta}^{\Delta}\mathbb{P}\left({{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\bm{T_{h}}}=P}\right)\cdot\mathbb{P}\left({{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\bm{B_{h}}}\geq\beta\Delta}\right)=o(1)$

We remind ourselves that

	$\displaystyle\bm{T_{h}}$	$\displaystyle\sim H\left({m,m\left({e^{-d}(1-p)+\left({1-e^{-d}}\right)q}\right),\Delta}\right)$
	$\displaystyle\text{and }\quad\bm{B_{h}}$	$\displaystyle\sim H\left({m\left({e^{-d}p+(1-e^{-d})(1-q)}\right),m\left({e^{-d}p}\right),P}\right).$

Now by the Chernoff bound for the hypergeometric distribution (Lemma A.2) and setting $z=P/\Delta$ , we establish the following two bounds for the probability terms:

(C.9)

\displaystyle\mathbb{P}\left({H\left({m,m\left({w+n^{-\Omega(1)}}\right),\Delta}\right)=P}\right)=\exp\left({-(1+n^{-\Omega(1)})\Delta\left({D_{\mathrm{KL}}\left({{{z}\|{w}}}\right)}\right)}\right)

	$\displaystyle\mathbb{P}\left({H\left({m\left({w+n^{-\Omega(1)}}\right),m\left({e^{-d}p+n^{-\Omega(1)}}\right),P}\right)\geq\beta\Delta}\right)$
(C.10)		$\displaystyle=\exp\left({-\left({1+n^{-\Omega}}\right)z\Delta\bm{1}\left\{{\beta>\frac{ze^{-d}p}{w}}\right\}zD_{\mathrm{KL}}\left({{{\frac{\beta}{z}}\\|{\frac{e^{-d}p}{w}}}}\right)}\right)$

(Note that the indicator in (C.10) appears due to the condition given by Lemma A.2) We reformulate the left-hand-side of (C.8) to

	$\displaystyle n\sum_{P=(1-\alpha)\Delta}^{\Delta}\exp\left({-(1+o(1))\Delta\left({D_{\mathrm{KL}}\left({{{z}\\|{w}}}\right)+\bm{1}\left\{{\beta>\frac{ze^{-d}p}{w}}\right\}zD_{\mathrm{KL}}\left({{{\frac{\beta}{z}}\\|{\frac{e^{-d}p}{w}}}}\right)}\right)}\right)$
	$\displaystyle\qquad\qquad=\left({1+n^{-\Omega(1)}}\right)n\max_{1-\alpha\leq z\leq 1}\Bigg{\{}\exp\Bigg{(}-(1+o(1))\Delta\Bigg{(}D_{\mathrm{KL}}\left({{{z}\\|{w}}}\right)+\bm{1}\left\{{\beta>\frac{ze^{-d}p}{w}}\right\}zD_{\mathrm{KL}}\left({{{\frac{\beta}{z}}\\|{\frac{e^{-d}p}{w}}}}\right)\Bigg{)}\Bigg{)}\Bigg{\}}$

where the second equality follows since the sum consists of $\Theta(\Delta)=\Theta(\log n)$ many summands. Since $\mathbb{P}\left({\mathcal{F}}\right)=1-n^{-\Omega(1)}$ for our choice of $c$ by Lemma C.2 rearranging terms readily yields that the expression in (C.7) is indeed of order $o(1)$ .

To see this, we remind ourselves that by definition $\Delta=cd\log\left({\frac{n}{k}}\right)=(1-\theta)cd\log(n)$ . Furthermore we plug in the definition for $w=e^{-d}p+\left({1-e^{-d}(1-q)}\right)$ . In the end we have to ensure that

1<(1-\theta)cd\Bigg{(}D_{\mathrm{KL}}\left({{{z}\|{w}}}\right)+\bm{1}\left\{{\beta>\frac{ze^{-d}p}{e^{-d}p+\left({1-e^{-d}(1-q)}\right)}}\right\}zD_{\mathrm{KL}}\left({{{\frac{\beta}{z}}\|{\frac{e^{-d}p}{e^{-d}p+\left({1-e^{-d}(1-q)}\right)}}}}\right)\Bigg{)}

We solve this inequality for $c$ . As we are only interested in a worst case bound, the assertion follows from the non-negativity of $D_{\mathrm{KL}}\left({{{*}\|{*}}}\right)$ .

∎

Proof of Theorem 2.2.

The theorem is now immediate from Lemma B.3, C.1, C.5 and C.6 and the fact that the choice of $\alpha,\beta$ and $d$ is at our disposal. ∎

Appendix D Comparison of the noisy DD and COMP bounds

The following section is intented to provide sufficient conditions under which the DD algorithm attains reliable performance requiring fewer tests than the COMP. However, these conditions are not necessary and DD might (and for all performed simulations does) require fewer tests than COMP for even wider settings.

Proof of Proposition 2.13.

In order to prove the proposition, we need to find conditions under which

\displaystyle\min_{\alpha,d}\max\left\{{b_{1}(\alpha,d),b_{2}(\alpha,d)}\right\}\geq\min_{\alpha,\beta,d}\max\left\{{c_{1}(\alpha,d),c_{2}(\alpha,d),c_{3}(\beta,d),c_{4}(\alpha,\beta,d)}\right\}

We write $\alpha^{*}$ and $d^{*}$ for the values that minimise the maximum of the two terms at the LHS, at which point we know that $b_{1}(\alpha^{*},d^{*})=b_{2}(\alpha^{*},d^{*})$ . Then it is sufficient to show that there exists $\beta^{*}$ such that

b_{1}(\alpha^{*},d^{*})=b_{2}(\alpha^{*},d^{*})\geq\max\left\{{c_{1}(\alpha^{*},d^{*}),c_{2}(\alpha^{*},d^{*}),c_{3}(\beta^{*},d^{*}),c_{4}(\alpha^{*},\beta^{*},d^{*})}\right\}

By inspection for any $\alpha$ and $d$ $b_{1}(\alpha,d)=c_{1}(\alpha,d)$ and $b_{2}(\alpha,d)\geq c_{2}(\alpha,d)$ since $\theta\in(0,1)$ .

Next, we will show that $b_{2}(\alpha,d)\geq c_{4}(\alpha,\beta,d)$ for any $\alpha,\beta$ in the respective bounds and $d\in(0,\infty)$ . Writing $w=e^{-d}p+(1-e^{-d})(1-q)$ , and recalling that by assumption that $\alpha\leq 1-w$ (or $w\leq 1-\alpha$ ) we readily find that

(D.1)

D_{\mathrm{KL}}\left({{{\alpha}\|{1-w}}}\right)=\min_{1-\alpha\leq z\leq 1}\left({D_{\mathrm{KL}}\left({{{z}\|{w}}}\right)}\right)\leq\min_{1-\alpha\leq z\leq 1}\left({D_{\mathrm{KL}}\left({{{z}\|{w}}}\right)+z\bm{1}\left\{{\beta>\frac{ze^{-d}p}{w}}\right\}D_{\mathrm{KL}}\left({{{\frac{\beta}{z}}\|{\frac{e^{-d}p}{w}}}}\right)}\right)

where the first equality follows since $D_{\mathrm{KL}}\left({{{\alpha}\|{1-w}}}\right)=D_{\mathrm{KL}}\left({{{1-\alpha}\|{w}}}\right)$ and $D_{\mathrm{KL}}\left({{{z}\|{w}}}\right)>D_{\mathrm{KL}}\left({{{1-a}\|{w}}}\right)$ for any $z>1-\alpha$ . The bound follows. Note that (D.1) indeed holds for any choice of $\alpha,\beta$ and $d$ in the respective bounds stated in the theorem.

Finally, we need to demonstrate that $c_{3}(\beta^{*},d^{*})\leq b_{2}(\alpha^{*},d^{*})$ . Since $\beta$ is not an optimisation parameter in $b_{2}(\alpha^{*},d^{*})$ and the bound in (D.1) holds for any value of $\beta$ , we can simply set it to the value that minimizes $c_{3}(\beta^{*},d^{*})$ which is $\beta=1/\Delta$ and for which we find

\displaystyle c_{3}(\beta^{*},d^{*})=\frac{\theta}{1-\theta}\frac{1}{d^{*}\log\left({1-e^{-d^{*}}(1-q}\right)}.

Thus, to obtain the desired inequality we need to ensure that for the optimal choice $\alpha^{*}$ from COMP

\displaystyle\theta D_{\mathrm{KL}}\left({{{\alpha^{*}}\|{e^{-d^{*}}(1-p)+\left({1-e^{-d^{*}}}\right)q}}}\right)

\displaystyle\leq-\log\left({1-e^{-d^{*}}(1-q)}\right)

Using the bound

	$\displaystyle\theta D_{\mathrm{KL}}\left({{{\alpha}\\|{e^{-d}(1-p)+\left({1-e^{-d}}\right)q}}}\right)$	$\displaystyle\leq-\theta\log\left({1-\left(e^{-d}(1-p)+\left({1-e^{-d}}\right)q\right)}\right)$
		$\displaystyle\leq-\log\left({1-\left(e^{-d}(1-p)+\left({1-e^{-d}}\right)q\right)}\right)$

which is obtained by setting $\alpha=1/\Delta$ , we find that $c_{3}(\beta^{*},d^{*})\leq b_{2}(\alpha^{*},d^{*})$ if

\displaystyle-\log\left({1-e^{-d^{*}}(1-q)}\right)\geq-\log\left({1-e^{-d^{*}}(1-p)+\left({1-e^{-d^{*}}}\right)q}\right)\Leftrightarrow e^{-d^{*}}p\geq q

∎

As mentioned before, due to bounding $b_{2}(\alpha^{*},d^{*})$ the result is not sharp. However, one immediate consequence of Proposition 2.13 is that DD is guaranteed to require fewer tests than COMP for the reverse Z channel.

Appendix E Relation to Bernoulli testing

In the noiseless case [26] shows that the constant column weight design (where each individual joins exactly $\Delta$ different tests) requires fewer tests to recover $\bm{\sigma}$ than the i.i.d. (Bernoulli pooling) design (where each individual is included in each test with a certain probability independently). In this section we show that in the noisy case, the COMP algorithm requires fewer tests for the constant column weight design than for the i.i.d. design, and derive sufficient conditions under which the same is true for the noisy DD algorithm.

To get started, let us state the relevant bounds for the Bernoulli design, taken from [48, Theorem 5] and rephrased in our notation.

Proposition E.1 (Noisy COMP under Bernoulli).

Let $p,q\geq 0$ , $p+q<1$ , $d\in(0,\infty)$ , $\alpha\in(q,e^{-d}(1-p)+\left({1-e^{-d}}\right)q)$ . Suppose that $0<\theta<1$ and $\varepsilon>0$ and let

	$\displaystyle m_{\text{COMP}}^{\text{Ber}}$	$\displaystyle=m_{\text{COMP}}^{\text{Ber}}(n,\theta,p,q)=\min_{\alpha,d}\max\left\{{b_{1}(\alpha,d),b_{2}(\alpha,d)}\right\}k\log(n/k)$
		$\displaystyle\text{where}\qquad b_{1}(\alpha,d)=\frac{\theta}{1-\theta}\frac{1}{kD_{\mathrm{KL}}\left({{{\alpha d/k}\\|{qd/k}}}\right)}$
		$\displaystyle\text{and}\qquad b_{2}(\alpha,d)=\frac{1}{1-\theta}\frac{1}{kD_{\mathrm{KL}}\left({{{\alpha d/k}\\|{(e^{-d}(1-p)+(1-e^{-d})q)d/k}}}\right)}$

If $m>(1+\varepsilon)m_{\text{COMP}}^{\text{Ber}}$ , COMP will recover $\bm{\sigma}$ under the Bernoulli test design w.h.p. given $\bm{G},\hat{\bm{\sigma}}$ .

Proposition E.2 (Noisy DD under Bernoulli).

Let $p,q\geq 0$ , $p+q<1$ , $d\in(0,\infty)$ , $\alpha\in(q,e^{-d}(1-p)+\left({1-e^{-d}}\right)q)$ and $\beta\in(e^{-d}p,e^{-d}(1-q))$ . Suppose that $0<\theta<1,\zeta\in(0,\theta)$ and $\varepsilon>0$ and let

	$\displaystyle m_{\text{DD}}^{\text{Ber}}$	$\displaystyle=m_{\text{DD}}^{\text{Ber}}(n,\theta,p,q)=\min_{\alpha,\beta,d}\max\left\{{c_{1}(\alpha,d),c_{2}(\alpha,d),c_{3}(\beta,d),c_{4}(\beta,d)}\right\}k\log(n/k)$
		$\displaystyle\text{where}\qquad c_{1}(\alpha,d)=\frac{\theta}{1-\theta}\frac{1}{kD_{\mathrm{KL}}\left({{{\alpha d/k}\\|{qd/k}}}\right)}$
		$\displaystyle\text{and}\qquad c_{2}(\alpha,d)=\frac{1-\zeta}{1-\theta}\frac{1}{kD_{\mathrm{KL}}\left({{{\alpha d/k}\\|{(e^{-d}(1-p)+(1-e^{-d})q)d/k}}}\right)}$
		$\displaystyle\text{and}\qquad c_{3}(\beta,d)=\frac{\theta}{1-\theta}\frac{1}{k\cdot D_{\mathrm{KL}}\left({{{\beta d/k}\\|{e^{-d}(1-q)d/k}}}\right)}$
		$\displaystyle\text{and}\qquad c_{4}(\beta,d)=\frac{\zeta}{1-\theta}\frac{1}{k\cdot D_{\mathrm{KL}}\left({{{\beta d/k}\\|{e^{-d}pd/k}}}\right)}\frac{}{}$

If $m>(1+\varepsilon)m_{\text{DD}}^{\text{Ber}}$ , DD will recover $\bm{\sigma}$ under the Bernoulli test design w.h.p. given $\bm{G},\hat{\bm{\sigma}}$ .

To compare the bounds of the Bernoulli and constant-column test design we employ the following handy observation.

Lemma E.3.

Let $0<x,y<1$ and $d>0$ be constants independent of $k$ . As $k\to\infty$

\displaystyle kD_{\mathrm{KL}}\left({{{\frac{xd}{k}}\|{\frac{yd}{k}}}}\right)=d\left(D_{\mathrm{KL}}\left({{{x}\|{y}}}\right)+v(x,y)\right)+o(1/k)

with

(E.1)

\displaystyle v(x,y)=y-x+(1-x)\log\left(\frac{1-y}{1-x}\right)\leq 0

Proof.

Applying the definition of the Kullback-Leibler divergence and Taylor expanding the logarithm we obtain

	$\displaystyle k\cdot D_{\mathrm{KL}}\left({{{\frac{xd}{k}}\\|{\frac{yd}{k}}}}\right)=$	$\displaystyle xd\cdot\log\left(\frac{x}{y}\right)+(k-xd)\left(\log\left(1-\frac{xd}{k}\right)-\log\left(1-\frac{yd}{k}\right)\right)$
		$\displaystyle=xd\cdot\log\left(\frac{x}{y}\right)+(k-xd)\left(-\frac{xd}{k}+\frac{yd}{k}+o\left(\frac{1}{k^{2}}\right)\right)$
		$\displaystyle=d\left(x\cdot\log\left(\frac{x}{y}\right)-x+y\right)+o(1/k)$
		$\displaystyle=d\left(D_{\mathrm{KL}}\left({{{x}\\|{y}}}\right)+y-x-(1-x)\log\left(\frac{1-x}{1-y}\right)\right)+o(1/k).$

We can bound $v(x,y)$ from above by writing the final term as $(1-x)\log\left(1+\frac{x-y}{1-x}\right)\leq(1-x)\frac{x-y}{1-x}=x-y$ , using the standard linearisation of the logarithm. ∎

We are now in a position to prove Proposition 2.15 and 2.16.

Proof of Proposition 2.15.

The lemma follows by comparing the bounds from Theorem 2.1 and Proposition E.1 and applying Lemma E.3. ∎

Proof of Proposition 2.16.

As evident from Corollary 2.8, the fourth bound $c_{4}(\alpha,\beta,d)$ vanishes under the Z channel. Now comparing the bounds from Theorem 2.2 and Proposition E.2, observing that $(1-\zeta)/(1-\theta)>1$ for $\zeta<\theta$ and applying Lemma E.3 immediately implies the lemma. ∎

Appendix F Notes on Corollary 2.3

Lemma F.1.

If $p+q<1$ the Shannon capacity of the $p-q$ channel of Figure 1 measured in nats is

(F.1)

C_{\text{Chan}}=D_{\mathrm{KL}}\left({{{q}\|{\frac{1}{1+e^{\phi}}}}}\right)=D_{\mathrm{KL}}\left({{{p}\|{\frac{1}{1+e^{-\phi}}}}}\right),

where $\phi=(h(p)-h(q))/(1-p-q)$ . This is achieved by taking

(F.2)

\mathbb{P}(X=0)=\frac{1}{1-p-q}\left(\frac{1}{1+e^{\phi}}-q\right).

Please note that the proof might be a standard result for readers from some research communities, but for others it might be less standard. Therefore we state it here to prevent the interested (but unfamiliar) reader from a long textbook search.

Proof.

Write $\mathbb{P}(X=0)=\gamma$ and $\mathbb{P}(Y=0)=T(\gamma):=(1-p)\gamma+q(1-\gamma)$ . Then since the mutual information

(F.3)

I(X;Y)=h(Y)-h(Y|X)=h\left(T(\gamma)\right)-\left(\gamma h(p)+(1-\gamma)h(q)\right),

we can find the optimal $T$ by solving

0=\frac{\partial}{\partial\gamma}I(X;Y)=(1-p-q)\log\left(\frac{1-T(\gamma)}{T(\gamma)}\right)-\left(h(p)-h(q)\right),

which implies that the optimal $T^{*}=1/(1+e^{\phi})$ . We can solve for this for $\gamma^{*}=(T^{*}-q)/(1-p-q)$ to find the expression above. As $\frac{\partial}{\partial^{2}\gamma}I(X;Y)<0$ it is indeed a maximum. Substituting this in (F.3) we obtain that the capacity is given by

$\displaystyle h(T^{})-\left(\gamma^{}h(p)+(1-\gamma^{*})h(q)\right)$	$\displaystyle=$	$\displaystyle h\left(\frac{1}{1+e^{\phi}}\right)-\left((T^{*}-q)\phi+h(q)\right)$
	$\displaystyle=$	$\displaystyle\log(1+e^{\phi})-\phi(1-q)-h(q)$
	$\displaystyle=$	$\displaystyle D_{\mathrm{KL}}\left({{{q}\\|{1/(1+e^{\phi})}}}\right)$

as claimed in the first expression in (F.1) above. We can see that the second expression in (F.1) matches the first by writing the corresponding expression as $D_{\mathrm{KL}}\left({{{1-p}\|{1/(1+e^{\phi})}}}\right)=\log(1+e^{\phi})-\phi p-h(p)$ , which is equal to (F) by the definition of $\phi$ . ∎

Note that this result suggests a choice of density for the matrix: since each test is negative with probability $e^{-d}$ , equating this with (F.2) suggests that we take

d=d^{*}_{{\rm ch}}=\log(1-p-q)-\log\left(\frac{1}{1+e^{\phi}}-q\right).

This is unlikely to be optimal in a group testing sense, since we make different inferences from positive and negative tests, but gives a closed form expression that may perform well in practice. For the noiseless and BSC case observe that $\phi=0$ , and we obtain $d^{*}_{{\rm ch}}=\log 2$ .

Appendix G Illustration of bounds for Z, reverse Z channel and the BSC

Acknowledgment

The authors would like to thank two anonymous referees for their detailed reading of this paper and for the suggestions they made to improve its presentation. Oliver Gebhard and Philipp Loick are supported by DFG CO 646/3.

References

[1] E. Abbe, A. Bandeira, and G. Hall. Exact recovery in the stochastic block model. IEEE Transactions on Information Theory, 62:471–487, 2016.
[2] B. Abdalhamid, C. Bilder, E. McCutchen, S. Hinrichs, S. Koepsell, and P. Iwen. Assessment of specimen pooling to conserve SARS-CoV-2 testing resources. American Journal of Clinical Pathology, 153:715–718, 2020.
[3] M. Aldridge. The capacity of Bernoulli nonadaptive group testing. IEEE Transactions on Information Theory, 63:7142–7148, 2017.
[4] M. Aldridge. Individual testing is optimal for nonadaptive group testing in the linear regime. IEEE Transactions on Information Theory, 65:2058–2061, 2019.
[5] M. Aldridge, L. Baldassini, and O. Johnson. Group testing algorithms: bounds and simulations. IEEE Transactions on Information Theory, 60:3671–3687, 2014.
[6] M. Aldridge, O. Johnson, and J. Scarlett. Improved group testing rates with constant column weight designs. Proceedings of 2016 IEEE International Symposium on Information Theory (ISIT’16), pages 1381–1385, 2016.
[7] M. Aldridge, O. Johnson, and J. Scarlett. Group testing: an information theory perspective. Foundations and Trends in Communications and Information Theory, 15(3–4):196–392, 2019.
[8] E. ArÄ±kan. Channel polarization: A method for constructing capacity-achieving codes for symmetric binary-input memory-less channels. IEEE Transactions on Information Theory, 55:3051â€“–3073, 2009.
[9] L. Baldassini, O. Johnson, and M. Aldridge. The capacity of adaptive group testing. Proceedings of 2013 IEEE International Symposium on Information Theory (ISIT’13), 1:2676–2680, 2013.
[10] C. Chan, P. Che, S. Jaggi, and V. Saligrama. Non-adaptive probabilistic group testing with noisy measurements: near-optimal bounds with efficient algorithms. Proceedings of 49th Annual Allerton Conference on Communication, Control, and Computing, 1:1832–1839, 2011.
[11] H. Chen and F. Hwang. A survey on nonadaptive group testing algorithms through the angle of decoding. Journal of Combinatorial Optimization, 15:49–59, 2008.
[12] I. Cheong. The experience of South Korea with COVID-19. Mitigating the COVID Economic Crisis: Act Fast and Do Whatever It Takes (CEPR Press), pages 113–120, 2020.
[13] A. Coja-Oghlan, O. Gebhard, M. Hahn-Klimroth, and P. Loick. Information-theoretic and algorithmic thresholds for group testing. IEEE Transactions on Information Theory, DOI: 10.1109/TIT.2020.3023377, 2020.
[14] A. Coja-Oghlan, O. Gebhard, M. Hahn-Klimroth, and P. Loick. Optimal group testing. Proceedings of 33rd Conference on Learning Theory (COLT’20), 2020.
[15] D. Donoho. Compressed sensing. IEEE Transactions on Information Theory, 52:1289–1306, 2006.
[16] R. Dorfman. The detection of defective members of large populations. Annals of Mathematical Statistics, 14:436–440, 1943.
[17] S. Ciesek E. Seifried. Pool testing of SARS-CoV-02 samples increases worldwide test capacities many times over, 2020. https://www.bionity.com/en/news/1165636/pool-testing-of-sars-cov-02-samples-increases-worldwide-test-capacities-many-times-over.html, last accessed on 2020-11-16.
[18] Y. Erlich, A. Gilbert, H. Ngo, A. Rudra, N. Thierry-Mieg, M. Wootters, D. Zielinski, and O. Zuk. Biological screens from linear codes: theory and tools. bioRxiv, page 035352, 2015.
[19] European Centre for Disease Prevention and Control. Surveillance and studies in a pandemic in Europe, 2009. https://www.ecdc.europa.eu/en/publications-data/surveillance-and-studies-pandemic-europe (last accessed on 2020-11-16).
[20] Oliver Gebhard, Max Hahn-Klimroth, Olaf Parczyk, Manuel Penschuck, Maurice Rolvien, Jonathan Scarlett, and Nelvin Tan. Near optimal sparsity-constrained group testing: improved bounds and algorithms. Arxiv-Preprint, 2021.
[21] Y. Gefen, M. Szwarcwort-Cohen, and R. Kishony. Pooling method for accelerated testing of COVID-19, 2020. https://www.technion.ac.il/en/2020/03/pooling-method-for-accelerated-testing-of-covid-19/ (last accessed on 2020-11-16).
[22] E. Gould. Methods for long-term virus preservation. Mol Biotechnol, 13:57–66, 1999.
[23] W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58:301:13–30, 1963.
[24] F. Hwang. A method for detecting all defective members in a population by group testing. Journal of the American Statistical Association, 67:605–608, 1972.
[25] S. Janson, T. Luczak, and A. Rucinski. Random Graphs. John Wiley and Sons, 2011.
[26] O. Johnson, M. Aldridge, and J. Scarlett. Performance of group testing algorithms with near-constant tests per item. IEEE Transactions on Information Theory, 65:707–723, 2018.
[27] O. Johnson and D. Sejdinovic. Note on noisy group testing: Asymptotic bounds and belief propagation reconstruction. Proceedings of 48th Allerton Conference on Communication, Control, and Computing, 2010.
[28] E. Knill, A. Schliep, and D. Torney. Interpretation of pooling experiments using the Markov chain Monte Carlo method. Journal of Computational Biology, 3:395â€“406, 1996.
[29] H. Kwang-Ming and D. Ding-Zhu. Pooling designs and nonadaptive group testing: important tools for dna sequencing. World Scientific, 2006.
[30] A. Lalkhen. Clinical tests: sensitivity and specificity. Continuing Education in Anaesthesia Critical Care & Pain, 8, 2008.
[31] S. Long, C. Prober, and M. Fischer. Principles and practice of pediatric infectious diseases. Elsevier, 2018.
[32] N. Madhav, B. Oppenheim, M. Gallivan, P. Mulembakani, E. Rubin, and N. Wolfe. Pandemics: Risks, impacts and mitigation. The World Bank:Disease control priorities, 9:315–345, 2017.
[33] D. M. Malioutov and M. Malyutov. Boolean compressed sensing: Lp relaxation for group testing. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 2012.
[34] C. McDiarmid. On the method of bounded differences. Surveys in Combinatorics, 1989: Invited Papers at the 12th British Combinatorial Conference, page 148â€“188, 1989.
[35] R. Mourad, Z. Dawy, and F. Morcos. Designing pooling systems for noisy high-throughput protein-protein interaction experiments using Boolean compressed sensing. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 10:1478–1490, 2013.
[36] L. Mutesa, P. Ndishimye, Y. Butera, J. Souopgui, A. Uwineza, R. Rutayisire, E. Musoni, N. Rujeni, T. Nyatanyi, E. Ntagwabira, M. Semakula, C. Musanabaganwa, D. Nyamwasa, M. Ndashimye, E. Ujeneza, I. Mwikarago, C. Muvunyi, J. Mazarati, S. Nsanzimana, N. Turok, and W. Ndifon. A strategy for finding people infected with SARS-CoV-2: optimizing pooled testing at low prevalence. Nature, 589:276â€“280, 2021. doi:10.1038/s41586-020-2885-5.
[37] H. Ngo and D. Du. A survey on combinatorial group testing algorithms with applications to DNA library screening. Discrete Mathematical Problems with Medical Applications, 7:171–182, 2000.
[38] U.S. Department of Health and Human Services. Pandemic influenza plan, 2017. https://www.cdc.gov/flu/pandemic-resources/pdf/pandemic-influenza-implementation.pdf (last accessed on 2020-11-16).
[39] World Health Origanisation. Global surveillance during an influenza pandemic, 2009. www.who.int/csr/resources/publications/swineflu (last accessed on 2020-11-16).
[40] M. Plebani. Diagnostic errors and laboratory medicine â€“ causes and strategies. Electronic Journal of the International Federation of Clinical Chemistry and Laboratory Medicine, 26:7–14, 2015.
[41] T. Richardson and R. Urbanke. Modern coding theory. Cambridge University Press, 2007.
[42] C. Sammut and G. Webb. Encyclopedia of machine learning. Springer, 2011.
[43] J. Scarlett. Noisy adaptive group testing: Bounds and algorithms. IEEE Transactions on Information Theory, 65:3646–3661, 2018.
[44] J. Scarlett. An efficient algorithm for capacity-approaching noisy adaptive group testing. Proceedings of 2019 IEEE International Symposium on Information Theory (ISIT’19), pages 2679–2683, 2019.
[45] J. Scarlett and V. Cevher. Converse bounds for noisy group testing with arbitrary measurement matrices. Proceedings of 2016 IEEE International Symposium on Information Theory (ISIT’16), pages 2868–2872, 2016.
[46] J. Scarlett and V. Cevher. Phase transitions in group testing. Proceedings of the 27th Annual ACM-SIAM Symposium on Discrete Algorithms(SODA’16), 1:40–53, 2016.
[47] J. Scarlett and V. Cevher. Near-optimal noisy group testing via separate decoding of items. IEEE Journal of Selected Topics in Signal Processing, 2017.
[48] J. Scarlett and O. Johnson. Noisy non-adaptive group testing: A (near-)definite defectives, approach. IEEE Transactions on Information Theory, 66(6):3775–3797, 2020.
[49] N. Thierry-Mieg. A new pooling strategy for high-throughput screening: the shifted transversal design. BMC Bioinformatics, 7:28, 2006.
[50] L. Wang, X. Li, Y. Zhang, and K. Zhang. Evolution of scaling emergence in large-scale spatial epidemic spreading. Public Library of Science ONE, 6, 2011.
[51] L. Wein and S. Zenios. Pooled testing for HIV screening: Capturing the dilution effect. Operations Research, 44:543â€“569, 1996.
[52] S. Woloshin, N. Patel, and A. Kesselheim. False negative tests for SARS-CoV-2 infection â€” challenges and implications. New England Journal of Medicine, 2020.

	$\displaystyle\bm{m}_{t}=$	$\displaystyle\left\|{\left\{{a:\bm{\sigma}^{*}(a)=t}\right\}}\right\|$
	$\displaystyle\bm{m}_{t}^{f}=\left\|{\left\{{a:\bm{\sigma}^{*}(a)=t,\hat{\bm{\sigma}}(a)\neq t}\right\}}\right\|$	$\displaystyle\quad\text{and}\quad\bm{m}_{t}^{u}=\left\|{\left\{{a:\bm{\sigma}^{*}(a)=t,\hat{\bm{\sigma}}(a)=t}\right\}}\right\|$

	$\displaystyle m_{\text{DD}}$	$\displaystyle=m_{\text{DD}}(n,\theta,p,q)=\min_{\alpha,\beta,d}\max\left\{{c_{1}(\alpha,d),c_{2}(\alpha,d),c_{3}(\beta,d),c_{4}(\alpha,\beta,d)}\right\}k\log(n/k)$
		$\displaystyle\text{where}\qquad c_{1}(\alpha,d)=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{q}}}\right)}$
		$\displaystyle\text{and}\qquad c_{2}(\alpha,d)=\frac{1}{dD_{\mathrm{KL}}\left({{{\alpha}\\|{1-w}}}\right)}$
		$\displaystyle\text{and}\qquad c_{3}(\beta,d)=\frac{\theta}{1-\theta}\frac{1}{dD_{\mathrm{KL}}\left({{{\beta}\\|{(1-q)e^{-d}}}}\right)}$
		$\displaystyle\text{and}\qquad c_{4}(\alpha,\beta,d)=\max_{1-\alpha\leq z\leq 1}\left\{{\frac{1}{1-\theta}\frac{1}{d\left({D_{\mathrm{KL}}\left({{{z}\\|{w}}}\right)+\bm{1}\left\{{\beta>\frac{ze^{-d}p}{w}}\right\}zD_{\mathrm{KL}}\left({{{\frac{\beta}{z}}\\|{\frac{e^{-d}p}{w}}}}\right)}\right)}}\right\}$

(C.1)	$\displaystyle\mathbb{P}\left({x\in\partial a{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\|x\in V_{0,\text{PD}}}}\right)$	$\displaystyle=1-\mathbb{P}\left({x\notin\partial a{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\|x\in V_{0,\text{PD}}}}\right)$
(C.2)		$\displaystyle=1-\sum_{i=0}^{\alpha\Delta}\mathbb{P}\left({\bm{N}_{x}=i\|x\in V_{0,PD}}\right)\binom{m-1}{\Delta-i}\binom{m}{\Delta-i}^{-1}$
(C.3)		$\displaystyle\leq 1-\big{(}1+n^{-\Omega(1)}\big{)}\sum_{i=0}^{\alpha\Delta}\mathbb{P}\left({\bm{N}_{x}=i\|x\in V_{0,PD}}\right)\left({1-\frac{1}{m}}\right)^{\Delta-i}$
(C.4)		$\displaystyle\leq 1-\big{(}1+n^{-\Omega(1)}\big{)}\sum_{i=0}^{\alpha\Delta}\mathbb{P}\left({\bm{N}_{x}=i\|x\in V_{0,PD}}\right)\left({1-\frac{1}{m}}\right)^{\Delta}=\big{(}1+n^{-\Omega(1)}\big{)}\left({\frac{\Delta}{m}+O(k^{-2})}\right)=\frac{d}{k}+O(k^{-2})$

	$\displaystyle\mathbb{P}\left({\big{\{}V_{0,\text{PD}}\cap\partial a\big{\}}=\emptyset}\right)$	$\displaystyle=\sum_{j=0}^{n}\mathbb{P}\left({\left\|{\bm{V_{0,\textbf{PD}}}}\right\|=j}\right)\mathbb{P}\left({\big{\{}V_{0,\text{PD}}\cap\partial a\big{\}}=\emptyset\Big{\|}\left\|{\bm{V_{0,\textbf{PD}}}}\right\|=j}\right)$
		$\displaystyle=\sum_{j=0}^{V}\mathbb{P}\left({\left\|{\bm{V_{0,\textbf{PD}}}}\right\|=j}\right)\left({1-\frac{d}{k}+O\left({k^{-2}}\right)}\right)^{j}+\sum_{j=V+1}^{n}\mathbb{P}\left({\left\|{\bm{V_{0,\textbf{PD}}}}\right\|=j}\right)\left({1-\frac{d}{k}+O\left({k^{-2}}\right)}\right)^{j}$
		$\displaystyle\geq\sum_{j=0}^{V}\mathbb{P}\left({\left\|{\bm{V_{0,\textbf{PD}}}}\right\|=j}\right)\left({1-\frac{d}{k}+O\left({k^{-2}}\right)}\right)^{V}=\mathbb{P}\left({\left\|{\bm{V_{0,\textbf{PD}}}}\right\|\leq V}\right)\left({1-\frac{d}{k}+O\left({k^{-2}}\right)}\right)^{V}$

	$\displaystyle m_{\text{DD}}^{\text{Ber}}$	$\displaystyle=m_{\text{DD}}^{\text{Ber}}(n,\theta,p,q)=\min_{\alpha,\beta,d}\max\left\{{c_{1}(\alpha,d),c_{2}(\alpha,d),c_{3}(\beta,d),c_{4}(\beta,d)}\right\}k\log(n/k)$
		$\displaystyle\text{where}\qquad c_{1}(\alpha,d)=\frac{\theta}{1-\theta}\frac{1}{kD_{\mathrm{KL}}\left({{{\alpha d/k}\\|{qd/k}}}\right)}$
		$\displaystyle\text{and}\qquad c_{2}(\alpha,d)=\frac{1-\zeta}{1-\theta}\frac{1}{kD_{\mathrm{KL}}\left({{{\alpha d/k}\\|{(e^{-d}(1-p)+(1-e^{-d})q)d/k}}}\right)}$
		$\displaystyle\text{and}\qquad c_{3}(\beta,d)=\frac{\theta}{1-\theta}\frac{1}{k\cdot D_{\mathrm{KL}}\left({{{\beta d/k}\\|{e^{-d}(1-q)d/k}}}\right)}$
		$\displaystyle\text{and}\qquad c_{4}(\beta,d)=\frac{\zeta}{1-\theta}\frac{1}{k\cdot D_{\mathrm{KL}}\left({{{\beta d/k}\\|{e^{-d}pd/k}}}\right)}\frac{}{}$