The low level of correlation observed in the CMB sky at large angular scales and the low quadrupole variance

We view the continued success of a six-parameter model in describing the statistical properties of cosmic microwave background (CMB) temperature anisotropy maps with millions of resolution elements as one of the major takeaway messages from the Planck mission: a simple model works extremely well. This success includes agreement between model and data via multiple statistics, most prominently the two-point correlation function (Planck XV, 2014; Planck XI, 2016), but also via the higher-order correlations expected from gravitational lensing (Planck XVII, 2014; Planck XV, 2016) and relativistic effects of our motion with respect to cosmic rest (Planck XXVII, 2014).

The strongest challenge¹¹1While others have described challenges to the success of the six-parameter model arising from some internal tensions revealed by tests using the two-point correlation function, e.g. Addison et. al. (2016), they are not highly significant (Planck LI, 2016). to this takeaway message, based on CMB maps alone, may be the existence of relatively large-scale patterns that have been described as “anomalous.” Some of these large-scale CMB anomalies are: the existence of an unexpectedly large cold spot (Cruz, Mart nez–Gonz lez, & Vielva, 2006, 2007, 2010; Zhang & Huterer, 2010), the preference for odd parity modes (Kim & Naselsky, 2010a, b, 2011), a hemispherical power asymmetry (Eriksen et. al., 2004, 2007; Hoftuft et. al., 2009; Hansen et. al., 2009; Akrami et. al., 2014; Planck XXIII, 2014; Planck XVI, 2016), the alignment of the lowest multipoles with the geometry of the solar system and with each other (de Oliveira–Costa et. al., 2004; Hansen et. al., 2004; Schwarz et. al., 2004; Land & Magueijo, 2005a, b; Bennet et. al., 2011; Copi et. al., 2015b), the low variance in low-resolution maps (Planck XXIII, 2014; Planck XVI, 2016), and the lack of correlation on the largest angular scales (Bennett et. al., 2003; Spergel et. al., 2003; Hajian, 2007; Copi et. al., 2007, 2009; Bennet et. al., 2011; Gruppuso, 2014; Copi et. al., 2015a). For a recent review of large-scale CMB anomalies, see Schwarz et. al. (2015).

One of the apparent anomalies that has received a lot of attention is the low value of the real space two-point correlation function at large angular scales. This feature can be seen in the earliest observations of the correlation function of the CMB as observed by the Cosmic Background Explorer Differential Microwave Radiometer (COBE–DMR) (Hinshaw et. al., 1996), though it was not described in this way at the time. The lack of correlations at large angles received more attention with the Wilkinson Microwave Anisotropy Probe (WMAP) first year data (Spergel et. al., 2003), when the most widely used precise statistic created to capture this feature of the sky was defined: the $S_{1/2}$ statistic. Spergel et. al. (2003) initiated the use of $S_{1/2}$, after having looked at the COBE–DMR and WMAP data, as a convenient way to show how unusual the lack of correlation of the CMB at large angles is under the assumption of $\Lambda$CDM, making $S_{1/2}$ an a posteriori statistic. The $S_{1/2}$ statistic has since been used to confirm the presence of the anomaly in the WMAP 3 year (Copi et. al., 2007) and 5 year (Copi et. al., 2009) data, as well as in the first (Copi et. al., 2015a) and second (Planck XVI, 2016) Planck data releases.

We now define $S_{1/2}$, starting with a generalized version:

Use of this statistic enables one to quantify how unusual the observed feature is, given $\Lambda$CDM (the null hypothesis). We use simulations to determine a distribution of $S_{1/2}$ values given $\Lambda$CDM, and find the fraction of simulations with lower $S_{1/2}$ values than that inferred from the data. This fraction of simulations that exceeds the statistic in question is commonly called a $p$–value. In a recent re-analysis of the observed lack of correlations on the CMB at large angles, Copi et. al. (2015a) calculated several $S_{1/2}$ values of the CMB using various WMAP and Planck maps, masks, and methods, and using simulations based on the $\Lambda$CDM theory, found $p$–values for $S_{1/2}$ of the observed cut-sky CMB. These vary from $0.00191$ to $0.00329$. Alternatively, rather than use $p$–values, one could perform a Bayesian analysis on $S_{1/2}$ in order to quantify how unexpected the observed value is. Efstathiou, Ma, & Hanson (2010) did such an analysis, in which they calculated an expected posterior distribution of theoretical $S_{1/2}$ values, given the observed value. This resulted in a very wide distribution of theoretical $S_{1/2}$ values, and the $\Lambda$CDM $S_{1/2}$ value was not strongly disfavoured, indicating that $S_{1/2}$ is actually a poor discriminator of theoretical models. In this work, we do not use the Bayesian approach, but rather stick with the computation of $p$–values.

There have been numerous theoretical attempts to define cosmologies for which a low value of $S_{1/2}$ is less unlikely. Some of these attempts examine basic features of $\Lambda$CDM, such as the Gaussianity and statistical homogeneity/isotropy of the primordial fluctuations. For example, Copi et. al. (2009) showed that the low-$\ell$ modes of the CMB seem to work together to cause the lack of correlation on large angles, suggesting a correlation among the modes, indicating a violation of statistical isotropy. Non-trivial topology could be a possible way to explain the low correlation on large angles, because in topological models that employ the spherical multi-connected manifolds, the power at large scales is naturally suppressed (Stevens, Silk, & Scott, 1993; Niarchou & Jaffe, 2006). However, constraints on such manifolds arise from the null results of searches for matched circles on the CMB. In a more recent work, Aurich & Lustig (2014) explore CMB correlations in a cosmology incorporating the Hantzsche–Wendt manifold, which is interesting because this manifold could easily escape detection by matched circle pairs. They calculate the distribution of $S_{1/2}$ for an ensemble of simulations in this topology, finding it reduced by about a factor of 2 compared to $\Lambda$CDM.

Here we argue that this attention is misplaced. We find the evidence is quite weak that the low $S_{1/2}$ value is due to anything other than either the low quadrupole value, or a human’s ability to identify unusual-looking features in a high-dimensional data set with many different ways it can be unusual, or both.

The role of the low-order multipoles and the quadrupole in particular in the lack of correlation at large angles has previously been investigated. Bernui et. al. (2006) examined the role of the low quadrupole in the correlation function by removing the quadrupole from both the observed WMAP data and the $\Lambda$CDM model, and found a qualitatively better agreement between observation and theory than with the quadrupole included. Furthermore, both the quadrupole-removed WMAP and $\Lambda$CDM correlation functions showed a lack of correlations at large angles, whereas the non-quadrupole-removed $\Lambda$CDM correlation function did not, indicating that the low quadrupole is largely, though not entirely, responsible for the lack of correlation at large angles. Hajian (2007) pointed out that the contribution to $C(\theta)$ from multipoles $\ell=2,3$ in the observed CMB is nearly equal and opposite to the contribution from the rest of the multipoles put together. Additionally, Copi et. al. (2009) found that by tuning the powers $C_{2},C_{3}$ of the $\Lambda$CDM prediction, they could obtain an $S_{1/2}$ expectation value lower than the observed $S_{1/2}$ value found from WMAP. However, no work of which we are aware has yet reconciled the low $p$–value of the $S_{1/2}$ statistic at $\sim 0.007$, with the larger $p$–value of the low quadrupole at $\sim 0.04$.

Despite this previous attention to the relationship between the low quadrupole and low correlation at large angles, we find an important result has been overlooked: namely, the impact of the low quadrupole on the distribution of $S_{1/2}$ values expected under $\Lambda$CDM. We demonstrate that once the probability of $S_{1/2}$ is conditioned on the observed value of the quadrupole, the value of $S_{1/2}$ is no longer anomalous, as the $p$–value rises to 0.08. This result has implications for solutions to the puzzle of low $S_{1/2}$: any change to the model that suppresses the ensemble average of the quadrupole variance is helpful. For example, several groups have explored the possibility of an era in the early universe before slow-roll inflation in which a “fast-roll” caused a large scale suppression of power, effectively creating a cutoff in the primordial power spectrum which would be observable as a lack of correlations on large scales today (Contaldi et. al., 2003; Lello et. al., 2014; Liu et. al., 2014; Gruppuso & Sagnotti, 2014).

Although the low value of $S_{1/2}$ is potentially understandable as due to a departure from $\Lambda$CDM that reduces the ensemble average of the quadrupole, the question still remains of whether $S_{1/2}$ is compatible with $\Lambda$CDM in the absence of any such modifications.

The standard criticism of the amount of attention given to the anomalies is the standard criticism of a posteriori statistics: their a posteriori nature makes their interpretation very challenging. Consider that for a map of the sky, even with a finite number of pixels or data points, there is an infinite variety of statistics that one can make up.²²2As a simple example, consider linear combinations of a finite number of data points. Given the freedom to choose arbitrary coefficients, there are an infinite number of possible combinations of these data. If one makes up 1,000 of them prior to examining some data set, then it is not evidence of a failure of a model if one of those statistics has a value that happens to be that extreme in only 1 out of 1,000 simulations. With a posteriori statistics, by definition, one creates the statistic after seeing the data. When such a statistic ends up having an extreme value, determining whether or not this indicates a failure of a model is difficult because it is not clear how many such statistics could have been made up, had the data looked different to begin with. Additionally, because the choice of statistic to use was informed by the data, this also makes simulation of the data creation and analysis process very difficult, and perhaps practically impossible. What statistic would a different, simulated data set, have led the analyst to choose? Now that an analyst is in the simulation loop, simulation is quite challenging, to say the least.

There are several features about the $S_{1/2}$ statistic which can be considered to have been selected a posteriori. Broadly speaking, the whole notion that $C(\theta)$ is low at large angles is a posteriori. More specifically, the choices to square $C(\theta)$, to integrate this quantity over angles, and to select an upper integration limit of $1/2$ were all a posteriori choices, thereby making the significance of the value of $S_{1/2}$ difficult to interpret. In order to address the critique that the upper integration limit was selected a posteriori, the Planck team (Planck XVI, 2016) recently implemented a process in which this limit in the definition of $S_{1/2}$ was allowed to vary over the whole range of $x$ values $(-1\leq x\leq 1)$. We repeat a similar process here, although with a different rationale. We know historically that there actually was no optimization over $x$ as a continuous variable – this point is indisputable as one can do the optimization and the result is not $x=1/2$. However, the choice of a statistic tailored to capture the apparently unusual near-zero values of the correlation function on large angular scales was clearly an a posteriori selection. We view the $S_{x}$ statistic as a one-dimensional proxy for the very high-dimensional space of all possible statistics from which $S_{1/2}$ was selected. It is a useful proxy as the impact of a posteriori choice can then be calculated. When we consider the effects of the a posteriori choice of the $S_{1/2}$ statistic, by considering other possible statistics that one could calculate, the significance of the $S_{1/2}$ anomaly decreases. However, as we are using a one-dimensional proxy, we expect the impact of selection from the larger space is underestimated. ³³3For completeness, we mention that Efstathiou, Ma, & Hanson (2010), in addition to providing the Bayesian analysis we mentioned earlier, also suggested that a two-dimensional generalization of $S_{1/2}$ could illustrate the impact of a posteriori choice on $p$–values  but did not make any clear quantitative claims about this impact other than to suggest it could be “an order of magnitude or more.”

One research direction motivated by the $S_{1/2}$ anomaly that we do see as valuable is a search for testable predictions of the hypothesis that $\Lambda$CDM is correct and the low value of $S_{1/2}$ is just a statistical fluke. Yoho et. al. (2014) use $\Lambda$CDM realizations conditioned on the low value of $S_{1/2}$ to develop a priori statistics for use with future data sets, such as CMB polarization data, designed to test the hypothesis that the low $S_{1/2}$ measurement is just the consequence of an unlikely $\Lambda$CDM realization. The conclusion of our work here is that the evidence is quite weak that the low $S_{1/2}$ value indicates anything other than a somewhat unlikely $\Lambda$CDM realization. We hesitate to call it a fluke however, as it is only a highly unusual realization if one does not correct for a posteriori effects. Names aside, we expect that fluke-hypothesis tests have the potential to confirm the “fluke” interpretation we favor here, or provide stronger evidence against $\Lambda$CDM.

The rest of this paper is organized as follows. In section 2 we describe the observational and simulated data and data processing needed to create $S_{1/2}$ $p$–values. In section 3 we describe the effect of the low quadrupole on the observed $S_{1/2}$ value and how we include the low quadrupole in our filtered-$\Lambda$CDM null hypothesis, and the effect that that has on the $S_{1/2}$ $p$–value. In section 4 we describe a means to quantify the impact of the a posteriori choice $x=1/2$ by exploring a more generic statistic $S_{x}$. We finish by combining these two methods and examining the resultant quadrupole-filtered look–elsewhere–corrected $p$–values.

In order to calculate how unlikely the $S_{1/2}$ statistic for the observed CMB is, given a particular cosmological model, one has to calculate $S_{1/2}$ for the observed universe as well as for an ensemble of simulated universes, created based on that model. When compared to a suitably created ensemble of simulations, the $p$–value of the $S_{1/2}$ statistic can be computed. In this analysis, we have treated the observed CMB map as similarly as possible to those created in simulations, for consistency. This includes the treatment of the pixelization, spatial frequency content, and masking.

The summation formulae for $C(\theta)$ and $S_{1/2}$ (equations 3, 4) make it easy to examine the relative importance of each multipole to $C(\theta)$ and $S_{1/2}$. We found that removing the lowest multipoles from the calculation of $S_{1/2}$ for both simulations and for the SMICA data drastically increased the $p$–value of the SMICA $S_{1/2}$ statistic (See Fig. 2). In fact, for the $\ell_{\rm min}$ $=4$ case, the SMICA value is quite near the middle of the simulated distribution. This shows that the low $p$–value of the standard $S_{1/2}$ statistic is highly influenced by the $C_{2}$ and $C_{3}$ of the observed CMB.

Here we revisit the question of the connection between $\ell=2$ and $S_{1/2}$. In particular, we ask how dependent the $S_{1/2}$ anomaly is on the low observed value of $C_{2}$. In our $\Lambda$CDM model, the ensemble average quadrupole variance is $1099.94~{\rm\mu K}^{2}$, whereas for the degraded and masked SMICA map, we measure a quadrupole variance of only $171.8~{\rm\mu K}^{2}$, which is lower than the ensemble average value by a factor of 0.156. However, the low observed quadrupole variance is by itself not exceedingly anomalous. To see this, we created an ensemble of $10^{5}$ simulated CMB skies, masked them, and measured their $C_{2}$ and $S_{1/2}$ values (see Fig. 3). In this ensemble, the cut-sky SMICA $C_{2}$ has a $p$–value of $3.9$ per cent, which is much higher than the $S_{1/2}$ $p$–value of $0.72$ per cent. This is consistent with a similar analysis by Efstathiou (2003), who calculated $C_{2}$ $p$–values using two differently measured WMAP $C_{2}$ values of $129$ and $212~{\rm\mu K^{2}}$,⁹⁹9The variances presented by Efstathiou (2003) were given in terms of $\Delta T^{2}_{\ell}=\ell(\ell+1)C_{\ell}/(2\pi)$. which resulted in $p$–values of $1.3$ and $3.6$ per cent, respectively. This suggests, as Copi et. al. (2009) point out, that the low quadrupole variance anomaly is not the same as the $S_{1/2}$ anomaly.

This joint $S_{1/2}$$-C_{2}$ distribution allows us to look at the ensemble relationships between the two values. As can be seen in the scatter plot and density contours of Fig. 3, the distribution has a golf-club like appearance, such that the upper right of the plot shows a thin and narrow structure, while the lower left is wedge shaped. The thin distribution in the upper right indicates that at high values, $S_{1/2}$ and $C_{2}$ are correlated with each other. However, this correlation breaks down at lower values, in the wedge-shaped part of the distribution. At these low values, a low value of $S_{1/2}$ indicates that $C_{2}$ must also be low, but the inverse, that a low value of $C_{2}$ indicates a low value of $S_{1/2}$, is not true. However, as we show below, conditioning an ensemble to only include low values of $C_{2}$ does have a significant impact on the distribution of $S_{1/2}$ values, and increases the SMICA $p$–value substantially.

We implemented a constraint on the quadrupole variance of our cut-sky simulations by generate–and–test filtering. We first created a simulation, masked it, then measured its quadrupole variance and compared it to the $\Lambda$CDM expectation value. If $C_{2}$ of the simulation fell between 0.1 and 0.2 times the $\Lambda$CDM expectation value (where 0.1 and 0.2 were chosen to simply bracket the observed ratio of 0.156 with round numbers), it was included in the ensemble of simulations. If not, we threw it out. We kept creating simulations until we had reached the desired number in the ensemble: $10^{4}$.

The results of this ensemble selection process are shown in Fig. 4 and Fig. 5. Comparing these to the previous versions without filtering (Fig. 1 and Fig. 2), we see that the ensemble of correlation functions has changed shapes considerably, converging toward the line indicating zero, and the $S_{1/2}$ distribution has correspondingly lowered toward zero. The SMICA $S_{1/2}$ value (unchanged) therefore appears to be in a much less unlikely place in this distribution, with a $p$–value of $8.24$ per cent. This non-anomalous $p$–value suggests that, at the very least, $S_{1/2}$ is not independent from the low quadrupole. Once we condition on the observed quadrupole value, the observed $S_{1/2}$ value is not that rare.

Finally, we look back at the joint $S_{1/2}$$-C_{2}$ distribution, and generalize the notion of the $p$–value to this 2 dimensional space. To do so, we created iso-probability density contours by approximating the probability density using kernel density estimation. Some of these contours are shown in Fig. 3, labelled by what fraction of the total number of points they contain. The SMICA value lies in a region which has a probability density that is not extremely low, lying just within the $93$ per cent contour, indicating a 2D $p$–value of $7$ per cent, less than a $2\sigma$ deviation from the densest region. To compare this to the 1D cases we need to consider that the definition of $p$–value that we have been using is only a 1-tailed statistic, and does not account for outliers that are extreme in the opposite direction, at equal or lower probability density. (Our density contours in 2D account for high and low values in all directions.) Therefore, to approximate an equivalent 2-tailed statistic, we multiply the 1-tailed 1D $p$–values by 2, giving us 2-tailed 1D $p$–values of $1.44$ per cent for $S_{1/2}$, and $7.8$ per cent for $C_{2}$, which is clearly an underestimate given the asymmetry of the 1D $C_{2}$ distribution. Thus we see that the observed values in the joint space occupy a probability density that is similar, in terms of how extreme it is, to the one-dimensional case of $C_{2}$. We conclude that there is no new evidence in this joint space for anomalous behavior.

The previous section provides a better description of the influence of the low quadrupole on the lack of correlations at large angles then was available before, suggesting that modifications to $\Lambda$CDM which lower the expected quadrupole contribution to $C(\theta)$ would help reduce the anomalous nature of $S_{1/2}$. However, we are still interested in whether $S_{1/2}$ is consistent with $\Lambda$CDM without any such modification. To that end, we now turn to addressing what is perhaps the most often criticized aspect of $S_{1/2}$: that it was created to capture a feature of the data by analysts who had already taken a look at the data. That is, $S_{1/2}$ was created a posteriori.¹⁰¹⁰10For a reductio ad absurdum description of a posteriori statistic selection in the context of CMB and $\pi$ anomalies, see Frolop & Scott (2016).

If, prior to looking at the data, one came up with 1,000 statistics to calculate from the data, and then applied them to the data, assuming they were all uncorrelated, one would expect a uniform distribution of $p$–values between 0 and 1. The expectation value for the number of statistics with $p$–values between 0 and 0.001 would be 1. Even if they were correlated, one could still perform simulations that would naturally take these correlations into account, and use them to calculate the probability that one of the statistics returned a $p$–value of less than 0.001, by counting the fraction of simulations for which this is the case; that is, we calculate the $p$–value of the $p$–value. Such a probability we call the look–elsewhere–corrected (LEC) $p$–value.

To perform the analogous calculation for a single a posteriori statistic, one chosen after inspection of the data, the simulation process would include the process of inspecting the data and inventing a statistic to capture what is perceived to be an unusual feature. This is obviously challenging, if not impossible, as it requires an automated model of the analyst him or herself, so that the simulated data are processed by the simulated analyst who identifies the unusual feature and creates a statistic designed to capture that feature in a single number.

The difficult aspect of the above procedure is the automation of the human process of selection of a statistic from a very high-dimensional array of possibilities. In the following we adopt a crude, but calculable, approximation of this process as the selection of one statistic from a merely one-dimensonal space of alternatives. We create this one dimensional space by extending one single aspect of the already well defined $S_{1/2}$ statistic: the choice of the upper endpoint of the $S_{1/2}$ integral, $1/2$. As described above, we have generalized $S_{1/2}$ to $S_{x}$, which has an arbitrary upper integration limit, chosen from the whole range of possible values: $-1\leq x\leq 1$.

Note that we are not claiming that this is the procedure that was followed historically. We know it is not because, as we will see, the choice of $x$ that returns the most extreme vaue of $S_{x}$ from the real data is $x=0.37$ rather than $x=1/2$. The resulting LEC $p$–value we obtain should be viewed as merely indicative of the type of correction that is plausible. We suspect that, if anything, our replacement of the high-dimensional space of alternative statistics with a one-dimensional one underestimates the size of the correction.

We proceed by using the generalized $S_{x}$ integral (equation 1) and calculating $S_{x}$ as a function of $x$ over the whole range of $x$ values for an ensemble of $10^{5}$ CMB simulations (without $C_{2}$ filtering this time). Then, to save computational time, we choose a subsample of these, the first $10^{4}$ curves, to evaluate further. For each $S(x)$ function in the subsample we find, for each value of $x$, a rank in the entire set of $10^{5}\ S_{x}$ curves. We divide these ranks by $10^{5}$ and they become (1-tailed) $p$–values, as a function of $x$: $p(x)$. Then, for each simulation in the subsample, we find its minimum $p$–value and corresponding $x$ and $S_{x}$ values. In order to treat the SMICA data in the same manner, we throw it in as just another member of this ensemble and find its optimal $x$, $S_{x}$, and $p(x)$ values with resect to the ensemble as well. Finally, using the ensemble of $p$–values, we can calculate the LEC $p$–value.

The result of this process is shown¹¹¹¹11 plotting package corner.py available at https://github.com/dfm/corner.py in Fig. 6, where the SMICA value is indicated by the blue horizontal and vertical lines. In this ensemble, the SMICA map, optimized for minimum $p$–value, has $x=0.37$, $S_{x}=1326~{\rm\mu K}^{2}$, and a nominal $p(x)=0.36$ per cent. This is lower than the $S_{1/2}$ $p$–value of $0.72$ per cent that we calculated earlier, which is as expected due to the optimization process. The LEC $p$–value is $2.94$ per cent.

To calculate the error on the LEC $p$–value, we estimated the sample variance of these quantities by doing some calculations with smaller $S_{x}$ ensembles. These smaller ensembles were constructed as subsets of the same set of $10^{5}\ S_{x}$ simulations that we used before, but with $10^{4}$ members chosen randomly, and with $p(x)$ curves calculated using only the corresponding $10^{4}\ S_{x}$ curves, rather than the entire set of $10^{5}$ of them, as we did previously.¹²¹²12This is a variation on the jackknife resampling test. Interestingly, these fell into two groups, with 27 of them having $x$ values near 0.35, and 5 of them being near -0.1. For those near $x=0.35$, the average $p$–value was $0.330$ per cent, and for those near $x=-0.1$, the average $p$–value was $0.346$ per cent. There does not appear to be a big difference in $p$–values between the two groups, but since our first sample had an $x$ value of 0.37, we compare it to the first group. Supposing that the sample variance is dominated by the number of $S_{x}$ curves used in the calculation, we expect the sample variance of these $p$–values to be approximately equal to the Poisson noise associated with the number of simulations used. Since the average $p$–value of $0.33$ per cent corresponds to 33 out of 10000 samples, we hypothesized that these 27 optimal $p$–values were drawn from a Poisson distribution with mean 33, and performed a Kolmogorov–Smirnov goodness-of-fit test. This resulted in a Kolmogorov–Smirnov $p$–value of 0.24, suggesting that these values are consistent with being drawn from a Poissonian distribution, as hypothesized. Therefore, we also suppose that the $S_{1/2}$ $p$–value that we obtained using $10^{5}\ S_{x}$ curves, also subject to a sample variance, can also be thought of as being drawn from a Poissonian distribution. Using the only $p$–value that we have ($0.361\%=361/10^{5}$) to derive an estimate of the mean, we find the square root of the Poissonian variance to be $\sqrt{361}=19$, constraining our measured $p$–value to be $0.361\%\pm 0.019$ per cent.

We need to now also account for the fact that we checked the entire range of possible $x$ values in order to find this optimal $p$–value (account for the look-elsewhere effect). We do this by examining the fraction of simulated data sets with x-optimized $p$–values less than the SMICA value of $0.36$ per cent, the LEC $p$–value. For this ensemble, this value is $2.94$ per cent. Using the small sample variance for the SMICA $p$–value described above, this corresponds to $2.94\%^{+0.14\%}_{-0.07\%}$.

The Planck team (Planck XVI, 2016) recently implemented a very similar process to this that was meant to account for the look-elsewhere effect caused by the a posteriori choice of the integration endpoint, and their analysis produced an LEC $p$–value of $2.1$ per cent¹³¹³13The Planck Collaboration used the opposite sense of $p$–value, such that they calculated the probability of finding values higher than the observed value, rather than lower than the observed value. for the SMICA map. However, their analysis was based on only $10^{3}$ simulations, whereas we used $10^{5}$. However, their simulations were much more extensive than ours were. Their simulations were based on their “8th Full Focal Plane simulation set” (FFP8), which includes simulation of instrumental, scanning, and data analysis effects, whereas ours were simply based only on a theoretical power spectrum with some simple Gaussian beam and pixel window smoothing. Using the a similar procedure as above to estimate sampling uncertainties (with $2.1\%=21/1000$ and $\sqrt{21}\simeq 4.6$) we find that their result is consistent with ours to within $1.8\sigma$ using their sampling uncertainties. It is reassuring to note that these two methods gave reasonably consistent results.

Finally, we combine both methods described here: the ensemble filtering by $C_{2}$, as well as the $p(x)$ optimization. In order to reduce computational time, we simply used $10^{4}$ quadrupole-filtered $S_{x}$ curves for this.¹⁴¹⁴14The choice of using fewer ($10^{4}$) simulations is also justified by the larger resultant $p$–value: large numbers of simulations are necessary for high $p$–value resolution, which is needed when $p$–values are small. Large numbers of simulations are therefore not needed for larger $p$–values. The result is shown in Fig. 7. In this ensemble, optimized for $p$–value, the SMICA data produced $x=0.367$ and $S_{x}=1326.5$, similar to the values found without $C_{2}$ filtering, but a much higher nominal $p$–value of $4.50$ per cent, and an even higher LEC $p$–value of $22.3$ per cent.

Others have previously noted connections between the low quadrupole and the near-zero correlation function on large angular scales, as quantified in the $S_{1/2}$ statistic. Here we quantitatively investigated the relationship between these statistics as expected in the $\Lambda$CDM model. We found that if one conditions on the observed low quadrupole power then the distribution of $S_{1/2}$ values in the $\Lambda$CDM model is shifted to much lower values and as a result the observed $S_{1/2}$ has a $p$–value of 0.08 rather than its unconditioned value of 0.007. We pointed out that departures from $\Lambda$CDM that suppress the quadrupole will thus naturally make the observed $S_{1/2}$ less unusual and potentially take it from a $p$–value one might call “anomalous” to one that is merely “somewhat unusual”, if that.

Second, we revisited the exercise in Planck XVI (2016), where $S_{1/2}$ was generalized to $S_{x}$ in order to investigate the impact of the a posteriori choice of the integration cutoff at $x=\cos\theta=1/2.$ They found that the process of optimizing the value of $x$ with the observed data leads to a $p$-value that is smaller than what one finds by doing the same process with 21 out of 1000 simulations; i.e., they found a look–elsewhere–corected (LEC) $p$–value of 2.1%. Repeating this exercise with 100 times as many simulations (but without the simulation of as many aspects of the instrument), we found an LEC $p$–value of $3$ per cent. Our result is consistent with the $2.1$ per cent value at the $2\sigma$ level given their uncertainties that we estimated from their finite number of samples.

Third, we combined these two techniques of both conditioning on the observed low quadrupole power and optimizing over $x$ values. This resulted in a quadrupole-filtered LEC $p$–value of $22$ per cent. Our point here is not that a proper calculation of the $p$–value has to include conditioning on the low quadrupole, but rather that these are by no means independent phenomena: One cannot point to both the low quadrupole and low $S_{1/2}$ as independent indicators of anomalous behavior on large scales.

The main thing we offer regarding the LEC $p$–value for $S_{1/2}$ (quadrupole-filtered or not) is our interpretation of the result. We view the selection of $S_{1/2}$ from the one-dimensional space of alternative statistics given by $S_{x}$ as a proxy for the choice of $S_{1/2}$ out of a very high-dimensional space of all possible statistics derivable from $C(\theta)$ or $C_{\ell}$. The value of the proxy is that it allows for calculation of an LEC $p$–value. Since the dimensionality of the alternative space is much lower than that of the space of all possible statistics, it arguably leads to an underestimate of the look-elsewhere correction. In conclusion, we have quantitatively demonstarted that a posteriori selection provides a viable explanation of the low observed value of $S_{1/2}$. The correct explanation for low $S_{1/2}$ might lie with physics beyond $\Lambda$CDM but there is not yet compelling evidence in favor of such an alternative conclusion. Tests of the so-called “fluke hypothesis” (Yoho et. al., 2014; O’Dwyer et. al., 2016) have the potential to alter this situation.

We thank Brent Follin, Silvia Galli, Marius Millea, Marcio O’Dwyer, and Douglas Scott for useful conversations.

Some of the results in this paper have been derived using the healpix (G rski et. al., 2005) package.

The theoretical power spectrum was computed using the class (Blas, Lesgourges, & Tram, 2011) package.

The cut-sky power spectra were calculated using the polspice (aka spice) (Chon et. al., 2004) package.

The observed CMB data and mask were provided by the Planck Legacy Archive (http://pla.esac.esa.int/pla/#home)