DESI 2024 II: Sample Definitions, Characteristics, and Two-point Clustering Statistics

DESI observes four classes of extra-galactic targets: quasars (QSO; [43]), luminous red galaxies (LRG; [44]), emission line galaxies (ELG; [45]), and a bright galaxy sample (BGS; [46]). All four of these classes of DESI targets were selected based on photometry from Data Release 9 (DR9) of Legacy Survey (LS) [47, 48] imaging. The LS data combines photometric data from multiple sources. DESI targeting in the North Galactic Cap (NGC) at declination $>32.375^{\circ}$ uses $g$ and $r$ band photometry obtained by the Beijing-Arizona Sky Survey (BASS; [49]) and the $z$ band photometry obtained by the Mayall z-band Legacy Survey (MzLS). At low declination and in the South Galactic Cap (SGC), all of the $g$, $r$, $z$ bands were observed using the Dark Energy Camera (DECam; [50]), as part of the Dark Energy Camera Legacy Survey (DECaLS) and the Dark Energy Survey (DES; [51]). Further details on all of these imaging programs are available in [47]. These regions are denoted, respectively, as the ‘North’ and ‘South’ photometric regions. Infrared photometry in the $W1$ and $W2$ bands from the WISE satellite [52, 53] is used over the entire sky.

In this paper, we describe the samples used in the DESI DR1 cosmological analyses. The samples are first defined by their target bits, encoded in the DESI_TARGET column of the target catalogs, which map directly to the priority with which targets are assigned fibers. When science targets compete for fibers, the one with the highest priority receives the assignment. Any given target can pass the selection cuts of multiple target classes and in such cases, the target is always assigned the highest of the potential priority values.

We create DR1 LSS catalogs for three (nearly) distinct target samples observed exclusively in dark time (LRG, ELG, QSO) and one observed exclusively in bright time (BGS). Below, we describe the target properties of the dark time tracers in the order of greatest to least priority, then describe the bright time sample. We finally discuss the associated random samples created to enable clustering measurements. In all cases, we describe any cuts applied at the level of targeting (i.e., without any information from spectroscopic observations) that produce the samples considered for DR1 LSS catalogs.

We use the ‘cumulative’ tile-based redshift catalogs and associated data products from the iron version of the DESI spectroscopic reduction pipeline Redrock [54, 55], which are released with DR1.⁴⁴4These are pairs of FITS files containing information on the redshift fits for all of the spectra observed on DR1 tiles. Targets observed on multiple tiles get multiple entries and these are matched to the potential target catalogs via the target ID, tile ID, and fiber ID. The metadata associated with the particular coadded spectrum is matched via tile and fiber to the potential target and random catalogs.

An exception to using the iron reductions is for data taken on the night of December 12th, 2021. When investigating trends between the spectroscopic success and the observation date, [20] found this night to have unusually low spectroscopic success. It was found that during the iron processing, a bug caused incorrect calibration data to be used, only for this night’s results. This issue was identified after the iron data was frozen and all the data taken for the night were reprocessed separately. We substitute the reprocessed data for the original data on the affected 8 dark and 9 bright tiles before constructing the LSS catalogs. These data will be released as a supplementary value-added catalog to the DR1 release.

To define the samples ultimately used for clustering analysis, we also use various additional data from the iron spectroscopic reductions that are produced for every tile (and will be released with DR1) but are not included in the redshift catalogs.

For BGS, we use the information obtained from fastspecfit [56] for $k$-corrections, which are used to define the absolute magnitude threshold used for the DR1 cosmology sample described in Section 3. For ELG samples, we use the [OII] emission line flux measurements and uncertainties produced in the emlin files, which are used in defining the ELG spectroscopic success criteria, as detailed in Section 3. We concatenate the information over all tiles and join it to the ELG potential target catalog via a match to the target ID, tile, and fiber.

For the QSO samples, in addition to Redrock redshifts we use the results produced per tile by the machine learning-based classifier QuasarNET and the MgII ‘afterburner’ [43], which are used to define a ‘good’ QSO. The observations of QSO targets that pass the QSO selection are evaluated per tile and are then concatenated into a QSO catalog that is later joined to the QSO potential target catalog via a match to the target ID, tile, and fiber. A similar process concatenates the QSO information that is determined from spectra that are coadded across tiles (when observations on multiple tiles exist) and separated into Healpix [57] pixels. These separate ‘Healpix’ QSO catalogs are used for Lyman-$\alpha$ forest analyses [9], but are not used for the LSS analyses except for some comparisons.

Finally, for all samples, we use the information in the ‘zmtl’ files, which include flags that indicate whether the DESI instrument was performing properly in terms of positioning, CCD wavelength coverage, calibrations, etc. The region of data flagged can be as large as a petal or as small as an individual fiber. We concatenate this information across all tiles, match it to the redshift catalog via the tile, fiber, and target ID, and store it in the column ZWARN_MTL. This information is used for the hardware veto described in Section 4.1.

The combination of all the data described in the previous two subsections provides information associated with every instance in which a DESI target or random could be reached by a DESI fiber positioner, which is recorded in the ‘combined information’ catalogs described in Section 3 of [12]. We define the DESI footprint⁵⁵5The footprint of DESI DR1 LSS samples can be seen in Figure 2, which is discussed further in Section 5.3 in the context of completeness variations. as the area containing such reachable targets, and subsequently apply a series of veto masks to this, as described below. The area of this footprint, and coverage properties within it, can be matched to a resolution of less than one arcsecond⁶⁶6This accuracy is assessed by comparing the physical position determined by the DESI fiberassign software to the actual physical position on the focal plane that a fiber positioner was instructed to move to at the time of observation. These differ, e.g., due to dynamic changes in the DESI optics. by randoms through the use of the DESI fiberassign software, as described in [12].

To create what we denote as the ‘full’ LSS catalogs, the data and random potential assignment catalogs are reduced to a sample with unique entries for each target ID. This process includes careful sorting to ensure that only the most relevant instance for each target is retained—most importantly, keeping good observations over non-observations—and is vital for obtaining accurate completeness corrections. The process is described fully in Section 4 of [12]. The resulting catalogs are split by target class and include one entry per target. No cuts are applied to these full catalogs based on analysis of the observed spectra, but all of the relevant information is included in order to enable quality cuts as desired. For instance, one can apply simple criteria to the columns provided in the catalogs to obtain a sample with ‘good’ redshifts within some desired redshift range. The fact that we do not apply cuts based on the spectra also means that we can quickly determine completeness statistics, in terms of both assignment and spectroscopic success.

Several additional processing steps are applied to the LSS catalogs to produce the final ‘clustering’ catalogs. First, the DR1 full catalogs are output in three stages: before any veto masks (‘full_noveto’), after fiducial veto masks (‘full’), and after applying vetoes based on imaging properties recorded in Healpix maps (‘full_HPmapcut’). These veto masks are described in Section 4. The full catalogs are used to determine corrections for variations in completeness (Section 5.3), imaging data properties (Section 6), and spectroscopic data properties (Section 7). Cuts on the spectroscopic information are then applied to the full catalogs to produce clustering catalogs, as described in Section 3. We summarize and define new weights included in the clustering catalogs in Section 8. These clustering catalogs are then used as the inputs for all results presented in the sections after Section 8. Versions v1.2 (used in [8, 32]) and v1.5 (used in [10, 33]) will be released publicly with DR1. The differences between the versions are detailed in Appendix B. All of the results presented in this work are based on version v1.5 of the DESI DR1 LSS catalogs unless otherwise noted.

To protect against confirmation bias in our DESI DR1 cosmological inference, we applied a blinding scheme to obscure the true cosmology during early analyses until the full large-scale structure analysis pipeline was finalised. This blinding scheme was applied at the catalog level, to produce blinded clustering catalogs for further analysis. The blinding was meant to alter three distinct pieces of information that can be extracted from the DESI 2-point measurement: 1) the location of the BAO feature; 2) the anisotropy in the clustering imparted via redshift-space distortions (RSD) due to structure growth; and 3) the large-scale scale-dependent bias that is generated by local primordial non-Gaussianity, $f_{\rm NL}$. The specific blinding method applied, and its validation using DESI simulations, is presented in [14]. We summarize the procedure here.

The BAO and RSD signatures were blinded by shifting the measured DESI redshifts, following the methods proposed in [58]. The measured redshifts were first altered in a way that would mimic a change in the dark energy equation of state parameters $w_{0}$ and $w_{a}$. To do this, the measured true redshifts were converted to comoving distances using the DESI fiducial cosmology, and then coherently shifted based on the expected difference in redshifts between an object at that same comoving distance from an observer in a cosmological model with hidden values of $w_{0}$ and $w_{a}$, and in the fiducial cosmological model. A further shift was applied to blind the RSD structure growth measurements. To do this, RSD effects present in the measured redshifts were approximately subtracted based on an estimate of the local displacement field and the fiducial growth rate, and then new RSD shifts were applied to match the effect of a blinded growth rate value $f$ (full details of the method can be found in [14]). The shift in $f_{\rm NL}$ was implemented by altering the weight column of the LSS catalogs, using the methodology described in [15].

To choose a $(w_{0},w_{a})$ pair for blinding, we produced a list of 1000 randomly sampled pairs, with the range of possible values bounded to keep the expected shift to the isotropic BAO scale measurement relative to its value in the fiducial cosmology to within 3% over the redshift range $0.4<z<2.1$. The order of the pairs was randomized and they were written to a file on disk. The first time that the DESI DR1 LSS ‘clustering’ catalogs were generated, a random integer was chosen as the row to select the $(w_{0},w_{a})$ pair used for DR1 blinding. The value of the integer was stored in a separate file, which was then read every subsequent time the LSS catalog production was run (following iterative improvements to the pipeline while the analysis remained blinded), so that the same blinding was consistently applied.

Rather than being drawn randomly, the relative shift in $f$ was automatically calculated, using a linear RSD model as described in [14], in order to approximately compensate for the expected change due to the $w_{0},w_{a}$ blinding in the monopole of the redshift-space clustering, but with a maximum allowed shift of up to 10% relative to its fiducial value. The procedure left the expected amplitude of the clustering monopole approximately unchanged by the blinding, which meant that, e.g., the clustering amplitude of the (blinded) data monopole would still be expected to match that of DESI mocks. However, the procedure imparted a shift in the amplitude of the higher-order multipoles determined by the unknown values of the $(w_{0},w_{a})$ pair, effectively blinding the true structure growth information in the data. The relative shift in $f_{\rm NL}$ was randomly chosen to be between $(-15,15)$, and the value applied to DR1 blinding was held fixed for all blinded catalogs by generating the value via a random seed determined from the random integer described above.

The application of the blinding scheme took the ‘full’ catalogs described in the previous subsection as inputs. These catalogs contain all of the selection function details that are described in Sections 5.3, 6 and 7, but the blinding procedure itself changed this $n(z)$. The completeness-corrected $n(z)$ was determined from the number density in the full catalogs and the redshifts were shifted as described above. The $n(z)$ was then re-measured and a weighting was applied to the blinded clustering catalog to make the blinded $n(z)$ match the original $n(z)$. The steps of adding radial information to the randoms and adding an ‘FKP’ [59] weight to optimize the expected signal to noise given the number density variations then proceeded as described in Section 8.

Six versions of the blinded DR1 LSS catalogs were produced and their clustering measurements analysed while the LSS pipeline was iteratively improved before the first unblinded version of the catalog was created. Changes to the LSS catalogs that occurred after unblinding are described in Appendix B.

All veto masks associated with individual components of the DESI instrument are grouped together to define ‘bad hardware’ regions that are to be masked from the LSS catalogs. When producing the LSS catalogs with a unique entry per target (data and randoms), we prioritize the cases that were reachable by good hardware over the cases that were not, as detailed in [12]. Data and randoms are flagged as bad if they can only be reached by a fiber defined as having bad hardware in the initial compilation. This minimizes the area lost to bad hardware (as such areas are likely to be recovered as good when a tile overlaps the area on a subsequent pass) and also allows us to determine the area lost. One can see in Table 4 that the area lost to bad hardware is only 2.3% in bright time and 3.1% in dark time.

Three distinct sources of information define the DR1 hardware veto:

• •

  The ZWARN_MTL information compiled from the spectroscopic pipeline outputs (see Section 2.2) contains flags that indicate whether an observation passes the cuts to count as observed in the MTL. We apply the same definition as part of the hardware mask.⁷⁷7A difference between what we use and what was used for MTL decisions, however, is that we are using the information as determined during the iron spectroscopic reductions, and the determination for the MTL is based on the ‘daily’ version of the spectroscopic pipeline.

• •

  We require a minimum template signal-to-noise ratio, TSNR2. These values are determined for each tracer type and are proportional to the effective observing time. They are determined per coadded spectrum, but are independent of the target observed to produce the spectrum; they use a fixed template and the estimated noise. Each is defined in [40]. In dark time, we apply a threshold $\texttt{TSNR2\_ELG}>80$ for all samples, while in bright time, we apply a threshold $\texttt{TSNR2\_BGS}>1000$. The spectroscopic success rates for spectra with TSNR2 values below these thresholds decline dramatically and these cuts remove less than 1% of the observed data. All tile and fiber combinations below these thresholds are marked as bad hardware.

• •

  We identify 60 poor-performing fibers, as defined in [20]. All data from those fibers are flagged as bad hardware. Figure 3 shows the LRG and BGS failure rate for each fiber on petal 5, highlighting this petal as it has the largest concentration of bad fibers, largely lying between FIBER 2675 and 2691. We describe the process used to identify bad fibers in [20] and show failure rates for the other petals and tracers there.

To account for the areas on the sky where a given target type could not be observed, we apply a priority veto mask. Similar to the hardware veto, the priority veto is applied to the initial compilation of reachable data and random targets and is based on the metadata associated with the potential assignments; i.e., it is determined purely from the fiber assignment information. For both data and randoms, before cutting to unique objects, the priority of every target assigned on the given tile and fiber is known and stored in the catalogs as the column PRIORITY_ASSIGNED. If the PRIORITY_ASSIGNED is greater than that of the sample under consideration, the object’s particular occurrence (associated with the given tile and fiber) is vetoed. Note that the given object can still be included (i.e., not ultimately vetoed) in the final catalog if it was a potential assignment on a different tile or fiber. For our dark time DESI DR1 samples, only QSO and rare high-priority strong lens candidates with priority 4000 cause priority vetoes. We choose not to have LRG targets cause priority vetos on ELGs, as 10% of ELGs have the same priority as LRGs and there is significant overlap between the samples in redshift. For BGS, only white dwarf candidates have a higher priority (2998) than our BGS sample and cause a priority veto.

The priority mask for the LRG and ELG samples is caused by the same QSO and strong lens candidates and it is thus the same area of 1666.7 deg² for both, which is 20% of the DR1 footprint. In the completed DESI survey, the impact of the priority mask will be much smaller, as the high-priority targets will already have been observed when a given area of the sky is revisited. One can compare coverage areas in Table 6 and observe that the area covered by dark time tracers matches to within 7% in areas covered by more than one tile, e.g., the area of the LRG sample that is covered by two or more tiles is 3390.6 deg² and for QSO it is 3634.0 deg². For BGS, the white dwarfs remove 25.9 deg² and for QSO, the strong lens candidates remove 38.5 deg². Full details of the implementation of the priority mask in the LSS pipeline can be found in [12].

An implicit assumption in the application of the priority mask is that the higher priority sample is not correlated with the sample being masked. This is not strictly true for QSO and LRG/ELG, as the samples overlap in redshift, though the angular correlations are strongly diluted due to the breadth of the QSO redshift distribution. This makes simulations that include all three tracers and are processed to produce LSS catalogs in the same way as for the real DESI data important. These are described in Section 11.

We apply two types of veto masks that are related to imaging conditions. One set, which we denote ‘bright object’, masks areas around bright stars, galaxies, and defects in the imaging. The bright object masks are different for each tracer type. The other, which we denote as ‘property’, masks data in regions with bad imaging conditions and we use the mask for all tracers. We provide the details for each below.

For the DESI DR1 cosmological analyses, our fiducial approach to fiber assignment incompleteness is to divide it into two components, in a way that mimics the SDSS approach [63, 61]. The full details of the calculations are provided in [12] and we repeat the basic definitions and concepts here.

The first component is analogous to the SDSS ‘close-pair’ weights. Recall that the full catalogs are split by target type and cut from the potential assignments catalog to unique TARGETID, after a careful sorting (described in [12]) that puts each object at the most relevant combination of tile and fiber. Thus, every target (observed or not) in the full LSS catalog is associated with a single combination of tile and fiber. By definition, the unobserved targets are at combinations of tile and fiber assigned to a different observed target. For unobserved targets, reducing the potential assignments catalogs¹¹¹¹11Again, this is fully detailed in [12]. to unique TARGETID prioritizes the instances of tile and fiber assigned to the given type. The result is that most unobserved targets in the full LSS catalogs are at combinations of tile and fiber that were used to observe the given target type. Every observed DESI target in each respective full catalog is given a completeness, $f_{\rm TLID}$, that is simply the inverse of the total number of unique DESI targets within the catalog at the given tile and fiber. This number is essentially the number of targets that were competing for the fiber.

The calculation of $f_{\rm TLID}$ does not account for all fiber assignment incompleteness. Some fraction of (unobserved) targets in the full catalogs will be at a tile and fiber that did not observe any target of the given type. These data thus do not influence any $f_{\rm TLID}$ calculations. These cases occur due to, e.g., the fiber needing to be assigned to a standard star or sky fiber to meet the minimum threshold. In the case of ELGs, it will also be due to an LRG being assigned to the combination of tile and fiber associated with the given ELG target. In such cases, the target that ultimately received the observation should only depend on randomized processes within the DESI targeting, such as the subpriority value of the target, or—for ELGs competing with LRGs—whether or not it was one of the 10% that were boosted to the LRG priority. We therefore expect such completeness effects to be distributed equally within any given set of overlapping tiles, which we denote $t_{\rm group}$. The $t_{\rm group}$ associated with a given point on the sky is the set of tiles that had a DESI fiber postioner included in the good hardware definition that could have reached the point. It can thus be determined for the data and random catalogs based on set of tiles for each TARGETID that are in the respective potential assignments catalog, after applying the bad hardware veto. We thus determine a completeness, $f_{\rm tile}$, per $t_{\rm group}$ that treats the targets that influenced the $f_{\rm TLID}$ calculation as if they were observed. The groups of overlapping tiles are analogous to SDSS ‘sectors’ and $f_{\rm tile}$ is thus analogous to $C_{\rm BOSS}$. Please see section 4.3 of [12] for the full details.

A separate way to evaluate DESI assignment completeness, and to enable more than point estimates of it, is through the production of alternative realizations of assignment histories. Such realizations are produced by changing the random seed that determines quantities such as the subpriority or whether an ELG is given the same priority as an LRG. Such changing of the random seed happens when creating the initial MTL and the impact on the assignment history can be simulated by running the fiberassign software using the same settings and updating the ‘alternative’ MTL in the same order as for DR1 observations. This process is described and validated in [13]; we denote it as the ‘altmtl’ process. A total of 128 altmtl realizations were produced for DR1 LSS catalogs. We apply the same hardware veto as described in Section 4.1 to the assignments determined for each realization and store the binary True/False information on whether each target was assigned for each realization within a bit array,¹²¹²12In practice, we store the results from the 128 realizations via two 64 bit arrays. which we store in the column BITWEIGHTS. Counting DR1 observations, we have 129 total realizations and the probability of assignment for any target is simply

where $N_{\rm assign}$ is the number of realisations in which the target is assigned.

For observed targets, we expect $p_{\rm obs}$ to be a close match to $f_{\rm TLID}f_{\rm tile}$. Detailed comparisons of the assignment completeness determined from the altmtl realizations and $f_{\rm TLID}f_{\rm tile}$ are presented in [22]. We describe the DESI DR1 LSS catalogs that use the altmtl data for completeness corrections in Appendix C. We recommend using the altmtl version of the LSS catalogs for small-scale clustering measurements.

To correct for the variations in completeness in the DR1 2-point functions, we produce weights based on the completeness definitions described in the previous subsection. For our fiducial LSS catalogs, we simply use

This is analogous to the SDSS close pair weight, as described in Section 5.1.

The $w_{\rm comp}$ defined in Eq. 5.2 does not account for the incompleteness that we have tracked via the $f_{\rm tile}$ determination. Given that we identify the same tile groupings in data and random samples, we choose to apply $f_{\rm tile}$ as a weight to the randoms. A small number of tile groupings are present only in randoms, i.e., there were no reachable targets within some tile groupings for a particular tracer. Such tile groupings are typically very small regions with many overlapping tiles. We assigned these areas $f_{\rm tile}=1$. The precise implementation is detailed further in Section 8.

Our fiducial method for including completeness weights in the DR1 LSS catalogs does not produce unbiased 2-point clustering statistics, because it does not account for the fact that the number of close pairs observed at small angular scales is highly incomplete, because of the physical limit on the minimum separation of neighbouring fibers in a single tile. To account for this, [23] develop a “$\theta$-cut” method to remove small angular separations from 2-point measurements in both configuration- and Fourier-space and include the impact of removing such information into a window matrix, to be convolved with the theoretical model. We remove pairs at angular separation scales less than 0.05 degrees, as spectra of targets at separations greater than this scale can be measured simultaneously, given the distance between fiber positioners. The $\theta$-cut method is the default choice in the DESI DR1 analyses to remove any biases in derived parameters due to fiber assignment incompleteness when using large-scale 2-point clustering measurements. Residual uncertainties related to the process are studied in [10], by comparing results obtained from realistic DR1 simulations (the ‘altmtl’ mocks described in Section 11.2) to the results of simulations without any fiber assignment incompleteness.

Alternatively, one can use the bit arrays determined from the alternative fiber assignment realizations described in the previous subsection to compute ‘pairwise inverse probability’ (PIP) weights which can be used to obtain unbiased 2-point clustering measurements, as described in [64]. This is not our fiducial method in the DR1 analysis for two main reasons. One is simply that the altmtl process required to obtain the bit arrays takes significant computing time¹³¹³13Currently, it takes several days on a NERSC Perlmutter CPU node with 128 physical cores to obtain 128 realizations, with the time dominated by the I/O feedback loop that must occur in the proper order. and it is not feasible to run this number of realizations on each of a large number of separate simulations. For instance, in the DESI 2024 cosmological analyses, 25 simulations of DESI DR1 were used for validation and 1000 were used to help determine covariance matrices (these are described in Section 11). The other is that PIP weights alone cannot correct for cases where there are 0 probability pairs. There are many such pairs in regions that have been covered by only one tile, which is a significant fraction of the DR1 footprint. For any group of targets that is reachable only to one single combination of tile and fiber, the probability of observing any pairs within the group is 0. The effect of the 0 probability pairs can be corrected via angular upweighting, but the results are no longer strictly unbiased. Any angular upweighting must rely on the angular clustering of the full target sample. The relative amount of area that is covered by only 1 pass will decrease as the DESI survey is completed, and the impact of 0 probability pairs will thus decrease. The fiducial methods to correct for fiber assignment incompleteness will be re-evaluated with each data release. Characterization of fiber assignment incompleteness issues in DR1, for all DESI tracers, is detailed further in [22].

While it is not our fiducial choice for DESI DR1 analysis, the use of PIP weights and angular up-weighting is our only option for accurately measuring small-scale clustering. The PIP weights will account for both the $f_{\rm TLID}$ and $f_{\rm tile}$ factors. Thus, we must not weight the randoms by $f_{\rm tile}$ when obtaining PIP-weighted clustering measurements. We discuss this further in Section 8 and Appendix C.

For our BGS sample, we use only three maps and the linear method for the regression to determine the imaging systematic weights. The three maps used are the $r$-band depth, the stellar density, and the HI column density.

The top panel of Figure 5 summarizes the imaging systematic validation tests for the BGS sample. As for Figure 6, we show the $\chi^{2}_{\rm null}$ obtained from the data relative to the mean of that from the mocks, displaying the results for the data with bars and the range of values obtained from the individual mocks with dotted lines. The results are shown for the individual maps used for the BGS null test, all with the fiducial imaging systematic weights applied. If the three above-listed maps are sufficient, we expect satisfactory results when testing all of the maps. We also show the results summed over all maps without (last column) and with (second to last column) the imaging systematic weights. One can observe that the improvement from using the weights is close to a factor of 2 for DECam and more moderate for BASS/MzLS.

For particular maps, three cases are highlighted. One is the HI column density in the DECam region, which is included as one of the three in the regressions. The trend against HI before and after applying weights can be observed in the bottom left panel of Figure 5: the $\chi^{2}_{\rm null}$ improves from 51.3 to 26.8. One can observe the result is just outside of the range found in the 25 mocks. The other two cases are similarly strong trends in the BGS density against the $\Delta$ EBV GR in both the DECam and BASS/MzLS regions. The result is more significant in the DECam region, as it includes more data. This map was not used in the regressions, as we expect the difference in $E(B-V)$ to include CIB contamination and the BGS density to correlate with CIB contamination. We use a solid curve to display the results for $\Delta$ EBV GR when substituting EBVnoCIB for the fiducial SFD $E(B-V)$. The trend becomes more moderate but remains significant. In terms of the $\chi^{2}_{\rm null}$ we determine, the fiducial value is 43.0 for the DECam region and reduces to 24.8 when using EBVnoCIB. For the data in the BASS/MzLS region, the corresponding $\chi^{2}_{\rm null}$ are 17.9 (which is within the range found in the mocks) and 9.7, respectively. This improvement in the $\chi^{2}_{\rm null}$ in both regions suggests the trends with $\Delta$ EBV GR are primarily driven by CIB contamination in SFD $E(B-V)$ and any regression applied that would remove the trends is likely to remove real clustering modes (traced by CIB).

DESI BAO [18] and Full-Shape analysis  [19] studies demonstrate that the results using DR1 BGS samples are robust even when not using the imaging systematic weights. For any analysis that instead proves sensitive to the imaging systematic weights, we suggest robustness tests against including the $E(B-V)$ difference when determining the imaging systematic weights.

For LRGs, we apply the linear regression method. The regressions are performed separately in each of the three LRG redshift bins. In all three, we regress against maps of stellar density, HI column density, $z$-band galaxy depth, $r$-band PSF size, and $W1$ PSF depth. These five maps are sufficient to pass all null tests in the BASS/MzLS imaging area, and are motivated by the results of studies by [65] and [17] on the LRG target sample. Based on the null tests in the DECam imaging area, one additional map was needed in each redshift bin, applied only to the regression in the DECam area: $r$-band galaxy depth for $0.4<z<0.6$, $g$-band galaxy depth for $0.6<z<0.8$, and the $z$-band PSF size for $0.8<z<1.1$.

Figure 6 shows the $\chi^{2}_{\rm null}$ values obtained for each tested map, relative to the mean $\chi^{2}_{\rm null}$ obtained across the 25 mocks. The results for the data are shown in bars, while the range of values obtained from each of the individual mocks is displayed with dotted lines. In the final two columns, we show the results summing across all of the maps, one when applying the weights and the other without applying weights. One can observe that the use of imaging weights decreases the total $\chi^{2}$ by more than a factor of two for all redshift ranges for the data in the DECam region, and for the $0.4<z<0.6$ data in the BASS/MzLS region. However, the weights make little difference for LRG data at $z>0.6$ in the BASS/MzLS region.

There are four combinations of imaging property map and redshift range where the $\chi^{2}_{\rm null}$ from the DR1 LRG data is significantly higher than obtained for any mock realization. These are all in the DECam region, which has the majority of the DR1 data and thus the least statistical uncertainty on these tests. We plot the trends for each in Figure 7. Two cases are for the $0.4<z<0.6$ redshift bin. One is the SFD $E(B-V)$ map with CIB removed (EBVnoCIB), which was used for validation but not in the regression. The trend is improved considerably by the imaging weights compared to the case without weights applied (‘raw’, using the dashed curve): the $\chi^{2}_{\rm null}$ improves from 50.9 to 21.1 when using the weights obtained from the linear regression (‘corrected’, points with error bars). The other case is for a trend with the depth in $r$-band, which we do include in the regression for this redshift bin. One can observe that a downward fluctuation at a depth of $\sim$ 1000 nanomaggies drives the discrepancy with the null expectation. The $\chi^{2}_{\rm null}$ improves from 30.4 to 20.6 when comparing the unweighted to weighted cases.

For the $0.6<z<0.8$ redshift bin, none of the 25 mocks have a greater $\chi^{2}_{\rm null}$ than found when testing the DR1 data against Gaia stellar density. The stellar density is included in the regression. One can observe in Figure 7 that the size of the $\chi^{2}_{\rm null}$ is driven by large fluctuations at particular stellar density values. The $\chi^{2}_{\rm null}$ improves from 52.1 to 35.8 when comparing the unweighted to weighted cases. Notably, the total sum of the $\chi^{2}_{\rm null}$ for the DR1 LRG data with $0.6<z<0.8$ is greater than found in any mock. We discuss this at the end of the subsection.

Finally, for the $0.8<z<1.1$ bin, there is a significant trend in the DECam data with the difference in $E(B-V)$ determined using the spectra of DESI stars and the SFD map used in targeting. It can be seen in the right panel of Figure 7. One can observe that there is a clear trend showing an approximately 5% decrease in the DR1 LRG density. One can further observe that the weights make little difference; the $\chi^{2}$ is indeed slightly worse with weights, 33.4 compared to 31.6. This map was not considered for use for regressions against the LRG data, as one component of the difference in $E(B-V)$ should be CIB contamination, which we expect to be correlated with the large-scale structure traced by LRGs. We can test this by determining $\Delta$EBV GR using EBVnoCIB instead of the fiducial SFD $E(B-V)$. When we do so, the result is plotted using a solid curve in Figure 7 and we find the $\chi^{2}$ is reduced to 13.2. The dramatic reduction in $\chi^{2}$ suggests that the trend is indeed primarily driven by the CIB contamination component of $\Delta$EBV GR and is thus not a trend we wish to regress out of the data, as it would remove real large-scale structure.

The residual trends for LRGs that have been identified as potentially significant are small compared total improvement provided by the systematic weights. DESI studies of the BAO [18] and full-shape [19] fits demonstrate that the cosmology results using our DR1 LRG samples are robust even when not using the imaging systematic weights. However, further study of the residual trends may be necessary to obtain robust $f_{\rm NL}$ results from the DESI DR1 LRG samples, as studied in [11]. For any analysis that proves sensitive to the inclusion of imaging systematic weights, we suggest robustness tests against splitting out the DES region, including the $E(B-V)$ difference when determining the weights, and any residual trends as a function of the $z$-band flux threshold, as motivated by the results of [17]. Many such tests are performed in [11].

For QSO, we apply the RF regression, using all the maps that we include for validation, except for the EBVnoCIB map. The RF regression method was developed with DESI QSO in mind [66]. Another unique aspect of the QSO regressions is that the DES and DECaLS regions are fit independently. In the validation plots, we still combine the two regions into the full DECam region.

The top panel of Figure 8 summarizes the imaging systematic validation tests for the QSO sample, in the same manner as already shown for LRG and BGS in the previous two subsections. One can observe that the overall improvement of the $\chi^{2}_{\rm null}$ across all of the maps is greater than a factor of five for the DECam region and a factor of two for the BASS/MzLS region. In the DECam region, the total $\chi^{2}_{\rm null}$ after applying weights is still right at the maximum edge of the mock distribution, which motivates careful study of potential systematic effects on the clustering of the sample. The importance of any residual uncertainty on 2-point measurements is studied in the context of $f_{\rm NL}$ in [11], structure growth measurements in [19], and BAO measurements in [18].

In terms of specific maps, the bottom panel of Figure 8 presents the four cases where the data result is most extreme compared to the mock distribution. In all cases, one can observe that the trends are significantly improved compared to the unweighted (‘raw’) case. In the DECam region, for the $z$-band PSF size the $\chi^{2}_{\rm null}$ improves from 38.5 to 15.3, and for EBVnoCIB it goes from 166 to 19.8. In the BASS/MzLS region, for the $r$-band PSF size the $\chi^{2}_{\rm null}$ improves from 22.2 to 14.5, and for the $g$-band PSF size, it goes from from 15.9 to 13.1. Given the improvement in the trends and the fact that all of these high $\chi^{2}_{\rm null}$ values are not far from the maximum found in the mocks, we do not find any maps to point to particular concerns with the DESI DR1 QSO sample.

For DESI ELGs, we use the SYSNet NN regression to obtain weights to correct for the trends with imaging properties. All of the maps used for validation are used for training, due to severe trends with imaging properties for this tracer, except for EBVnoCIB (which is expected to be redundant with HI). The full details on the settings used are provided in [18]

Figure 9 shows the results of imaging validation tests performed on the DR1 ELG sample, in a similar manner as presented in the previous three subsections. In the top panel, the data results are compared to the distribution of results recovered from our 25 mock samples. In the top two panels, one can see that for both the $0.8<z<1.1$ and $1.1<z<1.6$ redshift ranges, the total $\chi^{2}_{\rm null}$ results for the data are consistent with those from the mocks, and that there is more than a factor of five improvement in the total $\chi^{2}_{\rm null}$ comparing the weighted and unweighted cases.

The results when testing against four individual maps show $\chi^{2}_{\rm null}$ values that are outside of the bounds recovered from the 25 mock realizations. The trends in these cases are displayed in the bottom panel of Figure 9. In all cases, there is a large improvement in the trends when comparing before (‘raw’) and after (‘corrected’) applying the weights for imaging systematics. For the case of the EBVnoCIB map tested against the data with $0.8<z<1.1$ in the BASS/MzLS region, the $\chi^{2}_{\rm null}$ is 23.8 with weights and 189.3 without. Similarly, for $\Delta$ EBV GR in the same redshift range and region, the $\chi^{2}_{\rm null}$ is 18.2 with weights and 182.6 without. For both, one can observe deviations at the high end of the map quantity; only 2.7% of the BASS/MzLS region has a EBVnoCIB value greater than 0.1 and only 0.5% of the region has a $\Delta$EBV GR greater than 0.05. For the stellar density, $\chi^{2}_{\rm null}$ in the DECam region and $0.8<z<1.1$ is 13.4 after weighting and 654.2 before. Finally for the DR1 data with $1.1<z<1.6$ in the BASS/MzLS region, the weighted $\chi^{2}_{\rm null}$ is 12.5 and the unweighted is 204.6.

While the imaging systematic mitigation can produce results that are consistent with our mock tests, the impact of imaging systematics on the DESI ELG sample is by far the most severe of any of the DESI tracers. Further details on the ELG density variations, including how well they are predicted by image simulations, changes in the expected redshift distribution, and the relative impact on clustering measurements can be found in [18]. There, it is also demonstrated that despite the severity of the trends, there is negligible impact on BAO measurements. The potential impact on studies of the full shape of ELG clustering statistics is investigated in [19], where a method to account for systematic uncertainty related to imaging systematic is validated and applied to obtain DR1 results. We recommend similar testing and rigor be adopted for any LSS studies of the DR1 ELG sample.

The elimination of spurious fluctuations in the observed tracer density correlated with properties relating to how DESI spectra were observed, and the impact of this elimination, are detailed in [20, 21]. Significant trends were found and investigated for numerous properties, including the effective observing time, the survey speed, number of exposures or nights required to fully observe a target, the date of observation, and the focal plane position. One outcome of the investigation was the identification of issues with the DR1 processing of data taken on the night of Dec.  12th, 2021, and the ultimate replacement of that data in our DR1 LSS catalogs, as described in Section 2.2. Despite finding many significant trends, both studies show that the impacts of these trends on the two-point clustering statistics, even when uncorrected, are negligible.

For the corrections we ultimately apply to the DR1 LSS catalogs, for all tracers we model the success rate, $G_{z}$, as a function of the template signal-to-noise ratio, TSNR2. These TSNR2 values are determined for each tracer type and per coadded spectrum, but are independent of the specific properties of the target (e.g., its brightness) observed to produce the spectrum; they use a fixed template (different for each tracer type) for the signal and the estimated noise and the result is thus proportional to an estimated effective observing time. Each is defined in [40] and we use the TSNR2 associated with the particular tracer (i.e., TSNR2_ELG for ELGs). For the BGS, LRG, and QSO samples, we use the fiberflux of the observed target as a secondary variable in the modeling [20]. For ELGs, $G_{z}$ depends strongly on the precise redshift of the galaxy, due to noise from sky lines interfering with the ability to detect the [OII] doublet. The success rate is thus modeled as a function of TSNR2_ELG and the galaxy redshift, as described in [21].

In all cases, to apply weights we find the relative success at the given redshift or FIBERFLUX, $\phi_{\rm fib}$. For BGS, LRG, and QSO, the success model is asymptotic and the weight can thus be defined as

where $S_{\rm ratio}$ is the TSNR2 value and $G_{z}$ is a function modelling the success rate as a function of FIBERFLUX and TSNR2. As TSNR2 goes to infinity, $G_{z}$ must asymptote to a constant value at fixed FIBERFLUX; we use an error function to achieve this behavior, fit it to the observed redshift failure rates, and evaluate it in the limit of $\texttt{TSNR2}\rightarrow\infty$ to determine the numberator of Eq. 7.1. For the ELGs, a linear relationship is determined for the expected success rate as a function of TSNR2_ELG at each redshift. These linear relationships are normalized to have a mean of one, weighting by the effective observing time.

For all tracers, the modeling approach described above means that we do not over-correct for samples that have an overall poor spectroscopic success rate (i.e., when $G_{z}((\infty,\phi_{\rm fib}))$ is much less than 1); e.g., in the most extreme case, QSO at low fiberflux have a $\sim 30\%$ success rate and rather than upweight them all by a factor of $\sim$ 3, we only correct for the $\sim$ 10% variation in the success rate of the subsample as a function of effective observing time. The weighted observed density for any selection in fiberflux will have a null trend with TSNR2, and thus any impact of the DESI observing pattern is expected to be nulled, under the assumption that the fiberflux dependence captures any trends with redshift¹⁷¹⁷17This not being the case for ELGs necessitated the specific redshift dependence of their weights. and TSNR2 captures the DESI observing pattern.

Uncertainties in the estimation of the DESI redshift impact the measured clustering, but are not accounted for in the LSS catalogs. These uncertainties include the random scatter from standard measurement noise, systematic biases in the redshifts, and also catastrophic errors that are due to, e.g., mis-identification of emission lines or confusion with sky lines.

For LRG, redshift accuracy can be assessed using repeated observations and is estimated at a standard deviation of 40–60 km s^-1 [74, 75] (note that different statistics are used, with [74] quoting the standard deviation and [75] quoting the normalized median absolute deviation). Its dependence on magnitude is measured in [75] and on redshift in [74], who show that the distribution of redshift errors is best fit by a Lorentzian distribution. Comparison to BOSS redshifts in Section 9 shows that only 0.3% of the LRG targets in common differ by more than 0.002 in redshift ($\sim$250 km s^-1 at typical LRG redshifts), with a mean difference of 4 km s^-1 (very similar to the mean difference of 3.3 km s^-1 found in [75] from comparison to DEEP2).

Since BGS is brighter than LRG, its redshifts are more accurate: the standard deviation is 10 km s^-1 with little dependence on $r_{\textrm{fiber}}$ [75], with a mean difference of 6.5 km s^-1 compared to DEEP2 or 2 km s^-1 compared to BOSS, and a 0.2% outlier fraction.

While ELGs are faint, their redshifts are measured more accurately than LRGs due to their strong emission lines. Their redshift errors are 8–10 km s^-1 and best fit with a Lorentzian [75, 74, 76], again from repeat observations. The impact of ELG catastrophic errors are extensively studied in [21], who find 0.27% of ELG redshifts have catastrophic redshift errors $|\Delta v|>1000$ km s^-1. [21] characterize the origins of these catastrophic errors, and their impact on the galaxy clustering two-point statistics.

Quasar redshifts are more challenging, both due to the difficulty of centroiding the broad emission lines and due to systemic shifts in the high-ionization CIV line, which is observable by DESI at $z>1.6$. The more reliable MgII line redshifts out of the DESI wavelength range at $z=2.5$, making $z>2.5$ particularly problematic. The Redrock quasar templates were updated from the Early Data Release [77], improving the redshift accuracy and bias compared to early studies in [74, 78, 79], which used the old BOSS quasar templates. The LSS catalogs use the updated templates presented and validated in [77], whereas the Ly$\alpha$ analysis additionally applied a correction for the evolution of the Ly$\alpha$ mean flux, which further improves the quasar redshifts at $z>1.8$ [80]. Redshift precision is assessed by comparing redshifts in the SV Repeat Exposures, with an overall normalized median absolute deviation (NMAD) of 57 km s^-1, which rises with redshift from a low of $\sim$ 20 km s^-1 at $z\sim 0.8$ to a peak at $\sim 125$ km s^-1 at $z\sim 1.8$, and declines thereafter [77]. Redshift precision as a function of redshift is shown in Fig. 7 of [77]. The distribution of redshift errors has very heavy, non-Gaussian tails, and are better fit by a Lorentzian [74] or a combination of three Gaussians [78]. Quasar redshift accuracy is assessed via small-scale cross-correlations with LRG, ELG, and the Ly$\alpha$ forest [77]: biases in quasar redshift measurements will lead to a radial shift in the peak of the cross-correlation from zero. These biases are shown in Table 6 of [77] and are 30—60 km s^-1 at $0.8<z<2.1$. Finally, catastrophic redshift errors are assessed from the comparison of DESI automated and VI redshifts in [79] (using the old quasar templates), with 0.7% (4.5%) of quasars having offsets $>3000$ (1000) km s^-1 at $z<2.1$ and 1.8% (12.2%) at $z>2.1$. This paper uses the old BOSS quasar templates, but [77] finds that the catastrophic rate is very similar between the old and the new templates.

Random redshift errors have a similar impact on the two-point functions as the small-scale Finger of God velocity dispersion, though smaller because the redshift errors are much smaller than the virial velocities. For the BAO analysis of the clustering of BGS, LRG, ELG and QSO tracers, the small-scale velocity dispersion is modelled by a Lorentzian multiplying the broad-band power spectrum with free parameter $\Sigma_{s}$ which is taken to have a Gaussian prior [8]. The redshift error can be absorbed by $\Sigma_{s}$ and the BAO damping parameter $\Sigma_{\parallel}$ at the level of 1-2 $h^{-1}$ Mpc for quasars (and smaller for other tracers); as shown in [81], the BAO results are insensitive to this level of change in $\Sigma_{s}$ and $\Sigma_{\parallel}$. Likewise, the impact of redshift errors on the Ly$\alpha$ BAO analysis is presented in [82] and the impact on the BAO peak position is found to be negligible. For the analysis of the full shape of clustering statistics for discrete tracers, modifications to the simple Lorentzian Finger of God form (e.g. due to the heavy tails in the redshift error distribution) are effectively encapsulated by additional EFT counterterms [83]. The impact of ELG catastrophic errors on the power spectrum and correlation function is extensively studied in [21], who find shifts of $<0.2\sigma$ in the two-point functions, at a level that does not affect the derived parameters.

To assign redshifts to the random catalogs introduced in Section 2.1 while matching their redshift distribution to that of the data, we assign them redshifts and all associated weights from randomly chosen galaxies in the data catalogs. The random catalogs will thus have all of the same weights as the galaxies, but $w_{\rm comp}$, $w_{\rm zfail}$, and $w_{\rm imsys}$ are present purely to make sure that their weighted (normalized) redshift distribution matches the data. For ELG, LRG, and BGS, the assignment of these quantities to the randoms is done separately in the BASS/MzLS and DECam photometric regions. For QSO, the DECam region is further split into the DES region and DECaLS regions as the photometric selection of QSO is different in each region. We then multiply the sampled $w^{\prime}_{\rm tot}$ value by $f_{\rm tile}(t_{\rm group})$ (Section 5.1). Finally, we normalize the weights¹⁸¹⁸18The weights described in the following subsection do not affect the normalization, as they are applied equally to data and randoms. on the randoms such that the ratio of weighted data to weighted random counts is the same in each photometric region¹⁹¹⁹19The regions are the North and South for all tracers except QSO, which further divide the South region into DES and not DES.. Further details on the process are given in Section 7.2 of [12].

Assigning redshifts to the random points as described above is often described as ‘shuffled’ randoms and matches the approach applied to SDSS [63, 61] Out of the available options, it was determined to be the least biased method for BOSS CMASS galaxies [84]. However, given the radial distribution of the randoms exactly matches the data by construction, this method nulls purely radial clustering modes and introduces a radial integral constraint bias [85], whose effects must be modeled. We summarize the method and present details on all window effects accounted for in the DESI DR1 analyses in Section 10.1.2.

Based on the principles first presented in [59], to increase the expected signal-to-noise of our clustering measurements, we apply weights to data and randoms based on the observed number density as a function of redshift and $n_{\rm tile}$, which we denote $w_{\rm FKP}$. The completeness variations in DESI DR1 are large (see Figure 2). We thus incorporate the mean completeness (see Section 5.3) as a function of $n_{\rm tile}$ into our $w_{\rm FKP}$ calculation. The process is fully detailed in section 7.3 of [12] and we repeat some details below. In order to do so while accounting for the completeness weights²⁰²⁰20They are all greater than 1 and thus otherwise up-weight incomplete regions., we first divide the total data weight column $w^{\prime}_{\rm tot}$, for both data and random by the mean completeness weight as a function of $n_{\rm tile}$, $\langle w_{\rm comp}\rangle(n_{\rm tile})$. Thus, the final $w_{\rm tot}$, included for both data and randoms and meant to account for selection function variations, becomes

This is the column WEIGHT in the DR1 ‘clustering’ catalogs. In this way, the total data counts are normalized such that they sum to approximately the number of observed objects, but still fully account for the impact of variations due to fiber assignment incompleteness, imaging systematics, and redshift systematics.

The factoring of the weights applied above allows us to determine $w_{\rm FKP}$, taking the variation in observed number density as a function of redshift and survey coverage into account. First, we define

where $C_{\rm assign}(n_{\rm tile})$ assignment completeness values are determined in the same way as in Table 6 (selecting the particular discrete $n_{\rm tile}$ rather than applying a threshold)²¹²¹21The values of $C_{\rm assign}$ and $w_{\rm comp}$ vary with $n_{\rm tile}$ for the same reason, the fiber. In this manner, $n_{x}(z,n_{\rm tile})$ is the expected number density at the location of any galaxy or random point. We then define $w_{\rm FKP}$ following the standard convention

The value of $P_{0}$ is chosen separately for each tracer, given an approximate nominal value of the power spectrum monopole $P_{0}(k=0.15h{\rm Mpc}^{-1})$. The values used in the DR1 analysis are $P_{0,\rm BGS}=7000\;({\rm Mpc}/h)^{3}$, $P_{0,\rm LRG}=10,000\;({\rm Mpc}/h)^{3}$, $P_{0,\rm ELG}=4000\;({\rm Mpc}/h)^{3}$ and $P_{0,\rm QSO}=6000\;({\rm Mpc}/h)^{3}$. These values are only roughly consistent with the actual clustering amplitude of the respective DESI samples and it is likely that slightly more optimal choices can be adopted in future DESI analyses. These $w_{\rm FKP}$ are meant to improve the expected signal-to-noise of 2-point function measurements at the scales used for BAO analyses. For such calculations, one can simply multiply $w_{\rm tot}$ by $w_{\rm FKP}$ to obtain the total weight to apply to each data/random point. Analyses that use different scales or clustering estimates might wish to derive their own, more optimal, weighting to apply. Indeed, the primordial non-Gaussianity analysis presented in [11] derives alternative weights.

We describe separately below the methods for estimating the correlation function and the power spectrum.