Deep Learning for Line Intensity Mapping Observations: Information Extraction from Noisy Maps

We prepare a large set of training and test data. We use a fast halo population code PINOCCHIO (Monaco et al., 2013) that populates a cosmological volume with dark matter halos in a consistent manner with the underlying linear density field. We generate 300 (1000) independent halo catalogs with a cubic box of $280h^{-1}$ Mpc on a side for training (test)¹¹1In the rest of this Letter, we adopt $\rm\Lambda$CDM cosmology with $\Omega_{m}=0.32$, $\Omega_{\Lambda}=0.68$, and $h=0.67$ (Planck Collaboration VI, 2018).. The smallest halo mass is $3\times 10^{10}~{}\rm M_{\odot}$. We then assign $\rm H\alpha$ luminosities to the individual halos to obtain a three-dimensional emissivity field. The halo mass-to-luminosity relation is derived using the result of a hydrodynamics simulation Illustris-TNG (Nelson et al., 2019). We assume that the line luminosity is given by a function of the star-formation rate of the simulated galaxy as

where we adopt $A_{\rm line}=1.0$ mag, and $C_{\rm line}$ is a coefficient table computed using the photoionization simulation code Cloudy (Ferland et al., 2017).

We work with two-dimensional images (intensity maps) in order to make the best use of modern image translation methods, although, in principle, it is possible to construct neural networks that read and generate three-dimensional data (e.g., Zhang et al., 2019). We generate two-dimensional $\rm H\alpha$ intensity maps by projecting the three-dimensional emissivity fields along one direction.

For each realization of the training (test) data, 100 maps (1 map) with an area of $(0.85~{}\rm deg)^{2}$ are generated by projecting random portions of an emissivity field. A total of 30,000 training data and 1000 test data are generated in this manner. The intensity maps are pixelized with the angular and spectral resolution of SPHEREx listed in Table 1.

Finally, realistic mock observation maps are generated by adding Gaussian noise to the $\rm H\alpha$ intensity map. We adopt the noise level of ”SPHEREx deep” whose $5~{}\sigma_{\rm n}$ sensitivity per pixel at $\lambda=1.5\rm\mu$m is 22 mag, corresponding to $\sigma_{\rm n}=2.6\times 10^{-6}~{}\rm erg/s/cm^{2}/sr$. The maps are normalized by $1.0\times 10^{-4}~{}\rm erg/s/cm^{2}/sr$ before input to the networks.

We develop cGANs using the publicly available pix2pix code (Isola et al., 2016). The cGANs consist of two adversarial convolutional networks: a generator and a discriminator. The generator, consisting of 8 convolutional and 8 de-convolutional layers, outputs a map $G$ (”reconstructed map”) from an observed map $x$. The discriminator, consisting of 4 convolutional layers, returns a value $D$ for the input of $(x,y)$ or $(x,G[x])$ with $y$ denoting the $\rm H\alpha$ map. The value $D$ indicates the probability that the input is not $(x,G[x])$ but $(x,y)$. During the training, the two networks are updated repeatedly in an adversarial way; the generator is updated so that it deceives the discriminator (i.e., $D(G[x])$ should get closer to 1), while the discriminator is updated so that it gets better accuracy (i.e., $D(x,y)$ and $D(x,G[x])$ get closer to 1 and 0, respectively).

Specifically, the parameters in the generator (discriminator) are updated to decrease (increase) the loss function

where

Note that we include an additional term $\mathcal{L}_{\rm L1}$ that is known to ensure better performance by imposing the condition that the values of the corresponding pixels in the true and reconstructed maps should be close (Isola et al., 2016). In each round of training, the loss function is computed with a mini-batch. After some experiments, we set $\lambda=1000$ and batch size 4. The networks are trained for 8 epochs. We adopt these parameter values throughout the present study.

The probability distribution function (PDF) of line intensity is an excellent statistic that can constrain galaxy populations and their physical properties (e.g., Breysse et al., 2017). We test whether our networks also recover the PDF accurately.

We first note that, in general, a single set of networks do not reproduce pixel statistics of images/maps robustly. We thus resort to training multiple networks and take the mean of the statistics reconstructed by the ensemble of networks. This technique, called ”bagging,” is known to reduce generalization errors (Goodfellow et al., 2016), and has been applied to, for instance, de-noising weak lensing convergence maps (Shirasaki et al., 2019). In practice, we average the PDFs reconstructed by 5 networks that are trained with different datasets.

Figure 2 compares the PDFs of true and reconstructed maps. The vertical dashed line indicates the $1\sigma$ noise level. Our cGANs are able to reconstruct the PDF of the $\rm H\alpha$ intensity above 1$\sigma$. Note also that the scatter of the averaged PDF lies within the intrinsic scatter of the true $\rm H\alpha$ maps, i.e., within the so-called cosmic variance. Apparently, the networks tend to reconstruct the PDFs close to the average. This is simply caused by the bagging procedure. We have checked and confirmed that the variance of reconstructed PDFs by a single network is as large as the intrinsic one.

We further examine the ability of our cGANs to reconstruct the intensity power spectrum. To this end, we again adopt the bagging of 5 networks that are trained with different datasets. The red points with error bars and the shaded regions in Figure 3 show the power spectra of the reconstructed and true $\rm H\alpha$ maps, respectively. We also show the noise power spectrum, $P_{n}(k)$, and its variance, $P_{n}(k)/\sqrt{N_{k}}$, where $N_{k}$ is the number of modes in $k-\Delta k/2<|\bm{k}|\leq k+\Delta k/2$. We adopt $\Delta\log k=\Delta k/k=0.2$.

We notice that the reconstructed power spectra on large scales ($k\lesssim 0.5$) are systematically underestimated. This might be owing to the finite box size of the training data. To examine this, we train the cGANs with mock intensity maps with a larger area. For this test, we generate halo catalogs in a cubic volume of $700h^{-1}$ Mpc on a side (see Section 2.1). Then the smallest halos populated is degraded to $3\times 10^{11}~{}\rm M_{\odot}$, but we have confirmed that the mean $\rm H\alpha$ intensity (or the total luminosity density) is not significantly different from that with our default box size of $280h^{-1}$ Mpc. We set the side length of the pixel $l^{\prime}=2.0~{}\rm arcmin$ and the spectral resolution $R^{\prime}=41.5$. Each map has a ten times larger area of $(8.5~{}\rm deg)^{2}$. The noise level scales with the angular and spectral resolution as

where $l$, $R$, and $\sigma_{\rm n}$ are the original angular and spectral resolution and the noise level of SPHEREx (Table 1). The resulting noise level of the wide maps is $\sigma_{\rm n}^{\prime}=1.3\times 10^{-7}~{}\rm erg/s/cm^{2}/sr$. We adopt the same hyperparameters in the cGANs as in our default case except we set $\lambda=200$ and the normalization factor $1.0\times 10^{-6}~{}\rm erg/s/cm^{2}/sr$ for the low-resolution maps. The purple dots with error bars in Figure 3 show the power spectrum of the reconstructed wide maps. The light-pink shading indicates the 1$\sigma$ dispersion of the true $\rm H\alpha$ power spectra. Clearly, the large-scale (low-$k$) power spectrum is reconstructed more accurately compared to our default case. We note that the cGANs trained with the wider maps do not resolve point sources, but the peaks and voids in the reconstructed map correspond closely to the positions of groups/clusters and void regions.

Ideally, networks trained with intensity maps that have fine pixels and a large box size would be able to reconstruct both the positions of point sources (galaxies) and their large-scale clustering. Unfortunately, it becomes computationally more expensive if we set a larger number of pixels. The computational time for training roughly scales with the number of pixels, and the necessary number of training epochs could also increase. We thus suggest that one should generate training data depending on the purpose. In order to detect point sources robustly, one needs to train the networks using fine-pixel maps. If the primary purpose is to reconstruct the large-scale power spectrum, for cosmology studies for instance, then one needs to generate maps with a sufficiently large area (volume) but with coarse pixels. The reconstructed power spectra shown in Figure 3 suggest that one should adopt at least a several times larger area for training than the actual size of observed maps.

The line intensities of high-redshift galaxies are not well constrained observationally, and theoretical models of galaxy formation and evolution remain uncertain. Thus it is important to examine whether our GAN-based method can be applied robustly to data that are generated with different line emission models. To this end, we generate three additional sets of 1000 test data that have the same noise level but have different $\rm H\alpha$ line intensities. Two of the new test datasets are generated by simply multiplying the original $\rm H\alpha$ maps by a constant value, $c_{\rm H\alpha}=0.5$ and 2, as

These cases serve as a test of the networks’ reconstruction performance against the variation in the mean intensity. We also adopt a completely different $\rm H\alpha$ model of emission, Silva et al. (2018), in which the SFR-halo mass relation derived by Guo et al. (2013) is used to assign the $\rm H\alpha$ luminosities ($\propto$ SFR). The model reproduces the mean $\rm H\alpha$ intensity consistent with observations (Sobral et al., 2013, 2015).

Figure 4 shows examples of reconstructed images. In all the cases, the locations and amplitudes of bright peaks are reconstructed well, even though the mean intensity levels are quite different from the original data (see the color bars). It appears that the networks practically learn the noise properties and extract the signal robustly, even if the signal amplitude is different from the training data. It is remarkable that, in the cases with $c_{\rm H\alpha}=2$ and with Silva et al. (2018) model, the $\rm H\alpha$ signals are reconstructed as accurately as the default $c_{\rm H\alpha}=1$ case.

We also compare the one-point PDFs and the power spectra in Figure 5, where we plot the averages over 1000 test data with $c_{\rm H\alpha}=0.5$ (cyan line), 1 (red), and 2 (purple), respectively. The yellow lines show those with the Silva et al. (2018) model. The statistics of each reconstructed image are computed by taking the means of those reconstructed by 5 different networks. Both the one-point PDFs and the power spectra are reproduced well except the case of $c_{\rm H\alpha}=0.5$ that has effectively small signals. We thus conclude that our network is applicable to maps with different intensity models as long as we have a good understanding of the observational noise and if the line intensities are not too weak compared to the training data.