STag II: Classification of Serendipitous Supernovae Observed by Galaxy Redshift Surveys

Since DESI is not a survey designed to observe SN, the vast majority of DESI spectra will not be of a SN. As such, we use DESI spectra of real galaxies that have been combined with template SN spectra. These templates consist of both core-collapse SNe (Type Ib, Ic, and II) [31] and Type Ia SNe [32], allowing us to cover the full range of SN types we wish to be able to classify. We note here that this is a different set of supernova templates to those used in the first version of STag [20]. These models are first loaded into a script, with certain core-collapse models blacklisted due to being non-standard (namely SN 2013by, SN 2013fs, SN 2009bw, SN 2012aw, SN 2009kr, ASASSN14j, SN 2013am, SN 2008ax, SN 2008fq, SN 2009ip, iPTF13bvn, SN 2008D, SN 1994I, SN 2007gr, SN 2009b, and SN 2007ru).

A model is then randomly selected each time a simulated spectrum is generated, with a random phase in the range -10 to +30 days relative to maximum light (bolometric). One can then also manually choose a magnitude difference between the galaxy and the SN, which allows for variation in the flux ratio in the final simulated spectra: a flux ratio of 1.0 means that there is only SN light and a flux ratio of 0.0 means only galaxy light is present. Next a real DESI exposure is chosen from the main survey, importantly including the metadata corresponding to the observing conditions for that specific exposure. This allows for the SN model to be simulated as if it had been observed in the same way as the selected DESI spectrum. The newly simulated SN spectrum is then resampled to the correct wavelength range and added to the galaxy flux from the DESI spectrum, thus creating a simulated SN spectrum as if it had been observed by DESI. By following this process we generated a total of 400525 spectra, which was the total number of spectra containing both galaxy and SN light using the aforementioned process to generate $\sim$600000 total spectra (including spectra of just galaxies). Each class was roughly equally represented with 100205 Type Ia, 99982 Type Ib, 100117 Type Ic, and 100221 Type II spectra respectively. From this, the spectra were split into training, validation, and testing sets of size equal to 72%, 18%, and 10% of the total spectra, following typical values used in machine learning (though with a desire for a greater proportion of training data compared to the first version of STag [20] in order to maximise the possible variety in spectra trained on).

Whilst the entirety of training and testing of STag II was done using the simulated DESI spectra (we note here that simulated DESI spectra refers to the combination of real DESI spectra combined with model SN light, as described in Section 2.1), we also wanted to test its capabilities with actual SN spectra observed by DESI. There are not many examples (relative to the total number of DESI spectra), but there have been some fortuitous cases where DESI observed a galaxy that was host to a SN. Such SNe can be identified by comparing the RA and DEC from DESI with that given for SNe in the Transient Name Server (TNS)²²2https://www.wis-tns.org, then further refining the selections by choosing only DESI observations that occur within some suitable time frame around the reported TNS discovery date. For our case specifically we chose observations that were made $\pm 3$ weeks of the discovery date, giving us a far greater chance of the DESI spectra containing SN light.

Using these parameters we end up with a total of 8 DESI observations that match with a TNS report, covering a mean Julian date (MJD) range from 59319.4 to 59550.2, which we show in Table 1. One can quickly see an issue that has arisen, in that the reported redshift for some of the objects are different between DESI and TNS. The redshift calculated in DESI is done so by an algorithm called Redrock [33], which finds the best-fitting template in order to estimate the redshift. However, since DESI is not a SN survey these templates do not contain any examples of SN spectra, and as such any attempt by Redrock to estimate the redshift of a SN spectrum can result in an erroneous redshift. A good example of this is seen for the Redrock redshift of SN 2021zfs, which is significantly different to the redshift reported by TNS. Since an incorrect classification will result from the wrong redshift being used, it is important to ensure that the Redrock redshift is indeed reliable.

We were also fortunate to be working on the improvements to STag during the time when SN 2023ixf was first discovered. Since the decision was made by the DESI collaboration to spend time observing this SN, we were able to obtain these spectra and use STag to classify them. A brief description of SN 2023ixf and the results of the classification can be found in Appendix B.

What follows is a brief description of how STag works, for a more detailed overview of the architecture the reader is directed to our previous paper [20].

The tag probabilities are calculated using logistic regression, a function that rapidly transitions between 0 and 1; given by Equation 3.1 [50]:

Once a spectrum has all the tag probabilities calculated, these are then passed to a simple ANN which determines the best suited class using softmax regression, which is given by Equation 3.3:

With the introduction of a new dataset, it was necessary to also revisit the tags and recreate them making use of the new data. Some of the original tags had long wavelength ranges that potentially included other features (see Figure 1) and in the case of the hydrogen tag, utilised the whole spectrum. As such it was unclear exactly what it was using to determine the presence of hydrogen. This initially led to the recreation of all the original tags with stricter wavelength limits (see Figure 2), as well as the introduction of tags for individual hydrogen features.

With the new tags created, the neural network was repeatedly trained and tested with different combinations of included tags in order to determine which tags were necessary for the classification of a SN and which were superfluous. The result of this extensive training and testing was that the He i $\lambda$6678 tag (which is characterised by a doublet feature) was causing a lot of confusion between classifications and as such was removed. Specifically, since this feature was very seldom seen in spectra outside of Type IIb SNe, if it had even a $<10\%$ tag probability it was enough to reduce the softmax probability of the predicted class. This had the knock-on effect of also necessitating the removal the Type IIb class from STag’s functionality as there was no longer a suitable spectral feature to differentiate it from the over-arching Type II class. Another change that was borne out of these tests was that it was decided that an increase in the resolution of the spectra would help account for smaller changes in the tags. The original value of the spectral resolution was chosen to be $R=1024$ as this was the default setting used in DASH, selected so as to be able to distinguish between broad and narrow features whilst maintaining the desired speed of the classifier [19]. As such, to accommodate for a greater capability of catching the more subtle changes of spectral feature whilst also avoiding introducing too much noise or making STag drastically more computationally expensive, the number of points that make up the spectra was increased from 1024 to 1500. This value was chosen after trialling different spectral resolutions of up to 2048, whereupon it was ultimately decided that the value of 1500 allowed for the best balance of finer details and computational cost.

The Fe ii tag was also further reduced, instead of being one tag that encompassed three suspected features it is now a tag specifically for the Fe ii $\lambda$5170 feature as it was found to be the most frequent and readily identifiable. It was also decided that the distinction between an emission, absorption, and P-Cygni line profile for the He i $\lambda$5876 feature was unnecessary and likely causing more confusion than clarity when reporting on tag probabilities (it was not uncommon for spectra to have high probabilities for both absorption and emission tags [20]). As a result, only the absorption feature is now explicitly tagged for and results in a clear identifier of the He i $\lambda$5876 absorption feature.

To further strengthen the classifications, as well as to help the neural network to make the correct classification in the first place, a number of new tags were also introduced. These consist of a tag for Si ii $\lambda$4000, H$\alpha$ and a narrow H$\alpha$ $\lambda$6563 (henceforth referred to as n-H$\alpha$, see Appendix B for a more detailed description of these two hydrogen lines), and H$\beta$ $\lambda$4861. Whilst not requiring a new tag specifically, the spectral features of Si ii $\lambda$5876 and Na i D $\lambda$5876 both make use of the tag for He i at the same wavelength. All three of these features overlap at approximately the same wavelength so even with the more restricted ranges of the tags, it would still be impossible to create separate tags for each of these. We therefore make use of the fact that the He i $\lambda$5876 tag is now only for an absorption feature and that both Si ii $\lambda$5876 and Na i D $\lambda$5876 are also absorption features to effectively create a multi-purpose tag.

The final set of tags included in STag II are H$\alpha$, n-H$\alpha$, and H$\beta$ for Type II SNe, Ca ii H&K, Si ii $\lambda$4000, Si ii $\lambda$6355, and S ii for Type Ia SNe, He i $\lambda$5876 for Type Ib SNe, and Fe ii $\lambda$5170 as a general tag. The differences between the tags included in the two different versions of STag can be found in 2.

The equivalent width (W) of a line is defined as in Equation 3.4:

In order to see if there was extra information that could be used between the probability of a given tag and the equivalent width of the associated spectral feature, we also trialled including a nonlinear combination of these two values by passing the product of each pairing as separate inputs to the neural network. Ultimately it was determined that such nonlinear combinations offered no extra information for which the neural network could use to improve the classifications, with the exception of the Si $\lambda$4000 feature. It is unclear why this feature, and this alone, makes a difference on the accuracy of the classifications, however its inclusion does lead to a greater accuracy and so it continues to be utilised.

The inclusion of equivalent widths, and the changes to the tags, meant that the architecture of STag had to be changed. STag now consists of an input layer of 18 nodes, followed by three fully connected layers of 48 nodes each, then two layers of 96 nodes each, which then connects to two more layers of 48 nodes each, and finally a softmax output layer consisting of 4 nodes. Each of the hidden layers make use of the Rectified Linear Unit (ReLU) activation function [54]. This neural network was built using a combination of both keras [55] and TensorFlow [56] and has the architecture it does after extensive testing to find the best-performing number of hidden layers and nodes.

In order to give us a further measure of confidence in a classification beyond just the associated softmax probability, we also made use of what is known as the rlap score, as devised by [16]. A value of 0 would mean that there is no correlation between two spectra being considered, whilst it is typically accepted that a value above $\sim$5-6 suggests a reasonably strong correlation [16, 19]. This parameter is built off of first cross-correlating the spectrum of the SN with that of a template spectrum that is at $z=0$:

The rlap score is the product of two values: r, which is the cross-correlation height-noise ratio [57] which is given by Equation 3.6, and lap, which is a measure of the overlap in wavelength space of the two spectra being cross-correlated and is defined by Equation 3.7 [16].

We make use of 5 different phase bins, which are based on the phase range used to generate the DESI template spectra (and as such the DESI simulated spectra). The first bin encompasses all negative phase spectra (with respect to maximum light), the next 3 bins are separated by 7 days each, and the final bin is for all phases greater than 21 days since maximum light, up to 30 days.

The cross-correlation process was adapted from that used in DASH [19], with minor adjustments made to fit the pipeline of STag. These changes included, but are not limited to, adjusting the wavelength points to consider (since we increased the resolution of STag from 1024 to 1500 points) and a change to account for the reading of the different template spectra being used, but functionally it works the same as in DASH).

STag is highly dependent on accurate redshifts for it to be able to classify a spectra correctly, as such it is important that any redshifts being supplied are correct. However, STag is not designed to be something that can estimate a redshift. As such, we are not interested in producing a comprehensive framework for calculating the redshift of a SN, though we note that STag could be modified to do this by using the rlap score. Instead, we adopt the philosophy that if a redshift is significantly inaccurate, we accept that STag will simply be unable to return a satisfactory classification. We decided that should a spectrum initially fail the rlap criteria, this may be caused by an incorrect redshift and so we can then perform a check for a degree of inaccuracy. In the case of a relatively small inaccuracy it is worth being able to potentially use a more accurate redshift as this will lead to a better classification, as well as the possibility of a spectrum passing the rlap cut that it would have previously failed. The general principle we used was to check a series of redshifts within 10% of 1+z (where $z$ is the redshift returned by Redrock), resulting in a check of 5% either side of the given redshift. We chose to look at just 10 redshifts in the proposed range (balancing speed with enough redshift values to properly explore values around the given redshift), which means that each redshift is roughly 1% from its surrounding redshift values. What we found was that a spectra would pass the final rlap cut if the redshift was within 1% of the true redshift, whilst the rlap value would quickly fall off outside of this limit (see Figure 4). Therefore, should none of the 10 alternative redshifts pass the rlap criteria then STag will not return a classification and indicate to the user that it is possibly being caused by an incorrect redshift value.

We made use of the DESI simulated data (as detailed in Section 2.1) that had been set aside for use as testing data, which was a total of 40053 spectra (10% of the total spectra). All of these spectra were classified by STag II, before having the best rlap score calculated for each one. After applying our cut of only accepting classifications with an rlap score >6, the final number of spectra was 4338. We present the normalised confusion matrix for the classifications of these spectra in Figure 5.

We note a very high accuracy for all classes, with both Type Ia and Ib SN having a 99% accuracy. The classification success of Type Ic SN is lower at 88%, though this likely due to the lack of a clear, defining characteristic of the Type Ic class. We also consider the 100% success rate for Type II SNe to be misleading, as this is likely a result of the fact that there are simply fewer distinct examples of Type II SN used in creating the simulated data and so is learning the specific details of the SN models used, rather than of the Type II archetype overall. Still, assuming that the models used are truly representative of the full population of Type II SNe we expect that STag II is capable of classifying this type reliably (though, as a counter example, we refer the reader to Appendix B).

We classified the 8 SN listed in Table 1, including the functionality of STag to report back whether a redshift had been changed from the input value, and if so what the new redshift was. We find that we are able to correctly classify 2 of the SN after the rlap cut is applied, with Table 3 (see Appendix A) showing these along with any significant tag probabilities. This lack of completeness is a result of the mismatch between DESI observation date and the TNS discovery date, with the observation date often being before the reported discovery, which results in a lack of supernova light in the DESI spectra. We now discuss each of the 8 cases in detail, with explanations for a missing classification where applicable.

It is worth comparing the new version of STag with the first version, especially as it has undergone some significant changes. Whilst STag II can no longer classify Type IIb SN, the remaining 4 types are considered to be stronger due to the changes made to the tags and the inclusion of equivalent widths. The addition of the rlap score means that we now have a way of better quantifying whether a classification is trustworthy, with the added bonus of enabling us to give a prediction of the phase. The additional redshift checking is also an improvement for the reliability of STag. Finally, the fact that we have now trained with not only significantly more spectra than before, but these spectra now also include galaxy light and so are more representative of real data, meaning STag II does not require near-perfect spectra.