Identifying microlensing events using neural networks

Arguably, the most important legacy value of photometric surveys for gravitational microlensing events is opening-up a new branch of observational astronomy, now called time-domain astronomy. Paczyński (1986) proposed to monitor the brightness of millions stars in the Magellanic Clouds over a time scale from hours to years to search for gravitational microlensing caused by hypothetical dark, compact objects in the Milky Way halo, which—as suspected at that time—may have constituted dark matter. The practical aspect of this program seemed “formidable” to Paczyński (1986) – his proposal required frequent, repeatable observations of large high-stellar-density areas of the sky.

Setting aside practical (but essential) aspects of Paczyński’s proposal (photometric reductions of thousands images, measuring the brightness of millions of stars, creating efficient photometric databases, etc.), finding microlensing events in the data must have seemed challenging – the first microlensing event detected by the Optical Gravitational Lensing Experiment (OGLE; Udalski et al., 1993) was found among 1.1 million light curves of stars. All light curves passed through a variety of automated filters but in the last step human experts must have visually vetted hundreds of candidate objects.

The situation has not changed much since these first discoveries, despite the fact that the number of known microlensing events exceeded several thousands. Modern major microlensing surveys – OGLE (Udalski et al., 2015), MOA (Microlensing Observations in Astrophysics; Bond et al., 2001), and KMTNet (Korea Microlensing Telescope Network; Kim et al., 2016) are each observing hundreds of millions of stars and selecting rare microlensing events may require laborious efforts. For example, when building their event-finding algorithm, Kim et al. (2018) manually reviewed almost 400,000 light curves. The main disadvantage of such an approach – besides the number of work hours needed to manually vet the light curves – is the difficulty in quantifying the selection biases and measuring the event detection efficiency. Some events may be inadvertently rejected by a scanner and quantifying that effect is virtually impossible.

Broadly speaking, microlensing events may be detected either in real-time (as the event progresses) or in an offline search of archival data. The former problem being much more challenging than the latter – due to limited amount of information in the light curves. In this paper, we focus on the search for microlensing events in the archival data. From the historical point of view, finding microlensing events in real-time was instrumental for the success of the field (e.g., Udalski et al., 1994; Gaudi, 2012). The first generation surveys could not observe the sky with a high enough cadence to accurately cover light curves of interesting events, especially those by planetary systems. Thus, survey groups searched for probable microlensing events in real-time, distributed alerts, and only the most promising events were monitored with a higher cadence by follow-up teams, as advocated by Gould & Loeb (1992).

The observing capabilities of the current second-generation surveys have improved since then, so the role of follow-up observations – and alerts – has become less important. The survey telescopes monitor the central regions of the Galactic bulge with a 15–20 min cadence, which is sufficient for them to detect and characterize short-duration events and anomalies without the need of follow-up observations (e.g., Sumi et al., 2011; Yee et al., 2012; Shvartzvald et al., 2016; Mróz et al., 2017). This will also be true for the planned microlensing survey with the Nancy Grace Roman Space Telescope, which is expected to detect nearly 30,000 events (Penny et al., 2019). Roman will observe its microlensing field continuously with a cadence of 15 min. Moreover, many events detected by Roman will be inaccessible to ground-based telescopes owing to extremely high extinction, rendering their follow-up difficult. Thus, it is vital to devise robust automated algorithms for detecting microlensing events in the survey data.

The problem of finding single-lens events in the archival data has been discussed by the number of authors (e.g., Alcock et al., 2000; Popowski et al., 2001; Sumi et al., 2003; Hamadache et al., 2006; Tisserand et al., 2007; Wyrzykowski et al., 2009; Sumi et al., 2011; Wyrzykowski et al., 2015; Mróz et al., 2017; Kim et al., 2018). Usually, they use a series of selection cuts which narrow down a sample of potential events. This introduces a problem of a balance between the size and purity of the sample – too strict selection cuts may reject a significant fraction of genuine microlensing events, whereas the sample may be contaminated by non-microlensing light curves if the criteria are too loose.

While the problem of finding single-lens events is well understood, detecting binary microlensing events is much more challenging owing to a variety of possible light curve shapes (Liebig et al., 2015). Light curves of single-lens events can be described by a simple analytical formula (Paczyński, 1986), which renders their modeling easy and fast. On the other hand, modeling light curves of binary microlensing events requires a large amount of computational time: there are at least six highly non-linear parameters and finding the best-fitting solution requires a grid search over some of them. Thus, fitting the binary-lens models to all candidate light curves is (and likely will be) unfeasible. Any statistical studies of binary microlensing events are hampered by the lack of automated and robust detection algorithms. To our knowledge, no dedicated selection algorithms for binary-lens events are available in the literature.

The number of microlensing events expected in the future microlensing experiments forces us to consider fully-automated approaches. Here, we present two machine-learning neural-network-based classifiers for detecting single and binary microlensing events. Both algorithms are trained on a sample of microlensing events from the OGLE-IV survey (Mróz et al., 2017, 2019). We demonstrate their good performance on the OGLE-III sample of microlensing events. Moreover, we also demonstrate that these classifiers work well on a sample of microlensing events detected by the Zwicky Transient Facility (ZTF; Mróz et al., 2020), showing that they may be adapted to other experiments.

Although most studies on microlensing employ a traditional approach for finding microlensing events (in which a series of selection cuts is applied to the entire sample of light curves), a few notable studies used machine-learning techniques. We would like to review them in more detail. Wyrzykowski et al. (2015) used a Random Forest classifier to identify single-lens microlensing events in the OGLE-III data set. Their selection method was twofold. First, they used a series of selection cuts to narrow down a sample of possible microlensing events to $\sim 50,000$ light curves. Subsequently, they calculated 27 features for each light curve, which they used in their Random Forest classifier. The performance of the classifier was moderate: 96.7% of objects were correctly classified, but the false-positive rate was relatively high (27.7%). For that reason they added a second stage of classification, which allowed them to lower the false-positive rate to 6.7%. Similarly, Wyrzykowski et al. (2016) used the Random Forest classifier to select single-lens microlensing events exhibiting annual parallax effect.

Machine-learning-based techniques are also used to select a sample of microlensing events in data from the United Kingdom Infrared Telescope (UKIRT) survey (Chu et al., 2019). Chu et al. (2019) used three different classification methods: Random Forest, Support Vector Machine, and K-nearest Neighbor (of which the Random Forest classifier performs best, G. Bryden, priv. comm.) They extracted 66 features for every light curve, but in practice only 49 features were used for the best classifier performance. Their microlensing classifier has a 93.8% precision and 97.8% recall. Both Wyrzykowski et al. (2015) and Chu et al. (2019) trained their classifiers using a subset of manually-labeled data.

A Random Forest-based algorithm for detecting microlensing events in real-time was developed by Godines et al. (2019). They simulated light curves of constant stars, RR Lyrae and Cepheid variables, cataclysmic variable stars, and microlensing events, which they used as the training set. They calculated 47 light curve statistics for every object and then run the principal component analysis to select 44 most useful features. They achieved a 94% accuracy on simulated light curves. Godines et al. (2019) implemented their algorithm to search for ongoing microlensing events in the ZTF alert stream but its performance was affected by the lack of baseline data.

Mróz et al. (2017) and Mróz et al. (2019) presented two large samples of microlensing events identified in data from the OGLE-IV survey (Udalski et al., 2015). Both data sets were constructed in a traditional fashion – a series of selection cuts was applied to light curves of all objects observed by OGLE in the Galactic bulge. We were curious how efficiently this task can be performed by machine-learning techniques. Can they outperform traditional methods? First, strict selection cuts of Mróz et al. (2017, 2019) removed some number of genuine microlensing events from the sample. Second, the proposed criteria were designed to select point-lens point-source (PSPL) events and thus binary and anomalous events were rejected.

In this work, we use Deep Learning (DL) techniques, which are currently commonly used, both in scientific and industrial applications. For an overview of DL, we refer the reader to Géron (2019). In short, our networks are composed of several layers (see Figure 1), every layer is composed of many neurons (“units”) and is fully connected to the next layer. Every neuron performs a simple non-linear transformation of the input data (either input features or output from an earlier layer). The network is “trained” by adjusting the values of its parameters (weights and biases) so that the loss function (quantifying the difference between the network’s predictions and input labels) is minimized. We additionally use standard “dropout layers”, which prevent the network from overfitting.

We did not attempt to classify all OGLE light curves (including those of constant or periodic variable stars). We used the methods of Mróz et al. (2017) and Mróz et al. (2019) to preselect a sample of light curves “enriched” in microlensing events. We chose objects showing at least three consecutive data points (“bump”) that are at least $3\sigma_{\rm base}$ above the baseline flux $F_{\rm base}$, where $F_{\rm base}$ and $\sigma_{\rm base}$ are calculated using data points outside a 720 day window centered on the event (after removing $5\sigma$ outliers). We imposed the following four conditions: i) $\chi^{2}_{\rm out}/\mathrm{dof}\leq 2$, ii) $\chi_{3+}\geq 32$, iii) $n_{\rm DIA}\geq 3$, and iv) $A\geq 0.1$ mag. Here, $\chi^{2}_{\rm out}=\sum_{i}(F_{i}-F_{\rm base})^{2}/\sigma_{i}^{2}$ for data points outside the window (it quantifies variability in the baseline), $\chi_{3+}=\sum_{i}(F_{i}-F_{\rm base})/\sigma_{i}$ for data points within a bump, $n_{\rm DIA}$ is the number of data points within a bump detected on the subtracted images, and $A$ is the amplitude of the event. We additionally removed all candidates that were located close to each other and magnified in the same images – these are spurious detections. See Mróz et al. (2017, 2019) for a more detailed description of the method and selection cuts. Our previous tests indicate that the vast majority of genuine microlensing events ($\sim 95\%$) passes these criteria (Mróz et al., 2019).

We trained our DL algorithms on that sample “enriched” in microlensing events. We constructed two classifiers: the PSPL classifier is supposed to distinguish PSPL events from all other objects, whereas the binary-lens classifier is designed to recognize both PSPL and binary (anomalous) microlensing events.

In the first paragraph of Section 3 we asked ourselves whether DL classifiers can outperform traditional methods of selecting microlensing events. Comparing DL results with those of Mróz et al. (2017, 2019) is difficult because we used their microlensing events as the training sample. Our tests, based on a subsample of events not used in the training process, indicate that both classifiers are able to correctly classify $97.5-98.0\%$ of PSPL events. The training sample included 11,701 PSPL events, from which we should be able to select $\sim 11,400$, whereas the sample of Mróz et al. (2019) contains only 8002 events (that is, DL classifiers were able to select $\sim 40\%$ more events). However, Mróz et al. (2019) applied some physically-motivated cuts (such as limits on $u_{0}$ and source magnitude), so the real improvement is smaller than 40%. On the other hand, the samples of Mróz et al. (2017, 2019) were very pure ($\sim 99.5\%$), whereas our classifiers have a lower precision ($\sim 96\%$). Thus, the answer to the question raised in Section 3 depends on the scientific application of the data set. If a large sample size is needed, machine-learning techniques are better. If high purity is required, traditional methods may perform better.

The performance of our DL classifiers is as good as that of traditional techniques of finding microlensing events (and in some cases classifiers were even better than a human). This has several reasons. First, our training sample is based on real data and includes a significant fraction of artifacts and other non-microlensing light curves. Some previous classifiers were trained on artificially-created data which – when dealing with unknown objects – returned spurious results. Instead of creating synthetic light curves “from the scratch”, a better strategy may involve injecting microlensing signal into real light curves of constant stars (see Mróz et al. (2019) for details of such simulations). Such approach has an advantage that it preserves original noise in the data, which may be non-Gaussian and otherwise difficult to simulate.

Another explanation of our classifiers’ performance in the quality of the training and test data. In particular, their noise properties are well understood (Skowron et al., 2016) which enables us to use $\chi^{2}$ per degree of freedom as a robust and transferable statistics. Moreover, the light curve features used in our classifiers do not explicitly depend on cadence and number of observations. Thus, the classifiers – trained on OGLE light curves – work well at recognizing microlensing events in the ZTF data.

The current microlensing surveys have worked well without using machine-learning techniques to identify microlensing events. However, as we demonstrated, neural-network classifiers may be used to construct unbiased large samples of single-lens events or to recognize binary-lens events, which are otherwise difficult to identify using traditional methods. Our classifiers are able to correctly recognize $\sim 98\%$ of single-lens events and $80-85\%$ of binary-lens events in the test sets. As we expect that the future space-based surveys should be able to detect an order of magnitude more events than the current ground-based experiments, the use of DL techniques may be necessary to achieve the maximum science returns. High-cadence continuous observations from space are especially suited for machine-learning classification.

This work has made use of data from the OGLE survey. We would like to thank OGLE observers for their contribution to the collection of the photometric data used in this paper. We would like to thank Radek Poleski for sharing his classifications of OGLE microlensing events and Dmitry Duev for discussions on neural networks. We thank Dmitry Duev, Radek Poleski, and Andrzej Udalski for their comments on the manuscript.