The Solar System Notification Alert Processing System (SNAPS): Design, Architecture, and First Data Release (SNAPShot1)

Asteroids are the most numerous objects in the Solar System and as such act as tracers of the dynamical and physical evolution of our planetary system. At present, there are more than 1 million known asteroids. A comprehensive understanding of the evolution of our Solar System can therefore use the individual properties of asteroids as the fine-grained measurements from which broader conclusions can be drawn. Asteroids can be used to understand the dynamical and collisional history of the Solar System, the intrinsic material properties of primitive bodies, and help unravel the origin of life on Earth through understanding the degree to which asteroids brought water and organics to the early Earth. Individual asteroids are akin to pixels in a large image, and by building up our knowledge, one pixel at a time, the entire picture is revealed.

To gain this global understanding of the Solar System through asteroids, large datasets are required. The Vera C. Rubin Observatory will carry out the Legacy Survey of Space and Time (LSST) starting in 2024 and will revolutionize many fields of astronomy. LSST will observe more than 5 million main belt asteroids (LSST Science Collaboration et al., 2009). It is not only the number of asteroids to be observed that is significant, but the fact that most asteroids will be observed many hundreds of times, under relatively uniform conditions. This data set will be far richer — and far more scientifically valuable — than existing catalogs of asteroid measurements. However, the LSST data by itself (time, brightness, position) will not provide new insight into the processes that have driven the evolution of our Solar System. To reap the scientific reward from this data set, the scientific context and physical properties of the entire asteroid population must be understood.

The current LSST schedule of first light in 2023 and science operations in 2024 means that the next steps in understanding the history of the Solar System are not far away. This has motivated us to develop the necessary infrastructure now, to be prepared for the arrival of LSST data. Fortunately, the ongoing Zwicky Transient Facility (ZTF; Bellm et al. 2019) survey is a present-day analog of and precursor to LSST. We are using data from ZTF to develop, hone, and prepare for the coming LSST era.

All-sky surveys enable two important research goals: (1) understanding global trends, and (2) looking for rare events, as follows. In category 1, by deriving physical properties of millions of asteroids we can understand the global properties of the asteroid belt that reveal information about the formation and evolution of our Solar System. These are signatures that are only revealed through very large sample sizes. Examples here could include measuring asteroid lightcurve periods and amplitudes as a function of color (and implied composition), where the required densities and strengths may differ between rocky S-type asteroids and more primitive C-type asteroids. Large data sets can enable detecting subtle signatures, or slicing datasets (example: as a function of size) to reveal trends.

Objects detected in category 2 — detecting and characterizing rare events — also can place constraints on Solar System formation through probing subtle signatures that can indicate important processes. We refer to objects that exhibit rare behaviors as outliers, and there are two general kinds. Real-time outliers (category 2a) are objects whose properties change on short timescales. An example of this case is an asteroid that is found to be active. Population outliers (category 2b) are objects whose fundamental properties are unusual compared to the entire population. Examples here could include asteroids with very large lightcurve amplitudes; very short or long rotation periods; or asteroids with very unusual colors.

Studying outlier asteroids allows us to understand these rare events. As an example in category 2a, it is generally thought that most activity among asteroids is driven by sublimation of volatiles, but the occurrence rate of active asteroids is not well known (e.g., Jewitt et al. 2015). Measuring this rate from a uniform survey would improve our understanding of the volatile content of the asteroid belt and provide important new evidence for understanding the formation of the Solar System and the origin of life on Earth. (The LOOK project (Lister et al., 2022) has some goals and approaches that overlap with SNAPS, though LOOK has an overall greater emphasis on cometary science.) Thus, real-time outlier detections provide direct impact on these important outstanding science questions. Population outliers (category 2b) may also provide insight into the formation and history of the Solar System. For example, a small number of asteroids have very high amplitude lightcurves or very long rotation periods, both of which may indicate the need for significant internal strength (e.g., McNeill et al. 2018b and references therein). This too has implications for how asteroids formed, and how the Solar System formed.

ZTF does, and LSST will, broadcast alerts that report transient objects on the sky. These variable sources include supernovae, variable stars, and moving objects (and many other kinds of astrophysically variable objects). Moving objects are considered variable here because the brightness at a given sky location changes as a function of time, as an asteroid passes to and from that location. An alert includes measured data, some properties of the observed source, a postage stamp, and metadata, and alerts are distributed through a Kafka stream. LSST will broadcast alerts for around 10,000 variable sources detected in each 30 second visit¹¹1https://dmtn-102.lsst.io/DMTN-102.pdf. Moving objects may be 10% to 50% of all variable sources in a given LSST visit, depending primarily on ecliptic latitude. The ZTF data flow is about one tenth of this rate, with some 1000 alerts, and 100 asteroids, per visit, which corresponds to around 10,000 asteroids each night. The alerts (will) include measured properties of moving objects, including date/time; photometric properties (magnitude, uncertainty); a real/bogus score (likelihood that the source is real and is a point source); and properties of the PSF (e.g., elongation or extendedness). In the work described here, we focus only on known objects; in this case, the identity of the object is also transmitted with the measurement data. The orbital elements of each reported object are therefore known; in the work presented here, the orbital elements are not transmitted with the measurements and we obtain them through cross-referencing the Minor Planet Center’s catalogs.

At full scale, LSST will report measurements of $10^{5}$ – $10^{6}$ asteroids every night, for ten years. The total observational record, over ten years, will be a few billion unique measurements of more than 5 million unique asteroids. LSST is a carefully defined project, and key definitions for the LSST project — what LSST will and will not do — are clearly defined (see Ivezić et al. 2019 and many documents at the LSST Documentation Hub²²2https://www.lsst.io/). This data stream will enable science, but will not produce science. In order to carry out a wide range of deep scientific investigations, the detailed physical properties of these millions of asteroids must be derived outside of the LSST project.

This flood of data can only be handled with sophisticated automated software. The general approach to handling Solar System objects, or indeed any transient or time-varying science, is through brokers: software packages that automatically ingest the LSST (or ZTF) alert stream. Brokers have some intelligence and may have the ability to derive target properties, recognize outlier behavior, attach external measurements to create value-added data records, and issue further alerts. There are a number of brokers that are under development and that are oriented towards sidereal or astrophysical sources, or are general and will include some basic processing of Solar System observations³³3https://www.lsst.org/scientists/alert-brokers.

Individual asteroid measurements such as magnitude or position by themselves are generally not scientifically interesting (except for extendedness, as described below). However, for asteroids with enough measurements, additional properties can be derived, including Solar System absolute magnitude ($H$); photometric slope parameter ($G$); lightcurve period and amplitude; color; taxonomic classification, as estimated from color; and even albedo and diameter, as estimated from color (Ivezić & Ivezić, 2021).

In this paper we present SNAPS — the Solar System Notification Alert Processing System. SNAPS was designed from the outset to be a moving object broker and hence has a number of capabilities that specifically enable Solar System science. SNAPS has been designated an official downstream broker. Here we present, in Section 2, the SNAPS design and architecture and our technical work completed to date. In Section 3 we present SNAPShot1, our first data release. In Section 4 we demonstrate the accuracy of our automatically derived results. Section 5 presents several science results, among the many that will be enabled by our SNAPS catalog. In Section 6 we discuss the scalability of our approach to LSST operations, and Section 7 describes the next steps for SNAPS.

Element (a) is fully mature and is presented here. Element (b) is under development, and will be reported in a future paper. Element (c) is presently carried out, though in a manual, rather than automated or algorithmic, way. Element (d) is partially complete: the value-added database exists, and the public database portal and alert stream are under development.

Two timing requirements are imposed by the cadence of the sky surveys that are providing data to us: (1) Our data ingest and real-time processing cannot fall behind, which implies processing on average around 100 (ZTF) and 1000 (LSST) asteroids in the 30 second exposure times used by both surveys; and (2) Our population outlier detection scheme cannot fall behind, which means that all computations that are carried out during the following day must be completed between sunrise and sunset at the observatory.

For the first constraint, there are two key steps that must be carried out within a 30 second window: ingesting the data and identifying real-time outliers. Ingesting the data is fast ($<$1 second), and our present real-time outlier detection implementations — where we simply query observed properties such as elongation of the measured source — are also fast ($<$1 second). As we develop more sophisticated multi-dimensional outlier detection schemes we will ensure that the computational performance meets the requirements.

For the second constraint, we must compare the processing time for the entire database — or at least the subset that has changed since the previous day — to the available time, which could be as little as 10 hours during winter observing. For 10,000 objects, calculating all light curve related properties (color, period, amplitude) takes about 83 CPU-hours on a single AMD EPYC 7542 CPU core. The average number of asteroids observed in a full ZTF night may be about 10 times this amount, and LSST another factor of 10 greater. This computational need would take more than one day, if just a single CPU were available. Thus, our SNAPS pipelines run on NAU’s High Performance Computer (Monsoon) with multiple CPU/GPU nodes in order to complete the processing within a day. Almost all the 83 CPU hour effort is dedicated to the Lomb-Scargle periodogram processing. To address this computational load, we have created a GPU Lomb-Scargle implementation (Gowanlock et al., 2021) that is described in Section 6.2.1. With this GPU approach, we can readily complete the daytime processing within one day, using our project-specific node on Monsoon that has 2 AMD EPYC 7542 CPUs (32 cores each) and 4 Nvidia A100 GPUs.

To determine the accuracy of our derived physical properties we have created a population of synthetic asteroids; a detailed description of this synthetic asteroid population is given in the Appendix. Briefly, these asteroids are randomly assigned colors, lightcurve periods, lightcurve amplitudes, and several other properties from distributions that are similar to distributions in the literature (generally, from the Lightcurve Database (Warner et al., 2009)). These synthetic asteroids are “observed” — that is, a synthetic observational record is produced — with ZTF-like cadences and ZTF-like photometric errors. These datasets are then passed through our processing pipeline, which derives the various properties. Finally, we compare the derived properties to the input properties to assess the accuracy of our approaches.

Some results from analyzing this synthetic population are shown in Figures A.3–A.5. For our derived colors, around 60% of the solutions are within 10% of the assigned colors, and 90% of the solutions are within around 25% of the assigned colors. For our derived lightcurve amplitudes, around 60% of the solutions are within 10% of the assigned lightcurve amplitudes, and 90% of the solutions are within around 20% of the assigned amplitude. Finally, for our derived lightcurve periods, around 70% of the solutions are within 10% of the assigned periods (here we do not include aliases as “matching”), and 90% of the derived periods are within around a factor of 2. Based on this analysis we conclude that in general our pipeline is producing reliable results for our derived properties. However, as with any tool that carries out bulk processing of large datasets, any individual asteroid result may have a derived property that is not correct. The SNAPS database can readily be used for bulk analysis (i.e., distributions of periods or amplitudes), and solutions for most individual asteroids are correct. However, individual objects that are found to be unusual in our processing (example: unusually high amplitudes, or extreme colors, or long periods) should be examined carefully.

We compare our SNAPS results to other published results. Figure 15 shows our derived SNAPS H magnitudes (in $g$ and $r$ bands) compared to the $H$ values (taken to be $V$ band) in the current version of MPCORB.DAT from the Minor Planet Center. The correspondence is excellent, with $<$$H_{g}/H_{V}$$>$ = 1.03 and $<$$H_{r}/H_{V}$$>$ = 0.99 where $<>$ indicates the mean, for all objects, of the ratio of the absolute magnitudes. The standard deviation for both ratios is around 0.21 mag, presumably with components of around 0.1–0.15 mag, more or less, from each of the MPC and SNAPS catalogs – typical uncertainties for survey measurements. The offset between the two comparisons ($H_{g}$ and $H_{r}$ compared to $H_{V}$) simply reflects the mean color of all asteroids.

We compare our derived $g-r$ colors to those from SDSS MOC4 for objects that appear in both catalogs. We find $<$$(g-r)_{SNAPS}/(g-r)_{SDSS}$$>$ to be 0.98, with a standard deviation of 0.12 mag, again showing excellent agreement between our results and previously published results.

We compare our derived periods to values in the current version¹⁰¹⁰10https://www.minorplanet.info/php/lcdb.php of the Lightcurve Database (Warner et al., 2009). (Amplitudes cannot readily be compared since they may vary substantially from epoch to epoch due to changes in the pole orientation with respect to the observer.) There are 8748 objects that appear in both SNAPShot1 and the LCDB. Our period comparisons are shown in Figure 16. There are several clear signatures in this plot. (i) The 1:1 line where the two solutions agree is readily apparent. (ii) There are many SNAPS solutions at periods of 16 and 48 hours (see Figure 5). These are alias solutions that are products of the ZTF observing cadence and, in general, not the actual rotation periods for these asteroids, although some of these solutions are presumably correct (near where these alias distributions cross the 1:1 line). (iii) There is a line parallel to the 1:1 line, but shifted upward by a factor of 2 (in period space); this is a presumably another alias where SNAPS has derived a period that is twice that of the LCDB period. (iv) Several curved lines can be seen that together form a quasi-triangular shape with one vertex in the lower left. These are “pseudo-aliases” (VanderPlas, 2018) that result from interactions between the real period of an object and the strongest alias. These curves are defined by $1/P_{D}=\left((1/P_{R})\pm(1/$24\text{\,}\mathrm{h}$)\right)^{-1}$ where $P_{R}$ is the real period for the object and $P_{D}$ is the derived period (Ďurech et al., 2022).

Solutions in the LCDB are assigned quality codes, with “2” suggesting solutions that may be uncertain at the 30% level and “3-” or better indicating solutions that generally are reliable. Almost 97% of the targets that appear in both SNAPShot1 and the LCDB have LCDB quality codes of 3- or better. We find that around 60% of SNAPS periods are within 10% of the LCDB period given a quality code of 2 or better; for LCDB code of 3- or better, the 10% agreement rises to 67%. Of the remaining solutions, almost 20% are in the 16 and 48 hour aliases, and most of the rest are in the pseudo-aliases.

Figure 10 shows the derived rotation period and lightcurve amplitude for minor planets from SNAPS. There are two regions on this plot where the density of objects is interestingly low. The first is the relative lack of objects showing a long rotation period with a low amplitude. This is highly likely to be an observational bias as solving for these solutions with sparse data is difficult. The second, more interesting, effect can not simply be explained by observational biases: We observe a dearth of objects at short periods ($P<3$ h) with amplitudes larger than around 0.25 mag. This appears as an “empty triangle” in the upper left of Figure 10.

The “spin barrier” discussed in the context of asteroid rotation represents the critical spin rate at which a strengthless rubble pile of average density would undergo rotational fission (Pravec & Harris 2000). This effect will also be shape-dependent, as areas of the surface of an elongated body are much farther from the center of gravity of the object than would be the case for a more spherical object. It follows then that objects with shorter rotation periods would have a more stringent limit on elongation than an objects with longer periods.

To examine this in more detail we focus here on objects with estimated diameter $0.3<D<20$ km to ensure that the sub-population is likely to consist of primarily rubble piles (both smaller and larger objects may be substantially monolithic, though for different reasons). In SNAPShot1 there are 14,784 objects with $19.5>H>10.5$ (where we use these $H$ magnitudes to correspond approximately to 0.3 and 20 km, respectively) and $P<10$ h. Using the methodology of McNeill et al. (2018a), for each object we can derive a lower limit on the required strength needed to resist rotational fission.

Although we can carry out the strength modelling for every object in this subset, the derived cohesive strength is always a lower limit and for most objects the value yielded is 0 Pa. As a lower limit, 0 Pa does not provide any useful constraint, and we do not present these findings here. However, 1067 of the 14,784 objects in this subset require non-zero cohesive strength if they are indeed rubble piles held together only by self-gravity and the friction between constituent parts. This is a larger sample of strengths than has been previously determined. A histogram of these non-zero strengths is given in Figure 17. Note that for this calculation all objects are assumed to have typical S-type bulk density of $2500$ kg m^-3.

Figure 18 shows the periods and amplitudes for objects shown in Figure 10 that require non-zero strengths to resist rotational fission. The auxiliary axis shows the calculated (minimum) strength value. This result implies a boundary dependent on both period and amplitude beyond which internal cohesive strength is required. Since this boundary depends on size and object density it will not appear as a clear cut-off in a population of mixed size and taxonomy, as is seen in the SNAPS results in Figure 10 and in the derived strengths of objects in Figure 18.

We further explore this result with a synthetic population of objects having a range of periods and amplitudes, but a fixed diameter and bulk density of 3 km and 2500 kg m^-3 respectively. Figure 19 shows the boundary between the strength and strengthless regimes for objects of this size and density.

The conclusion of this analysis is as follows. Some rapidly rotating, low-amplitude SNAPS asteroids require strengths as large as 5000 Pa, but most minimum strengths are hundreds of Pascals (Figure 17). These values establish a lower limit on the strength within these bodies. The strengths required are sufficient that we must consider that there are mechanisms beyond simple friction between constituent pieces of the rubble pile preventing rotational fission. The values obtained are significantly lower than the internal strength of solid rock so we do not necessarily believe that this implies that all of these objects are monolithic in nature, as opposed to being rubble piles. Instead the strengths are similar to that of lunar regolith (Mitchell et al. 1972) so we consider the possibility that these objects may have some lunar-like cohesion caused by regolith on their surfaces. The lack of SNAPS objects with large lightcurve amplitudes and short periods is a real effect. Objects with these parameters — that is, in the “empty triangle” of Figure 10 — would require significant strength to maintain those elongations and rotation periods. The lack of objects in this area of parameter space indicates that indeed most asteroids have little or no strength.

Objects may be placed into this “empty triangle” corner of parameter space through (for example) collisions or the YORP effect, but unless they have sufficient internal strength they would soon shed material from their elongated ends, therefore increasing their rotation periods (through angular momentum loss) and decreasing their elongations. This would have the effect of moving those objects downward and to the right in Figure 10 — out of the “empty triangle” — until they cross the stability boundary implied in that figure and in Figure 19. Objects that appear to require significantly large strength — in other words, are found in or near the “empty triangle” — may be good targets to search for activity that is derived from mechanical mass loss.

McNeill et al. (2021) used sky-survey data from ATLAS to derive the mean shape distributions of the L4 and L5 Jupiter Trojan asteroids. They conclude that L4 asteroids are, on average, more elongated than L5 asteroids, but only if their period distributions are similar; the observations could be explained without any shape difference if the L5 asteroids rotate significantly more slowly than the L4 asteroids.

We can compare period distributions using our SNAPS catalog. There are 188 Jupiter Trojan asteroids that appear in SNAPShot1 (128 L4, 60 L5). Figure 20 shows the cumulative fractions for the period distributions for L4 and L5 Trojans. We find that the L4 Trojans are very slightly slower than the L5 Trojans (blue line slightly to the right of the red line). It is clear that the L5 Trojans are not significantly slower than the L4 Trojans (which would appear as a red line significantly to the right of the blue line), and in fact a K-S test between these two data sets indicates that the period distributions are not significantly different. Therefore, we rule out one of the potential alternative explanations given in McNeill et al. (2021), increasing the likelihood that the two clouds do indeed have different shape distributions and that the collisional interpretation in that paper is correct.

SNAPShot1, with more than 30,000 asteroids, contains a large number of asteroids that are not individually remarkable, but also a small number of objects that may be interesting. An exhaustive search for individual objects of interest is too cumbersome for any paper, or even any investigator; the data are available to enable a large number of people to carry out a wide range of science experiments. Here we present a few individual objects of note, to demonstrate several aspects of working with our database of derived properties.

Erasmus et al. (2021) present the discovery of very slowly rotating asteroids, with 39 objects each having periods greater than 1000 hours. This analysis was enabled by our SNAPS processing – in this case, our derived lightcurve periods. That paper presents the existence of such very slowly rotating objects, and estimated that very slow rotators must be at least 0.4% of all asteroids. In the current database we find that 633 asteroids (around 2%) have rotation periods greater than 1000 hours. We find 83 objects that have period solutions of exactly 5000 hours, our maximum allowed period, which could either indicate a true period that is longer than 5000 hours, or instead a low amplitude lightcurve; excluding these objects yields 550 very slowly rotation asteroids, or around 1.7% of the total sample. These very slow rotators would be extremely difficult to detect in classical lightcurve programs (one person, one telescope, one target at a time).

As an outgrowth of our SNAPS work, Navarro-Meza et al. (2021) showed that the derived size and shape distributions for asteroids near the detection limit of any large-scale sky survey may be biased due to non-zero lightcurve amplitudes. The catalogs produced from ZTF and LSST will need to be debiased in order to derive accurate distributions of these properties.

LSST will produce roughly an order of magnitude more data than ZTF. The two major computational tasks that need to be carried out are deriving properties of asteroids, and outlier detection to find outlying asteroids. As has been described in other ZTF processing pipelines for the detection of transient events, deriving periods is often the most computationally expensive task (Coughlin et al., 2021), and we address our solutions to this problem in Section 6.2.1. In Section 6.2.2 we briefly discuss what SNAPS will store in its database, and the computational resources available to SNAPS that will be used during the first few years of the survey.