Mapping ”Brain Terrain” Regions on Mars using Deep Learning

Brain terrain is a geologically young terrain that has the potential to enhance our understanding on the role of water in the recent geological history of Mars. “Brain terrain” is a descriptive term given to a decameter-scale surface texture on Mars that consists of labyrinthine ridges and troughs occurring in flat terrains, typically in topographic lows. The areas are named for their resemblance to the human brain or aquatic brain coral species. It is found primarily at mid-latitudes and is often associated with lineated valley fill (LVF), concentric crater fill (CCF) and lobate debris aprons (LDA) (e.g., Squyres 1978; Malin & Edgett 2001; Carr 2001; Mangold 2003). This terrain type shares morphological similarities to sorted stone circles on Earth, which are thought to form as the result of numerous freeze/thaw cycles of rock-bearing soil (Noe Dobrea et al., 2007). In Earth’s arctic environments, sorted stone circles and labyrinths are the result of a rock-bearing and water-rich soil layer that undergoes heave/contraction cycles as the result of freezing-thaw processes (Taber 1929; Taber 1930; Williams et al. 1989) whereby rock-soil segregation can systematically occur (Konrad & Morgenstern 1980) and lead to pattern formation (e.g. Werner 1999; Kessler & Werner 2003). The convective kinematics of this process are reasonably well explored (Goldthwait 1976; Williams et al. 1989; Hallet 2013) through the analysis of trenches and seasonal data from tilt meters and other field studies (e.g. Hallet & Waddington 2020). However, while the morphologies of brain terrain and sorted stone circles are similar, similarity in form does not necessarily imply the same underlying process. Competing hypotheses argue brain terrain formed by sublimation lag, polygon inversion, or stone sorting by freeze-thaw (Mangold 2003; Noe Dobrea et al. 2007; Levy et al. 2009), but its origin remains unresolved. Multiple classes of young geomorphic features on Mars suggest that thawing may have occurred in the recent geologic past and may still be ongoing, although significant controversy remains McEwen et al. (2011). It is therefore important to compare available hypotheses (Mangold 2003, Milliken et al. 2003, Costard et al. 2002, Kreslavsky et al. 2008, Cheng et al. 2021, Hibbard et al. 2022) by performing a careful and detailed study of this terrain.

Sorted stone circles on Earth are thought to form from cyclic freezing and thawing in permafrost regions, which drives the convection of stones, soil, and water in the active layer (Mangold, 2005). In western Spitsbergen, the stone circles consist of a central 2-3 m wide plug of soil surrounded by a 0.5-1 m wide ring of stones, with the plug domed up to 0.1 m and the stones rising up to 0.5 m (Kessler et al., 2001). Though stones are often cm-scale, meter-sized clasts in circles up to 20 m across occur as well (Trombotto 2000; Balme et al. 2009). Comparatively, we measure brain terrain cells on Mars to be $\sim$5-15 m with ridges 0.2-2 m tall, resembling terrestrial circles in scale (Kessler et al. 2001; Kääb et al. 2014). On Mars, brain terrain is relatively young based on crater counts (Morgan et al. in prep), though rigorous analysis is still needed.

The Mars Reconnaissance Orbiter (MRO) has collected images of the Martian surface for over 15 years and amassed over 50 terabytes of data. The High Resolution Imaging Science Experiment (HiRISE; 0.3 m/pixel resolution; McEwen et al. 2007) and the Context Camera (CTX; 6 m/pixel resolution; Malin & Edgett 2001) are two instruments onboard MRO that are routinely used to study geological landforms. The total volume of data from this mission poses new challenges for the planetary remote-sensing community. For example, each image includes limited metadata about its content, and it is time-consuming to manually analyze each image by eye. Therefore, there is a need for computational techniques to search the HiRISE and CTX image databases and discover new content.

Many algorithms can classify image content, often requiring preprocessing like edge detection or histogram of oriented gradients (Dalal & Triggs, 2005). For pattern recognition tasks like object classification, convolutional neural networks (CNNs) have emerged as a powerful approach and can now match human performance in object recognition tasks making them an excellent choice for an algorithm capable of learning optimal features from training data (He et al. 2015; Ioffe & Szegedy 2015). CNNs are particularly well-suited for image classification tasks because of their ability to learn optimal filters or feature detectors from the training data itself, eliminating the need for hand-crafted features or preprocessing steps like edge detection or histogram of oriented gradients. For planetary mapping, CNNs outperform other classifiers like support vector machines (Palafox et al., 2017), and can be fine-tuned to identify landforms in Martian images (Wagstaff et al. 2018; Nagle-McNaughton et al. 2020). Other relevant approaches for planetary exploration include terrain segmentation to inform navigation (Dai et al., 2022) and caption generation for planetary images (Qiu et al., 2020). To further improve computational efficiency, especially when dealing with large image archives, recent work has explored leveraging the frequency domain representations of images. Leveraging frequency domain data saves significant time when processing large archives by avoiding full decoding. This enables timely surveys to inform landing site selection and identify rare features like potential astrobiology targets. Image compression relies on frequency domain transforms like discrete cosine transforms in JPEG to remove redundant data. Recently, deep learning approaches have utilized these compressed representations for efficient near state-of-the-art object classification (Russakovsky et al. 2015; Gueguen et al. 2018; Chamain & Ding 2019).

We are interested in automating the detection of brain terrain with images from MRO/HiRISE because their resemblance to terrestrial periglacial features makes them promising regions to further investigate in the search for potential life on Mars, both past and present. To expedite the processing we leverage a classifier network (prior to segmentation) in the Fourier domain that can take advantage of JPEG compression by leveraging blocks of coefficients from a discrete cosine transform in lieu of decoding the entire image at the full spatial resolution. The hybrid pipeline approach cuts down on the total processing time compared to running a segmentation network at the full resolution on every image. In the sections below we will discuss how the training data is made, what goes into each of the networks we tested, and then quantify the performance and accuracy of predicting brain terrain.

We analyze observations of the Martian surface using the High Resolution Imaging Science Experiment onboard the Mars Reconnaissance Orbiter (MRO) (McEwen et al., 2007). Images from the primary and extended mission were used and are publicly available online¹¹1https://hirise.lpl.arizona.edu/. HiRISE has a 0.5 m telescope along with an optical camera however for the purposes of our study we use images from the red channel which has a photometric range between $\sim$550–800 nm. The MRO spacecraft orbits Mars at an altitude of around 300 km and depending on the pointing, the HiRISE camera is capable of imaging the surface at resolutions between $\sim$25-50 cm/px. The camera reads swaths of data at a time tracking the ground in spans of 20,000 pixels and combined images are usually 100,000’s pixels long. The vast amount of image data is stored with 10-bit precision as a JPEG2000 image. The image encoding algorithm is capable of compressing images to $\sim$5-10$\%$ their original size, e.g., 10.8 GB of data at the full spatial resolution ($ESP\_036917\_2210$) compresses into as little as $\sim$675 MB of space on disk. Decoding images at the full-resolution can often take up to a few minutes on a CPU depending on the image size (see Figure 1). The HiRISE archive contains over 50 TB of compressed data spanning $\sim$85,000 images (as of writing this paper). However, for our study we used a local snap shot acquired in 2020 with 58,209 images. Processing terabytes worth of images requires spending a non-negligible time decoding the data alongside applying any computer vision algorithm afterward.

In order to automate the detection of landforms on the Martian surface we need a computer vision algorithm capable of learning the ideal features for detection. The advantage of a neural network is that it can be trained to identify subtle features (e.g. in pixel or elevation data) inherent in a large data set. This learning capability is accomplished by allowing the weights and biases to vary in such a way as to minimize the difference (i.e. the cross entropy) between the output of the neural network and the expected or desired value from the training data. Our training data is separated into two classes; a generic ‘background‘ class that represents most types of land and a ‘brain terrain‘ class for our terrain of interest. The training data is split in a 3:1 ratio in order to provide a diverse range of background samples. This skewed distribution prevents bias towards the majority class and provides diverse negative samples, enabling effective discrimination of brain terrain from visually similar terrains (e.g. aeolian ridges or deflationary terrain) while minimizing false positives. We tested different ratios and found a bigger ratio (more background) created a higher rate of false negatives while smaller ratios yielded more false positives, including a 1:1 split. Crucially, this imbalanced scenario simulates learning from limited labeled examples, a common challenge when studying rare phenomena, thus building robustness for deployment on new data where brain terrain occurrences are sparse yet scientifically compelling across the vast Martian surface. It prevents the model from becoming biased towards the overrepresented majority class and instead forces it to learn highly discriminative features for precise identification.

When we started this study, we only had two hand-labeled examples of brain terrain and used active learning to build a more robust set of training data. The initial two hand-labeled examples of brain terrain came from the images published in Noe Dobrea et al. 2007. We used those labeled regions to train an initial classifier to identify similar structures in other HiRISE images. We then manually vetted the classifier’s detections, adding any false positives to the background training set and any missed detections of true brain terrain to the positive training set. This active learning process of training, manually vetting, and adding to the training sets was iterated on to steadily improve the classifier’s performance. Each new image added to our brain terrain training data was masked manually to avoid biasing based on outputs from a preliminary algorithm. The full list of images used to generate training samples is listed in Table 1.

It is important to train on a diverse population of background images since it needs to generalize to the $\sim$60,000 images in the HiRISE archive snapshot that we looked at for this study. We trained multiple networks for our study and used the one with the best results. We found shrinking the input window size usually degrades performance below 128 pixels, therefore all of the classifier networks we tested have bigger window sizes (either 128, 256, or 512 pixels; see Table 2). The networks are trained on data between $\sim$25-50 cm/px and supports the conclusion of other papers stating due to the convolutions and pooling operations the network is partially scale invariant (Xu et al. 2014; Xu et al. 2019; Wimmer et al. 2023). Obviously, if the brain terrain feature is larger than the window size this conclusion breaks down and the network needs a larger input window. We generate training tiles using the map-projected satellite images and ignore portions of the image containing the black border. A subset of our brain terrain training samples can be seen in Figure 2.

A hybrid pipeline approach allows us to save processing time by evaluating data at a lower resolution while maintaining most of the accuracy compared to a full-resolution network. The hybrid approach starts with a classifier for quickly evaluating images followed by a segmentation network for full-resolution pixel-scale inference. Multiple classifiers were tested using different architectures and input sizes which leads to differences in accuracy and evaluation speed (see Table 2). We tested two types of classifiers, one in the Fourier domain in order to leverage blocks of coefficients from an encoding process similar to JPEG and a spatial classifier for comparison. Both classifiers work with data at 1/16 resolution. The spatial classifier uses a normalization layer in the network to transform the input to have a mean of 0 and a standard deviation of 1. The Fourier domain classifier uses the JPEG encoding processing to tile an input into 8x8 tiles and then applies a discrete cosine transform to each tile before being reshaped into a smaller but deeper block for the network (see Figure 3). The tiling and encoding process can transform an input image of 256x256 into a data cube of size 32x32x64. The smaller window size is advantageous in convolutional neural networks since the first layer of the network usually reduces the dimensionality in the number of channels from 64 to 32. Additionally, the smaller image size requires fewer operations per layer and speeds up inference whereas, with a similar architecture for the spatial data, the first layer increases the dimensionality of the input since there is only one channel pertaining to color instead of frequency. Early on in our study we started with random forests and multi-layer perceptrons but quickly found more benefit in using bigger input sizes which are better suited for convolutional networks. We found a negative correlation between the input size and the number of false positives with every algorithm we tested and ultimately settled on window sizes of 128x128 or 256x256, depending on the classifier. The advantage of using a smaller window size like 128x128 is that we can make more training data for a given set of HiRISE images (see Table 2).

While the classifier network demonstrated high accuracy and computational efficiency, it lacked the precision required for detailed surface measurements. Detecting brain terrain at sub-meter scales necessitated a network capable of operating at the native HiRISE image resolution ($\sim$0.25-0.5 m/pixel). To this end, we designed a custom U-net architecture based on MobileNetV3 to build a segmentation network for pixel-level predictions (Howard et al., 2019). The MobileNet architecture was chosen due to its state-of-the-art accuracy on the CiFAR1000 test while requiring less training time and computational resources compared to larger CNN architectures like ResNet and VGG (He et al., 2015). Despite its name implying mobile applications, MobileNet was designed for ARM-based processors, which are growing in popularity for space-based operations due to their energy efficiency (Dunkel et al., 2022).

The U-net architecture is widely adopted for segmentation tasks due to its ability to learn both image context and object boundaries (Ronneberger et al. 2015; Garcia-Garcia et al. 2017;). It consists of a contracting encoder path that applies convolutions to downsample the input image and extract relevant features, and an expanding decoder path that upsamples these features to produce a pixel-wise segmentation mask at the same resolution as the input. The encoder uses a series of convolutional and pooling layers to downsample the input image, extracting higher-level semantic features at deeper layers. The decoder then upsamples these feature maps to produce full-resolution segmentation masks. Crucially, the decoder combines high-level semantic features from the encoder’s deepest layers with higher-resolution features from earlier layers via skip connections. This allows the U-net to integrate local information (from early layers) with global context (from deep layers) to make accurate pixel-wise predictions. Compared to standard convolutional architectures like VGG (a very deep stack of 3x3 convolutional layers), the U-net’s decoder path and skip connections enable much more precise segmentation around the boundaries of objects/regions. This precise segmentation is highly beneficial for delineating the intricate patterns of brain terrain at a high spatial resolution. We chose MobileNetV3 as the core encoder architecture due to its efficient design achieving high accuracy at low computational cost, as highlighted in the Howard et al. 2019. This allowed fitting a high-resolution U-net within memory constraints.

Due to the size and complexity of the U-Net architecture, we found improved performance by training the network in multiple stages. First, the encoder portion of the U-Net was trained to predict low-resolution (32x32) segmentation masks from the input images (512x512). As shown by the jitter in testing accuracy (Figure 4), the encoder alone did not generalize adequately to the testing data. Next, the full U-Net was assembled by appending the decoder portion and fixing the encoder weights. Only the decoder was trained, and conditioned on upscaling the low-resolution encoder outputs to the native image resolution. Skip connections from select encoder layers to decoder layers of equal depth were utilized, as per the U-Net design. Since the base network architecture (MobileNet) is not a U-Net, determining which layers to use for skip connections required empirical tuning. The number of upsampling layers in the decoder constrained possible skip connection sources in the encoder. Finally, end-to-end fine-tuning of the full U-Net was performed using a learning rate an order of magnitude lower than the previous training stages. The Adam optimizer was used for training, with binary cross-entropy loss, as there were only two classes of interest. A comparison of the classifier and segmentation network is in Figure 5.

To evaluate the performance of the different machine learning architectures for automating the detection of brain terrain, we used the F1 score as the primary metric. The F1 score provides a holistic measure by combining precision (the fraction of detected regions that are true positives) and recall (the fraction of actual brain terrain regions successfully detected). It ranges from 0 to 1, with a value of 1 representing perfect precision and recall. As shown in Table 2, the unet-512-spatial model achieved the highest F1 score of 0.998 on the test dataset, indicating excellent overall accuracy in correctly identifying brain terrain while minimizing both false positive and false negative detections. The F1 metric enables evaluating the critical trade-off between reducing false alarms and avoiding missed detections for this classification task. Maximizing the F1 score ensures the automated mapping approach has high fidelity for subsequent scientific analysis of the spatial distribution and morphology of brain terrains.

The HiRISE camera on MRO has been acquiring data of the Martian surface for over a decade. We acquired a snapshot of the archive in 2021 which contains 58,209 images subtending over 28.1 TB of space on disk. We have a problem of needing to find a rather rare terrain over many images and we need pixel-level boundaries in order to accurate measure the spatial extent (see Figure 6). Figure 1 shows the amount of time it takes to decode a single HiRISE image based on the decoding resolution and image size. The estimated time to decode the entire archive at the native resolution is about 170.3 hours and when we combine that with segmenting the dataset we get another $\sim$ 145.4 hours. These numbers are based on interpolating the results of our algorithm based on image size and quantity. In total, the algorithm would take about 14 days to process the entire archive using the segmentation algorithm. Utilizing a hybrid pipeline approach with a classifier that operates on lower resolution data was proved to be $\sim$ 90$\%$ as accurate as our segmentation algorithm (see Table 2). Since the classifier is about 15 $\times$ faster than the segmentation network and we evaluate images at 1/16 resolution with it, we process the entire archive in $\sim$21 hours.

While our methodology is quick, it also needs to be trustworthy and accurate. It is too time consuming for us to manually evaluate every image that comes out of our pipeline however we have the capacity to inspect the positive detections since they’re less numerous. Even though both algorithms flagged a handful of images we manually vetted a portion of them to ensure we had at least some reliable results to follow-up. Our vetting process consisted of manually inspecting the full-resolution images with the segmentation mask for brain terrain overlaid. We manually inspected 456 positive detections and rejected 255 as false positives. We can use the rejections to estimate a false positive rate of $\sim$0.4$\%$ (255/58209) and that value is remarkably similar to our estimated value in Table 2 which comes from a more idealized dataset. This outcome highlights the importance of encapsulating a diverse range of images in the training data in order to minimize false positives and ensure accurate results. One caveat about training on false positives (see section 2 about our active learning process) is that they can sometimes adversely effect the accuracy if the features are similar to brain terrain and there aren’t many positive samples. We don’t have a good prescription on how to systematically test this other than inspecting the image embeddings or using some form of hold-out validation during training.

The implications of identifying the processes responsible for the relatively young brain terrains on Mars are significant. The confirmed brain terrain detections are centered primarily around 40N (see Figure 7). This preferential distribution in latitude can provide important clues about the climate conditions necessary for the formation of brain terrain on Mars. A majority of the detections in the southern hemisphere centered around 40S latitude and are classified as candidates (images flagged by both our classifier and segmentation network) because they require manual confirmation. Due to the numerous candidate detections we didn’t have time to check them by hand and are releasing an interactive table on GitHub to help with labelling the $\sim$1100 candidates²²2https://github.com/pearsonkyle/Mars-Brain-Coral-Network. A future study by Noe et al. (in prep.) will explore correlations between the brain terrain locations and climate models when Mars had a different obliquity. This preferential distribution suggests the formation of brain terrain may be correlated with specific climate conditions, such as those related to past changes in Mars’ obliquity. If brain terrain formed through freeze-thaw processes similar to patterned stone circles on Earth, it would imply that cryoturbation has redistributed rock near the Martian surface and that it is largely confined to a narrow latitudinal bands. Integrating multiple remote sensing datasets, including high-resolution imagery, crater counts, thermal data, and rock counts, can provide robust tests of the competing hypotheses for brain terrain formation.

We developed a hybrid pipeline using convolutional neural networks to automate the detection of rare brain terrain in a large dataset of 58,209 HiRISE images of Mars. A Fourier domain classifier on compressed imagery allowed rapid screening before pixel-level segmentation mapping of landform boundaries, enabling processing of the 28TB archive in just 21 hours rather than what would have been 14 days. We identify 201 new images containing brain terrain with an additional 1141 candidates that will require vetting by hand to be confirmed. An interactive table of the results is available on Github for the community to explore and build upon³³3https://github.com/pearsonkyle/Mars-Brain-Coral-Network. The confirmed brain terrain detections are centered primarily around 40N with another small group around 40S. The preferential distribution in latitude can correlate to certain climate conditions based on Mars’ history but more work is required to quantify the correlation exactly. If brain terrain formed by freeze-thaw cycles similar to the processes that create patterned stone circles on Earth (Kessler et al. 2001; Kääb et al. 2014), it would imply that freeze-thaw processes redistribute rocks near the martian surface and that this process is largely limited to a narrow latitudinal band. In a future study we will map rock distributions in HiRISE images using shadow measurements to derive size and abundance similar to Golombek et al. 2012; and Huertas et al. 2006. Quantifying rock distributions will help test hypotheses related to possible cryoturbation and freeze-thaw processes in brain terrain formation. Integrating multiple remote sensing datasets, such as high-resolution imagery, crater counts, thermal data, and rock counting from shadow analysis, can provide robust tests of the competing brain terrain formation hypotheses. This, in turn, can reveal valuable insights into Mars’ recent climate, geology, and potential habitability, which have important implications for future landing site selection and the study of these rare Martian landforms. (Gallagher et al., 2011)

The research described in this publication was carried out in part at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. This research has made use of the High Resolution Imaging Science Experiment on the Mars Reconnaissance Orbiter, under contract with the National Aeronautics and Space Administration. We acknowledge funding support from the National Aeronautics and Space Administration (NASA) Mars Data Analysis Program (MDAP) Grant Number NNH19ZDA001N.