Deep learning for quality control of surface physiographic fields using satellite Earth observations

To expand V15 and V20 lake description (to V15X and V20X respectively) their salinity and time variability information was generated. Even though static permanent water fits better to describe inland water distribution on average all year round, some areas (in Tropics especially) could benefit from having monthly varying information as they have a very strong seasonal cycle, when size, shape and depth of a lake changes over the course of the year, leading to a significant change in modelling the lake temperature response. Similarly, saline lakes behave very differently to fresh water lakes since increased salt concentrations affect the density, specific heat capacity, thermal conductivity, and turbidity, as well as evaporation rates, ice formation and ultimately the surface temperature. These two properties of time variability and salinity are often related; it is common for saline lakes to fill and dry out over the course of the season, which naturally also affects the relative saline concentration of the lake itself. To create a monthly varying lake cover first 12 monthly fractional land-sea masks based on JRC Monthly Water History v1.3 maps for 2010-2020 were created. Since the annual lake maps were created taking into account a lot of additional sources the extra condition on the monthly maps that the monthly water is equal or greater than permanent water distribution from fractional land-sea mask is enforced. To create an inland salt lake cover map, the GLDBv3 salt lake list was used. First, in order to identify separate lakes on $\sim$ 1km resolution lake cover (by “lake cover" we refer the maximum lake distribution based on 12 monthly-varying lake covers), small sub-grid lakes and large lake coasts are masked, i.e. grid-cells that have water fraction less than 0.25. Next, the number of connected grid-cells in each lake (i.e. connected with sides only) in the 1-km grid is computed. Then only lakes that have 100 and more connected grid-cells are vectorised ¹¹1By “vectorised” we mean that we transfer these groups of 100+ grid-cells into a shapefile layer, then we check if inside the shapefile falls a latitude/longitude point from GLDB saline lake list; if yes, this shapefile is kept in the layer, if not – it is deleted from the layer. We apply this shapefile layer as a mask to filter at 1 km resolution fractional grid-cells with lake cover. , as at ERA5 resolution of ∼31km the grid-cells are quite large and can include a mixture of freshwater and saline lakes. Finally, saline lake vectors are selected by filtering vectors which have no saline lake point from GLDBv3 located – in total 147 large salt lake vectors, which were further used to filter non-saline lakes at  1km resolution lake cover, finally aggregated to  31km resolution. In the future it is planned to revisit this field and extend the list to include additional data. Note that all non-lake related climate fields such as vegetation cover or orography were updated in V20 field set compared to V15 only in relation to the changing lake fields (i.e. if fraction of lake in the grid cell increased then other fractions like vegetation or bare ground should have decreased accordingly).

The Lake Updates category shows significant improvements in LST predictability if using V20 field set instead of V15 – prediction accuracy increased globally (over 1631 grid-cells) on average by 0.37K. For the lakes category, the training noise in V20 was generally small $\sigma_{\rm V20}\sim 0.02$ K, with the V15 predictions a little more noisy with $\sigma_{\rm V15}\sim 0.07$ K, but this noise is much less than the improvement - as can be seen in Fig. 7 every V20 iteration significantly outperforms every V15 iteration. In Fig. 8 we plot the distribution of the mean LST error (averaged across each of the 4 trained VESPER iterations) for all lake grid points, for both V15 and V20. Evidently the V20 field significantly improve the high tail behaviour relative to V15, as well as shifting the median of the distribution to lower errors. Particular regions where the V20 physiographic fields notably improved performance were in Australia and the Aral sea (e.g. Fig. 9). These are two major regions where ephemeral lakes were removed and inland water distribution made up-to-date, as discussed in Section 2.2.1. In addition to the areas with a notable improvement in the prediction accuracy, there are some noteworthy regions where the predictions got worse (see red points in Figure 9) suggesting inaccuracies or lack of information in the updated surface physiographic fields. A few of the most noteworthy grid-cells (see red points highlighted with green circles in Figure 9 and also Figure 11) are:

• •

  Northern India. This grid-cell lies in the state of Gujarat, India, close to the border with Pakistan. Here $\delta_{\rm V20}=+4.21$, with $\sigma_{\rm V15}=2.54$ and $\sigma_{\rm V20}=0.416$. The lake fraction was increased from 0.59 in V15 to 0.71 in V20 field set, along with the lake depth increase from 2.58m to 3.76m. However, this point lies on a river delta within the Great Raan of Kutch, a large area of salt marshes (see Figure LABEL:fig:lakes5), known for having highly seasonal rainfall, with frequent flooding during the monsoon season and a long dry season. The surface itself also undulates with areas of higher sandy ground known as medaks, with greater levels of vegetation. It is evidently a complex and highly time variable area and additional static fraction of fresh water provided via V20 field set is not sufficient.

• •

  Salt Lake City, North America. This grid-cell lies within the Great Salt Lake Desert, just to the west of the Great Salt Lake, Utah, US. Predictions of VESPER_V20 are worse than VESPER_V15, with $\delta_{\rm V20}=+2.91$ ($\sigma_{\rm V15}=0.26$ and $\sigma_{\rm V20}=0.92$). Whilst the training noise is significant here, it is less than the $\delta_{\rm V20}$ value, and we can see from Fig 11 that the VESPER_V20 predictions consistently underperform the VESPER_V15 predictions. The lake fraction was completely removed from over 0.50 in V15 to 0.00 in V20 field set, meaning that the grid-cell is fully covered with bare ground in V20 field set. Whilst this area primarily is bare ground, satellite imagery also suggests the presence of a presumably highly saline lake (see Figure LABEL:fig:lakes3); in addition area has a large degree of orography and high elevation (∼1300m) which probably further complicates the surface temperature response. A more accurate description that accounts for the seasonality of the surface water and the salinity is necessary here.

• •

  Tanzania. There are two grid-cells of interest at the centre and northern edge of Lake Natron, which itself lies to south-east of Lake Victoria, in Tanzania. For both these points VESPER_V20 predictions are less accurate than VESPER_V15; for the central point ($\delta_{\rm V20}=+2.45$,$\sigma_{\rm V15}=0.12$ and $\sigma_{\rm V20}=0.81$, see also Figure LABEL:fig:lakes1) the lake fraction was increased from 0.04 in V15 to 0.39 in V20 field set; for the northern edge point ($\delta_{\rm V20}=+1.57$,$\sigma_{\rm V15}=0.13$ and $\sigma_{\rm V20}=0.51$) the lake fraction was also increased in V20 comparing to V15 field set along with a small decrease ($\sim$0.1) in the low vegetation fraction. However, Lake Natron is a highly saline lake that often dries out, with high temperatures, high levels of evaporation and irregular rainfall. It is a highly complex and variable regime that is not well described by simply increasing the fraction of permanent fresh water, and indeed results suggest that with current lake parametrization scheme it may be beneficial to keep the lake fraction low or introduce extra descriptor, e.g. salinity.

• •

  Algeria. This grid point lies in Algeria, at the northern edge of the Chott Felrhir, an endorheic salt lake ($\delta_{\rm V20}=+2.20$, $\sigma_{\rm V15}=0.41$ and $\sigma_{\rm V20}=0.49$). Similar to the Great Salt Lake Desert, the lake fraction was completely removed from 0.33 in V15 to 0.0 in V20. However, Chott Felrhir goes through frequent periods of flooding where the lake is filled by multiple large wadi, and corresponding dry periods where the lake becomes a salt pan. As with the Great Salt Lake Desert it is also a highly variable, complex area that may require additional consideration of the salinity and the seasonality.

• •

  Lake Chad This grid point contains Lake Chad, a freshwater endorheic lake in the central part of the Sahel ($\delta_{\rm V20}=+1.74$, $\sigma_{\rm V15}=0.33$ and $\sigma_{\rm V20}=0.98$). Here the lake fraction was modestly reduced from 0.63 to 0.47. However, Lake Chad is again a highly time variable regime with seasonal droughts and wet seasons. It is a marshy wetland area but the vegetation fractions in both V15 and V20 here are zero. Satellite imagery also shows a large fraction of the surface covered by water and vegetation (Figure LABEL:fig:lakes11).

• •

  Al Fashaga This grid point lies in a disputed region between Sudan and Ethiopia called Al Fashaga, close to a tributary of the Nile ($\delta_{\rm V20}=+0.94$, $\sigma_{\rm V15}=0.14$ and $\sigma_{\rm V20}=0.29$). The updated V20 fields increased the lake fraction at this point from $0$ to $0.14$. The grid cell contains the Upper Atbara and Setit Dam Complex. However, the dam was only recently completed in 2018 - during the training period the damn was still under construction. Consequently whilst the V20 field may be more accurate at the current time, during the period the model was training the V15 field was more accurate, since the damn was not yet built.

• •

  Lake Tuz. This grid cell contains a large fraction of Lake Tuz as well as the smaller Lake Tersakan, saline lakes in central Turkey ($\delta_{\rm V20}=+0.85$,$\sigma_{\rm V15}=0.25$ and $\sigma_{\rm V20}=0.34$). Here the updated physiographic field effectively removed all lake water, with the lake fraction decreasing from 0.14 to 0.005. Whilst the lake is shallow and does dry out in the summer, there is also a large fraction of surface water present (e.g. Fig LABEL:fig:lakes44s) and it is an over correction to completely remove all lake water at this point.

• •

  Lake Urmia. This grid cell contains Lake Urmia, which is another saline lake in Iran ($\delta_{\rm V20}=+0.81$, $\sigma_{\rm V15}=0.12$ and $\sigma_{\rm V20}=0.73$). The updated physiographic fields decreased the lake fraction at this point from 0.77 to 0.39. This was in response to the shrinking of Lake Urmia due to long-timescale droughts and the damming of rivers in Iran. However, this drought broke in 2019 and Lake Urmia is now increasing in size again - satellite imagery now shows a large fraction of the grid cell covered by water (Figure LABEL:fig:lakes444s).

The Vegetation updates category, restricts analysis to grid points with significant change to high vegetation cover, where the high vegetation cover is substituted with either low vegetation or bare ground, and vice versa. For this category the prediction accuracy of V20 decreased globally (over 58 grid-cells only) on average by 0.04K. However, this shift is much smaller than the training noise between successive VESPER iterations ($\sigma_{\rm V15}=0.04$, $\sigma_{\rm V20}=0.15$) and so it is hard to make definitive statements about the performance of the updated vegetation physiographic fields as a whole (see e.g. Fig 13). The best we can say is that the updated V20 vegetation fields offer no global improvement in the LST prediction accuracy.

If we isolate our analysis to individual grid points where the training noise is small (highlighted by $*$ points in Fig 13) we can discern that there are multiple locations where the high vegetation fraction was decreased (often quite drastically to zero), specifying that there should just be bare ground, but thorough inspection of these areas with satellite imagery revealed that they should in fact be covered with high vegetation (see e.g. Figure 12) and that updating the V20 high vegetation cover was erroneous for these grid-cells. Moreover, for this subset of less noisy grid points, the strength of the drop in LST predictability in VESPER_V20 comparing to VESPER_V15 is approximately linearly dependent to the degree of reduction in high vegetation fraction, when the vegetation is replaced with bare ground (i.e. $\delta_{\rm V20}$ is maximally positive when the grid-cell that was fully covered with forest becomes fully covered with bare ground – high vegetation cover is reduced to zero). These erroneous grid-cells in V20 vegetation fields are likely to appear during the interpolation. The errors in these regions will in turn corrupt the LST predictions and mitigate the gain from a more accurate representation of the lake water. The majority of grid cells in this category (57/58) are modified in this way where the high vegetation fraction is severely reduced, however due to the large degree of training noise and the small number of points, it is difficult to draw any definitive conclusions for the category as a whole.

The Glacier Updates category in general shows improvement in LST predictability in VESPER_V20 comparing to VESPER_V15 (see Table 4) – prediction accuracy increases globally (over 1057 grid-cells) on average by 0.22K ($\sigma_{\rm V15}=0.03$, $\sigma_{\rm V20}=0.02$), most notably around the Himalayas, the land either side of the Davis strait, as well as British Columbia and the Alaskan Gulf. Analogous to the Lakes Updates category whilst the introduction of the V20 glacier cover generally improves LST predictions, there is a small selection of grid points where the prediction gets worse. These are heavily concentrated in the southern hemisphere, in particular on the south-western edge of South America and the South Shetland Islands (which lie closer to Antarctica), and some parts of the Himalayas. This deterioration in performance in these areas is not due to erroneous update of V20 glacier cover, but related to the Aqua-MODIS data (i.e. sparse availability due to clouds, and less certain due to orography, see Figure LABEL:fig:modis_plot_1). Consequently, VESPER finds it difficult to make accurate predictions in this region and for these points there is often a large degree of training noise, with considerable overlap between VESPER_V15 and VESPER_V20. If grid- cells with scarce amount of Aqua-MODIS observations (i.e. mean number of Aqua-MODIS observations per day over the year per ERA5 grid-cell is >50) are removed from the analysis then the worst performing grid-cells become excluded, yet a few areas where VESPER_V20 underperforms VESPER_V15 remain. For example, there is a grid-cell in Chilean Patagonia that contains the Calluqueo Glacier, close to Monte San Lorenzo where $\delta_{\rm V20}=2.49$ ($\sigma_{\rm V15}=0.38$, $\sigma_{\rm V20}=0.62$). This grid-cell has been updated in V20 field set comparing to V15 by strongly increasing glacier cover from 0.0 to 0.44), decreasing low vegetation cover (from 0.22 to 0.12) and high vegetation cover (from 0.16 to 0.09) as well as modestly decreasing lake cover (from 0.02 to 0.007). According to satellite imagery (see Figure LABEL:fig:glacier1) the glacier only occupies a small fraction of the overall grid-cell, and the updated glacier cover may have been an over correction. Moreover, this is an complex orographic area with snowy mountain peaks at high altitude and deep valleys, therefore the temperature response due to the glacier feature could be atypical compared to e.g. the Alaskan Gulf or the Davis straight. There is also substantial vegetation cover in the valleys that may not be being properly described. A similar point is in the Chilean Andes (see Figure LABEL:fig:glacier2), by the Juncal Glacier with $\delta_{\rm V20}=1.26$ ($\sigma_{\rm V15}=0.68$, $\sigma_{\rm V20}=0.29$). Here V20 glacier cover was increased to 0.25 compared to 0.00 in V15. Again, this is may have been an over correction, as the glacier constitutes only a small fraction of the grid cell. As with the Calluqueo Glacier this is also an area with lots of orography and so could have an atypical temperature response. For both of these points VESPER managed to identify potential inaccuracies in updated glacier cover, and once again proved itself as a useful tool for quality control of surface physiographic fields.

The Lake Updates category shows no significant difference in LST predictability globally when using the V20X field set instead of V20, with $\delta_{\rm V20X}=\delta_{\rm V20}=-0.37$ (comparable training noise). For the Lake-ground category, there is a modest increase, with $\delta_{\rm V20X}=-0.84$ compared to $\delta_{\rm V20}=-0.83$ but this is within the training noise.

For some of the problematic lake grid-cells highlighted in Table 5, the addition of saline maps and monthly lake maps does improve the LST predictability relative to VESPER_V20. For the Great Salt Lake Desert, Chott Fehlrir, Lake Chad and Lake Urmia, VESPER_V20X is a notable improvement over VESPER_V20, with $\delta_{\rm V20X}$ = $0.248,0.726,0.029,0.22$ respectively. The difference in $\delta_{\rm V20X}$ and $\delta_{\rm V20}$ for these points is greater than the training noise. If we take as a case example the grid point in the Great Salt Lake Desert, the improvement in using VESPER_V20X over VESPER_V20 is 2.667K $\pm 1.10$ K. At this point there is a strong correction from the monthly lake maps (mean value 0.16) and the salt maps (mean value 0.56). This improvement is to be expected given the known strong salinity and time variability in the region, and so it is a nice confirmation to have these updated fields verified by VESPER. It is also notable that the variation in the monthly lake maps at this point is very large, with a standard deviation in the lake fraction over 12 months of 0.18. At the start of the year the corrections from the monthly maps are very large, then as the year progresses the magnitude of the corrections generally decreases as the lake dries out. Such a large variation is again difficult to ever capture with a static field.

It is however notable that a) for all of the problematic lake points that we have discussed $\delta_{\rm V20X}$ is positive and b) there are multiple points (e.g. Gujarat province) where VESPER_20X exhibits no improvement over VESPER_V20 within training noise. Given all the extra information provided to the more advanced VESPER_20X model this is unusual; it suggests that either i) some of the additional information is erroneous in these regions, or else ii) the temperature response is atypical to the rest of the globe. For point ii), this means that the additional information is not predictive in these regions. Including this additional information in our neural network increases the complexity of the model which may in turn increase its training noise. This is likely the reason behind point b) - the updated fields are not sufficiently informative but do increase the training noise and so we see no improvement from using VESPER_V20X. For example, for Gujarat province $\sigma_{\rm V20}=0.416$, but $\sigma_{\rm V20X}=1.04$. In order to explore the hypothesis of point i) we train one further model, VESPER_V15X (again, see Table 3 for a summary of all VESPER models used in this work). This VESPER iteration is analogous to VESPER_V20X, being simply the VESPER_V15 model with the additional monthly maps and salt lake fields included. Importantly it does not have the updated physiographic correction fields from V20. Globally, this model performs worse that the V20 models, as we might expect - for example in the Lake Updates category $\delta_{\rm V15X}$ = −0.20 ($\sigma_{\rm V15X}=0.02$) compared to $\delta_{\rm V20}$= −0.37 K. However, VESPER_V15X does perform well at a number of the these problematic lake points (see Table 5). For 7 out of the 9 selected lake points, VESPER_V15X outperforms VESPER_V20X. For example in Gujarat province the improvement in using V15X over V20X is $6.5K\pm 1.53$. This suggests that our hypothesis for point i) is correct and that for some grid points the V20 fields are less accurate than the V15 fields. For a subset of points VESPER_V15X also outperforms VESPER_V15 (e.g. for Gujarat province $\delta_{\rm V15X}=-1.26$) but the difference is typically within or close to the training noise (e.g. for Gujarat $\sigma_{\rm V15X}=1.12$) and so it is hard to draw any strong conclusions. These examples illustrates again how VESPER can identify particular regions where the fields are inaccurate, as well as emphasising the need more generally for accurate descriptions of seasonally varying inland water and saline lake maps in Earth system modelling.

Whilst the Vegetation Updates category explicitly deals with areas where the lake fraction does not change when going from V15 to V20, many of the grid points in this category do contain some kind of waterbody, often lying close to the coast or else containing lakes or large rivers. Information on the salinity and temporal variability of these water bodies could then influence the prediction accuracy. By providing the additional information in VESPER_20X, the error relative to VESPER_V15 is reduced modestly to $-3\times 10^{-4}$ although as we saw before with the vegetation category the noise is large $\sigma_{\rm V20X}=0.21$ and so it is difficult to draw any further definitive conclusions. Similar arguments apply to VESPER_15X.

We would expect the additional information provided by the V20X fields to be particularly effective for glacial grid points. Glacier ice is naturally found next to waterbodies which freeze and thaw over the year, and the salinity of water will also influence this freezing. Therefore accurate additional information from the monthly lake maps and the saline maps should prove useful in these more time variable regions. We do observe a small improvement globally, with $\delta_{\rm V20X}=-0.28$ compared to $\delta_{\rm V20}=-0.22$, however this difference is comparable to the training noise $\sigma_{\rm V20X}=0.06$. This training noise could be slightly deceptive; 3 out of our 4 VESPER_V20X iterations outperform every VESPER_V20 iteration in the Glacier Updates category. The 4th VESPER_V20X iteration is somewhat anomalous - the increased network complexity could mean that the model did not converge well for that particular iteration, for the glacier grid points. Since the updated V20 glacier fields are generally accurate globally, we saw no particular improvement in using VESPER_V15X to within the training noise. This suggests that the additional monthly lake maps are only useful if the underlying representation of static water is sufficiently accurate. Considering the particular glacier grid points we discussed previously in Section 3.1.3, the additional monthly lake maps were particularly useful for the Calluqueo glacier, with $\delta_{\rm V20X}=0.32$ compared to $\delta_{\rm V20}=2.49$ ($\sigma_{\rm V20}=1.59$, $\sigma_{\rm V20X}=0.73$). However we saw no improvement to within the training noise for the Juncal glacier

Thus far we have been focusing mainly on the $\delta_{\rm VM}$ metric averaged over the entire year of the test set. It is also of interest to explore how the prediction error for each of the 3 models varies with time. This is demonstrated in Fig 15 for each of the 4 updated categories that we have discussed.

The Lake-Ground Updates timeseries broadly follows the same general profile as Lake Updates, but the errors are larger - those grid points where the lakes have been replaced with bare ground were particularly poorly described in V15. Additionally, for Lake Updates we see two sharp decreases in the prediction error during ∼ April and September, which are not as strongly reflected in Lake-Ground. This is due to the geographic location of the grid points in each of the two categories; for the Lake Updates category the grid points are located primarily in the boreal zones and so are subject to freezing and thawing over the course of the year leading to a strong seasonality due to the lake mixing that we have discussed. The sharp drop in April corresponds to a time where the lakes are unfrozen and fully mixed. However the lakes in the Lake-Ground sub-category are less concentrated and much more evenly distributed over the globe and so do not exhibit such a strong seasonality. Consistent with our previous discussion, the training noise makes it difficult to separate the predictions of the VESPER model for the vegetation category across the year. All generations of VESPER_VM follow the same general trend, with errors maximal at the start and end of the year, and minimal during the spring and autumn months.

For the Glacier updates category, in order to deal with the separate warming and cooling seasonal cycles over the year, we separate grid points into the northern and southern hemispheres. For the northern hemisphere the errors peak for all models in the summer, again due to the lakes not being fully mixed. There is also an uptick in the prediction error for all models during the winter when the freezing is greatest - this indicates how ice cover can strongly influence the LST response. The familiar hierarchy of models is recovered; VESPER_V15 is generally outperformed by the more updated models. In turn VESPER_V20X is a general improvement over VESPER_V20 throughout the year, especially during the winter months where the training noise is minimal. Since this is the time when freezing is greatest, this suggests that the additional monthly maps and salt lake maps are particularly useful during this time. For the southern hemisphere the story is different. The errors are smallest during the middle of the year when we expect the freezing to be greatest. During the spring and autumn the errors are largest - this is correlated with a decrease in the number of observations suggesting that this is due to poorer data quality due to cloud cover. In the summer when the weather is clearer the errors start to decrease again. Given this variation in the data quality due to cloud cover it is difficult to draw any strong conclusions, and again for stronger performance cloud independent data should be used. What is obvious for the southern hemisphere glacier grid points is that the VESPER_V20 and VESPER_V20X models struggle to improve on VESPER_V15, unlike in the northern hemisphere. This suggests that the updated V20 fields are still insufficiently accurate for southern latitudes.

We have also discussed previously particular grid points that will likely show a large degree of temporal variability, or the lakes are saline, and as a consequence the static physiographic V15/V20 fields struggle to make accurate predictions (e.g. Table 5). In Figure 16 we present timeseries for two of these points: the Great Salt Lake Desert, Utah and Chott Felrhir, Algeria. Both these points were discussed in Sections 3.1.1, 3.2.1. We can see that for these two selected points the hierarchy of models no longer holds. Whilst there is a large degree of variability, and there is no clear separation between models for some parts of the year, generally it can be seen that VESPER_V20 performs worse than VESPER_V15. For the Great Salt Lake the inaccuracy when using the V20 physiographic fields is most pronounced during the summer months. April, May and June are some of the wettest months in this region. But the updated V20 fields specify a much smaller lake fraction than in V15 ($\sim 0.5$ compared to $0.0$). Consequently during this time the V20 fields are maximally inaccurate and the prediction error of the VESPER_V20 model grows accordingly.This indicates again that the updated V20 fields are in fact over-corrections for this area. The inclusion of monthly lake maps and salt lake maps in VESPER_V20X notably reduces the error during these summer months. For Algeria, we can we can see that VESPER_V20 underperforms VESPER_V15 throughout the entire year. For this grid point the lake was completely removed when updating the V20 fields, with the lake fraction reducing from $\sim 0.35$ to 0.0. This also appears to have been an over-correction. The separation between the models is most pronounced in the early months of the year; in the winter months both the prediction error and the variance increase - this period is the wet season in Algeria where the wadi which feed Chott Felrhir fill up. Similar to the Great Salt Lake Desert, the inclusion of the monthly lake maps in VESPER_V20X improves the prediction accuracy, most notably in the early months of the year. Again, later in the year the training noise is much greater and so it is harder to separate the predictions of the model, but on average VESPER_V20X outperforms VESPER_V20 over the entire year, highlighting the value of these additional physiographic fields. monthly fields.