Data Synopses Management based on a Deep Learning Model

Pervasive computing targets to the creation of smart environments around end users saturated with computing and communication capabilities to support novel applications. Pervasive services aim to be invisible, however, intelligent enough to facilitate users activities. Today, we are witnessing the provision of huge infrastructures where pervasive applications can be hosted. Such infrastructures deal with the Internet of Things (IoT) and Edge Computing (EC). Both of them try to ‘surround’ end users with smart devices, collect and process data to create knowledge adopted by various applications. It becomes obvious that in this new era of the Web, there are numerous opportunities to support intelligent and invisible services in a close distance with users. Hence, we are able to reduce the latency in the provision of the discussed services and increase the performance. The first ‘actor’ in this setting is the IoT device that may directly interact with users and their environment to collect data and perform simple processing activities [34]. IoT devices can, then, report their data in an upwards mode, to the EC infrastructure and Cloud for further processing. EC involves an ecosystem of heterogeneous nodes that become the hosts of the collected data and act as processing points to deliver analytics and knowledge [34]. We can observe a high number of distributed datasets present at the network edge opening the room for defining advanced services and support real time applications. The aim is to serve users or applications requests in the minimum time with the maximum performance. As the maximum performance we denote the provision of responses that fully match to the incoming requests. Obviously, responses are provided upon the available data and should be aligned with them.

As the EC supports a distributed environment with numerous datasets present at the ecosystem of nodes, the use of cooperative models is imposed to make nodes capable of exchanging data, queries/tasks, etc. The interaction between EC nodes aims at detecting the appropriate line of actions to efficiently respond to the incoming requests for processing. The reaserch community has already focused not only on the management of queries/tasks [20], [21], [22], [23], [24] but also on the management of the collected data [1]. However, due to the distributed nature of the EC, nodes should have a view on the data present in peers especially when we want to support efficient decision making locally. For instance, a data allocation action demands for the knowledge of the remote data at least in the form of synopses. Data synopses can be exchanged between EC nodes without burdening the network as they usually convey high level statistical information about the available data. The delivery of synopses seems to be more efficient than the exchange of large pieces of data, i.e., data migration between nodes [9]. Actually, we have two solutions for responding to queries/tasks requests when the relevant data are note present at the node receiving the request. The first solution deals with the queries/tasks migration upon the decision of finding the most appropriate node as seen by the corresponding synopsis. The second solution deals with the migration of the relevant data from the owner/peer to the node receiving the request. Evidently, the former model should be supported by an intelligent mechanism for exchanging the necessary statistical information for the data present in the ecosystem while the latter scheme burdens the network with large messages increasing the possibility of bottlenecks.

In this paper, we support the autonomous nature and the cooperation between EC nodes to serve queries/tasks demanded by users or applications. We focus on the first of the aforementioned solutions (i.e., queries/tasks migration) and propose a scheme for exchanging data synopses in the ecosystem to efficiently support decisions related to offloading actions. Synopses are updated every time new data arrive in an EC node, however, they should distributed when their ‘magnitude’ exhibit that significant new information is present. We propose a monitoring mechanism for the updated synopses and a model that detects when significant changes are present at every dataset. When this is true, we decide to deliver the synopses to peer nodes to have them informed about the new status of every dataset. We rely on a Deep Machine Learning (DML) model to learn the distribution behind the calculated synopses being able to estimate their future trends. Hence, in a proactive manner, we are able to estimate the appropriate time for delivering the updated synopses. More specifically, we adopt a Long Short Term Memory (LSTM) network which is a specific type of Recurrent Neural Networks (RNNs) [18]. The adopted LSTM is capable of incorporating data from the previous step to the upcoming steps of processing. Hence, LSTMs are capable of identifying dependencies on data that ‘legacy’ neural networks cannot do. The detection of such dependencies are critical in our scenario as synopses are updated in an ‘incremental’ manner, i.e., new data arrivals are affecting the statistical information of datasets that is related to the previously delivered synopses. We consider the trade off between the frequency of synopses distribution and the ‘magnitude’ of updates. We can accept the limited freshness of updates for gaining benefits in the performance of the network. We also define an uncertainty driven model under the principles of Fuzzy Logic (FL) [35] to decide when an EC node should distribute the synopsis of its dataset. The uncertainty is related to the ‘threshold’ (upon the differences of the available data after getting reports from IoT devices) over which the node should disseminate the current synopsis. We monitor the ‘statistical significance’ of synopses updates before we decide to distributed them in the network. We consider the trade off between the frequency of the distribution and the ‘magnitude’ of updates. We can accept the limited freshness of updates for gaining benefits in the performance of the network. We apply our scheme upon past, historical observations (i.e., synopses updates) as well as upon future estimations. Both, the view on the past and the view on the future (derived by the proposed LSTM) are fed into our Type-2 FL System (T2FLS) to retrieve the Degree of Distribution (DoD). Two DoD values (upon historical values and future estimations) are smoothly aggregated through a geometrical mean function [32] to finally decide the dissemination action. Our contributions are summarized by the following list:

• •

  We support monitoring activities for detecting the magnitude of the updated synopses;

• •

  We deliver an LSTM for learning the distribution and dependencies on continuous updates of data synopses for estimating their future realizations;

• •

  We propose an uncertainty driven model for detecting the appropriate time to distribute data synopses to peers;

• •

  We report on the experimental evaluation of the proposed models through a large set of simulations.

The paper is organized as follows. Section II presents the related work while Section III formulates our problem and provides the main notations adopted in our model. In Section IV, we present the envisioned mechanism and explain its functionalities. In Section V, we deliver our experimental evaluation and conclude the paper in Section VI by presenting our future research directions.

Resource management at EC has been studied in the past to reveal the requirements for hosting and processing data. This is because data processing demands for specific resources according to the complexity of the requested queries/tasks. The appropriate allocation of the available resources will guarantee the increased performance and the timely provision of the outcomes. A number of efforts try to deal with the resource management problem [6], [13], [39], [43]. Their aim is to address the challenges on how we can offload various tasks/queries and data to EC nodes taking into consideration a set of constraints, e.g., time requirements, communication needs, nodes’ performance, the quality of the provided responses and so on and so forth. A relevant study in the domain reveals that processing nodes may adopt the following three (3) schemes to perform the execution of queries/tasks [43]:

• •

  An integration model for aggregating data reported by multiple devices [44]. EC nodes have the opportunity to locally process the collected data before they transfer them to the Cloud. This approach limits the time for the provision of the final response as the processing is performed in close distance with end users;

• •

  A ‘cooperative’ scheme though which EC nodes interact with other devices (e.g., IoT devices, EC nodes) having processing capabilities to offload a subset of tasks [45]. In any case, the distance between the interacting devices should be low, otherwise, their interaction may be problematic;

• •

  A ‘centralized’ approach where EC nodes act as execution points for queries/tasks offloaded by IoT devices [36]. This approach sees EC nodes having increased computational resources compared to IoT devices, thus, they can perform more complicated processing. Arguably, EC nodes should incorporated a monitoring mechanism to detect possible overloading cases and take specific mitigation actions.

Additionally, for speeding up the processing at the EC nodes while being aligned with the requirements of requests, various efforts have proposed the use of caching [10], context-aware web browsing [37] and pre-processing actions [40].

Evidently, queries/tasks are executed upon huge volumes of data. The processing of large scale data demands for efficient techniques to timely deliver the outcomes. The support of synopses management is already identified by the research community as a means for having a view upon the data avoiding to perform time consuming activities, Synopses convey statistical information about the underlying data [4] and can be maintained in an incremental approach. The research community has connected the term ‘synopsis’ with (i) approximate query estimation [11]: the target is to estimate, in real time, responses given the query and without having any view on data. Obviously, the final goal is to detect the data that better ‘match’ to the incoming queries; (ii) approximate join estimation [5], [16]: join operations are usually time concuming and more complex compared to other types of operations (e.g., a simple select upon the available data). Hence, approximate solutions may limit the time required to conclude any join operation taking into consideration the trade off between the accuracy of results and the conclusion time; (iii) aggregates calculation [12], [15], [17], [28]: the aim is to provide aggregate statistics over the available data; (iv) data mining schemes [2], [3], [38]: there are services demanding for synopses instead of the individual data points, e.g., clustering, classification. This means that any decision is retrieved upon the high level statistical information for the available data. In any case, the adoption of data synopses aims at the processing of only a subset of the actual data. Synopses act as ‘representatives’ of data and usually involve summarizations or the selection of a specific subset [27]. Any limited representation may heavily reduce the need for increased bandwidth of the network and can be easily transferred in the minimum possible time. Examples techniques for the delivery of synopses deal with sampling [27], load shedding [7], [41], sketching [8], [33] and micro cluster based summarization [2]. The easiest one is sampling. It targets to the probabilistic selection of a subset of the actual data. Obviously, the appropriate selection of samples plays a significant role in the success of the decision making model where samples are processed. Load shedding aims to drop some data when the system identifies a high load, thus, to avoid bottlenecks. Sketching involves the random projection of a subset of features that describe the data incorporating mechanisms for the vertical sampling of the stream. Micro clustering targets to the management of the multi-dimensional aspect of any data stream towards to the processing of the data evolution over time. Other statistical techniques are histograms and wavelets [4].

We focus on the ecosystem of EC nodes that exhibit cooperative behaviour towards the execution of the received queries/tasks. Without loss of generality, we assume $N$ EC nodes depicted by the set $\mathcal{N}=\left\{n_{1},n_{2},\ldots,n_{N}\right\}$. Every node hosts the corresponding dataset, thus, $N$ geo-distributed datasets are available as depicted by the following set $\mathcal{D}=\left\{D_{1},D_{2},\ldots,D_{N}\right\}$. Datasets contain multivariate vectors reported by the IoT devices being connected with EC nodes. $n_{i}$ hosts the reports of ‘its’ IoT devices and formulates a dataset $D_{i}=\{\mathbf{x}_{j}\}_{j=1}^{m_{j}}$ with $m_{j}$ real-valued contextual multidimensional data vectors. Each data vector $\mathbf{x}=[x_{1},x_{2},\ldots,x_{d}]^{\top}\in\mathbb{R}^{d}$ involves the data related to $d$ dimensions. For instance, IoT devices may monitor a phenomenon and report sensory data related to it (e.g., they could monitor a fire event and report data for temperature, humidity, etc). Any processing activity in $n_{i}$ is performed upon $D_{i}$ and targets to the provision of analytics or knowledge. For instance, requests may demand for a regression analysis, classification, the estimation of multivariate and/or uni-variate histograms per attribute, non-linear statistical dependencies between input attributes and an application-defined output attribute, clustering of the contextual vectors, etc. $D_{i}$s are also the basis for delivering the discussed synopses. Let us denote a statistical synopsis by $\mathbf{s}$. $\mathbf{s}$ is depicted by $l$-dimensional vectors, i.e., $\mathbf{s}=[s_{1},s_{2},\ldots,s_{l}]^{\top}\subset\mathbb{R}^{l}$. As mentioned above $\mathbf{s}_{i}$ delivered by $n_{i}$ is the summarization of $D_{i}$ upon the data reported by the corresponding IoT devices. Obviously, there are $N$ synopses $\mathbf{s}_{1},\ldots,\mathbf{s}_{N}$ that have to be distributed in the ecosystem of EC nodes.

Let us focus on the behaviour of a specific EC node $n_{i}$. A similar approach dictates the behaviour of all the available nodes. Initially, $n_{i}$ is responsible to calculate locally the corresponding synopsis $\mathbf{s}_{i}$ upon $D_{i}$. This happens when the received data change the statistics of the underlying dataset (e.g., concept drift). Afterwards, $n_{i}$ tries to act in a cooperative manner and decide to exchange $\mathbf{s}_{i}$ regularly. The target is to inform peers about the changes in the statistics of its dataset, thus, to give them the opportunity to be aligned with new trends in $D_{i}$. Obviously, $n_{i}$, before sending $\mathbf{s}_{i}$, should take into consideration the trade off between the communication overhead and the ‘freshness’ of $\mathbf{s}_{i}$ delivered to peers. $n_{i}$ can share up-to-date synopses every time a change (even the smallest one) in the underlying data is realized at the expense of flooding the network with numerous messages. We have to keep in mind that the connection of the IoT and EC infrastructures involves numerous devices exchanging numerous messages to convey data, synopses, knowledge, etc. Hence, the frequency of the delivery of messages plays a significant role in the performance of the network. In any case, a frequent delivery of $\mathbf{s}_{i}$ will give the opportunity to peers to enjoy fresh information increasing the performance of decision making. An intermediate solution is to postpone the delivery of $\mathbf{s}_{i}$ and reduce the sharing rate expecting less network overhead in light of ‘obsolete’ synopses. The delay in delivering $\mathbf{s}_{i}$ can be dictated by the limited updates in $\mathbf{s}_{i}$ as the result of retrieving data that cannot significantly change the statistics of $D_{i}$. In this paper, we rely on the second approach and propose a model that monitors $\mathbf{s}_{i}$ and detects where significant changes in the underlying data are present before it decides to deliver the updated $\mathbf{s}_{i}$. The target is to optimally limit the messaging overhead. Our rationale is to monitor the ‘magnitude’ of the collected statistical synopsis before we decide a dissemination action. In this approach, there are two main problems. The first is related to if past observations are the appropriate basis for initiating the delivery of $\mathbf{s}_{i}$ while the second deals with the uncertainty in the adopted threshold that will ‘fire’ the delivery action. Thresholds are set into any decision making mechanism that tries to detect the appropriate time to initiate an action. For the first problem, we proposed the use of an LSTM/RNN capable of learning the dependencies of data, thus it will be easy to retrieve their future estimates. For the second problem, we proposed the use of a T2FLS to handle the incorporated uncertainty, i.e., our T2FLS results the DoD upon past synopsis observations and its estimated values. The proposed T2FLS tries to bridge the ‘gap’ between past observations and future trends of synopses. In any case, EC nodes are forced to disseminate synopses at pre-defined intervals even if no delivery decision is the outcome from our model. We have to notice that, to avoid bottlenecks in the network, we consider the pre-defined intervals to differ among the group of EC nodes. This ‘simulates’ a load balancing approach avoiding to have too many EC nodes disseminating their synopses at the same time.

Our LSTM/RNN and T2FLS are fed by the most recent $\mathbf{s}_{i}$. The RRN retrieves future estimates of $\mathbf{s}_{i}$ that are also fed into the proposed T2FLS. To the bast of our knowledge, the proposed model is one of the first attempts that combines a DML with an FL system to deliver a powerful decision making mechanism. The LSTM/RNN undertakes the responsibility of learning the data and their dependencies through time and the T2FLS focuses on the management of uncertainty in decision making. $n_{i}$ monitors significant changes in $\mathbf{s}_{i}$ as more contextual data are received from IoT devices. Based on the local monitoring activity, implicitly, we incorporate into the network edge the necessary ‘randomness’ in the conclusion of the final decision, thus, potentially avoiding network flooding. The discussed ‘randomness’ is enhanced by different data arriving to the available nodes and their autonomous decision making. Such ‘randomness’ can assist in limiting the possibility of deciding the delivery of synopses at the same time, thus, we can limit the possibility of overloading the network. We consider that at $t$ (a discrete time instance) a new $\mathbf{x}$ arrives in $n_{i}$. Afterwards, the corresponding synopsis $\mathbf{s}^{t-1}_{i}$ should be updated to conclude the new $\mathbf{s}^{t}_{i}$. Let $\mathbf{e}_{t}$ be the difference over the current, last sent synopsis $\mathbf{s}^{t-1}_{i}$ and the new, the updated one, $\mathbf{s}^{t}_{i}$. We name this error as the update quantum, i.e., the magnitude of the difference between $\mathbf{s}^{t-1}_{i}$ and $\mathbf{s}^{t}_{i}$. $n_{i}$ calculates $\mathbf{e}_{t}$ at consecutive time steps and, in a simplistic way, can be concluded by adopting the sum of differences between two consecutive synopsis for every dimension. In any case, we can rely on any desired synopses realization technique. $\mathbf{e}_{t}$ may have a positive or a negative trend, i.e., the new vector can increase or decrease the value of each dimension. For easiness in our calculations, we take into consideration the absolute value of any difference into the available dimensions. EC nodes should delay the delivery of $\mathbf{s}^{t}_{i}$ until they see that a significant difference, i.e., a high magnitude depicted by $\mathbf{e}_{t}$ is true. At that time, it is necessary to have the peer nodes informed about the new status of the local dataset. We define the update epoch as the time between disseminating $\mathbf{s}^{t-1}_{i}$ and $\mathbf{s}^{t}_{i}$. The update epoch is realized at pre-defined intervals, $T,2T,3T,\ldots$ ($T>0$). In this description, we focus on a single interval, e.g., $[1,2,\ldots,T]$ where EC nodes check the last $\mathbf{e}$ realizations and feed them into our LSTM/RNN and T2FLS to see if they excuse the initiation of the dissemination process. For sure, the dissemination of $\mathbf{s}^{t}_{i}$ will be concluded at $T$ if no relevant decision is made by our scheme. $n_{i}$ also ‘reasons’ over the time series of update quanta $\left\{\mathbf{e}_{t}\right\}$ with $t=1,2,\ldots,T$. It ‘projects’ the time series to the future through the adoption of our LSTM/RNN. Again, the projection of update quanta is fed into the T2FLS to generate the DoD upon the future estimations of $\mathbf{e}$.

Data management at the edge of the network is a significant research subject due to the reduced latency that end users can enjoy if processing is performed at the EC infrastructure. Numerous IoT devices report data towards the Cloud datacenter, thus, advanced data management applications should be provided at the EC as the intermediate point where processing can take place. A number of EC nodes may undertake the responsibility of hosting datasets and processing activities. Nodes should act in a collaborative manner to increase their performance. For instance, EC nodes may exchange processing tasks or data to conclude the desired outcomes as soon as possible. In this paper, we enhance the collaborative aspect of the EC infrastructure and propose a novel model for exchanging data synopses at the edge of network. The target is to have all EC nodes informed about the data present at their peers, thus, to take optimal decisions related to the management of the requested processing activities. We present a deep learning model and an uncertainty driven scheme to reason over the appropriate time to exchange data synopses. The deep learning model manages to learn the distribution of the concluded synopsis as the basis for retrieving future estimations. The uncertainty driven scheme deals with a set of rules applied upon past synopses observations and future estimates. This way, we combine two completely different technologies to realize an efficient system for the management of data synopses at EC. Our aim is to provide a decision making methodology that minimizes the number of messages circulated in the network, however, without jeopardizing the freshness of the exchanged statistical information. We discuss our model adopting the principles of the FL and present the relevant formulations. EC nodes monitor their data and decide when it is the right time to deliver the current data synopsis. Our experimental evaluation shows that the proposed scheme can efficiently assist in the envisioned goals being evidenced by numerical results. In the first place of our future research plans, it is to incorporate a rewarding mechanism for every ‘correct’ decision and present a system that learns on how to learn. Additionally, we want to involve more parameters in the decision making mechanism like a ‘snapshot’ of the current status of every EC node.