Discovering Frequent Gradual Itemsets with Imprecise Data

[11] proposes the first interpretation of gradual dependency as a co-variation constraint and models gradual itemsets using statistical linear regression. So, linear regression is performed between the attribute pairs and the validity of the graduality is relied on the normalised mean squared error $R^{2}$, together with the slope of the regression line. The gradual tendencies are revealed for attribute pairs that are strongly correlated with a strong slope of the regression line.

[13] was the first using data mining methods through an adaptation of the Apriori algorithm to extract the gradual itemsets. The authors evaluate the support of a gradual itemset by considering the proportion of objects couples that verify the constraints expressed by all the gradual items in the itemset. In that work, the reported gradual tendencies are based on correlation between attributes. The authors consider correlation in terms of the rankings induced by the attributes and not in terms of their values. Due to memory complexity, the algorithm reported in [13] is limited to the extraction of gradual itemsets with maximum length equals to $3$.

In the same way [7] proposes a gradual pattern extraction approach based on a heuristic for computing an approximate support value of a gradual pattern. For a given gradual itemset, this approach removes the objects (conflicts set) which prevent the maximum number of objects in the database to be ordered according to the gradual itemset. In this approach, the author constructs the candidates by ordering the transactions (objects) according to the values of the attributes. For the generation of a candidate gradual pattern of size $2$ for example, the author will order the transactions in a direction of graduality (increasing or decreasing) of the first item then, will associate the second item and check if the order of the transactions is respected according to the values of the item. In the case where the order is not respected, the author proceeds by a deletion of the transactions (set of conflict) preventing this order. The support of a gradual pattern is considered here as the cardinality of the longest ordered list of transactions respecting the gradual itemset. The major limitation of this approach is that it is not complete. Indeed, the use of a heuristic implies that in some cases the frequency of a gradual pattern may be undervalued. This phenomenon is explained by the fact that the author makes choices each time a set of conflicts is encountered. However, whatever the policy adopted for the choice to be made, it may have a loss of gradual reasons, this being explained by the fact that when generating a gradual itemset of size $n$, it is impossible to predict the best choice that will have an impact on the generation of gradual itemsets of size $n+1$. One solution is to keep all possible choices.

To deal with the disadvantage revealed in approach [7], [6] proposes a more complete approach named GRITE (GRadual ITemset Extraction) to automatically extract gradual itemsets from large databases. The authors consider the same definition of the support of a gradual pattern proposed in [7] and proposes a new method based on precedence graphs. In this method, the data are represented by a graph whose nodes are defined as the objects of the transactional database and the links stand for the precedence relation derived from the attributes taken into account. The author adopts a binary representation of the graph by a matrix. The support of the gradual pattern considered is defined as the length of the longest path in the graph. This approach makes it possible to efficiently generate itemsets of size $n+1$ from gradual itemsets of size $n$.

In [20], the authors propose an algorithm that combines the principles of several existing approaches and benefits from efficient computational properties to extract frequent gradual itemsets. In fact, they consider the formulation proposed in [13], and propose an algorithm that take into account the binary structure used in [6]. To evaluate the support of a gradual itemset, the authors use the Kendall tau ranking correlation coefficient that computes the number of object pairs which are consistent or inconsistent in the database, to be in agreement with the considered gradual itemset.

In [23], an approach based on B-Trees and OWA (Ordered Weighted Aggregation) operator [24, 25] is proposed to mine data streams for gradual patterns. [26] proposes the relational gradual pattern concept, which enables to examine the correlations between attributes from a graduality point of view in multi-relational data.

Fuzzy gradual patterns are revisited in [27] for noisy data where it is often hardly possible to compare attribute values, either because the values are taken from noisy data, or because it is difficult to consider that a small difference between two values is meaningful. An example of a fuzzy gradual pattern could be expressed as “the closer the age of an employee to 46, the higher his/her income”.

Recently, in [5, 28], the authors proposed an approach to extract gradual patterns in temporal data with an application on paleoecological databases to grasp functional groupings of coevolution of paleoecological indicators that model the evolution of the biodiversity over time. [29] introduces a generic method for extracting and analyzing gradual patterns in spatial data at several levels of granularity. The authors apply their method on the Health data to measure potentially avoidable hospitalization related with both societal and financial issues in public policies. More recently, in [30] propose fuzzy temporal gradual patterns to integrate the fact that a temporal lag may exist between changes in some attributes and their impact on others. These fuzzy temporal gradual patterns allow to detect the cases of relevant correlations between the attributes of a database whose changes in the value one attribute causes a ripple effect on other attributes with respect to time. [31] is interested to extract gradual itemset from property graphs where the attributes of the gradual itemsets are information from the graph, and are retrieved from the graph nodes or relationships.

It corresponds to the step 1 of Algorithm 1. Depending on the domain knowledge, the expert can set it using his knowledge. In this work, one of the formulas (1), (2) and (3) are applied on to the data. For these formulas, we set the values of the two parameters $k_{1}$ and $k_{2}$ equal to 1 and 0 respectively. These values ($k_{1}=1$ and $k_{2}=0$) are considered as their default values.

For the algorithms GRITE [6] and T-GPatterns presented in [5], which first transform the numerical database in categorical one, this threshold is applied during the transformation as following:

1. 1.

   Case of T-GPatterns approach presented in [5]

   The database $\Delta^{\prime}=\cal{T^{\prime}}\times$ $\cal{I^{\prime}}$ ($|T^{\prime}|=n-1$ and $|I^{\prime}|=|I|$) resulting from the application of the $Num2Cat$ function on the numerical database $\Delta=\cal{T}\times$ $\cal{I}$, is calculated as following :

   1. (a)

      $\forall t_{j}^{\prime}\in T^{\prime},t^{\prime}_{j}[i_{k}]="+"\iff t_{j+1}[i_{k}]>t_{j}[i_{k}]+\sigma_{i_{k}}$

   2. (b)

      $\forall t_{j}^{\prime}\in T^{\prime},t^{\prime}_{j}[i_{k}]="-"\iff t_{j+1}[i_{k}]<t_{j}[i_{k}]-\sigma_{i_{k}}$

   3. (c)

      $t^{\prime}_{j}[i_{k}]="o"$ else

   This function allows to generate more symbols ”o” than the function Num2Cat of the approach [5](see Table 4.a and 4.b) because graduality is considered if and only if the difference between values of attributes exceed the threshold. Table 4 presents how the numerical database of Table 1 is transformed without gradualness threshold application, by application of the formulas (1), formula (3) and formula (2) respectively.

2. 2.

   Case of GRITE algorithm [6].
   The step ”binary matrices generation” of GRITE is modified by the threshold introduction. Let $t_{1}$ and $t_{2}$ be two tuples and $x$ an attribute. The binary matrices are calculated as following:

   1. (a)

      The matrix $M_{1}$ of $x^{\geq}$ : $M_{t_{1},t_{2}}=1$ iff $t_{2}.x-t_{1}.x\geq\sigma_{x}$ and 0 otherwise.

   2. (b)

      The matrix $M_{2}$ of $x^{\leq}$ : $M_{t_{1},t_{2}}=1$ iff $t_{2}.x-t_{1}.x\leq\sigma_{x}$ else 0.

   Like in the first case, the resulting matrices will be less dense than the matrix obtained with the GRITE algorithm. Table 5 shows how the binary matrix is obtained from the numerical database of Table 1 respectively without the gradualness threshold on the gradual item $a_{1}^{\geq}$, with the gradualness threshold on the gradual item $a_{1}^{\geq}$ and the gradual item $a_{2}^{\geq}$. As the computation of the support of a gradual itemset consists in computing the length of the longest path in the graph represented by such binary matrices. We can see from the binary matrices of Table 5 (c) and 5 (d) that introducing gradualness threshold allows to cup the paths in the graph represented by the binary matrices of Table 5 (a) and 5 (b).

   It comes out from Table 5 (c) ($M_{a_{1}^{\geq}}$ with $\sigma=0.18$) that no path reaches to node $t_{2}$ and no path begins at node $t_{6}$. This is not the case of the Table 5 (a) ($M_{a_{1}^{\geq}}$ without threshold). Thus, by introducing gradualness threshold, we considerably reduce the search space during the mining process. The search space is even more reduced between the Table 5 (e) ($M_{(a_{1}^{\geq},a_{2}^{\geq})}$) and Table 5 (f) ($M_{(a_{1}^{\geq},a_{2}^{\geq})}$) with the gradualness threshold $\sigma_{a_{1}},\sigma_{a_{2}}$). More interesting, the Table 5 (f) can even be reduced by removing object $t_{4}$ as it is isolated; its line and its column do not have any relation with other objects, it is meaningless in a gradual context. This allows to gain memory, and run-time, as deleted objects are not considered during future joins. On Table 5 (e), $t_{4}$ is deleted: all bits from the $t_{4}$ column and $t_{4}$ line are set to $0$. Table 5 (g) represents the final matrix. From Table 5 (g), it is easy to see that the support of the gradual itemset ($a_{1}^{\geq},a_{2}^{\geq}$) is equal to $2$ as no path begins at nodes $t_{6}$, $t_{7}$, $t_{8}$ and no path reaches the nodes $t_{1}$, $t_{2}$, $t_{3}$, there is no path containing more than two objects. From this observation, we can even further reduce Table 5 (g) by removing the lines $t_{6}$, $t_{7}$, $t_{8}$ and the columns $t_{1}$, $t_{2}$, $t_{3}$ to obtain Table 5 (h) with only four lines and three columns.

For T-GPatterns [5], the remainded step is the procedure $searchCoevolution(Apriori(\Delta^{\prime},minSupp))$. In our proposal, this step is done exactly as proposed by the authors. Introducing the threshold constraints in GRITE or Paraminer algorithm consists of changing the processing step where the binary matrix associated to each attribute is calculated. All other steps (initialize, AND operator and DeleteAloneTuples) are not changed.

This proposition is also true if $x\in X$ is replaced by $X^{\prime}\subset X$.

It is clear that after applying the graduality threshold, the support of some gradual itemsets decreases. So during exploration, this fact should be taken into account.

Property 1 allows to perform dataset reduction before mining process. Hence the whole dataset is not required to compute candidate patterns for a given minimum support and gradualness threshold. The search space containing $x$ and all its supersets can be discarded.

The first experiment compares the execution time, the memory usage and the number of frequent gradual patterns for GRAPGT and ParaMiner on non temporal databases Cancer and ParaMiner-Data (cf Table 6).

Fig. 1.a shows the number of frequent gradual patterns for ParaMiner algorithm (purple curve), GRAPGT CV algorithm (green curve), GRAPGT SD algorithm (blue curve), and GRAPGT ST algorithm (orange curve) on the cancer database when varying the support, with a reduced number of attributes. Figure 1.b shows the runtime of all these algorithms on the same cancer database when varying support, while Figure 1.c, shows the memory usage.

It should be like to remind you that GRAPGT CV, SD and ST are modified versions of the ParaMiner algorithm to which we have integrated different gradual thresholds (coefficient of variation, standard deviation and standard deviation of the deviations). The number of frequents gradual patterns results show that GRAPGT considerably reduces this number than ParaMiner when database have a small attributes : for a value of support threshold equal to $0.2$ it extracts $7$ frequent gradual patterns when the gradual threshold is defined by equation or formule (2), while ParaMiner extracts $28761$. This is a considerable advantage that should not be overlooked for the end user (the expert), because it is easier for an expert to analyze $7$ frequent gradual patterns at the expense of $28761$; in addition, frequent gradual patterns extracted with our approach have an additional information : they are gradual patterns with a strong graduality power. The execution time results show that GRAPGT is faster that Paraminer for handling this database with small attributes when the support threshold increases : for a value of support threshold equal to $0.1$ it answers in $330$ ms when coefficient of variation considered like gradual threshold, while Paraminer needs $3$ hours and $16$ minutes. GRAPGT remains better in memory used compared to ParaMiner. Most the support is great, less execution time, number and memory are great.

Fig. 2 shows the number of frequent gradual patterns extracted (Fig. 2.a), the execution time (Fig. 2.b) and the memory usage (Fig. 2.c) for previous four algorithms when varying the support on numerical database ParaMiner-Data. We can see that figures Fig. 2.a, 2.b and 2.c each have $3$ curves instead of $4$. This observation is due to the fact that the ParaMiner algorithm without gradual threshold does not run until the end (it crashes memory) on the ParaMiner-Data database containing $109$ transactions and $4413$ attributes. This database is more dense than the previous one, as the complexity lies in the number of attributes which determines the number of frequent gradual patterns (problem of combinatorial explosion).

However, when we introduce the graduality threshold, we manage to execute and extract a reduced number of frequent gradual patterns (e.g. $5$ when the coefficient of variation is considered as gradual threshold for a support equal to $0.1$). This result shows that our GRAPGT approach is scalable compared to ParaMiner because it gives a result where Paraminer fails. It goes without saying that the introduction of the graduality threshold considerably reduces the execution time, the memory usage and the number of frequent gradual patterns on databases with a small or large number of attributes ; in addition, these results show that it is easier for the end user or the expert to analyze the frequent gradual patterns extracted from approaches taking the gradual threshold into account than the approaches not taking this threshold into account.

The next experiment compares the execution time, the memory usage and the number of frequent gradual patterns for GRAPGT and T-GPatterns on temporal databases Paleo and Air quality (cf Table 6). It is good to remember that the choice to compare our approach with the approaches of extraction of frequent gradual patterns on numerical temporal databases is due to the fact that the latter aims at the same objective as us : extract useful and relevant informations from a numerical database that respect certain constraints of the application domain. Taking these constraints into account has a positive impact for users. It makes it possible to reduce the number of frequent gradual patterns facilitating their analysis by the users.

Thus, Fig 3.a shows the number of frequent gradual patterns for T-GPatterns algorithm (purple curve), GRAPGT CV algorithm (green curve), GRAPGT SD algorithm (blue curve), and GRAPGT ST algorithm (orange curve) on the paleoecology database when varying the support. In this Figure, when minimum support is equal to $0.1$, the number of frequent gradual patterns extract by T-GPatterns without take into account gradual threshold is equal to $41867$; this number is considerably reduce when the gradual threshold is taken into account : it is equal to $72$ when the gradual threshold is define by equation 1 (GRAPGT CV), $6$ when the gradual is defined by equation 2 (GRAPGT SD) and $6255$ when the gradual threshold is defined by equation 3 (GRAPGT ST). Similarly, when the support increases, the number of frequent gradual patterns extracted from T-GPatterns remains greater than the number extracted from GRAPGT CV, GRAPGT SD and GRAPGT ST. In this context, these results show that it is easier for an user or an expert to analyze the frequent gradual patterns extracted from approaches taking the graduality threshold into account than the approaches not taking this threshold into account.

Fig. 3.b shows the evolution of the extraction time of the frequent gradual patterns of each algorithm as a function of the variation of the support. Here the end users will also save time if they integrate the gradual threshold into the mining process; this is easily seen in this Fig. 3.b. Fig. 3.c shows the memory usage for these four algorithms when varying support. It is easy to see that our GRAPGT algorithm consumes less memory than the T-GPatterns algorithm.

Fig 4 shows the evolution of the number (Fig. 4.a), the execution time (Fig. 4.b) and the memory usage (Fig. 4.c) of extraction frequent gradual patterns as a function of the variation of the support threshold after application of the T-GPatterns [5] algorithm (purple curve), GRAPGT CV algorithm (green curve), GRAPGT SD algorithm (blue curve), and GRAPGT ST algorithm (orange curve) on the air quality database. Unlike the paleoecological database, this database has a large number of attributes ($9358$).

The objective here is to show the impact of the gradual threshold on databases with a large number of attributes. We can easily see that the curves of the graph are monotonically decreasing which means that the number of frequent gradual patterns decreases when the support increases. In addition, the introduction of the gradual threshold considerably reduces the extraction time and number of frequent gradual patterns.

In this section, we present a list of the consistent (cf. Definition 8) gradual patterns that we were able to extract in the different databases that we presented above. We recall that a gradual pattern is said to be consistent if it has a great power of variation (increase or decrease).