GIFT: Guided and Interpretable Factorization for Tensors -
An Application to Large-Scale Multi-platform Cancer Analysis

We generate a gene group mask matrix $\mathbf{M}^{(2)}$ in a form of (gene - gene set) using the Hallmark data, which contain 50 important gene sets. Each column of mask matrix $\mathbf{M}^{(2)}$ corresponds to a gene set. If a gene $i_{n}$ is contained in a gene set $j_{n}$ then it is unmasked, i.e., $\mathbf{M}_{i_{n}j_{n}}^{(2)}$ is set to $0$; otherwise, the gene is masked, i.e., set to 1. If no prior-knowledge is known, the mask matrices are set to zero matrices with the size of corresponding factor matrices.

The PanCan12 dataset is represented as a 3-order tensor in a form of (patient - gene - platform; experiment value). Initially, the 4.7 version of the PanCan12 was downloaded from the Sage Bionetworks repository, Synapse (Omberg et al.,, 2013). The PanCan12 contains multi-platform data with mapped clinical information of patients group into cohorts of twelve cancer type: bladder urothelial carcinoma (BLCA), breast adenocarcinoma (BRCA), colon and rectal carcinoma (COAD, READ), glioblastoma multiforme (GBM), head and neck squamous cell carcinoma (HNSC), kidney renal clear cell carcinoma (KIRC), acute myeloid leukemia (LAML), lung adenocarcinoma (LUAD), lung squamous cell carcinoma (LUSC), ovarian serous carcinoma (OV), and uterine corpus endometrial carcinoma (UCEC). After download, probes of each platform are mapped to corresponding gene symbols. Then, subjects that have less than two evidence are removed from the dataset and genes that are not part of Hallmark set are removed. The resulting data for each platform are min-max normalized and is further normalized such that the Frobenius norm, i.e.,$\|A\|\equiv\sqrt{\sum_{i}\sum_{j}|a_{ij}|^{2}}$.

A tensor is a multi-dimensional array which is a generalization of a matrix and a vector. A mode or a way indicates each axis of a tensor, and an order is the number of modes or ways. We denote a tensor using boldface Euler script letters (e.g., $\boldsymbol{\mathscr{X}}$). A tensor $\boldsymbol{\mathscr{X}}\in\mathbb{R}^{I_{1}\times I_{2}\times\dots\times I_{N}}$ is an $N$-order tensor which has $N$ modes whose lengths are from $I_{1}$ to $I_{N}$. A vector and a matrix are regarded as a $1$- and a $2$-order tensor, respectively. We denote a matrix and a vector using boldface uppercase (e.g., $\mathbf{X}$) and lowercase letters (e.g., $\mathbf{x}$), respectively. The $i_{1}$th row of $\mathbf{A}$ is denoted by $\mathbf{a}_{i_{1}:}$, and the $i_{2}$th column of $\mathbf{A}$ is denoted by $\mathbf{a}_{:i_{2}}$.

Note that Equation (1) assumes missing entries of $\boldsymbol{\mathscr{X}}$ as zeros. Each column vector of a factor matrix generally represents each different concept. A higher value in a vector indicates that the corresponding element is highly related to the concept. Assuming a given tensor is movie rating data with (movie - user - time) triples, then a column vector in a movie-factor matrix can have a concept such as a horror or comic genre.

Many real-world tensors have missing values in them (i.e. partially observed). Applying standard Tucker factorization methods to the data triggers highly inaccurate results since they regard missing entries as zeros. Partially observable tensor factorization methods focus only on observed entries to tackle this problem, and a partially observable Tucker factorization is defined as follows.

Among many Tucker factorization methods (Smith and Karypis,, 2017; Oh et al., 2017a, ; Filipović and Jukić,, 2015), P-Tucker (Oh et al., 2017b, ) shows the best scalability and accuracy for partially observable tensors by focusing on observed entries of the tensors. The objective function of P-Tucker is the same to Equation (2), and P-Tucker uses a row-wise alternating least squares (ALS) to minimize the loss function. In detail, P-Tucker first chooses a row of a factor matrix to be updated while fixing all the others, and it computes three intermediate data $\delta_{(i_{1},...,i_{N})}^{(n)}$, $\mathbf{B}_{i_{n}}^{(n)}$, and $\mathbf{c}_{i_{n}:}^{(n)}$ defined as follows. Notice that $\mathbf{I}_{J_{n}}$ is a $J_{n}\times J_{n}$ identity matrix.

$\delta_{(i_{1},...,i_{N})}^{(n)}$ is a length ${J_{n}}$ vector whose $j$th entry is

$\mathbf{B}_{i_{n}}^{(n)}$ is a ${J_{n}\times J_{n}}$ matrix whose $(j_{1},j_{2})$th entry is

and $\mathbf{c}_{i_{n}:}^{(n)}$ is a length ${J_{n}}$ vector whose $j$th entry is

Using the above intermediate data, P-Tucker updates a row $a_{i_{n}:}^{(n)}$ by an update rule $\mathbf{c}_{i_{n}:}^{(n)}\times[\mathbf{B}_{i_{n}}^{(n)}+\lambda\mathbf{I}_{J_{n}}]^{-1}$. After updating factor matrices, P-Tucker calculates reconstruction error by the following rule.

If the error converges or the maximum iteration is reached, P-Tucker stops iterations and performs QR decompositions to orthogonalize factor matrices and update a core tensor accordingly. Note that (Oh et al., 2017b, ) suggests full details and proofs of the update process.

Although our previous method P-Tucker presents high scalability and accuracy, it is hard to interpret the results of P-Tucker since the factor matrices are dense and there is no direct interpretable connection between significant components, i.e. genes in each factor column that describes a group. Silenced-TF literally silences uninteresting or unnecessary parts of factor matrices and updates the rest using the same algorithm of P-Tucker.

More specifically, given a tensor $\boldsymbol{\mathscr{X}}\in\mathbb{R}^{I_{1}\times I_{2}\times\dots\times I_{N}}$ with observable entries $\Omega$, Silenced-TF of rank $(J_{1},\cdots,J_{N})$ finds a core tensor $\boldsymbol{\mathscr{G}}\in\mathbb{R}^{J_{1}\times\cdots\times J_{N}}$ and factor matrices $\mathbf{A}^{(1)}\in\mathbb{R}^{I_{1}\times J_{1}},\cdots,\mathbf{A}^{(N)}\in\mathbb{R}^{I_{N}\times J_{N}}$ which minimize the following objective function subjected to mask matrices $\mathbf{M}^{(1)}\in\mathbb{R}^{I_{1}\times J_{1}},\cdots,\mathbf{M}^{(N)}\in\mathbb{R}^{I_{N}\times J_{N}}$.

The difference in Silenced-TF compared to P-Tucker is in the selective updates of rows of factor matrices. To be more specific, given the mask matrices that encode the prior knowledge, Silenced-TF only updates an entry in the gene factor matrix, $a_{i_{n}j_{n}}^{(n)}$, when the corresponding masking element $m_{i_{n}j_{n}}^{(n)}$ is 0, and Silenced-TF sets the entry $a_{i_{n}j_{n}}^{(n)}$ in the gene factor matrix as a zero when $m_{i_{n}j_{n}}^{(n)}$ is 1. By masking, Silenced-TF offers interpretable gene factor matrices in addition to the benefits offered by P-Tucker: scalability and applicability for partially observed data.

Two potential weaknesses of Silenced-TF are low factorization accuracy due to many zeros in its factor matrices and inability to find genes that have a significant relationship to the gene group if they are specified as a member the gene group. To address these weaknesses, we propose GIFT which offers interpretability, high accuracy, and flexibility. GIFT tackles the problem by employing selective regularization of factor matrices. GIFT penalties proportional to $\lambda$ on masked entries of factor matrices during the update process. Thus, GIFT makes a distinction between the values of masked and unmasked genes for each gene group. Moreover, the accuracy of GIFT is similar to P-Tucker since masked entries can also have small values unlike the strict zero constraints of Silenced-TF.

GIFT encodes mask matrices $\mathbf{M}^{(n)}$ into its objective function as a regularization term. The specific loss function of GIFT is given by the following Equation (8).

The main difference between P-Tucker and GIFT is an existence of mask matrices $\mathbf{M}^{(n)}$ in a regularization term. GIFT uses $\mathbf{M}^{(n)}\ast\mathbf{A}^{(n)}$ instead of just $\mathbf{A}^{(n)}$, where $\ast$ denotes an element-wise multiplication. Through the specially-designed regularization, GIFT fully exploits prior knowledge encoded in $\mathbf{M}^{(n)}$. Compared to Silenced-TF, GIFT shows flexibility regarding the updates of masked entries. Instead of fixing them as zeros, GIFT imposes regularizations on them, which tend to make the values smaller, but not normally zeros.

Algorithm 1 describes how GIFT updates given factor matrices. When GIFT updates a row $a_{i_{n}:}^{(n)}$ (line 6), it requires a diagonal matrix $D$ which reflects masking information (line 5), while P-Tucker uses an identity matrix $\mathbf{I}_{J_{n}}$. The other parts of GIFT are the similar to that of P-Tucker. Table 1 shows a comparison of GIFT, Silenced-TF, and P-Tucker.

Given specific cancer type, which gene set is the most relevant to the cancer type? We first explain our discovery procedure and introduce several examples of (cancer - gene sets) relations found by GIFT.

We first compute an influence of each gene set on a patient and extract top-$k$ important gene sets for the patient. After that, we aggregate all top-$k$ gene sets of patients suffering from the given cancer and derive top-$k$ relevant gene sets to cancer by choosing top-$k$ frequent gene sets in aggregations. In detail, $\mathbf{a}_{i:}^{(1)}$ is a latent feature of $i$-th patient, and $\mathbf{G}=(\sum_{i=1}^{I_{3}}\boldsymbol{\mathscr{G}}_{::i})/I_{3}$ is a relation between gene sets and columns of a patient-factor matrix. Then, we use $\tilde{\mathbf{a}}_{i:}^{(1)}=\mathbf{a}_{i:}^{(1)}\mathbf{G}$ as an influence of each gene set on the $i$-th patient. The $j$-th element of $\tilde{\mathbf{a}}_{i:}^{(1)}$ indicates the influence of $j$-th gene set on the $i$-th patient. We extract top-$k$ most important gene sets for each patient by selecting top-$k$ highest values in $\tilde{\mathbf{a}}_{i:}^{(1)}$. Finally, we count the frequency of gene sets that appeared in the top-$k$ gene sets of all patients of a given cancer type. We regard the most frequent gene set as the most relevant one to the given cancer.

The first and second columns of Table 4 show (cancer - gene sets) relations discovered by GIFT. For breast cancer (BRCA), GIFT considers ‘Estrogen response late’ and ‘Bile acid metabolism’ gene sets closely related to breast cancer. It is well known that the estrogen plays a key role in the occurrence of breast cancer while the relation to ‘Bile acid metabolism’ gene set seems unnatural. However, (Murray et al.,, 1980) reveal that patients with breast cancer have significantly low fecal bile acid concentration than that of controlled patients. For ovarian cancer (OV), a relation to the ‘Interferon-gamma response’ gene set is supported by (Wall et al.,, 2003). They show that interferon-gamma causes apoptosis in human epithelial ovarian cancer. The ‘TGF beta signaling’ gene set is frequent among many types of cancer including Head and Neck Squamous Cell Carcinoma (HNSC), Lung adenocarcinoma (LUAD), Lung Squamous, Cell Carcinoma (LUSC), and Bladder carcinoma (BLCA). The reason is that the Transforming growth factor-$\beta$ (TGF-$\beta$) gene set is a tumor suppressor which affects many types of human cancers (Kretzschmar,, 2000).

Given a gene set, which genes are important or unimportant within it? Is there a gene not included but related to the gene set? A high value in the gene factor matrix indicates that the corresponding gene is highly related to the corresponding gene set. We sort the genes in each column of the gene factor matrix in descending order by their value and inspect high-value genes for each gene set.

We show how the discovered (gene sets - genes) relations are supported by biological facts with examples. The second and third columns of Table 4 show (gene sets - genes) relations retrieved by GIFT. The SKIL gene in the ‘TGF beta signaling’ gene set is identified to be important by GIFT. The SKIL gene encodes a protein which antagonizes TGF-$\beta$ signaling (Tecalco-Cruz et al.,, 2012). The PF4 gene in the ‘Angiogenesis’ gene set, reported to be important by GIFT, is known as an inhibitor of cell proliferation and angiogenesis (Bikfalvi,, 2004). The IRF7 gene in the ‘Interferon-gamma response’ gene set is also identified to be important, and the gene encodes interferon regulatory factor 7. The LPL gene in the ‘Angiogenesis’ gene set is unimportant according to the discovery result of GIFT. The LPL gene encodes lipoprotein lipase, an enzyme which hydrolyzes lipoprotein (Wang and Eckel,, 2009), thus it has low relatedness to angiogenesis. GIFT also reports the CASP8AP2 gene is closely related to the ‘Apoptosis’ gene set although the gene is not included in the gene set. The CASP8AP2 gene is associated with apoptosis of leukemic lymphoblasts in reality (Flotho et al.,, 2006).

Given specific cancer type, which genes affect the cancer type most? We suggest (cancer - genes) relations by combining two relations (cancer - gene sets) and (gene sets - genes) discovered by GIFT.

The first and third columns of Table 4 show (cancer - genes) relations found by GIFT. We regard gene sets in the second column of the table as bridges for (cancer - genes) relations. We deduce the IL17RB and TFF3 genes are significant to breast cancer since the genes are both important for the ‘Estrogen response late’ gene set and the gene set is the most relevant one to breast cancer. (Alinejad et al.,, 2017) showed that IL17RB is crucial in development and progression of breast cancer in effect. Moreover, (May and Westley,, 2015) reveal that the TFF3 gene promotes invasion and migration of breast cancer. GIFT also finds that the APOA1 gene in the ‘Bile acid metabolism’ gene set is highly related to breast cancer. High levels of APOA1 are known to be related to increased breast cancer risk (Martin et al.,, 2015). In the case of ovarian cancer, GIFT asserts a strong relation to the BST2 gene. High levels of BST2 have been identified in ovarian cancer (Shigematsu et al.,, 2017).