Detecting DeFi Securities Violations from Token Smart Contract Code

Before we explain DeFi in more detail, we define our system model. The Ethereum-based DeFi system model can be conceptualized as a five-layer system, consisting of a network layer, a blockchain consensus layer, a smart contract layer, the DeFi protocol layer, and an auxiliary services layer [21]. The network layer is concerned with communicating data across and within the various layers. It involves several elements, including network communication protocols and the Ethereum network; in particular, it includes communication among Ethereum peers/nodes. The consensus layer refers to the Ethereum blockchain’s consensus mechanism [21]. At the time of our research, this was still Proof-of-Work [22]. The consensus layer also encompasses nodes’ actions which rely on the consensus mechanism such as “data propagation”, “data verification”, executing transactions, and mining blocks. While the first two layers are implied in our research, this study primarily focuses on the smart contract layer. This includes the smart contract code that creates the ERC-20 tokens from which we derive our data set, and that creates the dApps that use them. The smart contract layer also includes transactions executed by smart contracts, the Ethereum Virtual Machine (EVM) state, and the state transition upon execution of DeFi transactions. The DeFi protocol layer refers to decentralized applications with which users interact, and the auxiliary service layer involves services that facilitate DeFi’s functioning, such as wallets and off-chain oracles. We describe these elements in more detail below and provide a visual representation thereof in Figure 1.

Ethereum functions as a distributed virtual machine and is the platform on which much of the DeFi ecosystem currently runs. This paper focuses on explaining the features of Ethereum which are most relevant to DeFi’s functioning.⁷⁷7For further details on Ethereum, see the Ethereum Yellow Paper [22].

In addition to holding balances, Ethereum accounts can store smart contract code and other information. A smart contract is a computer program that automatically carries out certain actions when specified conditions—such as payments—are met [3]. The smart contract code is immutable and publicly available on the blockchain. Smart contracts allow parties who do not trust one another to enter into contracts—rather than trusting each other or a third party to execute the contract, smart contracts ensure the terms will be executed as coded into the contract [1].

Before smart contracts are executed, they must be compiled into bytecode to be deployed and understood by the Ethereum Virtual Machine (EVM). Once compiled, Ethereum smart contracts take the form of a string of numbers (bytecode). The EVM is a stack-based environment⁸⁸8In stack-based programming, “all functions receive arguments from a numerical stack and return their result by pushing it on the stack.” These specific functions come from a set of pre-defined functions [23]., with a 256-bit stack size. It reads the bytecode as operational codes (opcodes) which are, essentially, a set of instructions (from a set of 144 possible instructions). The opcodes include actions like retrieving the address of the individual interacting with the contract, various mathematical operations, and storing information [24, 22].

Many DeFi projects also have associated tokens created by smart contracts, which either entitle holders to something within the dApp (analogous to a video game’s in-game currency) or serve as “governance tokens”. For example, UNI is the Uniswap decentralized exchange’s governance token [25]. Another example is the SCRT token which, in addition to being a governance token, is required to pay transaction fees on the Secret network [26]. Holders of governance tokens can vote on the future of projects; their voting power is proportional to the amount of governance tokens they hold. Most non-NFT DeFi tokens on Ethereum follow the ERC-20 (Ethereum Request for Comment) standard, which facilitates interoperability among projects. The ERC-20 standard allows for token capabilities like transferring among accounts, maintaining balances, and the supply of tokens, among others [27]. In this paper, we focus on ERC-20 tokens.

Developers create dApps which serve as the interfaces to execute these smart contracts. DeFi’s current core product offerings—including decentralized exchanges (dexes), lending products, prediction markets, insurance, and other financial products and services—are delivered through dApps [28]. Table 1 describes the primary products that make up the DeFi ecosystem.

Figure 2 depicts the process of dApp creation and execution through smart contracts, using Uniswap (a popular dex) as an example. As depicted in Figure 2, DeFi users must have a Web3 software wallet to hold DeFi tokens and interact with dApps. These wallets can be thought of as akin to a mobile banking application and exhibit similar features (sending transactions, showing balances, etc.). The difference is that, unlike with a banking application, users retain custody of their funds and can send transactions and execute other functions directly, rather than through an intermediary institution [29]. Using cryptographic digital signatures, users approve connections to their Web3 wallets, “sign in” to dApps, and approve interactions with the smart contracts on these platforms through their wallet.

An understanding of U.S. securities law is necessary prior to defining our DeFi threat model. DeFi has raised alarm in regulatory circles due to concerns over DeFi tokens’ potential conflict with existing U.S. securities laws [36]. U.S. securities are primarily governed at the federal level by the Securities Act and the Exchange Act, though the Sarbanes-Oxley Act, Trust Indenture Act, Investment Advisors Act, and Investment Company Act are also relevant. Enforcement of these laws is the responsibility of the Securities and Exchange Commission (SEC) and the Financial Industry Regulatory Authority (FINRA) [11].

The Securities Act relates to the offer and sale of securities. One of the key provisions which has been charged in cryptocurrency cases is Section 5, which requires the registration of the offer and sale of securities and stipulates specific provisions thereof [11].¹⁰¹⁰10For cryptocurrency case law involving the Securities Act, see [37, 38, 39]. Other sections detail the required registration information¹¹¹¹11Sections 7 and 10. and exemptions¹²¹²12Section 3, Section 4, Regulation S, Rule 144A, Regulation D, Rule 144, Rule 701, Section 28. [11]. Various SEC enforcement actions have successfully argued that ERC-20 tokens are securities (see, for example, [40]), arguing that they constitute investment contracts [41]. For further details on the application of the Howey Test (the SEC’s criteria for determining whether something is a security), as it relates to digital assets, see [41].

The Exchange Act specifies public companies’ reporting requirements and regulates securities trading through securities exchanges. It also handles securities fraud; under section 10(b) and 10b-5, fraud and manipulation in relation to buying or selling securities is illegal. One cannot make false or misleading statements (including by omission) in relation to the sale or purchase of securities, including those exempt from registration under the Securities Act. The registration of securities, securities exchanges, brokers, dealers, and analysts is covered in Sections 12 and 15 of the Exchange Act [11]. Section 12 regulates registration of initial public offerings, which is relevant for cryptocurrency initial coin offerings (ICOs).¹³¹³13For cryptocurrency case law involving the Exchange Act, see [37, 38, 39]. Finally, Section 13 relates to companies’ reporting obligations under the Exchange Act [11].

In practice, in addition to various registration and reporting violations, securities laws in the U.S. tend to cover the following fraudulent conduct: high yield investment programs, Ponzi schemes, pyramid schemes, advance fee fraud, foreign exchange scams, and embezzlement by brokers, among others [10].¹⁴¹⁴14For definitions of these offenses, see [13]. Financial frauds have been shown to impact financial stability, market integrity, and resource allocation and, in the case of cryptocurrency frauds, to have an impact on markets in traditional finance [18, 42].

We define our threat model in line with the U.S. securities laws described above. With that in mind, an “incident” is any activity that is in violation of these laws, such as failing to register a token as a security or an ICO or committing securities fraud. These actions may be the result of intentionally malicious behavior or naivete. Both cases “result in an unexpected financial loss” to users [21]. While we do not have an estimate of all of the losses individuals incurred as a result of DeFi securities violations, we know, for example that the Finiko ponzi scheme took $1.1 billion from victims in 2021 and rug pulls stole $2.8 billion worth of funds from victims in 2021 [43]. These incidents occur at the smart contract layer, the protocol layer, and the auxiliary layer of the DeFi system. In terms of the smart contract layer, this includes both the creation of the ERC-20 token itself (in the case of registration violations), as well as any malicious elements coded into the smart contracts such as Ponzi schemes, advance feed fraud, or certain types of exit scams. At the protocol level, the primary attack vector would be market manipulation [21]. Finally, at the auxiliary layer, we see both “operational vulnerability” (such as price oracle manipulation) and “information asymmetry” (such as smart contract honeypots) [21]. Information asymmetry is primarily at play in the case of securities fraud. Users are often unable to (or do not take the time to) analyze DeFi protocol smart contracts (and related security risks) prior to allowing them to utilize their assets. Users’ “understanding of a contract operation” is more likely to come from projects’ marketing materials than the contract source code itself [21].

The above threat model involves various assumptions. First, we assume that, based on the classification of the tokens used to construct our data set by the SEC, the DeFi tokens in question are securities under U.S. law. As discussed above, precedent has been established in this regard. Notably, this also means that many otherwise legitimate projects may be operating contrary to U.S. securities laws by virtue of the fact that they are not appropriately registered. The second assumption we make is that, in the case of frauds coded into the smart contract code, the developers of the DeFi tokens violating securities laws are behaving maliciously, rather than their violations being the result of errors. For this reason, patching or fixing smart contract code (as discussed in [44] and [45]) would not be a suitable way to allay this threat. Registration violations may result from either malicious intent or naivete.

Finally, we assume that prevention measures have failed in these instances and, therefore, the primary course of justice is detection and prosecution. While prevention is certainly preferable, it is not possible to prevent all crimes and, therefore, detection and prosecution remain an important way to remedy the threats discussed above. Figure 3 offers a visual representation of our threat model.

While the results of the studies described in Table 2 suggest that approaches of this nature can perform well for this task, a number of points of caution have also been raised. Literature concerning smart contract Ponzi scheme detection points to issues of overfitting, due to the imbalance of classifications in many data sets [49]. Studies have addressed this using the over- and under-sampling techniques [46, 49, 53, 54]. Other scholars [46] criticize the interpretability of results based on opcode features, i.e., why the presence of certain opcodes would point to criminality. Perhaps most fundamentally, however, these studies have given little consideration to whether machine learning techniques are actually necessary for this task; or, at least, superior to other, possibly simpler, approaches. While the methods applied undoubtedly show high performance, it remains possible that similar metrics could be achieved without recourse to these kinds of techniques.

Another issue with previous work is the repeated use of two particular data sets [48, 66]. Of the studies cited above, four use the Bartoletti et al. [66] data set [46, 50, 52, 51] and two use the Chen et al. [48] one [53, 54], while one study combined both data sets and added additional data [49]. While using the same data sets may be useful for comparing performance, it may be less useful in practice for combating fraud, with any shortcomings of these data sets having a polluting effect on the literature. Indeed, when manually inspecting the Bartoletti et al. [66] data set, one study [46] identified issues involving duplication and bias. Finally, though used to classify general fraud rather than Ponzi schemes, one article uses proprietary company data [59], which hinders reproducibility and evaluations of results.

The majority of existing related work uses smart contracts in general as opposed to ERC-20 token smart contracts (as [14] and we do). The only other work that specifically examines DeFi token smart contracts using machine learning is [14]. However, their work focuses on scam tokens in general (rather than securities violations) and on a single dApp (the dex Uniswap).

This paper seeks to fill these gaps by a) evaluating whether a machine learning approach is appropriate for identifying DeFi projects likely to be engaging in securities violations; b) examining securities violations more comprehensively (rather than just scam tokens or Ponzi schemes); c) investigating these violations across Ethereum-based DeFi, rather than specific sub-spaces, like decentralized exchanges; and d) examining the code-based features identified by our model in order to better explain its performance. We also develop an entirely new data set of violating and legitimate tokens.

To answer our question of whether it can be determined if a project may be engaging in securities violations from its token’s smart contract code using machine learning, we first required ground truth sets of both securities violations and legitimate tokens. One source of such information is provided by token lists compiled by DeFi projects or companies around particular themes. One function of token lists is to help combat token impersonation and scams; reputable token lists provide users with some assurance that the tokens that appear are not fraudulent. Uniswap posts lists which are contributed by projects in the community; users generally follow lists from projects they trust [67]. The lists contain information like project websites (important for avoiding phishing attempts), symbols, and their smart contract addresses.

Next, we used the Web3 Python package [74] to extract the bytecode for each of the tokens in our data set. We used an Infura node, which allows users to interface with the Ethereum blockchain through nodes the company runs, using their API¹⁷¹⁷17https://www.infura.io/. Our classification features come from the token smart contract bytecode we collected. We opted to use only code-based features (rather than transaction-based features, for example), following other recent studies which have achieved high levels of performance (including 100% precision and recall in [46]) in classifying Ethereum-based smart contract Ponzi schemes [46, 49, 15]. Furthermore, using only code-based features allows for classification as soon as smart contracts are deployed [51]—rather than waiting to examine the characteristics of associated transactions—and permits analysis of smart contracts with few transactions [46].

For this initial analysis, our aim was to keep the classifier as simple and as computationally inexpensive as possible. It is also the first classifier for Ethereum-based DeFi securities violations more broadly, so our aim was to obtain a baseline for this novel classification problem in order to determine if it is, in fact, suited to machine learning, rather than improving on the state of the art for previously addressed problems (as [49, 46] and others have done for smart contract Ponzi scheme classification).

EVM bytecode can be computationally “disassembled” into its corresponding opcodes. This process is illustrated in Figure 4. Following [51], we included a feature in our classifier for each opcode that appeared in our smart contracts, representing the frequency with which that opcode appeared in any given smart contract. We used the Pyevmasm Python package [75] to disassemble each contract’s bytecode into its equivalent opcodes and then used a counter to determine the number of times each opcode appeared in the contract.

We first used a random forest classifier to attempt to determine if a project was potentially engaging in securities violations. We chose a random forest classifier for the following reasons:

1. 1.

   Research involving data similar to ours achieved the best classification results with a random forest, compared with other classifiers ([14]; precision: 96.45%, recall: 96.79%, F1-score: 96.62%).

2. 2.

   While initial work on smart contract Ponzi schemes [51] has been optimized in later studies (for example, [49]), our goal is to achieve a baseline of performance for classifying Ethereum-based DeFi securities violations. Previous work [51] found the random forest algorithm performed the best on their data set, when compared with other standard classification algorithms (J48 decision tree and stochastic gradient descent).

3. 3.

   Given our primary goal of determining if machine learning methods are suitable for developing a classifier useful for law enforcement investigations, using a model with greater transparency and traceability is most informative.

Given the classification imbalance in our data, we used down-sampling of the majority class to balance it with the minority class. Specifically, we randomly sampled 47 smart contracts from the majority class (i.e., from the n=2,139 legitimate contracts) and ran a random forest classifier on the resulting balanced data set (i.e., 47 violations vs. 47 legitimate tokens). This procedure was repeated 100 times with different random samples, and we report the average performance of these 100 iterations.¹⁹¹⁹19Under-sampling, combined with properly executed cross-validation, performs well on highly imbalanced data sets [78]. While other, related work to ours [49] has used the Synthetic Minority Over-Sampling Technique (SMOTE) to train imbalanced data, we chose the more conservative under-sampling method. SMOTE combines majority class under-sampling and minority class over-sampling and synthesizes additional data for the minority class [79].

For each iteration, we used 70% of our data to train our model and 30% for our test set, following previous work in classifying smart contract Ponzi schemes [53]. We calculated accuracy, and weighted precision, recall, and F1-score to evaluate our model [80]. We calculated the means of these metrics across our 100 iterations to arrive at our final performance scores. We analyzed the average feature importance across the 100 iterations of our model and then built several subsequent models based on this information.

While, for the aforementioned reasons, we focused on random forest classification, in order to more comprehensively answer our first research question as to machine learning’s suitability for this classification task, we needed to build multiple kinds of models, including a simpler approach. For this reason, we also built logistic regression models from our data. As with our random forest classifier, we used down-sampling across 100 iterations, a 70%-30% train-test split, and calculated accuracy and weighted precision, recall, and F-1 score metrics. After analyzing feature importance, we constructed further models using different sets of features.²⁰²⁰20We did not build any more complex machine learning models, like neural networks, to answer our first research question because our data set is much smaller than those traditionally used to train deep learning models. Neural networks are much less interpretable than simpler machine learning models [81] and would therefore be less suitable for our purposes (where results could potentially be involved in legal proceedings) in any case.

To better understand the performance of our final model, we compared the frequencies with which the 10 opcodes from our final model occurred in each of our classes (violations and legitimate tokens). We conducted a t-test to assess whether the average frequencies were significantly different.

The findings from these comparisons are reported in Table 11, and support the analysis of feature importance in our final model. The mean frequencies of each of the features in our final model (reported in Table 11) were significantly different between the securities violations and legitimate token sets, with a much larger effect size for the most important feature (CALLDATASIZE) than for other features.

With the aim of better understanding our final model’s top three features, we randomly selected five contracts from our set of securities violations and five contracts from our set of legitimate tokens and analyzed their Solidity code. The contracts, the frequencies with which our model’s top three features occurred in their code, and the version of Solidity in which their code is written, can be found in Table 12.

We used Etherscan,²¹²¹21https://etherscan.io/ the Ethereum blockchain explorer, to obtain the Solidity code for each of these tokens. Next, we used Remix²²²²22https://remix.ethereum.org/—an Ethereum Integrated Development Environment that allows users to write, compile, deploy, and debug Ethereum-based smart contracts, including in virtual environments—to analyze the code. We compiled and deployed each smart contract on the Remix virtual machine.

We used Remix’s “debugger” tool to analyze the transactions deploying each section of the compiled contracts. The debugger tool allows a user to examine the opcodes for each transaction chronologically and highlights the corresponding line of Solidity code for each opcode (each line of Solidity code is compiled as several opcodes). It also gives information on the functions that the transaction is interacting with, the local Solidity variables, the Solidity state variables, and other information [82, 83]. However, given our goal of better understanding the features of our final classification model, our analysis focused on the opcode tool.

We examined all elements of the smart contracts involved in their deployment transactions. For each, each time one of our target opcodes appeared, we noted the specific aspect of the token smart contract and the corresponding line of Solidity code.

Though it is difficult to ascertain patterns which may be picked up by our classifier from visual examination of our code, we noticed that four of our five violating contracts (Dropil, Tierion Network Token, OpportyToken, and Boon Tech) had the same line of code which was resolved to the CALLVALUE opcode in the SafeMath portion of the smart contract: library SafeMath {. In our legitimate token smart contracts, the CALLVALUE opcode was only present in one of the five token contracts (the OST contract) in the SafeMath part of the contract. SafeMath is part of the OpenZeppelin smart contract development library, which allows developers to import standard, vetted, and audited Solidity code for, for example, ERC-20 tokens [84]. The SafeMath library, in particular, provides overflow checks for arithmetic operations in Solidity; arithmetic operations in Solidity “wrap” on overflow, which can lead to bugs [85]. SafeMath solves this issue by reverting transactions that result in operation overflows [85].²³²³23Notably, the SafeMath library was rendered superfluous by Solidity releases 0.8.0 and above (0.8.0 was released in December 2020 [86]) [87]. We initially hypothesized that violating tokens could utilize older versions of Solidity than legitimate ones due to the lengthy nature of the U.S. justice process. However, upon further inspection of the Solidity code for each token in our sample, they all rely on Solidity versions between 0.4.13 and 0.5.0 (as shown in Table 12), and all but Gladius include it in their code.

Additionally, when subsequently specifying the implementation of SafeMath for various arithmetic operations, the violating tokens use the “constant” function modifier, as opposed to the “pure” modifier used in the legitimate token smart contracts.²⁴²⁴24In later versions of Solidity, the “constant” modifier was changed to “view” [88]. These modifiers dictate whether or not a given function affects the Ethereum global state. The use of “constant” indicates that no data from the function is saved or modified, while “pure” adds the attribute that the function also does not read blockchain data [89]. Essentially, while both attributes specify that the function will not write to the Ethereum state, in the case of “pure”, the function also does not read state variables [90]. The “pure” attribute, being stricter about state modification, provides stronger assurance that arithmetic operations resulting in overflow will not (incorrectly) modify the contract’s state. It makes sense that the legitimate token contracts would provide this additional security and specificity.

We previously noted that each line of Solidity code in a smart contract resolves to multiple opcodes. Our smart contract analysis highlights this. In a previous iteration of our model, JUMPDEST was among the top most important features of our model. Various lines of the Solidity code, such as Contract BasicToken is ERC20Basic } involved both this opcode as well as CALLVALUE. However, we did not notice any distinctions in such lines of code between our violating and legitimate token classes.

CALLVALUE and LT were present numerous times in the aspects of the smart contracts we analyzed with the Remix debugger tool, though, based on our disassembly of these tokens’ bytecode, the full frequencies were not present in the portions of the smart contracts we analyzed. The CALLDATASIZE opcode was absent altogether. However, we were unable to execute other transactions in the Remix virtual environment in order to analyze the entirety of the smart contracts (though, we certainly explored a significant portion thereof for each). Furthermore, given our aim in using code-based features to develop a classifier which could be used immediately upon a token contract’s deployment, these are the aspects most relevant for our purposes.

Given our conclusions about the implementation of a particular library in the code being one distinguishing factor between our violating and legitimate token contracts, we sought to delve further into potential code reuse as a reason for our classifier’s performance. Prior studies have found that 96% of Ethereum smart contracts contain duplicative elements (though it is unclear if this is the case in the Ethereum-based DeFi ecosystem specifically)[91]. In that sense, if legitimate projects borrow code from other legitimate projects and projects violating securities laws borrow from other violating projects, smart contracts within each class would have a high degree of internal consistency.

Since ours is the first study to attempt to classify DeFi securities violations more broadly, we are unable to compare our model’s performance with previous studies. We do note that other studies which successfully built high performance classifiers for a related classification problem used more complex methods, which improved upon several previous studies addressing the same classification problem [46]. In addition, many previous studies, like ours, also used data with quite imbalanced classes, but they did not always account for this in building their models. Fan et al. [49], in particular, criticize previous work, including [51], on this basis. This may make the high performance metrics reported by some other studies slightly misleading. For example, only 3.6% of the smart contracts in the Chen et al. [48] data set are Ponzi schemes.

In contrast to previous studies, ours considers whether machine learning classification is necessary or whether a simpler model may suffice to solve the same problem. Previous studies have found that models built with, for example, logistic regression, have been much less effective than more complex ones [14]. We also found this to be the case.

Chen et al. [46] note the overall lack of interpretability of the results of classifiers with code-based features. The opcodes themselves have no obvious interpretation with respect to illegal activity, but, equally, there are not any opcodes which would offer a straightforward interpretation—i.e., there is no “STEAL” opcode or similar. The opcodes whose frequencies were among the most important features in our model were CALLDATASIZE, LT, CALLVALUE, SWAP3, EXP, CALLER, SHR, NUMBER, PUSH5, and ADDRESS. The only way to draw definitive conclusions from opcode-based features is to dissect the Solidity code from which they were compiled. Though it would have been impractical to dissect all 2,186 of the smart contracts in our data set, we gleaned some insight about our three most important features from a selection thereof, namely that developers of violating tokens may implement the SafeMath library differently in their code. In particular, they appear to use the “constant” modifier when describing how arithmetic operation overflows should be handled, which offers weaker assurances about the lack of state modification by these functions. It is, therefore, intuitive, for these reasons, that the use of the “pure” function would be associated with the legitimate tokens in this case. However, we note that, our analysis of transactions deploying the compiled Solidity code for certain contracts does not capture all of the opcodes whose frequencies were among the top 10 most important features in our model. Future research could also utilize other frameworks for analyzing token transfer behavior from token bytecode, such as TokenAware [94]. This would be particularly useful as well in instances where the contract Solidity code is not publicly available.

Like previous work [51, 46], we emphasize the usefulness of our classifier immediately upon deployment of the smart contract to the Ethereum blockchain and regardless of how many wallets interact with it. This is one of the key advantages of using code-only features for classification rather than transaction- or account-based features. This makes such a model not only useful as a retroactive tool for investigators, but also to prevent future fraud. It also enables investigators to monitor projects which may engage in securities violations in the future. Since investigations and prosecutions take such a long time (upwards of several years for complex cases), it is important for prosecutors to be able to begin gathering evidence as early as possible. However, we acknowledge that code-based analysis is merely one technique among many; future research could explore alternatives.

Our results point to the value of exploring the use of computational triage systems in the enforcement process. This is particularly important given that U.S. enforcement agencies seem to be relying heavily on submissions to their whistleblower programs [95]; a computational model could reduce reliance on whistleblowers and also avoid the government needing to pay out a portion of funds successfully recovered to whistleblowers (which can be millions of dollars [96]).

A classifier of the type we examine here would be more useful as a triage measure rather than a source of evidence due to issues surrounding the admissibility of machine learning-generated evidence in U.S. courts and also the risk for mis-classification. There may be questions about its admissibility under the Fifth Amendment, the Sixth Amendment, and the Federal Rules of Evidence, however legal scholars do not, ultimately, consider them impediments to its admission [97].²⁶²⁶26Though a complete discussion of the admissibility of machine learning evidence is outside of the scope of this paper, we provide a brief introduction here. The Fifth Amendment relates to an individual’s right to due process (this could arise in the context of the “black box” of machine learning calculations) and the Sixth Amendment includes the Confrontation Clause. This “black box” not only refers to inexplicable machine learning algorithms but also lay people’s likely lack of understanding about how these algorithms work. The Confrontation Clause requires experts to testify in person and submit to cross-examination. However, this is unlikely to be an issue, as the testimony of a machine learning expert should be satisfactory. The Federal Rules of Evidence around relevance, prejudice, and authenticity may be pertinent as well. Lawyers must further prove the accuracy of the evidence [98]. Some argue that under Rule 702 and Daubert v. Merrell Dow Pharmaceuticals, machine learning evidence would be admissible as expert testimony. Through Daubert, the court developed a set of four considerations for evaluating expert testimony [97]. Even if it is admissible, however, questions about its weight in court remain. In particular, explaining such evidence to a judge and jury (especially the “black box” calculations involved in developing machine learning models) may lead it being discounted. There is variation in levels of trust among jurors in machines in general and jurors must further trust the expert testimony which explains the machine learning tool [97]. This is exacerbated by the need for prosecutors to already explain complex concepts related to cryptocurrency in these cases. A machine learning-based tool would likely lead investigators to more compelling transaction-based evidence or qualitative evidence (for example, marketing material) which can be more easily understood by a judge and jury and which has been effective in prosecuting cryptocurrency-based financial criminal offenses in the past (See [99, 100]).

Considering the ambiguity around the Hinman standard²⁷²⁷27The “Hinman standard” refers to William Hinman’s 2018 speech which considered the level of decentralization of a project critical to determining whether it should be classed as a security [36]. in determining whether a project is sufficiently decentralized to avoid being classed as a security, a machine learning model could also serve as an additional tool in developers’ arsenal for determination thereof. Finally, we also perceive a machine learning model as potentially useful for people who are interested in participating in the DeFi ecosystem, as a way for them to research the validity of new projects to help protect themselves from fraud.

Our research may suffer from various limitations. The first is the potential for overfitting our model, particularly in the face of imbalanced data [49]. However, since we do not have a separate data set with known labels on which to test our model, overfitting could remain an issue despite our mitigation efforts. Ultimately, we chose a verified ground truth data set that was much smaller than our set of legitimate tokens. We do note that our data set may be making this classification problem simpler than it is in reality. We chose a list of generally reputable projects for our legitimate token set and those which are subject to government enforcement action for our securities violations. In this sense, we believe that, given the experimental nature of DeFi (and the high risk appetite of its participants), and the initial inclusion of a few violating tokens in the legitimate set, this set captures projects that exist in the middle, rather than at only extremes of offending and non-offending, which may be harder to classify. However, it would still be useful for future research to develop more data sets of DeFi securities violations to further test and refine our models, as necessary, using more advanced sequential machine learning techniques. As the number of verified violations increases, future research could also explore DeFi securities violations using more granular classes. These classes would show the different security patterns of the various types of securities violations and could further aid in prevention and detection efforts. It would also be useful for future research to compare the use of code-based features with models using account-based features or a combination thereof.

We note that, given the seeming importance of the implementation of the SafeMath library, in part, to our classifier’s performance, it may be the case that this classifier is less effective in classifying tokens created with Solidity versions 0.8.0 or later. However, this code did not account for all of our most important features. Future research could examine this, again, using an expanded data set.

We accept that there are some limitations around the use of a classifier like this in practice. Our future research will specifically explore how we could use a classifier like this to conduct an investigation and build a viable legal case. This will involve performing manual, in-depth analysis on tokens flagged when applying our classifier to other data sets and interactions with their smart contracts (similarly to Xia et al.’s [14] work on Uniswap scam tokens).

Chen et al. [46] raise the issue of bad actors using adversarial obfuscation methods to trick classifiers like the one we propose. We did not explicitly account for this possibility in building or model, nor did we test our model against known obfuscation techniques. However, this may be a useful avenue for future research to explore.

Further analysis of violating contracts, for example, using methods for analyzing token operational behavior from bytecode (such as TokenAware, which has successfully been applied to discrete instances of fraud [94]) would be fruitful. We encourage other scholars to pursue this line of research using our publicly available data set.

Finally, the jurisdictional focus of our research was limited in scope due to the nature of our data set. Legislation relating to cryptocurrencies is frequently changing; in particular, the European Union recently passed the Regulation on Markets in Cryptoassets (MiCA)[101]²⁸²⁸28Currently, the only implemented EU regulations that apply to cryptocurrencies relate to money laundering. The EU’s securities and investment regulations do not currently apply [102].. More generally, a global approach, incorporating legislation from multiple jurisdictions, would be a potential future goal. This would, of course, bring challenges, however, since the legality of particular applications—equivalent to the labels of training data, in technical terms—may vary across jurisdictions and change over time. Overcoming this challenge may require the adaptation of approaches from other domains.

One further prospect is that the output of a classification system such as this may be useful in detecting flaws or risks in novel DeFi functions. While the model would be trained on violations that had previously been detected, it is possible that cases may be identified as being risky even if they do not correspond to known flaws, but rather because they have underlying similarities to existing cases. In such a situation, manual inspection of cases identified as potentially fraudulent may offer insight into new forms of offending. This type of work would usefully contribute to the goal of preventing DeFi frauds and associated losses to individuals. Research on the prevention of such offenses is a crucial complement to the detection work presented in this paper.