ShadowGenes: Leveraging Recurring Patterns within Computational Graphs for Model Genealogy

ShadowGenes works for models that have been serialized using any format that includes a computational graph representation of the model, such as ONNX, TensorFlow, CoreML, OpenVino, or Caffe. ShadowGenes’ signature scanning technique only requires limited information from the computational graph and is completely independent of the underlying serialized model format or model framework. Therefore, the first step in the process is to create a format-agnostic computational graph that contains the essential information required for the subsequent steps. Three classes are used to represent the graph: Graph, Edge, and Node, which are shown in Appendix A.

While each model format stores its computational graph differently, when interacting with the serialized model file, development frameworks, such as ONNX and TensorFlow, typically expose an interface to access and extract the nodes and edges of the computational graph. This means that the agnostic graph can be constructed using the algorithm outlined below.

The pseudocode displayed in the above algorithm assumes that the names of the operations are unique because most computational graphs require unique names due to single static assignment ³³3https://github.com/onnx/onnx/blob/main/docs/IR.md; if that is not the case for a specific file format, the node lookup can be adjusted. In some formats, such as ONNX, some operations are stored under Graph.Input and Graph.Output rather than having all of the operations in the same place. If this is the case, the algorithm can be adjusted as needed to gather all of the operations required to construct the graph.

Once the agnostic graph is created, repeated subgraphs are extracted so signatures can be created for them, either programmatically or visually. Subgraphs are based on blocks of nodes rather than nodes themselves. Therefore, the first step of the process of extracting subgraphs is to extract the blocks from the graph.

A distinguishing pattern needs to be identified to build a signature for a specific purpose, such as identifying a model family or an overarching architecture. Analyzing a visualization of a model’s graphical representation can help achieve this.

Experimentation was carried out against models of the ONNX format, an intermediate representation that enables developers to work with different model formats in a standardized way.

The ONNX model zoo, hosted on GitHub, contains pre-trained models sorted into ‘classes’, which correspond to the concept of model families that have been used throughout this paper [7]. In total, there are 1,483 individual verified models across 104 model families in the ONNX model zoo. The signatures that we have created cover 101 of these families. ShadowGenes was tested on the models from these families to determine how well these signatures work. For this experiment, ShadowGenes only assigned each model to a single family.

Several classes defined within the ONNX model zoo contain multiple families. For example, ByobNet (Bring-your-own-Blocks) is used to implement multiple networks and, as a result, there are some models classified under ByobNet that were found to be of different families, including ResNet, RegNet, and GerNet. This had a negative impact on the efficacy of the signatures, in particular causing many false positives; ResNet signatures were generating false positives on ByobNet because of the ResNet models classified as ByobNet. This was remedied by leveraging the ONNX model zoo’s naming convention. Within these broader classes, such as EfficientNet and ByobNet, each model’s filename begins with a more specific descriptor of that model’s ‘sub-class’. For example, a model from the ByobNet class might have a filename beginning with ‘resnet’. We updated each model’s family label according to this naming convention, greatly improving detection accuracy.

Of the 104 model families, signatures were created for 101 of them. These signatures accurately identified 96.42% of the 1,483 models, returned a mean true positive percentage of 97.49%, with a precision of 99.51% and a false positive rate of 0.47%. A further breakdown of these results is shown in the table below.

Whilst the results are positive and demonstrative of a legitimate genealogy methodology, future work will be done to further improve the true positive rate, and test against a broader dataset to further validate the current signature database. That being said, the simplicity of the signature building process means incorporating new models and architectural changes would be easily achievable.

In early testing, ResNet models were producing too many false negatives. Results from our analysis indicated that this was because of a slight architectural difference between smaller and larger models. ResNet models can be defined by the number of convolutional layers they contain. Examples include ResNet18, Resnet50, and Resnet152. A typical, standard ResNet model with greater than 50 such layers is depicted in Figure 5. Referring back to this figure to explain: Block 2 is a linear flow consisting of three Convolutional operators and two Relu operators. In those with less than 50, such as ResNet18, Blocks 1 and 3 remain the same between the models, but ResNet18 contains only two Convolutional operators and one Relu operator in Block 2. In order to identify this subgraph with either of these linear flows within a model, the ‘*’ wildcard value was used within the second node of the signature, greatly reducing the false negative rate.

When building the signatures, it became apparent that those constructed for a specific model family would successfully identify a model compliant with ONNX opsets 17 and 18 but would result in false negatives for those compliant with ONNX opset 16 and below. This was particularly noticeable in models using transformer architecture. Upon further analysis, it was found that this was because the LayerNormalization ONNX operator was not introduced until opset 17, meaning the graph representations of opset 16 models had the LayerNormalization operator broken down into a distinct subgraph containing multiple blocks and operators, as shown in Figure 8.

Both images in Figure 8 represent the flow from the embeddings layer into the first encoder layer’s attention mechanism in a BERT model, the significant difference being that the LayerNormalization operator pictured on the right (opset 17) is broken down into the operators highlighted in grey on the left (opset 16). Identifying and adjusting for this difference significantly improved ShadowGenes’ accuracy, with the true positive rate increasing by fifty percent in the cases of several model classes.

Refining signatures to target a specific model family based on a single connected subgraph is not always possible, primarily because of architectural similarity between related models. To overcome this, ShadowGenes leverages what we refer to as ‘Combo’ signatures.

Most models contain sub-components that are individually present in several model families but only observed together in one specific model. A ‘Combo’ signature is a broader signature constructed to identify one sub-component, which, if used alone, would generate a high number of false positives. However, implementing multiple ‘Combo’ signatures as one enables us to target specific model families or architectures based on the unique make up of their disparate sub-components.

By way of example, the signature for the VisionTransformer model family is constructed of four ‘Combo’ signatures and will therefore trigger when this combination of its more generic sub-components are identified within the model. Two of these are highlighted in Figure 9, and the remaining two are shown in Figures 10 and 11.

The next ‘Combo’ signature used in the VisionTransformer model identifies the activation function subsection contained within a given layer and is shown in Figure 10 below.

The final sub-component used in the VisionTransformer model signature is one linear subgraph that leads to the output and is shown in Figure 11 below:

As stated, these subgraphs are commonly seen within multiple model families, but combining them allows us to create targeted signatures from sub-components only seen together in a particular model. A second important benefit of ‘Combo’ signatures is that they can be reused as building blocks for many targeted signatures. For example, the activation function subgraph shown in Figure 16 is commonly seen in models that leverage the transformer architecture and can be used in conjunction with other ‘Combo’ signatures to identify further models that use this architecture, and not just the VisionTransformer signature used here.

In addition to the above, further testing was conducted to determine whether or not ShadowGenes would trigger multiple signatures if applied to a multimodal model, one for each modality. When applied to a dataset of approximately 50,000 ONNX model files from HuggingFace, this was found to be the case.

For example, an optical character recognition (OCR) model [8] triggered two signatures, demonstrating a clear delineation between its component tasks. OCR models are used to extract text from images. For example, an OCR model that performs scanned document summarization must first extract text from the scanned image (image processing) and then summarize the extracted text (natural language processing).

The ResNet signature was triggered and, as can be seen in Figure 12, is related to the model’s initial input and earlier layers.

The signature developed for the Sequencer2D models within the test set was also triggered. Sequencer leverages LSTM (Long Short Term Memory) which has also been proposed for OCR tasks because of its contextual sequencing capabilities [9] [10]. This part of the model is shown in Figure 12 below.

The fact that both of these signatures triggered on the model indicates that, at the very least, the model receives an input image, performs an image recognition task, and then breaks the output down into a sequence. Knowing this is an OCR model helps us understand the component parts of the model, in that ResNet is likely used for text recognition within the image, and Sequencing / LSTM is used to contextualize the text to perform the desired task. However, even without this additional context, we could still infer that the model performs a sequencing task on an input image.

The depth-first search visits nodes and creates new blocks based on the execution flow criteria described in section 2.2.1. If a node has already been visited the search will terminate early to avoid running unnecessary computations, if a node has not already been visited the algorithm marks it as visited.

The ExtractBlocks algorithm checks the flow criteria by creating a new block if one of several criteria is met, as described below

Firstly, if a node’s number of non constant inputs is not equal to one, then a new block is made. If the count is zero, this signifies the start of a new branch, and therefore the start of a new block. If the count is greater than or equal to two, this represents a converging flow on the current node, indicating a new block has started. This check is performed prior to adding a node to a block.

After adding the current node to the current block, the algorithm checks if the node has two or more outputs, signifying a diverging execution flow, and therefore the end of a block. If there is only one output node, the next check is performed in order to determine if said output node has a non constant input node count greater than or equal to two, indicating that the next node is a new block due to a converging execution flow. If either of the above conditions are met, a new block is created and the current block is added to the list of blocks.

The algorithm then loops through all the node’s outputs and recursively calls the depth-first search again. Once the algorithm has looped through each output it creates a new block.

Throughout the process outlined above, lookups need to be performed tens of thousands of times. Therefore, simple data types such as integers are used for these lookups, these are combined with sets and tuples, because using hashable types greatly improves computational efficiency.