LLM-Guided Evolution: An Autonomous Model Optimization for Object Detection

The development of machine learning traditionally requires expertise in multiple areas, from data preprocessing to feature engineering, model selection, hyperparameter tuning, and deployment. As the demand for AI-driven solutions grows across industries and the complexity of machine learning tasks increases, Automated Machine Learning (AutoML) has emerged as a solution to address these challenges. AutoML automates the end-to-end process of developing machine learning models, enabling non-experts to build high-performance models efficiently while reducing the need for manual intervention.

AutoML consists of several key components that automate different stages of the machine learning pipeline. Data preprocessing and feature engineering involve data cleaning, transformation, and feature extraction, ensuring that datasets are prepared efficiently without manual intervention (Hutter et al., 2019). Model selection enables AutoML to explore various algorithms to identify the model with the best performance. Hyperparameter tuning alters model parameters and training configurations using techniques such as grid search and genetic algorithms to enhance accuracy and efficiency (Bergstra et al., 2011). For deep learning architecture design, NAS automates the discovery of optimal network architectures. Finally, model evaluation and deployment ensure that selected models are validated using cross-validation, allowing seamless integration into real-world applications (Olson and Moore, 2016). These components work together to streamline machine learning development, making AI more accessible and scalable across various domains (Ledell and Poirier, 2020).

Frequent in the development of AutoML solutions are Evolutionary Algorithm (EA)-based approaches (Olson and Moore, 2016; Zutty et al., 2015), which utilize a genetic encoding of machine learning pipeline choices and their hyperparameters.

AutoML has demonstrated its ability to surpass human-designed architectures in diverse domains, from image recognition (NASNet (Zoph and Le, 2017), EfficientNet (Tan and Le, 2019)) to structured data analysis (AlphaD3M (Drori et al., 2021)).

The overall goal of research in autoML is to discover higher-performing algorithms with fewer evaluations of candidate solutions or less total computational time. This paper considers an emerging new subfield of autoML which involves the usage of LLMs, described in Section 2.2.

The recent advancements in LLM capabilities have jump-started a new intersection between evolutionary computation and the employment of LLMs for automated machine learning (Wu et al., 2024). LLMs excel in code production due to the structure and rules followed in each language.

While traditional AutoML and evolutionary approaches rely on a representation or encoding of a genome for each algorithm, an evolutionary algorithm paired with an LLM is able to operate directly on code (Hemberg et al., 2024). Such an EA still retains its bio-inspired metaphor, where evaluation, selection, crossover, and mutation are in play. In this new form, evaluation and selection are handled using standard EA approaches, but the operators of crossover and mutation utilize LLMs, where each operation involves prompting an LLM with the code that is the individual.

Recently, this type of approach has been demonstrated several times on common benchmark problems such as image classification with the CIFAR-10 dataset (Morris et al., 2024; Nasir et al., 2024) or MNIST (Chen et al., [n. d.]) and the traveling salesman problem (Liu et al., 2023). An open question this paper seeks to answer is does this approach work well with less frequently published datasets and problems, where the LLM may not have been trained with as much information?

Across open literature, a variety of LLMs have been used for integration with EAs including: CodeGen-6B, Mixtral 8x7B, GPT-3.5 Turbo, Llama-2-70B-Instruct, and PaLM-62B. However, few papers compare the performance of LLMs within optimizations or blend their results.

Handcrafted neutral architecture requires domain knowledge of model design and large amounts of trail-and-error experiments to achieve promising performance for classification, detection, and segmentation tasking. As NAS demonstrated its efficacy in designing SoTA image classification models (Zoph and Le, 2017; Tan and Le, 2019), the NAS emerged to design detection backbones for object detection initially (Chen et al., 2019).

A state-of-the-art object detection system typically consists of a backbone, a feature fusion neck, a region proposal network (RPN), and an RCNN head. In one-stage detection models, such as YOLO, the detection pipeline consists of only a backbone and a head. NAS has emerged as a powerful tool to automate the design of detection architectures under given constraints, often guided by principles such as maximum entropy. The MAE-DET model was proposed to automatically design detection backbones using the Maximum Entropy Principle, eliminating the need for network training while still achieving state-of-the-art performance (Sun et al., 2022). Similarly, DAMO-YOLO was developed to optimize the backbone, neck, and head, resulting in a new model that surpasses previous YOLO series models on the COCO dataset (Xu et al., 2022).

In addition, EAutoDet was introduced to design both the backbone and Feature Pyramid Network (FPN) by constructing a supernet that jointly optimizes these modules using a differentiable NAS method (Wang et al., 2022). Another approach, Structural-to-Modular NAS (Yao et al., 2020), applies a multi-objective search algorithm to explore module combinations for backbone optimization, aiming to find efficient and effective architectures for object detection (Yao et al., 2020).

While none of the existing NAS methods for object detection have utilized Large Language Models, in this paper, we introduce a novel LLM-driven NAS approach. Specifically, we use a LLM-GE to intelligently modify neural network architectures through their module, layer, and hyperparameter descriptions in YAML configuration files, effectively enhancing YOLO’s performance on the KITTI dataset.

To enable meaningful evolution, LLMs must possess the core abilities and reasoning capabilities outlined in Section 3 to propose valid YAML configurations that can be successfully evaluated within the training framework, ultimately leading to performance improvements. For this study, we ran the evolution in two manners of operation. In the first, which we call Llama-3.3-GE1, we seeded our LLM-GE with the YAML configuration file split into the three blocks shown in Listing 1. In this manner of operation we seeded the evolution with only the configuration for YOLOv3. In the second manner of operation, which we call Llama-3.3-GE2, we made the following changes:

1. (1)

   We modified the representation of the configuration file to be a single block comprising the head, backbone, and hyperparameters.

2. (2)

   We modified the prompts to the LLM to instruct it to modify a specific part of the YAML file, such as the the head, backbone, or parameters.

3. (3)

   We included additional seeds into the starting population of the LLM-GE including YOLOv3-tiny, YOLOv9s, YOLOv10-tiny, and YOLOv11, in addition to the YOLOv3 seed used in the first manner of operation

Table 1 summarizes the framework’s performance over 50 generations for Llama-3.3-GE1 and 59 generations for Llama-3.3-GE2. Note that GE1 produced a 40.7% failure rate, where the YAML files during the evaluation process, while GE2 produced a 51.9% failure rate. A possible contribution to the increase could be the added variety of seeds, or the more open prompting style to the LLM.

Figure 2 illustrates the number of individuals on the Pareto frontier over time for each GE. The figure shows that the number of individuals on the Pareto frontiers for both of the GEs grows steadily throughout the generations. Note that the two experiments ran for different numbers of generations: LLM-GE1 stops after 50 generations, while LLM-GE2 stops after 59 generations. The GEs exhibit upward trends, suggesting continued search of the tradeoff space between objectives. Both trends can be characterized by a linear behavior and the trend of Llama-3.3-GE1 has a steeper slope than that of the Llama-3.3-GE2, indicating a faster improvement for the number of individuals in the Pareto Fronts.

In addition, to compare the overall performance of LLM-Guided Evolution with difference in seedings and content included in prompts, we evaluate the quality of generated solutions using hypervolume under the Pareto frontiers (for minimization of objectives). A smaller hypervolume indicates broader exploration of the Pareto-optimal space. The optimization objectives include the number of parameters, inference speed, precision, and recall, where GE minimizes the number of parameters and inference speed while maximizing precision and recall.

To compute the hypervolume shown in Figure 3, the number of parameters and inference speed are normalized between 0 and 1. Additionally, 1 - precision and 1 - recall are used to align all objectives with a minimization trend. As a result, the scores are contained within a hypercube of unit hypervolume.

Figure 3 shows the hypervolumes for Llama-3.3-GE1 and Llama-3.3-GE2. The blue connected squares denote the hypervolume for Llama-3.3-GE1 at each generation and the orange connected triangles denote the hypervolume for Llama-3.1-GE2 at each generation. The figure shows that Both Llama-3.3-GE1 and Llama-3.3-GE2 improve over time, and that Llama-3.3-GE1 achieves better hypervolume than Llama-3.3-GE2. It is interesting to note that the Llama-3.3-GE1 outperforms the other Llama-GE2 after 25 generations and there was a significant improvement from generation 26 to generation 27. Meanwhile, the hypervolume of Llama-3.3-GE2 reaches a plateau after 33 generations.

The figure also shows that at 0th generation, the hypervolumes of the two GEs are not the same. This could be caused by the randomness of creating the initial populations or by the difference in the set of seeds. The Llama-3.3-GE1 was only seeded with YOLOv3 while Llama-3.3-GE2 was seeded with additional seeds of YOLOv3-tiny, YOLOv9s, YOLOv10-tiny, and YOLOv11. We note a further investigation on the stochastic nature of this process with significantly more trials in our proposed future work.

In addition, Llama-3.3-GE1 discovered 135 Pareto-optimal YOLO variants in its final generation, while Llama-3.3-GE2 generated 128 Pareto-optimal YOLO variants in its final generation. By inspecting the Pareto front individuals, we observed common changes such as modifications to the number of layers, the width and depth multipliers, and the arguments of the layers (see Listing 1), especially in Llama-3.3-GE1. These observations suggest that Llama-3.3 relies solely on the seed model, YOLOv3, and lacks awareness of other layer types. In contrast, Llama-3.3-GE2, seeded with multiple state-of-the-art YOLO models, incorporated a wider variety of layers (See Listings 4-6).

Note that even though Figure 3 shows that the hypervolume for Llama-3.3-GE2 appears to stagnate around generation 34, Figure 2 shows that during this time, the evolution is still discovering a significant number of new individuals along the Pareto frontier, suggesting an increase in fidelity along the objectives in the trade-off space. It is a possibility that the Llama-3.3-GE2 is in a state of punctuated equilibrium.

In some cases, the LLM attempted to create new layers as Python code for the configuration. However, our current framework does not yet utilize this information. Future work will focus on integrating these generated layers into the model to further enhance YOLO’s architecture.

Listing 4 shows a modified YAML configuration file of the seed model. It was one of the Pareto front individuals proposed by Llama-3.3-GE2. The modifications made by the LLM-guided mutations and crossovers include changing the inputs for the Conv and Bottleneck layers and changing the number of layers.

The application of LLM-Guided Evolution to YOLO optimization generates promising YOLO variants with performance improvements across several key metrics, demonstrating the efficacy of this approach in optimizing the architecture and associated hyperparameters when seeded with a state-of-the-art object detection model for a challenging real-world dataset like KITTI. These improvements can be attributed to the framework’s ability to intelligently explore the configuration space through LLM-Guided mutations and crossovers. This autonomous framework successfully generated many YOLO variants, outperforming the original network’s performance of 92.5% mAP@50 on holdout data. A notable example generated by GE with Llama 3.3 achieves 94.5% mAP@50 on holdout data.

To understand the variety of individuals with unique contributions discovered across both GE1 and GE2, Figure 7 presents a parallel coordinate plot of the individuals on the Pareto frontier evaluated on the validation dataset, which the LLM-Guided Evolution framework uses to assess the performance of newly generated individuals. We combined the 135 Pareto-optimal individuals from GE1 with the 128 Pareto-optimal individuals from GE2, and kept only the individuals that remained Pareto-optimal. These individuals are co-dominant with respect to the objectives of model parameters, inference speed, precision, and recall, while also highlighting two key object detection metrics: mAP@50 and mAP@50-95. In the plot, blue diamonds connected by lines represent the performance of the remaining 11 individuals generated by Llama-3-3-GE1 and orange stars connected by lines indicate the performance of the remaining 63 individuals generated by Llama-3.3 GE2. The figure also shows the performance of YOLOv3, YOLOv3 SPP, YOLOv3 tiny, and other State-of-the-Art YOLO model for reference. We retrained these models on the same KITTI data with the default settings for training. Optimization of the training setting is not included in this study, but it remains a topic for future exploration.

Similar to Figure 7, Figure 8 presents the parallel coordinate plot of the Pareto front individuals evaluated on the holdout dataset. We reduced the set of individuals using the same procedure as applied to the validation set. The figure illustrates that some individuals outperform the reference models, including YOLOv3, YOLOv3 SPP, YOLOv3 tiny, YOLOv9s, YOLOv10tiny and YOLOv11. However, these improved variants come at the cost of higher computational complexity, making them more resource-intensive. In the plot, blue diamonds connected by lines represent the performance of 6 individuals generated by Llama-3.3-GE1 and orange stars connected by lines indicate the performance of 45 individuals generated by Llama-3.3-GE2. Listings 4-6 are examples 1-3 highlighted in Figure 8. They demonstrate that the LLM-GE is able to explore the configuration space by modifying numbers of layers, adding layers ( such as SPP layer and Conv layer), and changing the inputs of layers.

These visualizations effectively illustrate that the LLM-GE successfully evolved YOLO variants with trade-off between mAP and speed. This result highlights the flexibility and effectiveness of GE in real-world computer vision challenges, offering a novel paradigm for autonomous model optimization that combines LLM-driven reasoning with evolutionary strategies.