UBER: Uncertainty-Based Evolution with Large Language Models for Automatic Heuristic Design

Zijie Chen^1,2 Zhanchao Zhou^1,2 Yu Lu² Renjun Xu¹ Lili Pan³ Zhenzhong Lan²
¹ Zhejiang University ² School of Engineering, Westlake University
³University of Electronic Science and Technology of China
{chenzijie, lanzhenzhong}@westlake.edu.cn
Corresponding author.
Work in progress.

Abstract

NP-hard problem-solving traditionally relies on heuristics, but manually crafting effective heuristics for complex problems remains challenging. While recent work like FunSearch has demonstrated that large language models (LLMs) can be leveraged for heuristic design in evolutionary algorithm (EA) frameworks, their potential is not fully realized due to its deficiency in exploitation and exploration. We present UBER (Uncertainty-Based Evolution for Refinement), a method that enhances LLM+EA methods for automatic heuristic design by integrating uncertainty on top of the FunSearch framework. UBER introduces two key innovations: an Uncertainty-Inclusive Evolution Process (UIEP) for adaptive exploration-exploitation balance, and a principled Uncertainty-Inclusive Island Reset (UIIS) strategy for maintaining population diversity. Through extensive experiments on challenging NP-complete problems, UBER demonstrates significant improvements over FunSearch . Our work provides a new direction for the synergy of LLMs and EA, advancing the field of automatic heuristic design.

{python}

import numpy as np import random import math

def euclidean_distance(city1, city2): return math.sqrt((city1[0] - city2[0])**2 + (city1[1] - city2[1])**2)

def calculate_total_distance(route, distance_matrix): return sum(distance_matrix[route[i]][route[i+1]] for i in range(len(route)-1)) + distance_matrix[route[-1]][route[0]]

def two_opt(route, distance_matrix): best_route = route.copy() improved = True while improved: improved = False for i in range(1, len(route)-2): for j in range(i+1, len(route)): if j-i == 1: continue new_route = route[:i] + route[i:j][::-1] + route[j:] if calculate_total_distance(new_route, distance_matrix) < calculate_total_distance(best_route, distance_matrix): best_route = new_route improved = True route = best_route return best_route

@run def guided_local_search(cities, max_iterations=100, alpha=0.1): num_cities = len(cities) distance_matrix = np.zeros((num_cities, num_cities)) for i in range(num_cities): for j in range(i+1, num_cities): distance_matrix[i][j] = distance_matrix[j][i] = euclidean_distance(cities[i], cities[j]) init_distance_matrix=copy.deepcopy(distance_matrix) # Initialize route route = list(range(num_cities)) best