Machine Learning Infused Distributed Optimization for Coordinating Virtual Power Plant Assets

As global energy sectors transition towards sustainability, the role of Distributed Energy Resources (DERs) has become increasingly significant. However, the participation of DERs in competitive electricity markets remains a challenge [1]. While many DERs are capable of providing wholesale market services, they often individually fall short of the minimum size thresholds established by Independent System Operators (ISOs) and may not meet performance requirements [2]. As a solution to these challenges, Virtual Power Plants (VPPs) have emerged to aggregate diverse DERs, creating a unified operating profile for participation in wholesale markets and providing services to system operators [3]. Further promoting the aggregation of DERs, the Federal Energy Regulatory Commission’s (FERC’s) Order 2222, issued in September 2020, allowed DERs to compete on equal terms with other resources in ISO energy, capacity, and ancillary service markets [2]. The FERC regulatory advancement strengthens the position of DERs and VPPs in the market.

Despite their promising potential, the massive, decentralized, diverse, heterogeneous, and small-scale nature of DERs poses significant challenges to traditional centralized approaches, especially in terms of computational efficiency and speed. Centralized controls for VPPs require global information from all DERs, making them susceptible to catastrophic failures if centralized nodes fail and potentially compromising the privacy of DER owners’ information. To address these issues, there is a growing demand for efficient, scalable, distributed and decentralized optimization techniques. Our study aims to tackle these challenges and develop a solution that can efficiently harness the benefits of DERs, thereby unlocking the full potential of VPPs.

In this paper, we propose an ML-assisted method to replace the building blocks of the ADMM-based distributed optimization technique with neural approximators. Our method is referred to as $\mathcal{LOOP-MAC}$ (Learning to Optimize the Optimization Process for Multi-agent Coordination). We will employ our $\mathcal{LOOP-MAC}$  method to find a multi-agent solution for the power dispatch problem in DER coordination within a VPP. In the muti-agent VPP configuration, each agent may control multiple DERs. The proposed $\mathcal{LOOP-MAC}$  method enables each agent to predict local power profiles by communicating with its neighbors. All agents collaborate to achieve a near-optimal solution for power dispatch while adhering to both local and system-level constraints.

The utilization of neural networks expedites the search process and reduces the number of iterations required to identify optimal solutions. Additionally, unlike restoration-based methods, the $\mathcal{LOOP-MAC}$  approach doesn’t necessitate post-processing steps to enhance feasibility because local constraints are inherently enforced through a gauge mapping method [39], and coupled constraints are penalized through ADMM iterations. This paper advances our recent work in [32] that is focused on speeding up the ADMM-based DC-OPF calculations through efficient approximation of primal and dual sub-problems. While [32] tackled the DC-OPF problem, the present paper extends our previous model to incorporate individual VPP assets, addressing the DER coordination problem. In terms of methodology, [32] employs ML to facilitate both primal and dual updates of the ADMM method. This requires neighboring agents to share updated global variables, local copies of global variables, and Lagrangian multipliers. This work, however, replaces the two ADMM update procedures with a single data infusion step that reduces agents’ communication and computation burden.

The considered VPP consists of a number of $N_{\texttt{A}}$ agents, each denoted by index $i,i\in\mathcal{N}_{\texttt{A}}$. Every agent is responsible for aggregating a diverse set of DERs, which encompasses flexible loads (FLs), energy storage systems (ESSs), heating, ventilation, and air conditioning (HVAC) systems, plug-in electric vehicles (PEVs), and photovoltaic (PV) arrays, as shown in Fig.1. These agents might be connected to networks of different utilities. The primary objective of the VPP is to optimize the aggregate behavior of all agents while accounting for agents’ utility functions.

In this paper, we propose that the VPP operates within a two-settlement energy market, composed of a day-ahead and a real-time market. Upon the clearing of the day-ahead market, the VPP decides on hourly production schedules. The real-time market, also known as the imbalance market, is designed to settle potential day-ahead commitment violations. The real-time market productions are set in 5-minute increments. The production schedules every 5 minutes are denoted as $\mathbf{P}_{\texttt{Sch}}$.

The $\mathcal{LOOP-MAC}$  method is designed for the real-time market, where a VPP solves a dispatch optimization across its assets (agents) to honor its commitment over a given time scale, $[t_{\texttt{s}},t_{\texttt{e}}]$, where $t_{\texttt{s}}$ and $t_{\texttt{e}}$ represent the starting and ending times, respectively. Put differently, the VPP needs to fulfill the production schedule $\mathbf{P}_{\texttt{Sch}}=\left[{P}_{\texttt{Sch}}^{t}\mid t=t_{\texttt{s}},\ldots,t_{\texttt{e}}\right]$ while minimizing the overall cost of agents. Generally, the VPP implements 5-minute binding intervals ($\Delta t=5/60$ h) for the real-time market, and adopts look-ahead horizon $(t_{\texttt{e}}-t_{\texttt{s}})$, ranging from 5 minutes up to 2 hours [40], for the real-time dispatch optimization. The detailed dispatch optimization problem is presented next.

This subsection presents the centralized form of the power dispatch problem solved by a VPP over various assets for every time step $t\in[t_{\texttt{s}},t_{\texttt{e}}]$. The asset constraints are:

Agent-based problem-solving lends itself well to addressing the computational needs of the VPP coordination problem. In this subsection, we focus on finding a distributed solution for (21) (or (1)). While each sub-problem optimizes the operation of individual agents, communication enables individual agents to collectively find the system-level optimal solution.

In the context of distributed problem-solving, it is important to point out the unique challenges posed by coupling constraints such as (17). These constraints introduce intricate relationships among several agents where some variables of agent $i$ are tied with those of agent $j$. These coupled constraints prevent separating (21) into disjointed sub-problems.

As discussed in Section IIA, we define the variables present among multiple agents’ constraints as global variables, $\mathbf{u}^{i}_{\texttt{g}}$,

In contrast, the variables solely managed by non-overlapping constraints are referred to as local variables. That is, $\mathbf{u}^{i}=[\mathbf{u}^{i}_{\texttt{l}},\mathbf{u}^{i}_{\texttt{g}}]$. We refer to agents whose variables are intertwined in a constraint as neighboring agents.

The ADMM method finds a decentralized solution for (21) by creating local copies of neighboring agents’ global variables and adjusting local copies iteratively to satisfy both local and consensus constraints. The adjustment continues until alignment with original global variables is achieved, at which point the global minimum has been found in a decentralized manner.

In the power dispatch problem, we introduce $P_{\texttt{OCopy}}^{i,j,t}$, which is owned by agent $i$, and represents a copy of $P_{\texttt{O}}^{j,t}$. Then, coupled constraint (17) become a local constraint (23) and a consensus constraint (24):

Let $\mathbf{u}^{i}_{\texttt{g,Copy}}={[P_{\texttt{OCopy}}^{i,j,t}\mid t=t_{\texttt{s}},\ldots,t_{\texttt{e}}]}$ denote all local copies owned by agent $i$ imitating other neighboring agents’ global variables. Then, one could reformulate the problem (21) in accordance to $\mathbf{u}^{i}$ and $\mathbf{u}^{i}_{\texttt{g,Copy}}$ as,

where, $\mathbf{A}^{i}_{\texttt{eq}}$, $\mathbf{B}^{i}_{\texttt{eq}}$, $\mathbf{b}^{i}_{\texttt{eq}}$, $\mathbf{A}^{i}_{\texttt{ineq}}$, $\mathbf{B}^{i}_{\texttt{ineq}}$, and $\mathbf{b}^{i}_{\texttt{ineq}}$ in (25b) and (25c) capture the compact form of constraints (5)-(16), (23). And (25d) is the compact form of constraints (24). Here $\mathbf{I}_{\texttt{c}}^{i}$ is the element selector matrix that maps elements from vector $\bigoplus_{j\neq i}\mathbf{u}_{\texttt{g}}^{j}$ to vector $\mathbf{u}^{i}_{\texttt{g,Copy}}$ based on a consensus constraint (24). Each row of $\mathbf{I}_{\texttt{c}}^{i}$ contains a single 1 at a position that corresponds to the desired element from $\bigoplus_{j\neq i}\mathbf{u}_{\texttt{g}}^{j}$ and 0s elsewhere. Therefore, $\mathbf{I}_{\texttt{c}}^{i}[\bigoplus_{j\neq i}\mathbf{u}_{\texttt{g}}^{j}]$ represents the vector of global variables that are required to be imitated by agent $i$.

Let $\mathcal{S}_{\texttt{Local}}^{i}$ be the set of local constraints associated with agent $i$, i.e., (25b)-(25c). Therefore, the compact form of decentralized formulation at the agent-level as defined in (2).

The standard form of ADMM solves problem (25) (or (2)) by dealing with the augmented Lagrangian function $\mathrm{L}$:

where $\rho>0$ is a positive constant. $\boldsymbol{\lambda}^{i}$ denotes the vector of all Lagrangian multipliers for the corresponding consensus equality relationship between agent $i$’s copy and neighboring agent $j$’s global variable.

The search for a solution to (26) is performed through an iterative process (indexed by $[k],k=1,...,N_{\texttt{K}}$). All $N_{\texttt{A}}$ agents will execute this process simultaneously and independently before communicating with neighboring agents. At the agent level, these updates manifest themselves as follows,

The dual update equation, i.e., (27), modifies the Lagrangian multipliers to estimate the discrepancies between an agent’s local copy of variables (designed to emulate the global variables of its neighbors) and the actual global variables held by those neighbors. Subsequently, (28) provides an optimization solution leveraging prior iteration data from other agents.

It’s essential to note that agent $i$ doesn’t require all the updated values from other agents to update equations (27) and (28). Agent $i$ primarily needs:

• •

  Neighboring agents’ global variables: $\mathbf{I}_{\texttt{c}}^{i}[\bigoplus_{j\neq i}\mathbf{u}_{\texttt{g}}^{j}]$. In the context of the distributed DER problem, agent $i$ requires values of $P_{\texttt{O}}^{j,t^{[k-1]}}$ from their neighboring agent $j$.

• •

  Neighboring agents’ local copies mirroring agent $i$’s global variables: $\mathbf{I}_{\texttt{g}}^{i}[\bigoplus_{j\neq i}\mathbf{u}_{\texttt{g,Copy}}^{j^{[k-1]}}]$, where $\mathbf{I}_{\texttt{g}}^{i}$ functions as a selector matrix. In the distributed DER context, agent $i$ requires $P_{\texttt{OCopy}}^{{j,i,t}^{[k]}}$ from their neighboring agent $j$.

We use $\mathbf{u}^{i^{[k-1]}}_{\texttt{Other}}$ to represent the set of variables owned by other agents but are needed by agent $i$ to update (27) and (28). Finally, the intra-agent updates are represented by (4) and (3).

The standard form of ADMM guarantees the feasibility of local constraints by (4) and penalizes violations of consensus constraints by iteratively updating Lagrangian multipliers as (3). In what follows, we will propose a ML-based method to accelerate ADMM for decentralized DER coordination. The ADMM iterations will guide the consensus protocol, while the gauge map [39] is adopted to enforce hard local constraints.

This section provides a high-level overview of the $\mathcal{LOOP-MAC}$  method to incorporate ML to accelerate the ADMM algorithm. As shown in Fig. 2, instead of solving agent-level local optimization problems (4) by an iterative solver, we will train ${N}_{\texttt{A}}$ agent-level neural approximators $\xi^{i},i\in\mathcal{N}_{\texttt{A}}$ to directly map inputs to optimized value of agent’s optimization variables in a single feed-forward. The resulting prediction of each agent $i$, denoted as $\mathbf{u}^{i^{[k]}}$, will be trained to approximate the optimal solution of (2).

Pseudo code of the proposed $\mathcal{LOOP-MAC}$  method is given in Algorithm 1. $\mathcal{LOOP-MAC}$  method includes two steps for each iteration. First, each agent receives variables of prior iteration from neighboring agents. Second, each agent uses a neural approximator to predict its optimal values.

Violations of consensus constraints could be penalized by ADMM iterations. Further, we will design each neural approximator’s structure to guarantee that its output satisfies the local constraints, i.e., $\xi^{i}\in\mathcal{S}_{\texttt{Local}}^{i}(\mathbf{x}^{i})$. We adopt the $\mathcal{LOOP-LC}$ (Learning to Optimize the Optimization Process with Linear Constraints) model proposed in [39] to develop each neural approximator $\xi^{i}$. The $\mathcal{LOOP-LC}$ model learns to solve optimization problems with hard linear constraints. It applies variable elimination and gauge mapping for equality and inequality completions, respectively. The $\mathcal{LOOP-LC}$ model produces a feasible and near-optimal solution. In what follows, we will present the main components of $\mathcal{LOOP-LC}$ and how it applies to the VPP coordination problem.

We use the historical trajectories of ADMM (i.e. applied on historical power demands) for training purposes. Note that predicting the converged ADMM values is a time-series prediction challenge. Specifically, outputs from a given iteration are requisites for the subsequent iterations. This relationship implies that $\mathbf{u}^{i^{[k]}},\mathbf{u}^{i^{[k]}}_{\texttt{g,Copy}},\forall i$ are contingent upon $\mathbf{u}^{i^{[k-1]}}_{\texttt{Other}}$, derived from other agents’ outputs $\mathbf{u}^{j^{[k-1]}},\mathbf{u}^{j^{[k-1]}}_{\texttt{g,Copy}},\forall j\neq i$ from the prior iteration. To encapsulate this temporal dependency, our training approach adopts a look-ahead format, facilitating the joint training of all neural approximators in a recurrent manner, which ensures that prior outputs from different agents are seamlessly integrated as current inputs (see Fig. 4).

Suppose there are $N_{\texttt{D}}$ training data points, indexed and associated with their respective output by the superscript $(d)$. As an initial step, ADMM is employed to generate all values of optimization variables required for training. Concurrently, the optimal solution $\mathbf{u}^{i*(d)}$ pertaining to (2) is calculated. Subsequently, for $N_{\texttt{R}}$ recurrent steps, the loss function $f_{\texttt{L}}$ is defined as the cumulative distance $d$ between the prediction $\mathbf{u}^{i^{[k+r](d)}}$ and the optimal solution $\mathbf{u}^{i*(d)}$. This summation spans all agents, every recurrent step, every iteration ($k=1,..N_{\texttt{K}}$), and all data points, as delineated in (33).

Fig. 6 illustrates the cumulative computation time across all agents and test data points over iterations. The performance comparison is conducted among the decentralized setup employing ADMM solvers, our proposed $\mathcal{LOOP-MAC}$  method, and traditional centralized solvers.

From the case study, it is observed that the computational time required by the classical ADMM solver exceeds the centralized solvers solution time after approximately five iterations. Remarkably, our proposed $\mathcal{LOOP-MAC}$ method significantly outperforms the classical ADMM, achieving 500x speed up. Also, $\mathcal{LOOP-MAC}$  even surpasses the efficiency of the centralized solver in terms of computation speed.

Table II provides the average computational time for a single iteration on a single data point. An insightful observation from the results suggests that the $\mathcal{LOOP-MAC}$  method would require around 3300 iterations to match the computational time of centralized solvers. However, based on the convergence analysis that will be provided later, $\mathcal{LOOP-MAC}$  method demonstrates convergence in a mere 10 iterations.

Fig. 7 presents the optimality deviation rate for both the traditional ADMM algorithm and $\mathcal{LOOP-MAC}$  method. The deviation rate metric quantifies the degree to which the operational profiles of the DERs deviate from the optimal (derived from solving the centralized problem). It is evident that $\mathcal{LOOP-MAC}$  method achieves faster convergence. Moreover, $\mathcal{LOOP-MAC}$  showcases faster reduction of the deviation rate compared to the standard ADMM approach.

Similarly, Fig. 8 depicts the deviation rate of the VPP schedule for both the ADMM approach and $\mathcal{LOOP-MAC}$  method. This rate sheds light on the difference between the actual VPP production schedule and its planned output. In the context of our optimization problem, the deviation rate is equivalent to the feasibility gap rate of the coupled constraints, as shown in (17). Notably, $\mathcal{LOOP-MAC}$  excels in convergence speed and stability. The VPP schedule deviation rate declines more rapidly and remains stable using $\mathcal{LOOP-MAC}$ method, whereas the traditional ADMM method results in more oscillations and converges at a slower pace.

Table III summarizes post-convergence metrics for both algorithms across all agents, iterations, and test data points. While the minimum optimality deviation rate achieved by $\mathcal{LOOP-MAC}$  is slightly higher than that of the classical ADMM, our approach showcases a much lower variance and a significantly reduced maximum deviation. These results highlight $\mathcal{LOOP-MAC}$  method’s efficacy, especially when tasked with recurrently solving similar optimization problems. The observed improvements in variance and maximum deviation highlight the versatility and robustness of $\mathcal{LOOP-MAC}$  in varied problem scenarios. To sum up, the proposed $\mathcal{LOOP-MAC}$ solution speeds up the solution time of each ADMM iteration by up to 500X. Also, $\mathcal{LOOP-MAC}$  needs fewer iterations to converge, hence, the overall run time will be significantly shorter.