A Nash Equilibrium Solution for Periodic Double Auctions

Auctions are mechanisms that facilitate buying and selling of goods between market participants. A double auction consists of multiple buyers and sellers submitting their asks and bids to a market institution in order to procure a target unit of a commodity. A bid or ask consists of a price-quantity pair ($p,q$) indicating that the participant is willing to buy/sell $q$ units of the commodity at a unit price $p$. The market institution matches the buy bids with the sell asks to determine the clearing price and cleared quantities for all sellers and buyers. These type of auctions are very prevalent in stock exchanges [1] and energy markets [2]. For example, in energy markets, power generating companies are the sellers while energy brokers servicing retail customers are the buyers and a energy market regulator plays the role of the central market institution. Since the volume of trade is very high in such markets [3], it is prudent to design an optimal bidding strategy on behalf of a market participant to bring in profits and system efficiency to the ecosystem. The design of such optimal bidding strategies become more pronounced in a periodic double auction (PDA) setup (see Figure 1) wherein buyers and sellers participate in a (finite) sequence of auctions to exchange certain units of a commodity [4]. For example, an energy broker, armed with an estimated energy requirement for a future time slot, participates in day-ahead auctions, to procure the required energy from power generating companies by competing with other energy brokers. In these auctions, the broker will have more than one opportunity to procure the estimated energy by participating in a sequence of auctions. For the purpose of this exposition, such an auction set up, as depicted in Figure 1, is referred to as a periodic double auction (PDA). Evidently, in this PDA setup, an optimal bidding strategy involves planning across current and future time auctions and any small improvement in the bidding strategies of the market participants can lead to improved profits and system efficiency. Motivated by this, we formulate periodic double auctions as a Markov game and derive equilibrium solutions to devise optimal bidding strategies.

Equilibrium solutions for double auctions have been studied extensively in the past. For example, the work of Satterthwaite and Williams [5] proved the existence of non-trivial equilibria for $k$-double auctions. Analytical solutions for Nash equilibrium strategies for double auctions with average clearing price rule (ACPR) have been derived for the one buyer one seller single shot case for uniformly distributed valuations [6] and scale based strategies [7, 8]. However, all the above approaches were developed for single shot double auctions whereas sequential decision making in a multi-agent setting was required to study PDAs. In recent years, Markov game framework have been in use to devise bidding strategies in multi-shot auctions wherein approaches such as multi-agent Q-learning [9], multi-agent deep Q networks [10], deep deterministic policy gradients [11] are deployed. However, much of these works do not involve deriving analytical solutions for equilibrium strategies. As far as we know, this work is the first attempt to find analytical solutions for equilibrium strategies in a PDA set up and herein lies our main contribution.

We now elaborate on certain aspects of the PDA considered in this work as the equilibrium analysis would depend on these specifics [12]. First, we assume that the total supply available is enough to meet the overall demand requirement and that all the asks from the suppliers can be clubbed into a composite supply curve and is made known to all the buyers to make their bids. This implies that the resultant Markov game will have only the buyers as the game participants. Second, each PDA consists of fixed number of rounds, i.e., each buyer has the same fixed number of auctions to procure their respective estimated demand. Third, buyers estimate their respective procurement need before the start of the first auction of the PDA and the estimate is not altered during the course of the PDA. Fourth, the buyers do not attempt to buy more than their outstanding requirement in any auction. Fifth, in the case that a particular buyer is not able to procure her targeted units of the commodity even after exhausting all rounds of the PDA, it will be procured outside of the auction at a higher cost. Finally, we consider uniform payment rule (UPR), wherein, the clearing price decided through the auction mechanism is the same for all market participants. Specifically, we consider average clearing price rule (ACPR) [8] where the market clearing price (MCP) is arrived as the average of the last cleared bid and last cleared ask.

We begin by introducing a few notations. For any positive integer $K$, let $[K]$ denote the set $\{1,\cdots,K\}$. We consider a system of $N$ buyers participating in $H$ rounds of a PDA to procure multiple units of a commodity from a set of sellers. For the purpose of this exposition, we assume that the composite supply curve from all the sellers at round $h\in[H]$ consisting of $M^{h}$ asks is known to all the buyers and is given by $\mathcal{L}^{h}=\{(p^{h}_{1},q^{h}_{1}),(p^{h}_{2},q^{h}_{2}),\ldots,(p^{h}_{M_{h}},q^{h}_{M_{h}})\}$ where $p_{h}\in[0,p_{\max}]$ and $q_{h}\in[0,q_{\max}]$ are respectively the price and quantity components of the ask $(p^{h}_{m},q^{h}_{m}),\textrm{ with }\;m\in[M_{h}]$ and $p_{\max},q_{\max}$ are suitable upper bounds for the ask. The total supply available, at any round $h\in[H]$, is given by,

$\displaystyle Q^{\mathscr{S},h}=\sum\nolimits_{m\in[M_{h}]}q^{h}_{m}.$

(1)

Denote the outstanding requirement of a buyer $b\in[N]$ at round $h$ as $Q^{{b},h}\geq 0$. Let $\mathcal{Q}^{h}=\{Q^{1,h},Q^{2,h},\dots,Q^{N,h}\}$ be the vector that contains the outstanding requirement of all the $N$ buyers at round $h$. Let $\mathcal{B}^{h}$ denote the set of all bids placed by all buyers at round $h$, wherein, each buyer $b\in[N]$ places at most one bid consisting of price-quantity pair $(p^{b,h},q^{b,h})$ with $p^{b,h}\in[0,p_{\max}]$ and $q^{b,h}\in[0,Q^{{b},h}]$. The number of bids placed at round $h$ is given by $B^{h}\leq N$ and the total demand from all buyers round $h$ is given by

$${}Q^{\mathscr{D},h}=\sum\nolimits_{b\in[N]}q^{b,h}$$

(2)

with $q^{b,h}=Q^{b,h}$.

In this work, we consider average clearing price rule (ACPR) as the clearing mechanism. The ACPR is a special case of $k$-Double auction ($k\in\interval{0}{1}$), where $k=0.5$ for the ACPR. The MCP in $k$-Double auction is defined as $\lambda^{h}=k\cdot p^{h}_{d}+(k-1)\cdot p^{b,h}_{l}$, where $p^{h}_{d}$ and $p^{b,h}_{l}$ are the last cleared ask and bid prices respectively. In particular for ACPR the clearing price is $\lambda^{h}=\frac{p^{h}_{d}+p^{b,h}_{l}}{2}$. Here, the bids with bid price greater than $p^{b,h}_{l}$ are fully cleared and the bid with bid price $p^{b,h}_{l}$ is either fully or partially cleared. Similarly the asks with price lesser than $p^{h}_{d}$ are fully cleared and the ask with ask price $p^{h}_{d}$ is either fully or partial cleared. The cleared quantity of the last cleared ask and bid depends on the total cleared quantity $Q^{h}$. Moreover, this total cleared quantity in the clearing mechanism is given as $Q^{h}=\min\{\sum\nolimits_{j=1}^{d}q^{h}_{j},\sum\nolimits_{i=1}^{l}q^{b,h}_{i}\}$. An example of the ACPR mechanism with a cleared price and a total cleared quantity is shown in Figure 2.

Having described the clearing process of a double auction, we now model the PDA consisting of $N$ buyers with horizon $H$ as a finite horizon Markov game ¹¹1A Markov game is sometimes known as a stochastic game [13] specified by $\mathcal{M}=\langle N,S,A,C,P,H\rangle$. The ingredients of $\mathcal{M}$ are a finite set of players $N$; a state space $S$; for each player $b\in[N]$, an action set $A^{b}$; a transition probability $P$ from $S\times A\rightarrow S$, where $A=\times_{b\in N}A^{b}$ is the action profile, with $P(s^{\prime}|s,a)$ as the probability that the next state is $s^{\prime}\in S$, given the current state is $s\in S$ and current action profile is $a\in A$; and a payoff function²²2Although, we use the term payoff, $C$ actually specifies the cost function. $\mathcal{C}$ from $S\times A\rightarrow\mathbb{R}^{N}$, where the $b$-th coordinate of $\mathcal{C}$ is $C^{b}$, is the payoff to player $b$ as a function of state and action profile.

More specifically, we let the state at round $h$ denoted by $s^{h}$, to consist of $\{\mathcal{Q}^{h},\mathcal{L}^{h}\}$ and the action $a^{b,h}\in A^{b}$ by player $b\in[N]$ at round $h$ consists of at most one bid belonging to the bounded set $[0,p_{\max}]\times[0,q_{\max}]$. The payoff function $C^{b,h}:S\times A\rightarrow\mathbb{R}$ returns a scalar value to player $b$ specifying her cost of procurement (if any) for the auction at round $h$. More precisely,

$\displaystyle C^{b,h}(s^{h},a^{h})=\begin{cases}\lambda^{h}\cdot\alpha^{b,h}&\text{at non terminal state $s^{h}$}\\ \Psi\times Q^{b,h}&\text{when }h=H+1,\end{cases}$

where $a^{h}=(a^{b,h},a^{-b,h})\in A$ is the joint action set containing one action for each player at round $h$ with $a^{-b,h}$ specifying the $N-1$ actions of all players except $b$. In addition, we have $\lambda^{h}\geq 0$ to be the clearing price of the auction at round $h$, $\alpha^{b,h}$ is cleared quantity for the buyer $b$ at round $h$. The entity $\Psi\geq 0$ is the unit price of procuring the commodity outside of the $H$ auctions and $Q^{b,H+1}$ is the remaining units of the commodity to be procured by buyer $b$ after exhausting the $H$ rounds of the PDA. Given a state $s^{h}\in S$ and action profile $a^{h}\in A$, the next state at round $h+1$ is given by $s^{h+1}=\{\mathcal{Q}^{h+1},\mathcal{L}^{h+1}\}$, where $\mathcal{L}^{h+1}$ refers to the uncleared asks of the supply curve from round $h$ and $\mathcal{Q}^{h+1}=\{Q^{1,h+1},\cdots,Q^{N,h+1}\}$ with $Q^{b,h+1}=Q^{b,h}-\alpha^{b,h},\forall\;b\in[N]$.

At any round $h$, having seen the state $s^{h}$, the players choose their action based on a policy. A (Markov) policy for a player $b\in[N]$ is a collection of policies $\pi^{b}=\{\pi^{b,h}:S\rightarrow\Delta_{A_{b}}\}_{h=1}^{H}$ where each $\pi^{b,h}(\cdot|s^{h})\in\Delta_{A_{b}}$ specifies the probability of taking action $a^{h}\in A_{b}$ at state $s^{h}$. Let $\pi=(\pi^{b},\pi^{-b})$ be the joint policy containing one policy for each player $b\in[N]$ where $\pi^{-b}$ denotes the $N-1$ policies of all players except $b$. The value of a joint policy $\pi$ (not necessarily Markov), at round $h$, for any player $b$ is a function $V^{h}_{\pi}:S\rightarrow\mathbb{R}$ defined as below.

$$V_{\pi}^{h}(s)=\operatorname{\mathbb{E}}_{\tau\sim(P,\pi^{b},\pi^{-b})}\left[\sum\limits_{h^{\prime}=h}^{H+1}C^{b,h^{\prime}}(s^{h^{\prime}},a^{b,h^{\prime}},a^{-b,h^{\prime}})|s^{h}=s\right]$$

with $a^{b,h^{\prime}}\sim\pi^{b}$, $a^{-b,h^{\prime}}\sim\pi^{-b}$ and $\tau$ is a trajectory of the Markov game, generated by following the joint policy $\pi$. As the Markov game pertaining to this work involves cost minimization as the objective, the optimal policy for any player is to find a policy that minimizes the value function. However, in a multi-agent scenario, when other players act rationally, finding optimal policy is equivalent to finding (Nash) equilibrium solution which is the best response to rational behaviour of other participating agents. Hence, in this paper, for the PDA modelled as a Markov game, we look for MPNE [14, 15] solutions defined as below.

The perfectness of the Nash equilibrium is due to condition that the inequality in Definition (III.1) holds for every round $h\in[H]$ and for every element of the state space $S$. In the sequel, we propose a MPNE solution for the PDA problem described in Section I.

Having elaborated on the Markov game framework, we now describe a joint policy which is an MPNE for the PDA setup considered in this work wherein each buyer is allowed to place one bid per round of the Markov game. Note that here, the goal is to find MPNE in the space of deterministic policies.

Recall from Equations (1) and (2) that $Q^{\mathscr{D},h}$ and $Q^{\mathscr{S},h}$ denote total demand requirement and total supply available at round $h$. At each round $h$, let $[N^{h}]$ denote the set of $N$ players indexed by the decreasing order of their quantity requirement³³3For this work, we assume players quantity requirements are unique. Now, let $u_{h}$ be the index of the ask from the set $\mathcal{L}^{h}$ such that all of the demand requirement at round $h$ is met. That is, $u_{h}=\operatorname*{arg\,min}\nolimits_{j}\left(Q^{\mathscr{D},h}\leq\sum\nolimits_{m=1}^{j}q^{h}_{m}\right)$. At round $h$, denote $Q^{\mathscr{D}_{-b},h}=Q^{\mathscr{D},h}-Q^{b,h},\;b\in[N^{h}]$ as the demand requirement of all players except the player $b$. Let $v^{b}_{h},\;b\in[N^{h}]$ be the lowest index of the ordered set $\mathcal{L}^{h}$ such that the total supply available for the first $v^{b}_{h}$ asks satisfies the demand requirement of all players except $b$. That is, $v^{b}_{h}=\operatorname*{arg\,min}\nolimits_{j}\left(Q^{\mathscr{D}_{-b},h}<\sum\nolimits_{m=1}^{j}q^{h}_{m}\right)\;\forall\;b\in[N^{h}]$. Next, let us define index $z_{h}$ as $z_{h}=u_{h}-(H-h)$. Finally, let $\psi^{h}=\max\{1,\operatorname*{arg\,max}_{j}\{v^{j}_{h}\leq z_{h}\}\}$ as the player who bids $p_{z_{h}}$ and let $\phi^{h}$ as the player with the maximum requirement. Note that $\psi^{h}=\phi^{h}$ when $\psi^{h}=1$.

The joint policy $\pi^{*}$ for a player $b\in[N^{h}]$ at round $h\in[H]$ for state $s^{h}\in S$, can now be formulated as,

$$\pi^{b,h}_{*}(s)=\begin{cases}p^{b,h}=0,\;q^{b,h}=0&\textrm{ if }\;Q^{b,h}=0,\forall\;b\in[N^{h}]\\ \pi_{1}^{b,h}(s)&\textrm{ if }\;H-h\geq u_{h}-v^{\phi}_{h}\\ \pi_{2}^{b,h}(s)&\;\textrm{ Otherwise }\end{cases}$$

(3)

where the policies $\pi_{1}^{b,h}(s)$ and $\pi_{2}^{b,h}(s)$ are defined as,

$\displaystyle\pi^{b,h}_{1}(s)=\begin{cases}p^{b,h}=p_{v^{\phi}_{h}},\;q^{b,h}=Q^{b,h}&\textrm{ if }Q^{b,h}>0,\;b=\phi^{h}\\ p^{b,h}=p_{max},\;q^{b,h}=Q^{b,h}&\textrm{ if }Q^{b,h}>0,\;b\neq\phi^{h}\end{cases}$

$\displaystyle\pi^{b,h}_{2}(s)=\begin{cases}p^{b,h}=p_{z_{h}},\;q^{b,h}=Q^{b,h}&\textrm{ if }Q^{b,h}>0,\;b=\psi^{h}\\ p^{b,h}=p_{max},\;q^{b,h}=Q^{b,h}&\textrm{ if }Q^{b,h}>0,\;b\neq\psi^{h}\end{cases}$

Here, $p_{\max}$ is the maximum possible bid price and is greater than largest possible ask price i.e $p_{\max}>p_{M_{H}}$. The policy in (3) suggests that the player with the highest requirement would wait for other players to get their demand satiated provided there are enough rounds as determined by $H-h\geq u_{h}-v^{\phi}_{h}$. In this case, the player with highest requirement also determines the MCP. However, if there are not enough rounds ($H-h<u_{h}-v^{\phi}_{h}$), then the player $b\neq\psi$ would bid for the whole quantity at the highest possible price and the player ($\psi$) would bid a price that decides the clearing price. In the case when there is only one buyer left in the market, the policy in (3) recommends the player to follow the supply curve.

Having described the joint policy, we now evaluate the value of the policy $\pi_{*}^{b,h}$ at round $h$, for player $b\in[N^{h}]$ at state $s^{h}\in S$. To this end, the MCP $\lambda^{h}$, the total market cleared quantity $Q^{h}$ and the cleared quantity $\alpha^{b,h}$ for a buyer $b$ while adopting the policy $\pi_{*}^{b,h}$ at round $h\in[H]$ for state $s^{h}$ is tabulated in the Lemma below.

Having described the clearing implications for a buyer $b\in[N^{h}]$ for following the policy $\pi_{*}^{b,h}$ of Equation (3) at state $s^{h}\in S$, we now compute the value of the equilibrium policy for a buyer $b\in[N^{h}]$ which follows from Lemma IV.1. When $H-h\geq u_{h}-v^{\phi}_{h}$, we have,

$$V^{h}_{\pi^{b}_{*},\pi^{-b}_{*}}(s)=\begin{cases}\!p_{v^{\phi}_{h}}\times Q^{b,h},\;\;&\textrm{ if }\;b\neq\phi^{h}\\ \begin{aligned} \bigg{[}p_{v^{\phi}_{h}}\times\left(Q^{h}-\sum\nolimits_{b\in[N]\setminus\phi_{h}}q^{b,h}_{j}\right)\\ +\sum\nolimits_{k=h+1}^{H}p_{v^{\phi}_{k}}\times Q^{k}\bigg{]},\end{aligned}&\textrm{ if }\;b=\phi^{h}.\end{cases}$$

(4)

On the other hand, when $H-h<u_{h}-v^{\phi}_{h}$, we have,

$$V^{h}_{\pi^{b}_{*},\pi^{-b}_{*}}(s)=\begin{cases}\!p_{z_{h}}\times Q^{b,h},\;\;&\textrm{ if }\;b\neq\psi^{h}\\ \begin{aligned} \bigg{[}p_{z_{h}}\times\left(Q^{h}-\sum\nolimits_{b\in[N]\setminus\psi^{h}}q^{b,h}_{j}\right)\\ +\sum\nolimits_{k=h+1}^{H}p_{z_{k}}\times Q^{k}\bigg{]},\end{aligned}&\textrm{ if }\;b=\psi^{h}.\end{cases}$$

(5)

In this section, for the PDA considered in this exposition, we show that the policy in (3) is an MPNE in the space of all deterministic policies. More precisely, we need to show that, for all $b\in[N^{h}]$, for all $s\in S$, for all $h\in[H]$ and for any deterministic policy $\pi^{b}:S\rightarrow A_{b}$, we have

$$V^{h}_{\pi_{*}^{b},\pi_{*}^{-b}}(s)\leq V^{h}_{\pi^{b},\pi_{*}^{-b}}(s).$$

Denote the bid of buyer $b\in[N^{h}]$ at state $s$ and round $h$ as prescribed by the policy $\pi^{b,h}_{*}(s)$ as $(p_{*}^{b,h},q_{*}^{b,h})$. Further, recall that each bid of a player $b\in[N^{h}]$ belong to the bounded set $[0,p_{\max}]\times[0,q_{\max}]$ and at any round $h$, the player $b$ does not bid more than the outstanding demand requirement $Q^{b,h}$, the possible deviations available for a player $b$ at a state $s$ and round $h$ can be tabulated as below.

Given these deviations, we now show that, at any state $s$ and at any round $h$, a player $b\in[N^{h}]$ deviating from the policy $\pi^{b,h}_{*}(s)$ (Equation (3)) in any of the ways listed above (Table (II)) will not incur any less expenditure than what is accounted for via the value functions in Equations (4) and (5). To this end, we first provide results that will be used later in the analysis. The first result provides an insight into how the MCP varies across the rounds of a PDA.

Recall that the policy in Equation (3), suggests that $N-1$ players to bid at price $p_{\max}$ and the remaining player to bid at a specified (lower) price. The next result states that any deviations in the bid, by a player recommended to bid at price $p_{\max}$, at any round $h\in[H]$, might reduce the procurement cost.

Note that the policy in (3) is a Markov policy since it only depends on the present state $s$. Moreover, the inequality (6) holding for all $h\in H$ and $s\in S$ implies that the policy satisfies sub-game perfectness.

This section considers a simple numerical setup to demonstrate the efficacy of the Nash policies described in Section IV. Our setup consists of three players (buyers) in the market. The players go through a PDA simulator which has $H=24$ rounds to procure the required quantity. The quantity requirement of the four players (P0, P1, P2) at some round $h\leq H$ is given as $\mathcal{Q}^{h}=(232.18,164.6,90.7)$. The players P0 and P2 are the players with largest and smallest requirement respectively. The players know the supply curve (ask pattern) $\mathcal{L}_{h}$ which has 31 asks and the total supply $Q^{\mathscr{S}}=1502.38>Q^{\mathscr{D},h}=487.48$. We consider two values of $h$, namely $h=1$ and $h=23$, wherein the choice $h=1$ satisfies the condition $H-h\geq u_{h}-v^{\phi}_{h}$ and the latter does not.

Figure  3(a) compares the value function when all players adopt the policy in (3) with a joint policy in which player P0 deviates to a bid price higher than the prescribed price $p_{z_{h}}$ at $h=1$. The higher bid price of P0 results in higher cost because of increased MCP. In Figure  3(b), for $h=23$, the condition $H-h\geq u_{h}-v^{\phi}_{h}$ is not satisfied and hence the prescribed bid price of P0 is $p_{\max}$. When P0 bids less than $p_{\max}$ its bid priority decreases resulting in procurement outside of the PDA at higher cost $\Psi$ thereby increasing the overall cost. Figure  3(c), considers the case $h=24$, wherein the condition $H-h\geq u_{h}-v^{\phi}_{h}$ is not satisfied. Here, we consider the deviation by the minimum requirement player P2 to bid at a price $p\in(p_{v^{\psi}_{h}},p_{z_{h}})$ less than the prescribed price $p_{z_{h}}$. This deviation to a lower price, although results in lower cost of procurement at round $h$, leads to higher overall cost as the player has to buy more units of the commodity outside of the auction at higher price $\Psi\geq\beta\cdot p_{\max}$. Finally, in Figure 3(d), we consider average cost incurred by the players in 100 PDAs (each with $H=24$ rounds) with varying demand requirement. In each of these 100 PDAs, we let player P0 deviate from the prescribed Nash policy to the Zero intelligent (ZI) policy [16] and the corresponding value functions are compared with the value function for the Nash policy.

In this paper, we formulate optimal bidding strategies for a periodic double auction setting consisting of multiple buyers competing with each other to satisfy their respective demand. Each buyer has multiple opportunities to procure their need and the composite supply curve is known to all of them. The problem is modeled as a Markov game and we propose equilibrium solutions that could act as optimal bidding strategies when all buyers behave rationally. Apart from proving that the proposed policies are indeed MPNE, we also conducted simple numerical simulations to demonstrate the efficacy of the proposed solution framework. The PDA set up considered in this paper have applications in devising optimal bidding strategies for day-ahead electricity markets.

Although, in this work, we have considered only the case of adequate supply with one bid per auction per buyer, we believe that the case of multiple bids per auction and inadequate supply can be handled using the techniques developed in this work. Despite the fact that, the equilibrium solutions proposed here are for the complete information setting, they are still important for two reasons. First, as far as we know, ours is the first work to derive analytical equilibrium solutions for multi-shot auctions. Second, these policies can be used as a baseline to compare with a policy that is obtained in an incomplete information setting, which would be a direction of our future work.