MaskPlace: Fast Chip Placement via Reinforced Visual Representation Learning

The scalability and efficiency are two significant factors of autonomous chip layout design. Placement is one of the most challenging and time-consuming problems in the design flow, aiming to determine the locations of millions of circuit modules on a 2D chip canvas represented by a two-dimensional grid. A netlist can describe these modules, that is, a large-scale hypergraph consisting of massive macros (functional blocks such as memory) and standard cells (logic gates), where each macro and each standard cell can contain several or even hundreds of pins connected by wires, as shown in Fig.2.

Placing a large number of circuit modules onto the chip canvas is challenging because many performance metrics such as power consumption, timing, area, and wirelength should be minimized while satisfying some hard constraints such as placement density and routing congestion. For example, the wirelength (the length of wires that connect all modules) determines the delay and the power consumption of a chip [1]. Shorter wires often indicate less delay and less power consumption [2]. However, wirelength cannot be reduced by overlapping modules because the module density is a hard constraint to ensure that a valid and manufacturable chip layout has non-overlapping modules. More examples of the performance metrics are given in Fig.8 and Fig.9 in Appendix. As pointed out in [3], the design space of placement is larger than $10^{2,500}$ when there are just $1,000$ circuit modules, whereas neural architecture search (NAS) typically has a space of $10^{30}$ and the Go game has a state space of $10^{360}$.

Methods of chip placement can be generally divided into two categories, classic optimization-based approaches [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21] and learning-based approaches [3, 22, 23]. In the first category, hardware scientists often formulate placement as an optimization problem and relax the hard constraints. For example, let a pair of vectors $(\bm{x},\bm{y})$ denote the ($x,y$)-coordinate value of all circuit modules on a 2D canvas, the objective function of placement can be formulated as minimizing $\mathrm{WL}(\bm{x},\bm{y})$, subject to $\mathrm{D}(\bm{x},\bm{y})\leq\alpha$, where $\mathrm{WL}(\cdot,\cdot)$ and $\mathrm{D}(\cdot,\cdot)$ are the estimation functions of wirelength and density respectively, and $\mathrm{D}(\bm{x},\bm{y})\leq\alpha$ is a hard constraint with a very small density value $\alpha$, which ensures that all modules do not overlap. For instance, DREAMPlace [9] is a recent advanced method that minimizes $\mathrm{WL}(\bm{x},\bm{y})+\lambda\mathrm{D}(\bm{x},\bm{y})$, which relaxes the hard density constraint. However, it cannot directly produce a valid and manufacturable layout because the non-overlapping constraint is not satisfied after relaxation. These approaches often need a post-processing step, such as manual refinement and legalization (LG), to remove the overlapping in placement, resulting in two issues, (1) the wirelength may increase substantially after LG, and (2) no feasible solution can be found if the available chip area is insufficient before post-processing.

In the second category, reinforcement learning (RL) is employed to solve placement as a sequential decision-making problem, placing each circuit module at a time. Although the learning-based approaches are still in their early stage, they can produce promising results to automate the chip design flow end-to-end significantly without human effort. For instance, Graph Placement [3] and DeepPR [22] represent a netlist as a hypergraph, denoted as $G=(V,E)$, where $V$ represents a set of nodes, and each node is a module, and $E$ is a set of edges, which are the wires connecting all modules. They train RL agents to place one module at a time by maximizing the metric values as rewards. However, the hypergraph is not scalable to comprehensively encode information of a netlist. For example, the relative positions (offsets) of pins are discarded in [3, 22]. The wirelength estimation is inaccurate without the pin information, but encoding this rich information would make the hypergraph too complicated because each module can have hundreds of pins. Furthermore, placement on a large hypergraph requires heavy computations. Mirhoseini et al. [3] reduced computations by placing 15% of the modules using reinforcement learning (the remaining modules are placed by classic method), and Cheng and Yan [22] decreased the size (resolution) of module and chip canvas as shown in Table 1. Both of them sacrificed their placement performance.

To address the issues of prior arts, we propose a novel RL method, named MaskPlace, which can automatically generate a high-quality and valid layout (non-overlapping modules) within a few hours, unlike previous methods that need manual refinement to modify invalid placement, which may wait up to 72 hours for commercial electronic design automation (EDA) tools to evaluate the placement. MaskPlace casts placement as a problem of pixel-level visual representation learning for circuit modules using convolutional neural networks. This representation can comprehensively capture the configurations of thousands of pins, enabling fast placement in a full action space on a large canvas size e.g., 224$\times$224. As shown in Fig.2 and Table 1, MaskPlace has many attractive benefits that existing works do not have. MaskPlace is mainly for macro placement due to the problem size.

This paper has three main contributions. Firstly, we recast chip placement as a problem of learning visual representation to describe millions of circuit modules on a chip comprehensively. It opens up a new perspective for AI-assist chip placement. Secondly, we carefully design a new policy network that can capture and aggregate both the global and subtle information on a chip canvas, maximizing the reward of wirelength and ensuring non-overlapping placement efficiently. Thirdly, extensive experiments demonstrate that MaskPlace outperforms recent advanced methods on 24 public chip benchmarks. For example, MaskPlace can always produce a layout with 0% overlap while reducing wirelength up to 5$\times$ and 9$\times$ compared to Graph Placement [3] and DeepPR [22] respectively.

The placement quality can be measured by the HPWL (half perimeter wirelength), which estimates the wirelength with marginal computational cost [24]. Intuitively, Fig.2(e) illustrates a 2D chip canvas. Let $M^{i}$ and $P^{(i,j)}$ denote the $i$-th module and its $j$-th pin, respectively. A net contains a set of pins connecting modules by wires. For example, “Net 1” (in red) connects all four modules (i.e., $M^{1},M^{2},M^{3},M^{4}$) using wires through pins $P^{(1,2)}$, $P^{(2,2)}$, $P^{(3,2)}$, and $P^{(4,1)}$, while “Net 2” (in green) connects three modules (i.e., $M^{1},M^{2},M^{3}$) using wires through pins $P^{(1,1)}$, $P^{(2,1)}$, and $P^{(3,1)}$. HPWL estimates the wirelength by summing up the half perimeters of bounding boxes of all the nets, as shown by the red and green boxes in Fig.2(e). Intuitively, the half perimeter of a net bounding box equals the sum of its height and width. For example, HPWL in Fig.2(e) is $h_{1}+w_{1}+h_{2}+w_{2}$.

Given a netlist containing a set of nets, minimizing the wirelength can be treated as minimizing HPWL by placing modules to the optimal positions on a 2D chip canvas. To achieve a valid and manufacturable chip layout, we need to satisfy two hard constraints: (1) congestion constraint: the wire congestion should be lower than a desired small threshold to reduce chip cost, and (2) overlap constraint: the density should be minimized to achieve non-overlapping placement.

where $P_{x}$ and $P_{y}$ represent the $(x,y)$-coordinate value of a pin respectively, $\mathrm{Congestion}(\cdot)$ is the congestion function, $C_{\mathrm{th}}$ is a desired threshold, $\mathrm{Overlap}(\cdot)$ is the overlap function, and $M_{x},M_{y},M_{w},M_{h}$ represent the position, width, and height of modules respectively. Firstly, lower congestion often indicates shorter wirelength, which is crucial to reduce chip cost because the wire resources are limited on a real chip. Inspired by prior arts [3, 22], we employ the RUDY estimator [25] to estimate wire congestion. Details of RUDY can be found in the Appendix A.2. Secondly, the placement density calculates the overlapping region between every pair of circuit modules. It is time-consuming since its computational complexity is $\mathcal{O}(V^{2})$ where $V$ is the number of modules [1]. The proposed approach can ensure non-overlapping placement to avoid calculating this density metric explicitly in training, thus reducing computations while producing a valid layout.

Model Architecture Overview. Chip placement can be formulated as a Markov Decision Process (MDP) [26] by placing each module at a time. Fig.4 illustrates the overall architecture of MaskPlace, which trains a policy $\pi_{\theta}(a_{t}|s_{t})$ represented by a convolutional encoder-decoder network with parameter set $\theta$, and a value function $V_{\phi}(s_{t})$ represented by an embedding model with parameter set $\phi$. The policy network receives previous observations and actions as input $s_{t}$ and selects an action $a_{t}$ as output. Specifically, $s_{t}$ is a set of pixel-level feature maps that comprehensively capture the net and pin configurations in $M^{1:t-1}$, $M^{t}$, and $M^{t+1}$, where $M^{1:t-1}$ denotes the modules that have been placed in the previous time steps from $1$ to $t-1$, while $M^{t}$ and $M^{t+1}$ denote the modules to be placed at the current step $t$ and the next step $t+1$, respectively. Intuitively, MaskPlace looks one step forward to achieve better placement.

Although prior arts [3, 22] represented a netlist as a hypergraph as shown in Fig.2(f) where each node is a module, and each edge is a wire between two modules, they lost the information of pin offsets for each module. Unlike previous works, MaskPlace can fully represent massive net and pin configurations using three types of pixel-level feature maps, as shown in Fig.2(a-d), including position mask, wire mask, and view mask, as discussed below. Different masks are fused by convolutions to learn the state representation.

Position Mask. The position mask, denoted by $f_{p}\in\{0,1\}^{224\times 224}$, is a binary matrix of a canvas grid with size $224\times 224$ as shown in Fig.3, where value “$1$” means a feasible position to place a module. The purpose of the position mask is to guarantee no overlaps between modules (i.e., satisfy the overlap constraint) and to learn the relationship between placement and wirelength. Specifically, we slide a module $M^{t}$ (for example, $t=5$) on the entire chip canvas. The trajectory of the feasible positions (in green) can be labeled with “1”. Intuitively, we can check each position for each module using the cumulative sum array [27]. This naive approach has the computational complexity of $\mathcal{O}(N^{2})$ when a 2D canvas grid is divided into $N\times N$ cells. However, this simple approach is not efficient when $N$ is large. Therefore, since all modules are rectangles, we design an efficient generation algorithm, which iterates through all placed modules (in blue) and excludes positions that will cause overlap. In this case, all remaining positions are available for placement. The new algorithm is summarized in Appendix A.3, which costs $\mathcal{O}(V)$ for each module, where $V$ is the number of modules.

Wire Mask. The wire mask, denoted as $f_{w}\in[0,1]^{224\times 224}$, is a continuous matrix for representing how HPWL increases if we place a module $M^{t}$ in a specific position. Fig.5 shows a sample of wire mask, where each value means the increase of HPWL. The wire mask aims at finding the best position with the minimum increase of the wirelength. Intuitively, we can calculate the HPWL at each canvas position, leading to a complexity of $\mathcal{O}(N^{2}P)$, where $P$ is the total number of pins. However, a fast algorithm can be designed by considering the relationships between the pin offset, the net bounding box, and the linear property of the HPWL metric. For example, Fig.3 illustrates that the next module $M^{5}$ has two pins, $P^{(5,1)}$ and $P^{(5,2)}$, belonging to “Net 1” and “Net 2” respectively (Fig.2(e)). Fig.5 illustrates the increase of wirelength when placing $M^{5}$ at each canvas location. For instance, if $M^{5}$ is at the bottom-left corner, its Manhattan distance to the two net bounding boxes (in red and green) is $2+2=4$. To calculate the Manhattan distance more accurately, we move the net bounding box compared to the pin location. For example, since $P^{(5,2)}$ is located at $(2,1)$³³3We index the bottom-left corner as the origin $(0,0)$ in a two-dimensional coordinate., we move the bounding box of Net 2 (in green) in the direction $-\Delta^{(5,2)}=(-2,-1)$ to encode the information of pin offset. The time complexity can be reduced to $\mathcal{O}(NP)$. The algorithm can be found in Appendix A.3.

View Mask. The view mask, denoted as $f_{v}\in\{0,1\}^{224\times 224}$, is a global observation of the current chip layout, where the value “$1$” means a module has occupied this grid cell. Different from DeepPR [22] that assumed all modules have unit size, we consider real sizes of modules. For instance, if a module has size $w\times h$, it covers $\lceil wN/W\rceil\times\lceil hN/H\rceil$ cells in the canvas, where $W$ and $H$ represent the canvas size and $\lceil\cdot\rceil$ denotes the ceiling function.

Learning Algorithm. We train different blocks in Fig.4 as a whole using reinforcement learning. The detailed network architectures are provided in Appendix A.4. Firstly, we apply the above masks to represent the entire circuits and feed them to downstream networks. Secondly, a global feature encoder embeds the view mask of current placement and the wire masks of the following two steps into an embedding vector. Then we combine it with the positional embedding of the $t$-th circuit module in the value network to generate a scalar to evaluate the current state by fully-connected layers. Thirdly, a global mask decoder recovers a feature map of size $N^{2}$, which is fused with different position masks and wire masks in the policy network using $1\times 1$ convolutions to avoid the local signal diffusion. The policy network predicts a probability action matrix of size $N\times N$, indicating where to put the next module. Before sampling actions, we can remove unfeasible actions using the position mask. Finally, the congestion satisfaction block applies the congestion threshold on the probability matrix to select a final action.

Reinforcement Learning. We borrow the representative actor-critic diagram [28] and PPO2 framework [29] to train the policy $\pi_{\theta}(a_{t}|s_{t})$, where the state representation $s_{t}$ is listed in Table 11 in Appendix. The action $a_{t}$ is the canvas position (cell) to place the circuit module. Specifically, we treat the chip canvas as a grid and divide it into $N\times N$ cells, leading to $N^{2}$ possible actions. The objective function of the policy network can be formulated as

where the ratio $r_{t}(\theta)=\frac{\pi_{\theta}(a_{t}|s_{t})}{\pi_{\theta_{\mathrm{old}}}(a_{t}|s_{t})}$ and $\hat{A}_{t}=G_{t}-\hat{V}_{t}$ denotes the advantage function. We employ $G_{t}=\sum_{k=0}^{V-t-1}\gamma^{k}r_{t+k+1}$ that is the cumulative discounted reward and $\hat{V}_{t}$ is the estimated value produced by the value network. We update the the value network by optimizing its objective, $L_{\mathrm{value}}(\phi)=\hat{\mathbb{E}}\big{[}(G_{t}-\hat{V}_{t})^{2}\big{]}$.

Reward $r_{t}$. We treat HPWL as the reward because wirelength is the main optimization target in different performance metrics. This is different from prior arts [3, 22] that weighted combines HPWL and congestion as the reward, which introduces the weighting coefficient as an extra hyper-parameter to tune. Specifically, we achieve a dense reward by defining a partial HPWL, which only computes the currently placed pins. For example, the partial HPWL for $t$ modules can be defined as $\mathrm{HPWL}_{t}$. In other words, we compute $\mathrm{HPWL}_{t}$ after taking action $a_{t}$. The reward for the step $t$ is $r_{t}=\mathrm{HPWL}_{t-1}-\mathrm{HPWL}_{t}$, i.e., the opposite number of the increase of HPWL. Furthermore, instead of computing HPWL at each step, we can maintain the ranges of all net bounding boxes in one episode and update the changes of their sizes with a cost of $\mathcal{O}(P)$, where $P$ is the number of pins. Thus we can generate dense rewards while maintaining efficiency.

Training and Testing. Before training, we follow previous work [3] to sort the circuit modules according to the number of nets, areas, and the number of connected modules that have been placed to determine the place order. In training, we update the policy and value networks at each epoch by ignoring the congestion satisfaction block. When updating the value network, we stop the gradient back-propagated in the global mask encoder to avoid influence on the policy network. The detailed training setup is provided in Appendix A.5.

In the testing stage, for each step $t$, we obtain a probability matrix from the policy network and sample one place action $a_{t}$. Then, the congestion satisfaction block will check whether the congestion exceeds a threshold $C_{\mathrm{th}}$ after applying this action. If it exceeds, we uniformly sample a few actions, look up the corresponding values from the wire mask $f_{w}^{t}$, and estimate the congestion before taking these actions. We choose the action with the minimal value in $f_{w}^{t}$ and the congestion lower than $C_{\mathrm{th}}$. If all actions cannot satisfy congestion $C_{\mathrm{th}}$, we select the action with the minimal congestion and move to the next step. Detailed congestion satisfaction algorithm can be seen in Appendix A.3.

We extensively evaluate MaskPlace and compare it with several recent advanced placement methods, including NTUPlace3 [6], RePlAce [8], DREAMPlace [9], Graph Placement [3], and DeepPR [22], where NTUPlace3, RePlAce and DREAMPlace are optimization-based methods, whilst Graph Placement and DeepPR are learning-based approaches. All of them are evaluated on different public circuit benchmarks. All previous works are evaluated by following their experimental settings.

Benchmark. We evaluate MaskPlace in 24 circuit benchmarks selected from public datasets including the widely-used ISPD2005 [30], IBM benchmark suite [31], and Ariane RISC-V CPU design [32]. The number of evaluated benchmarks is three times more than previous work [9, 22, 3]. The statistics of benchmarks are given in Table 14 in Appendix A.6, where the largest circuit contains 1,293 macros, 22,802 pins, and more than a million standard cells, leading to a vast state space as aforementioned.

Main Results. Table 2 compares the HPWL results between all the above methods to place all macros. To enable a fair comparison, we evaluate all approaches⁴⁴4The random seed does not apply in a classic method NTUPlace3. by using five random seeds and report the means and variances. Since the original DeepPR method did not capture macro size (thus does not avoid overlap between adjacent macros because all macros have unit size), we extend DeepPR to model macro size to reduce the overlapping ratio. We name it “DeepPR-no-overlap”. Similar to prior works, we use the minimum spanning tree algorithm [33] to estimate routing wirelength [34]. From Table 2, we can see that MaskPlace achieves the lowest wirelength in six out of seven benchmarks (except “adaptec4” where it still outperforms all learning-centric methods). We also see that the conventional optimization-based approaches may fail when the circuit benchmark has high chip area usage, such as “bigblue3 ” and “ariane”. Also, MaskPlace gets the lowest wirelength compared with Graph Placement and simulated annealing [35] in the IBM benchmark, which is shown in Appendix A.7. This project website⁵⁵5laiyao1.github.io/maskplace visualizes and compares different placements.

Compare to Graph Representation. Since Graph Placement [3] is the recent advanced learning-based approach that employs hypergraph for placement, we compare MaskPlace with it in all four performance metrics, including HPWL, congestion, density, and overlap area ratio. Table 3 and 4 report the results. The overlap area ratio describes the ratio of the overlapping area between macros divided by the chip area. In Table 3, MaskPlace (soft constraint) means that the round function rather than the ceiling function is used to calculate the number of grid cells occupied by the placed macros. MaskPlace with soft constraints may produce better HPWL and congestion than its counterpart with hard constraints, but the overlap area ratio could not be zero because the constraints have been relaxed. From Table 3 and 4, we see that MaskPlace outperforms graph placement by large margins, especially in the ISPD benchmark, where the former reduces HPWL compared to the latter one by up to 80% while ensuring zero overlaps in all benchmarks. More results in the IBM benchmark can be found in Appendix Table 15.

Routing Wirelength. Table 5 compares the routing wirelength between MaskPlace and DeepPR [22]. We show that using the true wirelength as the reward would lower the efficiency and produce a sparse reward. We see that MaskPlace, which employs HPWL as the reward, can achieve 60% to 90% shorter routing wirelength than DeepPR, which directly used the true wirelength as the reward.

Standard Cells. Table 6 compares the HPWL of both the macros and the standard cells by using MaskPlace, DeepPR [22], and DREAMPlace [9], where DREAMPlace is employed to place the standard cells following the experimental setup in [22]. We can see that the proposed method surpasses the other approaches by up to 50% in the large benchmark “bigblue3”, which has more than a million standard cells.

This paper proposes MaskPlace, an RL-based placement method based on rich visual representation by learning position, wirelength, and view information. It helps the model take action effectively and efficiently without reducing the search space. We design a direct reward function based on practical scenarios and get satisfactory results on all key metrics. This work can facilitate the placement process and avoid undesired overlaps between modules. In the future, we will explore the standard cell placement by designing a suitable representation, which is an open problem for RL due to its vast space.

Limitation and Potential Negative Societal Impact. Chip design flow contains many stages, and our method shows its potential in a single stage. Similar to previous RL methods, it also requires an optimization method when placing millions of standard cells because RL’s state space is too large. Our method does not have potential harm to the public society at the moment.

We thank Xibo Sun answering questions about EDA. We also thank Runjian Chen for participating in our discussion. Ping Luo is supported by the General Research Fund of HK No.27208720, No.17212120, and No.17200622.

1. 1.

   For all authors…

   1. (a)

      Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? [Yes]

   2. (b)

      Did you describe the limitations of your work? [Yes]

   3. (c)

      Did you discuss any potential negative societal impacts of your work? [Yes]

   4. (d)

      Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]

2. 2.

   If you are including theoretical results…

   1. (a)

      Did you state the full set of assumptions of all theoretical results? [N/A]

   2. (b)

      Did you include complete proofs of all theoretical results? [N/A]

3. 3.

   If you ran experiments…

   1. (a)

      Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes]

   2. (b)

      Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]

   3. (c)

      Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes]

   4. (d)

      Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]

4. 4.

   If you are using existing assets (e.g., code, data, models) or curating/releasing new assets…

   1. (a)

      If your work uses existing assets, did you cite the creators? [Yes]

   2. (b)

      Did you mention the license of the assets? [N/A]

   3. (c)

      Did you include any new assets either in the supplemental material or as a URL? [Yes]

   4. (d)

      Did you discuss whether and how consent was obtained from people whose data you’re using/curating? [N/A]

   5. (e)

      Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]

5. 5.

   If you used crowdsourcing or conducted research with human subjects…

   1. (a)

      Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]

   2. (b)

      Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]

   3. (c)

      Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]