Effective Scaling of Blockchain Beyond Consensus Innovations and Moore’s Law

In Fig. 2a, we illustrate the confirmation latency (that is, the time interval for irreversibly confirming one transaction) of the leading blockchain designs. Considering that the bases used to judge "irreversibility" have differences, we classify the involved designs into two major categories: structure-oriented and Byzantine Fault Tolerance (BFT)-based ones, both of which contain plentiful derivatives.

The structure-oriented blockchain appeared first and stemmed from Bitcoin, which confirms transactions via the Nakamoto consensus. In this case, local ledger extends linearly and only the transactions on the longest chain among multiple conflicting forks will be confirmed. To guarantee the irreversibility of their transaction confirmations in such a competitive environment, clients wait for the proof of 6 subsequent blocks [18]. Given that each block creation roughly takes 600s, the latency fluctuates around 3600s. Since Bitcoin Cash [19] and Counterparty [20] share an equivalent architecture with Bitcoin, the confirmation latency has remained constant for several years. In 2015, Ethereum was developed, using Greedy Heaviest-Observed Sub-Tree (GHOST) protocol to accommodate an increased block-creation rate (15s per block) [21]. Accordingly, chain structure evolved from linearity to parental tree and the capability of forks was exploited. Then, the confirmation latency dramatically declined to less than 570s. BitShares 2.0 and EOS, emerging between 2015 and 2018, substituted permissionless competitions for geo-ordered block creations by permissioned block producers (BPs) [22]. Reduced amount of competitors and pre-configured consensus workflows are conducive to accelerating the process of reaching irreversibility. Thus, confirmations in BitShares 2.0 and EOS cost approximately 45s and 198s, respectively, outperforming the predecessors again. Since then, the improvement on latency stalled, even though IOTA further upgraded tree-structured ledger to a Directed Acyclic Graph (DAG) [23]. This is due to the fact that the structure-oriented designs view ledger extensions as the confirmation process, which is time-consuming and cannot be thoroughly alleviated through consensus innovations.

Under BFT-based designs, pending transactions call for commitments from no less than 2f + 1 peers (replicas), where f represents the sum of attackers in P2P network [24]. In Fig. 4a, we observe that the confirmation latency of BFT-based designs varies between 3-10s, with a slight drop over time.

Note that the confidence coefficient of "irreversibility" also influences clients’ judgements. If the submitted transaction only carries micro currency, appropriately relaxing the confidence interval could simplify the confirmation process without a risk of serious losses, for instance, zero-confirmation for Bitcoin. With respect to the standard of latency, Karame et al. [25] stressed that the validity of each pending transaction would better be determined within 30s, termed as fast payments. Although it seems that BFT-based designs have outperformed such a baseline, workloads of novel applications sparked by advanced techniques increase dramatically, leaving ever-decreasing time for confirmations. For example, the average latency of 5G-driven applications is less than 100ms [26]. Moreover, centralized counterparties, like Alipay, have already achieved instant confirmation. All the designs involved in Fig. 2a use the strategy of consensus-based scaling, which is the most widely-adopted scaling method. As both trends tend to slow down, we conclude that deficiency exists with respect to the confirmation latency.

Similarly, Fig. 2b depicts the trends of peak throughput (that is, the maximum ability of processing concurrent transactions per second), following the leading blockchain designs. Note that we compare the block creation strategies during the consensus workflows and thus classify the above designs into different categories.

From 2009 to 2015, Bitcoin and its imitators (called Altcoins) dominated the market and their consensus mechanisms are diverse versions of Proof-of-Work (PoW). Under PoW, the validity of blocks depends on creators’ workload of hash collisions trying to figure out tough cryptography puzzles, named mining [27]. Since all miners in the P2P network simultaneously commit massive resources into mining while only one can win the competition and create a block in each round, PoW is characterized by huge resource waste and low performance. Even though subsequent researchers revised standard PoW, such as by adopting the indicator "stake" to mitigate the computing power usage, thus proposing so-called hybrid consensus, PoW-based designs still hold weak throughput [28, 29] (from 3.3-6.67 TPS for Bitcoin to about 250 TPS for IOTA).

To reduce the resource overhead, researchers have carried out relentless efforts for redesigning lightweight consensus. BFT, initially aiming to achieve fault tolerance in distributed systems, conducts byzantine-style voting to create blocks. Since such process achieve higher time- and resource-efficiency, the peak throughput of BFT-based designs enters in the 1000 TPS-1500 TPS range.

After 2015, Graphene aroused widespread interest because this framework drastically stimulated the throughput to more than 3500 TPS through employing BP. For example, BPs of EOS take turns to create one batch containing 6 micro-blocks, after which they broadcast the proposals to neighbouring BPs within 3s [22]. Graphene-based designs are seriously questionable in terms of decentralization and security due to the prior configurations of permissions, geographic orders, even block-creation rates. That being said, authoritative BPs make EOS almost reach the theoretical upper bounds of throughput for current blockchain techniques, where the transaction processing speed over the whole P2P network is only capped at that of BPs. However, the overall throughput of blockchain still displays a gigantic, widening gap compared to the application demand or centralized counterparties. As shown in Fig. 2b, Alipay and Visa both bypassed 50000 TPS around 2015 [30]. Through strengthening datacenters, this indicator is growing by a factor of two per year. Conversely, consensus-based scaling no longer offers help, as all three trends flatten out. Given that innovations from other perspectives, such as multi-chain topology, are still in their infancy, it is clear that deficiency also exists in throughput.

The insights from Fig. 2 demonstrate that 10 years of consensus-based scaling fails to address the skyrocketing demand on performance. We focus on latency and throughput only because they are the most important performance metrics for blockchain. Recently, Facebook presented its financial infrastructure named Libra [33], whose consensus mechanism is Libra-BFT. Since the announced throughput of Libra-BFT (1000 TPS) is in line with our predictions for BFT-based designs, the accuracy of Fig. 2 gets further verified.

As we mentioned before, scalability reflects the performance tendency as more participants join in the blockchain network. To visualize the scalability of mainstream blockchain designs, Fig. 3 shows their throughput and latency versus the number of full nodes they can support at most. In addition, we take the validity of well-known scaling proposals, primarily those which underwent practical benchmarks, into consideration. In Fig. 3, we first analyse the items located in Areas 1-3.

Practical BFT (PBFT)-based designs within Area 1 excel in performance. In theoretical cases, they obtain a throughput over 10000 TPS and a latency only bounded by bandwidth. However, PBFT splits the consensus workflows into three successive phases, each of which executes network-wide multicast to collect commitments. For the P2P network composed of $n$ full nodes, the communication complexity required for one round of ledger extension is $O(n^{2})$ [24]. Despite an outstanding throughput in small-scale networks, such superiority drastically decays as more full nodes participate. Hence, the adaptability of PBFT-based blockchain is severely undermined, in which the P2P network can only accommodate dozens of (in [34], 16) peers.

Conversely, the throughput of PoW-based items marked in Area 2 is generally less than 2000 TPS and the latency keeps in the order of minutes. Although the huge resource consumption greatly affects the performance, this set of blockchain is capable of retaining stability when the network expands to thousands of full nodes. The corresponding communication complexity is just $O(n)$, or even $O(1)$ [8]. Such robustness enables blockchain to meet the requirements of not only small-scale but also globally-distributed deployments.

Some proposals within Area 3, mostly belonging to PBFT’s variations, seem to better balance between the performance and network scale. Unfortunately, the majority of them are short of practical benchmarks in large-scale network environments and thus their scalability is subject to further research [31]. Clearly, the blockchain within Areas 1-3 has distinct deficiencies in this metric.

Following three scaling trends in Fig. 3, we observe that the items within Area 4 obtain satisfying optimizations in regard to both performance and the network scale. Unlike the aforementioned techniques whose scaling relies on consensus innovations, the items in this area adopt well-studied consensus mechanisms, such as PoW and PBFT, while rebuilding the network topology. In particular, participants are clustered for maintaining multiple chains in parallel, instead of placing resources into one single chain. In this way, the large-scale network can be parallelized into smaller entities (side-chain or shard), which independently allocate resources [35]. With the cross-chain/shard communication mechanisms, peers from different entities are still accessible to exchange information. In this solution, performance of the entire blockchain network can rise positively with the entity number. For instance, Zilliqa clustered 600 peers in one shard then reached [1218, 1752, 2488] TPS in the P2P network with a scale of [1800, 2400, 3600] full nodes, respectively [36]. We reckon that topology-based scaling is able to overcome the roadblocks that impede current consensus-based scaling and provide a novel path towards effectively scaling up blockchain in the future.

Offloading workloads. Off-chain is a classic technique which enhances network concurrency by constructing channels [63]. In traditional on-chain topologies, myriad transaction confirmations concentrate on the blockchain network with limited capacity. Off-chain networks, in contrast, build independent and long-lived channels over which an arbitrary amount of transactions can be processed locally between individuals. The blockchain is contacted only when peers initialize and close channels, or the involved parties encounter conflicts, in which case the preset contracts will handle fair settlement. This topology therefore effectively offloads the vast majority of blockchain network’s workloads, accommodating more channels and peers. Simultaneously, both sides of channels are free of synchronizing network-wide message or participating in consensus, so the resource efficiency and performance at the application layer are significantly improved. Viewed from blockchain network, however, the resource efficiency keeps unchanged because of no architecture innovations.

The well-known Bitcoin Lightning network [64] and duplex micropayment channel [65] first explored off-chain to overcome the scalability bottleneck. Furthermore, Dziembowski et al. [66] presented the concept of virtual payment channel, which introduced smart contracts to bypass undependable intermediaries when processing cross-channel payments. In order to support general applications rather than cryptocurrencies, they further polished their proposal by building channels recursively and successfully constructed the state channel allowing arbitrarily complex smart contracts to execute [67]. All these designs achieved promising performance and a potential far higher than consensus-based scaling. Moreover, the security of off-chain technology has been elaborated in detail [68, 69]. Some researchers even claimed that their proposals can sustain dependability when the peers at both sides of one channel are malicious.

Enabling interoperability. Parallel-chain techniques, including side-chain, cross-chain, and child-chain, intend to enable the interoperability among multiple blockchains. This concept was first proposed by Back et al. [70], whose original goal was to build isolated side-chains for conducting low-cost experiments since the risk of arbitrary optimizations on the parent-chain (in [70], Bitcoin) is unbearable. Both side- and parent-chain are completely independent, which means they are maintained by different sets of peers. Through cross-chain communication mechanisms, like two-way peg protocols, numerous side-chains can be interconnected and allow for secure data transfers. From the perspective of transaction processing, parallel-chain partitions the workloads that previously belong to one single chain [71]. Suppose that each side-chain could process $\varphi$ transactions within 1s, the throughput of the whole blockchain network containing $\omega$ parallel chains reaches $\varphi\,\times\,\omega$ TPS, immensely lifting up the scalability. The child-chain is a hierarchical form of parallel-chain wherein various child-chains process the concurrent transactions, while the parent-chain completes the confirmations by verifying Merkle Tree roots [72].

Nano [73] and Chainweb [74] explored parallel-chain in the industrial and academic fields, both of which showed outstanding superiority over traditional single-chain systems, especially in throughput. Going forward, parallel-chain could switch the consensus mechanisms to adapt to different application scenarios, for example, adopting PoW for payment platforms and PoET for supply chains. As a result, the flexibility of blockchain likewise experiences significant growth. Since the parallel-chain scaling avoids altering the consensus workflows of the parent-chain, it is particularly suitable for the projects which have been put into widespread practical deployments. Considering that such projects have attracted considerable capital, any scaling attempt should be handled very conservatively.

Partitioning network. After 2016, the combinations of sharding techniques with blockchain have received in-depth research. The representative designs include NUS’s ELASTICO [75], EPFL’s OmniLedger [76], Visa’s RapidChain [77], and NUS’s TEE-assisted sharding system supporting general applications [78]. In blockchain, sharding-based scaling is realized by partitioning or parallelizing the heavy workloads among multiple smaller committees (shards). Accordingly, the basic unit of consensus and resource allocation changes to shards, where a reasonably small number of interconnected peers merely synchronize and store data within their own shard. Each shard can be regarded as an independent and fully functional blockchain, which processes disjoint sets of transactions without supervision from any trusted administrators. Thus, either for a single peer or the entire P2P network, the resource efficiency is maximized. An important breakthrough made through full sharding is that blockchain first attains the capability of linearly improving performance with the rising number of peers, thereby achieving an equivalent scalability as Visa.

Innovations of sharding designs concentrate on three key mechanisms: shard formation (initialization and reconfiguration), intra-shard consensus, and cross-shard communication. Since shard formation directly determines the intra-shard security, for example, the proportion of malicious peers in the newly generated shards, this process is required to be bias-resistant. Otherwise, attackers would get a great opportunity to gather in the same shard. ELASTICO [75], the first sharding-driven blockchain, leveraged PoW-based seeds as the randomness to assign peers into corresponding shards, which was proved to have hidden dangers. Motivated by this, Kokoris-Kogias et al. [76], Dang et al. [78], and Zamani et al. [77] proposed various randomness generation mechanisms based on RandHound protocol, TEE, and verifiable secret sharing protocol, respectively, to ensure the complete resistance towards bias.

Moreover, the intra-shard security decays over time because attackers tend to move towards those vulnerable shards, especially for the designs whose shard size is small. In order to hold high-security, sharding systems have to frequently reassign peers, named reconfiguration, which is undoubtedly unfavorable. Extending the reconfiguration interval needs to be specifically designed according to different intra-shard consensus. For PoW-driven shards, the Chu-ko-nu mining presented in Monoxide weakened the influence of high-end peers by diluting their computing power to multiple shards without affecting the resource efficiency [32]. Similarly, RapidChain refined the synchronous consensus protocol of Ren et al. and enhanced the robustness of BFT-driven shards [79]. PoW and BFT are the intra-shard consensus mechanisms commonly used in recent sharding designs. This is because they are well-proven and display excellent stability, and can thus be deployed in numerous parallel shards.

Apart from shard formation and intra-shard consensus, the last concern is about cross-shard communication. In sharding designs, atomicity, isolation, and liveness are indispensable properties for securely processing pending transactions whose inputs/outputs belong to multiple shards. Atomicity demands that cross-shard transactions ought to be either confirmed or aborted by all shards, otherwise the funds will be persistently locked. Similarly, isolation is defined as the independence and serializability of validating each cross-shard transaction owning multiple input/outputs. As the first work that achieves atomicity, OmniLedger used active clients as the coordinators between input- and output-shards. However, if the delegated clients are malicious, OmniLedger will fall into the denial of service and fail to process pending transactions within a finite latency [78]. Liveness just refers to the systems’ resistance towards such conditions. Fortunately, by combining two-phase protocols with BFT, Dang et al. [78] recently have addressed the concerns about isolation and liveness.

Trade-offs. Despite the enormous potential displayed from topology-based scaling, two trade-offs are yet to be judged. Recall that this approach scales up traditional single-chain designs via building channels, interconnecting isolated blockchains, or partitioning networks; the first trade-off is between the interoperability and complexity attributed to cross-chain/channel communication. After topology-based scaling, in most cases, the "one chain, one asset" maxim of blockchain industry will be destroyed [70]. Meanwhile, advanced topologies inevitably produce extra resource overhead. For example, Back et al. [70] adopted complicated two-phase protocols for secure transfers among heterogeneous parallel-chain, which might offset the increase in resource efficiency. Nonetheless, the overall benefits of cross-chain communication outweigh their potential risks, let alone the central role of interoperability for the future blockchain applications [80]. Therefore, this trade-off does not seem to be so confusing.

However, the second trade-off exists between the increased scalability and the decline in security and decentralization, as noted earlier, the "trilemma". In view of the finiteness of available resources and their growth rates owned by blockchain networks, simultaneously improving scalability, decentralization, and security is impossible. When developing Bitcoin, Nakamoto gave the top priority to absolute security, decentralization, and the robustness to accommodate thousands of peers, while he temporarily shelved the scalability by choosing a resource-intensive, low-performance PoW. The superb security just explains why PoW could support more than 90% of the worldwide digital assets [62]. Graphene, in contrast, attains a performance leap with the assistance of authoritative BPs; thus, EOS network exhibits a certain degree of partial decentralization.

Although researchers repeatedly emphasized their efforts towards holding security and decentralization, regardless of off-chain, parallel-chain, or sharding, most topology-based scaling designs still employed certain authoritative entities to empower some complex mechanisms. These entities include off-chain channels, final committee in ELASTICO, reference committee in RapidChain, and clients in OmniLedger. Additionally, the multi-chain networks after scaling are inevitably not as resistant to attackers as their prototypes, wherein all available resources are invested to protect only one chain. Nevertheless, if we arbitrarily refuse topology-based scaling because of several flaws, blockchain might sidestep a perfect chance for scaling from the order of magnitude.

Envisioning the potential of topology-based scaling, we advise that users could accept a slight attenuation of decentralization to embrace high-quality blockchain services, as Bitcoin also encounters similar issues from the Matthew effect. Meanwhile, researchers should keep on innovating to guarantee the security of multi-chain networks, eliminate the dependence on authoritative entities, and simplify cross-chain communication.

Accelerating time-consuming executions. Observing the intensive time and resource consumption of blockchain, many works have tried to accelerate or lighten certain operations being frequently performed. Although fundamentally overcoming the scalability bottleneck seems impossible, hardware-assisted scaling excels in feasibility because it mainly rebuilds the application or consensus layer, while retaining the network topology. In effect, customized FPGAs have attracted widespread interests for accelerating neural networks, machine learning, and blockchain [81, 82].

In most cases, FPGA accelerators are designed based on FPGA Network Interface Card (NIC) and connected to the host CPU (full nodes) via Peripheral Component Interconnect Express (PCIe). Since FPGA integrates NIC, RAM modules, and ALUs in one chip, the resource efficiency is much higher than that of standard CPUs. Furthermore, compared with CPU, FPGA can process more requests in one batch because of its stronger capability in parallel computing, which is vital for handling massive threads. Recent work has shown that employing FPGA accelerators instead of host CPUs for responding to outside requests can effectively offload the workloads for peers. Sakakibara et al. [44] proposed an FPGA-based NIC, which collaborated with off-chain channels in rapidly transferring digital assets without interacting with blockchains. Sanka et al. [83] designed an FPGA platform integrating hash tables and several hash cores for caching Key-Value Store (KVS)-styled data, achieving a 103-fold improvement in response speed over CPU storage. With the assistance from FPGA, full nodes communicate with clients only when the required data or commands are not satisfied by accelerators, thereby saving considerable resources for executing blockchain operations. As to outside clients, the throughput and latency of interacting with full nodes are optimized correspondingly, that is, a higher quality of service. Apart from FPGA, other hardware can also serve as blockchain accelerators. For example, Morishima et al. [84] applied GPUs in the local Patricia tree-structured KVS memory to reduce the time of searching metadata. Note that the KVS which is widely adopted in blockchain can combine hash tables or be organized as tree structures to adapt to FPGAs and GPUs, thereby improving the speed of data access [85, 86]. Luo et al. [87] presented a tracetrack memory based in-memory computing scheme for conducting encryption/decryption in blockchain, which achieved 8 folds of throughput per area compared with the state-of-the-art CMOS-based implementation. As to ASICs, considering the high development cost, they are temporarily unsuitable to accelerate blockchain operations except mining.

Lightening resource-intensive mechanisms. PBFT and its variants achieve high throughput (ideally, more than 10000 TPS) since the computation-intensive mining is abolished. Instead, the three-phase multicast is used to atomically exchange messages, which leads to a communication complexity of $O(n^{2})$ and cannot accommodate enough peers of public-chain applications. Therefore, researchers have long tried to perfect BFT and later realized that a feasible approach is to apply a hybrid fault model — some components are trusted and only fail by crashing, while the others remain BFT and are possible to launch any malicious attacks. Following this strategy, the three-phase multicast protocols could be lightened into two-phase ones, thereby eliminating the bottleneck on TPND that hinders BFT. Note that we exclude the hybrid fault model from consensus-based scaling because we merely discuss pure fault model in that part. The premise of hybrid fault model is to ensure the security and validity of trusted subsystems, so it calls for the assistance from trusted hardware technique. As early as 2004, Correia et al. [88] applied Trusted Timely Computing Base (TTCB), the tamper-proof and interconnected components distributed in host CPUs, to provide trusted ordering services for BFT. Chun et al. [89] presented two consensus mechanisms based on Attested Append-Only Memory (A2M), building a trusted log that makes any tampering attempts detectable.

However, A2M needs to store all certified messages as a part of the subsystem — a prominent defect [90]. The principal goal behind hybrid fault models is to control the size and complexity of protected components since the larger the trusted code is, the more the potential vulnerabilities expose. Following this principle, Kapitza et al. [41] implemented a small trusted component on FPGA-based platform, for improving the security and saving resources. MinBFT [91] and TrInc [92] employed trusted counters for attestations or to associate sequence numbers to each operation, making the protocol simpler and easier to deploy. TEE technique, for instance, Intel SGX, ARM TrustZone, and Sanctum has also been explored to create trusted subsystems, and gradually become prevalent. Behl et al. [90] proposed a highly parallelizable and formal hybrid consensus mechanism named Hybster, which used SGX to multiply the trusted subsystems. Dang et al. [78] installed A2M in SGX for optimizing PBFT and generated bias-resistant randomness for shard formations. Moreover, Liu et al. [93] developed a resource aggregation algorithm that combined TEE with lightweight secret sharing to reduce the communication complexity to $O(n)$.

These above works can be seen as the forerunners of the scaling attempts which no longer rely on refining consensus, while aiming to consume resource in a more efficient manner. Extensive experiments and deductions demonstrate the potentials of such proposals. However, they are still in an early stage of development since the statistics are merely staying in laboratories or private benchmarks.