Efficient Blockchain Network Simulation: A Closer Look

VIBES: SIMBA: An Efficient Simulator for Blockchain Applications SimBlock: A Blockchain Network Simulator Fast Blockchain Simulations for Large-scale Peer-to-Peer Networks by Yusuke Aoki, Kai Otsuki, Takeshi Kaneko, Ryohei Banno and Kazuyuki Shudo by Seyed Mehdi Fattahi, Adetokunbo Makanju, Amin Milani Fard by Lyubomir Stoykov, Kaiwen Zhang, Hans-Arno Jacobsen

Why Create a Blockchain Network Simulator? Issues blockchain network researchers currently face: It s hard to obtain information about the whole network in large-scale public blockchain networks Predicting the performance of a blockchain application during the design phase is difficult and evaluation after it is built could be expensive It s difficult to predict the behaviour of distributed ledgers That being said, it is hard to conduct experiments in practical large-scale blockchain networks, hence why distributed ledger technologies are important.

Architecture of Simulators Typically, distributed ledger technologies are abstracted into multiple interdependent layers (displayed on the right) [2]. https://ieeexplore.ieee.org/abstract/document/9500200

Simulator Model Types There are different types of simulator models (i.e., a particular type of mathematical model of a system) [1]. All simulators covered in this presentation follow a discrete-event simulation model (i.e., describes the system as a sequence of events, so it is possible to jump in time from one event to the next).

Quick Comparison Name Language/Frame- work Platform(s) Layer(s) Model Type Year VIBES Scala PoW blockchains consensus, data discrete-event 2017 SIMBA Python Bitcoin consensus, data, network discrete-event 2020 SimBlock Java PoW blockchains consensus, network discrete-event 2019

SimBlock Organization: 1. Propose the blockchain simulator 2. Show that the simulator could simulate an actual blockchain with good accuracy 3. Demonstrate that the simulator is useful in research

SimBlock SimBlock is a blockchain network simulator which simulates the P2P network of public blockchains (e.g., Bitcoin). It was designed as an event-driven simulator wherein each participating node behaves according to generated events, e.g., block generation and exchanging messages It can visualize nodes behavior and block propagation Claim: Unlike existing simulators, SimBlock can easily change behavior of nodes, enabling the influence of nodes behavior on blockchains to be investigated.

Parameters Block Parameters Node Parameters Block size Block generation interval Number of nodes Number of neighbor nodes Location of node Block generation capacity Network Parameters Network bandwidth Network propagation delay

Derived Metrics Arrival time of a message: derived from the propagation delay between the nodes and bandwidth Transmission time: derived from the message size and bandwidth between the regions Message reception: derived from the total time of the transmission time and propagation delay from the transmission event of the message

Details Actual hash calculation is not performed in the proposed simulator. Mining success time is calculated from the sum of the block generation capacity of all the nodes and block generation difficulty

Evaluation A comparative experiment was performed by reproducing the parameters used in an existing simulator introduced by Gervais et al. (Language: C++/NS3; year: 2016) Table I displays the parameters Gervais et al. and Aoki et al. used to reproduce the environment of Bitcoin, Litecoin, and Dogecoin

Evaluation Aoki et al. simulated it until 10,000 blocks were generated. The result of SimBlock, Gervais et al. s simulator and real data are shown below (only the fork rate in Bitcoin obtained a large error because Bitcoin uses a relay network that is not reproduced in their experiment).

Application Examples: Demonstrating That the Simulator is Useful in Research They conducted two experiments which display the influence of neighbor node selection algorithms and relay networks on the block propagation time. Purpose: One of the problems of existing blockchains is low transaction throughput (i.e., transactions per second) Reducing block propagation time can address this problem, hence shortening the propagation delay can safely reduce the block generation interval and improve the throughput

Reducing the Block Propagation Time Improve the efficiency of the network topology. A blockchain network is P2P without a central administrator. Therefore, the topology of the network is dependent on the manner of selecting the neighbor nodes of individual nodes. Construct a relay network for block distribution to reduce the block propagation time in blockchains.

Exploring Better Neighbor Selection Strategies Aoki et al. s algorithm makes each node attempt to connect with other nodes which send an INV message to it faster than its existing neighbors. A few neighbors are randomly selected from all known nodes to find better neighbors. They performed the experiments by using the same parameters that of Bitcoin.

Median of Block Propagation Time with a Neighbor Selection Technique x-axis: number of generated blocks y-axis: the median of the block propagation time (averaged every 100 blocks and then plotted) This figure shows the difference of block propagation time between Aoki et al. s algorithm and the normal one used in Bitcoin.

Assessment of Relay Networks Relay networks (e.g., bloXroute and Falcon) are networks dedicated to block propagation. They aim to improve propagation time, so that they could increase throughput similar to the neighbor selection strategies.

Median of the Block Propagation When Changing the Participation Rate of the Node to the Relay Network As displayed, high participation ratios entail improvement of propagation time. Even if the participation rate in the relay network is as low as 5%, the propagation time of all nodes is improved to less than 70% of the original propagation time.

Results Blocks are propagated faster with Aoki et al. s neighbor selection technique As the involved rate in the relay network increases, the difference in the propagation efficiency between the non-participating and participating nodes increases.

Criticisms

Criticisms from [1] SimBlock assumes that all nodes are honest; thus, they cannot be used to explore the strategies of the nodes malicious behaviors (e.g., selfish mining, double- spending) does not separate full nodes and miners

VIBES

VIBES Simulator is capable of conducting large-scale network simulation focused on the behavior of the blockchain network Two objectives of this simulator: Scalability (able to simulate large networks with thousands of nodes) Speed (ability to adjust the parameters for desired outcomes)

VIBES Architecture After a browser client sends input parameters to the web server, the orchestrator bootstraps the desired amount of nodes, configures them, and starts the simulation.

Input Parameters number of transactions per block / transaction size electricity cost smart contract time smart contract intensity and smart contract density percentage of attacker nodes percentage of failing nodes network topology area size latency bandwidth number of nodes number of miners block size block confirmation time

Output Metrics total time to process total number of transactions processed throughput (transactions per second) block propagation delay for 10%, 50% and 90% client bootstrap time cost per transaction probability of an attacker taking over at each stage a log of all transactions.

Addressing Scalability They propose the concept of fast-forward computing. For example, consider simulating a blockchain system where miners resolve consensus using Proof-of- Work, using a block time of 10 minutes like bitcoin each node makes a best guess on the time to complete the operation, then sends this information to the orchestrator, and asks for permission to skip ahead and label the block as mine Once the orchestrator has received a timestamp from every node, it gives permission to the node with the earliest timestamp ts to mine the block, assigns ts as the time of the mining operation, and fast-forwards the entire network to time ts All nodes reset their best guesses and continue with the next operation in their stack. The system works recursively until nodes have finished their work.

VIBES Web Client The web client is able to simulate the interaction between nodes via a time- lapse

VIBES Conclusion The rationale behind this simulator is to provide a blockchain simulator that is not confined to the Bitcoin protocol, trying to support large-scale simulations with thousands of nodes

Criticisms

Criticisms from [1] VIBES lacks the ability to simulate network behavior realistically, e.g., it uses fixed block and transaction propagation delays simulations of real-size public networks consisting of more than 10,000 nodes are complicated as this simulator utilizes a centralized coordinator, which is a severe performance bottleneck

SIMBA

SIMBA SIMBA extended BlockSim (a simulator by Faria and Correia) by adding Merkle tree elements to the blockchain nodes to improve efficiency. Name Language/Frame- work Platform(s) Layer(s) Model Type Year BlockSim Python Bitcoin, Ethereum consensus, data, network discrete-event 2019 In addition, it permits more realistic evaluations with the base tool to be performed

Block Verification Process mine a new block block verification Miner node block verification add to blockchain Other nodes

Evaluation Fattahi et al. evaluates SIMBA by comparing two metrics, verification time and propagation delay for a block, both when using a Merkle tree and without using a Merkle Tree.

Average Block Verification Time (ms) for Miner & Non-Miner Nodes n = 50 The graph shows that the average verification time per block without using Merkle trees is from 5 to 30 times higher than when using Merkle tree For space reasons, they did not present results for n = 100 and 300, but they claim the graphs look very similar.

Total Block Verification Time (ms) for Miner & Non- Miner Nodes n = 50, 100, and 300 The authors conclude that irrespective of the number of nodes in the network (n), the amount of time spent verifying blocks does not increase significantly for the network with Merkle trees while a linear increase can be expected without Merkle trees

Block Propagation Time Authors compare the block propagation time of the two simulations by calculating the mean absolute difference (MD) as for i j. For this experiment, they set n = 300, and block size to 100. The calculated MD value was less than 0.0001 almost no change in the propagation delay between using Merkle tree and without using it. : propagation time between node i and j in simulation without Merkle trees : propagation time between node i and j in simulation with Merkle trees

Results Using Merkle Trees can significantly improve the performance of nodes to verify the consistency of transactions in blocks, but does not significantly impact the performance of the network (i.e., Merkle Trees improve block verification, but not block propagation).

Criticisms

Criticisms from [1] BlockSim (what SIMBA extends)... is suitable only for investigations with a small number of nodes

Overall Criticisms from [1] The drawback of all simulators is that they typically use a fixed number of nodes, which is not in line with public blockchain networks. Most simulators are not actively developed and remain outdated. ...there is no one-size-fits-all simulator that could accurately simulate all layers of a PoW blockchain.

References [1] R. Paulavicius, S. Grigaitis, and E. Filatovas, A systematic review and empirical analysis of blockchain simulators, IEEE access, vol. 9, pp. 38 010 38 028, 2021, https://ieeexplore.ieee.org/document/9366733 [2] J. Polge, S. Ghatpande, S. Kubler, J. Robert, and Y. Le Traon, BlockPerf: A Hybrid Blockchain Emulator/Simulator Framework, IEEE Access, vol. 9, pp. 107 858 107 872, 2021, https://ieeexplore.ieee.org/abstract/document/9500200