Performance Analysis of Hadoop-Harp Applications on Big Red II
Explore the performance analysis of Hadoop-Harp applications on Big Red II, focusing on K-Means Clustering, FR Graph Drawing Algorithm, and WDA-SMACOF. The analysis includes execution time, speedup, and parallel efficiency, providing insights into their behavior with varying numbers of nodes. View detailed benchmarks and results to understand the scalability and efficiency of these applications on a large-scale computing environment.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Hadoop-Harp Applications Performance Analysis on Big Red II
Notes For K-Means Clustering and FR Graph Drawing Algorithm, I use execution time and speedup to explain the performance because They can be drawn in one picture. Due to the intensive computation, I find these applications are easy to achieve linear speedup when the number of nodes is small. Therefore the parallel efficiency is always around 1.
K-Means Clustering Execution Time and Speedup on Big Red II (10 Iterations Benchmark, Nodes: 8, 16, 32, 64, 128, JVM settings: -Xmx44000m Xms44000m -XX:NewRatio=1 -XX:SurvivorRatio=98) 6000 140 120 5000 100 Execution Time (Seconds) 4000 80 Speedup 3000 60 2000 40 1000 20 0 0 0 20 40 60 80 100 120 140 Number of Nodes 500m points 10k centroids Execution Time 5m points 1m centroids Execution Time 500m points 10k centroids Speedup 5m points 1m centroids Speedup
FR Graph Drawing Algorithm Execution Time and Speedup on Big Red II (477111 Vertices and 665599 Undirected Edges, 10 Iterations Benchmark, Nodes: 1, 8, 16, 32, 64, 128, JVM settings: -Xmx44000m Xms44000m -XX:NewRatio=1 -XX:SurvivorRatio=98) 10000 120 9000 100 8000 7000 Execution Time (Seconds) 80 6000 Speedup 5000 60 4000 40 3000 2000 20 1000 0 0 0 20 40 60 80 100 120 140 Number of Nodes Execution Time Speedup
WDA-SMACOF Execution Time on Big Red II (Full Application Benchmark, Nodes: 8, 16, 32, 64, 128, JVM settings: -Xmx54000m -Xms54000m, -XX:NewRatio=1 -XX:SurvivorRatio=98) 4000 3500 3000 Execution Time (Seconds) 2500 2000 1500 1000 500 0 0 20 40 60 80 100 120 140 Number of Nodes 100k points 200k points 300k points 400k points
WDA-SMACOF Parallel Efficiency on Big Red II (Full Application Benchmark, Nodes: 8, 16, 32, 64, 128, JVM settings: -Xmx54000m -Xms54000m -XX:NewRatio=1 -XX:SurvivorRatio=98) 1.2 1.0 0.8 Parallel Efficiency 0.6 0.4 0.2 0.0 0 20 40 60 80 100 120 140 Number of Nodes 100k points 200k points 300k points
WDA-SMACOF Speedup on Big Red II (Full Application Benchmark, Nodes: 8, 16, 32, 64, 128, JVM settings: -Xmx54000m -Xms54000m -XX:NewRatio=1 -XX:SurvivorRatio=98) 120.0 100.0 80.0 Speedup 60.0 40.0 20.0 0.0 0 20 40 60 80 100 120 140 100k points 200k points 300k points
