Challenges Faced in Budget Computing: Solutions and Strategies
Explore the world of budget computing with Christopher Harrison as he delves into the challenges of system engineering, collaboration with developers, computational job profiling, and more. Discover how users are solving problems, dealing with computational challenges in biostatistics, and thinking outside the box in the realm of computing. Venture into the realm of used equipment and learn the pros and cons of managing computational systems on a budget.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Computing on a budget, a system's engineers prospective Christopher Harrison HTCondor Week 2016 Systems Engineer Biostatistics and Medical Informatics School of Medicine and Public Health UW - Madison
Get to know your users (Think Journalism) What problems are my users solving? Who is doing the actual development? Closely collaborate with the development people When are the deadlines for projects? Where are the projects going? (not always known) How are the users expecting to solve their problems? Can early intervention lead to better outcomes? GOAL: Understand and Profile computational jobs of the users to understand requirements
UW - Biostatistics computational challenges Users are working on protected data sets (HIPPA) Genetic algorithms (Memory and I/O intensive jobs) CPU is not the bottleneck Data storage, I/O and memory are bottleneck Address data storage: FREEBSD ZFS NFS
Think outside the box Do we really need new? Do we need systems with warranties?
Pets Cattle Dotty, my dog rawhide-0(1-1023) VS Chairman Mao, People die every day Chris Harrison, Computers die every day
Venture into used equipment (ebay) Purchase #1 Purchase #1 Purchase #2 Purchase #2 42 x Dell C1100 26 x CISCO UCS C250 M2 72GB DDR3 192GB DDR3 Dual quad nehalem w/ HT Dual hex core Westmere 4 3.5 drive bays 10x 2.5 drive bays Price per: $460 Price per: $1799
Keeping track at home Pro Pro Con Con We added: Harder to procure, simplified bid process ~1300 CPU cores (w/ HT), ~8 TB of ddr3 mem No replacements on failures Treat systems as cattle, discard if dead Used equipment, more sysadmin time req. Tripled Condor pool at yearly replacement cost Useful life of individual machines is less the new Cost per unit ~8x less than new Replace equipment more frequently
Questions? Christopher Harrison harrison@biostat.wisc.edu
