HTCondor Week 18 Insights & Challenges
"Explore HTCondor Week 18 insights including the value of 18, the wisdom of Koheleth, and the evolving distributed execution environments. Discover the core hours, shared resources, and essential workflow steps in the HTCondor ecosystem."
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Welcome to HTCondor Week #18 (year 33 of our project)
The words of The words of Koheleth Jerusalem ~ 200 A.D. Jerusalem ~ 200 A.D. Koheleth son of David, king in son of David, king in Only that shall happen Only that shall happen Which has happened, Which has happened, Only that occur Only that occur Which has occurred; Which has occurred; There is nothing new There is nothing new Beneath the sun! Beneath the sun! Ecclesiastes, ( of David, and king in Jerusalem" alias Solomon, Wood engraving Gustave Dor (1832 1883) Kohelet, "son , Ecclesiastes Chapter 1 verse 9
A crowded and growing space of distributed execution environments that can benefit from the capabilities offered by HTCondor
But, we are also facing change! (complexity & scale)
1.42B core hours in 12 months! Almost all jobs executed by the OSG leverage (HT)Condor technologies: Condor-G HTCondor-CE Basco Condor Collectors HTCondor overlays HTCondor pools
Additional shared resources that require protection (allocation)
1. Request 2. Schedule 3. Allocate 4. Protect 5. Monitor 6. Reclaim 7. Account
Miron Livny John P. Morgridge Professor of Computer Science Director Center for High Throughput Computing University of Wisconsin Madison It is all about Storage Management, stupid! (Allocate B bytes for T time units) We do not have tools to manage storage allocations We do not have tools to schedule storage allocations We do not have protocols to request allocation of storage We do not have means to deal with sorry no storage available now We do not know how to manage, use and reclaim opportunistic storage We do not know how to budget for storage space We do not know how to schedule access to storage devices
Thank you for building such a wonderful and very much alive ( ) HTC community www.cs.wisc.edu/~miron
