Singularity Integration with Makeflow for HPC Environment
Explore the integration of Singularity with Makeflow in an HPC environment, focusing on compute nodes, parallel file systems, work queues, node anatomy, benefits, methods of usage, challenges, and more.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Integration of Singularity With Makeflow Kyle Sweeney University of Notre Dame ksweene3@nd.edu
Main design of an HPC Environment Compute Nodes with small discs Massive Parallel File System
Makeflow and Work Queue with HPC Work nodes with Workers Computer running Makeflow and Work Queue Massive Parallel File System
Anatomy of a Node with Workers Worker Scratch Space Symlink
Singularity No root Daemon! Does not manage a database of images! Does not attempt any kind of scheduling!
Main Benefits of Singularity Democratization of Resource Management! Everything needs to be done by the user And Singularity gives tools to do this
Using Singularity and Makeflow Two methods of using Singularity with Makeflow Global Wrapper: makeflow T wq N mytasks --singularity=<img> script.mf Will wrap every tasks in script.mf in Singularity singularity exec <img> sh c task1 Per-Task wrapper: Inside of your makeflow script, you can simply add singularity to the start of you task, and include the image file foo.out: singularity mytask myfile.img foo.in singularity exec myfile.img mytask foo.in o foo.out
Singularity and a Node with Workers Worker Scratch Space Symlink
Problems with Singularity Creating and Modifying an image file requires the user to be root Image Files are large Folders linked into Singularity
Biggest issue with integration Singularity attempts to bind 3 important locations into a container /tmp Current working directory $HOME/ When a user requests a worker, that worker s owner is that user Singularity attempted to bind the user calling Makeflow and Workqueue s home directory into the container
CPU Measurement Modified the shrimp workflow Performed this test 3 different ways Natively, Using Docker, and Using Singularity Each worker was requested to have 1 Core, 2000MB memory Performed this test around 150 times
Disc I/O Measurement Modified a test which tested the overhead of different file systems The test nests singularity calls (e.g singularity exec img1.img singularity exec img2.img my_task.sh) The images were made ahead of time, all existing flat, not nested inside of each other Required sudo to work, thus ran on an Ubuntu VM on my laptop
Conclusion Singularity fits into our Makeflow+Work Queue system very well, thanks to Singularity s democratization of resource management Requires users to create their own images to match their work Negligible CPU overhead, and minor I/O overhead
Singularity: http://singularity.lbl.gov Makeflow + Singularity out soon! CCL Website: http://ccl.cse.nd.edu Email: ksweene3@nd.edu
