Tim Cartwright OSG Virtual School Pilot - Self-Checkpointing July 17
In a virtual school pilot project by Tim Cartwright, self-checkpointing is discussed on July 17. The project involves saving checkpoints during iterations and handling exit codes efficiently. Learn more about the OSG Virtual School Pilot 2020 in this informative content.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Tim Cartwright OSG Virtual School Pilot 2020 Self-Checkpointing (Cartwright) July 17 1
One solution: OSG Virtual School Pilot 2020 Self-Checkpointing (Cartwright) July 17 2
OSG Virtual School Pilot 2020 Self-Checkpointing (Cartwright) July 17 3
checkpoint_exit_code = 77 transfer_output_files OSG Virtual School Pilot 2020 Self-Checkpointing (Cartwright) July 17 4
def save_checkpoint(iteration): cp_file = open(checkpoint_path, 'w') cp_file.write('%d\n' % (iteration)) # See Notes sys.exit(77) # ... for iteration in xrange(start, end + 1): do_science(iteration) if ((iteration - start + 1) % 1000) == 0: save_checkpoint(iteration) sys.exit(0) OSG Virtual School Pilot 2020 Self-Checkpointing (Cartwright) July 17 5
start, end = map(int, sys.argv[1:]) if os.path.exists(checkpoint_path): cp_file = open(checkpoint_path, 'r') cp_data = cp_file.readlines().strip() cp_file.close() cp_start = int(cp_data) if cp_start >= start: start = cp_start else: # Potential problem? OSG Virtual School Pilot 2020 Self-Checkpointing (Cartwright) July 17 6
learn OSG Virtual School Pilot 2020 Self-Checkpointing (Cartwright) July 17 7
