Shell Scripting for Genomic Computing: Basics and Tricks
Learn the basics of shell scripting with Bash for high-performance computing in genomic applications. Explore how to work with Bash, execute scripts, use language constructs, handle parameters, and set environment variables efficiently. Automate tasks and manage jobs on cluster systems with Bash scripting expertise provided by Michal Okoniewski from Scientific IT Services at ETH.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
High Performance Computing for genomic applications Basics of shell scripting with bash and related useful tricks Scientific IT Services Michal Okoniewski ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 1
Bash general information Bash is a command and scripting language of the Unix shell Bash commands are executed from the command prompt or a script In the cluster queueing system, bash scripts are typically wrapping several commands and are sent for execution Here we learn bash to use it with LSF to send jobs on the cluster Unix (incl. Liunx) and macOS systems speak bash as a native command language ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 2
How to work with bash Edit the script file, with #!/bin/bash on top Save it as *.sh Add the execution permissions chmod a+x my_test.sh chmod 755 my_test.sh Run it as ./my_test.sh . my_test.sh source my_test.sh ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 3
Hello world example ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 4
Useful bash language constructs Variables Setting name="mutant1.fastq.gz" Calling: $name or Loops for do done Conditional execution if then else fi ${name} ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 5
Parameters in the script ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 6
Parameters in the script - result ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 7
Useful bash language constructs Environment variables echo $PATH echo $SCRATCH Can be set globally in .bash_profile or .bashrc, eg: INCLUDE=$INCLUDE:$HOME/genomics/samtools-1.3.1:$HOME/genomics/samtools- 1.3.1/lib:$HOME/genomics/samtools-1.3.1/bam ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 8
Bash can be used to do automatization of (boring) tasks ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 9
Yet another example ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 10
Bash testing advice In the script, make sure where it is executed, eg start with cd /cluster/home/michalo/ME9 Development runs and testing of the bash scripts On the login nodes only if they are not using heavy or multithreaded computations Otherwise: in the interactive session on the compute node Scripts that do not use heavy computing, but take long time, eg. slow downloads can be run in a screen command sessions For tracing and debugging use echo command in critical steps of the script ID | SIS | | Michal Okoniewski, Scientific IT ETH 5/13/2025 11
Questions? Bash
