February Scripting Workshop

Introduction to Scripting Workshop February 23, 2016

Introduction George Garrett & The HPC Support Team Research Computing Services CUIT

Introduction Please Leave Feedback

Introduction Slides will be sent out afterwards.

Introduction What is a script?

Introduction What is a shell script?

Introduction What is a shell script? A file with shell commands.

Introduction Why use scripts?

Scripting Resources How Linux Works by Brian Ward Available as an E-book from Columbia University Libraries and at Safari Books Online

Scripting Resources Shell and Scripting Tutorial http://linuxcommand.org/

Scripting Resources Advanced Bash-Scripting Guide http://tldp.org/LDP/abs/html/

Cunix System: cunix.columbia.edu User: Your UNI

Access Windows Instructions 1. Search for putty on Columbia home page 2. Select first result 3. Follow link to Putty download page 4. Download putty.exe 5. Run putty.exe

Access Mac Instructions 1. Run terminal

Access Mac (Terminal) $ ssh UNI@cunix.columbia.edu Windows (Putty) Host Name: cunix.columbia.edu

Access Does everyone have access?

Quick Review shell $ pwd cd . ls user interface to the system standard prompt symbol print working (current directory) change directory current directory list directory contents

Quick Review cat sort grep echo hi sleep 5 man print a file sort the lines of a file print lines matching a pattern print hi wait 5 seconds command manual

Workshop Setup $ mkdir workshop $ cd workshop $ cp /tmp/workshop/* .

Command Line Example Word Count from Data Science at the Command Line - Jeroen Janssens

Lets Download Some Data Gutenberg Project: http://www.gutenberg.org Which famous novel should we use? Download plain text version using curl $ curl [URL] > novel.txt Trim header and footer, leaving only main text

Pipes $ cat alice.txt | grep rabbit Pipes connect output from one command to the input of another command

Pipes $ cat alice.txt | grep rabbit | sort You can keep on combining commands with more pipes.

Pipes $ cat alice.txt | grep rabbit | tr r w tr translate characters

wcount $ cat wcount cat alice.txt | tr

wcount $ wcount

wcount $ wcount -bash: wcount: command not found

wcount $ ./wcount

wcount $ ./wcount -bash:./wcount:Permission denied

wcount $ ls l wcount

wcount $ ls l wcount -rw-rw---- [ snip ]

wcount $ ls l wcount -rw-rw---- $ chmod +x wcount [ snip ]

wcount $ ls l wcount -rw-rw---- $ chmod +x wcount $ ls l wcount -rwxrwx--x [ snip ] [ snip ]

wcount $ ./wcount Should work this time.

file Determine type of file. $ file wcount

file Determine type of file. $ file wcount wcount: ASCII text

wcount Choose an editor nano Recommended default vi emacs

nano Nano commands are on back of cheat sheet. ^ means hold down control

Edit wcount $ nano wcount

#! Add #! to first line #!/bin/sh cat alice.txt | tr

#! $ ./wcount Still works.

#! Some #! first lines you might see #!/bin/sh #!/bin/bash #!/usr/bin/perl #!/usr/bin/python

file Has the file type changed? $ file wcount wcount: POSIX shell script text executable

Variables $ file=alice.txt $ echo $file alice.txt

Variables 1. Add file=alice.txt to wcount. 2. Replace cat alice.txt with cat and the variable.

Variables #!/bin/sh file=alice.txt cat $file | tr

Variables You could put $file in double quotes. Why put quotes around $file? cat $file | tr

Command Line Parameters We re going to change wcount so any file can be specified from the command line. $ ./wcount moby.txt

Command Line Parameters Change wcount so any file can be specified from the command line. $ ./wcount moby.txt $1

Command Line Parameters 1. Create a new file named param 2. Put the #! directive on the first line 3. On next line type: echo $1 4. Save and make executable 5. Run it