Regular Expressions: A Comprehensive Overview

John Carelli,Instructor Kutztown University carelli@kutztown.edu

Regular Expressions Character sequences used for string matching Matches longest pattern CSC252: John Carelli Source: Dr. Lisa Frye

.Any single character [] Any single character in [] -Range of characters, ex: [a-z] ^ If first character any character NOT in [] * Matches zero or more occurrences ^ At beginning of reg exp, matches beginning of line $ At end of reg exp, matches end of line \ Quote a special character (escape character) CSC252: John Carelli Source: Dr. Lisa Frye

+ Matchesone or more occurrences ? Matches zero or one occurrences | Boolean OR Note - implementation of regular expressions varies among tools http://www.greenend.org.uk/rjk/tech/regexp.html CSC252: John Carelli Source: Dr. Lisa Frye

\d numerical digits, [0-9] \w word character, [A-Za-z0-9_] \s whitespace [ \t\n\f\r] \b a word boundary (an anchor, like ^ and $, but for a word, not a line) CSC252: John Carelli Source: Dr. Lisa Frye

| Alternation ( or ) ex: cat|dog Repetition more granularity than * or + Includes two optional numbers in the braces Minimum Maximum ex: [1-9][0-9]{2,4} matches numbers between 100 and 99999 { } Group operator ex: Set(Value)? matches Set or SetValue ( ) CSC252: John Carelli Source: Dr. Lisa Frye

POSIX [:alnum:] [:alpha:] [:ascii:] [:blank:] [:digit:] [:lower:] [:space:] Description Alphanumeric characters Alphabetic characters ASCII characters Space and tab Digits Lowercase letters All whitespace characters,including line breaks [:upper:] [:word:] Uppercase letters Word characters (letters,numbers and underscores) CSC252: John Carelli Source: Dr. Lisa Frye

grep globally (match a)regular expression (and) print grep interprets basic regular expressions some special characters have to be escaped ex: ( ) | egrep (grep E) uses extended regular expressions escaping turns metacharacter meaning off fgrep (grep F) match literally turn off RE and metacharacters Source: Dr. Lisa Frye 8 CSC252: John Carelli

When in the Course of human events, it becomes necessary for one people to dissolve the political bands which have connected them with another, and to assume, among the Powers of the earth, the separate and equal station to which the Laws of Nature and of Nature's God entitle them, a decent respect to the opinions of mankind requires that they should declare the causes which impel them to the separation. We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty, and the pursuit of Happiness. CSC252: John Carelli Source: Dr. Lisa Frye

Regular Expressions - Try it! Match any line that contains the word which Match any line that contains a comma Match any line that begins with an o Match all blank lines Match any line that contains an uppercase letter Match any line that begins with a W , i ,or t Match any line that contains the word to only the word, not the embedded characters! CSC252: John Carelli Source: Dr. Lisa Frye

Stream EDitor utility sed [-n] program [filelist] sed [-n] -f program-file [filelist] sed "s/Powers/powers/g" declaration.txt > declaration.new CSC252: John Carelli Source: Dr. Lisa Frye

sed some basic options and commands syntax: sed [option] command [infile] writes to standard out can accept standard input options: -n suppress line printing -e additional command for processing -f read commands from a file commands: s do a substitution g substitute globally p print the current pattern space d delete the current pattern space I ignore case Source: Dr. Lisa Frye 12 CSC252: John Carelli

CSC252: John Carelli Source: Dr. Lisa Frye

sed substitution Most common sed usage is for substitution: sed s/one people/Folks/ declaration.txt sed s/the/THE/g declaration.txt sed s/the/THEM/2 declaration.txt (first occurrence) (all occurrences) (second occurrence) cat declaration.txt | sed s/one People/Folks/ig cat declaration.txt | sed s,one People,Folks,ig -n (no printing) and e (multiple commands) sed -n -e 's/one people/Folks/' -e 's/dissolve/melt/p declaration.txt note: p causes the matched line to print Source: Dr. Lisa Frye 14 CSC252: John Carelli

sed more Using sed for pattern matching: sed n /pattern/p file sed n 1,3p file sed 1,3d file (works like grep) (print first 3 lines) (delete first 3 lines) (delete from line 1 to line containing assume ) cat declaration.txt | sed '1,/assume/d (make all vowels upper case) sed f sedscript < declaration.txt sedscript file: s/a/A/g s/e/E/g s/i/I/g s/o/O/g s/u/U/g Source: Dr. Lisa Frye 15 CSC252: John Carelli

Try it! List only files in your directory (no directories) Delete leading blanks at beginning of every line of a file Replace all strings of blanks with one blank Delete empty lines or lines with only spaces in it Delete all lines from the beginning of the file up to, and including the first blank line CSC252: John Carelli Source: Dr. Lisa Frye