Perl
A language for Systems and Network Administration and Management
Nick Urbanik
nicku@vtc.edu.hk Department of Information and Communications Technology

Copyright Conditions: Open Publication License

(see http://www.opencontent.org/openpub/)

SNM — ver. 1.7

Perl - p. 1/123

What is Perl?
I I I I

I I

Perl is a programming language The best language for processing text Cross platform, free, open Microsoft have invested heavily in ActiveState to improve support for Windows in Perl Has excellent connection to the operating system Has enormous range of modules for thousands of application types

What is Perl? What is Perl? What is Perl? — 2 Compiled and run each time Perl is Evolving Eclectic Regular Expressions Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Perl - p. 2/123 Regular Expressions

SNM — ver. 1.7

What is Perl? — 2
I I I I

I I I I

Robust and reliable (has very few bugs) Supports object oriented programming Good for big projects as well as small Java 1.4 has borrowed one of Perl’s best features: regular expressions Perl has garbage collection The “duct tape of the Internet” Easy to use, since it usually “does the right thing” Based on freedom of choice: “There is more than one way to do it!” — TIMTOWTDI ™

What is Perl? What is Perl? What is Perl? — 2 Compiled and run each time Perl is Evolving Eclectic Regular Expressions Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Perl - p. 3/123 Regular Expressions

SNM — ver. 1.7

Compiled and run each time
I

I I

I

Perl is interpreted, but runs about as fast as a Java program Software development is very fast The Apache web server provides mod_perl, allows Perl applications to run very fast Used on some very large Internet sites: N The Internet Move Database N Macromedia, Adobe, http://slashdot.org/

What is Perl? What is Perl? What is Perl? — 2 Compiled and run each time Perl is Evolving Eclectic Regular Expressions Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Perl - p. 4/123 Regular Expressions

SNM — ver. 1.7

Perl is Evolving
I

Perl 6 will introduce many great features to make Perl N easier to use N Even more widely usable for more purposes N Even better for bigger projects

What is Perl? What is Perl? What is Perl? — 2 Compiled and run each time Perl is Evolving Eclectic Regular Expressions Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Perl - p. 5/123 Regular Expressions

SNM — ver. 1.7

Eclectic
I I I I

Borrows ideas from many languages, including: C, C++ Shell Lisp . . . even Fortran Many others. . .

What is Perl? What is Perl? What is Perl? — 2 Compiled and run each time Perl is Evolving Eclectic Regular Expressions Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Perl - p. 6/123 Regular Expressions

I BASIC I I

SNM — ver. 1.7

Regular Expressions
I I I I

One of the best features of Perl A new concept for most of you . . . But very useful! Used to: N extract information from text N transform information N You will spend much time in this topic learning about regular expressions — see slide 88

What is Perl? What is Perl? What is Perl? — 2 Compiled and run each time Perl is Evolving Eclectic Regular Expressions Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Perl - p. 7/123 Regular Expressions

SNM — ver. 1.7

Why should I learn it?
I

I

It will be in the final exam! N Okay, that’s to get your attention, but. . . Consider a real-life sys-admin problem: N You must make student accounts for 1500 students N TEACHING BEGINS TOMORROW!!! N The Computing Division has a multi-million dollar application to give you student enrollment data N . . . but it can only give you PDF files with a strange and irregular format for now (But Oh, it will be infinitely better in the future! Just wait a year or two. . . )

What is Perl? Example Problem Why should I learn it? The available data Sample data for new courses: Problems Solution in Perl — 1 Solution in Perl — 2 Solution in Perl — 3 But I can use any other language! Other Solutions may take Longer to Write The hello world program Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 8/123

The available data
I

I I I I

Has a variable number of lines before the student data begins Has a variable number of columns between different files Has many rows per enrolled student Goes on for dozens of pages, only 7 students per page!!!!!!! There are two formats, both equally peculiar!!!!

What is Perl? Example Problem Why should I learn it? The available data Sample data for new courses: Problems Solution in Perl — 1 Solution in Perl — 2 Solution in Perl — 3 But I can use any other language! Other Solutions may take Longer to Write The hello world program Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 9/123

Sample data for new courses:
What is Perl? Example Problem Why should I learn it?

15

N

CHAN Wai Yee 10-SEP-01 10-SEP-01

F 993175560

H123456(5) 21234567

28210216

The available CHEUNG data Sample data for new courses: Problems Solution in Perl — 1

WAI CHI

Solution in Perl — 2 Solution in Perl — 3 But I can use any other language! Other Solutions may take Longer to Write The hello world program Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 10/123

Problems
I

I

I I

There is a different number of lines above the student records There is a different number of characters within each column from file to file There are many files The format can change any time the computing division determines necessary

What is Perl? Example Problem Why should I learn it? The available data Sample data for new courses: Problems Solution in Perl — 1 Solution in Perl — 2 Solution in Perl — 3 But I can use any other language! Other Solutions may take Longer to Write The hello world program Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 11/123

Solution in Perl — 1
#! /usr/bin/perl -w use strict; my $course; my $year; while ( <> ) { chomp; if ( /ˆ\s*Course :\s(\d+)\s/ ) { $course = $1; undef $year; next; }
What is Perl? Example Problem Why should I learn it? The available data Sample data for new courses: Problems Solution in Perl — 1 Solution in Perl — 2 Solution in Perl — 3 But I can use any other language! Other Solutions may take Longer to Write The hello world program Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 12/123

Solution in Perl — 2
elsif ( m!ˆ\s*Course :\s(\d+)/(\d)\s! ) { $course = $1; $year = $2; next; } if ( my ( $name, $gender, $student_id, $hk_id ) = m{ \s\s+ # at leaset 2 spaces ( # this matches $name [A-Z]+ # family name is upper case (?:\s[A-Z][a-z]*)+ # one or more given names ) \s\s+ # at leaset 2 spaces ([MF]) # gender \s+ # at least one space (\d{9}) # student id is 9 digits # at leaset 2 spaces \s\s+ ([a-zA-Z]\d{6}\([\dA-Z]\)) # HK ID }x )
What is Perl? Example Problem Why should I learn it? The available data Sample data for new courses: Problems Solution in Perl — 1 Solution in Perl — 2 Solution in Perl — 3 But I can use any other language! Other Solutions may take Longer to Write The hello world program Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 13/123

Solution in Perl — 3
{ print "sex=$gender, student ID = $student_id, ", "hkID = $hk_id, course = $course, name=$name, ", defined $year ? "year = $year\n" : "\n"; next; } warn "POSSIBLE UNMATCHED STUDENT: $_\n" if m!ˆ\s*\d+\s+!; }
What is Perl? Example Problem Why should I learn it? The available data Sample data for new courses: Problems Solution in Perl — 1 Solution in Perl — 2 Solution in Perl — 3 But I can use any other language! Other Solutions may take Longer to Write The hello world program Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 14/123

But I can use any other language!
I

I

I

I will give you HK$200 if you are the first person to write a solution in another language in fewer keystrokes Note: the Perl solution given has: N comments N Plenty of space to show structure N . . . and handles exceptional situations (i.e., it is robust) To claim your $200 from Nick, your solution must have N similar space for comments N Similar readability and robustness N Be written in a general purpose language using ordinary libraries

What is Perl? Example Problem Why should I learn it? The available data Sample data for new courses: Problems Solution in Perl — 1 Solution in Perl — 2 Solution in Perl — 3 But I can use any other language! Other Solutions may take Longer to Write The hello world program Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 15/123

Other Solutions may take Longer to Write
I I I

This program took a very short time to write It is very robust For problems like this, Perl is second to no other programming language.

What is Perl? Example Problem Why should I learn it? The available data Sample data for new courses: Problems Solution in Perl — 1 Solution in Perl — 2 Solution in Perl — 3 But I can use any other language! Other Solutions may take Longer to Write The hello world program Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 16/123

The hello world program
print "hello world\n"
What is Perl? Example Problem Why should I learn it? The available data Sample data for new courses: Problems Solution in Perl — 1 Solution in Perl — 2 Solution in Perl — 3 But I can use any other language! Other Solutions may take Longer to Write The hello world program Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 17/123

Variables
I I I I

I

There are three basic types of variable: Scalar (can be a number or string or. . . ) Array (an ordered array of scalars) Hash (an unordered array of scalars indexed by strings instead of numbers) Each type distinguished with a “funny character”

What is Perl? Example Problem Variables Variables $Scalars: @Array %Hashes Conclusion Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O

SNM — ver. 1.7

Perl - p. 18/123 Regular Expressions

$Scalars:
I I I I

I

Start with a dollar sign Hold a single value, not a collection A string is a scalar, so is a number Since Perl is a loosely typed language, a scalar can be an integer, a floating point number, a character or a string. N Note that later you will see that a scalar can also hold a reference to another piece of data, which may also be an array or hash. Examples: $apple = 2; $banana = "curly yellow fruit";

What is Perl? Example Problem Variables Variables $Scalars: @Array %Hashes Conclusion Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O

SNM — ver. 1.7

Perl - p. 19/123 Regular Expressions

@Array
I I I

I

Starts with a @ Indexes start at 0, like in C or Java Each entry in an array is a scalar. N Multidimensional arrays are made by entry of an array being a reference to another array. See slide 37

What is Perl? Example Problem Variables Variables $Scalars: @Array %Hashes Conclusion Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O

SNM — ver. 1.7

Perl - p. 20/123 Regular Expressions

%Hashes
I I I I

Unfamiliar concept to many of you Like an array, but indexed by a string A data structure like a database See slide 43

What is Perl? Example Problem Variables Variables $Scalars: @Array %Hashes Conclusion Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O

SNM — ver. 1.7

Perl - p. 21/123 Regular Expressions

Conclusion
I

I I I I I I

Perl is optimised for text and systems administration programming Has great portability Is strongly supported by Microsoft Has three main built-in data types: Scalar: starts with $ Array: starts with @ Hash: starts with %

What is Perl? Example Problem Variables Variables $Scalars: @Array %Hashes Conclusion Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O

SNM — ver. 1.7

Perl - p. 22/123 Regular Expressions

An Overview of Perl
What is Perl? Example Problem

A language for Systems and Network Administration and Management: An overview of the language

Variables Perl Community An Overview of Perl Where do I get Perl? Where do I get Info about Perl?—1 Where do I get Info about Perl?—2 CPAN, PPM: Many Modules PPM: Perl Package Manager Mailing Lists: help from experts How to ask Questions on a List The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 23/123

File and Process I/O

Where do I get Perl?
I

I I I

For Windows, go to http://www.activestate.com, download the installer For Linux: it will be already installed For other platforms: go to http://www.perl.com This is a good source of other information about Perl

What is Perl? Example Problem Variables Perl Community An Overview of Perl Where do I get Perl? Where do I get Info about Perl?—1 Where do I get Info about Perl?—2 CPAN, PPM: Many Modules PPM: Perl Package Manager Mailing Lists: help from experts How to ask Questions on a List The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 24/123

File and Process I/O

Where do I get Info about Perl?—1
I

On your hard disk: N $ perldoc -f function I will look up the documentation for the built-in function (from the documentation perlfunc) N $ perldoc -q word I will look up word in the headings of the FAQ N $ perldoc perl I shows a list of much of your locally installed documentation, divided into topics N ActiveState Perl provides a Programs menu item that links to online html documentation

What is Perl? Example Problem Variables Perl Community An Overview of Perl Where do I get Perl? Where do I get Info about Perl?—1 Where do I get Info about Perl?—2 CPAN, PPM: Many Modules PPM: Perl Package Manager Mailing Lists: help from experts How to ask Questions on a List The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 25/123

File and Process I/O

Where do I get Info about Perl?—2
I

I

Web sites: N http://www.perl.com N http://www.activestate.com N http://use.perl.org See slide 123 for a list of books.

What is Perl? Example Problem Variables Perl Community An Overview of Perl Where do I get Perl? Where do I get Info about Perl?—1 Where do I get Info about Perl?—2 CPAN, PPM: Many Modules PPM: Perl Package Manager Mailing Lists: help from experts How to ask Questions on a List The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 26/123

File and Process I/O

CPAN, PPM: Many Modules
I

I

I

I

A very strong feature of Perl is the community that supports it There are tens of thousands of third party modules for many, many purposes: N Eg. Net::LDAP module supports all LDAP operations, Net::LWP provides a comprehensive web client Installation is easy: $ sudo perl -MCPAN -e shell cpan> install Net::LDAP Will check if a newer version is available on the Internet from CPAN, and if so, download it, compile it, test it, and if it passes tests, install it.

What is Perl? Example Problem Variables Perl Community An Overview of Perl Where do I get Perl? Where do I get Info about Perl?—1 Where do I get Info about Perl?—2 CPAN, PPM: Many Modules PPM: Perl Package Manager Mailing Lists: help from experts How to ask Questions on a List The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 27/123

File and Process I/O

PPM: Perl Package Manager
I I I

I

For Windows Avoids need for a C compiler, other development tools Download precompiled modules from ActiveState and other sites, and install them: C:\> ppm install Net::LDAP See documentation with ActiveState Perl

What is Perl? Example Problem Variables Perl Community An Overview of Perl Where do I get Perl? Where do I get Info about Perl?—1 Where do I get Info about Perl?—2 CPAN, PPM: Many Modules PPM: Perl Package Manager Mailing Lists: help from experts How to ask Questions on a List The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 28/123

File and Process I/O

Mailing Lists: help from experts
I I I I

There are many mailing lists and newsgroups for Perl When subscribe to mailing list, receive all mail from list When send mail to list, all subscribers receive For Windows, many lists at http://www.activestate.com

What is Perl? Example Problem Variables Perl Community An Overview of Perl Where do I get Perl? Where do I get Info about Perl?—1 Where do I get Info about Perl?—2 CPAN, PPM: Many Modules PPM: Perl Package Manager Mailing Lists: help from experts How to ask Questions on a List The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 29/123

File and Process I/O

How to ask Questions on a List
I

I

I I I

I

I receive many email questions from students about many topics Most questions are not clear enough to be able to answer in any way except, “please tell me more about your problem” Such questions sent to mailing lists are often unanswered Need to be concise, accurate, and clear see also Eric Raymond’s How to Ask Questions the Smart Way at http://catb.org/~esr/faqs/smart-questions.html Search the FAQs first—see slide 25

What is Perl? Example Problem Variables Perl Community An Overview of Perl Where do I get Perl? Where do I get Info about Perl?—1 Where do I get Info about Perl?—2 CPAN, PPM: Many Modules PPM: Perl Package Manager Mailing Lists: help from experts How to ask Questions on a List The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 30/123

File and Process I/O

Where is Perl on my system?
I I

I

ActiveState Perl installs perl.exe in C:\Perl\perl.exe Linux systems have a standard location for perl at /usr/bin/perl On some Unix systems, it may be installed at /usr/local/bin/perl

What is Perl? Example Problem Variables Perl Community The Shabang Where is Perl on my system? How OS knows it’s a Perl program—1 How OS knows it’s a Perl program—2 Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 31/123

How OS knows it’s a Perl program—1
I I I

To run your Perl program, OS needs to call perl How does OS know when to call Perl? Linux, Unix: N programs have execute permission: $ chmod +x program I OS reads first 2 bytes of program: if they are “#!” then read to end of line, then use that as the interpreter I OS doesn’t care what your program file is called N If program file is not in a directory on your PATH, call it like this: $ ./ program

What is Perl? Example Problem Variables Perl Community The Shabang Where is Perl on my system? How OS knows it’s a Perl program—1 How OS knows it’s a Perl program—2 Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 32/123

How OS knows it’s a Perl program—2
I

I

Windows: N OS uses the extension of the file to decide what to do (e.g., .bat, .exe) N Your program names end with .pl For cross platform support: N Put this at the top of all your programs: #! /usr/bin/perl -w N Name your programs with an extension .pl

What is Perl? Example Problem Variables Perl Community The Shabang Where is Perl on my system? How OS knows it’s a Perl program—1 How OS knows it’s a Perl program—2 Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 33/123

Language Overview
I I I I I

variables: scalars, arrays and hashes — §36–§52 compiler warnings, use strict; — §50–§52 operators, quoting — §53–§54 input and output — §55 statements: — §57 N if. . . elsif. . . else and unless statements — §58–§59 N while, for and foreach loops — §60–§66 I iterating over arrays and hashes — §66–§69 N Exit early from a loop with last, and next — §70 N “backwards” statements — §71–§72

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Language Overview Language Overview — 2 Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics

SNM — ver. 1.7

Perl - p. 34/123

Language Overview — 2
I

We also will examine: N subroutines, parameters and return statement — §76–§78 N array operations — §73–§75 N Error reporting: die and warn — §79 N Opening files — §80–§81 N executing external programs — §82–§86 N regular expressions — §88–§118 N Special input modes — §119–§121 N One line Perl programs — §122

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Language Overview Language Overview — 2 Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics

SNM — ver. 1.7

Perl - p. 35/123

Funny Characters $, @, %
I I I I

I

Variables in Perl start with a funny character Why? No problem with reserved words: can have a variable called $while, and another variable called @while, and a third called %while. Can interpolate value into a Double-quoted string (but not a single quoted string): my $string = "long"; my $number = 42.42; print "my string is $string ", "and my number is $number\n";

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 36/123 List Operations

SNM — ver. 1.7

Arrays
I

I I

I

I

Define an array like this: my @array = ( 1, 5, "fifteen" ); This is an array containing three elements The first can be accessed as $array[0], second as $array[1], the last as $array[2] Note that since each element is a scalar, it has the $ funny character for a scalar variable value In Perl, we seldom use an array with an index—use list processing array operations: push, pop, shift, unshift, split, grep, map and iterate over arrays with the foreach statement—see slide 66 N higher level.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 37/123 List Operations

SNM — ver. 1.7

Array Examples
I

I

I

Use the qw// “quote words” operator to help initialise arrays — see slide 54 See slide 66 for how the foreach loop works. my @fruit = qw( apple banana mandarin peach pear plum ); foreach my $fruit ( @fruit ) { print "$fruit\n"; } Note that these two are equivalent: my @fruit = qw( apple banana mandarin peach pear plum ); my @fruit = ( "apple", "banana", "mandarin", "peach", "pear", "plum" );

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 38/123 List Operations

SNM — ver. 1.7

More About Arrays
I

I

Instead of initialiasing the array as in slide 38, we can initialise the elements one by one: my @fruit; $fruit[ 0 ] = "apple"; $fruit[ 1 ] = "banana"; # ... $fruit[ 5 ] = "plum"; We can get a slice of an array: my @favourite_fruit = @fruit[ 0, 3 ]; print "@favourite_fruit\n"; N execute the program: $ ./slice.pl apple peach

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 39/123 List Operations

SNM — ver. 1.7

List Assignment
I

I

We can use a list of scalars whenever it makes some sense, e.g., N We can assign a list of scalars to a list of values Examples: my ( @a, $b, $c ) = ( 1, 2, 3 ); my @array = ( @a, $b, $c ); my ( $d, $e, $f ) = @array;

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 40/123 List Operations

SNM — ver. 1.7

Even More About Arrays
I

I

I

How many elements are in the array? See slide 42 print scalar @fruit, "\n" Does the array contain any data? See slide 59 print "empty\n" unless @fruit; Is there any data at the index $index? if ( defined $fruit[ $index ] and $fruit[ $index ] eq "apple" ) { print "found an apple.\n"; } N See perldoc -f defined. Also see perdoc -f exists.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 41/123 List Operations

SNM — ver. 1.7

Scalar, List Context
I

I

I

I

I

Each part of a program expects a value to be either scalar or list Example: print is a list operator, so if you print something, it is in list context If you look in the Perl Reference, you will see LIST shown as a parameter to many functions. N Any value there will be in a list context Many built-in functions, and your own functions (see perldoc -f wantarray), can give a different result in a scalar or list context force scalar context with scalar, e.g.,
print "the time is now ", scalar localtime, "\n";

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 42/123 List Operations

SNM — ver. 1.7

Hashes
I I I I

Hashes are probably new to you Like an array, but indexed by a string Similar idea was implemented in java.lang.HashTable Perl hashes are easier to use

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 43/123 List Operations

SNM — ver. 1.7

Initialising a Hash
my %hash = ( NL => ’Netherlands’, BE => ’Belgium’ ); I This creates a hash with two elements I one is $hash{NL}, has value “Netherlands”; I the other is $hash{BE} with value “Belgium” I The “=>” is a “quoting comma”. N It is the same as a comma, but it also quotes the string on its left. N So you can write the above like this: my %hash = ( ’NL’, ’Netherlands’, ’BE’, ’Belgium’ ); but the “=>” operator make it more clear which is the key and which is the value.
What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 44/123 List Operations

SNM — ver. 1.7

Hash Examples — 1
I

I

I

As with arrays, you make a new element just by assigning to it: my %fruit; $fruit{apple} = "crunchy"; $fruit{peach} = "soft"; Here, we made two hash elements. N The keys were "apple" and "peach". N The corresponding values were "cruchy" and "soft". You could print the values like this: print "$fruit{apple}, $fruit{peach}\n"; prints: crunchy, soft

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 45/123 List Operations

SNM — ver. 1.7

Hash Examples — 2
I

I

I

How to see if a hash is empty? See 59 print "empty\n" unless %fruit; How to delete a hash element? delete $fruit{coconut}; Hashes are often useful for storing counts (see slides 60–63 for more about while loops): my %wordcounts; while ( <> ) { chomp; ++$wordcount{$_}; }

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 46/123 List Operations

SNM — ver. 1.7

Hash slices
I

I

We can assign some values to part of a hash: $score{fred} = 150; $score{barney} = 100; $score{dino} = 10; We could use a list assignment (see §40):
( $score{fred}, $score{barney}, $score{dino} ) = ( 150, 100, 10 );

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 47/123 List Operations

. . . too long. A hash slice makes this easier:
@score{ "fred", "barney", "dino" } = ( 150, 100, 10 );

I

We can interpolate this too (see slides 36 and 54):
my @players = qw( fred barney dino ); print "scores are @score{@players}\n";

SNM — ver. 1.7

Another Hash Example
I

Often used to keep a count of the number of occurrences of data read in:
#! /usr/bin/perl -w use strict; our %words; while ( <> ) { next unless /\S/; # Skip blank lines my @line = split; foreach my $word ( @line ) { ++$words{$word}; } } print "Words unsorted, in the order they come from the hash:\n\n"; foreach my $word ( keys %words ) { printf "%4d %s\n", $words{$word}, $word; }

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 48/123 List Operations

I see slide 60 for while loop, slide 63 for while ( <> ), slide 66 for the foreach statement, slides 59 and 71 for the unless statement

SNM — ver. 1.7

Hashes are Not Ordered
I I

I

A big difference from arrays is that hashes have no order. The data in a hash will be available in only an unpredictable order. See slide 67 for how to iterate over hash elements

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Funny Characters $, @, % Arrays Array Examples More About Arrays List Assignment Even More About Arrays Scalar, List Context Hashes Initialising a Hash Hash Examples — 1 Hash Examples — 2 Hash slices Another Hash Example Hashes are Not Ordered Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Perl - p. 49/123 List Operations

SNM — ver. 1.7

Discipline—use warnings
I I I

Better to let compiler detect problems, not your customer Develop your program with all warnings enabled Either: N put -w as an option to perl when execute the program, i.e., I Make the first line of your program: #! /usr/bin/perl -w I Or better: put a line: use warnings; near the top of your program.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Discipline—use warnings
use strict and Declaring Variables

Examples of use strict and Variables Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 50/123

use strict and Declaring Variables
I

I

I

I

I

All programs that are more than a few lines long should have the pragma use strict; This turns on additional checking that all variables are declared, all subroutines are okay, and that references to variables are “hard references” — see perldoc strict. All variables that you use in your program need to be declared before they are used with either my or our. my defines a local variable that exists only in the scope of the current block, or outside of a block, in the file. N See perldoc my. our defines a global variable. N See perldoc our.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Discipline—use warnings
use strict and Declaring Variables

Examples of use strict and Variables Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 51/123

Examples of use strict and Variables
I

I

I

I

I

Without use strict, a variable just springs into life whenever you use it. Problem: a typing mistake in a variable creates a new variable and a hard-to-find bug! . . . so always start your programs like this: #! /usr/bin/perl use warnings; use strict; use warnings; enables compile time warnings which help find bugs earlier—see perldoc warnings After use strict, it will be an error to use a variable without declaring it with my or our. N Most code examples in these notes define variables with my or our

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Discipline—use warnings
use strict and Declaring Variables Examples of use strict and Variables

Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 52/123

Operators and Quoting
I

I I

Perl has all the operators from C (and so Java), in same precedence Has more operators for strings: Join strings with a dot, e.g. print "The sum of 3 and 4 is " . 3 + 4 . "\n";
Quote special characters with backslash, as in C or Java

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Operators and Quoting Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics

I

I I

print "\$value = $value\n"; Can quote all characters using single quotes:
print ’output of \$perl = "rapid";print \$perl; is "rapid"’;

I

Note that double quotes are okay in single quotes, single quotes okay in double quotes. Documentation in perldoc perlop.

SNM — ver. 1.7

Perl - p. 53/123

Quoting
I

I

Perl has lots of ways of quoting, too many to list here Meaning Interpolates Slide ’’ q// Literal No §53, §36 "" qq// Literal Yes §53, §36 ‘‘ qx// Command Yes §86 () qw// quote word list No §38,§70 // m// Pattern match Yes §94 s/// s/// Substitution Yes §115 y/// tr/// Translation No N See slide 36 for meaning of “interpolate” y/// or tr/// works just like the POSIX tr (translate) program in Linux.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Operators and Quoting Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics

SNM — ver. 1.7

Perl - p. 54/123

Input and Output
I

I

I

Read from standard input like this: my $value = <STDIN>; Note that there will be a newline character read at the end N To remove trailing newline, use chomp: chomp $value; N The word STDIN is a predefined filehandle. I You can define your own filehandles with the open built-in function. write to standard output with the list operator print N print takes a list of strings: print "The product of $a and $b is ", $a * $b, "\n";

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Input and Output What is Truth? Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics

SNM — ver. 1.7

Perl - p. 55/123

What is Truth?
I I I

I

Anything that has the string value "" or "0" is false Any other value is true. This means: N No number is false except 0 N any undefined value is false N any reference is true (see perldoc perlref) Examples:
0 1 0.00 "" "0.00" undef() # # # # # # becomes the string "0", so false becomes the string "1", so true becomes 0, would convert to the string "0", so false The null string, so false the string "0.00", neither empty nor "0", so true a function returning the undefined value, so false

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Input and Output What is Truth? Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics

SNM — ver. 1.7

Perl - p. 56/123

Statements for Looping and Conditions
I

I

We look at the following statements in the language: N if. . . elsif. . . else statements — §58 I The unless statement is similar to the if statement — §59 N while loops — §60 I processing input using while I The <> operator N for loops — §65 N foreach loops — §66 I iterating over arrays and hashes with foreach, while — §66–§69 N Exit early from a loop with last, and next — §70 We will also look at “backwards statements” — §71–§72

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Statements for Looping and Conditions
if Statements unless Statement while loop

Input with while The Special $_ variable while and the <> operator
while and the <> operator — 2 for loop foreach loop

Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 57/123

if Statements
I

I

if statements work as in C or Java, except: N braces are required, not optional N Use elsif instead of else if Example: if ( $age > $max ) { print "Too old\n"; } elsif ( $age < $min ) { print "Too young\n"; } else { print "Just right\n"; }

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Statements for Looping and Conditions
if Statements unless Statement while loop

Input with while The Special $_ variable while and the <> operator
while and the <> operator — 2 for loop foreach loop

Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 58/123

unless Statement
I

Same as if statement, N except that the block is executed if the condition is false:
unless ( $destination eq $home { print "I’m not going home.\n"; }

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview

corresponds to:
unless (
condition

){

if ( ! (

condition

)){

Data Types Good Practice Operators, Quoting Input, Output Statements Statements for Looping and Conditions
if Statements unless Statement while loop

statements. . .

;

statements. . .

;

}
I

}

else works, but I suggest you don’t use it N Use if. . . else instead

Input with while The Special $_ variable while and the <> operator
while and the <> operator — 2 for loop foreach loop

Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 59/123

while loop
I

Just as in C or Java N . . . but braces are required:
while ( $tickets_sold < 1000 ) { $available = 1000 - $tickets_sold; print "$available tickets are available. "How many do you want: "; $purchase = <STDIN>; chomp $purchase; $tickets_sold += $purchase; } ",

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Statements for Looping and Conditions
if Statements unless Statement while loop

I

Input with while The Special $_ variable while and the <> operator
while and the <> operator — 2 for loop foreach loop

Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 60/123

Input with while
I

I I

Input is often done using while: while ( $line = <STDIN> ) { process this $line } This loop will iterate once for each line of input will terminate at end of file

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Statements for Looping and Conditions
if Statements unless Statement while loop

Input with while The Special $_ variable while and the <> operator
while and the <> operator — 2 for loop foreach loop

Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 61/123

The Special $_ variable
I

I I

I I

I

Nearly every built-in input function, many input operators, most statements with input and regular expressions use a special variable $_ If you don’t specify a variable, Perl uses $_ For example, this while loop reads one line from standard input at a time, and prints that line: while ( <STDIN> ) { print; } while loop reads one line into $_ at each iteration. print statement prints the value of $_ if you do not tell it to print anything else. See the Perl Reference on page 2 under Conventions

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Statements for Looping and Conditions
if Statements unless Statement while loop

Input with while The Special $_ variable while and the <> operator
while and the <> operator — 2 for loop foreach loop

Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 62/123

while and the <> operator
I I

Most input is done using the <> operator with a while loop The <> operator processes files named on the command line N These are called command line parameters or command line arguments N If you execute it like this: angle-brackets.pl then you have no command line arguments passed to the program. N But if you execute it like this: angle-brackets.pl file_1 file_2 file_3 then the command line has three arguments, which here, happen to be the names of files.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Statements for Looping and Conditions
if Statements unless Statement while loop

Input with while The Special $_ variable while and the <> operator
while and the <> operator — 2 for loop foreach loop

Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 63/123

while and the <> operator — 2
I

We most often use the <> operator like this: while ( <> ) {
statements. . .

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Statements for Looping and Conditions
if Statements unless Statement while loop

I

} This loop does a lot. The pseudocode here shows what it does:
if there are no command line arguments, while there are lines to read from standard input read next line into $_ execute statements. . . else for each command line argument open the file while there are lines to read read next line from the file into $_ execute statements. . . close the file

Input with while The Special $_ variable while and the <> operator
while and the <> operator — 2 for loop foreach loop

Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 64/123

for loop
I

I

I

The for loop works as in C or Java, except that braces are required, not optional. Example: for ( $i = 0; $i < $max; ++$i ) { $sum += $array[ i ]; } Note that we rarely use this type of loop in Perl. Instead, use the higher level foreach loop. . .

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Statements for Looping and Conditions
if Statements unless Statement while loop

Input with while The Special $_ variable while and the <> operator
while and the <> operator — 2 for loop foreach loop

Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 65/123

foreach loop
I I I

I

I I I

The foreach loop iterates over an array or list. Most useful looping construct in Perl It is so good, that Java 1.5 has borrowed this type of loop to simplify iterators. An example: adds 1 to each element of an array: foreach my $a ( @array ) { ++$a; } $a here is a reference to each element of the array, so changing $a actually changes the array element. You can write “for” or “foreach”, Perl won’t mind.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Statements for Looping and Conditions
if Statements unless Statement while loop

Input with while The Special $_ variable while and the <> operator
while and the <> operator — 2 for loop foreach loop

Iteration Other Statements List Operations Subroutines Error Handling

SNM — ver. 1.7

Perl - p. 66/123

Iterating over a Hash
I

I

Referring to our example hash in slide 43, we can process each element like this: foreach my $key ( keys %hash ) { process $hash{$key} } N keys creates a temporary array of all the keys of the hash N We then looped through that array with foreach. More efficient is to use the each built in function, which truly iterates through the hash: while ( my ( $key, $value ) = each %hash ) { process $key and $value }

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Iterating over a Hash Iterating over a Hash in Sorted Order Iterating over a Hash in Sorted Order Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 67/123

Iterating over a Hash in Sorted Order
I

I I

Did we process the contents of %hash in alphabetical order in slide 67? N No. N So what do we do if we want to print the elements in order? I In order of key by alphabet? Numerically? I In order of element by alphabet? Numerically? Use built in sort function see perldoc -f sort

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Iterating over a Hash Iterating over a Hash in Sorted Order Iterating over a Hash in Sorted Order Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 68/123

Iterating over a Hash in Sorted Order
I I

I

You cannot sort a hash . . . but you can read all the keys, sort them, then process each element in that order: foreach my $key ( sort keys %hash ) { process $hash{$key} } N see perldoc sort A reverse sort: foreach my $key ( reverse sort keys %hash ) { process $hash{$key} } N see perldoc reverse

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Iterating over a Hash Iterating over a Hash in Sorted Order Iterating over a Hash in Sorted Order Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 69/123

Exit a Loop Early
I I

I

Java and C provide break and continue Perl provides last and next my @super_people = qw( Superman Robin Wonder Woman Batman Superboy ); foreach my $person ( @super_people ) { next if $person eq "Robin"; print "$person\n"; last if $person eq "Batman"; } What do you think this program will print?

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Exit a Loop Early “Backwards” Statements “Backwards” Statements—Examples List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 70/123

“Backwards” Statements
I

I

Put an if, while or foreach modifier after a simple statement. You can put a simple statement (i.e., with no braces), and put one of these afterwards: if EXPR unless EXPR while EXPR until EXPR foreach EXPR

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Exit a Loop Early “Backwards” Statements “Backwards” Statements—Examples List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 71/123

“Backwards” Statements—Examples
I

Examples: N print $1 if /(\d{9})/; is equivalent to: if ( /(\d{9})/ ) { print $1; } N # print unless this is a blank line: print unless /ˆ\s*$/; is equivalent to if ( ! /ˆ\s*$/ ) { print; }

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements Exit a Loop Early “Backwards” Statements “Backwards” Statements—Examples List Operations Subroutines Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 72/123

Array Operations—push and pop
I

The documentation for these is in the very loo–oong document perlfunc, and is best read with perldoc -f
push add a value at the end of an array, e.g., my @array = ( 1, 2, 3 ); push @array, 4; # now @array contains ( 1, 2, 3, 4 ) N Do perldoc -f push pop remove and return value from end of an array my @array = ( 1, 2, 3 ); my $element = pop @array; # now @array contains ( 1, 2 ) # and $element contains 3 N Do perldoc -f pop
Function

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Array Operations—push and pop Array Ops—shift and unshift
split and join

Subroutines Error Handling File and Process I/O Regular Expressions SNM — ver. 1.7 Other Topics Perl - p. 73/123

Array Ops—shift and unshift
shift remove and return value from the beginning of an array, e.g., my @array = ( 1, 2, 3 ); my $element = shift @array; # now @array contains ( 2, 3 ) # and $element contains 1
I
What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Array Operations—push and pop Array Ops—shift and unshift split and join Subroutines Error Handling File and Process I/O Regular Expressions SNM — ver. 1.7 Other Topics Perl - p. 74/123

Do perldoc -f shift

unshift add value to the beginning of an array, e.g., my @array = ( 1, 2, 3 ); unshift @array, 4; # now @array contains ( 4, 1, 2, 3 )
I

Do perldoc -f unshift

split and join
I I

Do perldoc -f split and perldoc -f join. split splits a string into an array:
my $pwline = "nicku:x:500:500:Nick Urbanik:/home/nicku:/bin/bash"; my ( $userid, $pw, $userid_number, $group_id_number, $name, $home_dir, $shell ) = split /:/, $pwline;

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Array Operations—push and pop Array Ops—shift and unshift split and join Subroutines Error Handling File and Process I/O Regular Expressions

I

I

Another application is reading two or more values on the same input line: my ( $a, $b ) = split ’ ’, <STDIN>; join is the opposite of split and joins an array into a string: my $pwline = join ’:’, @pwfields;

SNM — ver. 1.7

Other Topics

Perl - p. 75/123

Subroutines
I I

See perldoc perlsub Syntax: sub subroutine˙name {
statements. . .

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Subroutines Parameters — 1 Parameters — 2 Error Handling File and Process I/O Regular Expressions

}

SNM — ver. 1.7

Other Topics

Perl - p. 76/123

Parameters — 1
I

Subroutines calls pass their parameters to the subroutine in an list named @_. It is best to show with an example: #! /usr/bin/perl -w use strict; sub product { my ( $a, $b ) = @_; return $a * $b; } print "enter two numbers on one line: a b "; my ( $x, $y ) = split ’ ’, <STDIN>; print "The product of $x and $y is ", product( $x, $y ), "\n";

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Subroutines Parameters — 1 Parameters — 2 Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 77/123

Parameters — 2
I I

parameters are passed in one list @_. If you are passing one parameter, then the builtin function shift will conveniently remove the first item from this list, e.g., sub square { my $number = shift; return $number * $number; }

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Subroutines Parameters — 1 Parameters — 2 Error Handling File and Process I/O Regular Expressions

SNM — ver. 1.7

Other Topics

Perl - p. 78/123

Checking for Errors: die and warn
I

I

I

I

I

System calls can fail; examples: N Attempt to read a file that doesn’t exist N Attempt to execute an external program that you do not have permission to execute In Perl, use the die built in function with the or operator to terminate (or raise an exception) on error: chdir ’/tmp’ or die "can’t cd to tmp: $!"; die and warn both print a message to STDERR, but die will raise a fatal exception, warn will continue If no newline at the end of string, die and warn print the program name and line number where were called $! holds the value of the last system error message

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling Checking for Errors: die and warn File and Process I/O Regular Expressions Other Topics

SNM — ver. 1.7

Perl - p. 79/123

Files and Filehandles
I I I I

STDIN, STDOUT and STDERR are predefined filehandles You can define your own using the open built-in function Generally use all upper-case letters by convention Example: open for input: use strict; open PASSWD, ’<’, "/etc/passwd" or die "unable to open passwd file: $!"; while ( <PASSWD> ) { my ( $user ) = split /:/; print "$user\n"; } close PASSWD;

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Files and Filehandles Open for Writing Executing External Programs
system

Was system Call Successful? SNM — ver. 1.7 Was system Call Successful? — 2 Perl - p. 80/123 Backticks: ‘...‘ or qx{...} See the perl summary

Open for Writing
I

I I

To create a new file for output, use “>” instead of “<” with the file name. use strict; open OUT, ’>’, "data.txt" or die "unable to open data.txt: $!"; for ( my $i = 0; $i < 10; ++$i ) { print OUT "Time is now ", scalar localtime, "\n"; } close OUT; Note there is no comma after the filehandle in print To append to a file if it exists, or otherwise create a new file for output, use “>>” instead of “>” with the file name.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Files and Filehandles Open for Writing Executing External Programs
system

Was system Call Successful? SNM — ver. 1.7 Was system Call Successful? — 2 Perl - p. 81/123 Backticks: ‘...‘ or qx{...} See the perl summary

Executing External Programs
I

Many ways of doing this: N system built-in function N backticks N many other ways not covered here.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Files and Filehandles Open for Writing Executing External Programs
system

Was system Call Successful? SNM — ver. 1.7 Was system Call Successful? — 2 Perl - p. 82/123 Backticks: ‘...‘ or qx{...} See the perl summary

system
I

Example: my @cmd = (
’useradd’, ’-c’, "\"$name\"", ’-p’, $hashed_passwd, $id ); print "@cmd\n"; system @cmd; This also works:
system "useradd -c \"$name\" -p \"$hashed_passwd\" $id";

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Files and Filehandles Open for Writing Executing External Programs
system

I I

difference: second form is usually passed to a command shell (such as /bin/sh or CMD.EXE) to execute, whereas the first form is executed directly.

Was system Call Successful? SNM — ver. 1.7 Was system Call Successful? — 2 Perl - p. 83/123 Backticks: ‘...‘ or qx{...} See the perl summary

Was system Call Successful?
I

Check that the return value was zero:
system( "useradd -c \"$name\" -p \"$hashed_passwd\" $id" ) != 0 ){ print "useradd failed"; exit;

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types

if (

}

I

This is usually written in Perl more simply using the built in function die, and the or operator:

Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Files and Filehandles Open for Writing Executing External Programs
system

system( "useradd -c \"$name\" -p \"$hashed_passwd\" $id" ) == 0 or die "useradd failed";

Was system Call Successful? SNM — ver. 1.7 Was system Call Successful? — 2 Perl - p. 84/123 Backticks: ‘...‘ or qx{...} See the perl summary

Was system Call Successful? — 2
I

I usually prefer to call system like this: my @cmd = ( ’useradd’, ’-c’, "\"$name\"", ’-p’, $hashed_passwd, $id ); print "@cmd\n"; system @cmd == 0 or die "Can’t execute @cmd";

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Files and Filehandles Open for Writing Executing External Programs
system

Was system Call Successful? SNM — ver. 1.7 Was system Call Successful? — 2 Perl - p. 85/123 Backticks: ‘...‘ or qx{...} See the perl summary

Backticks: ‘...‘ or qx{...}
I I I

I

Perl provides command substitution Just like in shell programming, where the output of the program replaces the code that calls it: print ‘ls -l‘; Note that you can write qx{...} instead: print qx{df -h /}; N qx// is mentioned in slide 54

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Files and Filehandles Open for Writing Executing External Programs
system

Was system Call Successful? SNM — ver. 1.7 Was system Call Successful? — 2 Perl - p. 86/123 Backticks: ‘...‘ or qx{...} See the perl summary

See the perl summary
I

I I

The Perl summary on the subject web site provides. . . well, a good summary! Called perl.pdf Stored in same directory as these notes

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Files and Filehandles Open for Writing Executing External Programs
system

Was system Call Successful? SNM — ver. 1.7 Was system Call Successful? — 2 Perl - p. 87/123 Backticks: ‘...‘ or qx{...} See the perl summary

Regular Expressions
What is Perl? Example Problem

Regular Expressions are available as part of the programming languages Java, JScript, Visual Basic and VBScript, JavaScript, C, C++, C#, elisp, Perl, Python, Ruby, PHP, sed, awk, and in many applications, such as editors, grep, egrep.

Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting

Regular Expressions help you master your data.
— Sales Department.

Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 88/123 What do they look like? Example: searching for “Course:”

What is a Regular Expression?
I I

I

Powerful. Low level description: N Describes some text N Can use to: I Verify a user’s input I Sift through large amounts of data High level description: N Allow you to master your data

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 89/123 What do they look like? Example: searching for “Course:”

Regular Expressions as a language
I I

Can consider regular expressions as a language Made of two types of characters: N Literal characters I Normal text characters I Like words of the program N Metacharacters I The special characters + ? . * ˆ $ ( ) [ { | \ I Act as the grammar that combines with the words according to a set of rules to create and expression that communicates an idea

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 90/123 What do they look like? Example: searching for “Course:”

How to use a Regular Expression
What is Perl? Example Problem Variables Perl Community

How to make a regular expression as part of your program

The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 91/123 What do they look like? Example: searching for “Course:”

What do they look like?
I

I

I

In Perl, a regular expression begins and ends with ‘/’, like this: /abc/ /abc/ matches the string “abc” N Are these literal characters or metacharacters? Returns true if matches, so often use as condition in an if statement

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 92/123 What do they look like? Example: searching for “Course:”

Example: searching for “Course:”
I

I

I

Problem: want to print all lines in all input files that contain the string “Course:” while ( <> ) { my $line = $_; if ( $line =˜ /Course:/ ) { print $line; } } Or more concisely: while ( <> ) { print if $_ =˜ /Course:/; } or even: print if /Course:/ while <>;

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 93/123 What do they look like? Example: searching for “Course:”

The “match operator” =∼
I

I

I I

If just use /Course:/, this returns true if $_ contains the string “Course:” If want to test another string variable $var to see if it contains the regular expression, use $var =˜ /regular expression/ Under what condition is this true?

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 94/123 What do they look like? Example: searching for “Course:”

The “match operator” =∼ — 2
# sets the string to be searched: $_ = "perl for Win32"; # is ’perl’ inside $_? if ( $_ =˜ /perl/ ) { print "Found perl\n" }; # Same as the regex above. # Don’t need the =˜ as we are testing $_: if ( /perl/ ) { print "Found perl\n" };
What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language SNM — ver. 1.7 How to use a Regular Expression Perl - p. 95/123 What do they look like? Example: searching for “Course:”

/i — Matching without case sensitivity
$_ = "perl for Win32"; # this will fail because the case doesn’t match: if ( /PeRl/ ) { print "Found PeRl\n" }; # this will match, because there is an ’er’ in ’perl’: if ( /er/ ) { print "Found er\n" }; # this will match, because there is an ’n3’ in ’Win32’: { print "Found n3\n" }; if ( /n3/ ) # this will fail because the case doesn’t match: if ( /win32/ ) { print "Found win32\n" }; # This matches because the /i at the end means # "match without case sensitivity": if ( /win32/i ) { print "Found win32 (i)\n" };
What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language SNM — ver. 1.7 How to use a Regular Expression Perl - p. 96/123 What do they look like? Example: searching for “Course:”

Using !∼ instead of =∼
# Looking for a space: print "Found!\n" if
What is Perl?

/ /;

Example Problem Variables

Perl # both these are the same, but reversing the logicCommunity with The Shabang # unless and !˜ Language Overview print "Found!!\n" unless $_ !˜ / /; Data Types print "Found!!\n" unless !˜ / /; Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language SNM — ver. 1.7 How to use a Regular Expression Perl - p. 97/123 What do they look like? Example: searching for “Course:”

Embedding variables in regexps
# Create two variables containing # regular expressions to search for: my $find = 32; my $find2 = " for "; if ( /$find/ ) \{ print "Found ’$find’\n" }; if ( /$find2/ ) \{ print "Found ’$find2’\n" }; # different way to do the above: print "Found $find2\n" if /$find2/; I This is the meaning of the “Yes” under “Interpolates” in the table on slide 54 on the row for m//
What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language SNM — ver. 1.7 How to use a Regular Expression Perl - p. 98/123 What do they look like? Example: searching for “Course:”

The Metacharacters
What is Perl? Example Problem

The funny characters What they do How to use them

Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 99/123 What do they look like? Example: searching for “Course:”

Character Classes [...]
What my @names = ( "Nick", "Albert", "Alex", "Pick" ); is Perl? Example Problem foreach my $name ( @names ) { Variables if ( $name =˜ /[NP]ick/ ) { Perl Community print "$name: Out for a Pick Nick\n"; The Shabang else { Language Overview print "$name is not Pick or Nick\n"; Data Types } Good Practice } Operators, Quoting I Square brackets match one single character Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language SNM — ver. 1.7 How to use a Regular Expression Perl - p. 100/123 What do they look like? Example: searching for “Course:”

Examples of use of [...]
I

I I I

Match a capital letter: [ABCDEFGHIJKLMNOPQRSTUVWXYZ] Same thing: [A-Z] Match a vowel: [aeiou] Match a letter or digit: [A-Za-z0-9]

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 101/123 What do they look like? Example: searching for “Course:”

Negated character class: [ˆ...]
I I

Match any single character that is not a letter: [ˆA-Za-z] Match any character that is not a space or a tab: [ˆ \t]

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 102/123 What do they look like? Example: searching for “Course:”

Example using [ˆ...]
I

I

I

This simple program prints only lines that contain characters that are not a space: while ( <> ) { print $_ if /[ˆ ]/; } This prints lines that start with a character that is not a space: while ( <> ) { print if /ˆ[ˆ ]/; } Notice that ˆ has two meanings: one inside [...], the other outside.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 103/123 What do they look like? Example: searching for “Course:”

Shorthand: Common Character Classes
I

I I

I I I

Since matching a digit is very common, Perl provides \d as a short way of writing [0-9] \D matches a non-digit: [ˆ0-9] \s matches any whitespace character; shorthand for [ \t\n\r\f] \S non-whitespace, [ˆ \t\n\r\f] \w word character, [a-zA-Z0-9_] \W non-word character, [ˆa-zA-Z0-9_]

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 104/123 What do they look like? Example: searching for “Course:”

Matching any character
I I

The dot matches any character except a newline This matches any line with at least 5 characters before the newline: print if /...../;

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 105/123 What do they look like? Example: searching for “Course:”

Matching the beginning or end
I

I I

to match a line that contains exactly five characters before the newline: print if /ˆ.....$/; the ˆ matches the beginning of the line. the $ matches at the end of the line

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 106/123 What do they look like? Example: searching for “Course:”

Matching Repetitions: * + ?
I

{n,m}
What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

I

To match zero or more: N /a*/ will match zero or more letter ‘a’, so matches “”, “a”, “aaaa”, “qwereqwqwer”, or the nothing in front of anything! to match at least one: N /a+/ matches at least one “a” N /a?/ matches zero or one “a” N /a{3,5}/ matches between 3 and 5 “a”s.

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 107/123 What do they look like? Example: searching for “Course:”

Example using .*
$_ = ’Nick Urbanik <nicku@vtc.edu.hk>’; print "found something in <>\bs n" if /<.*>/; # Find everything between quotes: $_ = ’He said, "Hi there!", and then "What\’s up?"’; print "quoted!\n" if /"[ˆ"]*"/; print "too much!\n" if /".*"/;
What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language SNM — ver. 1.7 How to use a Regular Expression Perl - p. 108/123 What do they look like? Example: searching for “Course:”

Capturing the Match with (...)
I

I I

Often want to scan large amounts of data, extracting important items Use parentheses and regular expressions Silly example of capturing an email address: $_ = ’Nick Urbanik <nicku@vtc.edu.hk>’; print "found $1 in <>\n" if /<(.*)>/;

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 109/123 What do they look like? Example: searching for “Course:”

Capturing the match: greediness
I

Look at this example:

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

$_ = ’He said, "Hi there!", and then "What\’s up?"’; print "$1\n" if /"([ˆ"]*)"/; print "$1\n" if /"(.*)"/;
I I

I I I

What will each print? The first one works; the second one prints: "Hi there!", and then "What’s up? Why? Because *, ?, +, {m,n} are greedy! They match as much as they possibly can!

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 110/123 What do they look like? Example: searching for “Course:”

Being Stingy (not Greedy): ?
I I I

Usually greedy matching is what we want, but not always How can we match as little as possible? Put a ? after the quantifier:
*? +? ?? {n,}? {n,m}?

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

Match 0 or more times Match 1 or more times Match 0 or 1 time Match at least n times Match at least n, but no more than m times

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 111/123 What do they look like? Example: searching for “Course:”

Being Less Greedy: Example
I

We can solve the problem we saw earlier using non-greedy matching:

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

$_ = ’He said, "Hi there!", and then "What\’s up?"’; print "\$1\n" if /"([ˆ"]*)"/; print "\$1\n" if /"(.*?)"/;
I

These both work, and match only: Hi there!

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 112/123 What do they look like? Example: searching for “Course:”

Sifting through large amounts of data
I

I

Imagine you need to create computing accounts for thousands of students As input, you have data of the form: N Some heading on the top of each page N More headings with other content, including blank lines N A tab character separates the columns 123456789 H123456(1) 234567890 I234567(2) 345678901 J345678(3) ... ... 987654321 A123456(1)

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 113/123 What do they look like? Example: searching for “Course:”

Capturing the Match: (...)
# useradd() is a function defined elsewhere # that creates a computer account with # username as first parameter, password as # the second parameter while ( <> ) { if ( /ˆ(\d{9})\t([A-Z]\d{6}\([\dA]\))/ ) { my $student_id = $1; my $hk_id = $2; useradd( $student_id, $hk_id ); } }
What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language SNM — ver. 1.7 How to use a Regular Expression Perl - p. 114/123 What do they look like? Example: searching for “Course:”

The Substitution Operator s///
I

I

Sometimes want to replace one string with another (editing) Example: want to replace Nicholas with Nick on input files: while ( <> ) { $_ =˜ s/Nicholas/Nick/; print $_; }

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 115/123 What do they look like? Example: searching for “Course:”

Avoiding leaning toothpicks: /\/\/
I

I

I

I

Want to change a filename, edit the directory in the path from, say /usr/local/bin/filename to /usr/bin/filename Could do like this: N s/\/usr\/local\/bin\//\/usr/\bin\//; N but this makes me dizzy! We can do this instead: N s!/usr/local/bin/!/usr/bin/!; Can use any character instead of / in s/// N For matches, can put m//, and use any char instead of / N Can also use parentheses or braces: N s{...}{...} or m{...}

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 116/123 What do they look like? Example: searching for “Course:”

Substitution and the /g modifier
I I I I I I

If an input line contains: Nicholas Urbanik read “Nicholas Nickleby” then the output is: Nick Urbanik read “Nicholas Nickleby” How change all the Nicholas in one line? Use the /g (global) modifier: while ( <> ) { $_ =˜ s/Nicholas/Nick/g; print $_; }

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 117/123 What do they look like? Example: searching for “Course:”

Readable regex: /x Modifier
I

I I I

Sometimes regular expressions can get long, and need comments inside so others (or you later!) understand Use /x at the end of s///x or m//x Allows white space, newlines, comments See example on slide 13

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Regular Expressions What is a Regular Expression? Regular Expressions as a language

SNM — ver. 1.7

How to use a Regular Expression Perl - p. 118/123 What do they look like? Example: searching for “Course:”

Special Vars: Input Record Separator
I I

I

I

When I described the <> operator, I lied a little As while ( <> ) { ...} executes, it iterates once per record, not just once per line. The definition of what a record is is given by the special built-in variable the Input Record Separator $/ N default value is a newline, so by default read one line at a time But useful alternatives are paragraph mode and the whole-file mode

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics Special Vars: Input Record Separator

SNM — ver. 1.7

Paragraph, Whole-file Modes Perl - p. 119/123 localising Global Variables One Line Perl Programs

Paragraph, Whole-file Modes
I

I

I

I

To input in paragraph mode, put this line before you read input: $/ = ""; Then when you read input, it will be split at two or more newlines N You could split the fields at the newlines To slurp a whole file into one string, you can do: undef $/; $_ = <FILE_HANDLE>; # slurp whole file into $_ s/\n[ \t]+/ /g; # fold indented lines See perldoc -f paragraph, perldoc perlvar and perldoc -f local for important information on how to localise the change to $/.

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics Special Vars: Input Record Separator

SNM — ver. 1.7

Paragraph, Whole-file Modes Perl - p. 120/123 localising Global Variables One Line Perl Programs

localising Global Variables
I

I

I

It is not a good idea to globally change $/, (or even $_) N Your program may use other modules, and they may behave differently if $/ is changed. N Best to localise the change to $/ (or $_, . . . ) Example localising whole-file mode: my $content; open FH, "foo.txt" or die $!; { local $/; $_ = <FH>; } close FH; For paragraph mode, put: local $/ = "";

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics Special Vars: Input Record Separator

SNM — ver. 1.7

Paragraph, Whole-file Modes Perl - p. 121/123 localising Global Variables One Line Perl Programs

One Line Perl Programs
I I I I

Called “one liners” Just execute on the command line See perldoc perlrun Example: edits the files fileA and fileB N makes backups of the original files in fileA.backup and fileB.backup N substitutes all instances of “Silly” and replaces them with “Sensible”. Useful for editing configuration files in shell scripts, automating tasks
N

What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics Special Vars: Input Record Separator

I $ perl -pi ’.backup’ -e ’s/Silly/Sensible/g’ fileA fileB

I

SNM — ver. 1.7

Paragraph, Whole-file Modes Perl - p. 122/123 localising Global Variables One Line Perl Programs

References
I I I I I I
Learning Perl, 3rd Edition, Randal L. Schwartz and Tom Phoenix, ISBN 0-596-00132-0, O’Reilly, July 2001.
What is Perl? Example Problem Variables Perl Community The Shabang Language Overview Data Types Good Practice Operators, Quoting Input, Output Statements Iteration Other Statements List Operations Subroutines Error Handling File and Process I/O Regular Expressions Other Topics Special Vars: Input Record Separator SNM — ver. 1.7 Paragraph, Whole-file Modes Perl - p. 123/123 localising Global Variables One Line Perl Programs

N

The second edition is fine, too. Don’t bother with the first edition, it is too old.

Perl Reference Guide, Johan Vromans, handed out to each one of you, and will be handed out in the final examination. Become familiar with it. Perl for System Administration: Managing multi-platform environments with Perl, David N. Blank-Edelman, ISBN 1-56592-609-9, O’Reilly, July 2000. Perl Cookbook, 2nd Edition, Tom Christiansen and Nathan Torkington, ISBN 0-596-00313-7, O’Reilly, August 2003

N

The first edition is fine, too.

Don’t forget perldoc and all the other documentation on your hard disk. Object Oriented Perl, Damian Conway, ISBN 1-884777-79-1, Manning, 2000. — A more advanced book for those wanting to build bigger projects in Perl.