Create Shell Scripts

Overview

The Linux shell can control the system with

commands and perform file operations or start
applications
You can create a file that includes shell
commands and start this file like an
application

A Linux system offers different shell types

This is called a shell script

Scripts written for one shell may not work in another

shell

LINUX uses the bash shell as the default

bash = Bourne Again Shell

Create Shell Scripts

Overview

There are reasons to understand and create

shell scripts
You can automate many daily system tasks which
reduces time and effort to manage the system

They control the boot procedure and other system

functions

Shell scripting is relatively easy to learn

They run on almost every UNIX-like operating

system

There are also some minor disadvantages

Shell scripts are slow compared to other languages

They can use a lot of CPU power

Create Shell Scripts

Objectives
1. Understand the Course Project
2. Use Basic Script Elements
3. Use Variables and Command Substitution
4. Use Control Structures
5. Use Arithmetic Operators
6. Read User Input
7. Use Arrays
8. Finalize the Course Project
9. Use Advanced Scripting Techniques
10. Learn About Useful Commands in Shell
Scripts

Use Basic Script Elements

The Basic Rules of Shell Scripting

A shell programming language is a powerful

and complete programming language
A shell script is an ASCII text file containing
commands to be executed in sequence
To allow this the following permissions need to
be set on the file

r (readable) and x (executable) for the user

running it

These permissions are not granted by default on

a newly created file but you can do it with the
following command

chmod +x script.sh

Use Basic Script Elements

The Basic Rules of Shell Scripting

You can also run the script without setting the

permissions using sh or bash
/bin/sh is a symbolic link to /bin/bash

Example: sh script.sh or bash script.sh

The script directory must be in the user's search

path
Otherwise must bes started using full path names

A good way to do this is to create a /bin directory in the

users home directory and add it to the users PATH

Do this in the ~.bashrc

Example: export PATH=$PATH:~/bin

Always give script files a .sh extension so you

know that it is a shell script (Not required)

Use Basic Script Elements

The Request Tracker Entries in the Mailbox

File

Each mail message start with the line

Each mail message ends with #Ticket which is the

actual content of the mail

From wwwrun@linux-rwke.site ...

All other lines are headers which carry meta data and
transportation details of the mail

The #Ticket line has the information of the

request submitted through the web interface and
has multiple fields separated by the | character

The fields have the following meaning

Subject of Request | Email Address of Sender |

Request Category | Description of Request

Use Basic Script Elements

The Request Tracker Entries in the Mailbox

File

Start with a script that simply outputs the

mailbox file to the terminal
#!/bin/bash
#basic_script1.sh
cat /var/spool/mail/geeko

The first line tells Linux kernel to pass this file

to the bash shell to be executed
The second line is a comment since it starts
with #
The last line uses the cat command to open
the mailbox file and print it to the terminal

Use Basic Script Elements

The Request Tracker Entries in the

Mailbox File

When you execute the script the output

does not look very well-formed since we
are only interested in the mail, not the
headers so let's improve it
#!/bin/bash
#basic_script2.sh
cat /var/spool/mail/geeko | grep '#'

The output of cat is now passed to grep

which only prints the lines that include a #

Use Basic Script Elements

The Request Tracker Entries in the

Mailbox File

Another improvement would be to add a start

message to the script
#!/bin/bash
#basic_script3.sh
echo -e Welcome to the Request tracker. \nThe
following are open requests:\n
cat /var/spool/mail/geeko | grep '#'

The echo command is used to output text, which

is enclosed in double quotes, to the terminal

The option -e lets echo interpret backslash sequences

The sequence \n is used to creates a line break (newline) in the output

Use Basic Script Elements

The Request Tracker Entries in the

Mailbox File

Backslash sequences that can be used

with echo
\\ - Outputs a backslash
\a - Outputs an alert (beep tone)
\b - Outputs a backspace
\c - Outputs a suppress training new-line
\f- Outputs a form feed
\n - Outputs a new-line
\r- Outputs a carriage return
\t- Outputs a horizontal tab
\v - Outputs a vertical tab

Variables and Command

Substitution

Variables

Variables are important in all programming

languages
They are memory locations (containers) that
hold data
Instead of the actual data the variable is used
in the program code
For example
#!/bin/bash
#variable_1.sh
a=Geeko
echo Hello, my name is $a

Variables and Command

Substitution

Variables

The string geeko is assigned to the variable a

which is then used in the echo command
The important things are

When you assign a variable just use the name of

the variable

When you access the data of a variable use the $

before the variable name

When you assign data to a variable there can be

no spaces between the variable name, the = and
the data

The output would look like this

geeko@linux-rwke:~/Shell_Scripting> ./variables_1.sh
Hello, my name is Geeko

Variables and Command

Substitution

Variables

Variables can contain strings and numbers as

well
By default a variable can hold any kind of
data but it is possible to limit a variable to
a specific type using the declare command
It is also possible to assign the output of a
command to a variable or to use a
command directly where the output is
needed

Called a command substitution

Variables and Command

Substitution

Command Substitution

Means that the output of a command is used

in a shell command line or a shell script
Example that uses the output of the command
date to generate the output of the current date
#!/bin/bash
#command_subs_1.sh
echo Today is `date +%m/%d/%y`

Note the use of the backticks (not single

quotes)

Variables and Command

Substitution

Command Substitution

Instead of printing the output of a command to

the screen with echo, it can also be assigned
to a variable
#!/bin/bash
#command_subs_2.sh
TODAY=`date +%m/%d/%y`
echo Today is $TODAY

The output of date is assigned to the

variable TODAY which is then printed to
the screen with echo

Variables and Command

Substitution

Request Tracker

Now improve the Request tracker by

including the number of open requests
#!/bin/bash
#basic_subs_3.sh
tinum=`cat /var/spool/mail/geeko | grep '#' |
wc -l`
echo -e Welcome to the Request
tracker. \nThere are $tinum open
requests:\n
cat /var/spool/mail/geeko | grep '#'

Use Control Structures

Introduction

So far we have only learned to write

scripts that run sequentially from
beginning to end
We now learn how to use control
structures to make the execution of
parts of the script dependent on certina
conditions or to repeat script parts
We will learn how to
Create Branches

Create Loops

Use Control Structures

Create Branches

A branch in a script means that a part of

the script is only executed under a certain
condition
A common control structure for this uses
the if command
if condition
then
commands
fi

If the condition is true then one or more

commands are executed

Use Control Structures

Create Branches

The if statement can be extended with

an optional else statement
if condition
then
command1
else
command2
fi

Command2 is executed when the if

condition is false

Use Control Structures

Request Tracker

New we add an if structure to the Request

Tracker to modify the start message so that
if there are more than 10 open requests the
user sees a warning message
#!/bin/bash
#control_struc1.sh
tinum=`cat /var/spool/mail/geeko | grep '#' | wc -l`
if test $tinum -gt 10
then
echo -e Welcome to the Request tracker. \nThere are $tinum open
requests. You have to do something!\a
else
echo -e Welcome to the Request tracker. \nThere are $tinum open
requests:\n
fi
cat /var/spool/mail/geeko | grep '#'

Use Control Structures

Create Branches

The program test is used to test the condition

Almost all command line tools have a return
value of 0 which means that everything is OK
When a value other than 0 is returned an
error is indicated
An if condition is true when the program uses
for testing returns 0
Note that the lines after then and else are
indented which is not required but makes the
code easier to read

Use Control Structures

Create Branches

The program test can be used for many

things

test STRING1 = STRING2 The strings are equal

test STRING1 != STRING2 The strings are not equal
test INTEGER1 -eq INTERGER2 The integers are equal
test INTEGER1 -lt INTERGER2 INTEGER1 is less than INTEGER2
test -e FILE FILE exists

You can also use a different syntax for

test, where the expression is put in
square brackets and the test command
is left out
[12 -eq 14]

Use Control Structures

Create Branches

With if more complex structures can

also be used by using elif to add more
conditions in case the one in the
initial if statement was not true
if test $number -eq 10
then echo The value is 10
elif test $1 -eq 20
then echo The value is 20
else
echo I don't know
fi

Use Control Structures

Create Branches

Another way to create multiple branches

is the case statement
The expression in a variable is compared
with a number of expressions

Commands are executed for the first

expression that matches

case $number in
10) echo The value is 10;;
20) echo The value is 20;;
*) echo I don't know
esac

Use Control Structures

Create Loops

A loop is often used when a task needs to be

repeated more than once since it is better
than repeating the same code multiple times

There are several options for implementing a loop

Can use the for loop where the line with for
determines how many times to execute the
commands

With each pass of the loop the variable gets the

next value in the list after in

for variable in element1 element2 element3

do
commands
done

Use Control Structures

Create Loops

An example for loop

#!/bin/bash
#for_loop_1.sh

for i in 1 2 3
do
echo $i
done

Use Control Structures

Create Loops

The list defined after in is not always static

The following example loops through all
files in the current working directory and
renames files from upper to lower case

The * is expanded to a list of all the files by the

shell

#!/bin/bash
#for_loop_2.sh

for i in *
do
lower=`echo $i | tr | [upper:] [:lower;]`
mv $i $lower
done

Use Control Structures

Create Loops

Another way of creating a list uses

command substitution
The following example uses find to
create a list of all .mp3 files in the
current directory and all subdirectories
and they are deleted
#!/bin/bash
#for_loop_3.sh

for i in `find -name *.mps`

do
rm $i
done

Use Control Structures

Create Loops

Another loop construct is the while loop where

the commands are executed as long as the
condition is true
while condition
do
commands
done

Another loop construct is the until loop

where the commands are executed until the
condition becomes true
until condition
do
commands
done

Use Control Structures

Create Loops

We use the while loop to make the

output of the Request Tracker more
readable

#!/bin/bash
#while_loop_1.sh
...
while read line
do
if echo $line | grep '#' > /dev/null
then
echo $line | grep '#' | cut -d '|' --output-delimeter=', ' -f '2 3 4'
fi
done < /var/spool/mail/geeko

Use Control Structures

Create Loops

Explanation of while_loop_1.sh
The file /var/spool/mail/geeko is fed into
a while loop using input redirection (<)

The command read is used to process

the data line by line

For every cycle of the loop the variable

$line has one line

The loop is terminated when all lines are

processed

read returns a value not equal to 0

Use Control Structures

Create Loops

Explanation of while_loop_1.sh
In the loop, grep tests is a line contains a #

If so, cut is used to display only the subject,

email address and category

The cut options explained

-d '|'

- Defines delimiter used to split the input into fields

--output-delimiter=','
- Defines that a , separates the output fields

-f '2 3 4'
- Lets cut only output the fields 2, 3 and 4

Use Arithmetic Operators

Overview

Shell scripts often use values assigned to

variables for calculations
The Bash shell comes with built-in support
for arithmetic operations, but with the
following limitations
Only operations with whole numbers

All values are signed 64-bit values

When using Bash you might need to use

external commands

Always perform slower than built-in commands

Use Arithmetic Operators

Methods and Formats

Possible methods and formats for doing A=B+10

Use the external command expr
A=`expr $B + 10`

Use the Bash built-in command let

let A=$B + 10

Use arithmetic expressions inside parentheses or

brackets
A=$((B + 10)) or A=$[B + 10]

Use the built-in command declare

declare -i A ; declare -i B
A=B+10

Use Arithmetic Operators

Request Tracker

Use an arithmetic operator to assign a

number to each Request tracker request
#!/bin/bash
#arithmetic_1.sh
...
number=1
while read line
do
if echo $line | grep '#' > /dev/null
then
echo -e $number|$line | grep '#' | cut -d '|' --outputdelimeter=', ' -f '1 2 3 4 5'
((number++))
fi
done < /var/spool/mail/geeko

Read User Input

Overview

One way to read user input is to use the read

command which stores the read input in a
variable which can then be used to process
the user input
read VARIABLE

The script pauses at this point waiting for the

user input and then the Enter key
To tell the user what is expected use the echo
command
echo Please enter a value for the variable:
read VARIABLE

Read User Input

Request Tracker

Extend the Request Tracker to be more

interactive by allowing the user to enter a
command after the open requests have
been displayed
#!/bin/bash
#read_user_1.sh
...
input='blank'
until test $input = quit
do
number=1
while read line
do ... done < /var/spool/mail/geeko
read input
done

Use Arrays

Overview

Arrays are basically variables that hold

more than one value and use a
numerical index in square brackets to
identify the value in the arry
lines[1]=Hello World

To access a value in an array specify the

index and put curly braces around the
array name
echo $(lines[1])

Use Arrays

Request Tracker

Extend the Request Tracker to use

arrays
#array_1.sh
...
input='blank'
until test $input = quit
do
reqnr=1
while read line
do
if echo $line | grep '#' > /dev/null

then
lines[$reqnr]=#reqnr|$line
((reqnr++))
done < /var/spool/mail/geeko

Use Arrays

Request Tracker

Extend the Request Tracker to use

arrays

#array_1.sh
...
do
reqnr=1
...
for (9i=1;i<=${#lines[@]};i++))
do
echo -e $number|$line | grep '#' | cut -d '|' --outputdelimeter=', ' -f '1 2 3 4 5'
done
unset lines
read input
done

Use Arrays

Request Tracker

In the while loop the requests are stored in the array $lines
and the value of the index $reqnr is incremented in every
cycle of the while loop

In the for loop the content of the array is echoed to the

terminal using a different syntax of the for loop

For ((i=1;i<=${#lines[@]};i++)) means that the loop runs as

long as the variable $i is smaller or equal to the number of
elements in the $lines array

${#lines[@]} is a way to access the

number of elements in the array

At the end of the while loop the array lines are deleted
using unset command

Use Advanced Scripting

Techniques

Introduction

In this objective we learn some

advanced scripting techniques to help
solve common script development
problems
Use Shell Functions

Read Options with getopts

Use Advanced Scripting

Techniques
Use Shell Functions

Sometimes you need to preform a task

multiple times in a shell script can this can be
done using functions
Shell functions act like script modules and
make an entire script available under one
name
They are normally defined at the beginning of
a script
Can store several functions in a file and
include this file where the functions are needed

Use Advanced Scripting

Techniques
Use Shell Functions

Basic Syntax
functionname () {
commands
commands
}

To generate a function using the function

command
function functionname {

commands
commands
}

The function name can be any regular character

string

Use Advanced Scripting

Techniques

Use Shell Functions

A simple function that creates a directory

and then changes to that directory
mcd () {
mkdir $1
cd $1
}

The function can be called in a shell

script
...

...

mcd /tmp/new_directory

Use Advanced Scripting

Techniques

Use Shell Functions

The parameter /tmp/new_directory is an

argument
Within a function argument can be
accessed with the variables $1, $2, $3, ...
The following function can be used to
create a pause in a script that is then
resumed after pressing the Enter key
pause (){
echo To continue, hit RETURN.
read q
}

Use Advanced Scripting

Techniques
Use Shell Functions

You can also create functions that stop their

processing from within, similar to exiting a
loop with the commands break and continue
To exit a function, use the return command
If called without an argument the return
value is identical to the exit status of the
last command executed in that function

Otherwise the return value is identical to the

one supplied as an argument to return

Use Advanced Scripting

Techniques
Read Options with getopts

getopts is a shell built-in command which can

extract the script options provided on the command
line
The shell interprets arguments as command line
options only if they are prefixed with a -

Makes it possible to place options in any order so that

cp -dpR *.txt text/ is the same as cp -R *.txt -d text/ -p

getopts recognizes options in the same way

using the syntax getopts optionstring variable

The optionstring describes all options to be recognized

For example getopts abc declares a, b and c as the

options to be processed

Use Advanced Scripting

Techniques
Read Options with getopts

If a parameter is expected the option (such

as -m maxvalue) then it must be followed
by a : in the string, such as getopts m:
The optionstring is followed by a variable
to which all of the command line options
specified are assigned to as a list
The getopts command is mostly used in a
while loop together with a case statement
to define which command to execute for a
given option

Use Advanced Scripting

Techniques
Read Options with getopts

The getopts command is mostly used in a

while loop together with a case statement
to define which command to execute for a
given option
while getopts abc: variable
do
case $variable in

a)

echo The option -a was used. ;;

b)
echo The option -b was used. ;;
c ) option_c=$OPTARG
echo The option -c has been set. ;;
esac

done
echo $option_c

Useful Commands in Shell

Scripts

Introduction

This objective gives an overview of useful

commands that can be used in shell scripts
and is intended as a reference
Use the cat command

Use the cut command

Use the date command

Use the grep and egrep command

Use the sed command

Use the test command

Use the tr command

Useful Commands in Shell

Scripts

The cat Command

When combined with the here operator

(<<) a good choice to output several
lines of text from a script
Interactively, it is mostly run with a
filename as an argument and prints to
standard output

Useful Commands in Shell

Scripts

The cut Command

Use this command to cut out sections of

lines from a file so only the specified section
is printed on standard output

Can specify single or several sections

The command is applied to each line of text

as available in a file or on standard output
Use cut -f to cut out text fields
Use cut -c to work with specified characters
Use cut -d to specify a field separator

A tab is the default

Useful Commands in Shell

Scripts

The cut Command

Example to cut and print the first field to

standard output
cut -d : -f1 /etc/passwd
root
bin
daemon
...

Example to take the output of ls and cut

out everything from the thirty-fifth
character, pipe it to sort and have is sorted
by file size
ls -l somedir/ | cut -c 35- | sort -n

Useful Commands in Shell

Scripts

The date Command

Use whenever have a need to obtain a

date or time string
Output with no options
date
Fri Sep 03 14:18:12 CEST 2004

The output can be formatted using

options
The -I option prints the date and time
in ISO format

Same as +%Y-%m-%d

Useful Commands in Shell

Scripts

The grep and egrep Commands

Used to search one or multiple files for

certain patterns
The syntax is grep searchpattern filename ...
The output is all lines that contain the pattern
Many options are available to do things like
Only print the line numbers

Print the line along with leading and trailing

context lines

Search patterns can be supplied as regular

expressions
The regular grep has limited capabilities

Useful Commands in Shell

Scripts

The grep and egrep Commands

Can use egrep or grep -E for more complex

patterns
The regular expressions are in the standard
regex syntax
To avoid having special characters in the
seach patterns interpreted by the shell
enclose the pattern in quotation marks
tux@DA1:~> egrep (b|B)lurb file*
bash: syntax error near unexpected token |
tux@DA1:~> egrep (b|B)lurb file*

file1:blurb
file2:Blurb

Useful Commands in Shell

Scripts

The sed Command

The sed program is a stream editor used

from the command line, not interactively
Does text transformations on a line-byline basis
Can specify sed commands directly on
the command line or in a special
command script loaded by the program
The syntax is sed editing-command
filename

Useful Commands in Shell

Scripts

The sed Command

The available editing commands are singlecharacter arguments such as

d:
s:
p:
a:

Delete
Substitute (replace)
Output line
Append after

The output normally goes to standard output

and can also be redirected to a file
Each sed command must be preceded by an
exact address or address range specifying the
lines to which the editing command applies

Useful Commands in Shell

Scripts

The sed Command

Options can be specified to influence the

overall behavior of the program
-n, --quit, --silent

By default, prints all lines on standard output

This option suppresses the output so only prints
lines for which the p editing option has been given
to re-enable printing

-e command1 -e command2 ....

Necessary when specifying two or more editing

commands
Must be inserted before each additional editing
command

Useful Commands in Shell

Scripts

The sed Command

Options can be specified to influence the

overall behavior of the program
-f filename

Sometimes need to specify exact line or

lines to be processed and there are
special characters you can use

Can specify a script file from which to read its editing

commands

$ stands for the last line

Command to print only lines 1 through 9

sed -n '1,9p' somefile

Useful Commands in Shell

Scripts

The sed Command

Command to delete everything from line 10 to the end

and print the first 9 lines
sed '10,$d' somefile

Regular expressions can be used to define addresses

for a command and must be enclosed in forward
slashes
Command to print all lines that have the pattern
Murphy.* in them
sed -n '/Murphy.*/p' somefile

To perform several editing commands for the same

address need to enclose the commands in braces
sed '1,10{command1;command2}'

Useful Commands in Shell

Scripts

The sed Command

d
a
i
c
s

Important editing commands

sed 10,$d file

Delete line
sed 'a\text\text' file Insert text before specified line
sed 'i\text\text' file Replace specified lines with the text
sed '2000,$c\text' file Replace specified lines with the text
sed s/x/y/option
Search and replace
Patterns are regular expressions
y sed y/abc/xyz/
(yank) Replace every character
from
the set of source characters with the
character in the same position in the
destination characters

Useful Commands in Shell

Scripts

The sed Command

Options can use with the s command

for search and replace

I Do not distinguish between uppercase and lowercase

letters
g Replace globally
n Replace the nth matching pattern only
p Print the line after replacing
w Write the resulting text to the specified file rather than
printing it on stdout

Replace the first colon in each line

with a space
sed 's/:/ /' /etc/passwd

Useful Commands in Shell

Scripts

The sed Command

Replace all colons in all lines with a space

sed 's/:/ /g' /etc/passwd

Replace only the second colon in each line with a

space
sed 's/:/ /2' /etc/passwd

Replace all single vowels with double vowels

showing how matched patterns can be referenced
with \1 if the search pattern is given in
parentheses
sed -n 's/\([aeiou]\)/\1\1/Igp'

The I option ensures that the case is ignored

The g option causes characters to be replaced globally

The p option prints all lines processed in this way

Useful Commands in Shell

Scripts

The test Command

Exists as both a built-in command and as an

external command
Used to compare values and to check for file and
their properties, such as if it exists, is
executable, etc.
If the tested condition is true then test returns an
exit status of 0 and if not true returns an exit
status of 1
In shell scripts mainly used to declare conditions
to influence the operation of loops, branches and
other statements
The test syntax is test condition

Useful Commands in Shell

Scripts

The test Command

Test whether a file exists

-e

File exists

-f File exists and is a regular file

-d
File exists and is a directory
-x
File exists and is an executable file

Compare 2 files
-nt
-ot
-ef
link)

Newer than
Older than
Refers to same inode (hard

Useful Commands in Shell

Scripts

The test Command

Compare 2 integers
-eq

-ne
-gt
-lt
-ge
-le

Equal

Not equal
Greater than
Less than
Greater than or equal
Less than or equal

Useful Commands in Shell

Scripts

The test Command

Test strings
test -z string

True if string has 0 length

test string
True if string has nonzero length
test string1 = string2 True if strings are equal
test string1 != string2 True if strings are not equal

Combine tests
test ! condition
- True if condition is not true
test condition1 -a condition2 - True if both are true
test condition1 -o condition2 - True if either is true

Useful Commands in Shell

Scripts

The tr Command

Command translates (replaces) or deletes

characters
Reads from standard input and prints to
standard output
can replace regular characters or sequences of
regular characters and special characters, like \t
or \r
Standard syntax is tr set1 set2 where the
characters in set1 are replaced with the
characters in set2
To replace all lowercase characters with uppercase
cat text-file | tr a-z A-Z

Useful Commands in Shell

Scripts

The tr Command

To delete characters from set1 and print

the rest to standard output
tr -d set1

To delete the percent sign from the

original value of VAR and assign the
result as the new value of the variable
VAR='echo $VAR | tr -d %'

Replace a set of characters with a

single character
tr -s set1 char