Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
FOSSEE
Department of Aerospace Engineering IIT Bombay
1 / 83
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 2 / 83
8 9 10 11
Introduction
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 3 / 83
8 9 10 11
Introduction
What is the Linux OS? Free Open Source Operating System Free Free as in Free Speech, not Free Beer Open-Source Permit modications and redistribution of source code Unix-inspired Linux Kernel + Application software Runs on a variety of hardware Also called GNU/Linux
4 / 83
Introduction
Why Linux?
Free as in Free Beer Secure & versatile Why Linux for Scientic Computing? Free as in Free Speech Can run for ever Libraries Parallel Computing
5 / 83
Getting Started
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 6 / 83
8 9 10 11
Getting Started
Logging in
GNU/Linux does have a GUI Command Line for this module Hit Ctrl + Alt + F1 Login
7 / 83
Getting Started
Where am I?
8 / 83
Getting Started
What is in there?
9 / 83
Getting Started
New folders
mkdir creates new directories $ mkdir sees $ ls
Special characters need to be escaped OR quoted
10 / 83
Getting Started
Moving around
cd command changes the pwd $ cd sees $ pwd /home/user/sees/
Alternately written as cd ./sees Specifying path relative to pwd
$ cd /home/user/sees/
FOSSEE (IIT Bombay) Using Linux Tools 11 / 83
Getting Started
New les
touch command creates a blank le $ pwd /home/user $ cd sees $ touch first $ ls first
12 / 83
Getting Help
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 13 / 83
8 9 10 11
Getting Help
$ man man
FOSSEE (IIT Bombay) Using Linux Tools 14 / 83
Getting Help
$ ls -R
Create a new directory along with parents, if required
Getting Help
16 / 83
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 17 / 83
8 9 10 11
Removing les
rm is used to delete les $ rm foo rm works only for les; not directories
Additional arguments required to remove a directory
Copying Files
cp copies les from one location to another $ cp linux-tools/scripts/foo linux-tools/
New le-name can be used at target location
foo copied to new location with the name bar $ cp linux-tools/scripts/foo linux-tools/bar cp overwrites les, unless explicitly asked not to To prevent this, use the -i ag
Copying Directories
20 / 83
Moving Files
cp and rm would be one way mv command does the job Also takes -i option to prompt before overwriting
$ ls course sees became a sub-directory of course mv command doesnt over-write directories -i option is useful when moving les around mv to rename move to same location with new name
FOSSEE (IIT Bombay) Using Linux Tools 21 / 83
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 22 / 83
8 9 10 11
23 / 83
drwxr-xr-x
5 root users
The rst column shows the permission information First character species type of the le Files have -; Directories have d 3 sets of 3 characters for user, group and others
24 / 83
chmod command is used -R option to recursively change for all content of a directory
Change permissions of foo.sh from -rw-r--r-- to
25 / 83
Symbolic modes
Reference u g o a Operator + = Class user group others all
Description the owner of the le users who are members of the les group users who are not hte owner of the le or members of all three of the above; is the same as ugo
Description adds the specied modes to the specied classes removes the specied modes from the specied classes the modes specied are to be made the exact modes for the spe Name read write execute Description read a le or list a directorys contents write to a le or directory execute a le or recurse a directory tree
Using Linux Tools 26 / 83
Mode r w x
27 / 83
Looking at les
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 28 / 83
8 9 10 11
Looking at les
cat
$ cat foo.txt
Concatenates the text of multiple les
29 / 83
Looking at les
less
$ less wonderland.txt
q: Quit Arrows/Page Up/Page Down/Home/End: Navigation ng: Jump to line number n /pattern: Search. Regular expressions can be used h: Help
30 / 83
Looking at les
wc
Statistical information about the le the number of lines in the le the number of words the number of characters
$ wc wonderland.txt
31 / 83
Looking at les
$ head wonderland.txt -n option to change the number of lines $ head -n 1 wonderland.txt tail is commonly used to monitor les -f option to monitor the le Ctrl-C to interrupt $ tail -f /var/log/dmesg
FOSSEE (IIT Bombay) Using Linux Tools 32 / 83
Looking at les
cut
Allows you to view only certain sections of lines Lets take /etc/passwd as our example
root:x:0:0:root:/root:/bin/bash
View only user names of all the users (rst column)
$ cut -d : -f 1 /etc/passwd -d species delimiter between elds (default TAB) -f species the eld number
Multiple elds by separating eld numbers with comma
Looking at les
cut
Allows choosing on the basis of characters or bytes Example below gets rst 4 characters of /etc/passwd
34 / 83
Looking at les
paste
Joins corresponding lines from two different les
students.txt
Hussain Dilbert Anne Raul Sven
marks.txt
89 92 85 98 47 67 67 82 76 78 97 60 67 68 69
$ paste students.txt marks.txt $ paste -s students.txt marks.txt -s prints content, one below the other
If rst column of marks le had roll numbers? How do we get a combined le with the same output as above (i.e. without roll numbers). We need to use cut & paste together. But how?
FOSSEE (IIT Bombay) Using Linux Tools 35 / 83
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 36 / 83
8 9 10 11
$ cut -d " " -f 2- marks1.txt \ > /tmp/m_tmp.txt $ paste -d " " students.txt m_tmp.txt
or
$ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt The rst solution used Redirection The second solution uses Piping
37 / 83
Redirection
The standard output (stdout) stream goes to the display Not always, what we need First solution, redirects output to a le
$ command > file1 > creates a new le at specied location appends to a le at specied location
38 / 83
Redirection . . .
39 / 83
Redirection . . .
Following example shows stderr redirection Error is printed out in the rst case Error message is redirected, in the second case
$ cut -d " " -c 2- marks1.txt \ > /tmp/m_tmp.txt $ cut -d " " -f 2- marks1.txt 1> \ /tmp/m_tmp.txt 2> /tmp/m_err.txt 1> redirects stdout; 2> redirects stderr $ paste -d " " students.txt m_tmp.txt
FOSSEE (IIT Bombay) Using Linux Tools 40 / 83
Piping
$ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt - instead of FILE asks paste to read from stdin cut command is a normal command the | seems to be joining the two commands Redirects output of rst command to stdin, which becomes
input to the second command This is called piping; | is called a pipe
41 / 83
Piping
42 / 83
Tab-completion
Hit tab to complete an incompletely typed word Tab twice to list all possibilities when ambiguous completion Bash provides tab completion for the following.
1 2 3 4 5 6
File Names Directory Names Executable Names User Names (when they are prexed with a ) Host Names (when they are prexed with a @) Variable Names (when they are prexed with a $)
43 / 83
History
Bash saves history of commands typed Up and down arrow keys allow to navigate history
44 / 83
meta characters are special command directives File-names shouldnt have meta-characters
/<>!$%^&*|{}[]"~; $ ls file.*
Lists file.ext les, where ext can be anything
$ ls file.?
Lists file.ext les, where ext is only one character
45 / 83
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 46 / 83
8 9 10 11
sort
47 / 83
sort . . .
$ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt -\ | sort -t " " -k 2 -rn -t species the delimiter between elds -k species the eld to use for sorting -n to choose numerical sorting -r for sorting in the reverse order
48 / 83
grep
$ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt | grep Anne grep is case-sensitive by default
49 / 83
grep . . .
-i for case-insensitive searches $ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt | grep -i Anne -v inverts the search
To see everyones marks except Annes
$ cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt | grep -iv Anne
50 / 83
tr
tr translates or deletes characters Reads from stdin and outputs to stdout
Given, two sets of characters, replaces one with other The following, replaces all lower-case with upper-case
$ tr -s \n \n
51 / 83
tr . . .
-d deletes all specied characters
Only a single character set argument is required The following command removes carriage return characters (converting le in DOS/Windows format to the Unix format)
$ tr -cd [:alnum:]
52 / 83
uniq
uniq command removes duplicates from sorted input $ sort items.txt | uniq uniq -u gives lines which do not have any duplicates uniq -d outputs only those lines which have duplicates -c displays the number of times each line occurs $ sort items.txt | uniq -u $ sort items.txt | uniq -dc
53 / 83
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 54 / 83
8 9 10 11
Shell scripts
Simply a sequence of shell commands in a le To save results of students in results.txt in marks dir
#!/bin/bash mkdir ~/marks cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt - \ | sort > ~/marks/results.txt
55 / 83
Shell scripts . . .
56 / 83
57 / 83
echo
echo command prints out messages #!/bin/bash mkdir ~/marks cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt - \ | sort > ~/marks/results.txt echo "Results generated."
58 / 83
#!/bin/bash mkdir ~/marks cut -d " " -f 2- marks1.txt \ | paste -d " " students.txt - \ | sort > ~/marks/$1 echo "Results generated." $1 corresponds to rst command line argument $n corresponds to nth command line argument
It can be run as shown below
$ ./results.sh grades.txt
FOSSEE (IIT Bombay) Using Linux Tools 59 / 83
PATH
The shell searches in a set of locations, for the command Locations are saved in environment variable called PATH
60 / 83
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 61 / 83
8 9 10 11
Control Structures
if-else for loops while loops test command to test for conditions
A whole range of tests that can be performed STRING1 = STRING2 string equality INTEGER1 -eq INTEGER2 integer equality -e FILE existence of FILE
62 / 83
if
63 / 83
if-else
Checks whether argument is negative or not
#!/bin/bash if test $1 -lt 0 then echo "number is negative" else echo "number is non-negative" fi $ ./sign.sh -11
64 / 83
#!/bin/bash if [ $1 -lt 0 ] then echo "number is negative" else echo "number is non-negative" fi
spacing is important, when using the square brackets
65 / 83
if-else
An example script to greet the user, based on the time
#!/bin/sh # Script to greet the user # according to time of day hour=date | cut -c12-13 now=date +"%A, %d of %B, %Y (%r)" if [ $hour -lt 12 ] then mess="Good Morning \ $LOGNAME, Have a nice day!" fi
66 / 83
if-else . . .
if [ $hour -gt 12 -a $hour -le 16 ] then mess="Good Afternoon $LOGNAME" fi if [ $hour -gt 16 -a $hour -le 18 ] then mess="Good Evening $LOGNAME" fi echo -e "$mess\nIt is $now" $LOGNAME has login name (env. variable)
backquotes store commands outputs into variables
67 / 83
for
Problem Given a set of .mp3 les, that have names beginning with numbers followed by their names 08 - Society.mp3 rename the les to have just the names. Also replace any spaces in the name with hyphens. Loop over the list of les Process the names, to get new names Rename the les
68 / 83
for
A simple example of the for loop
for
Lets start with echoing the names of the les
tr & cut
Replace all spaces with hyphens using tr -s Use cut & keep only the text after the rst hyphen
while
Continuously execute a block of commands until condition becomes false program that takes user input and prints it back, until the input is quit
while [ "$variable" != "quit" ] do read variable echo "Input - $variable" done exit 0
72 / 83
Environment Variables
Pass information from shell to programs running in it Behavior of programs can change based on values of variables Environment variables vs. Shell variables Shell variables only current instance of the shell Environment variables valid for the whole session Convention environment variables are UPPER CASE
73 / 83
Environment Variables . . .
The following commands show values of all the environment variables
$ export PATH=$PATH:$HOME/bin
74 / 83
Miscellaneous Tools
Outline
1 2 3 4 5 6 7
Introduction Getting Started Getting Help Basic File Handling Linux File Hierarchy, Permissions & Ownership Looking at les The Command Shell Features of the Shell More text processing Simple Shell Scripts Control structures and Operators Miscellaneous Tools
FOSSEE (IIT Bombay) Using Linux Tools 75 / 83
8 9 10 11
Miscellaneous Tools
find
Find les in a directory hierarchy Offers a very complex feature set Look at the man page! Find all .pdf les, in current dir and sub-dirs
$ find . -type d
76 / 83
Miscellaneous Tools
cmp
77 / 83
Miscellaneous Tools
diff
> indicates content only in second le < indicates content only in rst le
78 / 83
Miscellaneous Tools
tar
tarball essentially a collection of les May or may not be compressed Eases the job of storing, backing-up & transporting les
79 / 83
Miscellaneous Tools
Extracting an archive
$ $ $ $
mkdir extract cp allfiles.tar extract/ cd extract tar -xvf allfiles.tar -x Extract les within the archive -f Specify the archive le -v Be verbose
80 / 83
Miscellaneous Tools
Creating an archive
81 / 83
Miscellaneous Tools
Compressed archives
82 / 83
Miscellaneous Tools
Bash reads /etc/profile, ~/.bash_profile, ~/.bash_login, and ~/.profile in that order, when starting up as a login shell.
83 / 83