Linux Essentials by Martin Mser

Data structures: Directories (Verzeichnisse) e.g. /Users/Fritz/Mail

Files (Dateien) Executables (Programme) Links e.g. /Users/Fritz/Mail/Maria/hi5.txt or /home/Fritz/letter.doc e.g. /usr/bin/MS/word.exe

I. Directories / First steps

Navigation through directory trees
What is in your current directory? What is your current directory? cd Mail cd .. cd - Type: ls Type: pwd (means list) (means print working directory)

change directory from current directory to directory with name Mail go one directory up in the directory hierarchy, e.g., from /Users/Mail/Fritz to /Users/Mail brings you back into the directory where you came from / ./ or type name of subdirectory or simply cd

absolute path names begin with a slash relative path names begin with Navigating to your home directory

cd ~

If you are Fritz and you are currently in any arbitrary directory, e.g., in /Users/Maria you can can navigate to your directory Mail in the two following ways: cd cd Mail cd ~/Mail If you are Fritz and you want to see whats on Marias Desktop then type: cd ~maria/Desktop ls

Creating and deleting directories

mkdir Masters rmdir Maria makes a directory called Masters in working directory removes directory called Maria in working directory Maria has to be empty; if it is not and you are sure that you want to delete all of its contents then type rm removes files rather than directories but including the option -r makes the deletion process recursively

rm -r Maria

Options and man pages

Many commands have options. Any option is preceded by a minus sign. For example ls -l gives you a lot longer list than simply ls. 1

(You get information about reading, writing, executing privileges, file size and date) If you want to now what options a command comes with, e.g., ls, then inquire like this: man ls in the manual pages (or short man pages) Man pages tend to be incomprehensible, at least to me, but they can often help your memory. If you want to combine two or more options, you do not need to repeat the minus sign, e.g., the two following commands are equivalent: ls -la and ls l -a -a in the list command plots all files, even those that are hidden, i.e., those that start with a period. For example .bash or .bash_history

Ascertaining the type of data structures in your working directory:

When you type ls l you get a rather long listing. Each line tends to start with a 10 letter combination. What does this tell us: First letter: d stands for directory, stands for file (theres more possibilities) Then we have three times three letters, which indicate read/write/execution privileges for you, your group, rest of the world, i.e., anybody who can access onto your directory (not necessarily with secure shell, see below, but also via browsers) The combination rwx (on positions 2,3,4) means that you can read, write, and execute the file. The combination r-- on positions 8,9,10, means that the rest of the world can read the file but it cannot write/alter the file or execute it. Directories need to be executable so that one can enter them and not only read the names of the files in them.

Some trivia
The reason why Linux is relatively safe is that regular users have no write privileges in sensitive areas, e.g., in the directories where the operational system is located. In addition, when you execute a program, the program inherits your priviledges and not those of the superuser, who is allowed to do essentially anything on a Linux box.

Exercises:
1. Create a directory tree, which you would find useful, or one that is similar to what is on your Windows PC 2. Create directories Mail/Dieter and Masters/Experiment/200912 and Masters/Experiment/201001 3. Go into directory 201001 and change your working directory directly to Dieter. 4. Repeat the previous exercise and see what happens when you type: cd ~/Mail/D and then hit the [tab] key (no space between D and tab) The [tab] key is what tends to make the command line faster than the mouse! 5. Find all files (directories) in your home that are hidden. What could their purpose be? 6. Can you say who can access your home directory?

II. Files / Second steps

Creating files
Most files are either binaries (compiled executables, compressed files, jpegs, word files, etc.) or ascii (simple text in data and text files, source code, LaTeX, etc.) Files can be created in many different ways, e.g., from executables such as Word or OpenOffice, or with Editors, or from an executable that you have compiled, or also from the command line. Note, mkfile is not an analogue to mkdir. One way to create an ascii file from the command line is to type: echo ls > my_first_file What have we done? echo is a command that usually makes its argument (in this case ls) be printed on the screen. If, however, it is used together with the greater sign, then the standard output (in this case the screen) is redirected to a file in our example a file with the name my_first_file. If the file exists, then its content will be replaced with the argument of the echo command. If it does not exist, then it will be overwritten. If you want echo to append to a file, then type, for example, echo ls -l >> my_first_file

Copying files & directories

cp my_first_file ~/Mail/. Copies the file without changing name into the directory ~/Mail. If a file with the same name already exists there, then the old file will be overwritten cp my_first_file new_file creates a copy in the working directory with new name cp i my_first_file new_file Linux will inquire if you want to override new_file if it already exists cp r Mail ~/Archive/. copies the whole directory tree into the directory ~/Archive.

Deleting and moving files

rm my_first_file removes the file my_first_file in working directory rm i my_first_file same as above, but will inquire if you really want to delete mv i old_f ../new_f moves a file to different directory and renames it

Displaying the content of (ascii) files on the command line / pipes

more my_first_file Displays the content of your file onto the screen. If the file does not fit onto the screen the file, it displays one screen at a time. When you hit the space bar, the next screen is displayed.

head -7 my_first_file similar to more but only for the first seven lines tail -7 my_first_file similar to more but only for the last seven lines A difference between more on one hand and head and tail on the other hand is that head and tail do not display the text in screen quanta. The latter can be changed as follows: head -128 my_first_file | more In Linux, the symbol | is called a pipe. It means that the output of the command (here head) is not to be sent to the standard output (usually screen), but into a pipe (meaning a file somewhere on a temporary directory that is not in your directory tree). The data then awaits instructions as to what to do with its content (here display it in screen quanta with more).

Some trivia
In principle, you need nothing but the commands echo, head, and tail to edit an ascii file. The only thing you would still need to learn is how you hide quotation marks when using echo. However, it is wise to resort to editors. Some of them are mentioned in the appendix. Even so, these three commands come in handy many times.

Exercises:
1. 2. 3. 4. Copy your file my_first_file from your home into your directory ~/Mail/Dieter. Delete the file my_first_file in while being in your home. If you ever typed out my_first_file, then repeat exercise 4 from the last chapter. If you had a long data file, and you were only interested in lines 50-55, how would you display them? And append them to an existing file? Hint: The command wc my_first_file will be useful. What does it do?

III Executables / Shell scripts

An executable is any program that runs on your machine. The command ls, for example, is an executable although as system command it is somewhat special. Word is another executable and a compiled C++ code is yet another one.

Making a file executable

Any ascii file that contains a series of Linux commands, such as our my_first_file (which should contain the two commands ls and ls l) can be turned into an executable shell script with the command chmod +x my_first_file changes the mode of the file to be executable chmod -x my_first_file changes the file to be no longer executable or a directory to be no longer accessible The complete chmod command works as follows chmod 754 my_first_file

Each number defines the privileges for you/your group/ the world. r = 4 , w = 2 , x = 1. 7 = 4 + 2 + 1 is equivalent to rwx 5 = 4 + 0 + 1 is equivalent to r x 4 = 4 + 0 + 0 is equivalent to r In our example, the world may read/copy but not execute the code (on your machine).

Executing a shell script (or other executable) in the working directory

Assume, we made the file my_first_file executable again. Then type: clear ./my_first_file You will see that the commands contained in my_first_file are executed as if you had just had typed them in yourself.

Make Linux find your shell script

Some shell scripts are useful for many purposes, but you may still be too lazy to memorize in what directory it is located and/or to always type the complete path name. You can then copy your file into a special directory, where you have many executables. In Linux and Univ many executable are located under root in /bin. What you can do is to create a directory with the same (or a similar) name in your home and copy all your shell scripts and executables there. And this is how you do it: cd mkdir bin cd - cp my_first_file ~/bin/. Now, if you use the bash shell (sorry, this is dialect dependent), edit your ~/.bashrc file and include the following two lines: PATH="$PATH:/Users/MyName/bin export PATH For a different dialect, you may need to replace Users with home or something else. PATH is a Linux variable. It contains all the directories in which your bash shell is going to search for executables. (.bashrc is executed each time you invoke the bash shell, e.g., when you open a terminal). Linux actually goes through the list of directories one after the other and takes the first executable that fits the description. So if you create a shell script called ls, then you can only execute it by calling it with its absolute path name If you want to know, where Linux finds an executable, e.g., where it finds the system call ls, then type: which ls

Pass an argument to your shell script

Will be discussed upon request / appendix.

Exercises
1. Create directories Dieter and Maria and Xaver in your ~/Mail directory. Allow yourself to access Dieter, to read the file names in Maria but not to acces it, and to change into Xaver, but not to be able to list the files.

2. Create a directory ~/myprograms. There design a shell script called make_my_file that creates an empty file. 3. Include myprograms in the list of directories to be searched by Linux.

IV Links
They are similar to links under Windows. The details are not important in a 4-hour tutorial.

V Customizing your account

Aliases
One line shell scripts are better written as aliases. An alias is a shorthand notation of a longer command that you can teach your shell in the .bashrc file. So you could include the following lines in .bashrc. alias ..='cd ..' now type only .. to go up one level in the directory tree alias cp='cp -i' .. to prevent accidental overwriting of old files alias del='rm -i' .. to prevent accidental deleting of old files alias myxterm='xterm -fn lucidasanstypewriter-18 &' .. gives you an X terminal with very large letters the ampersand & makes the command run in the background Aliases are particularly useful, when you frequently access or copy to remote computers, see section VII. You can also type an alias command on the command line, but it will be forgotten after you log off.

Prompt
It is nice to know the name of your working directory. You can see it if you include the following line in .bashrc PS1="\W\$ " PS1 is another shell variable

Make bash know the changes

source .bash_rc

Exercise
1. Create an alias and an executable in your bin directory that both have the same name, e.g., myls. (They may be doing different things.) Which one is executed by the shell when you type myls?

VI Job handling
Finding job IDs
ps aux shows the users and process ID (PID) of others and myself if you skip the a, youll only see yours 6

top

shows processes with highest most CPU-intensive time

Killing jobs
kill 36549 allows the job to with PID 36549 terminate certain input/output kill -9 36549 kills the job without further ado mycommand [CTRL C]

Sending jobs into the background

mycommand [CTRL Z] mycommand &

Bringing jobs to the foreground

jobs fg 2 fg lists all jobs running in the background brings job with background ID 2 to the foreground brings job with highest background ID to the foreground

VII Connecting to other Unix/Linux machines

Logging in remotely: ssh
A great advantage of Linux is the simplicity with which you can access other Linux computers, e.g., those that have your simulation or experimental results, or one where you share text files with collaborators. You simply call the command: ssh Y your_user_name@foreignhost.univ.edu The option Y makes sure that you can use the X terminal from where you called ssh (secure shell) as if it were managed from foreignhost, i.e., it exports the terminal. The Y option is less save but often better than the X option. Unfortunately, it seems to be difficult to ssh onto computers at Universitt des Saarlandes. This is the first time, ssh did not work for me. Whos on the system who tells you who is logged onto the system who am i tells you who you are and from where you logged on (e.g. console) These two commands and the following ones can also be called from your local host. Exiting the system exit logout [CTRL] D only works on the console on your local host (not any X terminal) also means end-of-file when keyboard is standard input

Copying to remote machine: scp

Another great advantage of Linux is how easy it is to copy files to and from other Linux computers. You can simply preced the cp command with s and thereby copy to a remote machine. If you copy frequently to the same machine (or even subdirectory in the

machine), you can create an alias or shell script, so that you only have to type something like my_scp_mars filename and your done. No mouse clicking and navigating through directories necessary._ scp p localfile your_user_name@foreignhost.univ.edu:Mail/. The option p preserves the file properties (date of creation, privileges, etc.) of your original file on your local computer, which copies it to your Mail directory in the home of the remote computer. If you want to copy from a remote computer to a local one, type something like: scp r your_user_name@foreignhost.univ.edu:Mail/. ~/Archive/. The option r means recursive copying, that is, the Mail directory with all its subdirectories including their contents will be copied. If the directory contains a lot of data, you may want to zip it up, which you can do on the command line. See appendix.

VIII Short cuts

Some of the following short cuts may be system or setting dependent, for example, it may require the lineediting mode to be set to emacs, which is the default in bash. TAB completes command or file/directory names if choice is unique TAB TAB suggests all possible completions (commands, files, directories) [CTRL] a makes the cursor jump to the beginning (anfang) of command line [CTRL] e makes the cursor jump to the end of command line [CTRL] c cancel job [CTRL] z suspend job [can then be sent to background with bg ] [CTRL] d end of file / exit [page up] goes up one command on your command line [page down] goes down one command on your command line

IV Miscellaneous
History
Each shell creates a history file, which saves the last few commands that you typed into the command line, e.g., .bash_history in your home. Assume you want to repeat a previous scp command that you used, then type history | grep scp then your history file will be dumped into a pipe (see section II). The command grep will make sure that only those lines will be displaced As a result of the history command, you will get lines such as: > 421 scp file1 fritz@zuse.de:. > 430 scp file2 maria@zuse.de:. If you want to repeat the first out of these two option, then simply type

! 421

executes line 421 in your .bash_history file

If you want to repeat the last out of these two option, then simply type ! scp executes the last in your .bash_history that started with scp. Never ever type !r or !rm These are absolute no goes! because you want to avoid the worst case scenario, which would be that the last command starting with r was rm r * and you are currently in your home or as root in the root of the directory tree. It is actually safest to type rm some_very_strange_letter_combination after you used rm together with a wild card. The history command is another reason why you can perform tasks that repeat very often extraordinarily quickly. Even if you have to change some parts of the last scp command, you could mouse those that you want to copy.

Wildcards
Sometimes you want to list or find files that satisfy certain search criteria, for example, all files that have the name fritz or Fritz in them. Then you can use wildcards (space fillers) to substitute for all other letters. For example, you could type ls *[fF]ritz* ... lists all files that contain either fritz or Fritz While * substitutes for any number of letters, ? will replace any single letter. For example ls *.f9? ... would list all Fortran 90 and Fortran 95 source codes (plus potentially other files), as those end with .f90 and .f95. respectively.

Finding files
Want to see a listing of all word files in a directory tree (such as home): find . name *.doc print the . tells linux to search in your working directory keep in mind that you can pipe this command or redirect into a file, which would be great if you wanted to make an inventory of all mp3 files, for example.

Finding stuff in files

grep mystring myfile will print to screen all lines in myfile that contain the string mystring.

Change ownership
chown newowner file1 file2 file3 changes ownership of said files (directories) to said user. You must have permissions to change the ownership, i.e., be owner or root.

Taring and compressing from the command line

Theres tons of flags and option, but I find the following lines the most useful ones: tar cvf - ./* > /tmp/archive.tar compress /tmp/archive.tar You tar the current directory into a tar ball, which will be located in /tmp. The latter is a standard directory in Linux. It is not in your home, and it will be emptied in regular intervals. The second line compresses the tar ball. And this is how you untar the whole thing: zcat archive.tar.Z | tar xvf -

Writing forbidden characters such as space

Windows allows empty spaces in names for files and directories. For example, New Report would be a legitimate file name. In Linux, these names are allowed too (though no intelligent person would create them), however, you have to tell Linux that you are not dealing with two files, but with just one. This can be done by hiding the not quite forbiddenspace with a slash. So you would type something like: ls New\ Report*

Change Passwd
Simply type on the command line: passwd you will be asked to type the old password once and the new one twice.

Viewing files
Postscript files: PDF files: jpegs, gifs, : ghostview gv, .. with xpdf, preview, gv gimp gv myfile.ps xpdf myfile.ps gimp myfile.jpg

Appendix: Special Programs

Webpages: http://linuxreviews.org/beginner/

A: Editors
pico / gedit Easy to use. Mostly for beginners. emacs
The standard choice for many developpers. Pretty efficient many shortcuts.

vi / vim
Awful in the beginning, but extremely fast if you know a few short cuts. My absolute personal favorite. 10

Most Linux now comes with a Windows like surface. If you right click onto a file on your desktop that ends with .txt, then your Linux will offer you text editors. In most cases, you should be able to execute that text editor also from the command line. .

B: Plotting / gnuplot
Webpages : http://www.duke.edu/~hpgavin/gnuplot.html gnuplot is a great program to display and fit data quickly and also does an OK job with 3d plots. For final figures, that is, those that make it into your thesis or into a publication, I would recomment xmgrace or something else (mathematica, maple,) To invoke gnuplot simply type: gnuplot

Elementary plotting
gnuplot> help plot gnuplot> plot cos(x) check that out, it is pretty self-explanatory known special functions are, e.g., exp, log, sinh powers are indicated as in fortran with ** otherwise, syntax is closer to that of C, C++ set xr [0:2*pi] plot data.dat u 3:2 w lp u is shorthand for using u 3:2 indicates that you plot 3rd column as x and 2nd column as y w lp is shorthand for linepoints default is to plot 1:2. You can also plot 0:2 (0 indicating the row number). If your file only consists of one column, the 0:1 is default. set log y unset log y plot data.dat u ($1-0.5):($2*log($3)) w lp When you perform mathematical operations on a column, you have to proceed it with a $ sign. Here we plot column 1 shifted by 0.5 as the x-axis and the product of column 2 and log of column 3 as y-axis. plot data.datw d, cos(x) You can plot several data files set xl pressure (GPa) xl is shorthand for xlabel [page up/down] Brings back the previous/next command. Which is usually to be preferred to (unless youre done and plot into postscript or something else) replot self-explanatory quit self-explanatory

gnuplot> gnuplot>

gnuplot> gnuplot> gnuplot>

gnuplot> gnuplot> gnuplot>

gnuplot> gnuplot>

Link on fit with gnuplot: http://theochem.ki.ku.dk/on_line_docs/gnuplot/gnuplot_21.html

11

Required data format for gnuplot

gnuplot likes simple ascii files / columns. Any line starting with a dagger # is ignored. Thus, comments and descriptions of the lines can be hidden rather well behind #.

Launching Linux commands from within gnuplot

gnuplot> ! ls l The exclamation mark tells gnuplot that you want to launch a shell command.

Saving and loading your gnuplot session

gnuplot> save mygnusession.gnu Creates a file with many, many commands. If you want to know there meaning try to ascertain it with help. gnuplot> load mygnusession.gnu Loads the file with the last set of your gnuplot commands. Typically, you would then want to launch the following command: gnuplot> !tail -2 mygnusession.gnu | head -1 which would dump the last plot command onto your terminal. This line can then be moused into gnuplot and modified for your purposes.

Saving your plot as a figure

gnuplot> set term post por col with set term you redefine what syntax gnuplot is going to send to the standard output (right now our stdo is the screen, but in a second we will redefine it as regular file). post postscript por portrait col color (default is black and white) alternatives to post would have been, e.g., pdf or jpeg or latex gnuplot> set out myplot.ps Now you changed the stdo to a file and will plot into it, when you say gnuplot> replot gnuplot> set size 0.5,0.3 gnuplot does not follow the principle, what you see is what you get. Often, you will want to resize your plot so that letter are bigger relative to symbols, etc. Resizing a graph for final output can be a bit tedious.

3D plots
gnuplot> splot data.dat u 1:2:3 There are many ways how you can modify 3D plots, which are learned quickly. These will be included in later iterations of this tutorial.

Defining variables
gnuplot> a=1.57 now you can simply use a as a variable. Certain letter combinations cannot be use as variable names

12

Fitting
gnuplot> fitf(x) = fp1*x**2 + fp2*x**3 gnuplot> fp1 = 1 fp2 = 1 Defines a function f(x) and defines and initializes parameters fp1 and fp2. gnuplot> fit fitf(x)data.dat u 1:2 via fp1, fp2 This will perform the fit and assign new values to the fitparameters. gnuplot> plot fitf(x),data.dat will plot the original function and the fitted values.

Exercises
1. Plot exp(sin(x)) in the range from 0,..,pi. Define a title of the plot as follows: gnuplot> set title I feel bad Write the output as postscript into a file myplot.ps. View the file with ghostview of equivalent from within gnuplot. Edit the postscript file from within gnuplot and change bad with good. 2. Download the data file datensatz.dat from http://www.lms.uni-saarland.de/Lehre/09ECMS/Week05/datensatz.dat Fit the data set to the function f(x) = a x2 + b x3. 3. Create a nice looking jpeg graph from the fit.

C: Passing arguments to shell script

Sometimes you may want to let the shell script know some information from the command line. For example, if you often convert files from ps to pdf, then you would have to type commands like pdf2ps myfile.pdf myfile.ps It would be faster if you only had to invoke a shell script, which would read as follows: mypdf2ps myfile You can achieve this, by passing myfile as a variable (the first variable) to the script: pdf2ps $1.pdf $1.ps

C: Awk
Great program to manipulate data on the command line and also to do some moderate programming. Many commands are similar to C++, but it does not need to be compiled. awk '{print $1 "\t" $2*$1^2} < input.dat > output.dat \t < > produces a tab standard input (usually keyboard) comes from file input.dat standard output (usually screen) goes to output.dat

13