Lab Manual Vol1 Nov13 2014-1

John Milton
Stefan van der Walt

Caleb Miles
Fernando Perez
Emily Nordhoff
Arthur H. Lee
Toru Ohira
Lab manual: Mathematics as a

laboratory tool
Vol. 1 (Fall semester)

November 13, 2014
Springer
v
Dedication
On behalf of the contributors I dedicate these laboratory experiences to the
late John D. Hunter (1968–2012). Although all of us knew John either as a col-
league and/or a teacher, few realized that many of the underlying pedagogical con-
cepts for these laboratories arose from 20 years of conversations between John and
me starting when he was my graduate student. I can still remember the day that
John, a second year PhD student, walked into my office and demanded that we no
longer use commercial software packages in our research, but instead use open-
source resources, in particular Python. My apprehensions were soon dispelled when
I checked out the costs of maintaining software licenses for my students after grad-
uation. Thus the decision to use Python as our programming language was born.
John did not have the patience to do boring, repetitive bench top experiments by
hand. Thus it was not surprising that a year later John had figured out how to get
a computer to run his all of his recording and stimulation protocols for his neuro-
physiological experiments on the motoneuron of the sea slug, Aplysia. Indeed, the
experiments described in his 2001 Journal of Neuroscience article were done while
we sat together talking, occasionally sipping a glass of wine, while the computer did
the work while flashing the information on computer screens. Thus the decision to
incorporate engineering concepts such as filtering and signaling and ultimately the
use of an Arduino board was made. John got tired about my insistence to prepare
figures for publication by hand since I was not satisfied with figures prepared using
commercial software packages. Beginning during his post-doctoral years in my lab
John developed his famous graphical software package, matplotlib. In 2004 I
moved to the Claremont Colleges and soon after John and I began taking about how
scientific computing could be taught to undergraduate science students in a liberal
arts environment. He invited me to a Scientific Python conference at Caltech, where
with Andrew Straw, we began to formulate the idea of a scientific computing work-
shop for undergraduate students. The first SciPy workshop was held in fall, 2007.
With the addition of Fernando Perez, the fall semester SciPy workshops continued
yearly until 2012. Many of the exercises that our students enjoyed, e.g. the Moiré
dot program in Lab 4, the bird song spectrogram in Lab 9 and the filtering moon
photograph exercise in Lab 9, are exercises that were initially developed for these
workshops. John emailed me from his hospital bed to tell me that he might not be
able to do the 2012 SciPy workshop later that fall. The greatest reward that a teacher
can receive is to be taught by their students. In knowing John D. Hunter I believe
that I have been well blessed.
John G. Milton
Preface
The weekly laboratory experiences represent the way we teach the material pre-
sented in Mathematics as a laboratory tool. We believe that a weekly 4-hour labo-
ratory experience is an essential component of a course that teaches mathematical
tools to researchers who work in the laboratory. These laboratory sessions provide a
unique opportunity for the lecturer to ensure that concepts are well understood and
can be implemented by every student. In our experience the “lights go on” when the
student sees the result predicted by their paper and pencil calculations evolve on the
computer that they control.
The laboratory exercises for the fall semester (Chapters 1-10) are given in Vol-
ume 1 and those for the spring semester are given in Volume 2. The primary pur-
pose of the first three laboratories is to introduce students into three programming
languages that are used throughout: LaTeX, Python, and XPPAUT. Many of the lab-
oratories are designed to introduce resources available in Python that are not always
well–documented, for example, resources related to the handling of jpg images and
animation. At the beginning of the course students log on to a Linux box that con-
tains these program and learn to use these as well as elementary commands neces-
sary to work in a Linux environment. As they become familiar with the use of these
programming languages they are required to install these programs on their personal
laptop or desktop computer. Most students accomplish this by the 6-th Lab and cer-
tainly before the 12-th Lab where this is a compulsory requirement. Installing the
software packages on each student’s laptop (which they do themselves) is essential
in order for them to translate the skills earned into their scientific activities after they
complete the course.
vii
Acknowledgments
The laboratory exercises were developed over many years and could not have been
developed without the volunteer efforts of many people. In particular we acknowl-
edge the efforts of Walter Cook (Harvey Mudd College) for the construction of
electronic circuits, Boyle Ke (Keck Science Center for IT support, Emily Nordhoff
for the development of thr Ardiuno lab and Caleb Mills (Class of ‘11, Claremont
McKenna College) for lab instruction and the development of basic Python pro-
grams.
ix
Contents
1 Communicating science(Lab 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Introducing XPPAUT (Lab 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Exercise 1: First XPPAUT program . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3 Exercise 2: Second-order DDEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Exercise 3: Phase plane representations . . . . . . . . . . . . . . . . . . . . . . . . 14
3 Introducing Python: Working with data (Lab 3) . . . . . . . . . . . . . . . . . . . 17

3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2 Exercise 1: Making figures using Matplotlib . . . . . . . . . . . . . . . . . . . . 20
3.3 Exercise 2: Adding a figure to a LaTeX document . . . . . . . . . . . . . . . 25
4 Working with arrays and eigenvalues (Lab 4) . . . . . . . . . . . . . . . . . . . . . 27

4.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2 Exercise 1: Stability using XPPAUT . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.3 Exercise 2: Stability using Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.4 Exercise 3: Moiré-Glass dot patterns . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.5 Exercise 4: Working with digital images . . . . . . . . . . . . . . . . . . . . . . . 37
5 The ’cubic’ nonlinearity: Excitable systems (Lab 5) . . . . . . . . . . . . . . . . 41

5.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.2 Exercise 1: The van der Pol oscillator . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.3 Exercise 2: Analog implementation of the van der Pol oscillator . . . . 43
5.4 Exercise 3: The Fitzhugh-Nagumo (FHN) neuron . . . . . . . . . . . . . . . . 45
5.5 Exercise 4: Reduced neuron models . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6 Bifurcation diagram: stopping spiking neurons with a single pulse

(Lab 6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
6.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
6.2 Exercise 1: Making one-parameter bifurcation diagrams . . . . . . . . . . 55
6.3 Exercise 2: Bifurcation diagram: Hodgkin-Huxley neuron . . . . . . . . . 58
xi
xii Contents
6.4 Exercise 3: Stopping spiking neurons with a single pulse . . . . . . . . . . 61
7 Lab 7: Filters and convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

7.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
8 Lab 8: Fourier series: Gibbs phenomenon and filtering (Chapter 8) . . 73

8.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
8.2 Exercise 1: Gibbs phenomenon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
8.3 Exercise 3: Convolution: Graphical interpretation . . . . . . . . . . . . . . . . 75
8.4 Exercise 4: Making a sine wave from a square wave . . . . . . . . . . . . . . 76
9 Lab 9: FTT and power spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

9.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
9.2 Exercise 1: Computing the power spectral density using mlab.psd() . 83
9.2.1 Power Spectral Density Recipe . . . . . . . . . . . . . . . . . . . . . . . . . 83
9.2.2 Example using mlab.psd() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
9.3 Exercise 2: Low-frequency signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
9.4 Exercise 3: FFT Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
9.4.1 Convolution: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
9.4.2 Filtering images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
9.5 Spectrograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
10 Phase resetting oscillators (Lab 10) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

10.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
10.2 Exercise 1: Phase resetting curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
10.3 Exercise 2: Arnold tongue diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
10.4 Exercise 3: Phase locking modes for standard circle map . . . . . . . . . . 100
10.5 Exercise 4: Phase locking modes for the periodically spiking HH
neuron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
10.6 Supplementary material: Phase resetting curves using XPPAUT . . . . 102
11 Blood cell dynamics: Oscillations, delays and chaos (Lab 11) . . . . . . . . 107
11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
11.1.1 Integrating DDEs using XPPAUT . . . . . . . . . . . . . . . . . . . . . . . 108
11.1.2 Integrating DDEs using PyDelay . . . . . . . . . . . . . . . . . . . . . . . 110
11.2 Exercise 1: Periodic anemias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
11.3 Exercise 2: Chaotic hematology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
12 Arduino boards: Using your laptop in the laboratory(Lab 12) . . . . . . . 115

12.1 Exercise 1: Blinking LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
12.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
12.1.2 The Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
12.1.3 The Potentiometer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
12.1.4 A few more things to know . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
12.1.5 The Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
12.2 Exercise 2: Measuring Tremor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Contents xiii
12.2.1 What is an accelerometer? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

12.2.2 The Code - Writing the Sketch . . . . . . . . . . . . . . . . . . . . . . . . . 123
12.2.3 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
12.2.4 Data from fingertip movements . . . . . . . . . . . . . . . . . . . . . . . . 127
A Appendix A: Finding data for analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
B Appendix B: Latex survival guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

B.1 LaTeX manuals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
B.2 LaTeX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
B.2.1 Text Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
B.2.2 Latex versus pdflatex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
B.2.3 Viewing the LaTeX file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
B.3 Learning LaTeX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
C Appendix C: Introduction of Python by A. H. Lee . . . . . . . . . . . . . . . . . . 145

C.1 Key references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
C.2 How to run a Python interpreter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
C.2.1 Interactive mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
C.2.2 Script mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
C.3 Basic data types and their values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
C.4 Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
C.5 Use of the print command in a script file . . . . . . . . . . . . . . . . . . . . . 149
C.6 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
C.7 Operating on values: operators and operands . . . . . . . . . . . . . . . . . . . . 151
C.8 Order of precedence among operators . . . . . . . . . . . . . . . . . . . . . . . . . . 152
C.9 Input from keyboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
C.10 Functions and packages (libraries) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
C.11 Conditionals: doing it conditionally . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
C.12 Loops: doing it repeatedly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
C.13 Compound data types: strings, lists, tuples . . . . . . . . . . . . . . . . . . . . . . 159
C.14 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
C.14.1 The for loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
C.15 Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
C.16 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
C.17 Tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
C.18 Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
C.19 Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
C.19.1 Text files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
C.19.2 Exception handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
C.20 More Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
C.21 NumPy, SciPy, etc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
xiv Contents
D Appendix D: The Scientific Python Ecosystem by F. Perez . . . . . . . . . . 181

D.0.1 Scientific Python: a collaboration of projects built by
scientists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
D.0.2 A note about the examples below . . . . . . . . . . . . . . . . . . . . . . . 183
D.1 Motivation: the trapezoidal rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
D.2 NumPy arrays: the right data structure for scientific computing . . . . 185
D.2.1 Basics of Numpy arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
D.2.2 Indexing with other arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
D.2.3 Arrays with more than one dimension . . . . . . . . . . . . . . . . . . . 189
D.2.4 Operating with arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
D.2.5 Linear algebra in numpy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
D.2.6 Reading and writing arrays to disk . . . . . . . . . . . . . . . . . . . . . . 196
D.3 High quality data visualization with Matplotlib . . . . . . . . . . . . . . . . . . 198
D.3.1 Image display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
D.3.2 Simple 3d plotting with matplotlib . . . . . . . . . . . . . . . . . . . . . . 203
D.4 IPython: a powerful interactive environment . . . . . . . . . . . . . . . . . . . . 203
D.4.1 The IPython terminal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
D.4.2 The graphical Qt console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
D.4.3 The IPython Notebook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
Chapter 1
Communicating science(Lab 1)
We would all like to be good citizens of the scientific community. An important

part of being a good citizen is being able to communicate results, papers, and ideas.
Since many of our colleagues live in different parts of the world and hence have
different resources available to them we cannot always assume that they have the
same commercial software package (including upgrades) for preparing manuscripts
that we do. Moreover, how can equations be exchanged easily by e-mail?
A modern day solution is to make use of the open source computing network
available on the Internet. In other words, we only require that our colleagues have
access to the Internet. There are all sorts of computer software packages available
on the Internet; many of them are exactly the same softwares that are presently used
at the cutting edges of scientific research (for example, check out the SciPy web-
site). A big advantage is that once we become familiar with these products we have
access to them anywhere in the world. Study abroad students from The Claremont
Colleges have already been able to use their experience with the open source com-
puting network to help students, workers and scientists in the countries they have
visited.
Computer setup: This lab manual has been written with the assumption that
your instructors have placed LaTeX, XPPAUT and Python onto a central Linux
computer that can be accessed either using computer terminal in a computer lab-
oratory or, perhaps, by logging into using your laptop. However, it is now relatively
easy to install all of these packages onto your own laptop for free. This is the pre-
ferred way to do these labs because you retain the use of these tools for your own
purposes after you complete the course1 .
• LaTeX: See for example latex-project.org/ftp.html. Most students
using PC’s first installed MikTek (miktek.org) followed by Winshell (e.g.
1 Beginning fall, 2014 we require all students in this course to install these computer packages
onto their personal laptops. We do this in three stages: Week 1 (LaTeX); Week 2 (XPPAUT) and
Week 3 (Python). Our experience is that installing these packages is easier for students with PC’s
than those possessing Mac’s. Nonetheless all students with perseverance and the help of IT can
accomplish these tasks.
1
2 1 Communicating science(Lab 1)
www.winshell.de). Latex if often provided on the disks that come with the
purchase of a new Mac.
• XPP/XPPAUT: See www.math.pitt.edu/˜bard/xpp.html.
• Python: We very strongly recommend that students install Enthought’s Canopy
Express (https://store.enthought.com). A huge advantage is that
packages required to work with images and animation can readily be installed
and managed through Canopy Express.
Today’s lab:
In this lab we address how to prepare manuscripts containing equations and fig-
ures. The software package we will use is called LaTeX. This is the same software
package that is used by the publishers of most journals and books that you read
which contain equations. LaTeX is a completely different concept than, for exam-
ple, a word processor such as WORD. Packages such as WORD are actually pro-
grams that produce the document the way that the programmers who designed this
package wanted it to be produced. In contrast, when we use LaTeX we write the
program ourselves that will be used to generate the document. In fact, for many of
you preparing a LaTeX document will be the first computer program that you have
written! Since we are writing the program ourselves we can get an article that looks
exactly like we want it to look. More importantly, the program is written using a text
editor2 and hence has the form of an ASCII file. This means that we can easily send
it to our colleagues by email and they can then run it locally on their own computer
to generate the document.
It is really not very hard to learn to use LaTeX. We focus on the skill set necessary
to make the most basic LaTeX file (Appendix B). Of course there are many advanced
features available in LaTeX that are available which you can learn about as you
search the Internet and/or discuss LaTeX with your colleagues. However, our goal
is to be able to collaborate with others. We cannot assume that our colleagues know
these advanced features or are even interested in learning how to use them. Thus, in
our experience, collaboration is most successful when lines of communication are
kept as simple as possible.
Most people learn LaTeX by doing it, i.e. their first experience is usually the re-
sult of having to prepare their own paper because the journal publisher has asked for
it. This is in fact how you will learn to use LaTeX. Your first assignment will be to
reproduce two pages from a bio-mathematics book (see section labeled deliverable
at end of the laboratory description). Of course it is necessary to know the basic
structure of the LaTeX file: the various commands that you need come from looking
them up as you need them. This is why the Internet is so useful. Indeed we had to
look up a number of commands that we had never used before in order to make up
these notes! In this way we know them now!
2 A text editor is an editor which does not add any hidden format commands to the document. In
other words, what you see is what you get. In Microsoft environments, the text editors are Notepad
and Wordpad; in LINUX two commonly used text editors are vi and emacs.
1 Communicating science(Lab 1) 3
Background: The main components of a LaTeX file are: 1) Preamble (Top mat-
ter), 2) Main body, 3) Math modes, 4) Figures and Tables, and 5) Bibliography. An
introduction to math modes is desribed in Appendix B which is available on the
website. The use of the basic commands are illustrated in today’s lab. We will dis-
cuss how figures are introduced into a document in Lab 3. We do not need to know
how to construct Tables or a bibliography for our purposes; however, these issues
are well documented on the Internet. Here we discuss the basic format of a LaTeX
file.
Preamble: The first line in the LaTeX program has the form
\documentclass[12pt]{article}
The term ’12pt’ means twelve points and sets the size of the print (This document
has been printed using 12pt.). Points are the units that type-setters use to define the
size of characters on the printed page of, for example, a newspaper page. If you
want smaller print you could use, for example, 10pt, or bigger, 14pt, and so on. The
term ‘article’ refers to the style that the paper will be prepared. Another style which
is often used is ’book’. Other styles are possible.
The preamble, or top matter, is everything from this line to the line
\begin{document}
A typical preamble for passing in assignments to this course is
\usepackage{graphicx,times}
\title{BIO-133: Assignment number xxx}
\author{Your Name\\
Your College}
The preamble tells the program what fonts and style to use, the title and author of
the article, and whether additional packages of commands need to be accessed in
order to produce the article. The preamble can be quite long since it is possible to
define new latex commands and environments. Additional packages are entered by
\usepackage. Here we added the package graphicx and times: graphicx
is the package that allow you to incorporate either *.eps or *.png into your document
(discussed in Lab 3) and times means that the font will be New York Times style.
Often journals, such as Nature, have there own packages and require you to declare
them in the usepackage statement. These *.sty files, and others, can readily be
located and downloaded from the Internet. In fact you could make up your own
*.sty file and include it here (minus the extension); however, most people don’t
do this.
Main Body: The main body of LaTeX file is everything that lies between the com-
mand
\begin{document}
and the command
\end{document}
The basic structure of the main body is
\begin{document}
\maketitle
text here
\section{your section heading here}
text here
\subsection{your sub-section heading here}
text here
\section*{your section heading here}
text here
\subsection*{your section heading here}
text here
...............
...............
\end{document}
The commands \begin{document} and \end{document} are manda-
tory. The line \maketitle is included only if you made use of the lines \title{}
and/or \author{} in the preamble (which you will need to do for your assign-
ments). The lines \section{} and \subsection{} generate, respectively, the
appropriate section and sub–section numbers, whereas the lines \section*{}
and \subsection*{} generate the same headings without the numbers.
Browser use: Anything that you would ever want to know about LaTeX can read-
ily be tracked down on the Internet. Thus you will find it useful to have a browser
open in a second window of your computer as you prepare your first LaTeX docu-
ment. Appendix B includes all of the math mode information that you will need for
this course.
Housekeeping: Perhaps the biggest issue when working with open source tools,
such as LaTeX, is dealing with the directory system of the computer. Most present
day computer users don’t have to worry about directories and paths: just point,
click, and drag! However, for computer programmers issues related to directories
and paths are very important. We need to make directories to organize our work in
a way that makes it easy for us to find things and we need paths so that a computer
program can find items that it needs to complete its calculations such as figure files,
data files, and where the outputs of the program should be written.
Paths always start from the root directory. On a PC the root directory is, by
default, the hard disk specified by C, D, E, and so on. When we are in root the
prompt on the command window will look like, for example, C:\. If you are a
user in a Linux environment, the root directory is your home directory (the system’s
administrator has access to the root directory).
Referring to Figure 1.1, the root directory can be thought of as the root of a tree
from which many branches arise which themselves have branches, and so on. Let us
Fig. 1.1 Example of path structure on a root directory. See text for discussion.
suppose that the root directory contains three sub–directories named apple, orange
and pear. If we are in root (i.e. on the screen we see C:\>), then we can type dir
or ls to obtain
<DIR> apple
<DIR> orange
<DIR> pear
The <DIR> do not appear when using Linux.
Within the directory apple we have two directories, Ant and Bee, in the direc-
tory Orange we have the directories Walk and Run, and so on. What is the path
for the directory Ant? The answer is
C:\Apple\Ant
Useful commands: These commands assume that either you are using the Com-
mand Prompt window on a PC or are working on a Linux terminal (including Mac
OSX).
cd This means change directory. If we are in the root directory and want to be in the
directory Orange we type
cd Orange
If we are in the directory
C:\Pear\Grass
and want to go to the directory
C:\Apple\Ant
we type
cd C:\Apple\Ant
If you want to move up one directory you can use the command cd ... Finally if
you want to move back to root you can use cd˜ in Linux and cd C:\ on a PC (in
the Command Prompt window).
pwd This command means pathway to directory. Another way to think of this com-
mand is that it answers the question, “Where am I?” Thus when you are in any
directory, you can obtain the path to that directory by typing pwd at the prompt.
mkdir You can create a directory with the command
mkdir name
rmdir You can remove a directory with the command

rmdir name
cp (copy) or mv (move) It is possible to move or copy files using the com-

mands mv and cp (on PCs, respectively, move and copy. The move commands
can be used to rename a file, e.g.
mv test.dat test_new.dat
What is the command to move a file named test.dat located in the directory Grass
to the directory Bee. The command depends on whether you are currently in Grass
or somewhere else. If in Grass the command is
mv test.dat C:\Apple\Bee\.
or, in Linux,
mv test.dat ˜\Apple\Bee\.
The ’.’ means that the moved file is to have the same name in the new directory. Of
course you could move and changed the file name with one command, namely,
mv test.dat ˜\Apple\Bee\new_name.dat
If we are not in the directory Grass we can still move the file by using the
command
mv ˜\Grass\test.dat ˜\Apple\Bee\.
It is important to organize our computer directories carefully so that you can
easily find things in the future. In your home directory make a directory called mtex
which will contain your LaTeX documents (see below). It is useful to make sub–
directories, each of which corresponds to one of the labs. For example, the document
prepared for today’s lab would be called Lab1 and so on. The path to this sub–
directory from the home directory is ˜/mtex/Lab1.
Questions to answer: You will be provided with 1.5-2 pages from a mathematical
biology textbook to reproduce. The LaTeXed article must be more than 1 page long
in order to complete the exercise described in Lab 3. So let us know if it isn’t. Each
of you will have a different 1-2 pages to use to prepare your document.
Deliverable: Your LaTeX document must be completed before we do Lab 3. You

will use your *.tex file to complete the exercise described in Lab 3 and then submit
a *.pdf at that time.
Chapter 2
Introducing XPPAUT (Lab 2)
In biological applications it is quite rare that the solutions of the appropriate differ-
ential equations can be obtained using paper and pencil. Thus we typically need to
use computer programs to integrate these equations and to use as tools for analyzing
their behaviors. In this course we use to types of computer programs: Python, a pro-
gramming language, and XPPAUT a programming package that is very well suited
for the analysis of many of the problems that arise in biology. Note that it is now
possible to run XPPAUT within the Python ecosystem (Appendix D); however, we
will not do this in the lab.
XPPAUT is a ‘quick and easy’ to use programming package for investigating
the differential equations [5]. An advantage of XPPAUT is that it can be used to
integrate discrete time systems, ordinary and delay differential equations, stochas-
tic differential equations and partial differential equations. In addition, XPPAUT is
particularly well suited for the investigation of the dynamics of excitable systems
composed of neurons and cardiac cells. Indeed, it is not uncommon that XPPAUT
programs can be located on the Internet for in the Supplemental materials for pub-
lished papers on journal websites1 . For example, the program we will use in Lab 6
to study the Hodgkin-Huxley equation, hh.ode, can readily be found on the Inter-
net. Can you locate this file? Hint: the program was developed by Micheal Guevara
at McGill University. Is it the same program that we have on our website!
2.1 Background
The lecture introduces the use of XPPAUT for the analysis of linear differential
equations. It is very easy to extend the use of XPPAUT to the study of nonlinear
differential equations. On the XPPAUT website check
Online Documentation
then check out
1 Typically it is necessary to access these materials through the journal website.
9
10 2 Introducing XPPAUT (Lab 2)
Writing ODE files

Reserved words & functions
Common commands
Browser use: All of the information needed to program in XPPAUT can be ob-
tained from the XPPAUT website. It is often possible to obtain additional help re-
garding the use of specific XPPAUT commands by typing, for example, in Google
xppaut heav. Finally, it is often possible to locate programs on the Internet that
can be used.
Housekeeping: Since XPPAUT file names have the form my_name.ode it is

useful to make a directory in the home directory called mode (in other words, my
ode files). Then we can make subdirectories in mode which contain XPPAUT pro-
grams used for specific purposes, for example, for Lab 2. This will be useful when
we require these programs for other purposes. An advantage of putting all of the data
files and other materials necessary to run a program in the same subdirectory, then
we don’t need to specify the directory paths in the program since, by default, the
computer program always searches for files in the directory in which it is running.
Make sure that you are in your home directory (type pwd), then we perform the
following steps
mkdir mode
cd mode
mkdir Lab2
cd Lab2
2.2 Exercise 1: First XPPAUT program
We illustrate the use of XPPAUT by using it to integrate the first–order ordinary

differential equation
dx
= −kx (2.1)
dt
Use a text editor, for example emacs, to create the file first_ode.ode. We re-
mind the reader that it is useful to create this file in the directory Lab2. All XPPAUT
files have the form
# Enter the equation (no spaces)
dx/dt=k*x
# Enter the initial conditions
init x=1
# Enter the parameter values
par k=-0.1
# Change default settings
@ TOTAL=20,dt=0.05,xlo=0,xhi=100,ylo=0,yhi=1.5,maxstor=100000
done
2.2 Exercise 1: First XPPAUT program 11
It should be noted that lines that begin with # are comments and are not acted
upon by XPPAUT. The purpose of these lines is to make the code more readable for
a user. However, you could simply write the program as
dx/dt=k*x
init x=1
par k=-0.1
@ TOTAL=20,dt=0.05,xlo=0,xhi=100,ylo=0,yhi=1.5,maxstor=100000
done
Useful hints:
1. TOTAL refers to the total time for the integration and dt is the step size
for the numerical integration. Thus if TOTAL=20 and dt=0.05, there will be
20 × 20 = 400 time steps. The parameters xlo,xhi,ylo,yhi refer, respec-
tively, to the dimensions of the x–axis and y–axis of the figure displayed by
XPPAUT and not to the variables that appear in the differential equation. The
parameter maxstor sets the total number of time steps that will be kept in
memory (default value is 5000).
2. Likely the most common mistake made using XPPAUT is to add spaces to the
entered equation to “make it look better”. Unfortunately the computer is not
impressed! Thus do not add spaces when you type in the equation. A good rule
of thumb is that when in doubt, no spaces!
3. The values of the initial conditions and parameters can readily be changed us-
ing the on–screen menus provided by XPPAUT. In other words, it really does
not matter which values you type into the *.ode program since you can easily
change them later.
4. The default integrator for an ordinary differential equation is a 4th –order Runge–
Kutta algorithm. This is a common integrator used to study the differential
equations that arise in mathematical biology. There are two ways that the inte-
grator can be changed: 1) on the XPPAUT start–up menu, click Numerics, then
Method; and 2) in the change default settings part of the ode program type, for
example, meth=euler. One of the very useful features of XPPAUT is the ease
by which it is possible to compare the effects of different numerical integrators
and their parameters on the simulation without writing new programs.
Running an XPPAUT program: The XPPAUT program is activated by typ-

ing the command
xppaut first_ode.ode &
In a Linux environment the & is added since we do not want the whole CPU to
be involved in doing this calculation. In other words, by adding the & we make it
possible to do other computations at the same time.
Now press ENTER. The computer screen will look like that shown in Figure 2.1.
Fig. 2.1 Starting XPPAUT opens this window on the computer screen. The tabs for opening the
additional windows ICs, BCs, Delay, Parem, Eqns and Data are displayed in a row on the left side
of the top of the opened window.
Six additional windows can be opened (ICs, BCs, Delay, Param, Eqns, Data) by
clicking on the appropriate icons. Since our goal is to integrate an ordinary differ-
ential equation we need to open the menus ICs and Parem. Click on these icons and
use your mouse to arrange the computer screen so that it looks like that shown in
Figure 2.2.
Fig. 2.2 This screen view was obtained by clicking on the Tabs ICs and Parem and then using the
computer mouse to re–arrange the panels.
To run the program click ‘OK’ in the ‘Initial Data’ window and ‘Parameters’
window and then ‘Go’ in the ‘Parameters’ window. The solution appears in the
main window as it is computed (see Figure 2.3).
2.3 Exercise 2: Second-order DDEs 13
Fig. 2.3 The X versus T plot was produced by first clicking on ‘OK’ on the ICs panel and then
‘Go’ on the Parem panel.
If you want to erase the solution, click on ‘Erase’. To quit the program, click on
‘File’ and then ’Quit’.
Questions to be answered:
1. In class we showed that the solution of (2.1) decreases exponentially when k < 0
and increases exponentially when k > 0. Do your simulations agree with this
expectation?
2. The mathematical solution of (2.1) is x(0) exp(kt), where x(0) denotes the initial
value (i.e., the number that appears in the IC window). Explain how can you
verify that the numerically estimated solution has this form (Hint: what is the
1/e time?)?
2.3 Exercise 2: Second-order DDEs
In order to use XPPAUT to integrate a second (and higher) order differential equa-
tion it is necessary to rewrite the differential equation as a system of two (2) first-
order differential equations, i.e.
dx
= f (x, y)
dt
dy
= g(x, y)
dt
where f and g are functions of x and y.
1. Write the second–order differential equation
Fig. 2.4 This window was obtained using a XPPAUT program that was written to integrate a
second–order linear differential equation. In preparation for re-plotting the data as a phase plane,
we have clicked on ‘Viewaxes’ and then ‘2D’ to open the new panel shown on the lower left. The
values in the panel were typed in.
d2x dx
2
+ b + cx = 0 (2.2)
dt dt
as a system of two first–order differential equations.
2. Write an XPPAUT to integrate this second–order differential equation. Call this
program second_ode.ode and put this file into the directory \mode\Lab2.
A screen shot of the program we wrote is shown in Figure 2.4.
3. What is the characteristic equation and what are its roots?
4. Determine values of the parameters b, c so that both roots are negative real num-
bers, complex numbers with negative real parts, and a pair of purely imaginary
numbers. In each case use your XPPAUT program to compute the solutions.
Are the solutions qualitatively the same or do they differ? If they differ, then
what are the differences?
2.4 Exercise 3: Phase plane representations
Up to now we have plotted x or y versus time. In these plots, time appears explicitly
(namely, it is the x–axis). However, there is another way to plot the data, namely x
versus y. This type of plot is referred to as a phase plane plot and is characterized
by the fact that time does not appear explicitly.
2.4 Exercise 3: Phase plane representations 15
Fig. 2.5 The phase plane solution in the right panel was produced by clicking on the ‘OK’ Tab on
the ‘2D View’ shown in Figure 2.4.
After running XPPAUT, click on Viewaxes, then 2D (see Figure 2.4). The table
that opens identifies the x–axis of the plot as time and the y–axis as the value of x.
The ranges are those we entered in the last line of the program second_ode.ode.
To obtain the phase plane representation, change the values so that y is plotted on
the y–axis and x is plotted on the x–axis. You will also need to adjust the ranges in
the two axis so that they span both positive and negative numbers, e.g.
xlo=-5,xhi=5,ylo=-5,yhi=5
Press ‘OK’ to obtain the phase plane representation (Figure 2.5).
1. For the values of b, c determined above, determine the phase plane for a pair
of negative real numbers, a pair of complex numbers with negative real part
and a pair of purely imaginary numbers. In our lectures we will describe the
shape of the phase plane representations as a spiral, a focus and a center. Can
you guess which pair of eigenvalues corresponds to which type of phase plane
representation?
Problem:
In lecture we discussed the conditions for steady state for the consecutive reaction
1k 2 k
A −→ B −→ C, (2.3)
1. Write down the differential equations that describe this consecutive reaction as
a system of three first–order differential equations.
2. Write a XPPAUT program to integrate these equations. Do the time courses of
A, B, and C resemble those we discussed in Chapter 3 on Mathematics as a
Laboratory Tool?
3. Show that as the ratio k2 /k1 becomes larger, the steady state condition for B
becomes more reasonable.
4. Keep track of the values of k1 , k2 you used in your simulations. In Lab 3 we will
show you how to write a Python program with matplotlib to make a figure.
At that time we will ask you to produce a figure that shows these results. Note
that you can save yourself some time by clicking on Data, then Write to save
the data file after you run each simulation.
Deliverable:
Use Lab2_template.tex to prepare the lab assignment.
Assignment hints:
• A useful LaTeX environment for writing up your lab assignment is the verba-
tim environment, namely \begin{verbatim} .... \end{verbatim}.
Whatever is placed within this environment appears in the LaTeX document as
written.
• You can save a figure in XPPAUT by clicking on Graphic stuff, then
Postscript. This makes a *.ps figure which is likely not going to be much
use to you since we can’t use such a figure in LaTeX and we don’t have a
postscript printer. A very powerful software package for dealing with figures is
called ghostscript. It is freely available and can be easily downloaded and
installed from the Internet on your own computer. Using this software package it
is possible to convert a *.ps figure to a *.pdf figure, by typing the command
ps2pdf name.ps
Chapter 3
Introducing Python: Working with data (Lab 3)
Software packages such as XPPAUT are well suited for analyzing the properties
of differential equations. However, in the laboratory we are also often interested in
analyzing the data collected experimentally. Typically this data is collected using
an instrument that is capable of exporting the data in the form of, for example, an
ASCII file. Thus an essential skill for laboratory researchers is the ability to write
our own programs to analyze collected data in whatever way we can imagine. In this
lab we introduce the use of Python for this purpose1 .
The important point concerning the use of Python for scientific programming is
the availability of packages that increase Python’s computational speed and useful-
ness (see Appendices C and D). A very useful Internet website is SciPy
http://docs.scipy.org/doc/
It is worthwhile to become familiar with the content on this website. It should be
noted that the term SciPy refers both to a package that can be imported as well as to
Scientific Python which means Python supplemented by packages. Many packages
are available (see the SciPy website) and more are being created. In this lab we
focus on two Python packages which are used in every Python program we will
write, namely, numpy, the numerics packages, and matplotlib, the graphics
package.
Question: What resources are available on the SciPy website for users interested
in neuroscience?, molecular biology?
3.1 Background
The lecture introduces Python programming skills that make it possible to import
data either generated by another computer program or collected by an experimental
device and then plot the observations using the matplotlib package. Appendices
1 Other possibilities are available such as MATLAB,OCTAVE, C++ and FORTRAN.
17
18 3 Introducing Python: Working with data (Lab 3)
C and D provide an introduction to, respectively, programming in Python and the

Python ecosystem, including an introduction to matplotlib. It is useful to glance
over these documents; however, only a small portion of this material is covered in
today’s lab. In the coming weeks we will cover most of this information.
Browser use: All of the information needed to write Python programs and use
the matplotlib package (and more) are available on the Internet. Most of the
Python commands we will use in this and the following laboratories are described
in Appendices C and D. By this point in the course the reader should realize how
useful it is to have access to the Internet while working with a computer (“everything
can be located on the Internet”). For example, information concerning how to use
any command in numpy, the numerical package for Python, plus an example of
the command’s use can readily be obtained on Google. For example, if we want to
understand how to use arange(), then type on Google
numpy arange
Not only will you get the information you wanted you will also be advised to use the
command linspace() instead! Other hits on Google will explain why it is better
to use linspace(). The SciPy website (type SciPy) in Google provides much
more information concerning on-line resources that are available (once on this web-
site look under Additional Documentation). It should be noted that SciPy
is an open source resource and thus it is not always well documented. Nonetheless
by talking to colleagues and exploring the Internet it is usually possible to find out
where things are located.
The most useful website for the matplotlib package is
http://matplotlib.org/
This is the website created by the late John D. Hunter, the creator of matplotlib.
Particularly useful are the list of plotting commands, the gallery and
the tutorial by Nicolas Rougier. The gallery shows a number of different styles
for making figures: see one that you like, click on it, and you obtain the Python code
used to produce the figure!
A great deal of confusion exists concerning whether the graphics package should
be imported into a Python program as matplotlib or pylab or some ver-
sion thereof. The package matplotlib is the whole graphics package; how-
ever, for many graphical applications the whole package is not needed. Thus it is
better to import small parts, or modules, of matplotlib, namely pylab and
matplotlib.pyplot. The module pylab is most convenient for use with
ipython where the emphasis is on interactive calculation and plotting. We will
introduce ipython in the spring laboratory sessions. matplotlib.pyplot is
most useful in a non-interactive programming setting in which the investigator is
exploring the data and obtaining clues to develop a program. By non-interactive
computing we mean that the investigator works with a program script (a program
saved in a file myname.py). This type of programming is used when the same tasks
are done over and over again (such as what the above program is done), i.e. there is
3.1 Background 19
no need for data exploration. Of course, in a course such as this, we require the use
of program scripts so that we can evaluate your progress.
Housekeeping: The names of Python programs have the form myname.py. As

we did in Lab 2 it is useful to organize our home directory. Make a directory called
pyfigs and inside this directory a sub-directory called Lab3.
As an illustration we export data generated by XPPAUT and use this data to make
a figure that we can place inside the LaTeX document we began making in Lab 1.
Run the program second_ode.ode using the values of the parameters, b and c,
that produced a spiral wave in the phase plane. Once the program has been run, click
on Data, then Write. Call the data file spiral.dat.
Where is the file spiral.dat located? It will likely be saved in the directory
that contains second_ode.ode. Move this file to /pyprogs/Lab3 by typing
mv spiral.dat /pyprogs/Lab3/.
Note that the ‘.’ means that we want to use the same file name. If you prefer to
rename the moved file you could type
mv spiral.dat /pyprogs/Lab3/new_name.dat
Now open the file spiral.dat with the text editor by typing
emacs spiral.dat &
Why did we add the ‘&’?
You should see three columns of numbers: the first column is time, the second
column is ‘x’, and the third column is ‘y’. Note that along each row the numbers
are separately by spaces. This observation means that the data file is a .tsv file,
i.e. a file of tab separated variables. Thus a better name for the data file would be
spiral.tsv. We can rename spiral.dat as spiral.tsv by typing
mv spiral.dat spiral.tsv
It is also possible that numbers in a data file are separated by commas, namely a
.csv file, where csv stands for comma separated variables. We could name such
files with the suffix .csv, i.e. myname.csv. As we will soon see, it is important to
know whether we have a .csv or a .tsv file when we write the computer program
to analyze the data.
A second question concerns how the columns are numbered. In Python the first
column is ‘0’, the second column is ‘1’, and so on. In some languages, such as
MATLAB, the first column is labeled as ‘1’; however, for most computer languages
the first column is labeled as ‘0’.
The data file spiral.tsv is an example of an array. In this case the array has
three columns and N rows where N=time/dt and dt is the integration time step.
3.2 Exercise 1: Making figures using Matplotlib
Python is a programming language such as C or Fortran. By itself Python is not

well suited for scientific research in the laboratory. However, it is possible to “put
Python on steroids” by importing packages which provide specific tools that are
useful for scientific applications. Two commonly used packages are numpy which
provides numerical tools based on arrays similar to those found in MATLAB and
matplotlib which provides graphical tools used, for example, to plot data in the
form of figures and tables. Other packages are available: a list can be found at
http://www.scipy.org/SciPy_packages
Our first goal is to develop a Python program that imports a data file (array) gen-
erated by some other program or a laboratory instrument, plot the data in the form of
a figure, and then save the figure in a form that is suitable for inclusion in a LaTeX
document. Researchers do this over and over again and thus it is important to have
such a program. This sounds like a tall task given that we are just beginning to learn
to write Python programs. A good way to start to program in Python is to consider
an illustrative example. Let’s examine the Python program data_plot.py
# Import the required packages
import numpy as np
import matplotlib.pyplot as plt
# Import the data file

X=np.loadtxt(’spiral.tsv’)
# Extract the data from the array

x0=X[:,0]
x1=X[:,1]
x2=X[:,2]
# Plot the data in the form of a figure

plt.plot(x1,x2,’k-’,linewidth=2)
plt.xlabel(’X’,fontsize=16)
plt.ylabel(’Y’,fontsize=16)
# Save the figure in an appropriate format

plt.savefig(’spiral.png’,dpi=600)
plt.savefig(’spiral.eps’,dpi=600)
# Show the figure on the compter screen

plt.show()
Comments:
1. Python gets it scientific usefulness by importing packages which contain func-
tions that are useful. These functions make it possible to integrate differential
equations, to perform statistical and time series analyses, to import photographs
and bird songs, to animate simulations, and so on. The basic numerical package
3.2 Exercise 1: Making figures using Matplotlib 21
is numpy and the graphics package is called matplotlib (or pylab). The
first lines of a Python program imports these packages, for example, using the
commands2
import numpy as np
It should be noted that other packages are available (see the SciPy website).
These would be imported if they are required for the program you are writing.
2. When beginning to program in Python it is useful to specify the package from
which function was obtained. Thus, for example if we want to use the function
plot() in matplotlib.pyplot we write
plt.plot()
and so on.
3. The numpy function which makes it possible to import a data file is
X=np.loadtxt(’myname.tsv’)
Note that this command associates the label ‘X’ with the whole data array. If
the data file to be imported is a *.cvs file, then this command becomes
X=np.loadtxt(’myname.csv’,delimiter=’,’)
It should be noted that there are only two other ways that an array in Python
can be constructed. We will discuss these as our Python programming skills
increase.
4. The next step is to extract from X the columns of data that correspond to time,
x, and y. The command that extracts a single column of data, say the 0-th col-
umn, from an array is x0=X[:,0]. This command associates x0 with the first
column in X.
This command is so useful that you should memorize it.
Note that we have labeled the extracted column with the small letter, in this
case x, which corresponds to the capital letter X used to specify the array. This
is not necessary, but is useful for helping a reader of your program (including
yourself at a later date) to better see how the program is organized, especially
when more than one data set is imported into the program.
5. The command plt.plot() is a very powerful plotting command with lots
of actions. Go to the Matplotlib website and look up how to use plot(). The
command used in the above program is the most basic form of the command and
plots the data using a black solid line (’k-’) of width 2. The PyLab website
also gives all of the commands we can use to make our figure look pretty.
2 There are other ways to import these packages; however, for learning it is best to start in this way
(see discussion).
6. It is useful to get into the habit of saving figures in two forms: myname.eps
for use in LaTeX documents and myname.png for use in pdflatex files. The
resolution is given by specifying the dpi. The default resolution for most sci-
entific journals is dpi=600.
Helpful hint: It should be noted that the above program can be used to import
any 3-column data array and plot the data as a figure. You will likely use such a
program many times over your scientific career. The good news is that we only
need to write (type) the whole program once in our lives! This is because we can
use a text editor, such as emacs, to modify the couple of lines of the program that
are needed for a specific use. Thus you might think about creating a subdirectory
in your home directory called template and then place a copy of the program
in this directory. It might be reasonable to give the file a generic name such as
plot_3column_data.py. Suppose you are working on your data in a subdi-
rectory called Mydata and decided that you wanted to plot the data. Of course you
could write a new program to do this. However, we are suggesting that you do the
following
mv ˜/template/plot_3column_data.py ˜/Mydata/myname.py
You could then open myname.py using a text editor such as emacs, make the neces-
sary corrections (change the name of the data file, change the variables to be plotted,
etc), and then run the program. Our guess is that this procedure will take much less
time than typing, and then debugging, a new program from scratch.
Differential equations using Python: The reader will no doubt wonder

why we used one programming language to integrate the differential equation and
another to prepare the figure. Is it possible to do both steps within the same program?
Fortunately the answer is YES. The following program entitled second_ode.py
shows how Python can be used to integrate a second order ordinary differential
equation and produce as output a figure suitable for publication.
import pylab as plt
import numpy as np
import scipy.integrate as integrate
a=1.0
b=2.0
t=np.linspace(0, 1, 100, endpoint=True)
#
# Define deriv() for second-order differential equation
def derivs(state,t):
x,y = state
deltax = y
deltay = a*x + b*y
return deltax,deltay
x0=1.0
y0=0.0
3.2 Exercise 1: Making figures using Matplotlib 23
z0=[x0, y0]
z=integrate.odeint(derivs, z0, t)
#
# Output is the array z[] containing two columns
# First column is ’0’ and contains x
# Second column is ‘1’ and contains y
#
x=z[:,0]
y=z[:,1]
#
# plot x versus time
# plt.plot(t,x,’k-’)
#
# plot y versus time
# plt.plot(t,y,’k-’)
#
# plot phase plane
#
plt.plot(x,y,’k-’)
#
plt.show()
Comments:
1. Functions that integrate ordinary differential equations are imported using the
statement
import scipy.integrate as integrate
2. The integrator odeint was developed by Lawrence Livermore National Lab-

oratories. It does an adaptive time-step to achieve a desired level of solution
accuracy. This software is designed to automatically switch between stiff and
non-stiff integration routines depending on the characteristics of the solution.
A list of full options can be obtained in I-python by importing the package
scipy.integrate and then typing odeint?.
3. The integration duration and time step is set using
np.linspace(start,stop,N,endpoint=True).
This function generates N points equally spaced on the closed interval [start,stop].
Note that the beginning and end points are included provided that we include
endpoint=True.
It is also possible to set the integration duration and time using
np.arange(start,stop,step).
This function is not currently favored since when using a non-integer step, the
results will often not be consistent.
4. The output of integrate.odeint() is the array z[]. The number of

columns in this array is equal to the dimension of the ordinary differential equa-
tion. Since we are integrating a first-order ordinary differential equation there is
only one column of numbers. The first column in a numpy array is 0, hence the
x are z[:,0].
5. Extension to higher order ODEs is straightforward; however, it is important
to remember to include all of the initial conditions that are required.
1. The Lotka-Volterra predator-prey equations are
dx
= k1 x − k2 xy
dt
dy
= k3 xy − k4 y
dt
Write a Python program to integrate this equation and prepare a *.png figure
that shows the phase plane portrait. It is useful to call this program lotka.py
and store it in the directory pyprogs.
2. The equations for the van der Pol oscillator are
x3

dx 1
= y− +x
dt ε 3
dy
= −εx
dt
where ε > 0 is a constant. Write a Python program to integrate this equation
and prepare a *.png figure that shows the phase plane portrait. It is useful to call
this program vdp.py and store it in the directory pyprogs.
3. The equations for the Fitzhugh-Nagumo model (FHN) spiking neuron are [9,
30] equation
dv
= f (v) − w + Iext
dt
dw
= bv − γw
dt
where v plays the role of membrane potential, w is a recovery variable, Iext is an
external applied current, f (v) is a cubic nonlinearity,
f (v) = v(a − v)(v − 1)
and the parameters are 0 < a < 1, b > 0 and γ > 0. Write a Python program
to integrate this equation and prepare a *.png figure that shows the phase plane
portrait. It is useful to call this program fhn.py and store it in the directory
pyprogs.
3.3 Exercise 2: Adding a figure to a LaTeX document 25
3.3 Exercise 2: Adding a figure to a LaTeX document
In order to introduce figures into our article we need to pay attention to the following
points:
1. We must include the user package graphicx in the preamble of the LaTeX
file.
2. The figure must be in the form of either a *.eps file, a *.png file, a *.jpg file or
a *.pdf file.
3. Both types of files can be produced by matplotlib.
4. When you use *.png or *.pdf or *.jpg (not *.jpeg), then you must use the
command pdflatex.
5. For publication the pictures must be *.eps and hence you must use latex. An
advantage of latex documents is that they are often smaller than documents
prepared using pdflatex. The disadvantage is that the document is produced
as a myname.dvi file. However, once again ghostscript comes to the
rescue: we use the command dvipdf to convert the file into a myname.pdf
file.
The figure environment in LaTeX has the form
\begin{figure}
\centering
\includegraphics[width=.5\textwidth]{filename.eps}
\caption{your text here}
\label{your label here}
\end{figure}
You will find that it is quite tedious to place figures into a LaTeX document.
The default mode for LaTeX is to put the figure at the end of the document. This
happens because either the figure is to big to put it where you hoped that it would go
or your placement conflicts in some way with items that are in the same region of
the document where you want the figure. A good first step is to latex the document
before a new figure is added and pick the place where you want the added figure to
appear: best choices are the top of the page or between paragraphs. Pick a reasonable
figure size, say 0.5, and add the figure environment statement to the appropriate
placed in your *.tex file. Now latex the file and see if the figure actually appeared
where you hoped. Then add the second figure and repeat the process. The point is
that you need to place the figure in the document sequential in the order first through
last. With a little practice you will get the hang of it!
Deliverable: Use Lab3_template.tex to prepare the lab assignment.

Chapter 4
Working with arrays and eigenvalues (Lab 4)
The purpose of this lab is to get familiar with manipulating data arrays in numpy.
Browser use: This lab introduces the use of a number of Python packages that
are particularly useful for working with images such as *.jpg pictures taken with
a cell phone. These packages include skimage and PIL (Python Imaging Li-
brary). We strongly recommend installing Canopy Express on your computer:
these packages can be readily installed using Package Manager to install
scikits-image. Unfortunately many of these packages are not well documented
and hence the only way to learn about using them is to use these packages and con-
sult the Internet: Chances are if you are having a problem, then someone else has
posted a question on the Internet about it and hopefully someone else has answered
it! In addition, Appendices C and D will be useful as references.
Housekeeping: In your home directory make two directories pyprogs and

pyimages. It is convenient to put the programs we write in this lab into the di-
rectory pyporgs since we will use them in future labs. The directory pyimages
will be used to store the programs that we will develop to manipulate images such
as *.jpg pictures.
4.1 Background
You can activate python in command mode, by typing the word python at the
command prompt, then hit ENTER. In Canopy Express the command mode window
is opened when you type Editor (it’s the lower of the two windows that open). It is
useful to do this to follow the discussion. Now type
import numpy as np
then ENTER, so that you have access to the numpy library of commands.
Here are two very important concepts in scientific computing that are often
under-emphasized:
27
28 4 Working with arrays and eigenvalues (Lab 4)
1. Not all numbers are the same: There are three basic number types: integers,
floating point number and complex numbers. An integer is a number such as 1,
2, 3, etc. Note that integers do not have a decimal point. A floating point number
is a number that has a decimal point, i.e 1.0, 2.2348016, and so on. A complex
number has the form floating point number ± j floating point number.
In Python we can specify the data type by using the command dtype. Thus it
we want our number to be short integers we could specify
dtype=np.int8
if a floating point number
dtype=np.float32
and so on.
For imaging we most often use dtype=unit8. This notation stands for 8-
byte, unsigned integers. Why are images limited to 8-bytes? There are two rea-
sons: 1). Most computer monitors can only display 8 bytes, and 2) The human
eye is only able to render about 8 bytes of visual information. For a useful list
of the possible dtypes see
http://docs.scipy.org/doc/numpy/user/basics.types.html.
If we keep in mind that computers store objects as 00 s and 10 s it is not hard
to imagine that we can specify dtypes that are not numbers. For example,
dtype=str means a string of letters like a word and dtype=bool refers to
either True or False. We can even use the command dtype=object to
refer to something that is none of the above.
2. An important data structure in numpy is the array. For an extensive dis-
cussion of arrays in Python see Appendix C and using arrays in numpy see
Appendix D. Here we briefly touch on some of the important points.
An array is a table of objects (usually numbers for scientists) having n rows
and m columns. For scientists it is convenient to think of an array as a table of
data. For example let us consider that the position of the hand during throwing
a baseball has been recorded by motion capture cameras. The output of the
camera system is typically a table of data in which the first column is time, the
second column is the position of the hand in the x=direction at time t, the third
column is the position in the y-direction at time t and the fourth column is the z
position of the hand at time t. Thus data array has n rows, where n is the nukber
of time samples, and 4 columns. The shape of this data array is (n, 4).
Suppose for simplicity that we have the array stored in the file test.dat
1 2
3 4
5 6
The shape of this array is 3 rows and 2 columns. We can load this data set into
our program using the command
4.1 Background 29
X = np.loadtxt(’test.dat’)
The command np.shape(X) tells us the shape of X and returns (3,2). We
can also use the command reshape() to change the shape of X. Thus
Y=np.reshape(X,(6,1)))
returns
1
2
3
4
5
6
3. Creating arrays: In the laboratory setting, the most common way to create
an array in a Python program is to read in data using the np.loadtxt()
command. In this case data file itself determines the way that the numbers are
represented, the number of rows and the number of columns. The question is
how to create an array of a given shape inside a computer program. In other
words for a situation in which we do not read in a data file as discussed above.
The answer to this question is important because we often use a computer simu-
lation of a model to predict what the behavior of the experimental observations.
Since at some point we will need to compare prediction to observation we need
to be able to make an array that contains the predictions of the model.
There are three ways to create an array and these are very similar in philosophy
The philosophy is the same: we create an array of the desired shape filled with
either 0’s, 1’s or random numbers and then replace these numbers by the number
generated by our computer simulations.
a. np.zeros(): In this case we create an arrays of 00 s of a required shape.
For example, the command
X=np.zeros((2,2),dtype=np.float32)
creates the array
0. 0.
0. 0.
b. The command np.ones() works in a similar way, except that instead of
10 s we get 00 s.
c. The command np.random.rand() creates an array of a given shape of
random floating point numbers, e.g.
np.random.rand(2,2)
produces
0.4823714 0.7398787
0.0082137 0.9087786
It should be noted that for np.zeros() and np.ones() we can specify the
dtype, but for np.random.rand() the dtype is always a floating point
number.
4. Extracting data from arrays: Each element in an array occupies position (i, j)
where i specifies the row and j specifies the column. Thus if we wanted to know
what is the value located in the second row, third position of array X we would
type X[2,3]. Note that an array position is enclosed by square brackets (in
contrast in MATLAB we would use round brackets, i.e. X(2,3)). Keep in
mind that the first element in the first row is X[0, 0].
In many languages including Python, the colon : is very special. By itself it
denotes all of the corresponding row or column. Thus if we wanted to extract
the first column from array X we would type
x0=X[:,0]
and if we wanted to first row we would type
x0=X[0,:]
A colon : separating two numbers a : b means rows (columns) a to b. Thus
[1:5] means from 1 to 5
Here are some other useful array manipulations:
[1:] means 1 to end
[len(a):1] means from length a to end
[::n] means each n-th item in entire sequence
Of course we can also combine these notations, for example, if X was an array
with shape (10, 10), then
X[1:5,7:10]
would mean the first five rows and the last four columns.
5. Saving computer generated arrays as data files:
Numpy lets you write arrays into files in two ways. In order to use these tools
well, it is critical to understand the difference between a text and a binary file
containing numerical data. In a text file, the number π (referred to as pi in
numpy) could be written as “3.141592653589793”, for example: a string of
digits that a human can read, with in this case 15 decimal digits. In contrast,
that same number written to a binary file would be encoded as 8 characters
(bytes) that are not readable by a human but which contain the exact same data
that the variable pi had in the computer’s memory.
The tradeoffs between the two modes are thus:

4.2 Exercise 1: Stability using XPPAUT 31
• Text mode: occupies more space, precision can be lost (if not all digits are
written to disk), but is readable and editable by hand with a text editor. Can
only be used for one- and two-dimensional arrays.
• Binary mode: compact and exact representation of the data in memory, can’t
be read or edited by hand. Arrays of any size and dimensionality can be saved
and read without loss of information.
Here we discuss how to read and write arrays in text mode. Appendix D dis-
cusses how to read and write arrays in binary mode.
The np.savetxt function saves an array to a text file, with options to control
the precision, separators and even adding a header. Suppose we have an array
X containing 10 numbers from 0 to 9 with shape (2,5) and we want to write it
to a file called test.out. The command is
np.savetxt(’test.out’, X, fmt=’%1.3f’, header="My dataset")
and X is
# My dataset
0.00 1.00 2.00 3.00 4.00
5.00 6.00 7.00 8.00 9.00
This is because we used the format option, fmt=’%1.2f’ to specify a floating
point number (hence the ‘f’), where the .2 indicates two decimal points. If we
wanted the numbers to be in exponential format (like in MATLAB), we would
choose fmt=’%.2e’ to obtain
# My dataset
0.00e+00 1.00e+00 2.00e+00 3.00e+00 4.00e+00
5.00e+00 6.00e+00 7.00e+00 8.00e+00 9.00e+00
We can read the data using np.loadtxt, namely

X=np.loadtxt(’test.out’)
If we write the command print X, we obtain
0. 1. 2. 3. 4.
5. 6. 7. 8. 9.
Why does the file look this way? There are two reasons. First, the default data
type for np.loadtxt() is float. Second, comment lines started with # are
ignored. If we want to change these defaults, we can look up the options for
np.loadtxt() on the Internet.
4.2 Exercise 1: Stability using XPPAUT
In lecture we emphasized three concepts for determining the qualitative behavior of

the differential equations we use to model biological phenomena: 1) fixed points,
2) the characteristic equation, and 3) local stability. Briefly, the fixed points are the
values of the variables for which all derivatives are equal to zero and ‘local stability’
refers to the response of the dynamical system to a small perturbation away from
the fixed point. Mathematically, the stability of the fixed point is obtained by first
linearizing the differential equations using Taylor’s theorem and then using the usual
Ansatz that x(t) ∼ eλt , to obtain a characteristic equation in terms of the eigenvalue,
λ . The roots of the characteristic equation corresponds to the eigenvalues that we use
to evaluate stability: the fixed point is stable if Re(λ ) < 0 and unstable if Re(λ ) > 0.
Performing all of these steps becomes increasingly difficult as the number of
variables included in the model increases. Since realistic biological models often
contain many variables, it quickly happens that we are not able to evaluate stability
analytically. Thus recourse to computer programming tools becomes necessary.
One possibility is to use a program such as XPPAUT. XPPAUT automatically
determines the eigenvalues from the phase plane representation. To see how this is
done consider the equation for a damped harmonic oscillator we studied in Lab 2.
Construct the phase plane representation and then click on ‘Sing pts’ (Figure 4.1). A
new window opens which shows the fixed point (bottom), gives the stability (upper
right) and classifies the eigenvalues.
Fig. 4.1 We obtained the classification of the fixed point and the eigenvalues by clicking on the
tab ’Sing pts’ after we had changed the axes to make a phase plane portrait.. Note that that the
classification of the eigenvalues and the location of the fixed point are shown in the window that
opens, but the actual values of the eigenvalues are printed out in the Terminal window.
The XPPAUT convention for classifying the eigenvalues (shown in panel at upper
left in figure) is
4.3 Exercise 2: Stability using Python 33
c− : number of complex eigenvalues with negative real part

c+ : number of complex eigenvalues with positive real part
r− : number of real negative eigenvalues
r− : number of real positive eigenvalues
Im : number of pure imaginary eigenvalues
The values of the eigenvalues are printed in the window that you used to start XP-
PAUT. Compare these values to the ones that you determined mathematically.
Exercise to do:
Use XPPAUT to run one of the *.ode programs you wrote last lab. Once you
have a solution, click on Sing pts. Are the fixed points and the eigenvalues values
the same as you calculated using paper and pencil?
4.3 Exercise 2: Stability using Python
When we integrate a differential equation using numpy it is necessary to re-write it

as a system of first-order differential equations, for example,
dx
= ax + by , (4.1)
dt
dy
= cx + dy ,
dt
where the a, b, c, d are constants. In class we determined the eigenvalues by first
reconstructing the differential equation from (4.1)
d2x dx
2
− (a + d) + (ad − bc)x = 0 ,
dt dt
and then obtaining the λ ’s (eignevalues) from the characteristic equation
λ 2 − (a + d)λ + (ad − bc) = 0 . (4.2)
We adopted this procedure because a course in linear algebra is not a pre-requisite

for this course. However, this procedure rapidly becomes tedious as the number, n,
of first-order differential equations increases, i.e.
dx1
= c11 x1 + c12 x2 + c13 x3 + · · · + c1,n ,
dt
dx2
= c21 x1 + c22 x2 + c23 x3 + · · · + c2,n , (4.3)
dt
. = ··· ,
dxn
= cn1 x1 + cn2 x2 + cn3 x3 + · · · + cn,n .
dt
In fact we know that when n > 4 it is unlikely we will be able to determine the λ ’s
analytically. Thus we must resort to numerical procedures.
The package np.linalg makes it easy to determine the eigenvalues for a sys-
tem of linear differential equations of arbitrary size. In order to understand this we
need to learn a little bit of linear algebra. First we re-write (4.3) as
    
dx1 /dt c11 c12 . . . c1n x1
dx2 /dt  c21 c22 . . . c2n   x2 
 . . .  = . . . . . . . . . . . . . . . . . (4.4)
    
dxn /dt cn1 cn2 . . . cnn xn
Equation (4.4) contains two n × 1 arrays, one in which the elements are dxi /dt and
one in which the elements are xi , and one n × n array containing the coefficients.
The n × 1 arrays corresponds to vectors and the n × n array containing n rows and n
columns corresponds to a matrix. Using this notation we can re-write (4.1) as

dx/dt ab x
= . (4.5)
dy/dt cd y
When we multiply a vector by a matrix we multiply a row of the matrix by the

vector element by element. Thus we multiple the first element of the first row of the
matrix by the first element of the vector, then the second element of the first row of
the matrix by the second element of the vector, and so on. In this way way you can
see that (4.5) is an alternate way of writing (4.1).
We can also write (4.4) as
dx
≡ y = Ax , (4.6)
dt
where the bold-face type is used to indicate matrices (capital letter) and vectors
(small letters). Equation (4.6) describes a linear transformation that maps a given
array x into a new array y. This statement will become much clearer when we do
the Glass-Moiré dot exercise (below). The important point for our discussion is that
the λ ’s (eigenvalues) can be determined from the matrix.
Important point: Although it is useful to think of vectors and matrices as kinds
of arrays (this in fact is what the programming language MATLAB does), in Python
vectors and arrays are not simply kinds of arrays, especially when it comes to mul-
tiplication. In other words in Python an array is not the same as a matrix.
The package numpy itself contains the package numpy.linalg which makes
available a number of useful functions for working with vectors and matrices. Our
4.4 Exercise 3: Moiré-Glass dot patterns 35
interest in this package stems from the fact that there is a function, eig(), which
outputs the eigenvalues (also the eigenvectors). The important point is that eig()
is not restricted to 2 × 2-matrices, but can be used to determine the eigenvalues
for n × n matrices where n ≥ 2. In order to use these functions it is necessary to
remember two things: 1) the subscripts of the coefficients, ci j indicate, respectively,
the row number (i) and column ( j) number, and 2) a matrix is constructed row by
row as
np.matrix([[x11, x12, .., xn], .., [xn1, xn2, .., xnn]])
Our discussion up to this point concerns only linear differential equations. For
nonlinear differential equations it will be necessary to linearize the equations about
a fixed point before we can use numpy.linalg.eig() to determine the eigen-
values. It is posssible to write a computer program that does all of these steps (in
fact XPPAUT incorporates this). However, this is an advanced topic. Note that in our
typical applications, the number of rows equals the number of columns. The output
of numpy.linalg.eig() is two nd.array’s: vals gives the λ ’s and vects
gives the eigenvectors. Here our interest here is on the eigenvalues.
Example: Write a computer program to determine the eigenvalues, λ ’s, for
dx
=y
dt
dy
= 3x − 2y
dt
Answer: A Python program that determines the eigenvalues for a system of linear differen-
tial equations is
import numpy as np
import numpy.linalg as linalg
X = np.matrix([[0.,1.],[3.,-2.]])
vals, vects = linalg.eig(X)
print vals
The result is λ = 1, −3. Question: Does this result agree with what you obtained using
paper and pencil?
4.4 Exercise 3: Moiré-Glass dot patterns
In the two-dimensional xy-plane, the values of x and y specify the coordinates of a

point. A common problem concerns the determination of the coordinates of a point
when the xy-plane is rotated by an angle θ . From introductory physics we know that
the answer to this problem is solved using a rotation matrix, R, [27]

x x
=R , (4.7)
y new y old
Fig. 4.2 A Moiré-Glass Pattern that resembles the flow patterns near a saddle point. The two colors
distinguish the position of the random rots before and after the 2D-plane was rotated by θ .
where
cos(θ ) − sin(θ )
R=
sin(θ ) cos(θ )
is the rotation matrix.
It has been observed that when a random dot pattern is scaled, rotated, and
superimposed over the original data, interesting visual patterns, known as Moiré-
Glass Patterns, emerge [1, 14]. Moiré-Glass dot patterns are actively studied in psy-
chophysics to understand how the nervous system perceives visual (sensory) stim-
uli. A 2-D random dot pattern can be generated using np.rand() which gener-
ates random numbers drawn a uniform probability distribution. Thus we can apply
transformations to the random dot field, X1 , using a scale, S, and rotation, R, namely
X2 = SRX1 . If the scale and rotation factors are small, the transformation is analo-
gous to a single step in the numerical solution of a second order ordinary differential
equation, namely, the discrete equation

x(t + 1) x(t)
= SR , (4.8)
y(t + 1 y(t)
where we have written out X1 and X2 in terms of their (x, y) components. The plot
of both X1 and X2 will reveal the structure of the vector field flow around the fixed
point. The fixed point corresponds to the invariant point under the transformation.
The eigenvalues of the transformation matrix M = SR determine the nature of the
fixed point.
The program moire_glass.py generates a Moiré-Glass dot pattern that re-
sembles the vector flows in the vicinity of a saddle point (Figure 4.2). Although we
have not discussed how to multiple two matrices, we can see what M is by adding a
4.5 Exercise 4: Working with digital images 37
print M command line to the program to obtain the result. Once you have M you
should be able to determine the eigenvalues by determining the roots of the equation
λ 2 − (a + d)λ + ad − bc = 0
An important point is that in contrast to continuous differential equations, the

stability for discrete time dynamical systems is determined by the modulus of the
eigenvalue, namely q
|λ | = Re(λ )2 + Im(λ )2 < 1
for stability and q
|λ | = Re(λ )2 + Im(λ )2 > 1
for instability. For the choices of the parameters in the program moire_glass.py
the eignevalues are 0.95, 1.05 and hence
|1.05| = 1.05 > 1
and
|0.95| = 0.95 < 1
This means we have one unstable eigenvalue and one stable eigenvalue. In other
words we have a saddle point. Determine sx and sy to see the Moiré dot pattern for
a node. However, in order to obtain a center and a spiral point it will be necessary
to add rotation (not too much!).
1. Rewrite (4.8) as a system of two equations.
2. Using this information, determine values of sx , sy , θ in the program to obtain
the vector field flow for a center and a spiral point. Save these figures as a *.png
file.
3. What happens as you increase θ ? (for example, does the pattern become clearer
or less clear?)
4.5 Exercise 4: Working with digital images
The images produced by a digital camera are most often in *.jpg format. The data
in a *.jpg file are presented in the form of an array and thus we can apply the same
tricks that we applied to the Moiré dot patterns to digital photographs. Indeed, an
infamous felon recently altered his own picture in this way to brag about a crime he
had performed, and displayed the picture on YouTube. This felon did not realize that
an image which has been changed by coordinate transformations, can be changed
back to the original. Thus the crook had unwittingly published his confession to the
crime for all to see!
Fig. 4.3 A *.jpg picture on the Mona Lisa (left) that is swirled (middle) and then reverse swirled
(right).
Skimage (a.k.a. scikit-image) is a collection of algorithms useful for im-

age processing and computer vision. Here we use some of the tools in Skimage to
create a swirled image similar to that that the felon used. Image swirling is a non-
linear image deformation that creates a whirlpool effect (Figure 4.3). This image
was produced using the program swirl_prog.py. This Python program repre-
sents a jump in programming skills from the programs we have considered up to
now and thus we do not expect you to be able to digest all of the issues at this point
in time. Here we discuss the basic concepts involved in swirling an image.
When applying a geometric transformation on an image, we typically make use
of a reverse mapping, i.e., for each pixel in the output image, we compute its cor-
responding position in the input. The reason is that, if we were to do it the other
way around (map each input pixel to its new output position), some pixels in the
output may be left empty. On the other hand, each output coordinate has exactly one
corresponding location in (or outside) the input image, and even if that position is
non-integer, we may use interpolation to compute the corresponding image value.
To perform a reverse mapping, namely a geometric warp, in skimage, you need
to provide the reverse mapping to the skimage.transform.warp function. For
example, consider the case where we would like to shift an image 50 pixels to the
left. The reverse mapping for such a shift would be
def shift_left(xy):
xy[:, 0] += 50
return xy
The corresponding call to warp is:
from skimage.transform import warp
4.5 Exercise 4: Working with digital images 39
warp(image, shift_left)
In order to understand the swirl transformation, consider the coordinate (x, y) in
the output image. The reverse mapping for the swirl transformation first computes,
relative to a center, (x0 , y0 ) using polar coordinates
θ = arctan(y/x)
q
ρ = (x − x0 )2 + (y − y0 )2
and then transforms them according to
r = ln(2) radius/5
φ = rotation
s = strength
θ 0 = φ + s e−ρ/r+θ
where “strength” is a parameter for the amount of swirl, “radius“ indicates the swirl
extent in pixels, and “rotation” adds a rotation angle. The transformation of “radius”
into r is to ensure that the transformation decays to ≈ 1/1000th within the specified
radius.
The program swirl_prog.py, takes a *.jpg picture, mona_lisa.jpg and
applies the swirl algorithm.
1. Take a picture with your cell phone and see if you can swirl part of the image
using swirl_prog.py. In its present form swirl_prog.py swirls a re-
gion centered on the center of the picture. See if you can change this program
so that it swirls some other region of the image. Save these as a *.png file.

Chapter 5
The ’cubic’ nonlinearity: Excitable systems (Lab
5)
Arguably one of the most important milestones in the evolution of cellular behaviors
was the development of excitability [28]. In terms of our black box analogy, an ex-
citable system is one which generates one type of output when the input is below a
threshold, and a qualitatively different input when the input exceeds the threshold1 .
Many scientists will think of a spiking neuron as THE excitable cell. When the
spiking neuron’s membrane potential is less than the spiking threshold the neuron
does not generate an action potential: action potentials are produced only when the
membrane potential exceeds the threshold. However, excitability is also a property
of all muscle cells (cardiac, skeletal, smooth), certain endocrine cells, such as the
pancreatic β -cell, the excitable green algae including Nitella, and even sea sponges.
From a mathematical point of view, excitability in excitable cells is most often asso-
ciated with the presence of a cubic nonlinearity in the relevant system of differential
equations.
5.1 Background
The simplest dynamical system which possesses a cubic nonlinearity is
dx
= x − x3 . (5.1)
dt
In class we showed that
1. There are 3 fixed-points: x = 0, x = ±1.
2. There is a double well dynamic potential, φ (x), described by
x2 x4
φ (x) = − + .
2 4
1 A more precise definition is developed in this laboratory exercise.
41
42 5 The ’cubic’ nonlinearity: Excitable systems (Lab 5)
The XPPAUT program, cubic.ode integrates (5.1). What happens when we

choose the initial value of x, x(0) < 0? x(0) > 0? These simulations show that bista-
bility is possible in a dynamical system which contains a cubic nonlinearity? This
laboratory and the one which follows demonstrate that many other behaviors are
also possible in differential equations that possess a cubic nonlinearity.
Browser use: XPPAUT is widely regarded as the outstanding computer program-

ming package for the investigation of excitable systems. This program is well docu-
mented on the XPPAUT website. In addition it is often possible to obtain XPPAUT
programs for various excitable system from the Internet, including the XPPAUT
website. Many will be tempted to use XPPAUT to complete this laboratory; how-
ever, some of the problems are actually easier to do using a Python program. This
observation emphasizes that before trying to solve a problem that one should con-
sider which computer programming tool is best suited for the program at hand. The
design and dynamical properties of the electronic circuit implementation of the van
der Pol oscillator is described by Ned J. Corron and a *.pdf can be obtained from
www.ccreweb.org/documents/physics/chaos/vdp2006.html
There is an error in Corron’s circuit diagram and this is corrected in Figure 5.1
(see ∗).
Housekeeping: Depending on whether you use Python or XPPAUT to do the

following exercises, make appropriate sub-directories in, respectively, pyprogs or
mode to put the programs and figures.
5.2 Exercise 1: The van der Pol oscillator
One of the very first examples to demonstrate that a stable periodic solution could
arise in a homogeneous system of ordinary differential equations (i.e. equations in
which ‘t’ does not appear explicitly) was the van der Pol oscillator [36, 37]. Here
the term ‘stable’ means that there is a ‘steady state’ oscillation that is independent
of the choice of initial conditions. This independence of the amplitude and period of
the oscillation on the choice of initial conditions is in sharp contrast to the neutrally
stable oscillations (centers) that we examined in the context of population cycles
(Lotka-Volterra equations). The van der Pol equation was first derived to describe
an electronic circuit containing a triode resistor whose resistance depends on the
applied current. However, it quickly became apparent that this equation would de-
scribe the dynamics of many other phenomena, including the heartbeat.
The van der Pol equations are
x3

1
ẋ = y− +x , (5.2)
k 3
ẏ = −kx .
5.3 Exercise 2: Analog implementation of the van der Pol oscillator 43
1. What are the critical points for (5.2)?

2. Classify the critical points with respect to type and local stability?
3. What is the behavior when x or y become very large?
4. Use the program vdp.py that you prepared in Lab 4 to answer the following
questions2 .
• Examine the plot of x versus time for different values of k, say k = 0.1,
k = 0.5 and k = 1.0. What seems to be happening to the waveforms as k
gets larger? The solutions when k is small are suggestiuve of a relaxation
oscillator. What aspect of the solutions justifies this terminology?
• Plot the phase plane (Lissajous diagram) for the van der Pol equation for the
above three values of k (do one value of k at a time). This diagram is con-
structed by plotting y versus x. Once you have the phase plane representation
superimpose the nullclines. The nullclines are the curves along which ẋ = 0
and ẏ = 0. Does the relationship between the phase plane trajectory and the
nullclines change as k changes?
• In what parts of the phase plane does the trajectory seem to be moving the
fastest, along the nullclines or as it jumps between them. Hint: You can es-
timate the speed at which the trajectory is moving by plotting data as points
rather than as a line (in other words in plt.plot() use the print option
’ko’ rather than ’k-’ ): the larger the gap between the data points the
greater the speed. You can increase dt, say from 0.05 to 0.1, to make it eas-
ier to identify the regions of faster and slower speeds. Keep in mind that if
you increase dt too much it is possible that the numerical algorithm used to
integrate (5.2) itself becomes unstable.
• What does the intersection between two nullclines correspond to?
• See question in Deliverables.
5.3 Exercise 2: Analog implementation of the van der Pol

oscillator
It is easy to get accustomed to the idea that solving differential equations is some-
thing that only can be done by a computer. However, before computers became read-
ily available, differential equations were sometimes solved using an appropriately
constructed electronic circuit to obtain an analog (continuous time) solution. One
advantage of this approach is that potential errors due to digitization are eliminated.
Another use of an analog circuit implementation is to establish the convergence of a
digital algorithm to the true solution in a novel situation, such as a dynamical system
under the influence of piecewise continuous feedback and noise (see, for example,
[11]). Finally, it can be easier to study the effects of various inputs and perturbations
on a dynamical system using an electrical circuit.
2Alternatively you could write a program in XPPAUT, vdp.ode, to do these exercises; however,
you will not be able to do all of them with the XPPAUT program.
Fig. 5.1 The van der Pol electronic circuit.
A variety of electronic circuits that implement the van der Pol oscillator can be
found using the Internet. The circuit diagram shown in Figure 5.1 was constructed
by Walter J. Cook, Harvey Mudd College. It is modified by the addition of the
connection labeled ∗ from a circuit designed by Ned J. Corron. The presence of this
connection is necessary for the circuit to produce an oscillation.
The circuit in Figure 5.1 integrates the modified van der Pol equation
dx
= y, (5.3)
dt
dy
= ε(α − x2 )y − x ,
dt
where α is an adjustable parameter. The components in this circuit have been chosen
so that ε = 0.1.
1. Vary a from ∼ 0 to 13.2. Use the oscilloscope to make a Lissajous plot, namely,
a plot of Va (proportional to x) versus Vb (proportional to y). Does the shape of
the Lissajous plot change as a changes? Are their corresponding changes in the
shape of the waveform of the oscillations as seen in plot of V versus time?
2. Write a computer program to integrate (5.3). Do changes similar to those ob-
served for the electronic circuit occur as the parameter α is varied?
5.4 Exercise 3: The Fitzhugh-Nagumo (FHN) neuron 45
(a) (b)
(c) (d)
Fig. 5.2 Analog simulator of van der Pol equation. a) A bipolar 15V power is required. b) The
value of α can be adjusted by turning the nob. c) The value of Va can be measured and α calculated
as 10Va . d). The outputs, Vx OUT and Vy OUT , can be used to construct the Lissajous diagram
(phase plane) using an oscilloscope. Remember to put the oscilloscope into ‘XY’ mode.
5.4 Exercise 3: The Fitzhugh-Nagumo (FHN) neuron
The Fitzhugh-Nagumo equation plays an important role in the analysis of excitable

systems. However, what do we mean by the term “excitable system”? What are the
important properties of this equation? The Fitzhugh-Nagumo equation is
dv
= f (v) − w + Iext , (5.4)
dt
dw
= bv − cw ,
dt
where 0 < a < 1, b > 0, and c > 0 and
f (v) = v(a − v)(v − 1) .
In the mathematical biology literature the Lissajous diagram is called the phase
plane. It is possible to use the phase plane to obtain a qualitative understanding of
the behavior of a dynamical system. Phase plane analysis is a very convenient tool
for the study of dynamical systems described by 2-D ODE. As for the van der Pol
oscillator, a key point in phase plane analysis is the construction of the nullclines.
The nullclines are the trajectories along which the derivative of one of the variables
is zero. Here we are concerned with 2D-ODEs. Hence there will always be two
nullclines. The v-nullcline is obtained by setting dv/dt = 0. When Iext = 0 we have
w = f (v) = v(a − v)(v − 1)
which we see is a cubic. Similarly we obtain the w-nullcline by setting dx/wt = 0

to obtain
b
w= v
c
which is the equation for a straight line.
Fig. 5.3 The phase plane constructed in XPPAUT for (5.4). The red line is the v-nullcline and the
green line is the w-nullcline. The parameter values are a = 0.25, b = 0.4 and c = 0.1. See text for
discussion.
XPPAUT contains a number of tools which facilitate phase plane analysis. Write
a XPPAUT program, fhn.ode, to integrate (5.4). Choose the initial conditions to
be v = 0, w = 0 and run the program. Figure 5.3 shows what happens when you 1)
use Viewaxes to plot w versus v (note the changes in the x-axis and y-axis), and
then 2) click Nullclines.
Now click on Initialconds and then Mouse (see Figure 5.4). By making
these selections you can use your mouse to pick the initial condition you want with
reference to the nullclines. Remember that in the phase plane the initial condition
is a point in the (v, w)-plane. Unfortunately you need to clink on Initialconds
and then Mouse each time you wish to use a different initial condition. Figure 5.4
shows the trajectories for (5.4) for two different choices of the initial conditions.
Take Iext = 0.
1. Use phase plane analysis to show that there will be either a single critical point
or three critical points.
5.4 Exercise 3: The Fitzhugh-Nagumo (FHN) neuron 47
Fig. 5.4 Once we have the phase plane displayed in XPPAUT we can pick the initial conditions by
using the mouse as described in the Discussion. These trajectories associated with the two initial
conditions illustrate that the FHN neuron is in its excitable regime. Explain.
2. Take a = 0.25 and Ia = 0. Determine b and γ so that there is one stable critical
point. Verify your prediction and show two solutions that verify your prediction.
3. Pick an initial condition for which w < 0 but greater than the v-nullcline. Use
phase plane analysis to guess what will happen and verify this with a simulation.
Now pick an initial condition for which w > 0 but is less than the v-nullcline.
Now what will happen. The difference between these two behaviors is what
defines an excitable system.
4. The two nullclines divide the phase plane into four regions each characterized
by the sign of dv/dt and dw/dt. Identify these four regions in the phase plane.
Using these results explain why the two choices in initial conditions in Fig-
ure 5.4 lead to two different behaviors in the trajectories. This is one way to
explain the phenomena of excitability.
5. Take a = 0.25 and Ia = 0. Determine b and γ so that there are three critical points
of which two are stable and one is unstable. Use the nullclines to identify which
critical points are stable and which are not. Remember that the 1/e time for v is
much shorter than that for w. Hence local stability is determined primarily by
the slope of the v-nullcline evaluated at the fixed-point. Verify your predictions
and show solutions that verify your predictions.
Take Iext > 0.

1. Use phase plane analysis to show that there will be either a single critical point
that is excitable (there are two possibilities), a limit cycle, or bistability.
2. Verify these predictions using numerical simulations.
3. See question in Deliverables.
5.5 Exercise 4: Reduced neuron models
Neurons can generate a wide variety of spiking patterns (Figure 5.5). In fact, neu-
roscientists often use the spiking pattern to identify the type of neuron that they
are working with. Eugene M. Izhikevich, a mathematical neuroscientist, realized
that the different firing patterns exhibited by neurons depended very much on the
properties of the v and w nullclines [23]. As we saw for the FHN neuron, the v-
nullcline (fast variable) has a cubic, or N-shape, and the w-nullcline, in general, a
sigmoid shape (the FHN neuron is the special case for which the w-nullcline is a
straight line.). From this perspective the different spiking patterns of neurons arises
from details related to where the intersection occurs between the nullclines and their
shape [21, 22]. A similar idea was developed independently by Hugh Wilson [38].
Fig. 5.5 Examples of the different spiking behaviors observed for cortical neurons. Figure was
been reproduced from izhikevich.org with permission.
The Izhikevich equations take the form

5.5 Exercise 4: Reduced neuron models 49
dv
= 0.04v2 + 5v + 140 − w + Iext , (5.5)
dt
dw
= a(bv − w) ,
dt
with the auxiliary after-spike reset condition3

v ←c,
if v ≥ 30mV, then
w ← w+d .
The reset condition means that when v ≥ 30mV, immediately set v to c and w to
w + d.
Fig. 5.6 Four neuronal firing types generated by the reduced neuronal model: regular spiking (RS),
fast spiking (FS), intrinsic bursting (IB), and chattering (CH). A single voltage trace for each
neuron type with a constant current input is shown in the lower four panels; the choices of the four
parameters, a, b, c, d for each spiking type are shown in the insets.
The Izhikevich model has the general form of an integrate and fire neuron model:
when the membrane potential exceeds a threshold it generates an action potential.
The occurrence of a spike immediately changes the spiking threshold (e.g. the neu-
ron become transiently refractory) and the membrane potential (it hyper-polarizes,
namely the membrane potential becomes slightly more negative than the neuron’s
3 The notation a ← b was introduced by Izhikevich to indicate that a is immediately replaced by b.

resting potential). Within the integrate-and-fire neuron framework, these changes

are described by the after-spiking reset conditions.
Background: The program four_neuron.py, written by the late John D. Hunter,

shows how the reset conditions are handled in a Python program. An example of
the output of this program showing four different spiking patterns is shown in Fig-
ure 5.6. For a very interesting application of the Izhikevich model neuron see
http://www.sciencebytes.org/2011/05/03/blueprint-for-the-brain/
We suspect that your programming skills in Python have not yet reached the stage
that you can comfortably understand the details of four_neuron.py. Don’t be
concerned! XPPAUT has a function, global, that makes handling the after spike
reset condition for (5.5) very easy. This function has the format
global sign condition {name1=form1;...}
The function global defines a global flag allowing XPPAUT to implement delta
functions. If sign=1, then the condition changes from negative to positive. If
sign=-1, the condition changes from positive to negative. If sign=0, the con-
dition vanishes identically. When the global flag is activated, each of the variables
in the {...} is changed according to the values of the formulas. For example the
statement
global 1 x-1 {x=0;y=y+a*sin(y)}
means that when the sign of x − 1 changes from negative to positive, immediately
then set x ← 0 and y ← y + a ∗ sin(y).
Example: The simplest integrate-and-fire model takes the form
dv
= k ∗ v + vrest + I (5.6)
dt
with the auxiliary after-spike resetting
if v ≥ vthres, then v ← 0 .
The XPPAUT program, IF.ode, integrates an integrate-and-fire neuron.
1. Write a XPPAUT program to integrate the Izhikevich neuron. Note that by con-
vention the parameters are labeled a, b, c, d with the meaning as given above.
Since investigators who use this model, for example in large scale neuron net-
work simulations, refer to these parameters in this way it is important to use the
same parameter definitions in your model.
2. Try the following parameter choices
regular_spiking = 0.02, 0.2, -65, 8
chattering = 0.02, 0.2, -50, 2
5.5 Exercise 4: Reduced neuron models 51
fast_spiking = 0.1, 0.2, -65, 2

intrinsically_bursting = 0.02, 0.2, -55, 4
thalamocortical = 0.02, 0.25, -65, 0.05
Do these parameter choices reproduce some of the spiking patterns shown in
Figures 5.5 and 5.6?

Chapter 6
Bifurcation diagram: stopping spiking neurons
with a single pulse (Lab 6)
The qualitative behaviors of a dynamical system can change when parameters are
changed. For example, a stable fixed-point can become unstable, or vice versa.
Moreover, for some dynamical systems it is possible that other types of dynami-
cal behaviors can arise when a fixed-point become unstable, such as a limit cycle
oscillation. If the mathematical equations are known then in principle it is possible
to summarize the parameter dependence of the dynamics in the form of a bifurca-
tion diagram. A bifurcation diagram provides a road map that can be used in the
laboratory to directly link the predictions of a mathematical model to experimental
observation. Are the predicted dynamics observed experimentally for the predicted
parameter values? Do the dynamics change appropriately as experimentally accessi-
ble parameters are changed? Even if a mathematical model has not been developed,
a demonstrated qualitative change in dynamics as a parameter is changed can spark
the interest of bio-mathematicians to work on the problem.
6.1 Background
In practice it is a challenging task to produce a bifurcation diagram both in mathe-

matical models and in the laboratory. A fundamental problem is that the number of
parameters in biological systems can be very large. At present it is not known how
to systematically explore the dynamics of a mathematical model using numerical
methods when the number of parameters is large. A very important computational
package for generating bifurcation diagrams in mathematical models in which the
number of parameters is small is called AUTO
http://indy.cs.concordia.ca/auto/
One of the reasons why bio-mathematicians use XPPAUT is that it affords an easy
access to AUTO [5]. AUTO uses a mathematical technique referred to as continua-
tion to follow a particular solution (e.g. a fixed-point or a limit cycle) as a parameter
changes. For each choice of the parameters the stability of a particular solution
53
54 6 Bifurcation diagram: stopping spiking neurons with a single pulse (Lab 6)
branch is determined using a linear stability analysis. As we will see, AUTO pro-
vides a number of tools to characterize the bifurcation of fixed-points and limit
cycles. AUTO can be tricky to use and can fail in a dramatic fashion for certain
problems. However, even though AUTO is over 25 years old it still remains a major
tool in the arsenal of an applied mathematician.
The experimental determination of a bifurcation diagram can be even more prob-
lematic since for a biological system the identify of all of the parameters is not
generally known and it may only be possible to alter some parameters, and even
then, the range of possible variation may be small. However, as we have seen in
lectures, sweeping a parameter, an experimentally attractive idea, can mis-identify
the location of the point where changes in stability occur. Moreover, as the edges of
stability are approached, the identification of the nature of the dynamics becomes
difficult due to slowing down phenomena. Nonetheless a number of experimental
paradigms have been developed in which it has been possible to at least partially
characterize the bifurcation diagram.
In this laboratory we illustrate the use of AUTO to characterize the onset of peri-
odic spiking in a neuron. This type of bifurcation is characterized by the replacement
of a stable fixed-point with a stable limit cycle and is referred to as a Hopf bifur-
cation. A surprising prediction of the mathematical analysis is that it is possible to
annihilate periodic spiking in a neuron by the application of a single, carefully-timed
pulse. We demonstrate this phenomenon using a numerical experiment. This phe-
nomenon was first demonstrated experimentally by R. Guttman and her colleagues
using the squid giant axon [17].
Browser use: A user’s guide for running AUTO under XPPAUT can readily be
located on the Internet
http://www.math.pitt.edu/˜bard/xpp/help/xppauto.html
In this laboratory we illustrate AUTO using only very simple examples. Readers
who wish to extend the use of AUTO to their own research are recommended to read
the chapter in Ermentrout’s guide to XPPAUT [5]. XPPAUT can be run in Python
(https://github.com/jsnowacki/xppy); however, we recommend first
learning to run AUTO in XPPAUT before taking it on in Python. The ode ver-
sion of the Hodgkin-Huxley (HH) equation, hh.ode, is the one written by Michael
Guevara (McGill University). You can either download this program from our web-
site or from the Internet by typing hh.ode Guevara McGill in a web browser.
The exercises involving the HH equation described here are modified from the com-
puter exercises developed by Michael Guevara to explore the dynamical properties
of the HH and Fitzhugh-Nagumo equations [16]. Students interested in the dynam-
ical behaviors of neurons and excitable systems are recommended to work through
the complete set of these exercises.
Housekeeping: It is useful to place all of the files related to the HH equation and
its analysis in a directory called, for example, HHeqn.
6.2 Exercise 1: Making one-parameter bifurcation diagrams 55
6.2 Exercise 1: Making one-parameter bifurcation diagrams
We illustrate the procedure for obtaining a bifurcation diagram for the saddle node
bifurcation discussed in class. The generic equation for this bifurcation is
dx
= µ + x2 , (6.1)
dt
where µ is the bifurcation parameter.
The steps to determine the bifurcation diagram for (6.1) are as follows:
1. Write an .ode program to integrate (6.1).
2. It is very important to ensure that the solution has settled onto either a fixed-
point or a limit cycle before using AUTO to continue the solution as a param-
eter changes. In the case of (6.1) we know that a fixed-point solution (x∗ < 0)
exists for µ < 0. Pick the initial condition x = −1 and run XPPAUT. The most
common cause of failure using AUTO is to use this program when the solu-
tion is not already on a branch. With this in mind we extend the integration
by clicking on Initialconds Last. This XPPAUT command extends the
integration by an amount defined by TOTAL, from the step at which the pre-
vious integration finished. Depending on the differential equation, the choice
of initial condition, and the parameter values it may be necessary to click on
Initialconds Last several times.
Fig. 6.1 Clicking on File and then AUTO opens up the AUTO window.
3. The AUTO window (Figure 6.1) is activated by clicking on Auto which

is located under File. The most important functions for our purposes are
Parameter, Axes, Numerics, Run and Grab. A complete description of
all of the commands in AUTO is provided in the user’s guide cited above. The
circle in the lower left of Figure 6.1 summarizes the number of stable and unsta-
ble eigenvalues for a given choice of the bifurcation parameter(s): the number
of crosses inside the circle corresponds to the number of stable eigenvalues (i.e.
eigenvalues with negative real part) and the number outside the circle corre-
sponds to the number of unstable eigenvalues (the number of eigenvalues with
positive real part). How many crosses do you see for (6.1) and are they stable
or unstable? How many crosses did you expect to see and why?
4. Click on Parameters. It is possible to declare up to 5 bifurcation parameters
in this box. By default, XPPAUT chooses the first five parameters in the *.ode
program. In our case there should be just one bifurcation parameter declared. Is
this correct?
5. Click on Axes and choose HiLo. Most of this box is already be filled out
correctly. We see that the variable X will be plotted on the Y-axis and we
are able to choose the range to vary the bifurcation parameter. Note that
Xmin , Xmax , Ymin , Ymax refer to the dimensions of the bifurcation diagram and
not the variables and thus Xmin , Xmax refers to the range over which µ is varied.
We suggest that you choose Xmin = −3 and Xmax = 1. The values Ymin ,Ymax re-
fer to the maximum and minimum values of a periodic orbit. Since (6.1) does
not possess a periodic orbit these values are irrelevant for our purposes (see
Exercise 2). Once you have completed the menu press OK.
6. Click on Numerics. For our purposes we can accept all of the default values
except those for Par Min and Par Max. Choose these values so that µ is
varied from Par Min = -3 to Par Max =1, then push OK.
Fig. 6.2 A partial bifurcation diagram computed for (6.1). We have clicked on Grab and thus we
see three special points indicated by crosses. Note that there are actually four crosses but one is not
shown. See text for discussion.
6.2 Exercise 1: Making one-parameter bifurcation diagrams 57
7. Click on Run Steady state. A thick black line will appear that extends
over the range that we varied µ (Figure 6.2. The fact that the line is thick means
that the fixed-point is stable (a thin black line indicates an unstable fixed-point).
If you do not see a line then the most likely causes is that you were not suf-
ficiently close to the stable fixed-point solution when you started. Go back to
Step 2.
8. Click on Grab. You will see a cross appear on the thick black line. You can
use the right and left arrow keys to move back and forth along this line. As you
move along this line in this way what happens to the eigenvalues (see circle in
lower left hand corner). Also if you look in the window below the bifurcation
diagram you can see more information pertaining to the current location of the
movable cross in the diagram, such as the value of µ and, in the case of a limit
circle, its period. Note that there are also crosses with numbers attached which
do not move as you use the arrow keys. The points are special points since it is
possible to continue the solution from them. Using the Tab key you can move
between the special points. When you are on one of these special points, the
bottom window provides information the point type. In our case you will see
EP which means end point. If you see MX then this means you screwed up and
it is back to step 2 that you go.
Fig. 6.3 The complete bifurcation diagram for (6.1) determined using AUTO.
9. In order to complete the bifurcation diagram it is necessary to continue to the

solution from one of the numbered special points. Use the Tab to position the
cross on the special point labeled 1. Press Enter. Click on Numerics and
change the sign of Ds. Note that when we change the sign of Ds we reverse
the starting direction for AUTO. Now click on Run and you should see the
bifurcation diagram that we calculated in class (Figure 6.3).
1. Use XPPAUT to determine the bifurcation diagram for the transcritical bifurca-
tion whose generic equation is
dx
= µx − x2 .
dt
2. Use XPPAUT to determine the bifurcation diagram for the supercritical bifur-
cation whose generic equation is
dx
= µx − x3 .
dt
3. Use XPPAUT to determine the bifurcation diagram for the subcritical bifurca-
tion whose generic equation is
dx
= µx + x3 .
dt
6.3 Exercise 2: Bifurcation diagram: Hodgkin-Huxley neuron
The determination of the bifurcation diagram for the HH equation by bio-mathematicians

made it possible for neuroscientists to test the prediction of this equation in the lab-
oratory using the squid giant axon (for a review see [16]). In this exercise we will
AUTO to characterize the onset of regular spiking of the squid giant axon when
a sufficient current is injected. The main difference in the use of AUTO for this
exercise is that we need to account for the appearance of a limit cycle.
Here are the steps we used to make this bifurcation diagram.
1. Download the program hh.ode from the class website. Note that this equation
has more parameters that can fit in the shown parameter box shown in XPPAUT.
You can maneuver up and down the parameter list by using the ∨, ∨∨, ∧, ∧∧
buttons located on the parameter box.
2. Run XPPAUT to integrate this equation when the parameter curbias=0. Use
the Initialconds Last button several times to ensure that the HH equa-
tion has settled onto the initial conditions of v, m, h, n (we obtained, respectively,
−59.9996379, 0.052955, 0.5959941, 0.317732).
3. After starting AUTO set the following parameters:
• In the Parameter window set the first parameter to be curbias (the
menu will show blockna which is the first parameter declared in hh.ode.
• In the Axes window click on Hi-Lo. The y-axis should be V and the x-axis
(called MainParm) should be curbias. Set Xmin = 0 and Xmax = 200 so
6.3 Exercise 2: Bifurcation diagram: Hodgkin-Huxley neuron 59
that curbias ranges from 0 to 200. Set Ymin = −80 and Ymax = 20 so that
the amplitude of the limit cycle oscillation ranges from −80 to 20.
• In the Numerics menu set Par min = 0 and Par max = 200 (this
sets the range over which the bifurcation parameter curbias will be varied.
Set Nmax=500 to limit the number of points that will be computed along a
given branch of the bifurcation diagram to 500. In addition set Npr=500
and Norm Max =150.
• Click on Run and then Steady state. You will see a thick line with 4
special points.
• Click on Grab and use the Tab to move between them. From the bottom
screen we see that special points 1 and 4 are end points (EP) and special
points 2 and 3 are labeled as Hopf bifurcation points (HB). Note that the line
between special points 2 and 3 is thin meaning that the fixed-point that exists
for this range of curbias values is unstable. Confirm this by looking at the
circle diagrams in the lower left hand corner of the AUTO screen.
• We can complete the bifurcation diagram by either starting at special point
2 and increasing curbias, or by starting at special point 3 and decreas-
ing curbias. Let’s start at special point 2. Use the Tab key to place the
cross at this point and then press Enter. Note that the Run menu changes.
Click on Periodic and the bifurcation diagram unfolds before your eyes
(Figure 6.4)!
• XPPAUT (and mathematicians) typically use lines to indicate the stability of
fixed-points (remember that thick lines correspond to stable fixed-points and
thin lines to unstable ones). Limit cycles are indicate by circles placed which
are positioned to indicate the maximum and minimum values of the oscil-
lation: filled circles correspond to stable limit cycle oscillations and open
circles to unstable limit cycle oscillations.
A bifurcation in which a stable fixed-point is replaced by a stable limit cycle is
called a Hopf bifurcation. Specifically the bifurcation that occurs at special point 2 is
a subcritical Hopf bifurcation and that which occurs at special point 3 is called a su-
percritical Hopf bifurcation (more specifically if we consider curbias increasing
we have a reverse supercritical Hopf bifurcation). We will discuss these bifurca-
tions in more detail later in the lectures. However, we can interpret the bifurcation
diagram to anticipate the properties of these bifurcations.
1. First consider the subcritical Hopf bifurcation that occurs at special point 2.
• What happens to the stability of the fixed-point at the bifurcation point? Is
the fixed-point destroyed at this Hopf bifurcation?
Fig. 6.4 Bifurcation diagram for an HH-neuron determined using AUTO. See text for discussion.
• Use Zoom to examine the bifurcation diagram close to special point 2. We

see that it is possible for a stable fixed-point and a stable cycle to co-exist.
This bistability is a characteristic of a subcritical Hopf bifurcation? What
does this mean (see also Exercise 3)?
• Suppose we start with a value of curbias much lower that special point
2 and slowly increase it. How do you expect the amplitude of the oscilla-
tion to grow: suddenly or gradually? The abrupt onset of a “high” amplitude
oscillation is a characteristic of a subcritical Hopf bifurcation.
• Now choose a value of curbias on the limit cycle and slowly decrease it.
At some point the oscillation will disappear and we will be left with a stable
fixed-point. Is the decreasing value of curbias at which the limit cycle
disappears the same value of increasing curbias which caused the limit
cycle? This observation illustrates the phenomenon of hysteresis.
2. Now consider the supercritical Hopf bifurcation associated with special point 3.
• What happens to the stability of the fixed-point at the bifurcation point? Is
the fixed-point destroyed at this Hopf bifurcation?
• How does the amplitude of the oscillation grow as the value of curbias is
decreased from special point 3: gradually or abruptly?
• Is this bifurcation associated with bistability?
• Is this bifurcation associated with hysteresis?
6.4 Exercise 3: Stopping spiking neurons with a single pulse 61
Fig. 6.5 Changes in membrane potential for a periodically spiking HH-neuron.
6.4 Exercise 3: Stopping spiking neurons with a single pulse
The special point 3 occurs when curbias equals ∼ 150µA/cm2 . This is a very
high current input for a neuron and likely is not biologically relevant. Thus biolog-
ical interest centers on the behaviors associated by the subcritical Hopf bifurcation
(special point 2).
Fig. 6.6 Stopping a periodically spiking HH-neuron with a single pulse. What stimulation param-
eters do you think we used? Can you accomplish the same thing with a smaller, briefer and better
time pulse? See text for discusison.
The practical implication of the presence of a subcritical Hopf bifurcation is be-

cause of its association with bistability. In particular, a non-spiking state of the neu-
ron co-exists with a periodically spiking state. Suppose we choose curbias close,
but smaller, than special point 2. It should be possible to switch between the two sta-
ble points by changing the initial conditions. How do we change initial conditions?
A simple way is to use a carefully timed electrical pulse.
The program hh.ode contains parameters that make it possible to introduce
an electrical pulse of width (duration) and height (amplitude) at a specified
time (tstart). Figure 6.5 shows the changes in membrane potential for a period-
ically spiking neuron. Guess at which point in the spike-to-spike interval where a
hyper-polarizing pulse will stop the neuron from periodically spiking (Figure 6.6).
At which point would a depolarizing pulse do the trick? Once the neuron is quies-
cent what kind of pulse is needed to start it spiking again.
To test your predictions set curbias = 13. Set the pulse height to be 9 and
the width to be 1.5. Determine a time at which a hyper-polarizing pulse stops the
spiking (Figure 6.6). Can you accomplish this task using an even briefer and and
smaller perturbation: sometimes an important goal is to determine the timing for
which the smallest and briefest pulse accomplishes the above task. Now try the
same exercise using a depolarizing pulse accomplishes the same tasks.
DELIVERABLES:
Chapter 7
Lab 7: Filters and convolution
One of the most remarkable properties of a linear dynamical system is its response
to sinusoidal inputs. As we learned in lecture, a sinusoidal input to a linear dynam-
ical system produces a sinusoidal output having the same frequency as the input.
However, the output sinusoid can differ in two ways from the input sinusoid: 1)
its amplitude, and 2) its phase. Measurements of the phase and amplitude of the
output to a sinusoidal input can be used to identify the differential equation which
best captures the input-output relationships of the dynamical system. The graphical
technique for estimating the transfer function from these experimental observations
is the Bode plot. Once the transfer function has been determined the response of the
dynamical system to any realizable input can be predicted. The purpose of this lab-
oratory is to use computer experiments to verify these observations and to introduce
the concepts of a filter and convolution.
7.1 Background
The lecture briefly reviews the concept of an impulse function, filtering and convo-
lution. A practical, but important, detail concerns the representation of a sine wave
wave in a computer program. We can write a simple program that plots a 10 second
interval of a sine wave for a given frequency, f , and a given time step.
import numpy as np
f=0.1
t=np.linspace(start,stop,N,endpoint=True)
x=np.sin(2*np.pi*f*t)
plt.plot(t,x,’k-’)
plt.show()
The period, T , is 1f , so when f = 0.1, we have T = 10. If we choose start

and stop to be, respectively, 0 and 10, then this would draw one cycle of a 0.1Hz
63
64 7 Lab 7: Filters and convolution
provided that N is chosen appropriately. The positive integer, N, is the number of

equally spaced time samples between, in this case, 0 and 10. It is important to
remember that by choosing the endpoint=True option for np.linspace()
both the starting and the end point are included, N points altogether. The time step,
dt is
stop-start
dt =
N
What is a good choice of the number of time steps, N? If we choose N = 10, the
dt = 1 and we would obtain 10 identical xalues of x since we are sampled once per
period. This is not a good choice of N since we cannot uniquely represent a func-
tion using a single value. Next we could try N = 20. In principle we can uniquely
represent a sine wave if we sample at twice the frequency (this is the concept of the
Nyquist frequency that we will discuss in the next lab). However, x would not look
like a nice sine wave. So we must pick N larger: the larger N, the smaller st, and
hence the more frequently the sine wave is sampled. However, it is also true that
the larger N the more computer memory it takes to store in this case x and t, and
the greater the number of calculations that the computer must make. Thus there is
a practical trade–off between the accuracy by which functions are to be represented
and the time it takes the computer to complete its calculations. The bottom line is
that as we change f we need to adjust ts. A reasonable choice might be to sample
the sine wave 10 − 100 times per period. Ideally we would like to keep the number
of sample points per period the same as we change f .
Browser use: Have available the document intro_python.pdf on your

browser for review, in particular, the sections that deal with how to declare a func-
tion. As always, information concerning the use of Python and matplotlib func-
tions can be obtained from the Internet using a browser.
Housekeeping: Labs 8 and 9 use some of the programs and results that we
develop in this lab. Thus it is useful to put today’s material in a sub-directory of
˜\pyprogs called, for example, convol, namely ˜\pyprogs\convol. This
will make it easier for you to find these results when we need them.
Exercise 1: Bode plots: low–pass filter

1. Write a computer program to integrate
dx
= k1 x + sin 2π f t , (7.1)
dt
where k1 is a constant.
2. Take k1 = −1.
3. Vary frequency over a large range (say f = 0.01 to f = 100) and measure the
amplitude. Plot the amplitude versus frequency.
4. Vary the frequency and measure the shift between the sine wave and the solution
of the equation. For example, we can plot the input and output sine wave n the
same figure. Plot the phase shift versus the frequency.
7.1 Background 65
5. Now change a to, let’s say, k1 = 10 and then k1 = 0.1. What happens?
6. Construct the Bode plot by choosing a suitable range of f . For each choice of f
enter the amplitude and phase shift of the steady–state solution and write these
values in a *.tsv file. Use this *.tsv file to construct the phase part of the
Bode plot.
7. You have just constructed your first low pass filter. Answer the following:
• The frequency on the input sine wave and the output sine wave is the same.
True of false?
• The output amplitude changes as a function of frequency. True or false?
• The output sine wave is phase shifted compared to the input sine wave. True
or false?
• Why is this type of filter called a low pass filter?
8. What is the transfer function for (7.1)?
9. Show that the impulse function, I(t,t 0 ), is
I(t,t 0 ) = k1 exp(k1t) .
(Remember that the impulse function corresponds to the case when the input
is a delta function.) Is this the same as the answer we derived in class (Section
7.3)?
10. The transfer function, C(s), for the thermometer is of the form
1
C(s) = .
1 + αs
Sketch the Bode plot. What is the corner frequency?
11. Consider the transfer function
10
C(s) = .
s(1 + 0.5s)(1 + 0.1s)
Sketch the Bode plot for the amplitude showing the contribution of each term.
Exercise 2: The convolution integral:

The concept of convolution is one of the most important concepts in science:
every time one uses a device to make a measurement, one essentially performs
a convolution. All physical observations are limited by the resolving power of in-
struments, and for this reason alone convolution is ubiquitous. This principle does
not only apply to laboratory instruments: our senses, e.g. vision, touch, smell, au-
dition, proprioception, are also convoluted by the sensory receptors which translate
physical stimuli into neural spikes trains. This convolution is mathematically as the
operation Z ∞
y(t) = x(t)h(t − u)du , (7.2)
−∞
where x(t) describes the input to the dynamical system and h(t − u) describes the
impulse function. In practice it is very easy to perform a convolution numerically. In
Python this is accomplished using the single command np.convolve() (similar
commands exist, for example, in MATLAB). In order to use the convolution integral
to solve (7.1) we need to know that the impulse function.
Convolution: Graphical interpretation

Although the mechanics of evaluating the convolution integral seem straight-
forward, it is quite difficult to understand in a simple way the process of evaluating
the convolution integral. The purpose of this exercise is to demonstrate a graphical
interpretation of convolution [3]. This graphical method is very useful and, for ex-
ample, we can use it to understand issues related to zero-padding and to appreciate
the difference between convolution and correlation (Lab 14).
Fig. 7.1 Graphical method for convolving a delta–function input (vertical dashed line in ’Multi-
plication panel’) with an alpha–type impulse response (solid line). See text for discussion.
The graphical convolution method divides the process of convolution into four
steps [3]:
1. Folding: Reflect the impulse response, I(t), about the ordinate axis to obtain
I(−t).
2. Displacement Shift I(−t) by an amount ∆t to obtain I(−t + ∆t).
3. Multiplication: Multiply I(−t + ∆t) by the input, b(t).
4. Integration: Determine the area under the curve produced by I(−t + ∆t) and
b(t).
Figure 7.1 illustrates this procedure in the evaluation of the alpha function re-
sponse, I(t), of a neuron to a single delta-function input, b(t). The step ‘Folding’
7.1 Background 67
means that we reflect the impulse response about the y-axis. The step ‘Displace-
ment’ means that we slowly shift the reflected impulse in the positive direction along
the x-axis. The step ‘Multiplication’ means that we multiple the reflected impulse
with the input. It is easiest to understand this step iteratively: for each time step do
the multiplication (and then Step 4), then displace the reflected impulse response an-
other time step to the right and repeat the process. In general this corresponds to the
shaded region under the two curves for which both functions are non-zero. Finally
the step ‘Integration’ calculates the area under the the two curves. In this example
the area is easy to determine since by the shifting property of the delta-function
(Section 6.3), the area is just the value of the alpha function at that point. The result
of convolution is the area computed in Step 4 as a function of time, namely as the
reflected impulse response is shifted iteratively to the right.
1. Use the graphical technique discussed in Section 8.3 to compute the convolution
for the following choices of b(t) and I(t)
• b(t) = constant and I(t) = e−t
• b(t) is a single square wave pulse and I(t) = e−t (This is animated on the
Wikipedia site for Convolution.)
• Both b(t) and I(t) are square wave pulses (This is animated on the Wikipedia
site for Convolution.)
Exercise 3: Computing the convolution

Numpy has a function called np.convolve(). Although it is tempting to think
that it would be easy to calculate the convolution of any function; there are some
subtleties.
The function np.convolve(r,s,mode=’full’) calculates the convolu-
tion integral, denoted (r ∗ s), as
N/2
(r ∗ s) j = ∑ s j−k rk
k=−N/2+1
where s is the discretely sampled signal and r is the discretely sampled impulse
function [32]. Unfortunately this approximation assumes that the length of r is the
same as that of s and that s is periodic. Thus we need to figure out how to get around
these constraints. The solution involves padding the data and impulse response with
zeros. This procedure is called zero padding.
The first constraint is easily dealt with since it is typically the case that the length
of impuse response, M, is much shorter than the length of the signal, N. Thus we
need to pad the impulse function with zero is make M = N. The second problem
is more subtle. Not only must the signal be considered to be periodic, but also the
impulse function. Therefore it happens that a portion at each end of the original
signal is erroneously wrapped around by the convolution with the response function.
In other words, the first output channel (s ∗ r)0 is contaminated wuth some wrapped-
around data from the far end of the data stream sN−1 , sn−2 , · · · . This we need to set
up a buffer zone of zero-padded values at the end of s in order to make the effects
of this contamination equal to zero. How many zeros do we need top add? We need
to pad the signal function on one end with the number of zeros equal to the length
of the response function.
The numpy function, np.pad(), is very useful for handling issues related to
zero padding
np.pad(array,pad_width,mode=None).
A useful feature of np,pad() is that we can pad to the left and/or to the right of
the array. For example, if
x=[1,2,3]
then the command
X=np.pad(x,(0,3),’constant’)
yields
X=[1,2,3,0,0,0]
and the command
X=np.pad(x,(3,0),’constant’)
yields the array
X=[0,0,0,1,2,3]
However, we are still not done. In terms of our fold-displacement-multiplication-
integration analogy for convolution, convolve() does not include the integration
step. Thus we need to multiple the output of np.convolve() by the time step.
1. Write a computer program that uses np.convolve() to construct the Bode
plot for (7.1). Again for each choice of F, measure the amplitude and phase
shift using the steady state solution.
Exercise 4: Neural inputs: The program convolve_neural_spikes_alpha.py

uses the convolution integral to estimate the sub-threshold membrane potential when
the impulse function is given by the alpha function
0
±km (t − t 0 )e−km (t−t ) if t ≥ t 0 ,
2
0
Iα (t,t ) = (7.3)
0 t < t0 .
The input spike train is modeled as a sequence of delta functions. Figure 7.2 shows
how the membrane potential changes for a given periodic excitatory input.
Hints:
7.1 Background 69
Fig. 7.2 The membrane potential of a neuron (bottom panel) in response to a pulsatile, excitatory
input whose timing corresponds to the timing of spikes in the spike train shown in the top panel.
The changes in the neuron’s membrane potential following the arrival of each EPSP is described
by (7.3) with km = 0.5.
1. A common use of def is to define and introduce functions that are used for cal-
culations in the program. The command r=impulse_response(t) evalu-
ates the function for all of the values of t. Note that the commands in the def
are indented by usually 4 spaces
2. Useful array manipulations:
[1:5] means from 1 to 5
[1:] means 1 to end
[len(a):1] means from length of a to end
[::n] means each n-th item in entire sequence
3. The function plt.subplot() has the form
\verb+plt.subplot(row,column,panel number)
Which panel would be drawn using the command plt.subplot(223)?
4. The package matplotlib.ticker includes a number of useful functions
that make it possible to, for example, change the spacing of the number on the
axis, etc. Labeling the subplots ax, bx, .. is necessary in order to use the
functions within matplotlib.ticker. In the case of a figure that has only
one panel, we can use the command plt.subplot(111).
Question to be answered:
An impulse function can be used to model the changes in blood concentration of
a drug ingested orally. For example, if we take (7.3) as a simple impulse function
for drug ingestion, then the sharp rising portion indicates the time course of the drug
going from the stomach to the blood (more typically minutes), and the falling phase
describes the clearance of drug from the body by the actions of the liver and kidneys
(typically hours to days). Consider that a doctor prescribes a drug for which the e−1
time is ∼ 6 hours and asks that the patient take the medication 4 times a day. Most
patients do not take the medication every 6 hours, but more often at 8am–noon–
5pm–midnight. Write a computer program to look at the effects of different dosing
schedules on blood levels to answer the following questions.
• Would it be better for the patient to take the medication three times a day, i.e.
every 8 hours?
• If the e−1 time is ∼ 4 hours, what would be the difference between the two dosing
schedules?
• If the e−1 time is ∼ 12 hours, what would be the difference between the two
dosing schedules?
Low-pass filters with two-poles

Here we discuss the transfer function for a second-order linear ordinary differen-
tial equation. As we discussed in lecture it is traditional and convenient to write the
differential equation in the form
1 d 2 x 2ε dx
+ + x = b(t) , (7.4)
ωn2 dt 2 ωn dt
where ωn = 2π fn is the natural frequency of the associated harmonic oscillator

(Note that ωn = 2π fn ). The damping ratio is ε:
ε > 1 : Overdamped ,
ε = 1 : Critically damped ,
1 > ε > 0 : Underdamped ,
ε = 0 : Undamped .
For the homogeneous ordinary differential equation
d2x dx
+ 2εωn + ωn2 x = 0 , (7.5)
dt 2 dt
the two eigenvalues are
p
(2εωn )2 − 4ωn2
2εωn ±
λ1,2 = ,
p2
= −εω2 ± ωn ε 2 − 1 .
When we determine the impulse function we need to remember that the λ1,2
are either be real negative numbers (the over-damped case) or a pair of complex
numbers with negative real parts (the under-damped case). This is because we can
7.1 Background 71
only define the impulse function for a dynamical system which in the absence of an
input has a stable fixed–point. For the over–damped case we have
ωn h i
I(t,t 0 ) = √ eλ1 t − eλ2 t . (7.6)
2 ε2 − 1
For the under-damped case we have
σ 2 + ω 2 −σt
I(t,t 0 ) = e sin(ωt) , (7.7)
ω
where σ is the absolute value of the real part of the complex eigenvalue and ω is the
imaginary part.
Consider the second–order linear differential equation
d2x dx
+ k1 + k2 x = sin 2π f t , (7.8)
dt 2 dt
where k1 , k2 are constants to be determined.
1. Choose the values of k1 , k2 such that both eigenvalues are real and negative.
In particular, pick the parameters so that one eigenvalues, λ1 , is equal to −1.
Construct the Bode plot when λ2 is smaller than λ1 and another when λ2 is
larger. Compare these Bode plots to the one we obtained for (7.1). You should be
able to sketch these Bode plots without using computer simulations (Of course
you are welcome to use a computer program to confirm the validity of your
sketches.) What are the differences?
2. Pick values of k1 , k2 such that the fixed–point of the homogeneous equation (i.e.,
the right–hand side is 0) is a stable spiral point.
• For these choice of parameters, what is the natural frequency?
• Write a computer program to show that as k1 approaches zero, (7.8) exhibits
resonance as f is closed to the natural frequency, fn . Your figure should show
the amplified solution (x versus t).
• Sketch the Bode plot for this case.
Deliverables:
Assignment should be submitted as a *.pdf prepared using LaTeX. Answer the
questions. In those cases in which the figures are sketched, attach the sketches to the
LaTeX document.
Chapter 8
Lab 8: Fourier series: Gibbs phenomenon and
filtering (Chapter 8)
8.1 Background
In class we used the Fourier theorem to construct a Fourier series representation of

a periodic square wave. If the periodic square wave is written as an odd function,
then the Fourier series is
1 2 2 2
g(t) = + sint + sin 3t + sin 5t + · · · .
2 π 3π 5π
Since odd numbers can be written as 2n − 1, where n is an integer, we can re-write
this series as
∞
1 2
g(t) = + ∑ sin(2n − 1)t . (8.1)
2 n=1 (2n − 1)π
There are two important properties of periodic square waves for bench top re-
searchers. The first is Gibb’s phenomenon. Figure 8.1 shows the Gibbs phenomenon
that arises when we increase the number of terms in (8.1). Sharp transitions, such
as the edge of the square wave, generate high frequency components in the Fourier
series. Gibbs’s phenomena produces artifacts in jpg images (see Section 8.4) and
also produces aliasing (see Lab 9) that makes the determination of power spectral
descriptions of neural spike trains [12] and the EKG of the heart beat difficult.
The second important property concerns the relationship between square and sine
waves. At first sight this does not seem to be that important. However, in electronics,
communication signals are often transmitted as square wave pulses and on arrival
to their destination are converted to sine waves. Moreover in an electronic circuit
it is often easier to create a train of periodic square waves rather than to generate
a sine wave. An example in biology arose in the measurement of the Bode plot for
the hyperosmolar response of yeast cells (Sections 7.4.2 and 7.6.2). From (8.1) we
see that a square wave can be thought of as a sine wave to which terms with higher
frequencies have been added. Thus an attractive idea to generate a sine wave is to
low-pass filter the train of periodic square waves to remove the higher frequency
terms, leaving the pure sine wave component behind.
73
74 8 Lab 8: Fourier series: Gibbs phenomenon and filtering (Chapter 8)
Fig. 8.1 Comparison of a square wave to the (8.1) using 50 terms. The little spikes at the corners
of the square wave (↓) represent the Gibbs phenomenon.
The purpose of this lab is to demonstrate Gibb’s phenomenon and then to use the
convolution integral to filter the square wave to make a sine wave. It is important to
keep in kind that this problem corresponds to the mathematical problem
Z ∞
output(t) = impulse response(u) input(t − u) du ,
−∞
where the input is the periodic square wave and the impulse response is that for a
RC circuit, namely,
I(t,t 0 ) = ke−kt .
It is observed that at least a two-pole low pass filter is required.
In Lab 9 we will show that we can also accomplish these goals using the fast
fourier transform (FFT). In this case we have
B( f ) = I( f ) f ilter( f ) .
Browser use: We use the function scipy.signal.square() to generate a

periodic train of square waves. The article written by R. Mark Stitt,
www.ti.com/lit/sbfapp3/sbfa003.pdf
provides background into the importance of converting square waves into sine waves
in industry. A number of animations which illustrate the act of convolution are avail-
able on the Internet. In particular, we recommend the Wikipedia-Convolution entry
since it includes animations of the two convolutions studied in this Lab. Informa-
tion concerning defining a function in Python and constructing a while loop can
either be obtained on the Internet or by consulting intro_python.pdf on our
website.
8.3 Exercise 3: Convolution: Graphical interpretation 75
House keeping: It is useful to place the programs related to convolution into a

directory labeled, for example, convol. This is because we will use these programs
in later labs and hence it is advantageous to be able to locate them quickly.
8.2 Exercise 1: Gibbs phenomenon
It is often necessary to evaluate a sum in a computer program, such as occurs in (??).

In Python sums can be evaluated using a while loop. For example to determine
y where
10
y = ∑ i2
i=1
we can write
sum=0
n=1
while n <11:
sum=sum+nˆ2
n=n+1
print sum
It is possible to use a for loop in Python; however, a while loop construct is
typically much faster.
1. Write a Python program using a while loop to compare g(t) to a periodic
square wave as a function of the number of terms n. Compare the solution to a
periodic 2π-periodic square wave.
2. Rewrite your program by constructing a function that evaluates g(t) for an in-
putted number of terms.
3. Modify this program by defining a function so that you can easily plot g(t)
for different numbers of n on the same graph. This plot is useful to see the
emergence of the square waves as the number of terms in the sum increases.
4. What happens to the Gibbs phenomenon when n becomes large?
8.3 Exercise 3: Convolution: Graphical interpretation
Note: This section has been moved to Lab 7.

76 8 Lab 8: Fourier series: Gibbs phenomenon and filtering (Chapter 8)
8.4 Exercise 4: Making a sine wave from a square wave
A sine wave can be created by low-pass filtering a square wave. It is necessary to use
at least a two pole low-pass filter to make wave that looks sinusoidal. A RC circuit
corresponds to a one pole low-pass filter. Thus it should not be surprising that one
way a sine wave is created from a square wave is to pass the square wave signal
through two consecutive RC-type circuits. Let’s use the convolution programs we
developed in Lab 7 to see how well this works.
We give you two useful hints:
1. A useful way to use scipy.signal.square() to generate a periodic train
of square waves is to write
t = np.linspace(0, 1, 100, endpoint=False)
plt.plot(t, signal.square(2 * np.pi * 3 * t))
To illustrate, these lines of code have been written to generate a 3 Hz periodic
train of square waves sampled at 100 Hz for 1 s.
2. The cut-off frequency for a RC circuit whose impulse response is kekt is 1/k.
Fig. 8.2 a) An input signal consisting of a 5 Hz periodic train of square waves. b) the output signal
after passing through a one pole low-pass filter, and c) the output signal after passing through
two consecutive one pole low-pass filters. Note that the signals shown correspond to steady state
solutions, i.e., a large number of transient have been allowed to die out. It should be noted that
using a one pole low-pass filter that it is possible to get other output waveforms. We have used
parameters that generate the “most sinusoidal shaped one”.
Questions to be answered
8.4 Exercise 4: Making a sine wave from a square wave 77
1. Write a computer program that uses the convolution program in Lab 7 to gen-
erate a sine wave from a square wave. remember to include the modification to
the convolution program that we discovered in Exercise 3. The basic idea is to
convolve the periodic square wave train with a RC filter and then convolve the
output with a second RC circuit. Figure 8.2 shows our attempt at this strategy.
Clearly it is clearly better to use two RC circuits, namely a two-pole low-pass
filter, in order to get an output that resembles a sine wave. In our case we choose
k for each RC circuit to be the same. Can you do better by choosing different
k’s for each low-pass filter?
2. According to the principle of linear superposition one would expect that the
result of passing a signal through a two-pole filter would be the same as passing
a signal through two consecutive one-pole filters having the same poles. To test
this hypothesis consider the impulse function for a second-order over damped
differential equation (Equation 6 in Lab 7). Pick two poles, λ1 and λ2 and write
a computer program to convolve the square wave used in the previous question
with this impulse response. Compare this result to that obtained by passing the
square wave signal through two consecutive RC circuits, the first with pole λ1
and the second with pole λ2 . Are they the same?
Deliverables: Use Lab8_template.tex to prepare the lab assignment.

Chapter 9
Lab 9: FTT and power spectra
The Fast Fourier Transform (FFT) is a fast and efficient numerical algorithm that
computes the Fourier transform. The power spectrum is a plot of the power, or
variance, of a time series as a function of the frequency1 . If G( f ) is the Fourier
transform, then the power spectrum, W ( f ), can be computed as
W ( f ) = |G( f )| = G( f )G∗ ( f )
where G∗ ( f ) is the complex conjugate of G( f ). We refer to the power spectrum

calculated in this way as the periodogram.
Currently, many investigators prefer to estimate the power spectral density using
matplotlib.mlab.psd(). This method is based on Welch’s averaged peri-
odogram method. Welch’s method reduces noise in the estimated spectrum at the
expense of reducing the frequency resolution (see below). For many of the experi-
mental uses of power spectra, the advantages of reducing the effects of noise out-
weigh the dis-advantages of reduced frequency resolution. As we showed in lecture
there is little practiacl difference between determining the periodogram versus the
power spectral density. However, the flexibility of mlab.psd() provides esuffi-
cient motivation to learn how to use this package.
Often overlooked is that the fact that W ( f ) is the power spectrum obtained for an
infinitely long time series measured with infinitely fine precision. In contrast, in the
laboratory we work with time series of finite length that are subjected to uncorre-
lated, random inputs (noise) and effects introduced by the process of measuring the
signal.
The purpose of this laboratory is to illustrate the use of mlab.psd() and ex-
plore a number of applications of the FFT and power spectra.
Browser: Essentially everything that you would ever want to know about the uses
of the FFT can be located on the Internet by asking the right questions. In Python,
1 For simplicity we have assumed that the independent variable is time; however, it could also
be a spatial dimension. We use the term power spectra as a collective term to include both the
periodogram and the power spectral density.
79
80 9 Lab 9: FTT and power spectra
the functions necessary to calculate the FFT are located in the numpy library called
fft. If we want to use the function fft(), we must add the following command
to the top matter of our program:
import numpy.fft as fft
Thus, the command for determining the FFT of a signal x(t) becomes fft.fft(x).
Of course, you could import the fft-package from numpy under a different name;
however, this might make the program less readable by others. Other functions re-
lated to the use of the FFT are located in scipy such as the library signal, i.e.
import scipy.signal as signal
The description of each command and how to use it can easily be obtained by using
your web browser and typing the name of the library together with the function that
you want to know about.
House keeping: Today we will illustrate the applications of the FFT and the
power spectra by working with real data. The deliverables take the form of figures
prepared using Matplotlib. We suggest that you create a directory for each figure
which contains both the data and the program(s) used to generate the required figure.
This is actually a good habit to acquire. One advantage is that you don’t need to
worry about specifying the required paths since, by default, a Python program looks
first in the directory they are running for required input files. A second advantage
occurs if, my goodness, you actually need to modify your figure at a later date
following comments made by a grader or a reviewer of your article (senior thesis,
publications, book).
9.1 Background
We illustrate the difference between W ( f ) and w( f ) by calculating the power spec-

trum of x(t) = sin(2π f0t), where f0 is a particular frequency. By definition, the
Fourier transform of sin(2π f t) is
Z ∞
G(sin 2π f0t)( f ) = e−2π j f t sin 2π f0tdt (9.1)
−∞
where we have distinguished a particular value of the frequency, f , as f0 . Using

Euler’s relation, we can write
e2π j f0 t − e−2π j f0 t
sin 2π f0t =
2j
Substituting into (9.1), we obtain
9.1 Background 81
e2π j f0 t − e−2π j f0 t
Z ∞
G(sin 2π f0t)( f ) = e−2π j f t dt
−∞ 2j
1 ∞ h −2π j( f − f0 )t
Z i
= e − e−2π j( f + f0 )t dt
2 j −∞
1
= [δ ( f − f0 ) − δ ( f + f0 )]
2j
j
= [δ ( f + f0 ) − δ ( f − f0 )]
2
Here we see that both the Fourier transform and the power spectrum, W ( f ), of
sin(2π f t) are predicted to be composed of two delta-functions, one centered at + f0
and the other at − f0 .
Fig. 9.1 Computer screen shot obtained after typing python at the Command prompt. The three
>>> means that we are operating in script mode.
Now let’s use Python to compute the FFT and the power spectrum, w( f ). Python
can be run directly from the command line, namely in an interactive mode2 (a
much more powerful and popular version of interactive Python programming is
iPython). To enable the interactive mode type python in your command win-
dow (Figure 9.1). Type the lines of Python code shown in Figure 9.2 to obtain the
FFT of a 1 Hz sine wave.
What does the output on the screen mean? First we note that there are 8 numbers
(the sin(2π f0t) was digitized to give 8 data points per second), all of the numbers
are written
√ in the form of a complex number, and Python uses the convention that
j = −1. By convention the FFT is outputted using reverse-wrap-around ordering.
For this example, the ordering of the frequencies is
[0, 1, 2, 3, 4, −3, −2, −1]
Note that the 8 discrete data points yield 8 Fourier coefficients and that the highest
frequency that will be resolved is the Nyquist frequency, N/2. The first half of the
list of numbers corresponds to the positive frequencies and the second half to the
2 Up to now we have run Python in its script mode, namely, we write computer program, name.py
and then run the program by typing python name.py (Did you remember to add the & ?). There
are two advantages: 1) we can use the same program over and over again, and 2) the program gives
us a permanent record of what we did. However, the interactive mode is useful when the goal is
simply to explore data.
Fig. 9.2 Obtaining the fft of a 1s 4Hz sine wave by running Python in script mode. Note that most
of the numbers for the fft are very small: there are two exceptions. See text for discussion.
negative frequencies. We suspect that most of you expected to see on the computer
screen
0.0000
-0.0000 -4.0000j
0.0000 0.0000j
0.0000 0.0000j
0.0000
0.0000 0.0000j
0.0000 0.0000j
-0.0000 +4.0000j
However, notice that many of the numbers are very, very small, for example of the
order 10−17 . These numbers simply represent numerical noise in the computer. If
we set mentally all numbers < 10−3 equal to 0, then we obtain the expected result.
It should be noted that recent releases of numpy.fft include the commands
rfft() and irfft(). The advantage of using rfft() over fft() becomes of
practical importance when we consider a problem that can arise when we compute
the inverse FFT using ifft(). Since we typically have a real signal, we expect
that the inverse should also be real. However, it can happen, due to rounding errors,
that ifft() contains complex numbers. The use of rfft() avoids this problem.
Thanks to the development of computer software packages, such as MATLAB,
Mathematica, Octave and Python, this task has become much easier. A particularly
useful function is Python’s matplotlob.mlab.psd() developed by the late
John D. Hunter3
matplotlib.mlab.psd(x, NFFT, Fs, detrend, window,
noverlap=0, pad_to, sides=, scale_by_freq)
3 We strongly recommend that readers do not use pylab’s version of psd(). The mat-
plolib.mlab.psd() produces the same power spectrum as obtained using the corresponding programs
in MATLAB.
9.2 Exercise 1: Computing the power spectral density using mlab.psd() 83
It is important to keep in mind that psd() does not calculate the periodogram,
but calculates the power spectral density using a mathematical method known as
Welch’s method. The advantages of using mlab.psd() are that it is very versatile
and for everyday use we can accept the default choices for most of the options.
Fig. 9.3 a) One-sided periodogram computed for 1s of a 4Hz sine wave using (9.1). b) One-sided
power spectral density computed using mlab.psd() for the same signal used in a).
9.2 Exercise 1: Computing the power spectral density using

mlab.psd()
9.2.1 Power Spectral Density Recipe
In the not too distant past, obtaining proper power spectra was a task that was beyond
the capacity of most biologists4 . However, thanks to the development of computer
software packages such as MATLAB, Mathematica, Octave, and Python, this task
has become much easier. Although it is relatively easy to write a computer program
that computes the periodogram, it is not so easy to write the program in a manner that
is convenient for many of the scenarios encountered by an experimentalist. For this
reason we strongly recommend the use of Python’s matplotlob.mlab.psd(),
4 For convenience we reproduce Section 8.4.5 here.
developed by the late John D. Hunter using the method described by Bendat and
Piersol [2].5 This function has the form
power, freqs = matplotlib.mlab.psd(x, NFFT, Fs,
detrend, window, noverlap=0, pad_to, sides=,
scale_by_freq)
The output is two one-dimensional arrays, one of which gives the values of the
power in db Hz−1 , the other the corresponding values of the frequency.
The advantage of matplotlib.mlab.psd() is its versatility. Assuming that
an investigator has available a properly low-pass-filtered time signal, the steps for
obtaining the power density density are as follows:
1. Enter the filtered and discretely sampled time signal x(t).
2. The number of points per data block is NFFT. NFFT must be an even number.
The computation is most efficient if NFFT = 2n , where n a positive integer;
however, with modern laptops, the calculations when NFFT 6= 2n are performed
quickly.
3. Enter Fs. This parameter scales the frequency axis on the interval [0, Fs/2],
where Fs/2 is the Nyquist frequency.
4. Enter the detrend option. Recall that it this a standard procedure to remove the
mean from the time series before computing the power spectrum.
5. Enter the windowing option. Keep in mind that using a windowing function
is better than not using one, and from a practical point of view, there is little
difference among the various windowing functions. The default is the Hanning
windowing function.
6. Typically accept the overlap default choice of 0 (see below for more details).
7. Choose pad to (see below). The default is 0.
8. Choose the number of sides. The default for real data is a one-sided power
spectrum.
9. Choose scale by freq. Typically, you will choose the default, since that
makes the power spectrum compatible with that obtained using MATLAB.
The versatility of mlab.psd() resides in the interplay between the options
NFFT, noverlap and pad to. The following scenarios illustrate the salient
points for signals for which the mean has been removed.
1. Frequencies greater than 1 Hz: The options available depend on the length of x,
L(x). If L(x) = NFFT, then the time is 1 sec and we can accept the defaults and
the command to produce the power spectrum. For example, when the digitiza-
tion rate is 256 points per second the command would be
mlab.psd(x,256,256)
We can increase the resolution by using the pad to option. For example for a
sample initially digitzied at 256 Hz, we can calculate the power spectrum for a
512 Hz digitization rate by using the command
5 Do not use PyLab’s version of psd(). The function matplolib.mlab.psd() produces the
same power spectrum as that obtained using the corresponding programs in MATLAB.
9.2 Exercise 1: Computing the power spectral density using mlab.psd() 85
mlab.psd(x,256,512,pad_to=512)
Finally we can average the time series by taking x to be an integer mutiple of
NFFT, Xn . This prodedure is useful when data is noisy and when we want to use
the noverlap option in addition to windowing to minimize power leakage.
Thus the command takes the form
mlab.psd(X_n,256,256,noverlap=128)
It should be noted that the optimal choice for noverlap is one-half of NFFT,
namely the length of the time series, including zero padding.
2. Frequencies less than 1 Hz: It is necessary to first determine the minimal length
of time series that is required. Suppose we wanted to look for periodic compo-
nents of the order of 0.1Hz. A 0.1Hz sinusoidal rhythm will have 1 cycle every
10 seconds. Thus the length of time series must be at least 10 seconds long.
However, we will clearly get a much better looking power spectrum the longer
the time series so maybe we should use 10 times longer, e.g. 100s. Thus the
command becomes
mlab.psd(x,6400,64)
It is important to note that the characterization of the frequency content of bi-
ological time series in the less than 1 Hz freqnecy range is made problematic
because of the effects of non-stationarity. Thus the length of the time series and
often the particular segment of the time series to be analyzed must be chosen
with great care. In fact, it is in the analysis of such time series that many of the
options available in mlab.psd() are most useful.
9.2.2 Example using mlab.psd()
Figure 9.3 shows the power spectrum determined for 1s of a 4 Hz sine wave sampled
at 64 Hz. Note that we have plotted the results using a linear scale for both axis.
What is the Nyquist frequency? It is 64/2 = 32 Hz. The periodogram is shown
in Figure 9.3a. We obtained the periodogram by squaring (term-by-term) the fft
shown in Figure 9.2. Since values except those at ±4Hz are zero we get two delta-
functions, each with with height 16. We have plotted the one-sided power spectrum.
The power spectral density is shown in Figure 9.3b. The power spectral density
looks different than the periodogram. Although both are centered at the expected
4Hz, the power spectral density is broader than the periodogram. A rough estimate
of the frequency resolution of a pure frequency in the power spectrum is the width of
the peak at one-half height: the bigger the half width the lower the frequency resolu-
tion. Thus the increased width of the 4Hz peak in Figure 9.3b reflects the decrease in
frequency resolution obtained when on uses Welch’s method to estimate the power
spectral density. Keep in mind that the advantage of Welch’s method occurs when
we have imperfect data of finite length subjected to random perturbations.
Fig. 9.4 Loglog plot of power spectral density determined using mlab.psd() for 1s of a 4Hz sine
wave. Note that we used the endpoint=False option in np.linspace(). What happens to the power
spectral density function when we choose the Default option, i.e. endpoint=True? Why?
Figure 9.4 shows the loglog plots of the power spectra shown in Figure 9.3. We
suspect that most of you are surprised by the appearance of loglog plot of the power
spectral density. Why does it look this way? To answer this question it is important
to remember that the power spectrum is actually being computed for the signal that
has been filtered using the rectangular sampling function. Thus the power spectrum
of a discrete frequency will resemble sinc(f). The reason that the ripples of the
sinc function are not apparent is because of the averaging (smoothing) that is part
of Welch’s method to determine the power spectral density.
1. Frequency resolution: The frequency resolution of the power spectral density
can be increased by increasing the length of the time series. In mlab.psd()
we can increase the sample length by increasing the number of points in the
data block, i.e. by changing NFFT, but by keeping Fs the same. For example,
if we had 10s of a 4Hz sine wave sampled at 64Hz, then NFFT = 640 and
Fs = 64. An alternate strategy is to pad with zeros. Modify sine_plot.py
to test these two methods, in each case observing what happens to the width of
the peak at half-height.
2. Aliasing: What happens if we add to xt a second sine term having a frequency
greater than 32Hz? Write a program in which you determine the power spectrum
for the signal
9.3 Exercise 2: Low-frequency signals 87
xt(t) = sin(2π f1t) + sin(2π f2t)

sampled at 64 Hz. Take, for example, f1 = 4 and f2 = 50. You will see two peaks
in the power spectrum; one at 4Hz and the other at second frequency less than
the Nyquist frequency, fnq . The appearance of a frequency component higher
than fnq as a frequency lower than fnq is referred to as aliasing. In other words,
when frequencies higher than the Nyquist frequency, f0 , are present, then these
higher frequencies will be falsely reproduced as lower frequencies. It is this
effect that is called aliasing. You can predict the aliased position of the high
frequency higher than the Nyquist frequency by using the relationship
sin(2π[2 fNyq − f ]tn + φ ) = − sin(2π f 0tn − φ ) ,
where φ is the phase and t 0 = 2 fNyq f . Thus for example, for a compact disk, the
Nyquist frequency is fnq = 22050. Hence a signal at a frequency of 34, 100 Hz
will look exactly like a signal at a frequency of 44, 100 − 34, 100 = 10, 000Hz.
Use this equation to predict where the aliased sine wave peak will appear in
your power spectrum?
3. The data file, sway.csv, is a two minute recording of the fluctuations in the
center of pressure (COP) for a healthy adult standing quietly on a force platform
with eye closed. The data is sampled at 200 Hz. The first column in the data
file is the x-coordinate (medio-lateral) of the COP and the second column is
p y-coordinate (anterior-posterior). Write a Python program to calculate X =
the
x2 + y2 and then determine the power spectral density using psd(). Do you
think that there is a significant periodic component present?
The data file, fish_1s_10000.tsv, is 1 second of the electrical signals
generated by a Keck Science Department electric fish sampled at 10000 Hz.
It is two columns of data: the first column is the time and the second column
is the voltage. The species of electric fish can often be determined from the
power spectrum [13]. A copy of the paper by Fugere and Kruhe on electric fish
power spectra will be supplied to you in the lab. Using the results of this paper
determine which type of electric fish corresponds to the one you calculated the
power spectrum for.
9.3 Exercise 2: Low-frequency signals
In biology, we are often interested in detecting frequencies that are less than 1
Hz. Examples include the resting adult respiratory rhythm (0.13 − 0.33Hz), circa-
dian rhythms (≈ 10−5 Hz), a 10-year population cycle (≈ 10−9 ). How do we use
mlab.psd() to calculate the power spectral density of such signals?
The important question to ask is what is the minimal length of time series that is
required. Suppose we wanted to look for periodic components of 0.1Hz. A 0.1Hz
rhythm will have 1 cycle every 10 seconds. Thus the length of time series must be
at least 10 seconds long. However, we will clearly get a much better looking power
spectrum the longer the time series so maybe we should use 10 times longer, e.g.
100s. In mlab.psd() we use the same trick as described above to increase the
resolution, namely, we set for example NFFT=6400 and Fs=64. We will return
to this problem in Laboratory 11. However, you should verify that this procedure
works for calculating the power spectral density of a 0.1 Hz sine wave.
There are a number of important issues related to the characterization of the
properties of low frequency signals. The most important are related to the fact that
as the frequency we wish to resolve become slower, the length of the time series
needed to resolve the signal rapidly becomes very long. As the length of the re-
quired signal becomes longer, issues related to the stationarity of the signal become
important. For example, there can be slow DC-baseline drifts in the outputs of mea-
surement devices, artifacts related to movement and sweating and as in the case of
the EEG, biological systems tend to naturally be continuously varying. Thus it is
possible that a low frequency signal of interest can be lost in the non-stationarity of
the process. Another problem arises because of the prevalence of noise in biologi-
cal system which typically contributes power to the spectrum proportional to 1/ f .
Consequently low-frequency signals can be lost in the noise.
9.4 Exercise 3: FFT Applications
9.4.1 Convolution:
Consider our black box analogy of a dynamical system. If the dynamical system is
linear then the output for any input can be determined using the convolution integral
as Z ∞
output (t) = impulse (t) input (t − u)du .
−∞
If we use the Fourier transform then
Output ( f ) = Impulse ( f ) Input ( f ) , (9.2)
where the capital letters have been used to indicate the Fourier transform. Since
the required Fourier transforms can be performed numerically using the FFT, (9.2)
provides another method for solving the convolution integral.
However, there are practical problems using the FFT to solve the convolution in-
tegral. The mathematical description of the integral transforms involves time series
that are infinitely long. On the other hand, experimentally attained time series are
always finite. Thus the will be effects, collectively referred to as end effects, that
effect the evaluation of the integral transform and consequently the evaluation of
the convolution integral. The practical solution is to pad the time series with zeros
[32]!
9.4 Exercise 3: FFT Applications 89
Performing the convolution integral assumes that 1) the length of the impulse
response function is the same as the length of the input time series, and 2) the input
time series is periodic. The solution to the first problem is straight-forward: we pad
the impulse response function by extending it with 0’s so that its total length equals
that of the input time series. In order to understand how the second problem can
be dealt with it is useful to refer to the graphical interpretation of the convolution
integral that we introduced in Laboratory 8. As a consequence of folding, displace-
ment and multiplication, a portion of each end of the input time series is erroneously
wrapped around by convolution with the impulse response function. Thus we need
to pad one end of the input time series with zeros to ensure that the convolution
integral is not contaminated from the effects of this wrap around effect. If the length
of the time series is N and of the impulse function is M, then the input time series is
padded at one end with N + M zeros [32].
The program used in Laboratory 7 used np.convolve() to solve the con-
volution integral can be modified to use np.fft.fft() to do the same exer-
cise. The program is called convolve_fft.py. We have modified this pro-
gram with the lines that pad with zeros deleted. You are asked to supply the miss-
ing lines (if you can’t figure this out you can check the lines in the programs
convolve_fft_neuron.py or convolve_fft_sine.py on the website).
9.4.2 Filtering images
Convolution of an input with a linear filter in the temporal domain is equivalent to

multiplication of the Fourier transforms for the input and the filter in the frequency
domain. This provides a conceptually simple way to think about filtering: transform
your signal into the frequency domain, dampen the frequencies that you are not
interested in by multiplying the frequency spectrum by the desired weights, and
then do an inverse transform to get the filtered signal.
To illustrate consider the picture shown in the top left corner of Figure 9.5 was
taken by a camera attached to a robot on the surface of the moon (moonlanding.png).
The picture is contaminated with high frequency noise related to the generation and
transmission of the image back to earth. How can we filter away the noise so that
we can see the sharper image shown in the lower left hand corner of Figure 9.5?
The power spectrum of a natural scene has the form
1
power (f) ≈
fα
where α ≈ 2 [7, 8, 35, 33, 34]. This observation means that the frequency con-
tent is predominantly in the low frequency range. By convention the 2-D Fourier
transform of an image is plotted such that the center of the image corresponds to
low frequencies and the high frequencies are located at the corners of the image.
Unfortunately, the np.fft.fft2() function in both MATLAB and Python plots
the data in exactly the opposite manner, namely high frequency components located
Fig. 9.5 High frequency noise filtering of a 2-D image in the Fourier domain. The upper two panels
show the original image (left) and its spectral power (right). The lower two panels show the same
data with the high frequency power set to zero.
in the center and low frequency components at the corners. Fortunately the func-
tion np.fft.fftshift() re-orders the frequencies correctly. The 2-D Fourier
transform of the image shown in the top left corner of Figure 9.5 is shown in the top
right corner.
A simple way to remove the noise is to use a brick wall filter. This is accom-
plished by setting the amplitudes of all of the frequency components greater that a
cut-off frequency to zero. The lower right-hand corner of Figure 9.5 shows the 2-D
Fourier transform of the brick all filtered image. This is not always the best way to
do this filtering: sharp edges in one domain usually introduce artifacts in another.
However, it is easy to do as a first pass and sometimes provides satisfactory results
(see lower left hand corner of Figure 9.5).
The program moon.py filters the images shown in Figure 9.5. As in the pre-
vious example, we ask you to supply the missing lines which perform the brick
wall filtering (if you can’t figure this out you can check the lines in the program
convolve_fft.py on the website.
9.5 Spectrograms
A spectrogram is a plot of the power spectrum as a function of time. For each time
(indicated on the x-axis) the variance of the power spectrum as a function of fre-
quency is plotted along the y-axis. A color or gray scale is used to indicate the
magnitude of the variance. In this way a 3-D plot is plotted in 2-D.
9.5 Spectrograms 91
Spectrograms are used to plot the power spectra describing the evolution of a
time-varying process. Perhaps the most common uses of a spectrogram are to ana-
lyze spoken words and the calls of animals, such as bird songs. Other applications
includes the analysis of music, sonar, radar and seismology.
Here we illustrate the use of a spectrogram by looking at a bird song. Most often
bird songs are stored in a *.wav file. A *.wav file is the standard Microsoft audio file
format. There are several Internet sites from which *.wav files can be downloaded
for analysis, for example,
http://www.ilovewavs.com/Effects/Birds/Birds.htm
Bird songs are also recorded in mp3 format. In this case the mp3 files can be con-
verted to *.wav files.
Fig. 9.6 Spectrogram for a goldfinch.
There are two practical problems. First, in our experience, *.wav file downloaded
from the Internet are often corrupted. Fortunately such files can often be restored by
using a software package called Audacity which can be freely downloaded. More-
over, a useful U-tube video explains how to use Audacity to recover a corrupted
*.wav file
http://www.youtube.com/watch?v=zaVr_WLikqs
The second problem is that Python currently has no audio playback software. There-
fore in order to hear the birdsong you will need to use a Quick time player or some
other software package to hear the bird song (Windows Media Player (PCs) or Quick
Time (PCs and Mac’s).
Figure 9.6 shows the birdsong of a goldfinch downloaded from the above bird
song website. The bird calls out twice. By looking at the spectrogram alone, can
you guess what the call sounds like? The Python program that generates this spec-
trogram is called spectro_bird.py. Modify this program so that the generated
spectrogram uses a color scale instead of a gray scale.

Chapter 10
Phase resetting oscillators (Lab 10)
The response of a dynamical system to brief perturbations underlines the concept

of the local stability of a fixed-point and consequently that of the bifurcation dia-
gram. It is natural to wonder about the responses of limit cycle oscillations to small
perturbations.
Fig. 10.1 Schematic representation of the effect of a single perturbation on a periodically spiking
oscillator. See text for discussion.
10.1 Background
Figure 10.1 summarizes the key observations for the response of an oscillator to a
brief stimulus. The dark vertical lines represent a marker event of the oscillation.
Examples of a marker event include the generation of an action potential, or neural
spike (this lab), heel contact with the ground during walking, the onset of mitosis in
periodically dividing cells, and so on.
Prior to the introduction of the stimulus, the oscillator produces one neural spike
per period, T . The perturbed inter-spike interval is T 0 . We consider the situation
that the rhythm is re-established “quickly” after the perturbation. This assumption
is equivalent to the observation that the effects of the perturbation disappear rapidly
on a time scale short compared to the period of the oscillator. The important point is
that the re-established rhythm has the same amplitude and inter-spike interval as it
did before it was perturbed. The lone exception occurs when the single pulses abol-
93
94 10 Phase resetting oscillators (Lab 10)
ishes the rhythm (as we saw in Laboratory 6). Although the rhythm is re-established
following the perturbation, subsequent spikes produced by the oscillator occur at
different times from what would have been observed in the absence of the perturba-
tion (compare solid and dashed lines in Figure 10.2. In other words the effect of the
brief perturbation is to simply phase shift the oscillator.
Fig. 10.2 Effect of a single 15nA, 5ms hyper-polarizing square wave pulse on a repetitively spiking
motoneuron of the sea slug, Aplysia. A 3nA depolarizing current was injected into the motoneuron
to cause it to spike periodically. The dashed lines show the unperturbed oscillator. Figure repro-
duced from [10] with permission.
A single stimulus may lengthen or shorten the inter-spike interval. In Figure 10.1
we show an example in which the perturbation lengthens the cycle, namely T 0 > T .
The effects of the perturbation depend on the magnitude of the stimulus and the time
at which it is applied. It is convenient to describe the timing of the perturbation in
terms of the phase during the cycle that it arrived, i.e.
t
φ= . (10.1)
T
It should be noted that φ is always a number between 0 and 1 and it is calculated us-
ing the unperturbed inter-spike interval, T . Thus a more correct way to write (10.1)
is to introduce the concept of the modulus, namely
t
φ= mod 1 .
T
Modulo 1 means the we consider only the remainder when we divide t/T by 1.
Hence 1.6 mod 1 = 0.6.
A plot of the phase shift of the oscillator as a function of the phase at which
the perturbation was introduced is called the phase resetting curve. Phase resetting
curves are further sub-divided depending on how the phase shift is measured into 1)
the phase response curve (PRC) and 2) the phase transition curve (PTC).
• The phase response curve (PRC) a plot of the change is phase, ∆ φ , as a function
of the phase, φ , at which the perturbation was delivered. As we showed in lecture
T0
∆φ = 1− (10.2)
T
10.1 Background 95
• The phase transition curve (PTC) is a plot of the new phase, φ 0 versus the old
phase, φ . As we showed in lecture
T0

0
φ = 1+φ − mod 1 , (10.3)
T
T0

= φ− mod 1 .
T
Fig. 10.3 Schematic representation of an Arnold tongue diagram. The horizontal dashed line is
related to the Devil’s staircase (see text for discussion).
A remarkable observation is that the effects of periodic stimulation of an oscil-

lator can be summarized in the form of an Arnold tongue diagram, such as shown
in Figure 10.3. In this diagram stable modes of N : M phase locking occur where
N is the number of cycles of the stimulus and M is the number of cycles of the
spontaneous rhythm and where N, M are positive integers. In practice the low-order
ratios are most often seen, e.g. 1 : 1, 1 : 2, 3 : 2, 2 : 3 and so on. For each N : M,
the phase locking patterns are arranged in tongues. As the stimulus amplitude (also
referred to as the coupling) becomes lower, the region in which a given N : M phase
locking mode is observed decreases. Thus for very low stimulus amplitudes it can
be difficult to maintain stable phase locking because of, for example, the effects of
noise.
The purpose of this lab is to demonstrate how the PRC and PTC are measured
and the uses of these curves for interpreting the responses of biological oscillators
to periodic stimulation in the form of an Arnold tongue diagram.
Browser use: The Scholarpedia page by Carmen Canavier on phase resetting

http://www.scholarpedia.org/article/Phase_response_curve
is excellent.
Housekeeping: Some of the laboratory exercises make use of some of the results
we obtained with the HH-equation in Laboratory 6. Thus it might be useful for
organizational purposes to work in the HHeqns directory. Alternately you might
choose to combine all of the exercises into a new directory called phase. If you
choose this option then move into HHeqns and copy hh.ode to phase using the
command1
cp hh.ode ˜/phase/.
What does the . mean? What does the ˜ mean?
10.2 Exercise 1: Phase resetting curves
XPPAUT’s Poincare Max/Min option in XPPAUT can be used to determine

the phase resetting curves when the equation for the oscillator is known (see Supple-
mentary Material at end of this lab). However, it is more relevant for those interested
in laboratory applications to construct the phase resetting curves point by point. It
is important to note that only three parameters can be measured: φ , T , and T 0 .
First we suggest that you do some house keeping. Move to directory HHeqn and
make a second copy of hh.ode by typing the command
cp hh.ode hh_prc.ode
Run hh_prc.ode by picking a value of curbias so that the periodic spiking
solution does not co-exist with a stable fixed-point (use the bifurcation diagram
for the HH neuron that you prepared in Lab 6 to determine a suitable value of
curbias). Click on Initialconds then Last several times to ensure that
you are on the limit cycle. Using your text editor, e.g. emacs, replace the initial
values of V, n, h, m with the values that appear in the initial values window. Save
hh_prc.ode, exit XPPAUT, and then re-start XPPAUT using the newly edited
version of hh_prc.ode. The advantage of using hh_prc.ode in this way is that
by clicking on Default in the initial conditions menu we ensure that each time we
run the program we start at the same place on the limit cycle.
It is easier to use XPPAUT to determine the phase resetting curves by choosing
an inter-spike interval and then varying the timing of the perturbation on this interval
(see below). For example we took curbias=25 and used the interval between the
spike that occurs at 15.92 and 19.84, so that T ≈ 3.92. Pick an amplitude for the
perturbation that is large enough to produce a phase shift. Note that we measured
these times by left clicking on the mouse and reading the (x, y)-coordinates of the
cursor below the graph. Use the cursor to check our estimate of T . An even simpler
way is to measure the distance between consecutive spikes off the screen using a
ruler. For the purposes of this exercise it does not matter.
Note that T and the magnitude of the perturbation are constant. The timing of
the perturbation is set in the parameter window. Thus by setting up the program as
described above, the only number we need to measure using the mouse cursor is
the time of the neural spike that occurs after the perturbation is delivered. In other
words we have reduced our work load.
1 The purpose of this strategy is to keep an unaltered version of the program so that we can use it
at another time.
10.2 Exercise 1: Phase resetting curves 97
Pick, for example, ≈ 20 values of the time to introduce the perturbation that span
the interval [15.92, 19.84]: this means that we increase the time of the perturbation
in 0.2s jumps. Use your text editor to create a data file (*.tsv) containing two
columns of data: the first column is the time that the perturbation was delivered and
the second column is the length of the perturbed inter-spike interval.
Questions to answer:
1. Write a Python program that uses the data you collected with the HH neuron to
calculate the PRC and the PTC.
2. What is the difference between the PRC and the PTC?
Comment: Although the PRC is widely used in the bio-mathematics literature, its
use in the laboratory can be problematic. For example it is possible that events
which occur after the perturbation is delivered are obscured by the perturbations
itself (namely, there can be a stimulus artifact). In these cases it is necessary to mea-
sure the phase difference asymptotically after several periods of the autonomous
cycle have elapsed. However, an ambiguity arises because phase differences of
−0.05, 0.95, 1.95 are asymptotically the same (remember that phase is defined mod-
ulo the period so that it is a number between 0 and 1). As we will see in this labora-
tory an advantage of the PTC is that it can be directly used in a circle map to predict
the responses to periodic stimulation (note that the PTC can be calculated form the
PRC) (see below). Thus it has been suggested that the PTC curve should be used in
routine laboratory work [10, 15].
1. Neurons exhibit two types of phase resetting responses referred to, respectively,
as Type 1 and Type 22 . Type 1 phase resetting means that phase is only advanced
by using brief depolarizing pulses. Type 2 phase resetting means that phase can
be either advanced or delayed depending on the timing of brief depolarizing
pulses. What type of phase resetting does a Hodgkin-Huxley neuron exhibit?
2. What is a simple mathematical function that approximates the PTC curve you
obtained for the HH neuron?
Comment: The differences in the phase resetting curves for neurons arises because
of the different mechanisms involved in spike initiation [4, 19]. Type 1 neurons
are capable of summing inputs over a broad range of frequencies and hence are
referred to as integrators. On the other hand, Type 2 neurons tend to resonate to a
preferred frequency and hence are referred to as resonators. Type 1 and 2 neurons
also display differences in their ability to synchronize with other neurons in a neural
network. Type 1 neurons cannot synchronize to purely excitatory synaptic inputs,
but can synchronize if the inputs are purely inhibitory. In contrast, Type 2 neurons
can synchronize to both excitatory and inhibitory neurons.
2 Be careful not to confuse Type 1 and Type 2 neurons with Type I and Type II excitability.
1. Is a HH neuron a resonator or an integrator?
2. Can a HH neuron synchronize in a neural network with purely excitatory inputs
(YES or NO)?
3. Can a HH neuron synchronize in a neural network with purely inhibitory inputs
(YES or NO)?
10.3 Exercise 2: Arnold tongue diagrams
In the laboratory it is easy to stimulate a neuron periodically. Thus at first sight it

would seem to be straightforward to construct the Arnold tongue diagram. However,
this approach is not the most reliable way to determine the Arnold tongue diagram.
First, a stable phase locking pattern does not mean that all of the solutions within
the Arnold tongue look the same (see below). For example, for 1 : 1 phase lock-
ing solutions can quantitatively differ by the phase in the cycle that the stimulus is
delivered. In other words the Arnold tongue groups solutions that are qualitatively
the same: 1 : 1 phase locked solutions all have the property that there is 1 stimu-
lus delivered per 1 cycle of the oscillator. Second, the phase locking properties of
oscillators to periodic stimuli is very much a work in progress. In fact, it is often
the case that stable phase locking patterns can be observed experimentally for stim-
ulus amplitudes that are larger than those for which a mathematical theory exists.
Moreover, the phenomena that occur between the low-order Arnold tongues is very
complex and only partially explained and explored. For example, it is known that
chaotic solutions can exist in these regions.
A reliable way to determine the Arnold tongue diagram for sufficiently small
stimulation amplitudes is to use the PTC. There are three steps in this procedure.
1. Construct circle map: The effect of a stimulus is to generate a new phase;
however, its value remains a number between 0 and 1. Thus the dynamics can
be described by the circle map
φn+1 = f (φn ) mod 1 ,
where f is a function to be determined3 . Specifically the circle map takes the

form
ts
φn+1 = g(φ , a) + mod 1 , (10.4)
T
3 We have defined the phase as a value between 0 and 1. Over the last century an elegant mathemat-
ical theory was developed by Poincaré and Arnold to analyze circle maps for situations in which
the amplitude is not too large, and then extended to applications by the work of Winfree, Canavier,
Glass, Ermentrout, and Strogatz. The mathematical definition of “not too large” is that there must
be a 1 : 1 correspondence between φ and φ 0 . Is this true for the phase resetting curves we have
studied in the lab today? Maps, f (φn , which exhibit this 1 : 1 property are said to be invertible.
Note that if the goal is to compare prediction to observation then an experimental challenge is to
ensure that the amplitude and duration of the stimulus is in an appropriate range.
10.3 Exercise 2: Arnold tongue diagrams 99
where, and this is the IMPORTANT POINT, g(φ , a) is the PTC curve, a is
the amplitude of the stimulus used to measure the PTC, and ts is the phase at
which the stimulus is delivered. Thus by measuring the PTC we may be able to
determine the responses of the oscillator over a range of appropriately chosen
stimulus amplitudes.
2. Calculate the rotation, or winding, number If we have an invertible map then
for all N : M, where N, M are relatively prime, positive integers (i.e. no common
divisor except 1) the solutions of the circle map are periodic. Moreover, for each
choice of the initial conditions the solution of the circle map asymptotically
approach a stable N : M phase locking pattern. The N : M values correspond to
the rotation, or winding number, ρ, and can be calculated as
∑nn=1 φn
ρ = lim . (10.5)
t→∞ n
In other words, all observed solutions within a given Arnold tongue are similar
because they have the same ρ. An important practical point is that (10.5) refers
to the cumulated phase and hence we do not take the modulus in evaluating this
expression.
Fig. 10.4 Cobweb diagram approach for graphically iterating the quadratic map (thick line) for
r = 3.25. The thin line shows xn = xn+1 . The alternating vertical (green) and horizontal (red) lines
illustrate the iterative process which is described in the text.
3. Identify the N : M pattern using a cobweb diagram If ρ is known then how do

we figure out the phase locking mode? The answer is make a cobweb diagram
using the program cobweb.py located on the website. A cobweb diagram is
way to graphically integrate an equation of the form
xn+1 = f (xn ) . (10.6)
For illustration consider the quadratic map,
xn+1 = rxn (1 − xn ) ,
where r is a constant. Choose r = 3.25 which corresponds to a period-2 cycle.

Figure 10.4 illustrates the cobwebbing procedure. Pick an initial value of x0 on
the xn axis. The value of x2 is found by drawing a vertical (green) line from x0
on the xn -axis to the thick line (i.e. 3.25x0 (1 − x0 ), then a horizontal (red) line
to x0 = x1 , and then a vertical (green) line (up or down as the case may be) to
the line xn = xn+1 , which yields x2 , then a horizontal (red) to intersect with
xn = xn+1 to obtain x3 , and so on.
10.4 Exercise 3: Phase locking modes for standard circle map
We illustrate this procedure by considering the standard circle map
φn+1 = [φn + b + a sin(2πφn )] mod 1 , (10.7)
where b is N/M. This map is invertible provided that a < 0.16. Comparing (10.7)
with (10.4) we have b = ts /T and g(φ , a) = φ + a sin(2πφn ).
Fig. 10.5 The devil’s staircase, namely a plot of the rotation number, ρ, as a function of the pa-
rameter b.
1. Use the program devils.py to determine the values of ρ as a function of b.
Setting a = 0.15 results in the Devil’s staircase shown in Figure 10.5.
2. What is the relationship between Figure 10.5 and the Arnold tongue diagram
shown in Figure 10.3? The Devil’s staircase is the Arnold’s tongue diagram
viewed along one horizontal plane (dashed line) which corresponds to experi-
ments done for one value of the stimulus amplitude. The “flat steps” correspond
to the cross-section of the low-order Arnold tongues for the given amplitude. By
inspecting the figure we see that the most prominent steps occur for ρ equal to
0.25, 0.33, 0.5, 0.67, 1.0. If you progressively decrease a you can, in principle,
generate the Arnold tongue diagram one slice at a time. Counter-intuitively this
is the most reliable way for generating an Arnold tongue diagram since trying
to accomplish this directly from the time series is difficult and often can lead
10.5 Exercise 4: Phase locking modes for the periodically spiking HH neuron 101
to measurement errors related to determining whether or not the solution has

settled onto its asymptotic solution. Run this program for different values of a
and use these observations to sketch the Arnold tongue diagram.
Fig. 10.6 Cobweb diagram for (10.7) for a choice of b corresponds to ρ = 32 with K = 0.15. This
solution correspond to a 2 : 3 solution: there are two contacts with the left branch of the circle map
for every three contacts of both branches to compete the cycle.
3. Modify the cobweb.py to identify the phase locking pattern that correspond
to the most prominent steps in the Devil’s staircase (Figure 10.5). For example,
for ρ = 2/3, we obtain the 2 : 3 phase locking pattern shown in Figure 10.6.
Why is this a 2 : 3 pattern. The value of m is equal to the number of steps to
go around the closed orbit in Figure 10.6 for one complete cycle. This is the
number of vertical dashed lines and hence we see that m = 3. The value of n is
the number of times per complete cycle that the closed orbit hits the branch of
the map on the right. We see that n = 2 and hence we have a 2 : 3 phase locking
pattern. Practice identifying phased locked patterns by trying different values
of the parameters. Note that within a given Arnold tongue the solutions can be
quite complex; however, they are all qualitatively described by the same n : m.
4. Notice that in Figure 10.6 we choose a very small value of a. What happens to
the cobweb diagram as a is increased (remember that a ≤ 0.16)? What happens
to the N : M phase locking pattern?)
10.5 Exercise 4: Phase locking modes for the periodically spiking

HH neuron
In Exercise 1 we measured the PTC for a periodically spiking neuron. Fit the ob-
served PTC with a continuous mathematical expression, such as a polynomial. For
example we choose PTC as
g(φ ) = [1 + φ + k ∗ sin(2πφ + π)] mod 1 .
In other words we approximated the PTC of the HH neuron as a standard circle map
phase shifted by π. What happens to the Devil’s staircase? Of course our choice of
function to fit PTC is not very accurate. Can you do better?
1. Modify devils.py and cobweb.py using your expression that describes
the PTC (remember to first make a copy of this program, give it a new name,
and work with the newly created program). The Devil’s staircase can be ob-
tained by varying ts over the range [0, T ]. For a value of ts that yields value
of ρ corresponding to a low-order mode locking solution, use your modified
cobweb.py to identify N : M.
10.6 Supplementary material: Phase resetting curves using

XPPAUT
XPPAUT contains a very useful function in nUmerics called Poincare Max/Min.

This function is based on the concept of a Poincaré section that we discussed in lec-
ture. Here we use this function to determine the period of an oscillator and then its
phase resetting properties. The basic idea is that we choose the Poincaré section to
correspond when the variable crosses a threshold value, say 0, in the positive direc-
tion. In other words, in the absence of noise, the time between successive crossings
of the threshold corresponds to the period.
This procedure works best for oscillations characterized by one maxima per pe-
riod. Here we illustrate the method by calculating the phase resetting responses of
the van der Pol oscillator
dx
= y,
dt
dy
= −x + y ∗ (1 − x2 ) .
dt
The program that you will need from the website is vdpphase.ode. Download
this program and open it up with your text editor (e.g. emacs).
The first task is to figure out how to make a pulse using XPPAUT. This can be
done by making use of the Heaviside function, heav(arg). This function is equal
to 0 if arg1 < 0 and is equal to 1 otherwise. Draw the shape of the pulse that would
be drawn using the command
pulse(t)=heav(t)*heav(pw-t)
10.6 Supplementary material: Phase resetting curves using XPPAUT 103
The program vdpphase.ode introduces the pulse using the command

dy/dt = -x + y*(1-xˆ2)+a*pulse(t-tp)
Draw a*pulse(t-delta).
The first step is to determine the period, T , of the van der Pol oscillation and the
first step of the procedure is to determine an appropriate initial condition. In order to
understand how this is done we need to remember the properties of the phase plane
and, in particular, that trajectories are traveled in the clockwise direction. Thus the
largest value of x is the value to the extreme right of the limit cycle (x is plotted
on the x-axis). If we choose this value at the initial condition, then one full period
corresponds to the time taken when x next achieves its maximum value.
Set a = 0 and run the program by clicking on Initialcond, then Last several
times to allow transients to die out (see Laboratory 6). Click on Data, then on
Find. Type in x and then 100. The number 100 is arbitrary. The only condition is
that it should be larger that you expect that the largest amplitude will be since it
ensures that we find the maximum. This command makes XPPAUT find the largest
value of x closest to 100. Click on Get to load this value an an initial condition (you
can also type this value into vdpprc.ode so that you can by-pass this procedure
the next time you use this program).
Fig. 10.7 Filling out the PoincareMax/Win menu.
Now click on nUmerics and then click Poincare map Max/Min to open
up the dialog box. Fill in the parameters as shown (Figure 10.7). These parameters
tell the program (from top to bottom) that the variable to monitor is x, that the
threshold (Poincaré section is placed at x = 0, that we want to measure crossings in
the upward (1) direction (the downward direction would be −1, and finally we stop
once a crossing has been detected. Click OK.
Now click on Transient and choose 4: this allows the integration to proceed
for awhile without looking at and storing values and hence makes the program
faster to run. The choice of 4 is arbitrary; however, it must be a positive number
smaller than the period. Exit the nUmerics window by pressing Esc. Now click
on Initialconds and then Go. When the program has finished click on Data,
then on Home. You will see that Time has a value of 6.6647 and this value is that
of the period. As above, you can type this value into vdpphase.ode so that you
can by-pass this procedure the next time you use this program.
We are finally ready to calculate phase resetting curves. Note that vdpphase.ode
uses an auxiliary function to calculate the phase response curve (PRC). You should
also define an auxiliary function to calculate the phase transition curve (PTC). A
problem with XPPAUT is that it does not have a command to calculate the modulus.
What strategies can you use to overcome this problem? Change the amplitude of the
perturbation to a = 3. Click on Initialconds, then Range to open up the dialog
box shown in Figure 10.8. Fill it out as shown and then click OK. Use Viewaxes
to put phase on the x-axis and prc on the y-axis.
Fig. 10.8 Filling out the Range menu.
Often we wish to compare the PRC’s obtained with different choices of a and
pw. Since the memory resources of the computer are limited it is not a good idea
to simply plot trajectories without pressing Erase. A better way is to “freeze” the
image by clicking on Graphic stuff and then Freeze On Freeze. In this
10.6 Supplementary material: Phase resetting curves using XPPAUT 105
way, the phase resetting curve is frozen onto the graph on the computer screen, but
the data is no longer stored in memory. Thus we free up computer resources for the
next calculation.
Chapter 11
Blood cell dynamics: Oscillations, delays and
chaos (Lab 11)
Time delays are intrinsic properties of all real, spatially extended biological systems.
These time delays arise because it takes time for information to travel from point A
to point B. For example, in the nervous system time delays arise because axonal
conduction velocities are finite. In the endocrine system, time delays arise because
blood circulation times are finite. Another reason time delays arise is because of
processing times. For example, it takes time for an organism to mature before it is
born and attains reproductive age, for a protein to be synthesized, and so on. An
important point is that if there is a time delay in the biological system, there should
be a time delay present in the relevant mathematical model.
Time delay differential equations (DDE) take the form, for example,
dx
+ k1 x = f (x(t − τ))
dt
where τ is the time delay, k1 is a constant, and x(t), x(t − τ) are, respectively, the
values of the variable at time t and t − τ. In order to solve a delay differential equa-
tion it is necessary to specify an initial function, Φ(s) where s ∈ [t0 − τ,t0 ]. If we
define the dimension of a dynamical system as the number of initial conditions that
must be specified to uniquely determine a solution, we see that a DDE is an infi-
nite dimensional differential equation. More importantly, for a DDE the dynamics
in the present depend on the dynamics in the past. This is completely different from
dynamical systems described by ordinary differential equations (ODE) in which dy-
namical behaviors are completely determined by the present.
Just a few years ago there were no software packages available to integrate DDEs.
At that time most investigators wrote there own computer code; most often using
an Euler integration scheme. One of the very first successful packages to integrate
DDE was XPPAUT. Since that time many software packages have become avail-
able, perhaps the most well developed are MATLAB’s dde23 and DDE-biftool.
Pydelay is an implementation of dde23 into Python1 .
1 Note that there is another package called PyDDE; however, we do not recommend using this
package.
107
108 11 Blood cell dynamics: Oscillations, delays and chaos (Lab 11)
11.1 Background
The lecture introduces XPPAUT and Pydelay for the integration of DDEs. XP-
PAUT is described in [5] and on the XPPAUT website and PyDelay is described
on the PyDelay website. Documentation concerning dde23 are the excellent pa-
pers by L. F. Shampine and S. Thompson (easy to download from Internet). The
discussion presented in these sources very much follows the presentation of the lab
material given below.
Browser: Overall, the use of software packages to integrate DDEs is not as well
documented on-line as the programs that integrate ordinary differential equations.
On the Internet an example of how to use XPPAUT to integrate a DDE can be found
by either typing
XPP examples
or by visiting the website
www.math.pitt.edu/xpp/help/xppexample.html
Documentation for Pydelay can be found at
http://pydelay.sourceforge.net/
or by simply typing Pydelay in a web browser.
Housekeeping: No new file extensions are needed for programs that integrate
DDEs. In other words they will have the file names myname.ode or myname.py.
However, you may find it useful to put your DDE programs into a sub-directory of
your home directory called delay.
11.1.1 Integrating DDEs using XPPAUT
We illustrate the use of XPPAUT for integrating DDEs with the Wright equation,
namely, the first-order DDE
dx
= k1 x(t − τ) . (11.1)
dt
In XPPAUT, the term x(t − τ) is represented as delay(x,tau). With this in mind
the program delay_first_order.ode becomes
dx/dt=k1*delay(x,tau)
par k1=-1,tau=0
init x=1
aux v=delay(x,tau)
@ TOTAL=500,delay=10,xlo=0,xhi=500,ylo=-5,yhi=5,maxstor=100000
done
11.1 Background 109
As for ordinary differential equations, the DDE is integrated by typing the com-
mand
xppaut delay_first_order.ode &
The computer screen will look like that shown in Figure 11.1.
Fig. 11.1 When XPPAUT is used to integrate a DDE it is necessary to open three of the tabs,
namely, ICs, Delay, and Param. We strongly recommend that the windows be arranged as shown
in this figure. In order to successfully integrate a DE it is necessary to click ‘OK’ in ALL three
windows (e.g. from top to bottom) and then click ‘Go’ in the bottom window. This tedious process
is necessary to ensure that the initial function is loaded correctly.
There are a number of points to keep in kind when using XPPAUT to integrate
DDEs.
1. The parameter tau refers to the value of the time delay used in the simulation.
The value of delay refers to the largest value of the delay that you might use.
Thus the value of delay must be greater than or equal to the tau value. It
should be noted that the value of delay reserves in memory the amount of
space that will be required to store the initial function. This observation should
be kept in mind in choosing the largest delay that you will use.
2. XPPAUT constructs the initial function Φ(s) in two parts: the value of X(s) at
t(0) and a second part that is a function defined on the interval [−τ, 0).
3. The functional part of the initial function can be introduced in two ways. First
we can use the Delay tab in the XPPAUT window (Figure 11.1). This is most
useful when Φ(s) is either a constant or is described by a reserved function
such as exp(). Second, Φ(s) can be constructed in the form of tables which
are read into the program (see below).
4. The most important point to remember is that each time a simulation is done it
is necessary to click ‘OK’ in both the ICs and Delay windows. We recommend
the procedure outlined in the legend to Figure 11.1.
5. It is often useful to define an auxiliary variable by adding the line
aux v=delay(x,tau)
to the program. This line defines the auxiliary variable, v, as the delayed value
of the value of the variable, x. This auxiliary variable can be use to construct a
phase plane representation of the dynamics by using Viewaxes. For example,
we can replace T by v in the Viewaxes window.
Questions:
1. What is the fixed-point for (11.1)?
2. The fixed-point for (11.1) becomes unstable when τ > | 2kπ1 |. For convenience
take k1 = −1.0. Pick different values of τ below this value. What happens to
the dynamics as τ → | 2kπ1 ?
3. How do the behaviors of (11.1) differ between the case when τ = 0 (an ordinary
differential equation) and the cases when τ 6= 0?
11.1.2 Integrating DDEs using PyDelay
We can also integrate DDEs using PyDelay. In our experience it is much easier to
use XPPAUT; however, using PyDelay has advantages. The program first_delay.py
integrates (11.1). Here we make a few comments.
1. The initial function, Φ(s), is created on [−τ, 0] using histdic. It is incorpo-
rated into the program through the command hist_from_arrays(histdic}.
There are other ways to do this, but the way we have done this is the most gen-
erally applicable. Note that in this example we have introduced Φ(s) = constant
by creating an array of 1’s and then multiplying by the constant.
2. The integrating time step is entered through the initial function, i.e. the inte-
grator uses the same dt as the increment in the array. For more complicated
DDEs, PyDelay varies the integration step. This results in a much more ef-
ficient and faster integration. This is done by adding the simulation parameter
dtmax which sets the maximum value of the integration time step.
11.2 Exercise 1: Periodic anemias
Background: It is generally accepted that in mammals there exists a pluripotential

stem cell (PPSC) population capable of providing differentiated cells for the ery-
throid series. Once a cell from the PPSC is committed to the erythroid series, it
11.2 Exercise 1: Periodic anemias 111
undergoes a series of nuclear divisions and enters a maturational phase for a period
of time before release into the blood as a mature erythrocyte. A fall in erythrocyte
numbers leads to a decrease in hemoglobin levels and thus in arterial oxygen ten-
sion. This decrease in arterial oxygen tension triggers the production of a substance,
renal erythropoietin (REP). Increased levels of REP, in turn, increase the cellular
production rate within the early committed erythroid series cells and, ultimately,
circulating erythrocyte numbers.
In an intriguing series of experiments [31] showed that the induction of auto–
immune hemolytic anemia (AIH) in rabbits is sometimes marked by a steady
depression of hemoglobin levels, and at other times by sustained oscillations in
hemoglobin concentration and reticulocyte levels with a period of 16–17 days. In
1979, Michael Mackey developed the model we will study in lab today to explain
these experimental observations [25].
Model:
Ė(t) = α(E(t − τ)) − β E(t) (11.2)

where
αθ n
α(E(t − τ)) =
θ n + E n (t − τ)
The important parameters for this model can be obtained from the literature and are
equal to:
τ = 5.7days
α = 7.62 × 1010 cells/kg/day
β = 2.31 × 10−2 day−1
θ = 2.47 × 1011 cells/kg
∗
Enorm ∗
= 3.3 × 1011 , where Enorm is the normal circulating numbers of erthyro-
cytes
n = 7.6
1. Write a computer program in XPPAUT to integrate equation (11.2).
2. You will find that the program becomes unstable whenever you run it. This is
because the numbers are pretty big. Re-write (11.2) using
E
Ê = ∗
Enorm
What is the equation in terms of this new variable using the values of the pa-
rameters given above?
3. Write an XPPAUT program to integrate this new version of (11.2). What hap-
pens when you run the program?
4. In auto-immune hemolytic anemia the peripheral destruction rate, β , is in-
creased. What happens as β is increased from 0.02 in steps up to, let us say,
0.5? For each value of β estimate steady state values of Ê. If there is an oscil-
lation estimate the minimum and maximum value of the oscillation. Plot these
values on the left axis as a function of β along the y-axis to generate a figure
that resembles the top panel in the left side of Figure 11.2. The graph shows a
Hopf bifurcation followed by a reverse Hopf bifurcation as β increases. Verify
that the middle and bottom panels of the left side of Figure 11.2 also describe
the behavior of (11.2).
5. It is easy of difficult to pinpoint the exact value of β at which the limit cycle
appears using numerical calculations. Describe the problem?
6. The right-hand side of Figure 11.2 shows blood cell populations of two strains
of rats following injection the same dose of 89 Sr [29]. In terms of the top panel
on the left–hand side of Figure 11.2 what initial and post-89 Sr values of β would
describe the responses of RBC synthesis in these two strains of rats?
7. Write a PyDelay program to integrate the differential equation for Ê. Do you
get the same results as you obtained for the XPPAUT program?
11.3 Exercise 2: Chaotic hematology
Background: The Mackey-Glass equation describes the growth of white blood cell
(WBC) populations [26]. This equation takes the form
dx Kµ
+ k1 x(t) = k2 x(t − τ) µ := f (x(t − τ)) , (11.3)
dt K + x(t − τ)µ
where k1 , k2 , K are constants and µ is the bifurcation parameter. The right-hand side
describes the dependence of f (x(t − τ)) on x(t − τ). When x(t) is low, f (x(t − τ)) ≈
k2 x(t − τ). Since k2 > 0, we have positive feedback. This is because low x leads to
the production of a hormone, granulopoietin, that stimulates production. However,
1
when x becomes high enough, f (x(t − τ)) ≈ x(t−τ) µ . Thus we have negative feed-
back. The type feedback described by f (x(t − τ)) is referred to as mixed feedback
since it combines both negative and positive feedback.
Figure 11.3 plots the dynamics of x predicted by (11.3) as the gain in the feedback
proportional to µ, increases. As µ increases, we first see a variety of periodic oscil-
lations (these arise by a series of bifurcations). When µ ≥ 9.70, we obtain a chaotic
solution. The Mackey-Glass equation was the first demonstration that chaotic dy-
namics could be generated by a plausible physiological control mechanism. In other
words, irregularly varying fluctuations in circulating blood cell numbers, such as ob-
served in patients with CML, may not simply represent lab error but can represent
the dynamics of a disease process. It should also be noted that when µ is increased
beyond the chaotic region, the fluctuations in x again become more periodic, indi-
cating that a reverse bifurcation can occur.
11.3 Exercise 2: Chaotic hematology 113
Fig. 11.2 a) A schematic representation of the control of PPSC regeneration. b) The qualitative
behavior of the model in (a) as β is increased. In the top panel, the dashed portion of the curve in-
dicates the range of β over which cell numbers oscillate with a period that increases as β increases
(middle panel). The solid lines bounding the dashed line indicate the maximum and minimum val-
ues of the oscillating cell numbers, from which the variation in the amplitude of the oscillation
with β can be determined (bottom panel). (c) Variations in hematocrit for two congenitally anemic
strains of mice after injection of a single dose of 0.5µCi/g body weight 89 Sr.
Subsequent work has shown that the Mackey-Glass equation does not provide
a valid description of WBC production. The main present–day use of the Mackey-
Glass equation is to provide a convenient way to produce chaotic signals to use as
bench marks for testing various time series analysis methods. The useful property
is that in the chaotic regime, the dimension of the chaotic attractor increases as τ
increases [6].
Fig. 11.3 Effect of changing the bifurcation parameter, µ, on the behavior of (11.3). Parameters:
k = 1, β = 2, K = 1, and τ = 2. The initial function is x(0) = 0.5 and x(s) = 0.5, s ∈ [−τ, 0).
1. Write a program using PyDelay to integrate (11.3). Note that this exercise is
much, much easier to do using PyDelay than using XPPAUT. Check to see
that you can reproduce the dynamics shown in Figure 11.3).
2. Use mlab.psd() to calculate the power spectrum for the four values of µ
used in Figure 11.3. Keep in mind that the time step of the integration must
be chosen to be short enough so that the numerical solution is valid, but the
length of the time series needs to be long enough to take into account that the
fluctuations of WBC certainly occur on time scales much longer than 1s (see
Laboratory 9).
3. What are the differences in the power spectrum between the periodic regime of
(11.3) (e.g. µ = 7) and the chaotic regime (e.g. µ = 9.69715).

Chapter 12
Arduino boards: Using your laptop in the
laboratory(Lab 12)
Up to now we have used our computer to prepare documents, collect information

from the Internet, analyze data and to study the dynamics of mathematical models of
biological processes. However, a computer can also be used to translate mathemat-
ical ideas into experimental paradigms. One way this becomes possible is through
the use of an A/D board. For example, analog data from a measuring device pro-
portional to the amplitude of a time-varying signal can be inputted to the computer
(A/D conversion). The computer can then perform some necessary calculations and
output a signal in the form of a voltage (D/A conversion) as an inputted stimulus to
the experimental preparation under study. In this way the computer can function as a
feedback controller or to make it possible to measure a phase resetting curve (PRC)
(Laboratory 10). Another possibility is that the computer can use inputted informa-
tion to turn on and off various measuring devices and stimulators to make possible
the design of automated recording and stimulation protocols [20]. This is because
devices can be turned off and on using TTL (transistor–transistor logic) pulses (the
inputs typically enter the measuring through connections located on the back of the
machine). These TTL pulses can be generated by the computer and outputted using
D/A conversion. In fact, the availability of inexpensive, credit–card size Strawberry
Pi computers (originally developed by Raspberry Pi to introduce computer program-
ming to elementary school children) has made it possible to build computers that are
dedicated to the performance of particular types of experiments or tasks.
Another strategy is to use the computer to program standalone electronic circuit
to perform tasks while the main computer does something else. A common exam-
ple arises in the collection of data from sensors. Today’s laboratory introduces the
Arduino board, an open–source electronics prototyping platform based on flexible,
easy-to-use hardware and software. The uses of this board are only limited by the
imagination of the user. Indeed one of our past students used an Arduino–type board
to program the amplifiers for their Alice Cooper–style band. The advantage of the
Arduino board stems from its open-source platform: many application programs can
be downloaded from the Arduino website (www.arduno.cc) and the Internet and
can often be readily implemented using inexpensive electronic components. For ex-
115
116 12 Arduino boards: Using your laptop in the laboratory(Lab 12)
ample, the light emitting diode (LED) we use for today’s lab cost $ 0.15 each (US)
and the accelerometer cost $ 11.95 each.
Browser use: All of the built-in functions for Arduino can be found on the Ar-
duino website: www.arduino.cc in the Reference section. Check it out! Data
sheets for various electronic components can readily be obtained on the Internet, of-
ten by entering the part number. For example, in the laboratory we use an accelerom-
eter (MMA7361). Type the name of the accelerometer, i.e., MMA7361, into a search
engine and click on links until you find it. You should download the data sheet for
MMA3761. In addition, websites that sell various are generally pretty good at sup-
plying the data sheet as well. We bought the components used in this laboratory
from SparkFun.
Housekeeping: Today’s lab is run entirely using your own laptop. You should
already have installed Python (version 2.5 or higher but not versions 3+) and Mat-
plotlib (version 1.1 or higher) on your personal laptop. In order to prepare for to-
day’s lab you will need to install the Arduino software on your laptop: go to the Ar-
duino website www.arduino.cc or type Arduino in your web browser), click
on Download tab at the top of the Home Page and follow the instructions appro-
priate for your computer. Ideally you should have this software installed on your
laptop before coming to today’s lab.
12.1 Exercise 1: Blinking LEDs
12.1.1 Background
We need to refresh our memories concerning electronic components and circuits.

• LED: stands for light-emitting diode. It can be turned on or off depending the
digital signal it receives. Each has two legs, one longer than the other. The longer
is the positive leg and is called the anode. The shorter is the negative leg and
called the cathode (labeled with a black stripe).
• Resistor: Each carries an electrical resistance. This resistance is delineated by the
colored stripes. It does not matter which direction they are connected. Each LED
requires a resistor to limit the electrical current it receives.
• Digital pins: For dealing with digital signals. For us, this means either a “HIGH”
or “LOW” signal. Pins 0 through 13 on the top row of the Arduino (see Fig-
ure 12.1).
• Ground: Circuits need to be connected to a ground. Not literally the ground, but
to a grounding pin. There are three ground pins on this Arduino and they are
labeled GND.
12.1 Exercise 1: Blinking LEDs 117
Fig. 12.1 An Arduino board connected to a LED using a breadboard and jumper wires. See text
for discussion.
12.1.2 The Code
The Arduino program is called a sketch. An Arduino computer program is very

similar to C++ code. Loops are enclosed in { } and a ; is used to end every line. A
// comments out everything after it and can be handy for making notes to yourself.
There are three basic, but required components to every Arduino sketch.
• Variable initialization: Establish the type, name, and initial value of the variables
that will be used in the sketch. The int indicates this variable is an integer.
We have named it led because it will be used for the LED experiment described
below. It is set to = 10 because we elected to use digital pin #10 on the Arduino
(Figure 12.1.
int led = 10;
• The setup: This program only runs once when you first upload the sketch to the
Arduino. pinMode is a built-in function that sets the pin you are using and how
you want to use it. In this sketch, we are using the led pin that we established
as digital pin #10 in the variable initializing section.
void setup()
{
pinMode(led, OUTPUT);
}
• The loop: This loop will run forever as long as the Arduino has power (connected
to your computer through the USB). digitalWrite is another built in func-
tion that, in this case, writes either a HIGH or LOW signal to the led pin. Writing
a HIGH signal passes current to the led pin and through the resistor and LED,
lighting it up. This particular loop turns the LED on, waits 1000ms (1 second),
turns off and waits 1000ms. Then the loop starts over again and the LED blinks
at 1 Hz
void loop()
{
digitalWrite(led, HIGH); // turn the LED on
delay(1000); // wait for a second
digitalWrite(led, LOW); // turn the LED off
delay(1000); // wait for a second
}
Figure 12.1 shows the Arduino board set that we will use to make the LED blink:
Time to make it happen:
• open the Arduino program and a new sketch (don’t forget to “save as” and name
it something)
• write the sketch (initializing variables, setup and loop)
• wire the circuit
• plug in the Arduino and upload the sketch
• watch the LED blink!
Fig. 12.2 Schematic diagram of a potentiometer. It does not matter which of the two connections
(A, B) is ground and which is power.
12.1 Exercise 1: Blinking LEDs 119
12.1.3 The Potentiometer
We can change the blinking rate by including a potentiometer (Figure 12.2) into the
circuit (Figure 12.3). A potentiometer is essentially a variable resistor. Each poten-
tiometer has three leads for ground, power and the wiper. Since the potentiometer
is basically a resistor it does not matter whether which connection, A or B, is the
power and which is the ground. By turning the dial of the potentiometer, you change
the amount of resistance on either side of the wiper which is connected to the center
pin of the potentiometer. This changes the voltage at the center pin (wiper). This
voltage is the analog voltage that we read as an input.
12.1.4 A few more things to know
• Analog pin: An analog pin receives data from a sensor. We will use the analogRead
function in the Arduino language. Arduinos have an analog-to-digital converter
which reads the voltage and converts it into a value in the range of 0-1023. There
are 210 values and hence we have a 10–byte A/D converter.
• 5V pin: This pin on the Arduino constantly delivers 5 volts of power to whatever
is connected to it. This is not enough to hurt a person, generally, but it is enough
to hurt a sensor. So... do not connect the wiper (middle lead) of a potentiometer
to 5V.
• Serial Port: Used for communication between the Arduino board and a computer.
We can use the Arduino’s built-in serial monitor to communicate with an Arduino
board. Click the serial monitor button in the tool bar and select the same baud
rate (communication speed) used in the call to begin(). The default is 9600
and this is what we will use.
12.1.5 The Code
The code used to read a potentiometer is as simple as the code we used to make the
LED blink:
• Initializing variables: For this sketch, two variables need to be initialized. One
for the pin that we’re going to use, and one for the reading from the potentiometer,
int pot = A0;
int sensorVal = 0;
• The setup: Just like with the LED sketch, pinMode sets the pot pin as an
INPUT. Serial.begin starts the connection with the Serial port so that we
can see the readings from the potentiometer. As mentioned earlier, the Arduino
communicates with the serial port at a certain speed. The default is 9600 and this
is what we will use.
void setup()
{
pinMode(pot, INPUT);
Serial.begin(9600);
}
• The loop: The signal from the potentiometer are read through the serial port
using the built-in analogRead function. Then, the readings are printed in the
serial monitor.
void loop()
{
sensorVal = analogRead(pot);
Serial.println(sensorVal);
}
Fig. 12.3 The Arduino board shown in Figure 12.1 now includes a potentiometer.
So, let’s hook it up and get this LED blinking! Notice how the negative, cathode,
end of the LED is connected to the GND and the longer, positive anode is in series
with a resistor in digital pin #10. To make this circuit, use the breadboards and
jumper wires to connect the Arduino to the bread board, as seen in the schematic.
12.2 Exercise 2: Measuring Tremor 121
Just remember, the bread board makes connections across the rows horizontally.
Because this potentiometer is basically a resistor, it does not matter which of the
two outer leads goes to either power or ground. The convention is that the red wire
is for power and the black wire is for grounding. Notice a potentiometer is connected
to an analog pin, not a digital pin. Remember the middle wiper lead CANNOT be
connected to 5V.
To obtain some readings using the potentiometer:

• write the sketch
• wire the circuit
• plug in the Arduino and upload the sketch
• open the serial monitor (upper right corner)
• turn the knob of the potentiometer
• watch the readings in the serial monitor
• How would you incorporate the potentiometer into the timing of the LED blinks?
To ask it in a different way: How could you use the potentiometer to control the
frequency of the blinking LED?
• How could you use the potentiometer to make a dinner and control the brightness
of the LED?
Implement your ideas and see if they works. It may not work perfectly the first
time so keep trying! Accomplishing this task will make a complete feedback loop
and help show you how these little Arduinos are actually quite powerful micro–
controllers.
12.2 Exercise 2: Measuring Tremor
Relax your hand and hold it in front of you with fingers outstretched. Have your lab
partner look at your fingertip and they will notice that your fingertips are moving
very, very slightly. Don’t worry, these movements are normal and are called tremor.
There are many contributions to your trembling fingertips. Physiological tremor
refers to frequency components in the 8 − 12Hz frequency range and is thought to
be related to the dynamics of the stretch reflex [24]. Evidence in support of this
hypothesis in the absence of these frequencies in some healthy individuals and in
patients with certain peripheral neuropathies and the magnitude of the latency for the
stretch reflex ([24] and this lab). However, there are many other contributions to the
fingertip tremor including those related to the beating of your heart (the so–called
ballisto–cardiogram). What would be expected frequency range of contributions to

the tremor related to your heart beat?
Other types of tremors arise in patients with diseases of the nervous system.
Examples include the pill-rolling tremor (∼ 4 − 6Hz) observed in patients with
Parkinson’s disease when their hand is resting and essential tremor, a familial dis-
order characterized by a tremor that worsens with movement. Thus neurologists
and neuro–scientists have directed considerable attention to the movements of the
fingertip.
How can we measure fingertip tremor? One solution is to spend a lot of money
and purchase an instrument designed especially to measure tremor. A more practi-
cal solution is to use an Arduino board to collect data from a triaxial accelerometer,
an electronic component that measures acceleration in 3D. Accelerometers are in-
expensive and it is likely that you can purchase the same accelerometer as in the
laboratory instrument for less than $ 50 (the accelerometer we use in this lab costs
$ 11.95). An additional advantage of measuring tremor in this way is that we may
be able to readily incorporate measurements of acceleration into novel experimental
paradigms. For example, a colleague developed a micro–accelerometer capable of
measuring limb movements in the developing chicken embryo F. Valero–Cuevas,
personal communication).
Now that we are motivated, let’s measure fingertip tremor!
Fig. 12.4 The MMA7361 triaxial accelerometer purchased from SparkFun. See text for pin iden-
tification.
12.2.1 What is an accelerometer?
Just like it sounds, an accelerometer measures acceleration. The acceleration that it

measures includes gravity and when the sensor is accelerated through a motion. The
accelerometer we are using is the triple axis accelerometer MMA7361. Triple axis
means that it can measure accelerations in all three planes: x, y, and z. On the front
of the accelerometer you can see the directions that each of the axes are pointing; x
and y are in the plane of the breakout board and the z direction is straight out of the
board. The accelerometer is attached to a breakout board which makes it possible to
make connections to the Arduino board so that the accelerometer can receive power
and measurements can be recorded. The specification sheet for MMA7361 can be
readily found on the Internet. Using this information we can identify the pins on the
The different pins on the breakout board are:
• Vcc: Every sensor requires power to run. This sensor operates on relatively low
power: only 3.3V. Be careful since the MMA7361 can not handle more voltage
than that. Remember with the potentiometer we used for the LED we could not
plug the wiper into the 5V pin on the Arduino. The same thing goes in this lab
but there are more pins to be careful about. Vcc can ONLY be connected to 3.3V
on the Arduino. It might be helpful to cover the 5V pin with tape or put a little
wire in it to remember. If you do accidentally connect Vcc to 5V, the sensor will
get warm and then hot and soon not work at all.
• GND: Just like we saw last time, everything that needs power also must be
grounded. This connects to the GND pin on the Arduino.
• Zout: This is where the measurement in the z plane comes out of the breakout
board. This connects to an analog pin.
• Yout: This is where the measurement in the y plane comes out of the breakout
• Xout: This is where the measurement in the x plane comes out of the breakout
• SLP: This is the sleep pin. It only understands a high or low signal so it will be
connected to a digital pin. Low is sleep and high is active. Its default position is
LOW so it must be given a HIGH signal to be able to get out any measurements.
• 0GD: This is a zero-g detection pin that is also a binary pin. It can detect when
the accelerometer is falling with the acceleration of gravity and can turn off the
rest of the sensor so it doesn’t get damaged when it eventually hits the floor. We
are not going to use this today.
• GSel: This pin allows you to indicate how sensitive you want the accelerometer
to be. It’s default is ±1.5g when given a LOW signal and ±6g when HIGH. We
may want to play around with this today.
• ST: This pin is a self-test pin used to check if the accelerometer is working cor-
rectly. Hopefully we won’t need this today at all.
12.2.2 The Code - Writing the Sketch
12.2.2.1 Variable Initialization
Just as we did for controlling the blinking of a LED, we need to write a sketch.
The first step is to write the variable initiation section. We advise that the sketch is
written to initialize the pins related to all of the variables that will be measured. This
organization is useful when it is necessary to change a pin since it is only necessary
to change one number at the top of the sketch instead of referring to the changed pin
throughout the rest of the code. Note that it does not matter what number pin you
assign to your pin variables, just that you get the correct type: digital or analog. Do
you remember the difference?
int zAxis = A0; // analog
int yAxis = A1; // analog
int xAxis = A2; // analog
int slp = 4; // digital
These three variables are going to be what we name the measurement coming
from the analog pin.
int z;
int y;
int x;
In addition we need to initialize two variables that will be sued for the timer (see
below). They are initialized as long variables is that they could potentially get very
large. For example we may decide to sample at 512 Hz for 10 minutes.
long currentTime = 0;
long lastTime = 0;
12.2.2.2 The Setup
In the setup, we use pinMode to set up those pins that we will actually use (don’t
forget about the Sleep pin! Go back up to where we were talking about what this
pin does to figure out how to set it up.). The other thing that you need to do in
the setup is to begin the communication with the serial port. This is done using
Serial.begin. Just remember that the setup only runs once so anything that
you only need to do once goes in the setup (hint: sleep pin).
12.2.2.3 The Loop
In the case of the blinking LED we used delay to make the loop. However, in this
case we’re going to use a timer. There are a number of advantages for using a timer:
1) using a timer prevents the loop from slowly down each time it is used; 2) it allows
greater flexibility and speed options when printing data; and 3) it is possible to keep
track of time efficiently. All of these advantages are important given that our ultimate
goal is to calculate a power spectrum in order to determine the frequency content
of tremor. The biggest thing to understand is the function millis(). Calling this
function sets that variable to how long the program has been running, but not in
seconds, in milliseconds. With that knowledge, take a look at this and see if it makes
sense. This is not going to be your whole loop and you don’t have to type this, just
look at it to understand · · · we just want to give you an idea of what a timer is:
void loop()
{
currentTime = millis();
if ((currentTime-lastTime) > 100)
{
lastTime = currentTime;
x = analogRead(xAxis);
Serial.println(x);
}
}
What happens if you change the 100 to a 10?
Now that we know what a timer is, we can build the rest of the loop. Try this example
out. You can tweak it as you want:
void loop()
{
currentTime = millis();
if ((currentTime-lastTime) > 10)
{
lastTime = currentTime;
z = analogRead(A0);
y = analogRead(A1);
x = analogRead(A2);
Serial.print("x: ");
Serial.print(x);
Serial.print("\ty: ");
Serial.print(y);
Serial.print("\tz: ");
Serial.print(z);
Serial.print("\n");
}
}
Once you’ve made the correct connections from the accelerometer to to the Ar-
duino, upload the sketch and move the sensor around while looking at the serial
monitor. Do you see the values changing? What do they mean?
12.2.3 Calibration
The first thing we need to do with these accelerometers to understand their data
is to calibrate them. The data from the potentiometer in the LED experiment took
the form of numbers that were being printed on the serial monitor that were in the
range of 0 to 1023. This is not specific to potentiometers but rather to the function
analogRead. Every time you read from an analog pin, the numbers produced will
be in this range. However, to make sense of these numbers we need to know what
the smallest number corresponds to? ±5V, ±1g, etc. Once we know the range we
can scale the remaining numbers as fraction of the whole range. The procedure of
determining the operating range of an accelerometer (or, in general, a sensor) is
called calibration.
Calibrating the sensor to gravity is the easiest. We will do this by holding the sen-
sor in different positions (i.e. the y axis pointed directly up) and taking the average
of the data and scaling it to gravity. Here is a suggestion of how we would do it:
• To make it simple, just print out one variable into the serial monitor at a time. So,
maybe you do just use the first loop we wrote out above.
• To begin with the easiest, hold the sensor with the y axis pointed up. Keep it in
this position until you have gathered the data necessary.
• Upload the sketch onto the Arduino and open the serial monitor. Make sure the
serial monitor is in auto-scroll mode.
• After a few seconds, uncheck the auto-scroll in the serial monitor and highlight
all the data from where you stopped to near the top and copy it. The reason to do
this is to get the average reading for the sensor in this position and then you can
scale it to gravity.
• It is convenient to past this list of data into an Excel file and then use Excel’s
=average function to find the average of the data set.
• Do this for all 6 (two for each axis) positions. Remember to print out the correct
axis.
• Then set up your excel file to resemble that shown in Figure 12.5.
Fig. 12.5 Example of an Excel file setup that is useful for calibrating the accelerometer.
• The big-sized numbers in the third row are the averages of the whole column
below each. Notice how the larger numbers in the 400’s and 500’s are when the
axis on the sensor is pointed up. The 1 and -1 in the cells above that are indicating
how we are going to scale the numbers. Essentially, we’re going to say that when
an axis is pointing up, that reading is equivalent to 1g or 9.8 m/s2
• Then we can use the Excel functions =slope and =intercept to find the
proper scaling. The x-values should be the analogRead values and the y-
values should be the 1 and −1. We’re using the simple mx + b equation to scale
the Arduino readings to gravity.
• Use these values (the slope and the intercept) to do some calculations in your
Arduino sketch. Every sensor is going to be a little bit different but you can use
the one we developed in this lab as a starting point.
z = analogRead(A0);
y = analogRead(A1);
x = analogRead(A2);
float xx = x*.00623-2.099;
float yy = y*.00600-2.170;
float zz = z*.00596-1.836;
• Notice how we used the values of m and b determined using Excel for the calcu-
lations in the Arduino loop (with a little bit of rounding).
• Once you’ve calibrated the sensor, now you can get meaningful data out of it!
Fig. 12.6 Triaxial accelerometer taped to index finger.
12.2.4 Data from fingertip movements
Once the accelerometer has been calibrated, rewrite the loop section in the sketch
so that it will print out x,y and z data (and maybe even the time, if you want) on
each line. Then when you highlight and copy from the serial monitor, the data will
be organized nicely in columns. Create data showing different things. Attach the
accelerometer to your index finger being careful to also secure the cables so that
they don’t move around (see Figure 12.6). Try just moving your hand back and
forth at first to see how you accelerate and decelerate to change directions. Do this
in one plane first and then make your movement more complicated. Finally collect
data when your hand is relaxed and fingers are outstretched. This is the data we will
analyze below.
An important question is to decide how you want to plot the data and also how
you will determine the power spectrum. Of course we are going to use a Python
program to accomplish these tasks. This means that we need to save the data in
a format that can be used in our Python program. PC users can paste it the data
into a plain text file (for example using Notepad) and then name it name.dat.
Mac users can paste their data into TextEdit. In TextEdit, under the format tab,
you can change the format to be plain text. Make a new folder and put the data
somewhere you’ll remember, like the desktop or in My Documents. This way, it
will be easier to find when you want to open it up using python.
Deliverables:

Appendix A
Appendix A: Finding data for analysis
Interesting data sets can be readily generated by integrating a differential equation.

However, many students are much more interested in working on real data sets. It is
surprisingly easy to obtain data for analysis.
Using an A/D board: By far the most exciting experience for science undergrad-
uates is to analyze data that they collected themselves. Present day undergraduate
teaching laboratories in biology, physiology and neuroscience have the ability to
collect and digitize biological signals (e.g. Backyard Brain, Phoenix, AD Instru-
ments). Using these devices it is relatively straightforward to obtain ASCII data files
for heartbeats, breathing, electromyographic (EMG), and electro-encephalographic
(EEG) data for analysis. A popular laboratory experience was to record the signals
generated by an electric fish (see Subsection ??).
In addition a number of useful, open source Python packages have been devel-
oped by Andrew Straw (Institute of Molecular Patholopgy, Vienna) that enable re-
searchers to use their personal computer to run experiments control an A/D board
and hence collect data from multiple instruments. These software packages can be
downloaded from
http://strawlab.org/
These include:
1. The PyUniversal Library which enables data to be collected through an A/D
board
https://code.astraw.com/projects/PyUniversalLibrary/
2. Vison Egg, a package for generating visual stimuli for psycho-physical experi-
ments
http://visionegg.org/
3. Motmot, a package for real–time collection and analysis of uncompressed digi-

tal images from a variety of courses
129
130 A Appendix A: Finding data for analysis
http://code.astraw.com/projects/motmot/
Electronic circuits: Many of the classic limit cycle oscillators of mathematical bi-
ology can be implemented as an electronic circuit. For example, Project 2 (Chap-
ter ??) shows how a circuit can be constructed to integrate the van der Pol oscillator.
Data generated by this circuit can be collected using an A/D board and the results
compared to those obtained usng numerical integration.
Internet data sets for analysis: A variety of databases exist on the Internet that
enable data sets to be freely downloadable. A very useful resource for obtaining
data from the Internet is Haddock and Dunn’s Practical Computing for Biologists
[18]. These include
1. PhysioNet provides access to large collections of recorded physiological data
sets
http://www.physionet.org/
2. Time Series Data Library, created by Rob Hyndman and now hosted by DataMarket.com,
provides about 800 time series from different sources
http://robjhyndman.com/TSDL/
3. Google’s Trends provide data concerning google hits foe search words as a
function of time
http://www.google.com/trends/
4. National Center for Biotechnology Information website provides sequences for

all known genes and, in some cases, the complete genome of an organism
www.ncbi.nlm.nih.gov
5. United States Census Bureau provides data sets for economic trends and demo-
graphics
http://www.census.gov/
6. A variety of EEG data sets are available, including

http://kdd.ics.uci.edu/databases/eeg/eeg.data.html
http://eeg.pl/epi
http://epilepsy.uni-freiburg.de/database
7. Birdsongs in the form of *.wav files can be downloaded and analyzed in the
frequency domain (see Chapter ??).
http://www.ilovewavs.com/Effects/Birds/Birds.htm
8. Stock history files can be downloaded from Yahoo! finance using the Python
program snip_stock.
A Appendix A: Finding data for analysis 131
Using a camera: Most students has access to a digital camera. The digital images
are in the format of a *.jpg file. These files can be readily handled in Python as
shown in Labs 4 and 9.
Using a scanner: A scanner provides another route to obtain data. In particular,
a figure can be scanned, converted to a *.png image, and the digitized using the
freely downloadable software package, Engauge
http://digitizer.sourceforge.net/
Appendix B
Appendix B: Latex survival guide
The philosophy in this course is that we would like to be good citizens of the scien-
tific community. An important part of being a good citizen is being able to commu-
nicate results, papers, and ideas. Since many of our colleagues live in different parts
of the world and hence have different resources available to them we cannot always
assume that our friends have the same software for preparing manuscripts that we
do. Moreover, since we use mathematics as our laboratory tool, how can equations
be exchanged easily by e-mail.
The solution we are choosing in this course is to make use of the open source
computing network. In other words, the only requirement that we require is that
our colleagues have access to the Internet. There are all sorts of computer software
packages available on the Internet; many of them are exactly the same softwares
that are presently used at the cutting edges of scientific research. A big advantage is
that once we become familiar with these products we have access to them anywhere
in the world. Study abroad students from The Claremont Colleges have already
been able to use their experience with the open source computing network to help
students, workers and scientists in the countries they have visited.
In this lab we address the problem as to how to manuscripts and figures to each
other. The software package we will use is called LaTeX. This is the same soft-
ware package that is used by the publishers of most of the scientific articles that you
read. LaTeX has a completely different concept than, for example, WORD. Pack-
ages such as WORD are actually programs that produce the document the way that
the programmers who designed this package wanted it to be produced. In contrast,
when we use LaTex we write the program ourselves that will be used to generate
the document. Since we are writing the program ourselves we can get an article that
looks exactly like we want it to look like. More importantly, the program is written
using a text editor (the same editor we will also use to write computer programs
that integrate equations) and has the form of an ASCII file. This means that we can
easily send it to our friends by email and they can then use it locally on their own
computer to generate the document.
133
134 B Appendix B: Latex survival guide
B.1 LaTeX manuals
There are several manuals for LaTeX that are available for free on the Internet.
Moreover you can also buy a variety of manuals at a bookstore. These notes give
a minimum background and set of commands that will be useful for this course.
However, as you get more and more experienced using LaTeX I suspect that you
will learn more commands.
B.2 LaTeX
There are three components to a LaTeX environment

1. An editor to edit and prepare tex files (named xxxxx.tex)
2. The TeX program itself (latex xxxxx.tex or pdflatex xxxxx.tex)
3. A way to view the result
There are many different combinations of the above components. In the past we
heavily used MiTeK which is a Latex package designed to work with Windows. In
this package all of these components are assembled into a single operating environ-
ment. This year we are switching to LINUX so that both PC and Mac users will
essentially be working in the same environment. In LINUX there are a number of
options available. However, they all work about the same; you will likely have your
favorite. Here are some guidelines.
B.2.1 Text Editor
A LaTeX file can be thought of as the instructions that the LaTeX computer program
uses to produce your article the way you told it to be produced. Like all computer
program files the code is produced using a Text Editor. Note that the important point
of a text editor is that the editor adds no editing commands itself. In other words you
are completely in control of exactly the way the article will appear in print. You no
longer have to worry about the WORD program doing lists the way you do not want.
This is why LaTeX is the program used by publishing companies, etc.
This year we are switching to a LINUX environment. At the beginning of the
course we will work in an openSUSE LINUX environment. Here, the text editor is
called emacs. When we switch to a Ubuntu LINUX environment, the text editor is
called gedit. It should be noted that in both environments you can install whatever
text editor you want (anything is likely better than Notepad/Wordpad); so you might
as well learn one and stick to it.
All LaTeX files have the form myfile name.tex, where the extension .tex is the
important part. Using the text editors on LINUX we can created a new file with the
B.3 Learning LaTeX 135
command emacs xxx.tex or gedit xxx.tex or whatever. If the file already

exists we can open it the same way.
A nice thing about LINUX is that you can easily find out which text editor your
system has by knowing some of the names of existing text editors and using ’Help’
or ’man’. It has become extremely easy to download these editors from the Internet
and install them on your system.
B.2.2 Latex versus pdflatex
When you use the command latex xxx.tex you generate a file with the ex-
tension xxx.dvi. When you use the command pdflatex xxx.tex you will
produce a xxx.pdf file.
The important difference between these files occurs when we try to incorporate a
figure into the document. When we use the command latex xxx.tex all of the
figures in the document must be xxx.eps type files. When we use the command
pdflatex xxx.tex all of the figures in the document must be xxx.png type
files.
Why are there two different ways to generate a LaTeX file? The advantage of
xxx.pdf files is that they do not require a postscript type printer in order to print.
All we need is Adobe Reader which can be freely downloaded from the Internet.
A second advantage of the use of pdflatex is that *.png, *.jpg, and *.pdf
figures can be incorporated into the document using the \includegraphics.
Why worry about xxx.eps files at all? Journals require that final figures be in
the xxx.eps format. Thus when most people prepare figures for their documents
they will prepare both a xxx.eps and a xxx.png version of the figure (as we will
see this is really easy to do).
B.2.3 Viewing the LaTeX file
In openSUSE the tool that is used to view a xxx.dvi file is xdvi xxx.dvi. The
tool that is used to view a xxx.pdf file is acroread xxx.pdf.
In Ubuntu the tool that is used to view a xxx.dvi file and a xxx.pdf is
evince xxxx.x.
B.3 Learning LaTeX
Most people learn LaTeX by doing it, i.e. their first experience is usually the result
of having to prepare their own paper because the journal publisher has asked for it.
This is in fact how you will learn to use LaTeX. It’s really not very hard. You do
need to know the basic structure of the LaTeX file: the various commands that you
need come from looking them up as you need them. This is where the Internet is so
useful (that’s where I found the links listed on your web page). Indeed I had to look
up a number of commands that I had never used before in order to make up these
notes.
Here we focus on preparing the LaTeX file you will use to submit your assign-
ments. In the next few weeks I will show you how to include a bibliography into
your document and to cite references within the text. You should regard this as your
first step into the scientific world!
Structure of a LaTeX file
There are three main parts of a LaTeX file:

1. Preamble
2. Body
3. Math modes
4. Bibliography
Preamble
The preamble is everything from the beginning of your file to the line \begin{document}.
It tells the program what fonts and style to use, the title and author of the article, and
whether additional packages of commands need to be accessed in order to produce
the article. The basic structure of the preamble is
\documentclass[12pt]{article}
\usepackage{graphicx,times}
\title{BIO--133: Assignment number xxx}
\author{Your Name\\
Your College}
The term ’12pt’ means twelve points. Points are the units that type–setters use.
This article has been printed using 12pt. If you want smaller print you could use, for
example, 10pt, or bigger 14pt and so on.
The term ‘article’ just refers to the style that the paper will be prepared. The other
style often used is ’book’. Many other styles are possible.
Under \usepackage{} you get to include user packages that are not in the
basic LaTeX package. A common one is graphicx which allows you to include
figures (either *.eps or *.png)into your document(see below). The user packages
times means that the font will be New York Times style. Often journals, such as
Nature, have there own package and require you to include it at this line. In fact you
could make up your own *.sty file and include it here (minus the extension).
Main Body
The basic structure of the main body is
\begin{document}
\maketitle
text here
\section{your section heading here}
text here
\subsection{your sub-section heading here}
text here
\end{document}
The commands \begin{document} and \end{document} are manda-
tory. The line \maketitle is included only if you made use of the lines \title{}
and/or \author{} in the preamble (which you will need to do for your assign-
ments).
The lines \section{} and \subsection{} generate, respectively, the ap-
propriate section and sub–section numbers, whereas the lines \section*{} and
\subsection*{} generate the same headings without the numbers.
Math modes
Of course the whole point of LaTeX is to easily write text that includes equations
and mathematical symbols. The fact that we use only a text file and the software
can be downloaded from the Internet by anyone who has a connection is where the
popularity and power of this program comes from. Indeed scientists and mathemati-
cians are so familiar with the commands that many use them in their e-mails to each
other.
The commands are actually pretty simple and once you get the hang of it you can
guess what the command without having to look it up. However, until your brain
accumulates the commands that you most frequently use it is a bit of a nuisance
going back and forth between manual and your document.
Inserting mathematical expressions into a text line
You use $..$ to insert equations and mathematical symbols into text. For example,
we have $math expression$, where $symbol$ is.
What about equations? The first question to ask is whether you want the equation
numbered or not. If you do not want the equation numbered, then
\[
your equation, e.g. a=b
\]
If you want the equation numbered, but don’t what to refer to it in your text, then
\begin{equation}
your equation here, e.g. a=b
\end{equation}
Finally, if you want the equation numbered and want to be able to refer to that
equation in the text, then
\begin{equation}
your equation here, e.g. a=b
\end{equation}
The big difference between the last two options is the inclusion of the line
\label{} in the latter. You can label anything this way, e.g. a figure, table, section
heading, chapter, and so on. A useful hint is to use a label that you can remember
and which in some way describes what it is that you have labeled. Here’s how you
refer to it in the text
(\ref{label}) yields (label)
Chapter˜\ref{label} yields Chapter label
Symbols
LaTeX is fun because you get to use all sorts of interesting symbols. Here are some
symbols that are frequently used in latex documents
α \alpha β \beta τ \tau ϑ \vartheta

π \pi γ \vartheta φ \phi δ \delta
κ \kappa ρ \rho ϕ \varphi ε \epsilon
λ \lambda µ \mu σ \sigma ψ \psi
ω \omage η \eta ξ \xi Γ \Gamma
Λ \Lambda Ψ \Psi ∆ \Delta Ω \Omega
Θ \Theta Π \Pi Φ \Phi
R H
∑ \sum \int \oint ∏ \prod
cos \cos sin \sin tan \tan log \log
± \pm ≤ \le \ll ≥ \ge

\gg • \bullet ◦ \circ \diamond
≈ \approx ∼ \sim ' \simeq ∝ \propto
≡ \equiv ∈ \in ← \leftarrow → \rightarrow
For a comprehensive list of all of the Latex symbols that are available see
http://www.tex.ac.uk/tex-archive/info/symbols/comprehensive/symbols-a4.pdf
Super and sub-scripts
Subscripts and superscripts are made, respectively, with the commands _ and ˆ.
Sums and integrals (under symbols on previous page) have upper and lower lim-
its. We can use the symbols _ and ˆ to introduce these limits.
The command \sum_{i=1}ˆ{n} produces
n
∑
i=1
and the command \int_{a}ˆ{b} produces

Z b
Two special considerations must be kept in mind when type setting integrals.
1. There should be a little extra space between the differential operators, e.g. dx
and the integrands preceding it. Thus
\intˆ{b}_{a} f(x) dx should be \intˆ{b}_{a} \ dx

2. According to the International Standards Organization the ’d’ in dx should be
upright. This means that if we want to be compliment with the world we need
to type
\intˆ{b}_{a} f(x) \ \mathrm{d}x
Useful mathematical stuff
Roots: The command for the n-th root of an expression is

\sqrt[n]{expression}
For example \sqrt[3]{2} produces
√
3
2
In the case of a square root you do not need the [2].
Fractions: The command for a fraction is

\frac{expression_a}{expression_b}
For example, \frac{aˆ2+bˆ2}{y+x} yields
a2 + b2
y+x
Automatic sizing of brackets: Mathematics often contains bracketing symbols,

usually in pairs that enclose part of the formula. Obviously when we print the text
we want these brackets to be scaled appropriately with respect to the mathematical
expression(s) they surround. This is accomplished by typing
\left lbrack expression \right rbrack
where lbrack indicates [,\{,( and rbrack indicates ],\},) and we note
that the brackets {,} require a \.
The math environments:
The math environments have the basic structure

\begin{environment}
commands here
\end{environment}
The math environments that we will use are called array, eqnarray, and
figure. As you will see the first two environments allow you to include systems
of equations, matrices, etc into your text and the last one introduces figures. There
is also a math environment that introduces tables; however, we will not use this
environment in this course.
The array environment: One of the most useful mathematical environments is

the array environment. A single argument specifies the number of columns and the
alignment of items within the columns. For each column, there is a single letter in
the argument that specifies how items in the column should be positioned: c for
centered, l for flush left and r for flush right. Within the body of the environment,
adjacent rows are separated by // and adjacent items within a row are separated by
an & character. It should be noted that array is activated in math mode (in contrast
to eqnarray). For example, the array environment
\[
\begin{array}{clcr}
a+b+c & uv & x-y & 46 \\
a+b & u+v & z & 1234 \\
a + 3u/v & uvˆ2 & 5176
\end{array}
\]
generates
a + b + c uv x − y 45
a+b u+v z 123
a u/v uva 5436
Now the reason why array is so important is that we can use it in so many differ-
ent ways. For example how can be rewrite the above equation as a matrix
 
a + b + c uv x − y 45
 a+b u+v z 123 
a u/v uva 5436
All we need to do is type the commands

\[
\left( \begin{array}{clcr}
a+b+c & uv & x-y & 45 \\
a+b & u+v & z & 123 \\
a & u/v & uva & 5436
\end{array}
\right)
\]
A frequent problem is to type set equations of the following form

y if y > 0
x=
z + y otherwise
The becomes in LaTeX

\[
x= \left\{ \begin{array}{ll}
y & \mbox{if $y>0$} \\
z+y & \mbox{otherwise}
\end{array}
\right.
\]
where we see how to type words inside of math mode (use \mbox{}) and what to
do if you include only one member of a bracket pair (type a . where the missing
lbrack or rbrack should be).
The eqnarray environment: The eqnarray environment is especially useful for

typing multi–line equations. In many ways the eqnarray environment works the
same way that the array environment works.
Here are some useful examples (note adding the ‘*’ deletes the equation number).
\begin{eqnarray}
x & = & 17y \\
y & > & a + 189 + \nonumber \\
& & b + 78
\end{eqnarray}
generates
x = 17y (B.1)
y > a + 189 +
b + 78 (B.2)
where we see that the addition of \nonumber removes the equation number from
the line it is typed on. On the other hand if we replaced eqnarray by eqnarray*
no equations numbers would appear.
If the formula is too long to put into a single line, then we must tell LaTeX that the
‘-‘ or ‘+’ is a binary operator. This is done by starting the formula with an invisible
first term, produced by \mbox{} with a null argument. Thus the commands
\begin{eqnarray*}
y & = & a + b + c + d + e + f + g \\
& & \mbox{} + h + j + k + l
\end{eqnarray*}
generates
y = a+b+c+d +e+ f +g
+h+ j+k+l
Another way to divide an equation across multiple lines is to split it as follows

\begin{eqnarray*}
\lefteqn{a+b+c+d =} \\
& & e+f+g+h+ \\
& & j + k + l + m + n
\end{eqnarray*}
where we obtain
a+b+c+d =
e+ f +g+h+
j+k+l +m+n
The figure environment In order to introduce figures into our article we need to
pay attention to the following points:
1. We must include the user package graphicx in the preamble of your LaTeX
file.
2. The figure must be in the form of either a *.eps file or a *.png file.
3. Both types of files can be produced by MATPLOTLIB (Pylab).
4. When you use *.png, then you must use the command pdflatex. This ap-
proach is good since you can use Adobe Acrobat Reader (can be downloaded
for free from the Internet) to print out your article on your own printer.
5. If you have access to a postscript printer (most of us do not), then you can use
*.eps figures and the LaTeX command to process your article.
The environment has the form
\begin{figure}
\includegraphics[width=.9\textwidth]{filename.eps}
\caption{your text here}
\end{figure}
Appendix C
Appendix C: Introduction of Python by A. H.
Lee
This tutorial assumes that the reader has no previous programming experience but
has access to an installation of Python. It is operating system independent (good
for both Widows or OS X) and intended to be read in the given order. A table of
contents is given at the end.
C.1 Key references
There is an excellent book available free online that will teach you the basics of
Python and some important concepts in computer science at the same time. It
is: Think Python: How to Think Like a Computer Scientist by Allen B. Downey
(http://www.greenteapress .com/thinkpython/thinkpython.html),
which I will refer to as [Downey] in the remainder of this tutorial.
C.2 How to run a Python interpreter
Two different ways of running a Python program are described. Use script mode if
you have a long list of Python commands to run (or execute) in order; otherwise
interactive mode may be more convenient.
C.2.1 Interactive mode
First, start a Python session. Here, I am using prompt> to mean whatever prompt
you happen to be on when you start a command session with Windows or OS X.
Below are two sample sessions using python and ipython in that order on a Mac
OS X in an interactive mode.
145
146 C Appendix C: Introduction of Python by A. H. Lee
prompt> python
Enthought Python Distribution -- www.enthought.com
Version: 7.1-2 (64-bit)
Python 2.7.2 |EPD 7.1-2 (64-bit)| (default, Jul 27 2011, 14:50:45)

[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "packages", "demo" or "enthought" for more information.
>>> 3 + 4
7
>>> exit()
prompt> ipython
[TerminalIPythonApp] Staging bundled ipython_config.py from default
into u’/Users/alee/.ipython/profile_default’

Type "copyright", "credits" or "license" for more information.
IPython 0.11 -- An enhanced Interactive Python.

? -> Introduction and overview of IPython’s features.
%quickref -> Quick reference.
help -> Python’s own help system.
object? -> Details about ’object’, use ’object??’ for extra details.
In [1]: 3 + 4
Out[1]: 7
In [2]: exit()
C.2.2 Script mode
You would prepare a Python script file containing a Python program and execute the
program.
First, using a text editor of your choice create a file named hello.py containing
the following as its content:
print "\nHello world!"
print "Welcome to the world of Python!"
print "We will have some fun!!!"
This script file contains only three lines of print commands. To run this script, do
the following:
C.2 How to run a Python interpreter 147
1. Open a command (terminal) window. Here again I am assuming that you get
prompt> as the prompt when you start your command window.
2. Go to the folder where hello.py is located. The cd command will be useful
as you move around the folders in your file system along with the ls command
for Mac and dir command for Windows.
3. Try the python command with the script file (hello.py in this case) as fol-
lows. The result of executing the program will be displayed.
prompt> python hello.py
Hello world!
Welcome to the world of Python!
We will have some fun!!!
prompt>
The program files that I included in this tutorial are all valid Python script files.
Try the following and see what happens:
prompt> ipython hello.py
You can run a script from ipython in addition to the usual interaction with the
interpreter. Here is a log of one such interaction:
prompt> ipython

IPython 0.11 -- An enhanced Interactive Python.

In [1]: pwd # To see where I am in the file system

Out[1]: u’/Users/alee’
In [2]: cd python_2013 # To move to the folder python_2013

/Users/alee/python_2013
In [3]: pwd
Out[3]: u’/Users/alee/python_2013’
In [4]: execfile("hello.py") # Finally to run the script

Hello world!
Welcome to the world of Python!
We will have some fun!!!
In [5]: exit()
Some of these interactions, e.g., the execfile command, are also available in
a regular interactive python session. However, pwd and cd (the usual unix com-
mands) are not available in regular python although they are available in ipython.
Whether you are using an interactive mode or a script mode, you can import other
modules into your python program. See the ”Modules” section later for detail.
C.3 Basic data types and their values
These are basic type values Python supports and some examples are given below
without much explanation. If you are not familiar with them, [Downey] is a good
source for more information.
• Integers: 34, 2, -300, etc.
• Real numbers (floating point numbers): 3.4, 2.0, -300.0, 3.14159, etc.
• Booleans (logical values): True and False.
• Strings: ’A’, ’a’, ”A”, ”a”, ”This is a string of letters”, ”Hello world!”, etc.
Strings are defined by enclosing a string of text in a pair of single or double
quotes. Python as in most languages recognizes them as a single entity (value)
that can be manipulated much like a number (integer or a floating point number)
can be, of course using some operators that make sense with strings.
Try the following with Python. The function type given an argument, say 3,
returns the type of the value, e.g., integer for 3. The integer type is represented
as <type ’int’> in Python.
>>> type (3)
<type ’int’>
Now, real numbers (also known as floating point numbers) without listing output
(try it yourself to see what output you get):
>>> type (3.14)
And, two boolean values:
>>> type (True)
>>> type (False)
And, strings this time:
>>> type ("3")
>>> type ("It is a nice day outside.")
C.5 Use of the print command in a script file 149
As you may have recognized it by now, any value the language Python, or any pro-
gramming language in general, deals with has a type associated with the value. By
using the type information a language processor (interpreter in the case of Python)
understands what it is dealing with and distinguishes one type of value from another.
That way, it would not try to do things that would not make sense, for example, try-
ing to add a number to a string. It also uses the type information to decide how much
memory to use to represent a value in memory. An integer is typically represented
using a smaller amount of memory (say 32 bits) than the amount of memory used
to represent a real number (say 64 bits). So, type information for any value we use
in a program is important for the language processor as well as for the programmers
who write application programs.
In this tutorial we will deal with only a small number of types which will still be
sufficient for the kinds of programs that we will write.
Types such as int, float, boolean are called primitive data types. Later we
will see some compound data types such as string, list, tuples, etc.
C.4 Comments
Any line starting with a pound sign (#) in your Python program is considered a
comment and will be ignored by the Python interpreter. I often add my explanation
as a comment in a program file.
C.5 Use of the print command in a script file
Create a Python script file named print.py containing the following two lines:
3 + 4 + 2 # Line 1
print 4 + 8 # Line 2
Now run that script in script mode, i.e., run it like this:
prompt> python print.py
then you will see output for only one of the two lines.
The python interpreter in script mode will evaluate each line of your program
(script) and do exactly what you asked it to do. Line 1 above tells the interpreter
to evaluate the expression (3 + 4 + 2) and do nothing else. So, it will evaluate
it and produce 9 and that is all it does. Line 2 tells the interpreter to evaluate
the expression (4 + 8) and print it to the standard output device, which is your
computer screen. Prepare your script files with a print command sprinkled here and
there if it would make the program easier to understand when you run it.
You might wonder what is going on in an interactive mode as in ipython. When
Python runs in an interactive mode, the interpreter works a little differently in that
it interprets a line and furthermore prints the result to the screen which makes sense
since you are expecting to see the result on the screen. When you are in script mode,
most likely you will write a series of lines of code and many of those lines will be
for some intermediate steps of your program. You wouldn’t want to see the value of
each line of your entire program printed on the screen. That would be too confusing.
C.6 Variables
In most programming we store a value with a name so we can refer to the value
repeatedly by the name, often updating the value if necessary. Once a name is given
a value, we can even update the value of the variable by assigning a new value, using
an assignment operator (=), as many times as we want if the logic of the program
that we write demands it. Let us see some examples next.
Establishing a variable name with a value associated with the name (again omit-
ting output values – please try them as you read):
>>> message = "What’s up?"
>>> message
Updating the value of the variable with a new value:
>>> message = "What’s down?"
>>> message
Establishing a name with a value initially and then updating the value as you com-
pute in your program is a pattern we use a lot, specially in Python. So, remember
this pattern.
The variable message here happens to have a string value at this point, but you
can even assign an integer as its value at this point in the program like this:
>>> message = 234
>>> message
But, it is usually not a good idea (not a good programming style) to mix different
types of values for a given variable unless it is logically necessary to do so in your
program. Programs are much easier to understand if you keep the variables with
values of the same type throughout the lifetime of the variables. In other languages
such as Java changing the type of a variable like this in mid-stream is not even
allowed. Here are some more variables with some values assigned.
>>> n = 22
>>> n
>>> n = 23
>>> n
>>> pi = 3.14159
>>> pi
C.7 Operating on values: operators and operands 151
We already know how to get the types. Let us try them again here.
>>> type (message)
>>> message = "back to string again"
>>> type (message)
>>> type (n)
>>> type (pi)
We can try some boolean variables, i.e., the variables with boolean values (aka log-
ical values, namely, True or False). For example,
>>> larger = (45 > 10)
>>> larger
>>> larger = (45 < 10)
>>> larger
>>> same = (34 == 34)
>>> same
>>> type(same)
You will find variables useful as you write more complex programs.
C.7 Operating on values: operators and operands
Combining simple data values you can construct an arbitrarily complex expression
using operators available in Python. I will try to give you a flavor of them here. You
can find other operators and form more complex expressions.
>>> 22 + 33 - 10
Create some variables with a value assigned to each and try some expressions:
>>> hour = 5
>>> minute = 30
>>> hour - 2
>>> hour*60 + minute
>>> (3+4)*(40/10)+100
>>> this = "Ham"
>>> that = " and cheese"
>>> more = " for breakfast!"
The + operator has a different meaning depending on what type of operands it op-
erates on. If it is dealing with numbers, it will just add as usual. If the operands are
of string type, it will concatenate the operands.
>>> this+that+more
>>> this + that + more
’==’ is the operator that tests if its two operands are equal or not. The result will
be True or False. Be careful not to use ’=’ when you actually mean ’==’. ’=’ is an
assignment operator, whereas ’==’ is an equality testing operator. What would be
the type of issame after the line below is executed?
>>> issame = this == that
If you are not sure of operator precedence between two operators, use a pair of
parentheses to indicate your intention like this. ’not’ is the negation operator.
>>> isequal = (not (this == that))
>>> issame
>>> type(issame)
>>> isequal
>>> type(isequal)
C.8 Order of precedence among operators
The rules on operator precedence in Python is the same as what you would expect
for common operators, e.g., 4 + 3 * 2 evaluates to 10, not 14. If you are not
sure, use parentheses to indicate your intention. We have also seen some operations
on strings in the previous section.
C.9 Input from keyboard
There are two built-in functions for getting keyboard input, and they are: raw input
and input. Let us try these. When you run the program below, try ’4 + 5’ (with-
out the quotes) as your input both times and see what the differences are between
raw input and input. What you type in as input will not be evaluated if you use
raw input:
>>> n = raw_input("Enter a numerical expression, e.g., 3 + 4: ")
>>> n
This one will evaluate the input and print the evaluated result:
>>> n = input("Enter a numerical expression, e.g., 3 + 4: ")
>>> n
C.10 Functions and packages (libraries)
Let us start with something that we are all familiar with. In Math, we first define and
next apply a function in 2 steps in that order like this:
C.10 Functions and packages (libraries) 153
(1) Definition: f(x) = 3xˆ2 + 2x + 1

(2) Application: f(2) = 12 + 4 + 1 = 17
f(3) = 27 + 6 + 1 = 34
Once you define a function, you can call (apply/invoke/use) that function as many
times as you wish.
In Python, the idea is the same. We only happen to be using a different language:
different syntax, same semantics (at least for now):
(1) Definition:
def f (x):
return 3*x**2 + 2*x + 1
(2) Application:
f(2)
which would compute 17 as its value.
Let’s look at it in more detail. In Python, a function must be defined using the fol-
lowing syntax:
def <name>(<list_of_parameters>):
<statements>
You can make up any name you want for the function that you create, except that you
can’t use a name that is a Python keyword. For example, def is a keyword reserved
in the language Python, and you must avoid using it as a name that you create. The
list of parameters specifies what information, if any, you have to provide in order
to use the function. There can be one or more of statements inside the function,
but they must be indented from the def. In my examples, I will use the standard
indentation of four blank spaces.
Function definitions are the first of several compound statements we will see, all
of which have the same pattern:
1. A header, which begins with a keyword and ends with a colon (:). In case you
have not noticed it, the right parenthesis, the left parenthesis, and the colon are
all required. Seemingly insignificant letters like these are part of the language
syntax and Python like any other programming language is very picky about the
exact syntax.
2. A body consisting of one or more Python statements, each indented the same
amount from the header. In a function definition, the keyword in the header is
def, which is followed by the name of the function and a list of parameters
enclosed in parentheses. The parameter list may be empty, or it may contain
any number of parameters. In either case, the parentheses are required as is the
colon (:).
Here are a few examples:
>>> def f(x):
return 3*x**2 + 2*x + 1
>>> f(1)
>>> def g(x, y, z):

return x * y + z
>>> g(2, 3, 4)
Try the above fragments of Python code in the given order.
pow for power and abs for absolute are two of the many functions that are
already defined in the language that we can use. Those language provided functions
are called built-in functions as opposed to the ones that we programmers create,
which are called user-defined functions. For example, f and g above are examples
of user-defined functions.
>>> f(pow(abs(-2), 3))
There are some libraries (packages) that need to be imported into the current module
before we can use any function that is defined in the library. math is one such library
that contains many math related functions. We import a library by the import
command like below:
>>> import math
Let us define a function named distance that computes the distance between two
points: p1(x1, y1) and p2(x2, y2):
>>> def distance(x1, y1, x2, y2):
dx = x2 - x1
dy = y2 - y1
dsquared = dx**2 + dy**2
result = math.sqrt(dsquared)
return result
>>> distance(0, 0, 3, 4)
Note that when we want to call a function defined in a library (aka package) other
than the default package, we must specify the name of the package followed by a
dot (.) followed by the function that we want to use in the package. Note that we
did not have to use any package qualifier when we used the ones like print and
def because they are included in the base Python language package, i.e., default
package. A few more example functions follow: isEven tells if a given number is
even or odd. % computes modulo and == tests to see if two values are equal or not
producing a boolean value, i.e., True or False.
>>> def isEven(x):
return ((x % 2) == 0)
>>> isEven(3)
C.11 Conditionals: doing it conditionally 155
>>> def isDivisible(x,y):

return (x % y) == 0
>>> isDivisible(30, 3)
Exercise 1. Define a function that converts a Celsius temperature value to a
Fahrenheit temperature and call it with some values such as 100, 0, -40, etc. I will
provide my answer to this exercise at the end of this file, but try it yourself before
you see mine.
Exercise 2. This time let us combine functions and input. Define a function that
reads a number from the user through keyboard input, calls the function that con-
verts the number as Celsius temperature into Fahrenheit temperature, and returns the
computed Fahrenheit temperature. After that you can run this program three times:
once each with 100, 0, and -40 as input values.
Exercise 3. Define a function with three string parameters and returns a string
that is the result of concatenating the three with a space as a separating character
between strings. Now, call the function with three strings as its actual arguments.
Exercise 4. See Section 3.15 of [Downey] if you want to see more exercise prob-
lems.
Sample solutions to Exercises 1 and 2.
def c2f(c):
return c * (9.0/5.0) + 32
def c2ftest():
n = input("Enter a number: ")
n = c2f(n)
return n
c2ftest() # use 100, 0, or -40
# Note that c2ftest may be rewritten more succinctly this way:

def c2ftest2():
return c2f(input("Enter a number: "))
c2ftest2() # use 100, 0, or -40
C.11 Conditionals: doing it conditionally
You need to understand boolean expressions to understand conditionals. If you are

not familiar with them, consult [Downey].
absolute taking one argument returns the absolute value of the given number.
To compute the absolute value, it uses a conditional expression, i.e., if...else...
Both syntax and semantics of a conditional expression is intuitive.
>>> def absolute(x):
if x < 0:
return -x
else:
return x
>>> absolute(-34)
A conditional expression can be of the cascaded if...elif... ...else...
form if you need to have multiple cases to handle, again intuitive (see examples be-
low).
Note that if you define functions with the same name multiple times, the last one
overwrites all the previous ones even if their signatures are different.
>>> def absolute(x):
if x < 0:
return -x
elif x > 0:
return x
else:
return 0
>>> absolute(-34)
>>> absolute(34)
>>> absolute(0)
Exercise 1. Write a boolean function that determines if a given speed value is
within the legal freeway speed on I-10 outside of the city limits. Once you define
the function, write a piece of code that calls the defined function and prints the
computed result (True or False).
Exercise 2. Given a score out of 100, return a letter grade for the score. Let’s
assign the letter grade using the grading scale given by the following table:
above 89: A
above 79 and below 90: B
above 69 and below 80: C
above 59 and below 70: D
below 60: F
and call the function at least five times, each time producing a different letter grade.
It is okay to produce the final grade as a string value, e.g., ”A” for an A.
A side note: In Python a function name cannot be overloaded (unlike how it is
in Java. Java supports function name overloading, but not in Python.). That is, you
cannot define a function with different parameter lists to mean different functions.
For example, if you redefine absolute with two parameters in the same session
where absolute was defined earlier with one parameter, the earlier one is lost.
So, you can call absolute with one argument okay here:
>>> absolute(-23)
C.12 Loops: doing it repeatedly 157
>>> def absolute(x, y):

return x + y
But, you cannot call absolute with one parameter here any longer, thus I com-
mented it out here.
# >>> absolute(-23)
#
# Of course, this one below would certainly be okay.
#
>>> absolute(2, 3)
C.12 Loops: doing it repeatedly
Let us see what ’while’ does in the program below. ’while’ is used to cre-
ate a while loop. That is, the body of the while loop, the two lines indented inside
the ’while’ loop in the example below, will be executed repeatedly as long as the
boolean expression (n > 0) evaluates to True. So, for this style of loop to termi-
nate eventually the value of n (called a loop control variable) must change inside the
loop body in such a way that the boolean expression is moving toward evaluating to
True.
If you call countdown with a negative number initially, what would happen?
Well, it will get into an infinite loop... sort of... it will run until the computer gets
into a corrupted state or it runs out of memory depending on how the underlying
interpreter behaves. At least you see a possible problem! If you do not update the
value of the loop control variable inside the while loop, we will end up with a similar
result. Create a file named loops.py with the following as its content and run.
# We know what import does, right?
import math
# precondition: n > 0 [the condition that must satisfy at the time the
# function is called, i.e., before it is executed.]
def countdown(n):
while n > 0:
print n
n = n - 1
print "Blastoff!"
countdown(5)
# Let us write a function that prints when called a log table.

# The character sequence ’\t’ generates a tab character.
#
def print_table():
x = 1.0
while x < 10.0:
print x, ’\t’, math.log(x)
x = x + 1.0
# An annotation as part of output (human readability is important!)

print "print_table()"
# Finally, generate the table itself by calling the function.

print_table()
# Let us improve the earlier definition of print_table by making it more

# general. We will parameterize the function by allowing the ’from’ value
# and the ’to’ value to be user definable as the function is called. This
# is one of the very important techniques in programming - generalization!
# abstraction! That is, we abstract with functions and generalize them
# when it makes sense to do so.
def print_table2(fr, to):
while fr < to:
print fr, ’\t’, math.log(fr)
fr = fr + 1.0
print "print_table2(10.0, 20.0)"
# As before generate a table, but using two values as actual arguments for
# the two parameters that we introduced in the definition.
#
print_table2(10.0, 20.0)
# What would happen if you called it this way?

#
print "print_table2(30.0, 20.0)"
print_table2(30.0, 20.0)
See how I used the print statements liberally in the code above to make it more
readable when we execute the program.
Exercise 1. Write a function that prints a table of a number, its square, and its
square root in a range defined by two numbers. For example, numbers between 10
and 20 and their squares and square roots in a table. Once the function is defined,
call it and be sure that your program generates the table correctly.
Exercise 2. Write a function that reads two numbers from keyboard and use
those two numbers as lower and upper limits of the range that you would need for
the function that you wrote in Exercise 1 above. If the input values that your function
reads in are invalid, e.g., first number is larger than the second, your program must
read a pair of numbers repeatedly until you have two valid numbers. Once you have
a good pair of numbers, then you can call the function defined in Exercise 1 above.
C.14 Strings 159
To make it a little more interesting, define a separate function that determines if

two given numbers are valid or not, and use it as part of your reader function. Yes,
you could easily do all that in a single function, but this will give you a chance to
practise with multiple functions that make up a program. In fact, as you will learn
later if you continue to write more complex programs, this is a good way to deal
with complexity in your program design.
Exercise 3. See Section 7.9 of [Downey] if you want to see more exercise prob-
lems.
C.13 Compound data types: strings, lists, tuples
So far we have mainly dealt with primitive data type values (and a little bit about
strings). That is, we have seen the following four data types: int (for integers),
float (for real numbers), bool (for booleans), and str (for strings).
Strings are compound data whereas the other three are atomic. That is, a string
is made up of smaller pieces, namely characters. To be more precise, a string is a
sequence of characters.
Our main focus so far has been learning the basic language constructs and pro-
gramming elements such as operators, conditionals, loops, functions, input, etc. us-
ing simple data types.
Using the basic language constructs that we have already learned, we can enrich
the data types. We do so by learning compound data types. The compound data types
we will learn include strings (in more detail), lists, and tuples. To write interesting
programs, we will need to know these compound data types as well. Let us do it.
C.14 Strings
A string is an ordered sequence of characters.

String is a data structure and just like integer is a data structure. Integer is a very
simple data structure whereas string is a more complex data structure. Every time
you encounter a new data structure in any language that you learn, figure out four
things:
1. How to create an instance of that data structure and give a name to it. Here is an
example of how to create a string variable with a value:
sports = "basketball"
2. How to view the value of the data structure? This is what we call read access.
Here are a few examples:
letter = sports[0] # to access the first character
sports[1] # to access the second character
3. How to change the value of the data structure, if possible? For example, if we
want to modify the contents of sports so that it holds "Baseball" instead
of "baseball", we would have to assign a new character in that location of
the sports string by using an assignment operator like this:
sports[0] = ’B’
but modifying an element in place in string is not allowed in Python. So, we call
string an immutable data structure in Python. The best you can do is to create a
new string with a ’B’ in the first location and reassign the whole string to be
the new value of the variable sports. When we study lists later, we will be
able to modify an element in a list in place. Stay tuned. Changing the value of a
variable is called write access, and there are two forms of write access we can
think of when we deal with a string.
a. Changing a character in the string, e.g., by doing something like:
sports[0] = ’B’
but that is not allowed with a string because string is immutable in Python.
b. Setting a new string to be the value of the variable, e.g.,
sports = Baseball
which is allowed in Python.

4. How to destroy the data structure when we are done using it. In a language such
as C or C++ you would have to worry about that, but not in Python.
So, there are only three things to worry about when you encounter a new data
structure in Python. Sometimes, only two as with strings. I will tell you which
ones to worry about each time a new data structure is introduced below.
Let us see some strings and related functions that we can use to manipulate
strings. Define a variable with the string value ’banana" as its initial value:
>>> fruit = "banana"
String is a compound value and we can access individual pieces in it. Here we are
accessing the second character in fruit.
>>> letter = fruit[1]
>>> letter
Indices in Python are 0-based as in most other programming languages. So, fruit[1]
is accessing the second character in the string.
>>> fruit[0]
C.14 Strings 161
Length of a string is available too.

>>> len(fruit)
More looping using strings would be instructive. Let us print fruit using a while
loop:
>>> i = 0
>>> while i < len(fruit):
print fruit[i]
i = i + 1
Exercise: Create a function that takes one string argument and prints the letters
backward.
C.14.1 The for loop
We have learned one kind of a loop in Python so far: the while loop. Well,
there is another looping construct in Python: the for loop. As you can see be-
low, it has a very simple intuitive form. This loop is often referred to as the
for-each loop because that is what it really means. So, the example below can
be read as: ’for each letter referred to as c in the string
fruit, print the character c’, thus printing each letter in the string
to the standard output device, i.e., the computer screen. Remember that the body of
a for loop as with a body of a while loop or a function needs to be indented.
>>> for c in fruit:
print c
Here is a simple yet fun thing to do. I think you can figure out what this does:
>>> prefixes = "JKLMNOPQ"
>>> suffix = "ack"
>>> for letter in prefixes:
print letter + suffix
Slicing a string: by using the ’:’ operator we can specify a range within the string.
>>> s = "Peter, Paul, and Mary"
>>> s[0:5] # 0 to 5, but 0 inclusive, 5 exclusive
>>> s[7:11]
>>> s[17:21]
>>> s[:3] # 0 to 3, but 0 inclusive, 3 exclusive
>>> s[17:] # 17 to the end, with 17 inclusive
Comparing strings: we can use the usual comparison operators that we use for num-
bers to compare two strings.
word = "apple"
if word == "banana":
print "Yes, we have a banana!"
else:
print "No, we don’t have a banana!"
if word < "banana":

print "Your word, " + word + ", comes before banana."
elif word > "banana":
print "Your word, " + word + ", comes after banana."
else:
print "Yes, we have a banana!"
Finding a letter in a string: we can define a function that does that like this:
def find(str, ch):
index = 0
while index < len(str):
if str[index] == ch:
return index
index = index + 1
return -1
find("apple", ’l’)
find("apple", ’p’)
Counting: count the occurrences of a letter in a string:
def count(str, ch):
count = 0
for char in str:
if char == ch:
count = count + 1
return count
count("apple", ’p’)
There is a built-in string module that we can import:
import string
To see what things are included in the module:
dir(string)
Two of those are digits and find:
string.digits
string.find
C.15 Lists 163
If you want to find the documentation on find, try this. Similarly you can find
documentation on other functions as well.
string.find.__doc__
Based on the information obtained from the documentation, we can now call find:
string.find("banana", "nan")
C.15 Lists
Remember that a string is an ordered sequence of characters? Well, a list is an or-

dered sequence of values of any type. Both strings and lists and other things that
behave like an ordered sequence are called sequences.
Since a list is a data structure, think about four things about a new data structure:
(1) How do you create one? (2) How do you access an element in it? (3) How do
you modify an element in it in place if possible? (4) How do you destroy it when we
don’t need it any more? (don’t worry about destroying one in Python)
Let us see some lists and related functions that we can use to manipulate lists.
Also think about where lists would be useful as you write your programs.
Let us create a list on the fly and print it to the standard output device:
>>> [10, 20, 30, 40, 50]
Once more with different type of elements:
>>> ["Let", "us", "stop", "spams!"]
Once more with an element itself as a list:
>>> ["hello", 2.0, 5, ["nested", 2, True]]
Here is another way of creating a list:
>>> range(1, 7) # 1 inclusive, 7 exclusive
From 0 to 9 inclusive this time:
>>> range(10)
How about 1 to 10 in 2 or 3 increments?:
>>> range(1, 10, 2)
>>> range(1, 10, 3)
Let us see how we access elements in a list. First create a list and make sure it is
indeed created:
>>> numbers = [10, 20, 30, 40, 50]
>>> numbers
Remember that strings are immutable? Well, lists are mutable, which means that
we can modify an element in a list in place. Let us try it and see if indeed the
modification was done correctly in place.
>>> numbers[2] = 3
>>> numbers
Use a for loop as we did with a string:
>>> for n in numbers:
print n
Concatenating two lists together:
>>> a = [1, 2, 3]
>>> b = [3, 4, 5]
>>> a + b # note that neither a nor b has been changed.
>>> c = a + b
>>> c # note that neither a nor b has been changed. c is new.
What would you expect this to do? Well, verify your answer by running it.
>>> [0] * 4
How about this one? Again run it to see what it does.
>>> [1, 2, 3] * 3
Slicing lists, much like we did with strings:
>>> alist = [’a’, ’b’, ’c’, ’d’, ’e’, ’f’]
>>> alist[1:3] # 1 inclusive, 3 exclusive
>>> alist[:4] # 0 to 3 inclusive
>>> alist[3:] # 3 and the rest
>>> alist[:] # everything
Deleting elements of a list in place:
>>> del alist[2] # This is a destructive operation
>>> alist
>>> del alist[2:4] # 2 to 3 inclusive
>>> alist
Inserting elements into a list: this is useful enough that we create a function for it:
def insert(u, v, i):
u[i:i] = v
numbers
insert(numbers, [111], 1)
numbers
insert(numbers, [22, 33, 44], 2)

C.15 Lists 165
numbers
insert(numbers, [[222, 333, 444]], 2)

numbers
insert(numbers, range(1000, 2000, 500), 2)

numbers
def sum(u):
sum = 0
for e in u:
sum = sum + e
return sum
def ave(u):
return sum(u) / len(u)
def makeList(u):
return [u, ave(u)]
insert(numbers, [makeList(range(1, 5))], 2)

numbers
Converting strings to lists: sometimes it is useful to convert a string into a list and
manipulate it as a list:
s1 = "Stop that!"
list(s1)
Some basic functions dealing with lists:
def del_head(u):
del u[0]
aa = [11, 22, 33, 44, 55]

del_head(aa)
aa
def tail(u):
return u[1:]
tail(aa)
aa
x = range(1, 11)
x
What does this do? For each element in x double it and include
them all in a list - neat, huh?
y = [j * 2 for j in x]
y
Matrices can be represented as lists in Python. For example,
# | 1 2 3 |
# | 4 5 6 |
# | 7 8 9 |
m = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
m
m[1]
m[1] = [40, 50, 60]
m
m[1][2]
m[1][2] = 300
m[1][2]
We can print the 3 rows in order like this:
for r in m:
print r
We can print the 9 elements in order like this:
for r in m:
for e in r:
print e
This is a rather inefficient way to deal with matrices in Python. To deal with matrices
more efficiently, learn how to use Numpy arrays.
C.16 Modules
Tuples and dictionaries are also compound data types and we will study them, but
let us study modules first before we continue with them.
Namespace is a concept used in programming languages. As we create functions,
thus function names, we may want to create the same function name to mean differ-
ent things in different contexts. For example, count may mean counting students
or beans depending on which program you are dealing with. Rather than naming
them studentCount and beanCount we may want to name them both count,
but in different contexts. We create a namespace in which names are contained. So
C.16 Modules 167
if we have a namespace that contains all the names that deal with students and an-
other containing all the names that deal with beans, then we can use the same name,
e.g., count, as long as we distinguish one from the other by using the name of the
namespace it belongs to. Namespace is used to manage names in a programming
language. After all, a program consists of a bunch of names such as variable names,
function names, etc. When there are many names, it is easier to deal with them if
we partition them into different spaces. Python like many other languages supports
this mechanism and it is implemented by modules.
A module is a file containing Python definitions that are intended to be used in
other Python programs. There are many Python modules that come with the lan-
guage Python as part of the standard library. We have seen two of those already: the
string module and the math module.
Let us learn how to create a module of our own now. All we need to create a
module is a Python program file with a .py extension in the filename. For example,
functions.py that I created earlier is one such module. Let us create one as
follows: we will create a file named seqtools.py that contains two definitions
named remove at and insert at. The file would contain the following:
# seqtools.py
def remove_at(seq, pos):

return seq[:pos] + seq[pos+1:]
def insert_at(seq, pos, item):

return seq[:pos] + item + seq[pos:]
Now, that we created a module seqtools.py, let us learn how to use it. Start
a Python session in an interactive mode, and try the following.
>>> from seqtools import remove_at
>>> s = "This is fun!"
>>> remove_at(s, 3)
’Thi is fun!’
As you can see, we are using the ’from . . . import’ command to import a
specific name (remove at) from a module (seqtools). We have imported only
one definition out of two in seqtools.py.
In this particular situation we imported the name (remove at) into the current
module, which is the top-level, interactive module. Once you import it, you can use
the imported name as if the name was created in the current module that you are in.
One bad thing about the way it was just done is the following: if you already had
the same name in the current module, the old one will be lost. To avoid this situa-
tion, there is another way of importing. Here again I am showing you an interactive
session using an interactive mode to illustrate how it is done, like this:
>>> import seqtools
>>> s = "This is fun!"
>>> seqtools.remove_at(s, 3)
’Thi is fun!’
As you can see here we are importing everything from the module seqtools. Note
that we don’t say ’import seqtools.py’, but rather ’import seqtools’
without the file extension (.py). Since there are two definitions in seqtools, both
are imported. If we import it this way, we must specify the module name along with
the name in the module if we want to use a name in the imported module.
We use the dot (’.’) operator to separate the module name and a name in the
module, e.g., seqtools.insert at. Suppose there was already a definition
named insert at in the current module. Then, insert at alone would refer
to the one in the current module, and seqtools.insert at would refer to the
one from seqtools module.
Note that this works because we have the file seqtools.py located in the
same folder where the current python session was started at. If the module to be
imported is located in a different folder, we would have to do a little more work. I
will explain that a little later. Some programming languages use packages to mean
modules. I may use that term inadvertently in the future, and you will know what I
mean if I do.
The ’from . . . import’ version also has a wildcard option: We can use
’from modulename import *’ to mean import everything from the named
module, e.g.,
>>> from seqtools import *
This would import all the names from seqtools into the current module thus
overwriting any existing names if there are names that are the same in the current
module as the ones being imported from seqtools.
The dir(. . . ) function is useful. Once you import a module, you can use the
dir function to see all the functions defined in the module. For example, try the
following:
>>> import math
>>> dir(math)
Getting back to the case where the module that you want to import is located
elsewhere, what would we have to do if the module to be imported is located in
a different folder than the folder where the current python session was started at?
Well, if the module file is located in one of the folders that is included in the current
’path’ for the python session, then you don’t need to do anything. If not, we will
have do some more work. So, how do we find out if the folder where the module to
be imported is located is in the path or not? Try this in your current Python session:
>>> import sys
>>> sys.path
The value of sys.path is returned and it will show all the folders that are included
in the path. If the folder where the module is located is in the the path, you are done.
If not, we have to include that folder in the path. Assuming that the module to be
C.16 Modules 169
imported is located in the folder: /Users/alee/python, this is what we would

do:
>>> import os
>>> os.path.join ## Just to see if join is available
>>> <function join at 0x00A12F70>
>>> sys.path ## Tried on a Mac machine (Unix)
## see init.py for Windows version
[’’,
’/Library/Frameworks/Python.framework/Versions/5.1.1/bin’,
’/Library/Frameworks/Python.framework/Versions/5.1.1/lib/python25.zip’,
(some omitted)
u’/Users/alee/.ipython’]
>>> sys.path.append(os.path.join("/Users", "alee", "python"))

>>> sys.path
[’’,
’/Library/Frameworks/Python.framework/Versions/5.1.1/bin’,
’/Library/Frameworks/Python.framework/Versions/5.1.1/lib/python25.zip’,
(some omitted)
u’/Users/alee/.ipython’,
’/Users/alee/python’]
Now, you can see that /Users/alee/python is included in the path, and you
should be able to import a module located in that folder, as you can see below:
>>> import seqtools
>>> s = "Thisisfun..."
>>> seqtools.remove_at(s, 3)
’Thiisfun...’
Note that remove at(s, 3) without seqtools. would not work. However,
if you imported it using ’from seqtools import *’ then remove at(s,
3) alone would work.
We can encapsulate all this stuff that I just did into a file named init.py so that
we can use it to add something into the path. You may then use it every time you
want to do something like this. If you are using Windows, create init.py with the
following as its contents. Of course, you will have to use the appropriate names that
are meaningful for your situation. (The one I used above works for Mac OS X.):
import sys
import os
sys.path.append(os.path.join("C:\\", "alee", "python"))
With init.py available, just do the following right after you start a new Python
session:
>>> execfile("init.py")
and then continue with whatever you wanted to do in your current Python session.
C.17 Tuples
Remember that a string is an ordered sequence of characters? Also remember that a

list is an ordered sequence of values of any type? We said both strings and lists and
other things that behave like ordered sequences are called sequences. Also remem-
ber that strings are immutable and lists are mutable?
A tuple, like a list, is a sequence of items of any type. Unlike lists, however, tuples
are immutable. The values of a tuple are indexed by integers as with the values in a
list or string. As you will see below, sequences (strings, lists, and tuples) share some
common operators.
Since a tuple is a data structure, think about four things you want to know about
a new data structure: (1) How do you create one? (2) How do you access an element
in it? (3) How do you modify an element in it in place if possible? Note that tuples
are immutable! (4) How do you destroy it when we don’t need it any more? (don’t
worry about destroying one).
Let us see some tuples and related functions that we can use to manipulate tuples.
Also think about where tuples would be useful in programming. Syntactically, a
tuple is a comma-separated sequence of values.
a = 2, 3, 4, 6, 7, 10
a # (2, 3, 4, 6, 7, 10)
Although it is not necessary, it is conventional to enclose tuples in parentheses:
b = (20, 30, 40, 60, 70, 100)
b # (20, 30, 40, 60, 70, 100)
To create a tuple with a single element, we have to include the final comma. Without
the comma, Python treats (5) as an integer in parentheses.
tup = (22,)
type(tup) # <type ’tuple>
Another way to create a tuple is the built-in function tuple. With no argument, it
creates an empty tuple:
t = tuple()
t # ()
If the argument is a sequence (string, list, or tuple), the result is a tuple with the
elements of the sequence:
t = tuple(’apple’)
t # (’a’, ’p’, ’p’, ’l’, ’e’)
Most list operators work on tuples. The bracket operator indexes an element:
tup = (’a’, ’b’, ’c’, ’d’, ’e’)
tup[0] # ’a’
And, the slice operator selects a range of elements:
C.17 Tuples 171
tup[1:3] # (’b’, ’c’)

As expected you can’t modify an element in place–immutable!
# tup[2] = ’K’ # this would produce an error. What kind of error? Try it!
Although we can’t replace an element in place, we can rebind the variable with a
new tuple:
tup = (’a’, ’b’, ’K’, ’d’, ’e’)
Tuple assignments: It is often useful to swap the values of two variables. Tuple
assignment is convenient. To swap the values of a and b we can do the following:
a
b
a, b = b, a
a
b
The left side is a tuple of variables; the right side is a tuple of expressions. Each
value is assigned to its corresponding variable. All the expressions on the right side
are evaluated before any of the assignments. The number of variables on the left and
the number of values on the right have to be the same. More generally, the right side
can be any kind of sequence (string, list, or tuple). For example, to split an email
address into a user name and a domain, you could write:
addr = ’monty@python.org’
uname, domain = addr.split(’@’)
Tuple as return values: Functions can return tuples as return values. For exam-
ple, we could write a function that swap two parameters like this:
def swap(x, y):
return y, x
Then, do the following to swap:
a, b = swap(a, b)
What would this definition of swap (called swapbad) do?
def swapbad(x, y):
x, y = y, x
If we call this function like this:
swapbad(a, b)
then changing x inside swapbad makes x refer to a different value, but it has no
effect on a in the main frame. Similarly, changing y has no effect on b.
This would do the right thing, but a function like this would be redundant. I am
using this example to illustrate local variables and their relationship with variables
outside a function, i.e., global variables.
def swapgood(x, y):

x, y = y, x
return x, y
a, b = swapgood(a, b)
Lists and tuples: zip is a built-in function that takes two or more sequences
and zips them into a list of tuples where each tuple contains one element from each
sequence. This example zips a string and a list:
s = ’abc’
t = [0, 1, 2]
zip(s,t) # [(’a’, 0), (’b’, 1), (’c’, 2)]
You can use tuple assignment in a for loop to traverse a list of tuples:
t = [(’a’, 0), (’b’, 1), (’c’, 2)]
for letter, number in t:
print number, letter
Each time through the loop, Python selects the next tuple in the list and assigns the
elements to letter and number. Try it to see what output you get if it is not obvious.
If you combine zip, for, and tuple assignment, you get a useful idiom for
traversing two (or more) sequences at the same time. For example, has match
takes two sequences, t1 and t2, and returns True if there is an index i such that
t1[i] == t2[i]:
def has_match(t1, t2):
for x, y in zip(t1, t2):
if x == y:
return True
return False
If you need to traverse the elements of a sequence and their indices, you can use the
built-in function enumerate. Try the following:
for index, element in enumerate(’abc’):
print index, element
C.18 Dictionaries
All of the compound data types that we have studied so far–strings, lists, and tuples–
are sequence types, which use integers as indices to access the values that are con-
tained in them. If you try to use any other type as an index, you get an error.
A dictionary is also a compound data type. It is Python’s built-in mapping type.
It maps keys, which can be any immutable type, to values, which can be any type,
just like the values of a list or tuple.
C.18 Dictionaries 173
Abstractly and conceptually, a dictionary is a sequence of key-value pairs. For

example, a phone book is a dictionary: name-number pairs, i.e., mapping of names
to phone numbers.
Remember we should know how to (1) create a new instance of this type, (2)
do write access, (3) do read access, and (4) destroy it when you are done using it?
Well, we will do (1), (2), and (3) by writing an example that maps English words to
Spanish equivalents. For this dictionary, the indices are strings. As usual we won’t
worry about destroying one since it will be done automatically by the Python sys-
tem when you stop using the data structure that you created. It is called garbage
collection. We are talking about destroying the entire dictionary instance itself, not
an entry in the dictionary. For individual entries, we will be able to remove a value
from a dictionary which will then be destroyed by the language system if it is not
shared by any other data structure.
One way to create a dictionary is to start with an empty dictionary and start
adding elements, i.e., pairs. Here is how you do it. I am creating a dictionary named
’e2s’ (English to Spanish) initialized with an empty dictionary.
e2s = {}
This is how you put a key-value pair, i.e., ’one’ as the key and ’uno’ as the value
of the pair:
e2s[’one’] = ’uno’
One more with the two-dos pair:
e2s[’two’] = ’dos’
This is how you retrieve the value associated with a key:
e2s[’one’] # Just to do a read access
e2s[’one’] # If you want it to be printed on the screen
This will display the contents of the entire dictionary so far:
e2s
The elements of a dictionary appear in a comma-separated list as you can see in the
printed result above–you are using an interpreter to execute these lines, aren’t you?
Each entry (pair) contains an index and a value separated by a colon. In a dictio-
nary, the indices are called keys, so the elements are called key-value pairs. Dictio-
naries are also called association lists, the original term used in the Lisp program-
ming language for the same concept.
You can create one dictionary with some initial pairs explicitly like this:
e2s = {’one’: ’uno’, ’two’: ’dos’, ’three’: ’tres’}
Order of the pairs in a dictionary is not important. You can use a list of tuples to
initialize a new dictionary:
t = [(’a’, 0), (’b’, 1), (’c’, 2)]
d = dict(t)
d # {’a’: 0, ’b’: 1, ’c’: 1}
Combining dict and zip yields a concise way to create a dictionary:

d = dict(zip(’abc’, range(3)))
d # {’a’: 0, ’c’: 2, ’b’: 1}
Here is another dictionary:
inventory = {’apples’: 430, ’bananas’: 312, ’oranges’: 525, ’pears’: 217}
inventory
Deleting an entry (pair) from a dictionary:
del inventory[’pears’]
inventory
Updating the value of a pair:
inventory[’apples’] = inventory[’apples’] + 10
inventory
Updating the value of a pair once more:
inventory[’oranges’] = 0
inventory
Note: index (key) must be unique in a dictionary, i.e., no two keys can be identical
although values do not need to be unique. This is like a function in math! Length of
a dictionary:
len(inventory)
Retrieve all the keys in a dictionary into a list:
e2s.keys()
Retrieve all the values in a dictionary into a list:
e2s.values()
Retrieve all the pairs in a dictionary into a list:
e2s.items()
Check if a key is contained in a dictionary
e2s.has_key(’one’)
e2s.has_key(’deux’)
If you want to iterate through the pairs and add 10 to each:
for key in inventory.keys():
inventory[key] = inventory[key] + 10
inventory.values()
If you want to iterate through the pairs and add 10 to each this way this time for
key in e2s.keys():
C.18 Dictionaries 175
e2s[key] = e2s[key] + str(10)

e2s.values()
Combining items, tuple assignment, and for, you get the idiom for traversing the
keys and values of a dictionary. Try the following:
for key, val in e2s.items():
print val, key
It is common to use tuples as keys in dictionaries (primarily because you can’t use
lists). For example, pbook (a phone book dictionary) might map from last-name,
first-name pairs to phone numbers.
first = ’john’
last = ’doe’
number = 9096070410
pbook = {}
pbook[last,first] = number
pbook[’jones’,’amy’] = 2133223333
The expression in the brackets is a tuple. We could use tuple assignment to traverse
this pbook.
for last, first in pbook:
print first, last, pbook[last,first]
This loop traverses the keys in pbook, which are tuples. It assigns the elements of
each tuple to last and first, then prints the name and the corresponding phone
number.
Using an alias vs. a copy: Let us create another map first:
opposites = {’up’: ’down’, ’right’: ’wrong’, ’true’: ’false’}
This assignment creates an alias where both names reference the same identical
dictionary object.
alias = opposites
The ’copy’ command below creates a distinct copy of the original. There are two
copies now with different names: one called opposites and another called cp.
cp = opposites.copy()
Exercise 1. We can write a function that counts the number of occurrences of
a letter in a string. Or, a more general version of this problem would be to form a
histogram of the letters in the string, that is, how many times each letter appears.
Such a histogram might be useful for compressing a text file. Because different
letters appear with different frequencies, we can compress a file by using shorter
codes for common letters and longer codes for letters that appear less frequently.
Dictionaries provide an elegant way to generate a histogram. Here is a simple
program that would do that:
>>> letter_counts = {}
>>> for letter in "Mississippi":
... letter_counts[letter] = letter_counts.get(letter, 0) + 1
...
>>> letter_counts
{’M’: 1, ’s’: 4, ’p’: 2, ’i’: 4}
We start with an empty dictionary. For each letter in the string, we find the current
count (possibly zero) and increment it. At the end, the dictionary contains pairs of
letters and their frequencies. It might be more appealing to display the histogram in
alphabetical order. We can do that with the items and sort methods:
>>> letter_items = letter_counts.items()
>>> letter_items.sort()
>>> letter_items
[(’M’, 1), (’i’, 4), (’p’, 2), (’s’, 4)]
You have seen the items method before, but sort is one of the first methods you
have encountered that applies to lists. There are several other list methods, including
append, extend, and reverse. Consult the Python documentation for details
on these methods and some additional ones.
C.19 Files
We have now seen enough basic data structures that you will need in Python to write
some interesting programs. We will now study how to deal with data in a file.
While a program is running, its data is in memory. When the program ends, or
the computer shuts down, data in memory disappears. One of the easiest ways of
storing the data permanently is to save it in a file. Files are usually stored on a hard
drive, USB drive, CD, etc.
When there are a large number of files, they are often organized into directories
(also called folders) in the file system of your computer. Each file is identified by a
unique name, or a combination of a file name and a directory name.
Working with files is a lot like working with notebooks. To use a notebook, you
have to open it. When you are done, you have to close it. While the notebook is
open, you can either write in it or read from it. In either case, you know where you
are in the notebook. Most of the time, you read the whole notebook in its natural
order, but you can also skip around.
All of this applies to files as well. To open a file, you specify its name and indi-
cate whether you want to read or write. Opening a file creates a file object. In this
example, the variable f refers to the new file object.
f = open("test.dat", "w")
f
# <open file ’test.dat’, mode ’w’ at 0x2aaaaab80cd8>
C.19 Files 177
The open function takes two arguments: the first is the name of the file, and the
second is the mode. Mode "w" means that we are opening the file for writing. If
there is no file named test.dat, it will be created when you are accessing it with
the ’w’ mode. If there already is one, it will be replaced by the file we are writing.
When we print the file object, we see the name of the file, the mode, and the
location of the object. To put data in the file we invoke the write method on the
file object:
f.write("Now is the time")
f.write("to close the file")
Closing the file tells the system that we are done writing and makes the file available
for reading:
f.close()
Now, we can open the file again, this time for reading, and read the contents into a
string. This time, the mode argument is "r" for reading:
f = open("test.dat", "r")
If we try to open a file with the read mode that does not exist, we get an error:
#f = open("test.cat","r")
# IOError: [Errno 2] No such file or directory: ’test.cat’
f = open("test.dat","r")
Not surprisingly, the read method reads data from the file. With no arguments, it
reads the entire contents of the file:
text = f.read()
text
Now is the timeto close the file
read can also take an argument that indicates how many characters to read:
f = open("test.dat","r")
f.read(5)
Now i
If not enough characters are left in the file, read returns the remaining characters.
When we get to the end of the file, read returns the empty string:
f.read(1000006)
s the timeto close the file
f.read()
<Nothing this time>
Writing a function that copies a file: It reads and writes up to fifty characters at a
time. The first argument is the name of the original file; the second is the name of
the new file:
def copyFile(oldFile, newFile):

f1 = open(oldFile, "r")
f2 = open(newFile, "w")
while True: # infinite loop
text = f1.read(50)
if text == "":
break # break out of the current loop
f2.write(text)
f1.close()
f2.close()
return
The break statement is new. Executing it breaks out of the current loop; the flow
of execution moves to the first statement after the loop. In this example, the while
loop is infinite because the value True is always true. The only way to get out of
the loop is to execute break, which happens when text is the empty string, which
happens when we get to the end of the file.
C.19.1 Text files
A text file is a file that contains printable characters and whitespace, organized into
lines separated by newline characters. Since Python is specifically designed to pro-
cess text files, it provides methods that make the job easy.
To demonstrate, we will create a text file with three lines of text separated by
newlines:
f = open("test1.dat", "w")
f.write("line one\nline two\nline three\n")
f.close()
The readline() method reads all the characters up to and including the next
newline character (’\n’):
f = open("test1.dat", "r")
f.readline()
readlines() returns all of the remaining lines as a list of strings:
f.readlines()
In this case, the output is in a list format, which means that the strings appear with
quotation marks and the newline character appears as the escape sequence \n.
At the end of the file, readline returns the empty string and readlines
returns the empty list:
>>> f.readline()
C.19 Files 179
>>> f.readlines()
[]
Exercise 1: The following is an example of a line-processing program. filterFile
makes a copy of oldFile, omitting any lines that begin with a #:
def filterFile(oldFile, newFile):
f1 = open(oldFile, "r")
f2 = open(newFile, "w")
while True:
text = f1.readline()
if text == "":
break
if text[0] == ’#’:
continue
f2.write(text)
f1.close()
f2.close()
return
The continue statement ends the current iteration of the loop, but continues loop-
ing. The flow of execution moves to the top of the loop, checks the condition, and
proceeds accordingly. Thus, if text is the empty string, the loop exits. If the first
character of text is a hash mark, the flow of execution goes to the top of the loop.
Only if both conditions fail do we copy text into the new file.
C.19.2 Exception handling
Whenever a runtime error occurs, it creates an exception. Usually, the program stops
and Python prints an error message. For example, dividing by zero creates an ex-
ception:
>>> 55/0
ZeroDivisionError: integer division or modulo
So does accessing a nonexistent list item:
>>> a = []
>>> a[5]
IndexError: list index out of range
Or, accessing a key that isn’t in the dictionary:
>>> b = {}
>>> b[’what’]
KeyError: what
In each case, the error message has two parts: (1) the type of error before the colon,
and (2) specifics about the error after the colon. Normally, Python also prints a
traceback of where the program was at the time of error, but we have omitted that
from the examples.
Sometimes, we want to execute an operation that could cause an exception, but
we don’t want the program to stop. We can handle the exception using the try and
except statements. For example, we might prompt the user for the name of a file
and then try to open it. If the file doesn’t exist, we don’t want the program to crash;
we want to handle the exception. Here is one way to handle the situation:
filename = raw_input(’Enter a file name: ’)
try:
f = open (filename, "r")
except:
print ’There is no file named’, filename
The try statement executes the statements in the first block. If no exceptions occur,
it ignores the except statement. If any exception occurs, it executes the statements
in the except branch and then continues.
We can encapsulate this capability in a function: exists takes a filename and
returns True if the file exists, False if it doesn’t:
def exists(filename):
try:
f = open(filename)
f.close()
return True
except:
return False
C.20 More Python
The online tutorial that I wrote can be found at http://www.cmc.edu/pages/faculty/alee/

python/. This tutorial is a subset of the online version which covers some addi-
tional topics: classes, objects, and arrays.
C.21 NumPy, SciPy, etc.
The online tutorial at http://www.cmc.edu/pages/faculty/alee/python/

also discusses these topics briefly, which are useful for scientific computing.
You can find additional information on these topics at http://fperez.org/py4science/
starter kit.html.
Appendix D
Appendix D: The Scientific Python Ecosystem by
F. Perez
While the Python language is an excellent tool for general-purpose programming,

with a highly readable syntax, rich and powerful data types (strings, lists, sets, dic-
tionaries, arbitrary length integers, etc) and a very comprehensive standard library,
it was not designed specifically for mathematical and scientific computing. Neither
the language nor its standard library have facilities for the efficient representation
of multidimensional datasets, tools for linear algebra and general matrix manipula-
tions (an essential building block of virtually all technical computing), nor any data
visualization facilities.
In particular, Python lists are very flexible containers that can be nested arbitrarily
deep and which can hold any Python object in them, but they are poorly suited to
represent efficiently common mathematical constructs like vectors and matrices. In
contrast, much of our modern heritage of scientific computing has been built on top
of libraries written in the Fortran language, which has native support for vectors and
matrices as well as a library of mathematical functions that can efficiently operate
on entire arrays at once.
D.0.1 Scientific Python: a collaboration of projects built by

scientists
The scientific community has developed a set of related Python libraries that provide
powerful array facilities, linear algebra, numerical algorithms, data visualization and
more. In this appendix, we will briefly outline the tools most frequently used for
this purpose, that make “Scientific Python” something far more powerful than the
Python language alone.
For reasons of space, we can only describe in some detail the central Numpy
library, but below we provide links to the websites of each project where you can
read their documentation in more detail.
First, let’s look at an overview of the basic tools that most scientists use in daily
research with Python. The core of this ecosystem is composed of:
181
182 D Appendix D: The Scientific Python Ecosystem by F. Perez
• Numpy: the basic library that most others depend on, it provides a powerful ar-
ray type that can represent multidmensional datasets of many different kinds and
that supports arithmetic operations. Numpy also provides a library of common
mathematical functions, basic linear algebra, random number generation and
Fast Fourier Transforms. Numpy can be found at http://numpy.scipy.
orgnumpy.scipy.org
• Scipy: a large collection of numerical algorithms that operate on numpy arrays
and provide facilities for many common tasks in scientific computing, including
dense and sparse linear algebra support, optimization, special functions, statis-
tics, n-dimensional image processing, signal processing and more. Scipy can be
found at http://scipy.orgscipy.org.
• Matplotlib: a data visualization library with a strong focus on producing
high-quality output, it supports a variety of common scientific plot types in
two and three dimensions, with precise control over the final output and for-
mat for publication-quality results. Matplotlib can also be controlled interac-
tively allowing graphical manipulation of your data (zooming, panning, etc)
and can be used with most modern user interface toolkits. It can be found at
http://matplotlib.sf.netmatplotlib.sf.net.
• IPython: while not strictly scientific in nature, IPython is the interactive envi-
ronment in which many scientists spend their time. IPython provides a powerful
Python shell that integrates tightly with Matplotlib and with easy access to the
files and operating system, and which can execute in a terminal or in a graphical
Qt console. IPython also has a web-based notebook interface that can combine
code with text, mathematical expressions, figures and multimedia. It can be found
at http://ipython.orgipython.org.
While each of these tools can be installed separately, in our opinion the most con-
venient way today of accessing them (especially on Windows and Mac computers)
is to install the
http://www.enthought.com/products/epd\_free.php
Free Edition of the Enthought Python Distribution which contain all the above.
Other free alternatives on Windows (but not on Macs) are
http://code.google.com/p/pythonxy}{Python(x,y)
and the
http://www.lfd.uci.edu/\ensuremath˜gohlke/pythonlibs
packages developed by Christoph Gohlke.
These four ‘core’ libraries are in practice complemented by a number of other
tools for more specialized work. We will briefly list here the ones that we think are
the most commonly needed:
• Sympy: a symbolic manipulation tool that turns a Python session into a com-
puter algebra system. It integrates with the IPython notebook, rendering results
in properly typeset mathematical notation. http://sympy.orgsympy.org.
D Appendix D: The Scientific Python Ecosystem by F. Perez 183
• Mayavi: sophisticated 3d data visualization; http://code.enthought.

com/projects/mayavicode.enthought.com/projects/mayavi.
• Cython: a bridge language between Python and C, useful both to optimize
performance bottlenecks in Python and to access C libraries directly; http:
//cython.orgcython.org.
• Pandas: high-performance data structures and data analysis tools, with power-
ful data alignment and structural manipulation capabilities; http://pandas.
pydata.orgpandas.pydata.org.
• Statsmodels: statistical data exploration and model estimation; http://
statsmodels.sourceforge.netstatsmodels.sourceforge.net.
• Scikit-learn: general purpose machine learning algorithms with a common
interface; http://scikit-learn.orgscikit-learn.org.
• Scikits-image: image processing toolbox; http://scikits-image.
orgscikits-image.org.
• NetworkX: analysis of complex networks (in the graph theoretical sense);
http://networkx.lanl.govnetworkx.lanl.gov.
• PyTables: management of hierarchical datasets using the industry-standard HDF5
format; http://www.pytables.orgwww.pytables.org.
Beyond these, for any specific problem you should look on the internet first, before
starting to write code from scratch. There’s a good chance that someone, some-
where, has written an open source library that you can use for part or all of your
problem.
D.0.2 A note about the examples below
In all subsequent examples, you will see blocks of input code, followed by the re-
sults of the code if the code generated output. This output may include text, graphics
and other result objects. These blocks of input can be pasted into your interactive
IPython session or notebook for you to execute. In the print version of this docu-
ment, a thin vertical bar on the left of the blocks of input and output shows which
blocks go together.
If you are reading this text as an actual IPython notebook, you can press
Shift-Enter or use the ‘play’ button on the toolbar (right-pointing triangle) to
execute each block of code, known as a ‘cell’ in IPython:
# This is a block of code, below you’ll see its output

print "Welcome to the world of scientific computing with Python!"
Welcome to the world of scientific computing with Python!

D.1 Motivation: the trapezoidal rule
In subsequent sections we’ll provide a basic introduction to the nuts and bolts of
the basic scientific python tools; but we’ll first motivate it with a brief example that
illustrates what you can do in a few lines with these tools. For this, we will use the
simple problem of approximating a definite integral with the trapezoid rule:
1 N
Z b
f (x) dx ≈ ∑ (xk − xk−1 ) ( f (xk ) + f (xk−1 )) .
a 2 k=1
Our task will be to compute this formula for a function such as:
f (x) = (x − 3)(x − 5)(x − 7) + 85

integrated between a = 1 and b = 9.
First, we define the function and sample it evenly between 0 and 10 at 200 points:
def f(x):
return (x-3)*(x-5)*(x-7)+85
import numpy as np
x = np.linspace(0, 10, 200)
y = f(x)
We select a and b, our integration limits, and we take only a few points in that
region to illustrate the error behavior of the trapezoid approximation:
a, b = 1, 9
xint = x[logical_and(x>=a, x<=b)][::30]
yint = y[logical_and(x>=a, x<=b)][::30]
Let’s plot both the function and the area below it in the trapezoid approximation:
plt.plot(x, y, lw=2)
plt.axis([0, 10, 0, 140])
plt.fill_between(xint, 0, yint, facecolor=’gray’, alpha=0.4)
plt.text(0.5 * (a + b), 30,r"$\int_aˆb f(x)dx$", horizontalalignment=’center’, fontsize=20);
D.2 NumPy arrays: the right data structure for scientific computing 185
Compute the integral both at high accuracy and with the trapezoid approximation
from scipy.integrate import quad, trapz

integral, error = quad(f, 1, 9)
trap_integral = trapz(yint, xint)
print "The integral is: %g +/- %.1e" % (integral, error)
print "The trapezoid approximation with", len(xint), "points is:", trap_integral
print "The absolute error is:", abs(integral - trap_integral)
The integral is: 680 +/- 7.5e-12

The trapezoid approximation with 6 points is: 621.286411141
The absolute error is: 58.7135888589
This simple example showed us how, combining the numpy, scipy and matplotlib
libraries we can provide an illustration of a standard method in elementary calculus
with just a few lines of code. We will now discuss with more detail the basic usage
of these tools.
D.2 NumPy arrays: the right data structure for scientific

computing
D.2.1 Basics of Numpy arrays
We now turn our attention to the Numpy library, which forms the base layer for the
entire ‘scipy ecosystem’. Once you have installed numpy, you can import it as
import numpy
though in this book we will use the common shorthand

import numpy as np
As mentioned above, the main object provided by numpy is a powerful array.

We’ll start by exploring how the numpy array differs from Python lists. We start by
creating a simple list and an array with the same contents of the list:
lst = [10, 20, 30, 40]

arr = np.array([10, 20, 30, 40])
Elements of a one-dimensional array are accessed with the same syntax as a list:
lst[0]
10
arr[0]
10
arr[-1]
40
arr[2:]
array([30, 40])
The first difference to note between lists and arrays is that arrays are homoge-
neous; i.e. all elements of an array must be of the same type. In contrast, lists can
contain elements of arbitrary type. For example, we can change the last element in
our list above to be a string:
lst[-1] = ’a string inside a list’

lst
[10, 20, 30, ’a string inside a list’]
but the same can not be done with an array, as we get an error message:
arr[-1] = ’a string inside an array’
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/home/fperez/teach/book-math-labtool/<ipython-input-13-29c0bfa5fa8a> in <module>()
----> 1 arr[-1] = ’a string inside an array’
ValueError: invalid literal for long() with base 10: ’a string inside an array’
The information about the type of an array is contained in its dtype attribute:
arr.dtype
dtype(’int32’)
Once an array has been created, its dtype is fixed and it can only store elements
of the same type. For this example where the dtype is integer, if we store a floating
point number it will be automatically converted into an integer:
arr[-1] = 1.234
arr
array([10, 20, 30, 1])
Above we created an array from an existing list; now let us now see other ways in
which we can create arrays, which we’ll illustrate next. A common need is to have
an array initialized with a constant value, and very often this value is 0 or 1 (suitable
as starting value for additive and multiplicative loops respectively); zeros creates
arrays of all zeros, with any desired dtype:
np.zeros(5, float)
array([ 0., 0., 0., 0., 0.])
np.zeros(3, int)
array([0, 0, 0])
np.zeros(3, complex)
array([ 0.+0.j, 0.+0.j, 0.+0.j])
and similarly for ones:
print ’5 ones:’, np.ones(5)
5 ones: [ 1. 1. 1. 1. 1.]
If we want an array initialized with an arbitrary value, we can create an empty

array and then use the fill method to put the value we want into the array:
a = empty(4)
a.fill(5.5)
a
array([ 5.5, 5.5, 5.5, 5.5])

Numpy also offers the arange function, which works like the builtin range
but returns an array instead of a list:
np.arange(5)
array([0, 1, 2, 3, 4])
and the linspace and logspace functions to create linearly and logarithmically-
spaced grids respectively, with a fixed number of points and including both ends of
the specified interval:
print "A linear grid between 0 and 1:", np.linspace(0, 1, 5)

print "A logarithmic grid between 10**1 and 10**4: ", np.logspace(1, 4, 4)
A linear grid between 0 and 1: [ 0. 0.25 0.5 0.75 1. ]

A logarithmic grid between 10**1 and 10**4: [ 10. 100. 1000. 10000.]
Finally, it is often useful to create arrays with random numbers that follow a spe-
cific distribution. The np.random module contains a number of functions that can
be used to this effect, for example this will produce an array of 5 random samples
taken from a standard normal distribution (0 mean and variance 1):
np.random.randn(5)
array([-0.08633343, -0.67375434, 1.00589536, 0.87081651, 1.65597822])
whereas this will also give 5 samples, but from a normal distribution with a mean
of 10 and a variance of 3:
norm10 = np.random.normal(10, 3, 5)
norm10
array([ 8.94879575, 5.53038269, 8.24847281, 12.14944165, 11.56209294])
D.2.2 Indexing with other arrays
Above we saw how to index arrays with single numbers and slices, just like Python
lists. But arrays allow for a more sophisticated kind of indexing which is very pow-
erful: you can index an array with another array, and in particular with an array of
boolean values. This is particluarly useful to extract information from an array that
matches a certain condition.
Consider for example that in the array norm10 we want to replace all values
above 9 with the value 0. We can do so by first finding the mask that indicates where
this condition is true or false:
mask = norm10 > 9

mask
array([False, False, False, True, True], dtype=bool)
Now that we have this mask, we can use it to either read those values or to reset
them to 0:
print ’Values above 9:’, norm10[mask]
Values above 9: [ 12.14944165 11.56209294]
print ’Resetting all values above 9 to 0...’

norm10[mask] = 0
print norm10
Resetting all values above 9 to 0...

[ 8.94879575 5.53038269 8.24847281 0. 0. ]
D.2.3 Arrays with more than one dimension
Up until now all our examples have used one-dimensional arrays. But Numpy can
create arrays of aribtrary dimensions, and all the methods illustrated in the previous
section work with more than one dimension. For example, a list of lists can be used
to initialize a two dimensional array:
lst2 = [[1, 2], [3, 4]]
arr2 = np.array([[1, 2], [3, 4]])
arr2
array([[1, 2],
[3, 4]])
With two-dimensional arrays we start seeing the power of numpy: while a nested
list can be indexed using repeatedly the [ ] operator, multidimensional arrays sup-
port a much more natural indexing syntax with a single [ ] and a set of indices
separated by commas:
print lst2[0][1]
print arr2[0,1]
2
2
Most of the array creation functions listed above can be used with more than one
dimension, for example:
np.zeros((2,3))
array([[ 0., 0., 0.],

[ 0., 0., 0.]])
np.random.normal(10, 3, (2, 4))
array([[ 11.26788826, 4.29619866, 11.09346496, 9.73861307],

[ 10.54025996, 9.5146268 , 10.80367214, 13.62204505]])
In fact, the shape of an array can be changed at any time, as long as the total
number of elements is unchanged. For example, if we want a 2x4 array with numbers
increasing from 0, the easiest way to create it is:
arr = np.arange(8).reshape(2,4)
print arr
[[0 1 2 3]
[4 5 6 7]]
With multidimensional arrays, you can also use slices, and you can mix and
match slices and single indices in the different dimensions (using the same array
as above):
print ’Slicing in the second row:’, arr[1, 2:4]

print ’All rows, third column :’, arr[:, 2]
Slicing in the second row: [6 7]

All rows, third column : [2 6]
If you only provide one index, then you will get an array with one less dimension
containing that row:
print ’First row: ’, arr[0]

print ’Second row: ’, arr[1]
First row: [0 1 2 3]
Second row: [4 5 6 7]
Now that we have seen how to create arrays with more than one dimension, it’s
a good idea to look at some of the most useful properties and methods that arrays
have. The following provide basic information about the size, shape and data in the
array:
print ’Data type :’, arr.dtype

print ’Total number of elements :’, arr.size
print ’Number of dimensions :’, arr.ndim
print ’Shape (dimensionality) :’, arr.shape
print ’Memory used (in bytes) :’, arr.nbytes
Data type : int32

Total number of elements : 8
Number of dimensions : 2
Shape (dimensionality) : (2, 4)
Memory used (in bytes) : 32
Arrays also have many useful methods, some especially useful ones are:
print ’Minimum and maximum :’, arr.min(), arr.max()

print ’Sum and product of all elements :’, arr.sum(), arr.prod()
print ’Mean and standard deviation :’, arr.mean(), arr.std()
Minimum and maximum : 0 7

Sum and product of all elements : 28 0
Mean and standard deviation : 3.5 2.29128784748
For these methods, the above operations area all computed on all the elements of
the array. But for a multidimensional array, it’s possible to do the computation along
a single dimension, by passing the axis parameter; for example:
print ’For the following array:\n’, arr

print ’The sum of elements along the rows is :’, arr.sum(axis=1)
print ’The sum of elements along the columns is :’, arr.sum(axis=0)
For the following array:

[[0 1 2 3]
[4 5 6 7]]
The sum of elements along the rows is : [ 6 22]
The sum of elements along the columns is : [ 4 6 8 10]
As you can see in this example, the value of the axis parameter is the dimension
which will be consumed once the operation has been carried out. This is why to sum
along the rows we use axis=0.
This can be easily illustrated with an example that has more dimensions; we
create an array with 4 dimensions and shape (3,4,5,6) and sum along the axis
number 2 (i.e. the third axis, since in Python all counts are 0-based). That con-
sumes the dimension whose length was 5, leaving us with a new array that has
shape (3,4,6):
np.zeros((3,4,5,6)).sum(2).shape
(3, 4, 6)
Another widely used property of arrays is the .T attribute, which allows you to
access the transpose of the array:
print ’Array:\n’, arr

print ’Transpose:\n’, arr.T
Array:
[[0 1 2 3]
[4 5 6 7]]
Transpose:
[[0 4]
[1 5]
[2 6]
[3 7]]
We don’t have time here to look at all the methods and properties of arrays, here’s
a complete list. Simply try exploring some of these IPython to learn more, or read
their description in the full Numpy documentation:
arr.T arr.copy arr.getfield arr.put
arr.squeeze arr.all arr.ctypes arr.imag
arr.ravel arr.std arr.any arr.cumprod
arr.item arr.real arr.strides arr.argmax
arr.cumsum arr.itemset arr.repeat arr.sum
arr.argmin arr.data arr.itemsize arr.reshape
arr.swapaxes arr.argsort arr.diagonal arr.max
arr.resize arr.take arr.astype arr.dot
arr.mean arr.round arr.tofile arr.base
arr.dtype arr.min arr.searchsorted arr.tolist
arr.byteswap arr.dump arr.nbytes arr.setasflat
arr.tostring arr.choose arr.dumps arr.ndim
arr.setfield arr.trace arr.clip arr.fill
arr.newbyteorder arr.setflags arr.transpose arr.compress
arr.flags arr.nonzero arr.shape arr.var
arr.conj arr.flat arr.prod arr.size
arr.view arr.conjugate arr.flatten arr.ptp
arr.sort
D.2.4 Operating with arrays
Arrays support all regular arithmetic operators, and the numpy library also con-
tains a complete collection of basic mathematical functions that operate on arrays.
It is important to remember that in general, all operations with arrays are applied
element-wise, i.e., are applied to all the elements of the array at the same time. Con-
sider for example:
arr1 = np.arange(4)
arr2 = np.arange(10, 14)
print arr1, ’+’, arr2, ’=’, arr1+arr2
[0 1 2 3] + [10 11 12 13] = [10 12 14 16]
Importantly, you must remember that even the multiplication operator is by de-
fault applied element-wise, it is not the matrix multiplication from linear algebra (as
is the case in Matlab, for example):
print arr1, ’*’, arr2, ’=’, arr1*arr2
[0 1 2 3] * [10 11 12 13] = [ 0 11 24 39]
While this means that in principle arrays must always match in their dimension-
ality in order for an operation to be valid, numpy will broadcast dimensions when
possible. For example, suppose that you want to add the number 1.5 to arr1; the
following would be a valid way to do it:
arr1 + 1.5*np.ones(4)
array([ 1.5, 2.5, 3.5, 4.5])
But thanks to numpy’s broadcasting rules, the following is equally valid:
arr1 + 1.5
array([ 1.5, 2.5, 3.5, 4.5])
In this case, numpy looked at both operands and saw that the first (arr1) was a
one-dimensional array of length 4 and the second was a scalar, considered a zero-
dimensional object. The broadcasting rules allow numpy to:
• create new dimensions of length 1 (since this doesn’t change the size of the array)
• ‘stretch’ a dimension of length 1 that needs to be matched to a dimension of a
different size.
So in the above example, the scalar 1.5 is effectively:

• first ‘promoted’ to a 1-dimensional array of length 1
• then, this array is ‘stretched’ to length 4 to match the dimension of arr1.
After these two operations are complete, the addition can proceed as now both
operands are one-dimensional arrays of length 4.
This broadcasting behavior is in practice enormously powerful, especially be-
cause when numpy broadcasts to create new dimensions or to ‘stretch’ existing
ones, it doesn’t actually replicate the data. In the example above the operation is
carried as if the 1.5 was a 1-d array with 1.5 in all of its entries, but no actual array
was ever created. This can save lots of memory in cases when the arrays in question
are large and can have significant performance implications.
The general rule is: when operating on two arrays, NumPy compares their shapes
element-wise. It starts with the trailing dimensions, and works its way forward, cre-
ating dimensions of length 1 as needed. Two dimensions are considered compatible
when
• they are equal to begin with, or

• one of them is 1; in this case numpy will do the ‘stretching’ to make them equal.
If these conditions are not met, a ValueError: frames are not aligned
exception is thrown, indicating that the arrays have incompatible shapes. The size
of the resulting array is the maximum size along each dimension of the input arrays.
This shows how the broadcasting rules work in several dimensions:
b = np.array([2, 3, 4, 5])
print arr, ’\n\n+’, b , ’\n----------------\n’, arr + b
[[0 1 2 3]
[4 5 6 7]]
+ [2 3 4 5]
----------------
[[ 2 4 6 8]
[ 6 8 10 12]]
Now, how could you use broadcasting to say add [4, 6] along the rows to arr
above? Simply performing the direct addition will produce the error we previously
mentioned:
c = np.array([4, 6])
arr + c
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/home/fperez/teach/book-math-labtool/<ipython-input-45-62aa20ac1980> in <module>()
1 c = np.array([4, 6])
----> 2 arr + c
ValueError: operands could not be broadcast together with shapes (2,4) (2)
According to the rules above, the array c would need to have a trailing dimension
of 1 for the broadcasting to work. It turns out that numpy allows you to ‘inject’ new
dimensions anywhere into an array on the fly, by indexing it with the special object
np.newaxis:
(c[:, np.newaxis]).shape
(2, 1)
This is exactly what we need, and indeed it works:
arr + c[:, np.newaxis]
array([[ 4, 5, 6, 7],
[10, 11, 12, 13]])
For the full broadcasting rules, please see the official Numpy docs, which de-
scribe them in detail and with more complex examples.
As we mentioned before, Numpy ships with a full complement of mathematical

functions that work on entire arrays, including logarithms, exponentials, trigono-
metric and hyperbolic trigonometric functions, etc. Furthermore, scipy ships a rich
special function library in the scipy.special module that includes Bessel, Airy,
Fresnel, Laguerre and other classical special functions. For example, sampling the
sine function at 100 points between 0 and 2π is as simple as:
x = np.linspace(0, 2*np.pi, 100)

y = np.sin(x)
D.2.5 Linear algebra in numpy
Numpy ships with a basic linear algebra library, and all arrays have a dot method
whose behavior is that of the scalar dot product when its arguments are vectors (one-
dimensional arrays) and the traditional matrix multiplication when one or both of
its arguments are two-dimensional arrays:
v1 = np.array([2, 3, 4])
v2 = np.array([1, 0, 1])
print v1, ’.’, v2, ’=’, v1.dot(v2)
[2 3 4] . [1 0 1] = 6
Here is a regular matrix-vector multiplication, note that the array v1 should be

viewed as a column vector in traditional linear algebra notation; numpy makes no
distinction between row and column vectors and simply verifies that the dimensions
match the required rules of matrix multiplication, in this case we have a 2 × 3 matrix
multiplied by a 3-vector, which produces a 2-vector:
A = np.arange(6).reshape(2, 3)
print A, ’x’, v1, ’=’, A.dot(v1)
[[0 1 2]
[3 4 5]] x [2 3 4] = [11 38]
For matrix-matrix multiplication, the same dimension-matching rules must be

satisfied, e.g. consider the difference between A × AT :
print A.dot(A.T)
[[ 5 14]
[14 50]]
and AT × A:
print A.T.dot(A)
[[ 9 12 15]
[12 17 22]
[15 22 29]]
Furthermore, the numpy.linalg module includes additional functionality

such as determinants, matrix norms, Cholesky, eigenvalue and singular value de-
compositions, etc. For even more linear algebra tools, scipy.linalg contains
the majority of the tools in the classic LAPACK libraries as well as functions to
operate on sparse matrices. We refer the reader to the Numpy and Scipy documen-
tations for additional details on these.
D.2.6 Reading and writing arrays to disk
Numpy lets you read and write arrays into files in a number of ways. In order to
use these tools well, it is critical to understand the difference between a text and a
binary file containing numerical data. In a text file, the number π could be written as
“3.141592653589793”, for example: a string of digits that a human can read, with
in this case 15 decimal digits. In contrast, that same number written to a binary file
would be encoded as 8 characters (bytes) that are not readable by a human but which
contain the exact same data that the variable pi had in the computer’s memory.
The tradeoffs between the two modes are thus:
• Text mode: occupies more space, precision can be lost (if not all digits are written
to disk), but is readable and editable by hand with a text editor. Can only be used
for one- and two-dimensional arrays.
• Binary mode: compact and exact representation of the data in memory, can’t be
read or edited by hand. Arrays of any size and dimensionality can be saved and
read without loss of information.
First, let’s see how to read and write arrays in text mode. The np.savetxt func-
tion saves an array to a text file, with options to control the precision, separators and
even adding a header:
arr = np.arange(10).reshape(2, 5)
np.savetxt(’test.out’, arr, fmt=’%.2e’, header="My dataset")
!cat test.out
# My dataset
0.00e+00 1.00e+00 2.00e+00 3.00e+00 4.00e+00
5.00e+00 6.00e+00 7.00e+00 8.00e+00 9.00e+00
And this same type of file can then be read with the matching np.loadtxt
function:
arr2 = np.loadtxt(’test.out’)
print arr2
[[ 0. 1. 2. 3. 4.]
[ 5. 6. 7. 8. 9.]]
For binary data, Numpy provides the np.save and np.savez routines. The
first saves a single array to a file with .npy extension, while the latter can be used
to save a group of arrays into a single file with .npz extension. The files created
with these routines can then be read with the np.load function.
Let us first see how to use the simpler np.save function to save a single array:
np.save(’test.npy’, arr2)
# Now we read this back
arr2n = np.load(’test.npy’)
# Let’s see if any element is non-zero in the difference.
# A value of True would be a problem.
print ’Any differences?’, np.any(arr2-arr2n)
Any differences? False
Now let us see how the np.savez function works. You give it a filename and
either a sequence of arrays or a set of keywords. In the first mode, the function will
auotmatically name the saved arrays in the archive as arr 0, arr 1, etc:
np.savez(’test.npz’, arr, arr2)
arrays = np.load(’test.npz’)
arrays.files
[’arr_1’, ’arr_0’]
Alternatively, we can explicitly choose how to name the arrays we save:

np.savez(’test.npz’, array1=arr, array2=arr2)
arrays = np.load(’test.npz’)
arrays.files
[’array2’, ’array1’]
The object returned by np.load from an .npz file works like a dictionary,
though you can also access its constituent files by attribute using its special .f
field; this is best illustrated with an example with the arrays object from above:
print ’First row of first array:’, arrays[’array1’][0]

# This is an equivalent way to get the same field
print ’First row of first array:’, arrays.f.array1[0]
First row of first array: [0 1 2 3 4]

First row of first array: [0 1 2 3 4]
This .npz format is a very convenient way to package compactly and without
loss of information, into a single file, a group of related arrays that pertain to a
specific problem. At some point, however, the complexity of your dataset may be
such that the optimal approach is to use one of the standard formats in scientific data
processing that have been designed to handle complex datasets, such as NetCDF or
HDF5.
Fortunately, there are tools for manipulating these formats in Python, and for
storing data in other ways such as databases. A complete discussion of the possibil-
ities is beyond the scope of this discussion, but of particular interest for scientific
users we at least mention the following:
• The scipy.io module contains routines to read and write Matlab files in .mat
format and files in the NetCDF format that is widely used in certain scientific
disciplines.
• For manipulating files in the HDF5 format, there are two excellent options in
Python: The PyTables project offers a high-level, object oriented approach to
manipulating HDF5 datasets, while the h5py project offers a more direct map-
ping to the standard HDF5 library interface. Both are excellent tools; if you need
to work with HDF5 datasets you should read some of their documentation and
examples and decide which approach is a better match for your needs.
D.3 High quality data visualization with Matplotlib
The http://matplotlib.sf.netmatplotlib library is a powerful tool capa-

ble of producing complex publication-quality figures with fine layout control in two
and three dimensions; here we will only provide a minimal self-contained introduc-
tion to its usage that covers the functionality needed for the rest of the book. We
encourage the reader to read the tutorials included with the matplotlib documenta-
tion as well as to browse its extensive gallery of examples that include source code.
Just as we typically use the shorthand np for Numpy, we will use plt for the
matplotlib.pyplot module where the easy-to-use plotting functions reside
(the library contains a rich object-oriented architecture that we don’t have the space
to discuss here):
The most frequently used function is simply called plot, here is how you can
make a simple plot of sin(x) for x ∈ [0, 2π] with labels and a grid (we use the semi-
colon in the last line to suppress the display of some information that is unnecessary
right now):
D.3 High quality data visualization with Matplotlib 199
x = np.linspace(0, 2*np.pi)
y = np.sin(x)
plt.plot(x,y, label=’sin(x)’)
plt.legend()
plt.grid()
plt.title(’Harmonic’)
plt.xlabel(’x’)
plt.ylabel(’y’);
You can control the style, color and other properties of the markers, for example:
plt.plot(x, y, linewidth=2);
plt.plot(x, y, ’o’, markersize=5, color=’r’);

We will now see how to create a few other common plot types, such as a simple
error plot:
# example data
x = np.arange(0.1, 4, 0.5)
y = np.exp(-x)
# example variable error bar values

yerr = 0.1 + 0.2*np.sqrt(x)
xerr = 0.1 + yerr
# First illustrate basic pyplot interface, using defaults where possible.

plt.figure()
plt.errorbar(x, y, xerr=0.2, yerr=0.4)
plt.title("Simplest errorbars, 0.2 in x, 0.4 in y");
A simple log plot

D.3 High quality data visualization with Matplotlib 201
x = np.linspace(-5, 5)
y = np.exp(-x**2)
plt.semilogy(x, y);
A histogram annotated with text inside the plot, using the text function:
mu, sigma = 100, 15

x = mu + sigma * np.random.randn(10000)
# the histogram of the data

n, bins, patches = plt.hist(x, 50, normed=1, facecolor=’g’, alpha=0.75)
plt.xlabel(’Smarts’)
plt.ylabel(’Probability’)
plt.title(’Histogram of IQ’)
# This will put a text fragment at the position given:
plt.text(55, .027, r’$\mu=100,\ \sigma=15$’, fontsize=14)
plt.axis([40, 160, 0, 0.03])
plt.grid(True)
D.3.1 Image display
The imshow command can display single or multi-channel images. A simple array
of random numbers, plotted in grayscale:
from matplotlib import cm

plt.imshow(np.random.rand(5, 10), cmap=cm.gray, interpolation=’nearest’);
A real photograph is a multichannel image, imshow interprets it correctly:
img = plt.imread(’stinkbug.png’)
print ’Dimensions of the array img:’, img.shape
plt.imshow(img);
Dimensions of the array img: (375, 500, 3)

D.4 IPython: a powerful interactive environment 203
D.3.2 Simple 3d plotting with matplotlib
Note that you must execute at least once in your session:
from mpl_toolkits.mplot3d import Axes3D
One this has been done, you can create 3d axes with the projection=’3d’
keyword to add subplot:
fig = plt.figure()
fig.add_subplot(<other arguments here>, projection=’3d’)
A simple surface plot:
from mpl_toolkits.mplot3d.axes3d import Axes3D

from matplotlib import cm
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1, projection=’3d’)
X = np.arange(-5, 5, 0.25)
Y = np.arange(-5, 5, 0.25)
X, Y = np.meshgrid(X, Y)
R = np.sqrt(X**2 + Y**2)
Z = np.sin(R)
surf = ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap=cm.jet,
linewidth=0, antialiased=False)
ax.set_zlim3d(-1.01, 1.01);
D.4 IPython: a powerful interactive environment
A key component of the everyday workflow of most scientific computing environ-

ments is a good interactive environment, that is, a system in which you can execute
small amounts of code and view the results immediately, combining both printing
out data and opening graphical visualizations. All modern systems for scientific
computing, commercial and open source, include such functionality.
Out of the box, Python also offers a simple interactive shell with very limited
capabilities. But just like the scientific community built Numpy to provide arrays
suited for scientific work (since Pytyhon’s lists aren’t optimal for this task), it has
also developed an interactive environment much more sophisticated than the built-
in one. The http://ipython.orgIPython project offers a set of tools to make
productive use of the Python language, all the while working interactively and with
immedate feedback on your results. The basic tools that IPython provides are:
1. A powerful terminal shell, with many features designed to increase the fluidity
and productivity of everyday scientific workflows, including:
• rich introspection of all objects and variables including easy access to the
source code of any function
• powerful and extensible tab completion of variables and filenames,
• tight integration with matplotlib, supporting interactive figures that don’t
block the terminal,
• direct access to the filesystem and underlying operating system,
• an extensible system for shell-like commands called ‘magics’ that reduce the
work needed to perform many common tasks,
• tools for easily running, timing, profiling and debugging your codes,
• syntax highlighted error messages with much more detail than the default
Python ones,
• logging and access to all previous history of inputs, including across sessions
2. A Qt console that provides the look and feel of a terminal, but adds support
for inline figures, graphical calltips, a persistent session that can survive crashes
(even segfaults) of the kernel process, and more.
3. A web-based notebook that can execute code and also contain rich text and fig-
ures, mathematical equations and arbitrary HTML. This notebook presents a
document-like view with cells where code is executed but that can be edited
in-place, reordered, mixed with explanatory text and figures, etc.
4. A high-performance, low-latency system for parallel computing that supports the
control of a cluster of IPython engines communicating over a network, with opti-
mizations that minimize unnecessary copying of large objects (especially numpy
arrays).
We will now discuss the highlights of the tools 1–3 above so that you can make them
an effective part of your workflow. The topic of parallel computing is beyond the
scope of this document, but we encourage you to read the extensive documentation
at
http://ipython.org/ipython-doc/rel-0.12.1/parallel/index.html
\end{varbatim}
and
\begin{verbatim}
http://minrk.github.com/scipy-tutorial-2011/}{tutorials
on this available on the IPython website.
D.4.1 The IPython terminal
You can start IPython at the terminal simply by typing: $ ipython which will
provide you some basic information about how to get started and will then open
a prompt labeled In [1]: for you to start typing. Here we type 264 and Python
computes the result for us in exact arithmetic, returning it as Out[1]:
Python 2.7.2+ (default, Oct 4 2011, 20:03:08)
IPython 0.13.dev -- An enhanced Interactive Python.

In [1]: 2**64
Out[1]: 18446744073709551616L
The first thing you should know about IPython is that all your inputs and outputs
are saved. There are two variables named In and Out which are filled as you work
with your results. Furthermore, all outputs are also saved to auto-created variables
of the form NN where NN is the prompt number, and inputs to iNN. This allows
you to recover quickly the result of a prior computation by referring to its number
even if you forgot to store it as a variable. For example, later on in the above session
you can do:
In [6]: print _1
18446744073709551616
We strongly recommend that you take a few minutes to read at least the basic
introduction provided by the ? command, and keep in mind that the %quickref
command at all times can be used as a quick reference “cheat sheet” of the most
frequently used features of IPython.
At the IPython prompt, any valid Python code that you type will be executed
similarly to the default Python shell (though often with more informative feedback).
But since IPython is a superset of the default Python shell; let’s have a brief look at
some of its additional functionality.
Object introspection
A simple ? command provides a general introduction to IPython, but as indicated
in the banner above, you can use the ? syntax to ask for details about any object. For
example, if we type 1?, IPython will print the following details about this variable:
In [14]: _1?
Type: long
Base Class: <type ’long’>
String Form:18446744073709551616
Namespace: Interactive
Docstring:
long(x[, base]) -> integer
Convert a string or number to a long integer, if possible. A floating
[etc... snipped for brevity]

If you add a second ? and for any oobject x type x??, IPython will try to provide
an even more detailed analsysi of the object, including its syntax-highlighted source
code when it can be found. It’s possible that x?? returns the same information as
x?, but in many cases x?? will indeed provide additional details.
Finally, the ? syntax is also useful to search namespaces with wildcards. Sup-
pose you are wondering if there is any function in Numpy that may do text-related
things; with np.*txt*?, IPython will print all the names in the np namespace
(our Numpy shorthand) that have ‘txt’ anywhere in their name:
In [17]: np.*txt*?
np.genfromtxt
np.loadtxt
np.mafromtxt
np.ndfromtxt
np.recfromtxt
np.savetxt
Tab completion
IPython makes the tab key work extra hard for you as a way to rapidly inspect
objects and libraries. Whenever you have typed something at the prompt, by hitting
the <tab> key IPython will try to complete the rest of the line. For this, IPython
will analyze the text you had so far and try to search for Python data or files that
may match the context you have already provided.
For example, if you type np.load and hit the key, you’ll see:
In [21]: np.load<TAB HERE>
np.load np.loads np.loadtxt
so you can quickly find all the load-related functionality in numpy. Tab completion
works even for function arguments, for example consider this function definition:
In [20]: def f(x, frobinate=False):
....: if frobinate:
....: return x**2
....:
If you now use the <tab> key after having typed ‘fro’ you’ll get all valid Python
completions, but those marked with = at the end are known to be keywords of your
function:
In [21]: f(2, fro<TAB HERE>
frobinate= frombuffer fromfunction frompyfunc fromstring
from fromfile fromiter fromregex frozenset
at this point you can add the b letter and hit <tab> once more, and IPython will
finish the line for you:
In [21]: f(2, frobinate=
As a beginner, simply get into the habit of using <tab> after most objects; it should
quickly become second nature as you will see how helps keep a fluid workflow
and discover useful information. Later on you can also customize this behavior by
writing your own completion code, if you so desire.
Matplotlib integration
One of the most useful features of IPython for scientists is its tight integration
with matplotlib: at the terminal IPython lets you open matplotlib figures without
blocking your typing (which is what happens if you try to do the same thing at the
default Python shell), and in the Qt console and notebook you can even view your
figures embedded in your workspace next to the code that created them.
The matplotlib support can be either activated when you start IPython by passing
the --pylab flag, or at any point later in your session by using the %pylab com-
mand. If you start IPython with --pylab, you’ll see something like this (note the
extra message about pylab):
ipython --pylab
Python 2.7.2+ (default, Oct 4 2011, 20:03:08)
IPython 0.13.dev -- An enhanced Interactive Python.

Welcome to pylab, a matplotlib-based Python environment [backend: Qt4Agg].

For more information, type ’help(pylab)’.
In [1]:
Furthermore, IPython will import numpy with the np shorthand, matplotlib.pyplot
as plt, and it will also load all of the numpy and pyplot top-level names so that you
can directly type something like:
In [1]: x = linspace(0, 2*pi, 200)
In [2]: plot(x, sin(x))

Out[2]: [<matplotlib.lines.Line2D at 0x9e7c16c>]
instead of having to prefix each call with its full signature (as we have been doing
in the examples thus far):
In [3]: x = np.linspace(0, 2*np.pi, 200)
In [4]: plt.plot(x, np.sin(x))

Out[4]: [<matplotlib.lines.Line2D at 0x9e900ac>]
This shorthand notation can be a huge time-saver when working interactively (it’s a
few characters but you are likely to type them hundreds of times in a session). But we
should note that as you develop persistent scripts and notebooks meant for reuse, it’s
best to get in the habit of using the longer notation (known as fully qualified names
as it’s clearer where things come from and it makes for more robust, readable and
maintainable code in the long run).
Access to the operating system and files
In IPython, you can type ls to see your files or cd to change directories, just
like you would at a regular system prompt:
In [2]: cd tests
/home/fperez/ipython/nbconvert/tests
In [3]: ls test.*
test.aux test.html test.ipynb test.log test.out test.pdf test.rst test.tex
Furthermore, if you use the ! at the beginning of a line, any commands you pass
afterwards go directly to the operating system:
In [4]: !echo "Hello IPython"
Hello IPython
IPython offers a useful twist in this feature: it will substitute in the command the
value of any Python variable you may have if you prepend it with a $ sign:
In [5]: message = ’IPython interpolates from Python to the shell’
In [6]: !echo $message

IPython interpolates from Python to the shell
This feature can be extremely useful, as it lets you combine the power and clar-
ity of Python for complex logic with the immediacy and familiarity of many shell
commands. Additionally, if you start the line with two $$ signs, the output of the
command will be automatically captured as a list of lines, e.g.:
In [10]: !!ls test.*
Out[10]:
[’test.aux’,
’test.html’,
’test.ipynb’,
’test.log’,
’test.out’,
’test.pdf’,
’test.rst’,
’test.tex’]
As explained above, you can now use this as the variable 10. If you directly want
to capture the output of a system command to a Python variable, you can use the
syntax =!:
In [11]: testfiles =! ls test.*
In [12]: print testfiles

[’test.aux’, ’test.html’, ’test.ipynb’, ’test.log’, ’test.out’, ’test.pdf’, ’test.rst’, ’test.
Finally, the special %alias command lets you define names that are shorthands for
system commands, so that you can type them without having to prefix them via !
explicitly (for example, ls is an alias that has been predefined for you at startup).
Magic commands
IPython has a system for special commands, called ‘magics’, that let you control
IPython itself and perform many common tasks with a more shell-like syntax: it uses
spaces for delimiting arguments, flags can be set with dashes and all arguments are
treated as strings, so no additional quoting is required. This kind of syntax is invalid
in the Python language but very convenient for interactive typing (less parentheses,
commands and quoting everywhere); IPython distinguishes the two by detecting

lines that start with the % character.
You can learn more about the magic system by simply typing %magic at the
prompt, which will give you a short description plus the documentation on all
available magics. If you want to see only a listing of existing magics, you can use
%lsmagic:
In [4]: lsmagic
Available magic functions:
%alias %autocall %autoindent %automagic %bookmark %c
%cd %colors %config %cpaste %debug %dhist %dirs
%doctest_mode %ds %ed %edit %env %gui %hist %history
%install_default_config %install_ext %install_profiles
%load_ext %loadpy %logoff %logon %logstart %logstate %logstop
%lsmagic %macro %magic %notebook %page %paste %pastebin
%pd %pdb %pdef %pdoc %pfile %pinfo %pinfo2 %pop %popd
%pprint %precision %profile %prun %psearch %psource %pushd %pwd
%pycat %pylab %quickref %recall %rehashx
%reload_ext %rep %rerun %reset %reset_selective %run %save %sc
%stop %store %sx %tb %time %timeit %unalias %unload_ext %who
%who_ls %whos %xdel %xmode
Automagic is ON, % prefix NOT needed for magic functions.

Note how the example above omitted the eplicit % marker and simply uses lsmagic.
As long as the ‘automagic’ feature is on (which it is by default), you can omit the %
marker as long as there is no ambiguity with a Python variable of the same name.
Running your code
While it’s easy to type a few lines of code in IPython, for any long-lived work
you should keep your codes in Python scripts (or in IPython notebooks, see below).
Consider that you have a script, in this case trivially simple for the sake of brevity,
named simple.py:
In [12]: !cat simple.py
import numpy as np
x = np.random.normal(size=100)
print ’First elment of x:’, x[0]

The typical workflow with IPython is to use the %run magic to execute your script
(you can omit the .py extension if you want). When you run it, the script will ex-
ecute just as if it had been run at the system prompt with python simple.py
(though since modules don’t get re-executed on new imports by Python, all sys-
tem initialization is essentially free, which can have a significant run time impact in
some cases):
In [13]: run simple
First elment of x: -1.55872256289
Once it completes, all variables defined in it become available for you to use inter-
actively:
In [14]: x.shape
Out[14]: (100,)
This allows you to plot data, try out ideas, etc, in a %run/interact/edit cycle that
can be very productive. As you start understanding your problem better you can
refine your script further, incrementally improving it based on the work you do at
the IPython prompt. At any point you can use the %hist magic to print out your
history without prompts, so that you can copy useful fragments back into the script.
By default, %run executes scripts in a completely empty namespace, to better
mimic how they would execute at the system prompt with plain Python. But if you
use the -i flag, the script will also see your interactively defined variables. This lets
you edit in a script larger amounts of code that still behave as if you had typed them
at the IPython prompt.
You can also get a summary of the time taken by your script with the -t flag;
consider a different script randsvd.py that takes a bit longer to run:
In [21]: run -t randsvd.py
IPython CPU timings (estimated):

User : 0.38 s.
System : 0.04 s.
Wall time: 0.34 s.
User is the time spent by the computer executing your code, while System is the
time the operating system had to work on your behalf, doing things like memory
allocation that are needed by your code but that you didn’t explicitly program and
that happen inside the kernel. The Wall time is the time on a ‘clock on the wall’
between the start and end of your program.
If Wall > User+System, your code is most likely waiting idle for certain
periods. That could be waiting for data to arrive from a remote source or perhaps
because the operating system has to swap large amounts of virtual memory. If you
know that your code doesn’t explicitly wait for remote data to arrive, you should
investigate further to identify possible ways of improving the performance profile.
If you only want to time how long a single statement takes, you don’t need to put
it into a script as you can use the %timeit magic, which uses Python’s timeit
module to very carefully measure timig data; timeit can measure even short state-
ments that execute extremely fast:
In [27]: %timeit a=1
10000000 loops, best of 3: 23 ns per loop
and for code that runs longer, it automatically adjusts so the overall measurement
doesn’t take too long:
In [28]: %timeit np.linalg.svd(x)
1 loops, best of 3: 310 ms per loop
The %run magic still has more options for debugging and profiling data; you should
read its documentation for many useful details (as always, just type %run?).
D.4.2 The graphical Qt console
If you type at the system prompt (see the IPython website for installation details, as
this requires some additional libraries):
$ ipython qtconsole
instead of opening in a terminal as before, IPython will start a graphical console that
at first sight appears just like a terminal, but which is in fact much more capable than
a text-only terminal. This is a specialized terminal designed for interactive scientific
work, and it supports full multi-line editing with color highlighting and graphical
calltips for functions, it can keep multiple IPython sessions open simultaneously in
tabs, and when scripts run it can display the figures inline directly in the work area.
Fig. D.1 The IPython Qt console: a lightweight terminal for scientific exploration, with code,
results and graphics in a soingle environment.
The Qt console accepts the same --pylab startup flags as the terminal, but you
can additionally supply the value --pylab inline, which enables the support
for inline graphics shown in the figure. This is ideal for keeping all the code and
figures in the same session, given that the console can save the output of your entire
session to HTML or PDF.
Since the Qt console makes it far more convenient than the terminal to edit
blocks of code with multiple lines, in this environment it’s worth knowing about
the %loadpy magic function. %loadpy takes a path to a local file or remote URL,
fetches its contents, and puts it in the work area for you to further edit and execute. It
can be an extremely fast and convenient way of loading code from local disk or re-
mote examples from sites such as the http://matplotlib.sourceforge.
net/gallery.htmlMatplotlib gallery.
Other than its enhanced capabilities for code and graphics, all of the features of
IPython we’ve explained before remain functional in this graphical console.
D.4.3 The IPython Notebook
The third way to interact with IPython, in addition to the terminal and graphical Qt
console, is a powerful web interface called the “IPython Notebook”. If you run at the
system console (you can omit the pylab flags if you don’t need plotting support):
$ ipython notebook --pylab inline
IPython will start a process that runs a web server in your local machine and to
which a web browser can connect. The Notebook is a workspace that lets you exe-
cute code in blocks called ‘cells’ and displays any results and figures, but which can
also contain arbitrary text (including LaTeX-formatted mathematical expressions)
and any rich media that a modern web browser is capable of displaying.
In fact, this document was written as a Notebook, and only exported to LaTeX for
printing. Inside of each cell, all the features of IPython that we have discussed before
remain functional, since ultimately this web client is communicating with the same
IPython code that runs in the terminal. But this interface is a much more rich and
powerful environment for maintaining long-term “live and executable” scientific
documents.
Notebook environments have existed in commercial systems like Mathemat-
ica(TM) and Maple(TM) for a long time; in the open source world the http:
//sagemath.orgSage project blazed this particular trail starting in 2006, and
now we bring all the features that have made IPython such a widely used tool to a
Notebook model.
Since the Notebook runs as a web application, it is possible to configure it for
remote access, letting you run your computations on a persistent server close to your
data, which you can then access remotely from any browser-equipped computer. We
encourage you to read the extensive documentation provided by the IPython project
for details on how to do this and many more features of the notebook.
Finally, as we said earlier, IPython also has a high-level and easy to use set of
libraries for parallel computing, that let you control (interactively if desired) not
just one IPython but an entire cluster of ‘IPython engines’. Unfortunately a detailed
discussion of these tools is beyond the scope of this text, but should you need to
Fig. D.2 The IPython Notebook: text, equations, code, results, graphics and other multimedia in
an open format for scientific exploration and collaboration
parallelize your analysis codes, a quick read of the tutorials and examples provided
at the IPython site may prove fruitful.
References
1. I. Amidror. Glass patterns as Moiré effects: new surprising results. Optics Lett., 28:7–9, 2003.
2. J. S. Bendat and A. G. Piersol. Random Data: Analysis and Measurement Procedures, Second
Edition. John Wiley & Sons, New York, 1986.
3. E. O. Brigham. The fast Fourier transform and its applications. Prentice Hall, Englewood, N.
J., 1988.
4. B. Ermentrout. Type I membranes, phase resetting curves and synchrony. Neural Comp.,
8:979–1001, 1996.
5. B. Ermentrout. Simulating, analyzing, and animating dynamical systems: A guide for XPPAUT
for researchers and students. SIAM, Philadelphia, 2002.
6. J. D. Farmer. Chaotic attractors of an infinite–dimensional dynamical system. Physica D,
4:366–393, 1982.
7. D. J. Field. Rotation between the statistics of natural images and the response properties of
cortical cells. J. Opt. Soc. Amer. A, 4:2379–2394, 1987.
8. D. J. Field and N. Brady. Visual sensitivity, blur and the sources of variability in the amplitude
spectra of natural scenes. Vision Res., 37:3367–3382, 1997.
9. R. Fitzhugh. Impulses and physiological states in theoretical models of nerve membranes.
Biophys. J., 1:445–466, 1961.
10. J. Foss and J. Milton. Multistability in recurrent neural loops arising from delay. J. Neuro-
physiol., 84:975–985, 2000.
11. J. Foss, F. Moss, and J. Milton. Noise, multistability, and delayed recurrent loops. Phys. Rev.
E, 55:4536–4543, 1997.
12. A. S. French and A. V. Holden. Alias–free sampling of neuronal spike trains. Kybernetik,
5:165–171, 1971.
13. V. Fugère and R. Krahe. Electric signals and species recognition in the wave–type gymnoti-
form fish apteronotus leptorhynchus. J. Exp. Biol., 213:225–236, 2010.
14. L. Glass. Moiré effect from random dots. Nature, 223:578–580, 1969.
15. L. Glass and M. C. Mackey. From Clocks to Chaos: The rhythms of life. Princeton University
Press, Princeton, New Jersey, 1988.
16. M. R. Guevara. Dynamics of excitable cells. In A. Beuter, L. Glass, M. C. Mackey, and M. S.
Titcombe, editors, Nonlinear dynamics in physiology and medicine, pages 87–121, New York,
2003. Springer.
17. R. Guttman, S. Lewis, and J. Rinzel. Control of repetitive firing in squid axon membrane as a
model for a neuron oscillator. J. Physiol. (London), 305:377–395, 1980.
18. S. H. D. Haddock and C. W. Dunn. Practical Computing for Biologists. Sunderland, Mas-
sachusetts, USA, 2011.
19. A. L. Hodgkin. The local electric changes associated wirh repetetive action in a non–
modulated axon. J. Physiol., 107:165–181, 1948.
215
216 References
20. J. D. Hunter and J. G. Milton. Synaptic heterogeneity and stimulus–induced modulation of

despression in cnetral synapses. J. Neuroscience, 21:5781–5793, 2001.
21. E. M. Izhikevich. Simple model of spiking neurons. IEEE Trans. Neural Net., 14:1569–1572,
2003.
22. E. M. Izhikevich. Which model to use for cortical spiking neurons? IEEE Trans. Neural Net.,
15:1063–1070, 2004.
23. E. M. Izhikevich. Dynamical systems in neuroscience: The geometry of excitability and burst-
ing. MIT Press, MIT, 2007.
24. O. C. J. Lippold. Oscillation in the stretch reflex arc and the origin of the rhythmical 8–12 c/s
component of physiological tremor. J. Physiol., 206:359–382, 1970.
25. M. C. Mackey. Periodic auto-immune hemolytic anemia: An induced dynamical disease. Bull.
Math. Biol., 41:829–834, 1979.
26. M. C. Mackey and L. Glass. Oscillation and chaos in physiological control systems. Science,
pages 287–289, 1977.
27. J. B. Marion. Classical dynamics of particles and systems, 2nd edition. Academic Press, New
York, 1970.
28. R. W. Meech and G. O. Mackie. Evolution of excitability in lower metazoans. In Invertebrate
Neurobiology, pages 581–616, New York, 2007. Cold Spring Harbour Laboratory Press.
29. J. G. Milton and M. C. Mackey MC. Periodic haematological diseases: Mystical entities or
dynamical disorders? J. Royal College of Physicians of London, 23:236–241, 1989.
30. J. S. Nagumo, S. Arimoto, and S. Yoshizawa. An active pulse transmission line simulating a
nerve axon. Proc. IRE, 50:2061–2070, 1962.
31. J. S. Orr, J. Kirk, K. G. Gary, and J. R. Anderson. A study of the interdependence of red cell
and bone marrow stem cell populations. Brit. J. Haematol., 15:23–34, 1968.
32. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Numerical recipes: The art
of scientific computing, third edition. Cambridge University Press, New York, 2007.
33. D. L. Ruderman. Origins of scaling in natural images. Vision Res., 37:3385–3398, 1977.
34. D. L. Ruderman and W. Bialek. Statistics of natural images: scaling in the woods. Phys. Rev.
Lett., 73:814–817, 1994.
35. D. J. Tolhurst, Y. Tudmor, and T. Chao. Amplitude spectra of natural images. Ophthalm.
Physiol. Optics, 12:229–232, 1992.
36. B. van der Pol. Forced oscillations in a circuit with nonlinear resistance (receptance with
reactive triod). Phil. Mag. (London, Edinburgh, and Dublin), 3:65–80, 1927.
37. B. van der Pol and J. van der Mark. The heart beat considered as a relaxation oscillator, and
an electrical model of the heart. Phil. Mag. (7th ser.), 6:763–775, 1928.
38. H. R. Wilson. Spikes, decisions and actions: dynamical foundations of neurosciences. Oxford
University Press, New York, 1999.

Lab Manual Vol1 Nov13 2014-1

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Lab Manual Vol1 Nov13 2014-1

Caricato da

Copyright:

Formati disponibili

John Milton

Stefan van der Walt

Lab manual: Mathematics as a

Vol. 1 (Fall semester)

2 Introducing XPPAUT (Lab 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Introducing Python: Working with data (Lab 3) . . . . . . . . . . . . . . . . . . . 17

4 Working with arrays and eigenvalues (Lab 4) . . . . . . . . . . . . . . . . . . . . . 27

5 The ’cubic’ nonlinearity: Excitable systems (Lab 5) . . . . . . . . . . . . . . . . 41

6 Bifurcation diagram: stopping spiking neurons with a single pulse

6.4 Exercise 3: Stopping spiking neurons with a single pulse . . . . . . . . . . 61

7 Lab 7: Filters and convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

8 Lab 8: Fourier series: Gibbs phenomenon and filtering (Chapter 8) . . 73

9 Lab 9: FTT and power spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

10 Phase resetting oscillators (Lab 10) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

12 Arduino boards: Using your laptop in the laboratory(Lab 12) . . . . . . . 115

12.2.1 What is an accelerometer? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

A Appendix A: Finding data for analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

B Appendix B: Latex survival guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

C Appendix C: Introduction of Python by A. H. Lee . . . . . . . . . . . . . . . . . . 145

D Appendix D: The Scientific Python Ecosystem by F. Perez . . . . . . . . . . 181

We would all like to be good citizens of the scientific community. An important

rmdir You can remove a directory with the command

cp (copy) or mv (move) It is possible to move or copy files using the com-

Deliverable: Your LaTeX document must be completed before we do Lab 3. You

Writing ODE files

Housekeeping: Since XPPAUT file names have the form my_name.ode it is

2.2 Exercise 1: First XPPAUT program

We illustrate the use of XPPAUT by using it to integrate the first–order ordinary

Running an XPPAUT program: The XPPAUT program is activated by typ-

2.3 Exercise 2: Second-order DDEs

2.4 Exercise 3: Phase plane representations

C and D provide an introduction to, respectively, programming in Python and the

Housekeeping: The names of Python programs have the form myname.py. As

3.2 Exercise 1: Making figures using Matplotlib

Python is a programming language such as C or Fortran. By itself Python is not

# Import the data file

# Extract the data from the array

# Plot the data in the form of a figure

# Save the figure in an appropriate format

# Show the figure on the compter screen

Differential equations using Python: The reader will no doubt wonder

2. The integrator odeint was developed by Lawrence Livermore National Lab-

4. The output of integrate.odeint() is the array z[]. The number of

f (v) = v(a − v)(v − 1)

3.3 Exercise 2: Adding a figure to a LaTeX document

Deliverable: Use Lab3_template.tex to prepare the lab assignment.

Housekeeping: In your home directory make two directories pyprogs and

The tradeoffs between the two modes are thus:

We can read the data using np.loadtxt, namely

4.2 Exercise 1: Stability using XPPAUT

In lecture we emphasized three concepts for determining the qualitative behavior of

c− : number of complex eigenvalues with negative real part

4.3 Exercise 2: Stability using Python

When we integrate a differential equation using numpy it is necessary to re-write it

λ 2 − (a + d)λ + (ad − bc) = 0 . (4.2)

We adopted this procedure because a course in linear algebra is not a pre-requisite

dxn /dt cn1 cn2 . . . cnn xn

When we multiply a vector by a matrix we multiply a row of the matrix by the

4.4 Exercise 3: Moiré-Glass dot patterns

In the two-dimensional xy-plane, the values of x and y specify the coordinates of a

An important point is that in contrast to continuous differential equations, the

|1.05| = 1.05 > 1

4.5 Exercise 4: Working with digital images

Skimage (a.k.a. scikit-image) is a collection of algorithms useful for im-