Python:

a Powerful,
Easy-to-Use,
Open-Source
Scripting
Language
Stephen Ferg
Bureau of Labor
Statistics
1

What we're
going to talk
about...

Executive Summary

Open-source applications -- or at least

some of them -- are good.

Scripting languages are good.

Python is the best scripting language.

Time, money, and innovative energy...

can be saved.
Productivity, speed, and quality ...
can be improved.
3

Topics

Scripting languages
Python
Issues surrounding use of Python

Open-Source Software

What the experts think

Where we might find Python useful
4

What is a "scripting"
language?

Interpreted

Untyped or dynamically typed

requires a run-time interpreter or

virtual machine

No data declarations

No compilation step
5

In the beginning...

System programming languages

Assembler, C
Fortran, Cobol, Algol
PL/1, Pascal, Basic, C++, Java

Command languages

JCL
TSO CLists, CMS "execs"
Batch files
6

"Little Languages" appear...

awk
sed
grep

Scripting languages evolve

UNIX shell
scripting
languages
Rexx
Tcl, TK

Perl
Python
PHP
Ruby
SAS

Many have higher-level object-oriented

features that make them powerful
application development languages in
their own right.

The defining characteristic

of scripting languages...

Vastly increased
productivity!

Programmers can write roughly the same

number of lines of code per year regardless
of language.
Productivity = the number of machine
instructions that a programmer can produce
perAssembly
year.
Language
One machine instruction per line
System programming languages
3-7 machine instructions per line
Scripting languages
Hundreds to thousands of instructions
per line
"Scripting: Higher Level Programmingfor the 21st Century" by John K. Ousterhout
IEEE Computer magazine, March 1998 -- http://home.pacbell.net/ouster/scripting.html

10

Language Levels and Productivity

1000
Scripting Languages
Instructions/Statement

VB

100
Python, Perl,
Ruby, TCL
Java

10

C++
C

Assembler

None

System Languages
Degree of Typing

Strong

From "Scripting: Higher Level Programming for the 21st Century" by John K. Ousterhout.
11
This version prepared by Dana Moore and updated by Stephen Ferg

This leads to...

increasing use of
scripting languages.

12

John Ousterhout
For the last fifteen years a fundamental change
has been occurring in the way people write
computer programs. ...from system programming
languages to scripting languages. This article
explains why scripting languages will handle many
of the programming tasks of the next century
better than system programming languages.
Scripting languages represent a different set of tradeoffs than
system programming languages. They give up execution speed
and strong typing but provide significantly higher programmer
productivity and software reuse.
This tradeoff makes increasing sense as computers become
faster and cheaper compared to programmers.
Scripting: Higher Level Programmingfor the 21st Century - IEEE Computer 1998

13

Robert C. Martin
I think there is a trend in language that will become
more and more evident as the decade progresses. I think
we are seeing an end to the emphasis on statically typed
languages like C++, Java, Eiffel, Pascal, and Ada.
I expect to see an ever increasing use of dynamically
typed languages, such as Python, Ruby, and even
Smalltalk. These languages, and languages of their kind,
will be mainstream industrial languages in the coming
years.
14

Tim O'Reilly
Why Scripting Languages
Matter
People
are so stuck in the personal computer paradigm that
they don't recognize that the nature of applications has
undergone a profound change in the last decade, with most of
the new killer apps running on what has been called the LAMP
platform (Linux-Apache-MySQL-PHP | Perl | Python). People
understand the importance of Linux, Apache and MySQL... but
they still struggle with understanding the "P" in LAMP.
The reason why dynamic languages like Perl, Python, and PHP
are so important is key to understanding the paradigm shift.
Unlike applications from the previous paradigm, web
applications are not released in one to three year cycles. They
are updated every day, sometimes every hour.
15

Agile programming
languages
We really should stop
calling them
"scripting" languages.
"Agile" languages
would be more accurate.
Kevin Altis and Ward Cunningham

16

Python a great agile programming

language!

Powerful
Easy-to-learn
Easy-to-use
Open-Source

"Python" from
"Monty Python's Flying Circus"
17

Python language features

Derived from ABC, Modula-3, and C

Object-Oriented
Dynamically typed
Interpreted
Cross-platform (Unix, Windows, etc.)
Extensible
Flexible
18

Python's most obvious

feature
Uses indentation as a control structure

no DO.. END
no BEGIN..END
no { .. }
19

Indentation as a controlstructure
for i in range(20):
if i%3 == 0:
print i
if i%5 == 0:
print "Bingo!"
print "---"

0
Bingo!
------3
------6
------9
------12
------15
Bingo!
------18
-----

20

Sample Python Code

See the handouts distributed with

this presentation.

For a quick overview of Python's

features:
http://www.ferg.org/python_slides/index.html
21

Python's advantages

Productivity and Ease-Of-Use

Maintainability
Flexibility (OO & functional
programming)
Power
Plays well with other languages

Jython compiles to Java byte code

Extensible

Easy to extend with, or call from, C

22

Python some history

Developed by
Guido van Rossum
in 1991.

A fan of
Monty Python's Flying
Circus

23

Guido van Rossum

Python BDFL - Benevolent Dictator for Life

1999 - Dr. Dobb's Journal Excellence in

Programming Awards

2002 Free Software Foundation (FSF)

Award for the Advancement of Free
Software
24

"Doctor Fun has the dubious distinction of being the first web cartoon.
Doctor Fun was not, however, the first cartoon on the Internet." - http://www.ibiblio.org/Dave/index

25

T-shirt Slogan
Python:
Programming the way Guido
indented it

26

Who is using Python?

What are they doing with it?

Industrial Light & Magic , maker of the

Star Wars films, uses Python
extensively in the computer graphics
production process.

Disney Feature Length Animation uses

Python for its animation production
applications.
27

Google , a leading internet search

engine, is powered by Python.

Yahoo uses Python for its groups site.

The Inktomi (formerly Infoseek, now

part of Yahoo) search engine uses
Python.

IBM and Philips have used Python to

create the business practice logic for
factory tool control applications.
28

NASA uses Python in several large

projects, including a CAD/CAM system
and a graphical workflow modeler used
in planning space shuttle missions.

The National Institutes of Health (USA)

and Case Western Reserve University
are building cutting-edge
genetic analysis software with Python.
29

The National Weather Service (USA) uses

Python to prepare weather forecasts.

Python is also used for this purpose at

the
Swedish Meteorological and Hydrological
Institute
and at TV4 Sweden .

Chandler, the new open-source crossplatform Personal Information Manager

being developed by Mitch Kapor, is being
written in Python and wxWindows.
30

Lawrence Livermore National Laboratori

es
is basing a new numerical engineering
environment on Python.

The
Theoretical Physics Division at Los Alam
os National Laboratory
uses Python to control large-scale
physics codes on massively parallel
supercomputers, high-end servers, and
clusters.
31

US Navy uses Python & Zope for a

web based workflow system

US Dept. of Agriculture - Python &

Zope for massive collaboration

Should we be using
Python? ....
32

Let's look at some...

Issues to Consider
when Evaluating
a Programming Language

33

Are capabilities an issue?

Frank Stajano

"Batteries included" philosophy

Standard distribution includes extensive
module library
Many other modules available
34

The Python Standard Library

GUI
strings
regular expressions
database connectivity
HTTP, CGI, HTML, XML
numeric processing
debugger
object persistence
35

Is execution speed an issue?

Probably 10 times slower than a system
language, but ...

Modern processors generally make language speed

a non-issue

Many applications are limited by speed of database

or network connection, not programming language

Ease-of-use makes implementing optimization

algorithms easier possible to beat even C
programs

Easy to write interface to C extension modules for

optimization
36

"In terms of run time and memory

consumption, scripting languages
often turn out better than Java
and not much worse than C or C+
+."
An empirical comparison of C, C++, Java, Perl, Python,
Rexx, and Tcl for a search/string-processing program
University of Karlsruhe, Germany
Technical Report 2000-5, March 10, 2000
http://wwwipd.ira.uka.de/~prechelt/Biblio/jccpprtTR.pdf
37

Is dynamic typing an issue?

"It might seem that the typeless nature of
scripting languages could allow errors to
go undetected, but in practice scripting
languages are just as safe as system
programming languages."
"Scripting: Higher Level Programmingfor the 21st Century" by John K. Ousterhout
IEEE Computer magazine, March 1998 -- http://home.pacbell.net/ouster/scripting.html

38

"I'd been a statically typed bigot for quite a

few years. Four years ago I got involved with
Extreme Programming. ... I liked the
emphasis it put on testing.
About two years ago I noticed I was
depending less and less on the type system
for safety. My unit tests were preventing me
from making type errors.
So I tried writing some applications in
Python, and then Ruby. I found that type
issues simply never arose. My unit tests kept
my code on the straight and narrow. I simply
didn't need static type checking."
Robert C. Martin

39

Is productivity an issue?

"5-10 times productivity (really!)"

Bruce Eckel

"I find that I'm able to program about

three times faster [in Python] than I

could in Java, and I was able to program
in Java about three times faster than I
could in C."

Andy Hertzfeld

"The results indicate that, for the given

programming problem, 'scripting languages'
(Perl, Python, Rexx, Tcl) are more productive
than conventional languages."

University of Karlsruhe, Germany, Technical Report 2000-5, March 2000

40

When a 20,000 line project went to

approximately 3,000 lines overnight,
and came out being more flexible
and robust ... I realized I was on to
something really good.

-- Matthew "Glyph" Lefkowitz

...the lines of Python code were 10%

of the equivalent C++ code.

-- Greg Stein, eShop

41

"Programming is fun again!"

Over and over on comp.lang.python there

are messages:

"Now that I've discovered Python, I enjoy

programming again!"

"Now I am back programming in Java because

the projects I'm working on call for it. But I
wish I was programming in Ruby or Python ..."
Robert C. Martin

42

Is maintainability an issue?

"I realized that the flexibility of

dynamically typed languages makes
writing code significantly easier. Modules
are easier to write, and easier to
change."
Robert C. Martin -- http://www.artima.com/weblogs/viewpost.jsp?thread=4639

"Python excels at rapid creation of

maintainable code"
Bruce Eckel

43

Is weirdness an issue?
"Python's use of whitespace stopped
feeling unnatural after about twenty
minutes.
I just indented code, pretty much as I
would have done in a C program
anyway, and it worked."

Eric S. Raymond
44

How weird is it, really?

"Most people who use Python consider the indentation
syntax to be an important, if not downright critical,
feature of the language. It forces you to write readable
code, which in turn fosters code maintainability. It's a
big win, once you get past the initial shock.
In any structured programming language, the
indentation of blocks really does mean something. Most
Python users think that enforcing consistency in
indentation is not only good software engineering, it's
simple common sense. The end result is code that is so
well laid out that it resembles something akin to
poetry."
Mark Lutz, author of Programming Python

45

Is support an issue?
Python is an "open-source" language.
It has no vendor.
Does that mean we'll have support
problems?
What about...
Vendor longevity?
Consulting & training support?
Books and reference materials?
Tools? IDEs, debuggers, screen-painters?
46

What is "Open-Source"?

a distribution license for source code

source code is available without $$$ charge

code may be changed, customized, enhanced
GPL Gnu Public License
Python license unlike the GPL, you may
distribute a modified version without making
your changes open source.

a development style and a culture...

47

48

The Cathedral & the

Bazaar
Linux is subversive. Who would have thought ... that
a world-class operating system could coalesce as if
by magic out of part-time hacking by several
thousand developers scattered all over the planet,
connected only by the tenuous strands of the
Internet?

Certainly not I... I believed there was a certain

critical complexity above which a more centralized, a
priori approach was required. ... the most important
software needed to be built like cathedrals, carefully
crafted by individual wizards or small bands of
mages working in splendid isolation, with no beta to
be released before its time.
49

The Cathedral & the

Bazaar
The Linux style of development came as a surprise.
No quiet, reverent cathedral-building hererather,
the Linux community seemed to resemble a great
babbling bazaar of differing agendas and approaches
out of which a coherent and stable system could
seemingly emerge only by a succession of miracles.
The Linux world not only didn't fly apart in confusion
-- it seemed to go from strength to strength at a
speed barely imaginable to cathedral-builders.

50

Some open-source products

"LAMP"

Linux
Apache
MySql
PHP | Perl | Python

Apache has overwhelmingly dominated the Web

server market since 1996.
PHP is the most popular Apache module, running on
almost 10 million domains (over a million IP
addresses).
"MySQL threatens to do for databases what Linux
has done for operating systems." Tim O'Reilly

51

Is Open-Source software
used in the Federal
Government?

See earlier list of Python users

NIH, NASA, Navy, Agriculture, Weather

Service

In 2002, a Mitre study found 115

FOSS products in use in DoD

http://egovos.org/pdf/dodfoss.pdf

Why would a Federal agency

use open-source
software? ...

52

Government Computer News

November 20, 2000

The NASA Acquisition Internet Service (NAIS) development team

adopted open-source software several years ago and we plan to
expand its use in the agency-wide procurement system.
We were using a proprietary Web development application that
promised interoperability with another vendors database software. It
failed to interoperate, however... Then we discovered Perl and have
been using it for the last five years to develop and support all NAIS
applications.
Recently, price restructuring for a commercial DBMS threatened to
consume most of the NAIS budget. We decided to convert NAIS to
MySQL. Our tests showed MySQL could perform NAIS functions faster.
Cost of the optional technical support was about 1 percent of that for
the commercial product.
Technical support for MySQL has been excellent when we needed it,
plus there are hundreds of Web sites that offer free help and support
for such open-source products.
We plan to evaluate the Apache HTTP Server to correct limitations of
the commercial Web server we currently use.
53

eGov & Open-Source

Center of Open Source & Government

(http://egovos.org/index.html)

EGOVOS - high-level international

conference on OSS ("Libre Software"),
interoperability and open standards in
government

October 2002 & March 2003 - Washington, DC

EGOVOS3 - 24-26 November, 2003 at
UNESCO headquarters in Paris
54

The Open Source Reference Book 2003 What Local/National Governments, the
Defense Establishment, and The Global
1000 Need To Know About Open Source
Software (November 2003)

... will provide a Generally Regarded As Safe

(GRAS) list of Open Source software to identify
mature and useable Open Source projects

... will list Open Source software that is NIAP*

or Common Criteria evaluated

*NIAP: National Information Assurance Partnership NIST security

certification
55

So... Is Open-Source Safe?

Vendors and products vary widely in both

the commercial and open-source arena.

The fact that a piece of software is

commercial is no guarantee of its quality,
or of its vendor's long-term survival.

The best open-source software is as good

as the best commercial software.
56

The Bottom Line

Each product and vendor should be

evaluated on its own merits,
regardless of whether it is
commercial or open-source.

Python is in the same league as the

best software anywhere, commercial
or open-source.
57

Is vendor longevity an
issue?
What if Guido got run over by a bus?

58

The Python Software

Foundation

A non-profit organization for advancing opensource technology related to Python

Holds Python's intellectual property rights.

Produces the core Python distribution,

available to the public free of charge.

Establishes PSF licenses, ensuring the rights

of the public to freely obtain, use,
redistribute, and modify intellectual property
held by the PSF.
59

Is mindshare an issue?
The Python community is very
active and growing rapidly

International Python Conference

EuroPython conference

IPC - in USA since 1992

in Europe since 2002

Python for Scientific Computing

Workshop

SciPy - in USA since 2002

60

Newsgroup
Activity

comp.lang.*
December
2002

Statistics compiled
by
Aaron K. Johnson.

java
c++
c
perl
python
basic
ruby
lisp
tcl
pascal
smalltalk
fortran
cobol

26953
19913
13874
10486
9647
7909
6466
6132
5256
4229
2398
2355
1845
61

1
2
3
4
5
6
July 2003
7
8
9
10
based on the
11
number of hits
12
returned by a
Google search 13
14

TIOBE
Popularity of
Programming
Languages
Index

Java
44.3
C 36.8
C++
33.2
Perl
18.3
(Visual) Basic
15.5
PHP
7.6
SQL
6.0
C#
3.5
JavaScript 3.3
Delphi/Pascal/Kylix
3.1
Python 2.6
COBOL 2.3
SAS
2.2
Fortran
1.9

Index is available at

62

Is online support an issue?

comp.lang.python -- Outstanding!!

http://groups.google.com/groups
?&group=comp.lang.python

63

Consulting and Training

Resources?

Not much!

Python is probably too easy-to-learn and

easy-to-use to support much of a training/
consulting industry. You can learn it out of a
book!
A couple of useful consulting resources...

Zope Corp.
Fourthought, Inc. - XML tools for Python and XML
and web-based applications.

64

Is Ease-of-Learning an
issue?

Python is famously easy to use and easy to

learn.

I talked my colleagues into using Python for

our Computer Science 1 course this fall. ... In
the past I would be swamped during office
hours with students wanting help deciphering
C++ compiler errors. This semester almost
nobody has stopped by for syntax issues.

-- Dave Reed on Python In Education mailing list

65

Books & Reference

Materials?

66

Online Materials?

Python distribution includes:

Tutorial, Language Reference

Extensive Standard Library documentation

"How to Think Like a Computer Scientist with

Python" http://greenteapress.com/thinkpython/

"Python Programming an Introduction to Computer

Science" http://mcsp.wartburg.edu/zelle/python/

"Dive Into Python"

http://diveintopython.org/index.html

Too many others to list...

67

Tools? - IDEs
IDLE comes
with
Python
WingIDE
excellent
IDE with
visual
debugger
$35 and $180 -- http://wingide.com/

68

Visual Python
Python plug-in for Visual Studio .NET.
Python-specific features within the familiar Visual Studio environment.
Visual Python integrates seamlessly with Visual Studio .NET, allowing
programmers to leverage features of Microsoft's popular development
tool suite.
http://www.activestate.com/Products/Visual_Python
69

Tools? Screen Painters

A screenshot of wxDesigner

70

What Do the Experts Think

of Python?

71

Bruce Eckel
His book Thinking in C++ was given
the Software Development Jolt Award
for best book published in 1995.
Thinking in Java received Java World Reader's Choice Award
and Java World Editor's Choice Award for best book, the
Java Developer's Journal Editor's Choice Award for books,
the Software Development Productivity Award in 1999, the
third edition received the Software Development Magazine
Jolt award for best technical book, 2002.
One of "the industry's leading lights" ( Windows Tech
Journal, September 1996).
72

Why I Love Python

2001 Bruce Eckel
MindView, Inc.
Training & Consulting
5343 Valle Vista
La Mesa, CA 91941
Bruce@EckelObjects.com
www.MindView.net
http://64.78.49.204/pub/eckel/LovePython
.zip

73

The language you speak affects what

you can think. "Python fits my brain."

Python excels at rapid creation of

maintainable code

Programmer productivity is the most

important thing. 5-10 times
productivity (really!)
74

Simplicity really does make a

difference.
I can remember many Python idioms
because theyre simpler.
One more reason I program faster in
Python.

I still have to look up how to open a file

every time I do it in Java.
75

Python & The Tipping

Point
It is possible to write
programs to
automate every
task. But you dont.
Python makes it easy
enough
76

Eric S. Raymond
Well-known Unix guru,
Linux advocate, and
author
The Cathedral and the Bazaar
www.catb.org/~esr/writings/cathedral-bazaar/cathedralbazaar/
The New Hacker's Dictionary
http://www.jargon.8hz.com/jargon_toc.html
Homesteading the Noosphere
http://www.firstmonday.dk/issues/issue3_10/raymond/
77

Why Python?
This could be an opportunity to get some
hands-on experience with Python...
I charged ahead and roughed out some
code ...

http://pythonology.org/success&story
=esr
78

I noticed I was generating working code

nearly as fast as I could type. When
you're writing working code nearly as
fast as you can type, it generally
means you've achieved mastery of the
language. But that didn't make sense,
because it was still day one ...
This was my first clue that, in Python, I
was actually dealing with an
exceptionally good design.
79

Not that it took me very long to learn

the feature set.
This reflects another useful property of
Python: it is compact -- you can hold
its entire feature set (and at least a
concept index of its libraries) in your
head.
80

The long-term usefulness of a language

comes from how well and how
unobtrusively it supports the day-to-day
work of programming, which consists
not of writing new programs, but mostly
reading and modifying existing ones.
So the real punchline of the story is this:
weeks and months after writing
fetchmailconf [my Python program], I
could still read the code and grok what it
was doing without serious mental effort.
81

Martin C. Brown

Author and Perl expert

Perl: The Complete Reference

Perl Annotated Archives

... and ...

... and ...

82

Nicholas Petreley
ComputerWorld columnist

One of my favorite programming

languages is Python. It seems I don't go
a week these days without someone
asking me what I know about Python, so
it seems to be gaining quite a following
in mainstream IT.
November, 2002

83

The Bottom Line...

"Use the Best Tool for the Job: Put Both a
Scripting and Systems Language in Your
Toolbox"- Bill Venners
http://www.artima.com/commentary/langtool.html

Python would be a useful tool in the

toolboxes of our developers, DBAs, and LAN
administrators,
for situations where....
84

A command-language is too underpowered, and a systems

programming language would be
overkill.

Speed and minimizing effort are

important

One-time, throw-away programs

Internal utilities
Prototyping
Test scaffolding

85

Cross-platform portability is important

System administrators need learn only

one scripting language
Prototype/develop on one platform,
deploy on another (e.g. Windows NT and
Unix)

Readability & maintainability are

important
The Python people also piped up to say

XMLeverything's
processingjust fine here but then

they always do. I really must learn that

language.
XML Is Too Hard For Programmers
Tim Bray, co-author of the original XML 1.0 spec

86

Ease-of-learning is important
An application is written in four different
languages (Java, C, Perl, and Unix shellscript) because it was built by four
different developers who were expert in
four different languages.
Everybody knows this is a problem, but
nobody has time to learn another
language.
One solution -- a single common language
that is both powerful enough to handle a
wide variety of tasks, and easy enough to 87

More Online Information

http://www.python.org is the Python home page

Chapter 1 of Internet Programming with Python is

available online. It discusses reasons for using
Python.
http://www.fsbassociates.com/books/pythonchpt
1.htm

Python Compared to Other Languages

http://www.python.org/doc/Comparisons.html
88

The End
Questions? Comments?

89