Porting
to
Python
3.

0
Anthony
Baxter

anthony@python.org
arb@google.com
 these
slides:

www.interlink.com.au/anthony/tech/talks/OSCON2008
 introduction
www.interlink.com.au/anthony/tech/talks/OSCON2008
www.interlink.com.au/anthony/tech/talks/OSCON2008
 past
release
mgr

www.interlink.com.au/anthony/tech/talks/OSCON2008
not
for
2.6/3.0
www.interlink.com.au/anthony/tech/talks/OSCON2008
 caveat

www.interlink.com.au/anthony/tech/talks/OSCON2008
www.interlink.com.au/anthony/tech/talks/OSCON2008
 spokesman

www.interlink.com.au/anthony/tech/talks/OSCON2008
 spokesman

www.interlink.com.au/anthony/tech/talks/OSCON2008
 duh
www.interlink.com.au/anthony/tech/talks/OSCON2008
 or
not...
www.interlink.com.au/anthony/tech/talks/OSCON2008
FAIL
python
3.0
Python
3000
aka
Py3k
aka
p3yk
3.0
where
we
break
all
your
code
it’s
not
that
bad
 +
in
a
good
cause
promise!
 backwards

incompatible
quite
a
lot
imprecise
more
precisely:
almost
every
program
will
need
changes
almost
every
program
will
need
changes
↓.
but
1st
some
justification
some
rationalizations
Python:
16
years
old
first
release
1991
b/w
compat

important
 pre‐1.0
lots
of
changes
1.0
➙
2.0
list.append()
socket.bind()
made
them
agree

with
docs
2.0
➙
2.6
no
b/w
incompatible

changes
well,
none
important
to
published
APIs,
anyway
play
with
internals
get
hurt
like
all
16
year
olds
annoying
features
3.0
the
chance
to
fix

things
mistaikes
duplication
duplication
just‐plain‐ugly
mistakes
coerce()
no
use
to
anyone
backticks
duplication
TOOWTDI
there’s
only
one
way
to
do

it
not
true
 map()
list
comprehensions
string
methods

string
functions
apply(foo,
args,
kwargs)

foo(*args,
**kwargs)
email
package

rfc822
module
how
did
this
happen?
we
learned
&c
the
ugly
`backticks`
repr()
 ew
hard
to
read
formatted
wrong
 reduce()
sum()
replaces
95%
of
uses
print
>>
sys.stderr,
“oh
noes”
changes
add
bloat
smaller
language
==
better
language
“python
fits
in
your
brain”
 easier
to
use
easier
to
learn
who
wins?
NCSS
2008
professional
programmers
 builtins
by
version
1.0.1
(1994):
40
1.5.2
(1999):
54
2.1
(2001):
69
2.5
(2006):
82
language
complexity

$
things
we’ve
learned
stdio
not
the
best
optional
unicode
==
badness
so...
porting
to
3.0
2.x
↓
2.6
↓
3.0
2.6
interim
release
same
day
as
3.0
 python
2.6
‐3
turns
on
warnings
>>>
{}.has_key(1)
__main__:1:
DeprecationWarning:
dict.has_key()
not
supported

in
3.x;
use
the
in
operator
 python2.6
from
__future__
 python2.6

from
future_builtins
import
enables
some
backports
 2to3
ships
with
2.6,
3.0
mechanical
rewrites
handles
a
lot
never
going
to
be
perfect
unit
tests
onto
the
changes
language
changes
removed:
 2to3

<>
↓
!=
 2to3

the
most
basic
change
2to3
handles
it
 2to3

Note
this

 2to3

`backticks`
↓
repr()
 removed:
print
statement
OMGWTFBBQ?
print()
now
a
function
print
a,
b,
c
⇓
print(a,
b,
c)
 print
>>
fp,
someoutput
⇓
print(someoutput,
file=fp)
 print
>>
abomination
“C++
does
it”
that
makes
it
worse
 print
somevalue,
⇓
print(somevalue,
end=˝
˝)
 new
print
feature:
customize
separators
tag=˝blink˝
print(˝<˝,
tag,
˝>˝,
sep=˝˝)







⇓
˝<blink>˝
print
˝,˝.join((a,b,c,d))
⇓
print(a,b,c,d,
sep=˝,˝)
 2.6
from
__future__
import
print_function
new
string
formatting
str.format()
method
“name:
{0}
{1}”.format(first,last)
fmt
=
“{num}
{item}”
fmt.format(num=3,
item=”sheep”)
format()
builtin
>>>
format(3.0,
"06.1f")
'0003.0'
new
keyword:
nonlocal
you
ain’t
from
round
here
Python
2.x:
can
read
from

enclosing
scopes
can’t
write
to
them
that’s
where
nonlocal

comes
in
def
enclosing():




a=4




def
inner():








nonlocal
a








a
=
5




inner()




print(a)

5
new
keywords:
as
True
False
None
>>>
False=1


File
"<stdin>",
line
1
SyntaxError:
assignment
to
keyword
‘as’
was
already
a
kinda‐
keyword
basic
types
strings
say
Olá
to
űņĩçôđë
uniçode
strings
default
2.x
mixed
strîng
types
disastrous
switching
produced
šubtle

mistakes
 2to3

basestring
gone
spelt
‘str’
now
 2.6
>>>
from
__future__
import
unicode_literals
>>>
type('')
<type
'unicode'>
but
sometimes
you
want
bytes
network
protocols
binary
file
formats
bytes()
to
the
rescue
 byte
literals

b˝HELO
mail.python.org˝

uses
default
encoding
:‐(
 bytes([0x0A,
0x0B,
0x20])

any
iterable
producing
0‐256
bytes(somestring,
encoding)
explicit
is
better
than
implicit
particularly
in
this
case
encoding
guessing
fraught
with
peril
repr()
produces
unicode
 new
ascii()
builtin
and
“%a”
string
format
one
last
point
on
unicode:
non‐ASCII
identifiers


>>>
børkbørkbørk=True


>>>
print(börkbörkbørk)


True
(source
files:
UTF‐8
by

default)
numbers
 2to3

no
more
long()
ints
are
longs
by
default
>>>
sys.maxint
Traceback
(most
recent
call
last):


File
"<stdin>",
line
1,
in
<module>
AttributeError:
'module'
object
has
no

attribute
'maxint'
 2to3

no
more
123L,
either
integer
division
>>>
1/2
0.5
 old‐style:




>>>
1//2



0
true
division
has
been

“coming
soon”
since
2.2
...
in
2001
turn
on
new
division

python2.x
‐Qnew
dicts
 2to3

obvious
change:
death
to
has_key()
↓
key
in
dictobj
also
dict.iter*
 2to3

also
dict.iter*
 instead:
dictionary
views
 stolen
from
java
(everyone
steals
from
everyone)
the
open
source
way
 set‐like
view
of
underlying
data
not
a
copy
behaves
as
you’d
expect
 for
k
in
dict:
for
k,v
in
dict.items():
....
unless
you’re
being
evil...
if
you
need
a
list
of
keys:
list(d.keys())
 modifying
a
dict
while

iterating
still
undefined
behaviour
sets
set
literals
{1,
2,
3,
4}
dicts,
without
the
colons
but
ambiguity:
 {}?
empty
dict?
empty
set?
{}
==
dict()

empty
set:
set()
set
comprehensions
iterables
 2to3

next()
 
__next__()
 new
feature
unpacking:
first,
*rest,
last
=
resultlist
head,
*rest
=
somelist
*junk,
tail
=
somelist
 2to3

map(),
filter(),
zip()
↓
iterators
exceptions
no
more
string
exceptions
exceptions
must
derive
from

BaseException
 2to3

2‐arg
raise

raise
MyExc,
val
↓
raise
MyExc(val)
 2to3

3‐arg
raise:
raise
exc,
val,
tback









↓
raise
exc(value).with_traceback(tback)
sys.exc_value
&c
gone
not
thread
safe
sys.exc_info()
 2to3

except
exc,
value
↓
except
exc
as
value
classes
new
classes
only
old‐style
classes
 new
style
only
old‐style
classes
 class
decorators


@RegisteredMuppet


class
Kermit():








.....
@decorator
class
MyClass:




pass












↓
class
MyClass:




pass
MyClass
=
decorator(MyClass)
metaclasses
kinda
funky

metaclasses
inherited
 on
the
subject

class
A(B,
C,
metaclass=M):




....
metaclasses
in
3.0
more
powerful

mwahahahahahaha
not
going
into
detail

PEP
3115
ABCs
Abstract
Base
Classes
the
problem
duck
typing
g
sometimes
you
care
 assert
is‐a‐sequence
hasattr(obj.__contains__)
?
hasattr(obj.__iter__)
?
hasattr(obj.__getitem__)
?
from
collections
import
MutableSequence
MutableSequence.register(MyMagicSequence)
m
=
MyMagicSequence()

#
these
are
both
True:
isinstance(m,
MutableSequence)
issubclass(MyMagicSequence,
MutableSequence)
MutableSequence

Abstract
Base
Class
collections,
numbers
numbers
‐
PEP
3141
 define
your
own:
class
TVCharacter(metaclass=abc.ABCMeta):




pass

class
Muppet:




pass

TVCharacter.register(Muppet)
 abstract
methods

class
TVCharacter(metaclass=abc.ABCMeta):




@abc.abstractmethod




def
getPerformer(self):








pass
class
Muppet(TVCharacter):




pass

>>>
m
=
Muppet()
Traceback
(most
recent
call
last):


File
"<stdin>",
line
1,
in
<module>
TypeError:
Can't
instantiate
abstract
class

Muppet
with
abstract
methods
getPerformer
>>>
class
MyRational(numbers.Rational):

...


pass
...

>>>
m
=
MyRational()
Traceback
(most
recent
call
last):


File
"<stdin>",
line
1,
in
<module>
TypeError:
Can't
instantiate
abstract
class
MyRational
with

abstract
methods
__abs__,
__add__,
__ceil__,
__eq__,

__floor__,
__floordiv__,
__le__,
__lt__,
__mod__,
__mul__,

__neg__,
__pos__,
__pow__,
__radd__,
__rfloordiv__,
__rmod__,

__rmul__,
__round__,
__rpow__,
__rtruediv__,
__truediv__,

__trunc__,
denominator,
numerator
 @abstractproperty
behaves
as
you’d
expect
functions
keyword
only
args
can’t
be
positional
 buggy
example:
def
doMagic(arg1,
arg2,
flag=False):



....

doMagic(1,2,3)
#
flag
=
3
??!?
 the
change:
keyword
args
after
*args
def
doMagic(arg1,
arg2,
*extras,
flag=False):




if
extras:








raise
TypeError(“oops”)




....
def
doMagic(arg1,
arg2,
*,
flag=False):




....

>>>
doMagic(1,2,3)
Traceback
(most
recent
call
last):


File
"<stdin>",
line
1,
in
<module>
TypeError:
doMagic()
takes
exactly
2

positional
arguments
(3
given)
annotations
attaching
metadata
to

arguments
 syntax:
def
func(arg:
expression)
‐>
returnValue:




pass
expression
can
be
anything
 documentation:
def
doEvil(plan:
“a
plan”)
‐>
“evil
results”:




....
 types
def
PumpGas(gallons:
float,













price:
float)
‐>
float:
 more
complex
still:


def
processFiles(*files:
“one
or
more
filenames”,





delete:
“delete
when
done”
=
False)
‐>
“a
boolean”:
Python
ignores
annotations
stored
in
__annotations__
3rd
party
tools
 e.g.
optimizers,
docs,
IDEs,

language
bridges,

typechecking,
...
 2to3

funcobj.func_*
↓
s/func_(.*)/__\1__/
files
text
vs
binary
text
files
must
have
encoding
produces
unicode
binary
files
produce

bytestrings
new
I/O
layer
stdio
is
std
but
kinda
...not
good
 raw
I/O
buffered
I/O
text
I/O
 ABCs

TextIOBase
BufferedIOBase
RawIOBase
 e.g.

FileIO
SocketIO
BytesIO
all
implement
RawIOBase
you
probably
don’t
need
to

care
except
for:
>>>
open('/tmp/fozzie',
'w')
<io.TextIOWrapper
object
at
0x3c2970>

>>>
open('/tmp/fozzie',
'wb')
<io.BufferedWriter
object
at
0x3b7c90>

>>>
open('/tmp/fozzie',
'rb')
<io.BufferedReader
object
at
0x3c2970>
modules
absolute
imports
 import
foo
always
imports
from
the
top

level
import
from
same
package:

from
.
import
foo
 import
from
parent:

from
..
import
foo
 2.6
from
__future__

import
absolute_import
builtins
 2to3

already
covered:
map,
filter,
zip,
print
 2to3

new
builtin:
exec()
replaces
exec
stmt
 2to3

callable()
just
call
it
or
check
for
__call__
 2to3

buffer()
↓
memoryview()
buffer()
terminally
broken
stdlib
 goals:
PEP
8
compliance
some
structure
cleaning
out
cruft
 PEP
0008
“Modules
should
have
short,

all‐lowercase
names.”
...oops
BaseHTTPServer
cPickle
cStringIO
HTMLParser
....
either
renamed,
or

structure
current
structure:
duplications
 and
near
duplication

bsddb
gdbm
dbm
dumbdbm
urllib
vs
urllib2
cleaning
this
stuff
up
new
packages:
http
html
xmlrpc
json
dbm
urllib
 lots
of
renames

cStringIO
 
io.StringIO
 lots
of
renames

SocketServer
 
socketserver
httplib
→
http.client
urllib
and
urllib2
merged
new
package
new
package:
http
http
contents
client
server
cookies
cookiejar
cruft
“batteries
included”
some
of
the
batteries
old
and
busted
 lots
of
purges
e.g.
old
email
modules
old
hash
modules
old
platform
modules
 thread
gone
use
threading
UserDict
and
friends
↓
subclass
from
dict
2to3
only
catches
some
of

these
warn
on
import
in
2.6
 porting

approach
take
2.5
code
get
working
on
2.6
turn
on
‐3
flag
while
True:





run
through
2to3





run
unit
tests
under
3.0





fix
2.x
code
 eventually...
drop
2.x
version
switch
to
3.x
version
things
that
are
not

good
fwd
compat
of
modules
 ugly
fix:

if
sys.version_info
<
(3,0,0):






from
httplib
import
HTTPConnection
else:


from
http.client
import
HTTPConnection
pickle‐compatibility
 3rd
party
pkgs

also
need
updates

C
API
changed,
too
 if
you
are
a
pkg
author

please
don’t
break
your
API
in

3.0
compat
version
 temptation:
“since
3.x
is
breaking

everything
anyway”
your
users
will
hate
you
double‐upgrade,
yuk
things
not
in
py3k
case
insensitivity
death
to
lambda
implicit
self
macros
not
a
complete
rewrite
not
rewriting
the
new
VM
other
implementations
are

doing
that
CPython
is
reference

implementation
the
future
3.0
is
the
one
chance
get
things
right
don’t
expect
a
4.0
2.x
not
going
away
 2.7 *

more
3.x
compat
 2.x.y
bugfix
releases
for
as
long
as
needed
your
future
plans
please
try
3.0
betas
(it’s
easier
to
get
bugfixes
in)
think
about
3.0
2.x
is
not
going

away
 2.x
is
not

going
away
will
continue
to
be
supported
2.7
in
2010
maintenance

releases
both
2.x
and
3.x
that’s
it
for
now

Q&A