WELCOME TO C C LAB

1
WELCOME TO CC LAB

• The Recursion, Pickling, and Debugging Edition

2
ByBen and Ivy!
3

• This weeks lesson is brought to you by

4
This weeks lesson is brought to you by
5
The Ministry of Funny Walks
6
Because Life is Too ShortTo Walk Through it
Normally
7
And without knowing what RECURSION is
8
WHAT IS RECURSION
9
A FUNCTION WHOSEDEFINITION CONTAINSA CALL TO
THAT SAME FUNCTION
10
recursion ri-kur-zhuhn

• return
• a computer programming technique involving the
  use of a procedure, subroutine, function, or
  algorithm that calls itself one or more times
• See recursion

11
recursion ri-kur-zhuhn

• Recursion simply means applying a
• function as a part of the definition of
• that same function.

12
Thus the definition of GNU (the source of much
free software) is said to be recursive because
GNU stands for 'GNU's Not Unix'. (GNU is part
of the definition of GNU!)
13
Recursion is found in Mathematics and Nature
14
Recursion in Mathematics
Factorial n! Fibonacci sequence
15
Fractals
A fractal is generally "a rough or fragmented
geometric shape that can be subdivided in parts,
each of which is (at least approximately) a
reduced-size copy of the whole, a property
called self-similarity http//en.wikipedia.org/wi
ki/Fractals
16

• These objects display self-similar structure
  over an extended, but finite, scale range.
  Examples include clouds, snow flakes, crystals,
  mountain ranges, lightning, river networks,
  cauliflower or broccoli, and systems of blood
  vessels and pulmonary vessels.

17
Recursion in Python
Recursion can be a powerful technique for
solving some types of problems.
18
Recursion in Python
The key to making this work is that there must be
a terminating condition so that the function
branches to a solution at some point and breaks
the cycle.
19
Recursion in Python
(The GNU definition fails this test and so gets
stuck in an infinite loop).
20
THUS, IN PYTHON
21
DAILY DOUBLE
22
def factorial(n) if n 1 return
1 else return n factorial(n-1)
23
WHAT IS FACTORIAL?
24
OR
25
WHAT IS AN EXAMPLE OF A RECURSIVE FUNCTION?
26
If you bring up the topic of recursion to a
Computer Science major, they might refer to the
Tower of Hanoi
27
The Tower of Hanoi puzzle was invented by the
French mathematician Edouard Lucas in 1883.
28
The problem We are given a tower of 8 disks,
initially stacked in increasing size on one of 3
pegs. The objective To transfer the entire
tower to one of the other pegs, moving only one
disk at a time and never a larger one onto a
smaller.
29
The procedure In order to move a tower of h
disks from a peg f onto a peg t, r being the
remaining third peg Step 1 If h gt 1 then
first use this procedure to move the h - 1
smaller disks from peg t to peg r. Step 2 Now
the largest disk, i.e. disk h - 1 can be moved
from peg f to peg t. Step 3 If h gt 1 then
again use this procedure to move the h - 1
smaller disks from peg r to peg t.
30
Although it still may sound complicated, dont
get too bogged down on these details. The gist
here is that the solution to the Tower of Hanoi
lies in recursion.
31
In Python code
def move(d, f, t) print 'moving
disk print d s --gt s' (d, f, t) def
hanoi(h, f, t, r) if h gt 0
hanoi(h-1, f, r, t) move(h-1, f, t)
hanoi(h-1, r, t, f)
32
I know what youre thinking.
33
When will I ever use recursion???
34
Well, recursion is very useful is in processing
lists.
35
Consider the trivial case of printing each
element of a list of strings using a function
printList
def printList(L) if L print
L0 printList(L1)
Take a moment to talk about what this does. What
is the for loop equivalent?
Taken from http//www.freenetpages.co.uk/hp/alan.g
auld/tutrecur.htm
36
For a simple list that's a trivial thing to do
using a simple for loop. But consider what
happens if the List is complex and contains other
lists within it.
Taken from http//www.freenetpages.co.uk/hp/alan.g
auld/tutrecur.htm
37
If we can test whether an item is a List using
the built-in type() function and if it is a list
then we can call printList() recursively. If it
wasn't a list we simply print it.
Taken from http//www.freenetpages.co.uk/hp/alan.g
auld/tutrecur.htm
38
def printList(L) if its empty do nothing
if not L return if it's a list call
printList on 1st element if type(L0)
type() printList(L0) else
no list so just print print L0
now process the rest of L
printList(L1)
Taken from http//www.freenetpages.co.uk/hp/alan.g
auld/tutrecur.htm
39
Now if you try to do that using a conventional
loop construct you'll find it very difficult.
Recursion makes a very complex task
comparatively simple. There is a catch (of
course!). Recursion on large data structures
tends to eat up memory so if you are short of
memory, or have very large data structures to
process the more complex conventional code may be
safer.
Taken from http//www.freenetpages.co.uk/hp/alan.g
auld/tutrecur.htm
40

• Things to Remember
• Recursive functions call themselves within
  their own definition
• Recursive functions must have a non-recursive
  terminating condition or an infinite loop will
  occur.
• Recursion is often, but not always, memory
  hungry

Taken from http//www.freenetpages.co.uk/hp/alan.g
auld/tutrecur.htm
41
And Now for Something Completely Different
42
How To Make A Pickle
43

• Data Persistence with pickle
• Python Object Serialization

44

• If you're working with objects and you want to
  transfer them to another computer or save them
  from the current application for reloading into
  that application or another one on the same
  computer later on, you need to serialize the
  objects

45

• The pickle module can turn many Python data types
  into a stream of bytes and then recreate the
  objects from the bytes.

46

• The pickle module implements a fundamental, but
  powerful algorithm for serializing and
  de-serializing a Python object structure.

47

• Pickling is the process whereby a Python object
  hierarchy is converted into a byte stream, and
  unpickling is the inverse operation, whereby a
  byte stream is converted back into an object
  hierarchy.

48

• Data within OO programming languages is stored in
  the heap and as its name implies the heap isn't
  neat.

49

• In the heap there's pointers and information all
  over the place and even if you save an object
  correctly into a file or database, without extra
  information you can't read it back properly.

50

• Serialization is the process that adds the extra
  information you'll need

51

• The pickle module has an optimized cousin called
  the cPickle module.
• As its name implies, cPickle is written in C, so
  it can be up to 1000 times faster than pickle.

52

• However it does not support subclassing of the
  Pickler() and Unpickler() classes, because in
  cPickle these are functions, not classes.

53

• Most applications have no need for this
  functionality, and can benefit from the improved
  performance of cPickle.

54

• Other than that, the interfaces of the two
  modules are nearly identical.

55
(No Transcript)
56

• Warning The pickle module is not intended to be
  secure against erroneous or maliciously
  constructed data.
• Never unpickle data received from an untrusted or
  unauthenticated source.

57

• By default, the pickle data format uses a
  printable ASCII representation.
• This is slightly more voluminous than a binary
  representation.

58

• The big advantage of using printable ASCII is
  that for debugging or recovery purposes it is
  possible for a human to read the pickled file
  with a standard text editor.

59

• There are currently 3 different protocols which
  can be used for pickling.
• Protocol version 0 is the original ASCII
  protocol and is backwards compatible with earlier
  versions of Python.
• Protocol version 1 is the old binary format
  which is also compatible with earlier versions of
  Python.
• Protocol version 2 was introduced in Python
  2.3. It provides much more efficient pickling of
  new-style classes.

60

• HIGHEST_PROTOCOL
• The highest protocol version available.
• This value can be passed as a protocol value.

61

• The dumps method writes any object to a string,
  and the loads method reads the object back from a
  string.

62

• The reloading program does need to have any
  imported modules that were used within the dumped
  object, of course.

63

• If you want to write objects straight to a file,
  you can use the dump method instead, and reload
  using the load method.

64

• dump(obj, file, protocol)
• Write a pickled representation of obj to the open
  file object file. This is equivalent to
  Pickler(file, protocol).dump(obj).
• If the protocol parameter is omitted, protocol 0
  is used. If protocol is specified as a negative
  value or HIGHEST_PROTOCOL, the highest protocol
  version will be used.
• file must have a write() method that accepts a
  single string argument. It can thus be a file
  object opened for writing, a StringIO object, or
  any other custom object that meets this
  interface.

65

• import cPickle
• import myuser
•  
• team
• team.append(myuser.User("Lisa","lisa_at_wellho.net"))
  
• team.append(myuser.User("Leah","leah_at_wellho.net"))
  
• team.append(myuser.User("Christine","christine_at_wel
  lho.net"))
•  
• print "storing this team"
• for member in team
•     print "Team member",member.getname(),\
•         "has email",member.getemail()
•  
• fh open("teamfile.txt","w")
• dump to a file, but dumps to a string
• Use dumps if you want to pickle and store in a
  database

66

• dumps(obj, protocol)
• Return the pickled representation of the object
  as a string, instead of writing it to a file.
• If the protocol parameter is omitted, protocol 0
  is used. If protocol is specified as a negative
  value or HIGHEST_PROTOCOL, the highest protocol
  version will be used.

67

• load(file)
• Read a string from the open file object file and
  interpret it as a pickle data stream,
  reconstructing and returning the original object
  hierarchy. This is equivalent to
  Unpickler(file).load().
• file must have two methods, a read() method that
  takes an integer argument, and a readline()
  method that requires no arguments. Both methods
  should return a string. Thus file can be a file
  object opened for reading, a StringIO object, or
  any other custom object that meets this
  interface.
• This function automatically determines whether
  the data stream was written in binary mode or not.

68

• import cPickle
• import myuser
•  
• fh open("teamfile.txt","r")
• load from a file.
• use loads to load from a string (which you
• may in turn have read back from a database)
• teamstored cPickle.load(fh)
• fh.close()
•  
• print "Let's see what we just loaded"
• for member in teamstored
•     print "Team member",member.getname(),\
•         "has email",member.getemail()

69

• noload()
• This is just like load() except that it doesn't
  actually create any objects. This is useful
  primarily for finding what's called persistent
  ids that may be referenced in a pickle data
  stream.
• Note the noload() method is currently only
  available on Unpickler objects created with the
  cPickle module. pickle module Unpicklers do not
  have the noload() method.

70

• What can be pickled and unpickled?

71

• The following types can be pickled
• None, True, and False
• integers, long integers, floating point
  numbers, complex numbers
• normal and Unicode strings
• tuples, lists, sets, and dictionaries
  containing only picklable objects
• functions defined at the top level of a module
• built-in functions defined at the top level of
  a module
• classes that are defined at the top level of a
  module
• instances of such classes whose __dict__ or
  __setstate__() is picklable

72

• The term "shelving" is often used in association
  with pickling.
• Objects pickled can be stored in strings, files,
  databases ... and the shelve module allows you to
  use an anydbm 'simple' database for the purpose.

73

• For the benefit of object persistence, the pickle
  module supports the notion of a reference to an
  object outside the pickled data stream.
• Such objects are referenced by a persistent id,
  which is just an arbitrary string of printable
  ASCII characters.

74

• To define external persistent id resolution, you
  need to set the persistent_id attribute of the
  pickler object and the persistent_load attribute
  of the unpickler object.

75

• To pickle objects that have an external
  persistent id, the pickler must have a custom
  persistent_id() method that takes an object as an
  argument and returns either None or the
  persistent id for that object.
• When None is returned, the pickler simply pickles
  the object as normal.
• When a persistent id string is returned, the
  pickler will pickle that string, along with a
  marker so that the unpickler will recognize the
  string as a persistent id.

76

• To unpickle external objects, the unpickler must
  have a custom persistent_load() function that
  takes a persistent id string and returns the
  referenced object.

77

• import pickle
• from cStringIO import StringIO
• src StringIO()
• p pickle.Pickler(src)
• def persistent_id(obj)
• if hasattr(obj, 'x')
• return 'the value d' obj.x
• else
• return None
• p.persistent_id persistent_id
• class Integer
• def __init__(self, x)
• self.x x
• def __str__(self)
• return 'My name is integer d' self.x

78

• For the simplest code, use the dump() and load()
  functions.
• Note the code that follows that a
  self-referencing list is pickled and restored
  correctly.

79

• import pickle
• data1 'a' 1, 2.0, 3, 46j,
• 'b' ('string', u'Unicode string'),
• 'c' None
• selfref_list 1, 2, 3
• selfref_list.append(selfref_list)
• output open('data.pkl', 'wb')
• Pickle dictionary using protocol 0.
• pickle.dump(data1, output)
• Pickle the list using the highest protocol
  available.
• pickle.dump(selfref_list, output, -1)
• output.close()

80

• The following example reads the resulting pickled
  data.
• When reading a pickle-containing file, you should
  open the file in binary mode because you can't be
  sure if the ASCII or binary format was used.

81

• import pprint, pickle
• pkl_file open('data.pkl', 'rb')
• data1 pickle.load(pkl_file)
• pprint.pprint(data1)
• data2 pickle.load(pkl_file)
• pprint.pprint(data2)
• pkl_file.close()

82
(No Transcript)
83
Using the Python Debugger Module
or Pdb for short
84
Debugging - the process of locating and fixing
bugs in your code - is an important process that
all programmers must learn. Having a debugging
strategy or, better yet, knowing how to use a
debugging tool can save you time and headaches.
85
The debugger module for Python is called
pdb. To use it, include the following at the
top of your code import pdb
86
Find the point in your program at which youd
like to start debugging, or tracing, and write
the line pdb.set_trace()
87
Now run your code as usual.
88
When your program encounters the line with
pdb.set_trace() it will

• Stop
• Display the "current statement" (that is, the
  line that will execute next) and
• Wait for your input.

You will see the pdb prompt (Pdb)
89
At the (Pdb) prompt, press "n" (for "next") on
your keyboard, and then press the ENTER key.
This will tell pdb to execute the current
statement. Keep doing this -- pressing "n", then
ENTER.
90
Eventually you will come to the end of your
program, and it will terminate and return you to
the normal command prompt.You've just done your
first debugging run!
91
(No Transcript)
92

Quitting with q
Quitting the Pdb at any time is easy. When you
see the (Pdb) prompt, just press "q" (for "quit")
and the ENTER key.
93
Printing with p
The most useful thing you can do at the (Pdb)
prompt is to print the value of a variable.
Here's how to do it. When you see the (Pdb)
prompt, enter "p" (for "print") followed by the
name of the variable you want to print. And of
course, you end by pressing the ENTER key.
94
For example, you can print the value of the
variables a, b, and c this wayp a, b, c
95
When does pdb display a line?
Suppose you have progressed through the program
until you see the line final a b
c and you give pdb the command p final
96
When does pdb display a line?
You will get a NameError exception. This is
because, although you are seeing the line, it has
not yet executed. So the final variable has not
yet been created.
97
When does pdb display a line?
Now press "n" and ENTER to continue and execute
the line. Then try the "p final" command again.
This time, when you give the command "p final",
pdb will print the value of final.
98
See where you are with l
Sometimes you may forget where you are in your
program. That's where the "l" (for "list")
command comes in. "l" shows you, on the screen,
the general area of your code that you are
executing.
99
See where you are with l
By default, it lists 11 lines of code. The line
of code that you are about to execute (the
"current line") is right in the middle, and there
is a little arrow --gt that points to it.
100
Other Commands
c for continue s to step into subroutines r
like continue, but to the end of the subroutine
101
Try to correct your code .
One of the nice things about the (Pdb) prompt is
that you can do anything at it -- you can enter
any command that you like at the (Pdb) prompt.
102
Try to correct your code .
So you can, for instance, enter this command at
the (Pdb) prompt. (Pdb) var1 "bbb" You
can then continue to step through the program and
see if it makes a difference.