Lisp: An Introduction

1
Lisp An Introduction

• Lisp List Processing language
• Developed by John McCarthy, MIT, in the late
  1950s
• the original idea was to implement a language
  based on mathematical functions that McCarthy was
  using to model problem solving
• a grad student implemented the first interpreter,
  and it was quickly adopted for AI programming
• symbolic computing
• list processing
• recursion
• early Lisp was based almost entirely on recursive
  and so
• Lisp had no local variables nor iterative control
  structures (loops)
• and since Lisp was interpreted, there was no
  compiler for early Lisp

2
More on Lisp

• In part because this was early in the history of
  high level languages, and in part because of the
  desire to support AI
• Lisp used dynamic scoping
• Lisp had no compiler
• Lisp had no local variables
• Lisp was slow
• But, because Lisp was interpreted
• It was easy to prototype systems and build larger
  scale systems than a compiled language
• Lisp had features not available in other early
  languages
• Recursion
• Linked lists
• Dynamic memory allocation
• Typeless variables

3
Lisps, Lisps and More Lisps

• The original Lisp was both inefficient and
  difficult to use
• Other Lisp dialects were released by people
  trying to further the language
• they added a compiler, local variables, static
  scoping, etc
• while Scheme is the most recognized Lisp to come
  out of these efforts, others include Qlisp,
  Elisp, Interlisp, etc
• Specialized hardware was manufactured to support
  Lisp
• hardware that had large heap memory spaces and
  optimized routines for garbage collection
• interestingly, the first GUI was implemented for
  Lisp machines
• however, many researchers were put off by the
  high cost of these specialized machines, so there
  was also a push for implementing many of the
  necessary Lisp hardware mechanisms in
  general-purpose computers leading to both
  diversity (among compilers and dialects) and
  innovation
• but because of all the different Lisps, sharing
  of ideas and code became awkward if not
  prohibitive

4
Toward a Standard Common Lisp

• DARPA sponsored an effort in the early 1980s to
  develop a standard for Lisp, and so Common Lisp
  (CL) was born
• However, among the Lisp community, there were
  hard-fought battles
• West coast users often used Interlisp
• East coast users often used MACLisp
• European communities may have used yet other
  Lisps
• and Scheme was often taught in colleges
• CL would take the best attributes found in
  MacLisp with some influence from Zetalisp, Scheme
  and Interlisp
• it would also provide a number of imperative
  features (e.g., loops)
• ANSI created a standard for CL around 1990
• the Common Lisp Object System (CLOS) was added to
  CL making the language roughly equivalent in size
  and capability to C
• while not as recognized as C, Common Lisp is a
  very useful and flexible language, we will study
  it here

5
Goals of CL

• Commonality a common dialect of Lisp for which
  extensions (for given hardware) should be
  unnecessary
• Portability and Compatibility
• Consistency prior Lisps were internally
  inconsistent so that a compiler or interpreter
  might assign different semantics to a correct
  program
• Expressiveness and Power
• Efficiency for instance, optimizing compilers
  and instructions that perform efficient
  operations on lists
• Stability changes in CL will be made only with
  due deliberation

6
Some CL Uses

• CL is a very successful language in that
• it is extremely powerful as an AI tool
• those that use it love it
• And yet CL is a very uncommon language because
• it is very complicated
• most software development is in C (with some in
  Java, C, C, Python, Ada, etc) such that few
  programmers would choose to use CL for
  development
• The most important software developed from CL
  includes
• Emacs
• G2 (a real-time expert system)
• AutoCAD
• Igor Engraver (musical notation editor and
  publisher)
• Yahoo Store
• Up until the early 1990s, most AI research was
  still done in Lisp, most using Common Lisp
• however, once C added objects, many researchers
  switched to C because
• their graduate students had more familiarity with
  C
• obtaining C compilers is often cheaper and
  easier than CL compilers and environments
• C programs will run faster than CL programs
  (not always true, but generally true)

7
Why Study CL?

• Learn about functional programming
• Learn how to utilize recursion better
• Learn about symbolic computing
• Gain experience in other languages and syntax
• Get a better idea of such concepts as scope,
  binding, memory allocation, etc
• Learn a little about AI problem solving
• But primarily learn a new language
• one that is very VERY different from what you
  have already experienced
• but also a very rich and complex language
• CL contains 978 defined symbols, macros and
  functions not including anything available in
  libraries or what you might define yourself

8
Lisp Basics

• Everything in a Lisp program is either
• An atom
• a single literal value which could be
• a number
• a symbol like hello
• not to be confused with a string, a symbol cannot
  be subdivided in any way
• a character
• T or nil
• A list
• lists are denoted by ( )
• lists can contain nothing (empty list), one or
  more atoms, sublists, or some combination of
  these
• In Lisp, all variables are pointers that point to
  an atom or a list, but like Java, these pointers
  do not need dereferencing (unlike C or C)
• there are also sequences of different types
  (vectors, arrays, strings, structures, objects),
  these are neither lists nor atoms

9
Variables, Lists, Pointers

• All variables are in fact pointers
• A variables pointer points to an item in heap
  memory
• like Java, there is no dereferencing (unlike C or
  C)
• unlike Java, all items are pointed to whether
  they are objects, strings, arrays, numbers,
  atoms, whatever
• Lists are not only pointed to by the variable
  that defines the list, but each list element
  contains a pointer to the next item
• The structure of a list item is actually two
  pointers
• the CAR is the pointer to the item
• the CDR is the pointer to the next list item
• this structure looks like this
• This structure is known as a cons cell

10
List Structures

• A List then consists of a group of these cons
  structures, each structure has
• one pointer (the car) that points to the atom or
  sublist
• one pointer (the cdr) that either points to the
  remainder of the list or is nil (like null in
  Java or NULL in C)

( A ( D E F ) B C )
Cons cells are needed to make lists The cdr
should be a pointer to the next cons cell in the
list (or nil if this is the last cons cell, as
with the cells storing C and F) but in some
cases, you might put a datum in the cdr, this
would result in a dotted pair (a . b)
a b
11
Pointers and Pointers

• Things can get confusing in Lisp because the
  variables are pointers, and the things that they
  point to may also be pointers
• Look at the list example on the last slide, a
  given cons cell has two pointers, one pointing to
  the car and one to the cdr (which is usually
  another cons cell or the value nil)
• When you declare a variable, you are allocating
  memory to store the pointer, not the datum
• (let (a b) ) ? allocates space for a and b
• (setf a 5) ? allocates space for 5, adjusts a to
  point at 5
• (setf b 6) ? allocates space for 6, adjusts b to
  point at 6
• (setf b 5) ? will this allocate a new 5 or adjust
  b to point at the already existing 5? This
  differs depending on the implementation!

a
b
nil
nil
a
5
b
6
5
?
12
Lisp Code vs. Data

• Lisp code is placed inside of lists
• All Lisp instructions are function calls
• All Lisp function calls are written in prefix
  notation (name param1 param2 )
• All Lisp function calls return a value, which can
  be used in another call
• example (square ( 3 5)) ? ( 3 5) returns 8,
  (square 8) returns 64
• Lisp data are most commonly placed in lists
• So in Lisp, data and code have the same look to
  them, this makes it easy to write code that
  manipulates code rather than data (code that can
  generate code for instance)
• - there are exceptions to this statement that
  we will explore throughout this course

13
Lisp Interpreter

• One of the more unique aspects of Lisp (as
  opposed to say Java or C) is that it is an
  interpreted language
• You are given a command line where you can type
  in commands where a command can be
• a single executable statement (e.g., ( x y))
• a definition (defining a new function, defining a
  variable, defining a class, instantiating an
  object, etc)
• you see the response immediately and you dont
  have to have already compiled anything, this lets
  you test out code while you write it
• The Interpreter works on a REPL cycle
• Read read the latest item typed in at the
  command line (ended by ltentergt)
• Eval evaluate what was typed in (execute it)
• Print print the result to the screen for
  immediate feedback
• Loop do it all over again
• Note you can type definitions and instructions
  in an editor and load it all at once or compile
  it and load the compiled file

14
Example Session with REPL
CL-USER 1 gt 'hello HELLO CL-USER 2 gt (print
'hello) HELLO HELLO CL-USER 3 gt (setf a
'hello) HELLO CL-USER 4 gt a HELLO CL-USER 5 gt
'a A
Type in a statement, it is evaluated hello
evaluates the symbol hello print is a function,
it prints the item and also returns the
item setf is a function that has a side effect
of adjusting a pointer and returns the value that
the pointer is pointing to evaluate a returns
what a points to notice here the quote mark says
do not evaluate, just return
15
Example Session
CL-USER 40 gt (defun findmin (lis) (let ((temp
(car lis))) (dolist (a (cdr lis)) (if (lt a
temp) (setf temp a))) temp)) FINDMIN CL-USER
41 gt (defun sortlist (lis) (let (temp (size
(length lis)) (sortedlist nil)
(currentmin 1000)) (dotimes (i size) (setf
temp (findmin lis)) (if (lt temp currentmin)
(setf currentmin temp)) (setf lis (remove temp
lis)) (setf sortedlist (append sortedlist
(list temp)))) sortedlist)) SORTLIST CL-USER
42 gt (sortlist '(5 3 4 7 2 1 6)) (1 2 3 4 5 6 7)
16
The Debugger

• Unfortunately, the run-time environment is where
  most of your errors will be caught
• Every time the interpreter does not understand
  something, you are thrown into the debugger
• We will explore the debugger in detail later in
  the semester, here are a couple of comments
• the debugger lets you inspect the run-time stack,
  you can see the values of local variables and
  parameters, and the order that functions were
  called
• you can correct some mistakes and then resume
  execution without having to abort what you are
  doing, fix errors and start over
• For now though, we will simply abort out of the
  debugger any time an error arises
• This differs from implementation to
  implementation, but in LispWorks, look at the
  options and then type c where is the number
  of the abort option (for instance, c 1)

17
Example With Debugger
CL-USER 43 gt (sortlist b) Error The variable B
is unbound. 1 (continue) Try evaluating B
again. 2 Return the value of B instead. 3
Specify a value to use this time instead of
evaluating B. 4 Specify a value to set B to.
5 (abort) Return to level 0. 6 Return to top
loop level 0. Type b for backtrace, c ltoption
numbergt to proceed, or ? for other
options CL-USER 44 1 gt c 3 Enter a form to
be evaluated '(5 3 2 4 1) (1 2 3 4 5)
18
Example Continued
CL-USER 62 1 gt (sortlist b) Error The
variable B is unbound. 1 (continue) Try
evaluating B again. 2 Return the value of B
instead. 3 Specify a value to use this time
instead of evaluating B. 4 Specify a value to
set B to. 5 (abort) Return to level 1. 6
Return to debug level 1. 7 Return a value to
use. 8 Supply a new first argument. 9
Return to level 0. 10 Return to top loop level
0. Type b for backtrace, c ltoption numbergt to
proceed, or ? for other options CL-USER 63 2
gt c 4 Enter a form to be evaluated '(5 3 4 6 2
1) (1 2 3 4 5 6)
19
Example Continued
CL-USER 66 gt (sortlist) Error Call ((LAMBDA
(LIS) (DECLARE (SPECIALSOURCE )
(LAMBDA-NAME SORTLIST)) (BLOCK SORTLIST
(LET SORTEDLIST)))) has the wrong number
of arguments. 1 (abort) Return to level 0. 2
Return to top loop level 0. Type b for
backtrace, c ltoption numbergt to proceed, or ?
for other options CL-USER 67 1 gt
b Interpreted call to SORTLIST Call to
SPECIALEVAL-NOHOOK Call to IVPROCESS-TOP-LEVEL
Call to CAPICAPI-TOP-LEVEL-FUNCTION Call to
CAPIINTERACTIVE-PANE-TOP-LOOP Call to
(SUBFUNCTION MPPROCESS-SG-FUNCTION
MPINITIALIZE-PROCESS-STACK)
20
Getting Output Dribble

• You can send output to a text file, which we will
  cover later in the semester
• for now, to output your entire session in the CL
  environment, use dribble
• form (dribble filename)
• from this point forward, everything that you type
  in and everything that is returned (even debugger
  messages) are sent to the textfile filename
• to stop (or turn off) the dribble, type (dribble)
• if filename is just the name of a file, the
  file is created and stored in the current
  directory
• you can specify a directory, but since \ is an
  escape character (much like in Java), you have to
  specify \\
• as in (dribble C\\csc375\\output1.txt)
• if you specify a filename that already exists,
  anything that you dribble to the file is appended
  (the file is not overwritten)
• this allows you to join a previous session

21
Defining Functions

• There are a number of built-in functions in CL
• We will explore them throughout the semester
• For now, we briefly consider how to define a
  function
• The basic form is
• (defun name (params) body)
• name is the name of your function to have, it can
  be any legal CL name (and as long as the name is
  not a pre-defined CL name but it can be a name
  that you had previously used yourself)
• params are the names of parameters being passed
  into the function, they can be any legal name,
  and there are no types associated with them in
  the definition, the list can be empty as in ( )
• body is one or more CL function calls, after the
  last one is called, whatever value that last
  function call returns is returned by this function

22
Examples
(defun square (x) ( x x)) (defun largest (x y)
(if (gt x y) x y)) if (x gt y) return x
else return y (defun printnumbers (x) (if
(gt x 0) (progn
(printnumbers (- x 1)) (print
x)))) recursive version, if (x gt 0)
recursively call the function with x-1, then
print x (defun printnumbers2 (x) (do
((i 1 ( i 1))) (( i ( x 1)))
(print i))) here, a do loop is used
to iterate from i1 to x, stopping once i x1

• Type in (square 5) and the function returns 25
• x is 5
• the function performs ( 5 5)
• progn is used to denote a block of function calls
  where the result of the last function is returned
  from the block

23
Recursive vs Iterative

• Here we can clearly see the value of recursion
  when implementing simple operations
• the recursive version is short and matches our
  mathematical notion of factorial
• the iterative version requires a local variable
  for the product

(defun factorial (x) (if (lt x 1) 1 ( x
(factorial (- x 1))))) if x lt 1 then return
1, otherwise return x factorial(x-1) (defun
factorial2 (x) (let ((y 1)) y is a
local var 1 (do ((i 1 ( i 1)))
loop on i (( i ( x 1)))
until i x1 (setf y
( y i))) y y i y)) return y
as the last action
24
Lisp and Variables

• There are generally three kinds of variables
• Global variables defined in the run-time
  environment (at the command line) or outside of a
  function in a file
• these can be accessed within a function, but only
  having been declared as special
• Local variables defined in a function using the
  let statement, or within the scope of a specific
  instruction, such as the loop counter(s) in a do
  statement
• these can only be accessed inside their scope, as
  soon as their scope ends, you can no longer
  access them
• Parameters much like local variables, but they
  are available throughout the entire function

25
Variable Types

• As described earlier, variables are actually
  pointers, but unlike C, the pointers are not
  typed
• This means that the value being pointed to by a
  pointer can be of one type and later the pointer
  can point to a value of another type
• With a few exceptions, variables in CL are not
  typed
• notable exceptions to not typing variables are
  objects and structures
• However, all variables are type checked at
  run-time to make sure that a value is being
  treated correctly
• So ( 5 a) will yield a run-time error rather
  than an incorrect value

26
Values

• There are generally 3 kinds of values
• Numbers
• pretty self explanatory although Lisp contains
  several types of numbers
• integer, float, fraction, complex (contains an
  integer or float portion and an imaginary
  portion) and more
• you can combine types in an operation, Lisp will
  convert the answer to whichever type is necessary
• numbers are not restricted in size other than the
  size of what your computers memory can store!
• Atoms
• do not confuse these with strings, an atom is
  atomic (you cant for instance do substring or
  charAt type operations), the atom however can
  represent any symbolic item (word or words)
• Lists
• as we discussed earlier
• There are also characters, strings, arrays,
  structures, objects and more that we will discuss
  in detail later in the semester

27
Numbers in More Detail

• Unless otherwise indicated, numbers are stored as
  
• Integers
• Fractions
• You can override the default by indicating
• Floating point (in either decimal or scientific
  notation)
• Hexadecimal (x precedes the value as in x54B3E)
• Octal (o as in o5102)
• Any base up to 36 of the form brvalue as in
  6r3051 or 36rAZ9Q5
• base 36 includes all 10 digits and the 26 upper
  case letters
• Complex c(value1 value2) value1 value2i
  (i is the square root of -1)
• The result of any arithmetic operation will be in
  the form of the widest value (e.g. int fraction
  fraction)
• Except for / which will place integer divisions
  as a fraction

28
Operations on Numbers

• Arithmetic functions
• , -, , /
• Note that / will always return quotient and
  remainder, there is nothing equivalent to
  integer division as we see in Java
• But there are functions for truncate, round,
  ceiling, floor and mod
• (truncate 5 3) ? 1 (equivalent to 5 / 3)
• (mod 5 3) ? 2 (equivalent to 5 3)
• Common Lisp will always reduce fractions as much
  as possible, so (/ 6 8) ? ¾ and (/ 6 2) ? 3
• Now consider ( c(3 5) c(2 1)) ? c(5 6)
• But ( c(3 2) c(3 -2)) ? 13, why?
• Comparison functions
• lt, gt, , /, lt, gt
• EQL is similar to but more restrictive, it says
  numbers must be equal and be the same type
• ( 5 5.0) is T, (EQL 5 5.0) is nil

29
Preventing Evaluation

• One problem in Lisp is that everything entered at
  the command line is evaluated
• ( 3 5) ? evaluate , apply it to the evaluation
  of 3 and 5
• ( a b) ? evaluate , apply it to the evaluation
  of a and b
• a and b are variables, so evaluating a returns
  the value it points to (if a 5 and b 4, then
  ( a b) returns 9)
• (length lis) returns the number of elements in
  lis
• What if lis is not a variable, but the list (a b
  c)?
• (length (a b c)) ? this does not return 3,
  instead it will probably cause an error because
  (a b c) is interpreted as call function a
  passing parameters b and c

30
Quote

• So, to avoid some thing being evaluated, use
  (quote thing) as in (length (quote (a b c)))
• we can use quote to generate a symbol as in
  (quote hello) or a list as in (quote (a b c))
• We will often use (quote ) so Lisp gives us a
  shortcut way of specifying it,
• Sometimes it will be confusing as to when to
  quote something and when not to
• In general, if you have a list of data, you will
  want to quote the list so that the first item is
  not interpreted as a function call
• If you are referencing a variable, do not quote
  it, if you are referencing an atom, quote it
• NOTE has an entirely different meaning so
  make sure you use the forward quote or (quote ),
  we will discuss the backward quote later in the
  semester

31
Special Forms

• I said earlier that all Lisp code is function
  calls
• There are some notable exceptions called special
  forms
• Here are some special forms
• defun used to define a new function as already
  discussed
• if and cond Lisp conditional statements
• quote return the item without evaluating it
• let bind variables to memory locations,
  possibly initializing them
• and, or and returns either T or nil, or returns
  either nil or the first non-nil item
• progn declare a block of functions to make up
  the body of some function or instruction,
  returning the value returned by the last function
• all of these are actually macros, we will cover
  how to define macros later in the semester, but
  this gives you the flexibility of writing
  operations that are different from functions

32
Basic I/O

• CL has very powerful I/O facilities
• However, to get started, we will look at two
  simple I/O operations
• (read) get the next input from the keyboard and
  return it
• (setf x (read)) accepts one input and stores
  it in x
• (print x) sends x to the run-time window, but
  only works on a single item
• if you want to print multiple items, put them in
  a list as in
• (print (list x y z))
• Note this output will look like this (5 3 1)
  rather than 5 3 1
• Later on, we will examine formatted input and
  output, inputting from other sources, outputting
  to different windows, and file I/O

33
Basic Control Structures

• We will explore the control forms later, here are
  four to get started
• if and if-else
• (if (condition) then)
• (if (condition) then else)
• then what to return if the condition is true
• else what to return if the condition is false
• these will usually be function calls, but could
  be values to return
• (if (/ y 0) (/ x y) 0) divide x by y or return
  0 if y 0
• iteration dotimes and dolist
• (dotimes (var num) )
• iterates from 0 to num-1, setting var to each
  value one at a time
• (dotimes (i n) (print ( i 1))) prints the
  numbers 1 through n
• (dolist (var lis) )
• iterates for each item in the list lis, with var
  assigned to each
• (dolist (a lis) (print a)) prints each item of
  lis one at a time

34
Putting It All Together

• We will rely heavily on the REPL interpreter
• Start up Common Lisp
• Type in a function, if it is interpreted
  correctly, there will be no error
• Now you can call your function to test it out
• If it isnt right, redefine it with another defun
  statement
• Evolve your code over time
• Alternatively, type your code into an editor
• This allows you to write your code as a series of
  functions so that you can get a more global idea
  of what you have to do, and save your code in a
  text file
• Copy and paste your code from the editor into the
  interpreter
• Or, save the file and load the entire file at one
  time (usually you will only do this after you
  know all of your individual functions work!)

35
Apropos

• Cant find the right symbol (function) for some
  particular application but you know its something
  like foobar? Use Apropos
• (apropos foobar)
• (apropos foobar packagename)
• apropos will look up all defined entities with
  the string foobar in it and return the list
• if you specify a package, it only lists those
  items in the given package
• most likely, you will want to limit your search
  to the Common Lisp package, cl
• (apropos reverse) returns somewhere around 46
  entries whereas (apropos reverse cl) returns
  just 2