Foundations of Artificial Intelligence

1
Foundations of Artificial Intelligence

• Chapter 7 The predicate calculus
• Luc De Raedt

2
Predicate calculus

• Limitations of propositional calculus
• Language and its syntax
• Semantics (informally)
• Quantifiers
• Representing Knowledge
• Resolution
• Prolog
• This part is based on
• Nils Nilsson, AI a new synthesis, Morgan
  Kaufman, 1998, chapters 15, 16 and 17

3
The language and its syntax (restricted version)

• Components
• An infinite set of object constants strings
  starting with a capital or a numeral
• E.g.
• An infinite set of function constants of all
  arities strings starting with lowercase and
  superscripted with arity
• E.g.
• An infinite set of relation constants of all
  arities strings beginning with uppercase and
  superscripted with arity
• E.g.
• Traditional connectives

4
Terms (restricted)

• An object constant is a term
• A function constant f of arity n followed by n
  terms ti in parentheses is a term
• Examples

5
Well-formed formulae (restricted)

• Atoms A relation constant F of arity n followed
  by n terms ti is an atom
• Examples
• Propositional wffs any expression formed with
  the propositional calculus and atoms as
  propositions.
• Example

6
Semantics (restricted version)

• Worlds
• The world can have an infinite number of objects,
  called individuals, the domain
• The world can have an infinite number of
  functions f that map n-tuples of individuals into
  individuals
• The world can have an infinite number of
  relations over individuals,

7
Interpretations

• An interpretation maps
• object constants to individuals
• n-ary function constants onto n-ary functions
• n-ary relation constants onto n-ary relations
• Given an interpretation, an atom has
  the value True iff the relation denoted by R
  holds among for those individuals denoted by the
  ti

8
(No Transcript)
9
Models and related notions

• An interpretation satisfies a wff if the wff has
  the value True under the interpretation
• An interpretation that satisfies a wff is a model
  of that wff
• Any wff that has the value True under all
  interpretations is a tautology (it is valid)
• Any wff that does not have a model is
  unsatisfiable
• If a wff ? has value True under all those
  interpretations for which each of the of wffs in
  a set ? has value true then ? logically entails
  ?, notation
• Two wffs are equivalent iff their truth values
  are identical under all interpretations
• Essentially, the notions from propositional logic
  carry over

10
Knowledge

• Does the following knowledge base (set of
  formulae) have a model ?

11
Quantification

• How to express
• every object is red ?
• there is an object which is red ?
• For finite domains for infinite domains ?
• Use variables and quantifiers !

12
Predicate calculus

• Now introduce
• An infinite set of variable symbols start with
  lowercase
• Each variable is also a term !
• The universal quantifier? and the existential
  one ?
• If ? is a wff and v is a variable then the
  following are also wffs
• Example
• Scoping ? is within the scope of the
  quantifier

13
Variables, scoping, and quantifiers

• Scoping
• We will assume that all wffs are closed, i.e.
  that all variables are quantified, i.e. occur
  with the scope of a quantifier.
• closed
• Open
• Equivalent
• Not equivalent !

14
Semantics

• has the value true (under a given
  interpretation) iff ?(v) has the value true for
  all assignments of the variable symbol v to
  individuals
• has the value true (under a given
  interpretation) iff ?(v) has the value true for
  an assignments of the variable symbol to
  individuals

15
An example
16
Rules of inference

• Some equivalences (De Morgan)
• Universal instantiation
• From infer where a is
  substituted for x throughout ?

17

• Existential Generalization
• From infer where a has
  been replaced by x throughout ?
• Universal instantiation and Existential
  generalization are sound inference rules

18
Wumpus

• Time could be incorporated as well

19
Some sentences

• Not all students take both history and biology
• Only one student failed history (requires )
• No person likes a smart vegetarian
• No person likes a professor unless the professor
  is smart

20
Clausal Form

• Clauses are universally quantified disjunctions
  of literals all variables in a clause are
  universally quantified

21
Towards Resolution

• Examples
• We need to be able to work with variables !
• Unification of two expressions/literals

22
Terms and instances

• Consider following atoms
• Ground expressions do not contain any variables

23
Substitution
24
Composing substitutions

• Composing substitutions s1 and s2 gives s1 s2
  which is that substitution obtained by first
  applying s2 to the terms in s1and adding
  remaining term/vars pairs to s1
• Apply to

25
Properties of substitutions
26
Unification

• Unifying a set of expressions wi
• Find substitution s such that
• Example
• The most general unifier, the mgu, g of wi has
  the property that if s is any unifier of wi
  then there exists a substitution s such that
  wiswigs
• The common instance produced is unique up to
  alphabetic variants (variable renaming)

27
Disagreement set

• The disagreement set of a set of expressions wi
  is the set of subterms ti of wi at the
  first position in wi for which the wi
  disagree

28
Unification algorithm
29
Towards Resolution

• Examples
• We need to be able to work with variables !
• Unification of two expressions/literals

30
Predicate calculus Resolution

• John Allan Robinson (1965)

31
Example
32
A stronger version of resolution

• Use more than one literal per clause

33
Factors
34
Resolution
35
(No Transcript)
36
Resolution

• Properties
• Resolution is sound
• Incomplete
• But fortunately it is refutation complete
• If KB is unsatisfiable then KB - ?

37
Refutation Completeness

• To decide whether a formula KB w do
• Convert KB to clausal form KB
• Convert ?w to clausal form ?w
• Combine ?w and KB to give ?
• Iteratively apply resolution to ? and add the
  results back to ? until either no more resolvents
  can be added, or until the empty clause is
  produced.

38
Converting to clausal form

• To convert a formula KB into clausal form
• Eliminate implication signs
• Reduce scope of negation signs
• Standardize variables
• Eliminate existential quantifiers using
  skolemization
• Same as in prop. logic

39
(No Transcript)
40
Skolemization

• General rule is that each occurrence of an
  existentially quantified variable is replaced by
  a skolem function whose arguments are those
  universally quantified variables whose scopes
  includes the scope of the existentially
  quantified one
• Skolem functions do not yet occur elsewhere !
• Resulting formula is not logically equivalent !

41
Skolemization examples

• A well formed formula and its skolem form are not
  logically equivalent. However, a set of formulae
  is (un)satisfiable if and only if its skolem
  form is (un)satisfiable.

42
Converting to clausal form

• 5. Convert to prenex form
• Move all universal quantifiers to the front
• 6. Put the matrix in conjunctive normal form
• Use distribution rule
• 7. Eliminate universal quantifiers
• 8. Eliminate conjunction symbol
• 9. Rename variables so that no variable occurs in
  more than one clause.

43
(No Transcript)
44
Using resolution to prove theorems
45
Resolution Search Strategies

• Ordering strategies
• In what order to perform resolution ?
• Breadth-first, depth-first, iterative deepening ?
• Unit-preference strategy
• Prefer those resolution steps in which at least
  one clause is a unit clause (containing a single
  literal)
• Refinement strategies
• Unit resolution allow only resolution with unit
  clauses

46
Input Resolution

• at least one of the clauses being resolved is a
  member of the original set of clauses
• Input resolution is complete for Horn-clauses but
  incomplete in general
• E.g.
• One of the parents of the empty clause should
  belong to original set of clauses

47
Linear Resolution

• Linear resolvent is one in which at least one of
  the parents is in the initial database or is
  ancestor of the other parent
• Refutation complete
• Many other resolution strategies exist

48
Set of support

• Ancestor c2 is a descendant of c1 iff c2 is a
  resolvent of c1 (and another clause) or if c2 is
  a resolvent of a descendant of c1 (and another
  clause) c1 is an ancestor of c2
• Set of support the set of clauses coming from
  the negation of the theorem (to be proven) and
  their descendants
• Set of support strategy require that at least
  one of the clauses in each resolution step
  belongs to the set of support

49
Answer extraction

• Suppose we wish to prove whether KB (?w)f(w)
• We are probably interested in knowing the w for
  which f(w) holds.
• Add Ans(w) literal to each clause coming from the
  negation of the theorem to be proven stop
  resolution process when there is a clause
  containing only Ans literal

50
(No Transcript)
51
Horn clause logic and Prolog

• SAT for propositional clausal theories is
  NP-complete, but for Horn-clauses it is
  polynomial
• Inference with first order Horn clauses is also
  more efficient/easier than with full clauses
• Horn clauses are the basis of Computational
  Logic, Logic Programming and the Prolog
  programming language
• Horn clauses correspond to rules in a knowledge
  base
• Very effective model for knowledge representation
• We just the basics, so called pure Prolog

52
Horn logic/Prolog
53
Proof tree

• - Moves
• Moves - Bat_ok, Liftable
• - Bat_ok, Liftable
• Bat_ok -
• - Liftable
• Liftable -
• -

54
SLD-derivation

• - Moves
• - Bat_ok, Liftable
• - Liftable
• -

• Contains the sequence of goals that arise during
  the computation

55
p.4
These slides are adapted from Peter Flachs
book Simply Logical, John Wiley, 1994.
56
London Underground in Prolog (1)
p.3

• Connected(Bond_street,Oxfd_circus,Central)
  -Connected(Oxfd_circus,Tottenham_court_road,Cent
  ral)-Connected(Bond_street,Green_park,Jubilee)-
  Connected(Green_park,Vharing_cross,Jubilee)-Conn
  ected(Green_park,Piccadilly_circus,Piccadilly)-C
  onnected(Piccadilly_circus,Leicester_square,Piccad
  illy)-Connected(Green_park,Oxfd_circus,Victoria)
  -Connected(Oxfd_circus,Piccadilly_circus,Bakerlo
  o)-Connected(Piccadilly_circus,Charing_cross,Bak
  erloo)-Connected(Tottenham_court_road,Leicester_
  square,Northern)-Connected(Leicester_square,Char
  ing_cross,Northern)-

57
London Underground in Prolog (2)
p.3-4

• Two stations are nearby if they are on the same
  line with at most one other station in between
• Nearby(Bond_street,Oxfd_circus).Nearby(Oxfd_circu
  s,Tottenham_court_road).Nearby(Bond_street,Totten
  ham_court_road).Nearby(Bond_street,Green_park).N
  earby(Green_park,Charing_cross).Nearby(Bond_stree
  t,Charing_cross).Nearby(Green_park,Piccadilly_cir
  cus).
• or better
• Nearby(x,y)-Connected(x,y,l).Nearby(x,y)-Connec
  ted(x,z,l),Connected(z,y,l).

58
p.5

• Compare
• Nearby(x,y)-Connected(x,y,l).Nearby(x,y)-
  Connected(x,z,l),Connected(z,y,l).
• with
• Not_too_far(x,y)-Connected(x,y,l).Not_too_far(x
  ,y)- Connected(x,z,l1),Connected(z,y,l2).

59
Proof tree
Fig.1.2, p.7
-Nearby(Tottenham_court_road,w)
Nearby(x1,y1)-Connected(x1,u1,l1)
Connected(Tottenham_court_road,Leicester_square,N
orthern)-
60
p.7
-Nearby(w,Charing_cross)
Nearby(x1,y1)-Connected(x1,z1,l1),Connected(z1,y
1,l1)
Connected(Bond_street,Green_park,Jubilee)
Connected(Green_park,Charing_cross,Jubilee)
61
Recursion (1)
p.8

• A station is Reachable from another if they are
  on the same line, or with one, two, changes
• Reachable(x,y)-Connected(x,y,l).Reachable(x,y)-
  Connected(x,z,l1),Connected(z,y,l2).Reachable(x,
  y)- Connected(x,z1,l1),Connected(z1,z2,l2),
  Connected(z2,y,l3).
• or better
• Reachable(x,y)-Connected(x,y,l).Reachable(x,y)-
  Connected(x,z,l),Reachable(z,y).

62
Fig. 1.3, p.9
-Reachable(Bond_street,w)
Reachable(x1,y1)-Connected(x1,z1,l1),
Feachable(z1,y1)
Connected(Bond_street, Oxfd_circus,Central)
Reachable(x2,y2)-Connected(x2,z2,l2),
Reachable(z2,y2)
Connected(Oxfd_circus, Tottenham_court_road, Centr
al)
Reachable(x3,y3)-Connected(x3,y3,l3)
Connected(Tottenham_court_road, Leicester_square,
Northern)
63
p.12
Reachable(x,y,Noroute)-Connected(x,y,l). Reachabl
e(x,y,route(z,r))-Connected(x,z,l),
Reachable(z,y,r). -Reachable(Oxfd_
circus,Charing_cross,r).r route(Tottenham_court
_road,route(Leicester_square,Noroute))r
route(piccadilly_circus,Noroute)r
route(Picadilly_circus,route(Leicester_square,Noro
ute))
64
Proof tree

• -Above(A,C)
• Above(x,y)-On(x,z),Above(z,y)
• -On(A,z),Above(z,C)
• On(A,B)-
• -Above(B,C)
• Above(x1,y1)-On(x1,y1).
• -On(B,C)
• On(B,C)-
• -

65
SLD-derivation

• -Above(A,C)
• -On(A,z),Above(z,C)
• -Above(B,C)
• -On(B,C)
• -

66
SLD-tree

• -Above(A,C)
• / \
• -On(A,C) -On(A,z),Above(z,C)
• -Above(B,C)
• / \
• -On(B,C) -On(B,z),Above(z,C)
• - -Above(C,C)
• / \
• -On(C,C) -On(C,z),Above(z,C)

67
SLD-resolution

• SLD
• Linear resolution with Selection rule for
  Definite clauses
• SLD-derivation
• Summarizes one proof path
• SLD-tree
• Summarizes all possible proofs
• Substitutions
• May be written explicitly

68
SLD-tree

• Summarizes all possible proofs of the goal
• Prolog selects leftmost literal in goal to
  resolve upon !
• Clauses to resolve the literal with are chosen
  according to clause order !
• Prolog traverses the tree depth-first in order
• Prolog is incomplete (can get trapped into
  infinite paths in the tree)
• Prolog is memory efficient (depth-first)
• Alternatively, iterative deepening or breadth
  first (which would be complete)

69
p.44-5
Student_of(x,t)-Follows(x,c),Teaches(t,c).Follow
s(Paul,Computer_science).Follows(Paul,Expert_syst
ems).Follows(Maria,Ai_techniques).Teaches(Adrian
,Expert_systems).Teaches(Peter,Ai_techniques).Te
aches(Peter,Computer_science).
-Student_of(s,Peter)
-Follows(s,c),Teaches(Peter,c)
-Teaches(Peter,Ai_techniques)
-Teaches(Peter,Computer_science)
-Teaches(Peter,Expert_systems)
70
List processing examples
71

• - M(B,c(A,c(B,nil)))
• - M(B,c(B,nil))
• -
• Is B a member of A,B ?

72

• - M(w,c(A,c(B,nil)))
• \
• - - M(w,c(B,nil)
• wA / \
• - -M(w,nil)
• wB
• Generate members w of A,B
• Backtracking returns all solutions !
• Prolog returns
• Yes wA Yes wB Fail.

73

• - M(B,w)
• / \
• - -M(B,w2)
• wc(B,w1) \
• - -M(B,w4)
• wc(u1,c(B,u2))
• infinite tree
• Generate lists w that B is a member of ?

74
(No Transcript)
75
Alternative more compact list notation

• Use
• AB for cons(A,B)
• Use for Nil
• So, cons(a,cons(b,nil)) would be written as
• a b or
• a,b
• A,B x a list with at least two elements (A
  and B)
• Note Standard prolog uses following
  convention variables start with an upper case (or
  _)
• predicates, constants/function symbols
  start with a lower case

76
An illustration Natural Language Processing

• Very often employ
• Grammar formalisms
• Logic
• Unification
• We very brief introduction to
• Definite clause grammars
• Notation
• we sometimes use list notation instead of the
  cons(a,cons()) (built-in to Prolog)
• Prolog notation is different than that used by
  Nilsson
• DCG and Prolog notation predicates, function,
  and constants start with a lower case variables
  start with an upper-case
• We use DCG notation for DCG but stick to
  Nilssons notation for Prolog !

77
Two tasks in NLP

• Just to give an idea / to illustrate
• Computer science point of view !
• Parsing
• Analyzing the syntactic structure of sentences
• Construct the parse tree
• Interpretation
• Determine the meaning (internal representation
  thereof) of the sentence
• Internal representation can then be used to
  reason
• Example Application
• NLP interface to a database

78
Context-free grammar
p.132
sentence --gt noun_phrase,verb_phrase. noun_phrase
--gt proper_noun. noun_phrase --gt
article,adjective,noun. noun_phrase --gt
article,noun. verb_phrase --gt intransitive_verb. v
erb_phrase --gt transitive_verb,noun_phrase. articl
e --gt the. cons(the,Nil) adjective --gt
lazy. adjective --gt rapid. proper_noun --gt
achilles. noun --gt turtle. intransitive_verb -
-gt sleeps. transitive_verb --gt beats.
79
Parse tree
p.133
80
p.133
81
Translating CFG in Prolog
82
Difference lists in grammar rules
p.135
noun phrase
verb phrase
VP2
VP1
NP1
Sentence(np1,vp2)-Noun_phrase(np1,vp1),Verb_phr
ase(vp1,vp2)
83
Translating CFG in Prolog
84
Non-terminals with arguments
p.137
sentence --gt noun_phrase(N),verb_phrase(N). noun_p
hrase(N) --gt article(N),noun(N). verb_phrase(N) --
gt intransitive_verb(N). article(singular) --gt
a. article(singular) --gt the. article(plural)
--gt the. noun(singular) --gt turtle. noun(plura
l) --gt turtles. intransitive_verb(singular) --gt
sleeps. intransitive_verb(plural) --gt sleep.
85
Translate to Prolog

• Translation is automatic !
• Most Prolog implementation can directly cope with
  DCG notation

86
Constructing parse trees
p.137-8
sentence(s(NP,VP)) --gt noun_phrase(NP),verb_phrase
(VP). noun_phrase(np(N)) --gt proper_noun(N). noun_
phrase(np(Art,Adj,N)) --gt article(Art),adjective(A
dj), noun(N). noun_phrase(np(Art,N)) --gt
article(Art),noun(N). verb_phrase(vp(IV)) --gt
intransitive_verb(IV). verb_phrase(vp(TV,NP)) --gt
transitive_verb(TV), noun_phrase(NP). article(ar
t(the)) --gt the. adjective(adj(lazy)) --gt
lazy. adjective(adj(rapid)) --gt
rapid. proper_noun(pn(achilles)) --gt
achilles. noun(n(turtle)) --gt
turtle. intransitive_verb(iv(sleeps)) --gt
sleeps. transitive_verb(tv(beats)) --gt beats.
-Sentence(Achilles,Beats,The,Lazy,Turtle,Nil,t)
t s(np(pn(Achilles)), vp(tv(Beats),
np(art(The), adj(Lazy),
n(Turtle))))
87
Parse tree
p.133
88
Interpretation (simplified !)
p.140

• The meaning of the proper noun Socrates is the
  term socrates
• proper_noun(socrates) --gt socrates.
• The meaning of the property mortal is a mapping
  from terms to literals containing the unary
  predicate/functor mortal
• property(X,mortal(X))) --gt mortal.
• The meaning of a proper noun - verb phrase
  sentence is a fact obtained by applying the
  meaning of the verb phrase to the meaning of the
  proper noun
• sentence(Fact)--gt
• proper_noun(X),is,property(X,Fact).
• - Sentence(Socrates,Is,Mortal,fact).
• fact mortal(Socrates).
• Can easily be extended to convert simple natural
  language into logical statements.

89
p.140

• A transitive verb is a binary mapping from a pair
  of terms to literals
• transitive_verb(Y,X,likes(X,Y)) --gt likes.
• A proper noun instantiates one of the arguments,
  returning a unary mapping
• verb_phrase(Y,M) --gttransitive_verb(Y,X,M),proper_
  noun(X).

90
p.140-1

• sentence(fact(L)) --gt
• proper_noun(X),verb_phrase(X,L).sentence(cla
  use(H,B)) --gt
• every,noun(X,B),verb_phrase(X,H). NB.
  separate determiner rule removed, see later
• verb_phrase(M) --gt is,property(M).
• property(M) --gt a,noun(M).property(X,mortal(X))
  --gt mortal.
• proper_noun(socrates) --gt socrates.
• noun(X,human(X)) --gt human.

91
p.141
?-Sentence(c,Nil,s). c clause(human(x),human(x))
s Every,Human,Is,A,Human c
clause(mortal(x),human(x))s Every,Human,Is,Mor
tal c fact(human(Socrates))s
Socrates,Is,A,Human c fact(mortal(Socrates))
s Socrates,Is,Mortal
92
Real NLP

• Real Grammars
• Are much more complicated
• Account for detailed syntactic and semantic
  aspects of language
• Take ages to develop
• Extension for NLP database interface of present
  grammar is possible
• Assert interpreted facts/clauses in Database
• Logical reasoning possible
• Extend with queries, e.g. What Who Where
  queries
• Who is mortal ?
• What is Socrates ?

93
Prolog

• Use sicstus (installed in Pool) or download
  SWI-prolog
• http//www.swi.psy.uva.nl/projects/SWI-Prolog/
• Beware of syntax !
• File append.pl
• append(nil,X,X).
• append(cons(X,Y),Z,cons(X,W))-append(Y,Z,W).
• sicstus
• ?-consult(append).
• Yes
• ?-append(cons(a,nil),cons(b,cons(c,nil)),X).
• X cons(a,cons(b,cons(c,nil)))
• Yes
• Good books on Prolog
• Peter Flach, Simply Logical, Wiley
• Ivan Bratko, Prolog programming for AI, Addison
  Wesley.

94
Prolog

• Good books on Prolog
• Peter Flach, Simply Logical, Wiley
• Ivan Bratko, Prolog programming for AI, Addison
  Wesley.
• Sterling and Shapiro, The art of Prolog, MIT Press