Logic, Language and Learning

1
Logic, Language and Learning

• Chapter 6
• NLP in Prolog
• Luc De Raedt
• (Some slides by Flach, Kramer and Covington)

2
Outline

• Natural Language Part
• We follow
• Simply logical (from Flachs book)
• NLP for Prolog programmers (by Covington)
• An introduction to NLP through Prolog (by
  Matthews)
• Today
• Introduction
• phrase structure grammars
• definite clause grammars (DCGs)

3
Introduction
4
Natural language processing

• Use of computers to understand (natural)
  languages
• Natural languages are
• form, not substance
• competence versus performance
• arbitrary
• discrete
• using duality of patterning
• equally complicated, change constantly

5
Levels of linguistic analysis (1)

• Prosodyrhythm and intonation of spoken language
• Phonologyhow sounds are used in languagehow
  simple phonemes (units of sound) are combined in
  spoken language
• Morphologyhow words are formed from components
  (morphemes) -
• inflection (word forms)
• derivation (create new words from existing ones)

6
Levels of linguistic analysis (2)

• Syntaxhow sentences are constructed from words
• Semanticshow expressions of speech (words and
  sentences) relate to their meaning
• The president likes spaghetti.
• Pragmaticshow context influences the meaning of
  expressionshow sentences are used in
  communication
• Can you open the door ?

7
Template matching (1)

• Example Eliza
• Stimulus-response pair- I am ltstatementgt.- How
  long have you been ltstatementgt?
• Read the input- While the input is not bye -
  Choose a stimulus-response pair - Match the
  input to the stimulus - Generate the reply from
  the response and the above match - Output
  the response - Read the next input

8
Template matching (2)

• A simple template system
• What files are on drive B? gt dir b
• Delete all my files. gt erase .
• Run the word processor gt wp
• Simplification rules
• discard unnecessary words and deal with
  equivalent words
• Translation rules
• map a template to the formal language

9
Template matching (3)

• Simplification rules. sr(theX,X).
  sr(isX,X). sr(disk,in,driveX,driveX).
  sr(disk,inX,driveX). sr(diskX,driveX)
  . sr(what,filesX,filesX). ...
• simplify(List,-Result)simplify(,).simpl
  ify(List,Result) - sr(List,NewList), !,
  simplify(NewList,Result).simplify(WWords,WN
  ewWords) - simplify(Words,NewWords).

10
Template matching (4)

• tr(files,on,drive,X,'?', 'dir
  ',X,'').tr(X,files,on,drive,Y,'?', 'dir
  ',Y,'.',X).tr(copy,all,files,from,drive,X,to
  ,drive,Y,'.', 'copy ',X,'.
  ',Y,'').tr(quit,quit).translate(Input,Re
  sult) - tr(Input,Result),
  !.translate(_,) - write('I do not
  understand.'), nl.

11
Keyword analysis (1)

• employee(1001, 'Doe, John P.', 1947,01,30,
  'President', 100000).
• employee(1002, 'Smith, Mary J.', 1960,09,05,
  'Programmer', 30000).
• employee(1003,'Zimmer, Fred', 1957,3,12,
  'Sales rep', 45000).
• process_queries - repeat, write('Query '),
  read_atomics(Words), translate(Words,_,Query)
  , findall(_, call(Query), _), Words
  quit, !.

12
Keyword analysis (2)

• Look for specific words and respond to each word
  in a specific way
• E.g. Show me all programmers with salaries over
  25000.
• action display
• Title programmer
• Salary gt 25000

13
Keyword analysis (3)

• Translation

14
Keyword analysis (4)

• Problem how to store definitions of words
• Lambda Abstraction
• Idea explicitely specify the argument
• mortal(socrates) gt Socrates is mortal
• (?x)mortal(x) gt function of x is mortal
• In mathematics
• let f(x)4x2
• f (?x)4x2
• Main idea of Lisp / functional programming

15
Keyword analysis (5)

• Lambda abstraction in Prolog

action(show,Xdisplay_record(X)).action(display,X
display_record(X)).action(delete,Xremove_record
(X)). test(programmer,Xtitle(X,'Programmer')).t
est(programmers,Xtitle(X,'Programmer')).test(sal
esrep,Xtitle(X,'Sales rep')).test(salesreps,Xti
tle(X,'Sales rep')). relation(over,YZ(YgtZ)).re
lation(under,YZ(YltZ)). argument(salary,XYsala
ry(X,Y)).argument(birthdate,XYbirth_date(X,Y)).
argument(N,_Y(YN)) - number(N).
16

Keyword analysis (6) translation using lambda
abstraction
translate(WWords,X,(Queries,Q)) -
action(W,XQ), !, translate(Words,X,Querie
s). translate(WWords,X,(Q,Queries)) -
test(W,XQ), !, translate(Words,X,Queries)
. translate(Arg1,W,Arg2Words,X,(Q1,Q2,Q3,Querie
s)) - relation(W,YZQ3), !,
argument(Arg1,XYQ1), argument(Arg2,XZQ2),
translate(Words,X,Queries). translate(,_,tru
e). translate(_Words,X,Query) -
translate(Words,X,Query).
17
Phrase Structure Grammars
18
Grammars and parsing

• Syntactic level best understood and formalized
• Derivation of grammatical structure
  parsing(more than just recognition)
• Result of parsing mostly parse treeshowing the
  constituents of a sentence, e.g. verb or noun
  phrases
• Syntax usually specified in terms of a grammar
  consisting of grammar rules

19
Phrase structure
S
NP
VP
D
N
NP
V
PP
P
NP
N
D
N
D
the
dog
a
cat
into
the
garden
chased
20
Notation

• S sentence
• D or Det Determiner (e.g., articles)
• N noun
• V verb
• P preposition
• NP noun phrase
• VP verb phrase
• PP prepositional phrase

21
Phrase structure rules
S -gt NP VPNP -gt D NVP -gt V NPVP -gt V NP
PPPP -gt P NPD -gt theD -gt aN -gt dogN -gt
catN -gt gardenV -gt chasedV -gt sawP -gt into
22
Phrase structure

• Formalism of context-free grammars
• Nonterminal symbols S, NP, VP, ...
• Terminal symbols dog, cat, saw, the, Ø, ...
• Recursion
• The girl thought the dog chased the cat

VP -gt V SN -gt girlV -gt thought
23
Grammars and parsing background

• top-down parsing vs.bottom-up parsing
• context-free grammars vs.context-sensitive
  grammars

24
Context-free grammar
sentence --gt noun_phrase,verb_phrase. noun_phrase
--gt proper_noun. noun_phrase --gt
article,adjective,noun. noun_phrase --gt
article,noun. verb_phrase --gt intransitive_verb. v
erb_phrase --gt transitive_verb,noun_phrase. articl
e --gt the. adjective --gt lazy. adjective --gt
rapid. proper_noun --gt achilles. noun --gt
turtle. intransitive_verb --gt
sleeps. transitive_verb --gt beats.
25
Parse tree
26
Context-sensitive grammar

• sentence --gt noun_phrase,
  plurality,
  verb_phrase.
• noun_phrase --gt article, noun.
• plurality --gt singular.
• plurality --gt plural.
• verb_phrase --gt intrans_verb.
• article --gt the.
• noun, singular --gt turtle, singular.
• noun, plural --gt turtles, plural.
• singular, intrans_verb --gt sleeps.
• plural, intrans_verb --gt sleep.

27
Parsing example grammar sentence

• S -gt NP VPNP -gt Det NVP -gt V NP
• The engineer repaired the boiler.

28
Top-down parsing with phrase structure grammar (1)
29
Top-down parsing with phrase structure grammar (2)
30
Bottom-up parsing with phrase structure grammar
(1)
31
Bottom-up parsing with phrase structure grammar
(2)
32
Definite-Clause Grammars
33
DCGs (1)

• Definite clause grammarspowerful built-in
  grammar formalism
• Special notation for grammar,internally
  automatically translated into definite clauses
• More powerful than context-free grammarsuse of
  arguments and arbitrary Prolog goals in addition

34
Phrase structure grammar in Prolog

• S -gt NP VPNP -gt Det NVP -gt V NP
• s(S) - concat(NP,VP,S), np(NP),
  vp(VP).np(NP) - concat(Det,N,NP),
  det(Det), n(N).vp(VP) - concat(V, NP,
  VP), v(V), np(NP).

35
Proof tree of parse
36
DCGs (2)

• Sentences represented in liststhe, rapid,
  turtle, beats, achilles
• Grammar rulesentence --gt noun_phrase,
  verb_phrase.
• Prolog clausesentence(S) - noun_phrase(NP),
  verb_phrase(VP),
  concat(NP, VP, S).

37
DCGs (3)

• Terminal symbolverb --gt sleeps.
• Prolog fact verb(sleeps).
• In order to parse, pose a query, e.g.?-
  sentence(the,rapid,turtle,
  beats,achilles).

38
More precisely DCG notation
nonterminal symbol --gt expansion
where expansion is any of the following

• A nonterminal symbol such as np
• A list of terminal symbols (dog, as,well)
• A null constituent represented by
• A Prolog goal enclosed in braces, such as
  write(Found NP)
• A series of any of the preceding expansions
  joined by commas

39
Phrase structure grammar in DCG notation...
s --gt np, vp.np --gt d, n.vp --gt v, np.d --gt
the.d --gt a.n --gt dog.n --gt cat.n
--gt gardener.n --gt policeman.n --gt
butler.n --gt garden.v --gt chased.v --gt
saw.
40
...written in Prolog
s(L1, L) - np(L1,L2), vp(L2, L).np(L1,L) -
d(L1,L2), n(L2,L).vp(L1,L) - v(L1,L2),
np(L2,L). d(theL,L).d(aL,L).n(dogL,L).
n(catL,L). n(gardenerL,L).n(policemanL,
L).n(butlerL,L).v(chasedL,L).v(sawL,L)
. ?- s(the,dog,chased,the,cat, ).
41
DCGs and difference lists

• Reminiscent of difference lists
• sentence(NP1-VP2) - noun_phrase(NP1-VP1), verb_
  phrase(VP1-VP2).
• Query?- sentence(the,rapid,turtle,
  beats,achilles-).

42
Difference lists in grammar rules
noun phrase
verb phrase
VP2
VP1
NP1
sentence(NP1-VP2)-noun_phrase(NP1-VP1),verb_phr
ase(VP1-VP2)
43
Automatic conversion

• Automatic conversion of phrase structure rules
  intoProlog

s --gt np, vp.np --gt d, n.vp --gt v, np.
s(L1, L) - np(L1,L2), vp(L2, L).np(L1,L) -
d(L1,L2), n(L2,L).vp(L1,L) - v(L1,L2), np(L2,L).
44
Meta-level vs. object-level
PARSING
GRAMMAR
META-
s --gt np,vp
?-phrase(s,L)
LEVEL
s(L,L0)- np(L,L1), vp(L1,L0)
?-s(L,)
OBJECT-
LEVEL
45
Proof tree-like structure for DCGs
46
SLD-tree-like structure for DCGs
47
Non-terminals with arguments
sentence --gt noun_phrase(N),verb_phrase(N). noun_p
hrase --gt article(N),noun(N). verb_phrase(N) --gt
intransitive_verb(N). article(singular) --gt
a. article(singular) --gt the. article(plural)
--gt the. noun(singular) --gt turtle. noun(plura
l) --gt turtles. intransitive_verb(singular) --gt
sleeps. intransitive_verb(plural) --gt sleep.
Agreement
48
Case Marking
pronoun(singular,nominative) --gt
heshe pronoun(singular,accusative) --gt
himher pronoun(plural,nominative) --gt
they pronoun(plural,accusative) --gt
them sentence --gt np(Number,nominative),
vp(Number) vp(Number) --gt v(Number),
np(_,accusative) np(Number,Case) --gt
pronoun(Number,Case) np(Number,_) --gt det,
n(Number)
He sees her. She sees him. They see her. But
not Them see he.
49
Subcategorization
vp --gt v(1). v(1) --gt sleep vp
--gt v(2), np. v(2) --gt chase vp --gt v(3),
np, np. vp --gt v(4), s.
Verb Complement Example Sleep None The cat
slept Chase One NP The cat chased the
dog Give Two NP John gave Bill the
book Say sentence John said he loved Mary.
50
Constructing parse trees
sentence(s(NP,VP)) --gt noun_phrase(NP),verb_phrase
(VP). noun_phrase(np(N)) --gt proper_noun(N). noun_
phrase(np(Art,Adj,N)) --gt article(Art),adjective(A
dj), noun(N). noun_phrase(np(Art,N)) --gt
article(Art),noun(N). verb_phrase(vp(IV)) --gt
intransitive_verb(IV). verb_phrase(vp(TV,NP)) --gt
transitive_verb(TV), noun_phrase(NP). article(ar
t(the)) --gt the. adjective(adj(lazy)) --gt
lazy. adjective(adj(rapid)) --gt
rapid. proper_noun(pn(achilles)) --gt
achilles. noun(n(turtle)) --gt
turtle. intransitive_verb(iv(sleeps)) --gt
sleeps. transitive_verb(tv(beats)) --gt beats.
?-phrase(sentence(T),achilles,beats,the,lazy,turt
le)T s(np(pn(achilles)), vp(tv(beats),
np(art(the), adj(lazy),
n(turtle))))
51
Prolog goals in grammar rules
numeral(N) --gt n1_999(N). numeral(N) --gt
n1_9(N1),thousand,n1_999(N2), N is
N11000N2. n1_999(N) --gt n1_99(N). n1_999(N) --gt
n1_9(N1),hundred,n1_99(N2), N is
N1100N2. n1_99(N) --gt n0_9(N). n1_99(N) --gt
n10_19(N). n1_99(N) --gt n20_90(N). n1_99(N) --gt
n20_90(N1),n1_9(N2),N is N1N2. n0_9(0) --gt
. n0_9(N) --gt n1_9(N). n1_9(1) --gt
one. n1_9(2) --gt two. n10_19(10) --gt
ten. n10_19(11) --gt eleven. n20_90(20) --gt
twenty. n20_90(30) --gt thirty.
?-phrase(numeral(2211),N). N two, thousand,
two, hundred, eleven
52
DCGs summary

• Specifying the grammar one obtains the parser
  (almost) for free!
• DCGs extend context-free grammars in two
  respects
• arguments can be added to non-terminal symbols
• Prolog goals can be added to the right-hand sides
  of grammar rules

53
Interpretation and Answering Questions
54
Interpretation (1)

• The meaning of the proper noun Socrates is the
  term socrates
• proper_noun(socrates) --gt socrates.
• The meaning of the property mortal is a mapping
  from terms to literals containing the unary
  predicate mortal
• property(Xgtmortal(X)) --gt mortal.
• The meaning of a proper noun - verb phrase
  sentence is a clause with empty body and head
  obtained by applying the meaning of the verb
  phrase to the meaning of the proper noun
• sentence((L-true)) --gt proper_noun(X),verb_phrase
  (XgtL).?-phrase(sentence(C),socrates,is,mortal)
  .C (mortal(socrates)-true)

55
Exercise 7.4

• A transitive verb is a binary mapping from a pair
  of terms to literals
• transitive_verb(YgtXgtlikes(X,Y)) --gt likes.
• A proper noun instantiates one of the arguments,
  returning a unary mapping
• verb_phrase(M) --gt transitive_verb(YgtM),proper_no
  un(Y).

56
Interpretation (2)

• sentence((L-true)) --gt proper_noun(X),
  verb_phrase(XgtL).sentence((H-B)) --gt
  every, noun(XgtB),
  verb_phrase(XgtH).verb_phr
  ase(M) --gt is property(M).
• property(M) --gt a, noun(M).property(Xgtmor
  tal(X)) --gt mortal.
• proper_noun(socrates) --gt socrates.
• noun(Xgthuman(X)) --gt human.

57
Interpretation (3)
?-phrase(sentence(C),S). C human(X)-human(X)S
every,human,is,a,human C
mortal(X)-human(X)S every,human,is,mortal
C human(socrates)-trueS socrates,is,a,huma
n C mortal(socrates)-trueS
socrates,is,mortal
58
Determiners

• Determiner sentences have the form every/some
  noun verb-phrase (NB. meanings of some
  sentences require 2 clauses)
• sentence(Cs) --gt determiner(M1,M2,Cs),noun(M1),ve
  rb_phrase(M2).
• determiner(XgtB, XgtH,(H-B)) --gt every.
• determiner(skgtH1,skgtH2,(H1-true),(H1-true)
  --gt some.
• ?-phrase(sentence(Cs),D,human,is,mortal).
• D every, Cs (mortal(X)-human(X))
• D some, Cs (human(sk)-true),(mortal(sk)-tr
  ue)

59
Questions

• question(Q) --gt who, is, property(XgtQ).
• question(Q) --gt is, proper_noun(X), property
  (XgtQ).
• question((Q1,Q2)) --gt is, some, noun(skgtQ
  1), property(skgtQ2).

60
Querying a rulebase

• handle_input(Question,Rulebase)-
  phrase(question(Query),Question), question
  prove_rb(Query,Rulebase),!, solveable
  transform(Query,Clauses), transform
  phrase(sentence(Clauses),Answer), to answer
  show_answer(Answer), nl_shell(Rulebase).