Constraint Based Systems

1
Constraint Based Systems

• Reading Chapter 6
• Chess article (see links)

2
Agenda

• Finishing up search for machine translation
• Constraint satisfaction

3
Hill climbing

• function HillClimbing(problem, initial-state,
  queuing-fn)
• node ? MakeNode(initial-state(problem))
• while T do
• next ? Best(SearchOperator-fn(node,cost-fn))
• if(IsBetter-fn(next, node)) then continue
• else if(GoalTest(node)) then return node
• else exit
• end while
• return Failure

MT (Germann et al., ACL-2001) node ?
targetGloss(sourceSentence) while T do next
? Best( LocallyModifiedTranslationOf(node))
if(IsBetter(next, node)) then continue else
print node exit end while
4
Types of changes

• Translate one or two words (j1e1j2e2)
• Translate and insert (j e1 e2)
• Remove word of fertility 0 (i)
• Swap segments (i1 i2 j1 j2)
• Join words (i1 i2)

5
Example

• Total of 77,421 possible translations attempted

6
(No Transcript)
7
(No Transcript)
8
How to search better?

• MakeNode(initial-state(problem))
• RemoveFront(Q)
• SearchOperator-fn(node, cost-fn)
• queuing-fn(problem, Q, (Next,Cost))

9
Example 1 Greedy Search MakeNode(initial-state(p
roblem))
Machine Translation (Marcu and Wong,
EMNLP-2002) node ? targetGloss(sourceSentence) w
hile T do next ? Best( LocallyModifiedTranslat
ionOf(node)) if(IsBetter(next, node)) then
continue else print node exit end while
10
Climbing the wrong peak
What sentence is more grammatical? 1. better bart
than madonna , i say 2. i say better than bart
madonna ,
Can you make a sentence with these words? a
and apparently as be could dissimilar firing
identical neural really so things thought
two
11
Language-model stress-testing

• Input bag of words
• Output best sequence according to a linear
  combination of an
• ngram LM
• syntax-based LM (Collins, 1997)

12
Size 3-7 words long

• Best searched
• 32.3 i say better than bart madonna ,
• Original word order
• 41.6 better bart than madonna, i say

SBLM trained on an additional 160k WSJ
sentences.
13
Cryptograms

• QFL HCVPS
• PX V ANSWLCEZK NCJVS PQ XQVCQX QFL BPSZQL RNZ
  JLQ ZT PS QFL BNCSPSJ VSW WNLX SNQ XQNT ZSQPK RNZ
  JLQ QN QFL NEEPGL CNHLCQ ECNXQ

14
Cryptograms

• QFL HCVPS
• THE BRAIN
• PX V ANSWLCEZK NCJVS PQ XQVCQX
• IS A WONDERFUL ORGAN IT STARTS
• QFL BPSZQL RNZ JLQ ZT PS QFL BNCSPSJ
• THE MINUTE YOU GET UP IN THE MORNING
• VSW WNLX SNQ XQNT ZSQPK RNZ JLQ QN
• AND DOES NOT STOP UNTIL YOU GET TO
• QFL NEEPGL
• THE OFFICE CNHLCQ ECNXQ
• ROBERT FROST

15
Constraint Satisfaction Problems

• Standard representation allows a general-purpose
  approach to search
• Set of variables, X1, X2 Xn
• Each variable has domain Di of possible values
• Set of constraints, C1, C2, Cn
• State assignment of values to some or all
  variables
• Xiv,i , Xjvj
• Solution complete assignment that satisfies all
  constraints
• Some solutions maximize an objective function

16
Cryptograms

• QFL HCVPS
• PX V ANSWLCEZK NCJVS PQ XQVCQX QFL BPSZQL RNZ
  JLQ ZT PS QFL BNCSPSJ VSW WNLX SNQ XQNT ZSQPK RNZ
  JLQ QN QFL NEEPGL CNHLCQ ECNXQ

17
Cryptograms

• QFL HCVPS
• THE BRAIN
• PX V ANSWLCEZK NCJVS PQ XQVCQX
• IS A WONDERFUL ORGAN IT STARTS
• QFL BPSZQL RNZ JLQ ZT PS QFL BNCSPSJ
• THE MINUTE YOU GET UP IN THE MORNING
• VSW WNLX SNQ XQNT ZSQPK RNZ JLQ QN
• AND DOES NOT STOP UNTIL YOU GET TO
• QFL NEEPGL
• THE OFFICE CNHLCQ ECNXQ
• ROBERT FROST

18
CSP algorithm

• Depth-first search often used
• Initial state the empty assignment all
  variables are unassigned
• Successor fn assign a value to any variable,
  provided no conflicts w/constraints
• All CSP search algorithms generate successors by
  considering possible assignments for only a
  single variable at each node in the search tree
• Goal test the current assignment is complete
• Path cost a constant cost for every step

19
Local search

• Complete-state formulation
• Every state is a compete assignment that might or
  might not satisfy the constraints
• Hill-climbing methods are appropriate

20
Dimensions of CSPs

• Discrete/finite domains
• Map coloring
• 8 queens
• Nonlinear constraints
• No general solution
• Unary constraints
• Absolute constraints

• Infinite domains
• Scheduling calendars (discrete)
• Schedule experiments for Hubble
• Linear constraints
• Binary/N-ary constraints
• Preferences
• Prof. McKeown prefers after 1pm

21
(No Transcript)
22

• M GWDX URNMRPR MD QD QCXRLNMCR, QNXFWZOF M QH
  ULMDOMDO Q KFQDOR WC ZDGRLARQL. AWWGS QNNRD

23

• M GWDX URNMRPR MD QD A BCDE A
  AD DQCXRLNMCR, QNXFWZOF M QH E
  A E
• ULMDOMDO Q KFQDOR WC AD AD
  D C ZDGRLARQL.
• DB
• AWWGS QNNRD
• B
  D

24

• M GWDX URNMRPR MD QD A BCDI A
  AD DQCXRLNMCR, QNXFWZOF M QH I
  A I
• ULMDOMDO Q KFQDOR WC AD AD
  D C ZDGRLARQL.
• DB
• AWWGS QNNRD
• B
  D

25

• M GWDX URNMRPR MD QD A BCDO A
  AD DQCXRLNMCR, QNXFWZOF M QH O
  A O
• ULMDOMDO Q KFQDOR WC AD AD
  D C ZDGRLARQL.
• DB
• AWWGS QNNRD
• B
  D

26

• M GWDX URNMRPR MD QD A BCDU A
  AD DQCXRLNMCR, QNXFWZOF M QH U
  A U
• ULMDOMDO Q KFQDOR WC AD AD
  D C ZDGRLARQL.
• DB
• AWWGS QNNRD
• B
  D

27

• M GWDX URNMRPR MD QD A BCE A
  AE EQCXRLNMCR, QNXFWZOF M QH
  A
• ULMDOMDO Q KFQDOR WC AE AE
  E C ZDGRLARQL.
• EB
• AWWGS QNNRD
• B
  E

28
What additional knowledge about language would be
helpful?
29
General purpose methods for efficient
implementation

• Which variable should be assigned next?
• in what order should its values be tried?
• Can we detect inevitable failure early?
• Can we take advantage of problem structure?

30
Order

• Choose the most constrained variable first
• The variable with the fewest remaining values
• Minimum Remaining Values (MRV) heuristic
• What if there are gt1?
• Tie breaker Most constraining variable
• Choose the variable with the most constraints on
  remaining variables

31
Order on value choice

• Given a variable, chose the least constraining
  value
• The value that rules out the fewest values in the
  remaining variables

32

• M GWDX URNMRPR MD QD I
  I I AQCXRLNMCR, QNXFWZOF M QHA
  I A I A
• ULMDOMDO Q KFQDOR WC I I
  A A ZDGRLARQL.
  A AWWGS QNNRD
• 
  A

33

• M GWDX URNMRPR MD QD I N
  I IN ANQCXRLNMCR, QNXFWZOF M QHA
  I A I A
• ULMDOMDO Q KFQDOR WC IN IN A
  AN ZDGRLARQL. N A AWWGS
  QNNRD
• 
  A N

34
Forward Checking

• Keep track of remaining legal values for
  unassigned variables
• Terminate search when any variable has no legal
  values

35

• M GWDX URNMRPR MD QD I
  I I AQCXRLNMCR, QNXFWZOF M QHA
  I A I A
• ULMDOMDO Q KFQDOR WC I I
  A A ZDGRLARQL. AWhen M
  I selected D (F N S T)
• When QA selected D (N S T)

36

• M GWDX URNMRPR MD QD I DONT EL I E E
  IN ANQCXRLNMCR, QNXFWZOF M QHA FTERL I FE
  ALTHO GH I AM
• ULMDOMDO Q KFQDOR WC R IN G I NG A
  HANGE OFZDGRLARQL. NDERWEAR
• AWWGS QNNRD
• WOODY
  ALLEN

37

• M GWDX URNMRPR MD QD I DONT CEL I E E IN
  ANQCXRLNMCR, QNXFWZOF M QHA FTERL I FE
  ALTHO GH I AM
• ULMDOMDO Q KFQDOR WCCR IN G I NG A
  HANGE OFZDGRLARQL. NDERWEAR
• Select UC -gt the domain of P is empty. Requires
  backtracking even before taking the next branch.
  Avoids selection Z, K next before discovering the
  error.

38

• M GWDX URNMRPR MD QD I DONT BEL I E VE IN
  ANQCXRLNMCR, QNXFWZOF M QHA FTERL I FE ALTHO
  UGH I AM
• ULMDOMDO Q KFQDOR WCBR IN G I NG A CHANGE
  OFZDGRLARQL.UNDERWEAR
  AWWGS QNNRD
• WOODY
  ALLEN

39
Speed-ups
Problem Backtracking BTMRV Forward Checking FVMRV
USA (gt1000K) (gt1000K) 2K 60
N-Queens (gt40000K) 13,500K (gt40000K) 817K
Zebra 3,859K 1K 35K .5K
Random1 415K 3K 26K 2K
Random2 942K 27K 77K 15K
40
Other types of improvements

• Constraint propagation
• Propagate implications of a constraint on one
  variable onto other variables. Not only values,
  but constraints between other variables
• Intelligent backtracking
• Conflict-directed backtracking Save possible
  conflicts for a variable value on assignment.
• Back jump to assignment of values

41
Crossword Puzzles