David W. Etherington

1
Improving Coalition Performance by Exploiting
Phase Transition BehaviorFinal Briefing

• David W. Etherington
• Andrew J. Parkes
• CIRL, University of Oregon

2
Administrative

• Project Title Improving Coalition Performance
  by Exploiting Phase Transition Behavior
• Program Manager Vijay Raghavan
• PI Names David Etherington, Andrew Parkes
• PI Phone Numbers 541-346-0472, 0434
• PI E-Mail Addressesether, parkes_at_cirl.uoregon.e
  du
• Company/Institution CIRL / University of Oregon
• Contract Number F30602-00-2-0543
• AO Number K273/00
• Award Start Date 06/27/2000
• Award End Date 11/30/2003 (?)
• Agent Name/Organization Gary Chaffins,
• Office of Research Services Administration, Univ.
  of Oregon

3
Problem Description/Objective

• Develop lightweight, robust mechanisms, not
  subject to computational cliffs, to facilitate
  the coordination of autonomous teams
• Challenges
• strict real-time constraints
• stringent communication and coordination
  restrictions
• scaling
• Approaches
• static implement architectures guaranteed not to
  raise difficult problems
• dynamic ensure that hard problems can be
  detected and made manageable on the fly

4
Problem Description Objective

• Model peaks and cliffs in computational/communicat
  ion cost and develop mechanisms to help ANT
  systems avoid them
• theoretical and experimental results
• Develop infrastructure and tools to
• detect infeasibility transitions by monitoring
  derived constraints and phase transition info
• relax constraints to avoid infeasibility
• develop resource-bounded distributed algorithms
• aggregate search information to guide ANT
  coalitions

5
Improving Coalition Performance by Exploiting
Phase Transition Behavior
Etherington/Parkes CIRL/University of Oregon
New Ideas
Problem and Challenge

• Innovative monitoring of derived constraints to
  detect
• imminent computational cliffs
• misalignment of coalition problem structures
• Use of local solution clusters to enhance
  robustness of negotiation architectures
• Application of effectively parallelizable
  techniques to enhance robustness with limited
  communication

Enhanced Locality
Robustness indications
Emphasize local decisions
Impact
FY03 Schedule

• Avoidance of non-local dependencies
• reduced communication/coordination needs
• better scaling
• Better solutions faster, more predictably
• distributed resource allocation, logistics,
  tracking
• mitigate computational cliffs in large problems
• More robust solutions to operational problems
• real time adaptation of distributed workload

• 1QFY03
• Distributed version of PARIS
• 2QFY03
• Genericize key ideas
• Robust control of distributed systems
• 3QFY03
• Distributed avoidance of cliffs
• 4QFY03
• End of project

6
Overview

• Scaling behavior
• thresholds vs phase transitions
• sequential vs parallel
• Coarse-grained search and local robustness
• Enhanced representations and learning

7
Local Search Experimental Results

• Simple local search on standard SAT benchmark

n
Satisfiability phase transition is at 4.2
8
Scaling is Poorly Predicted by Small n

• Linearsuper-polynomial threshold in large
  theories
• algorithm-dependent
• semantics are unclear (not backbones)
• marks boundary of ANTS-accessible region?

Threshold is not due to closeness to PT
9
Thresholds and Predictable Dynamics

• Below the cliff the algorithm is
• well-behaved
• low variance/analyzable
• relatively insensitive to algorithm tuning
• Above the cliff
• badly-behaved
• too slow
• high variance/unpredictable

10
Fine-grained Distribution

• Few variables/constraints per coalition
• problem arises in trying to coordinate
• Algorithms of interest will be
• highly localized
• demonstrated effectiveness in sequential case(?)
• Good candidate random walk iterative repair
  algorithms
• Reasonable match for ANTs because
• repairs can selected and done in parallel
• repairs can be selected by local negotiations

11
Satisfiability Problems WSAT

• Sequential WSAT
• P random assignment
• loop
• c randomly selected violated clause
• l heuristically selected literal from c
• flip value of l in P
• Parallel WSAT
• P random assignment
• loop
• parallel foreach violated clause c
• l(c) heuristically selected literal
  from c
• flip all l(c) in P
• Equivalent to having O(n) processors.

12
Parallel WSAT on Satisfiable Problems

• Random 3SAT. Almost always satisfiable for
  cls/var
• Average parallel time. Lines are best-fit
  quadratics in log(n)

clause/ variable ratios
time
Number of variables, n
Sequential search is O(n). Using O(n) parallel
processors, reduces it to O(log(n)2).
13
Scaling for Optimization

• Under-constrained find a satisfying assignment
• Over-constrained
• find assignments violating fewest constraints
• Good-enough soon-enough context
• harder to define meaning of scaling because we
  need to define how good a solution we obtain.
• Working definition
• set target quality to what sequential WSAT can
  achieve in a controlled time
• e.g. O(n) or O(n log(n))
• find time for parallel search to achieve target
  quality
• Allows useful comparison of sequential/parallel
  scaling

14
Scaling Sequential vs Parallel WSAT
Over-constrained random 3SAT

• Select target qualities so that
• sequential WSAT takes O(n)
• then parallel WSAT takes O(1)

• Select target qualities so that
• sequential WSAT takes O(n log(n) )
• then parallel WSAT takes O(log(n))

Result eliminating interfering repairs greatly
aids the scaling
15
Lessons Phase Transitions Thresholds

• Degradation of scaling can be abrupt even for
  local search. Clean threshold between
• Acceptable
• linear for centralized algorithm
• parallel performance is also good polylog
• small variance
• Unacceptable
• exponential
• large variance
• Thresholds algorithm-dependent, and distinct from
  phase transition
• This applied even to local search previous
  perception was that it is less susceptible

16
Overview

• Scaling behavior
• Coarse-grained search and local robustness
• coping with change
• solution clusters
• Enhanced representations and learning

17
Coupling and Robust Solutions

• Coarse-grained behavior depends critically on
  scope of interaction between coalitions/systems.
• Motives
• Sensible partition choice requires that first
  optimize interactions
• Failure of interaction should suggest a
  re-partition

• Using locally robust solutions can reduce search
  costs.

18
Self-Reliance and Cooperation

• Self-reliance
• prefer solutions that rely less on other
  coalitions
• coarse-grained least-constrained heuristics

• Cooperation
• avoid solutions that over-constrain others
• coarse-grained least-constraining heuristics
• Advantages
• reduce need for renegotiation
• reduce communication with other coalitions

19
Public/Private Internal/External Variables
public variables
Node or coalition
Communication of values for variables

• Coarse-grained distribution suggests division
• private variables values not needed by other
  nodes
• public variables values shared between nodes
• internal owned by node, but used by others
• external used by node, but owned by another

20
Robust Solutions from Intra-Coalition Search

• Fast distributed search benefits from decreased
  dependencies between sub-solutions.
• Local solution should maximize decoupling
• avoid distracting neighbors
• avoid being distracted by neighbors
• Approach add soft constraints to favor solutions
  under local control
• Example for the constraint pub(1) v priv(2)
  also add soft priv(2)

21
Soft Constraints for Self-Reliance

• Each constraint involves
• internal vars controlled by the coalition, e.g.
  Use(S1), Use(S2)
• external vars controlled by some other
  coalition, e.g. Use(S3)
• Bias solutions toward reliance on internally
  controlled choices.
• e.g. given C1s constraint Use(S1) or Use(S2)
  or Use(S3)
• add the soft auxiliary constraint Use(S1) or
  Use(S2)

Coalition C1
Coalition C2
S2
S3
Comms
S1
22
Effectiveness of Self-Reliance

• Two coalitions
• separately find solutions
• merge in single round
• Naive
• no attempt to be robust
• Robust
• biased to self-reliance

Violated Constraints
Constraints

• Result simple bias towards robust, self-reliant,
  solutions
• significantly improves performance
• reduces need for further negotiation rounds

23
Quantifying Robustness

• How far can solutions be tweaked before breaking?
• Achievable robustness the maximum percentage of
  variables that can be reset in some solution
• higher percentage means more robust
• Given the constraint x or y, the solution
  xytrue is the most robust possible
• either of x or y can be reset
• achievable robustness is 100

24
Robustness Phase Transition

• Random 3-SAT
• achievable robustness depends on clause/ variable
  ratio
• sharp transition from almost always achievable to
  almost never achievable

Average achievable robustness (with 10,90th
percentiles)
Clause/Variable ratio
Phase transition lets us predict achievable
robustness.
25
Good and Robust

• Q nominal solution quality R solution
  robustness

R
cliff region
achievable region (below PT line)
Q
26
Good and Robust

• Expected solution quality (utility) f(R,Q)
  
• Q nominal solution quality R solution
  robustness

R
Contours of
cliff region
Target values of R and Q (depends on interaction
of f and PT)
achievable region (below PT line)
Q
27
Good and Robust

• Expected solution quality (utility) f(R,Q)
  
• Q nominal solution quality R solution
  robustness

R
Contours of
cliff region
Target values of R and Q (depends on interaction
of f and PT)
achievable region (below PT line)
Q
Upper computation trajectory minimizes time in
cliff region

• Robustness must be a goal from the outset.
• Complexity arguments underlie this insight.

28
Cooperation

• A solution cluster is a set of solutions with
• small set of forced variables
• other variables are relatively unconstrained
• Use to reduce constraints on other members

29
Solution Clusters

• Inside coalition
• generate initial solution
• scan for variables whose values are inessential,
  and unset them
• Instead of total assignment, T, send
• partial assignment
• residual constraints

30
Cluster Experiment

• Interacting coalitions. Each coalition
• accepts conditions from previous coalition
• solves its local problem, if still satisfiable
• sends to next coalition, one of
• simple total solution
• small cluster minimal effort to compute
• larger cluster more effort to compute
• all solutions for reference only
• Measure
• probability of success without backtracking

31
Effectiveness of Cooperation
Measure probability of successful integration of
local solutions.

• Result Solution clusters can markedly improve
  success rates.

32
Summary

• Focusing on robust solutions
• enhances self-reliance and cooperation
• aids in coping with lack of knowledge
• reduces communications demands
• Phase transitions and computational cliffs
• help predict achievable robustness
• guide anytime increases in robustness
• help manage be robust vs act in time conflict

33
Lessons System-of-systems Robustness

• Defn System-of-systems internals have
  significant complexity
• Interactions enhanced by local robustness
• Be strong self-reliance
• Be nice cooperation solution clusters
• Phase transition effects do show up
• Can treat robustness as an objective and have the
  PT in quality, but now is multi-dimensional
• Robustness vs. optimization tradeoff
• Solution clusters rely on properties of the PT

34
Overview

• Scaling behavior
• Coarse-grained search and local robustness
• Enhanced representations and learning
• pseudo-Boolean
• OPARIS

35
SAT/CSP vs. PB

• SAT/CSP representations are bad for counting
• representation and reasoning both blow up
• Pseudo-Boolean (PB) representation is better
• linear inequalities on Boolean, 0/1, variables
  e.g. x1 x2 x3 x4 2 x5 3

36
Basic Ideas of OPARIS

• Takes PB representation
• Based on standard SAT methods zChaff, Berkmin
• branch propagate, rather than iterative repair
• conflict driven learning zChaff, and others
• rapid restarts Cornell inspired
• tight, but highly adaptive, focus on current
  region of search tree Berkmin
• Optimization, not just decision, problems
• takes objective function, and tightens it, until
  times out or proves optimal

37
PB OPARIS

• Comparable raw speed to best SAT solver
• Uses conflict analysis to discover entailed
  constraints (conflict clauses, no-goods) to
  reduce redundant search.
• The internal reasoning exploits piggybacking
• allows learning of more complex constraints

38
Piggybacking in PB

• Search State d0, b1, c1, e0, f0
• Unsat constraints abcd 3 a e f
  1
• SAT blame only d0, e0, f0 generates
  only b v e v f
• PARIS also blame b1, c1,
  generates bcdef 3
• which is equivalent to multiple clauses

39
Pigeon-Hole Problem

• Can n pigeons be placed into n-1 holes?
• Can n planes be repaired by n-1 mechanics?
• SAT provably exponential effort to solve
• PB polytime proofs exist, due to piggybacking
• Does OPARIS see these gains automatically?

40
OPARIS on Pigeonhole
PARIS

zchaff


sec

preproc
solve

hole9.cnf

3

0

0

hole10.cnf

24

0

0

hole11.cnf

156

0

0

hole50.prs



95

0

• Preprocessing by OPARIS is the step that enables
  the piggybacking

41
Application to SNAP

• OPARIS is a general (PB) solver
• not limited to ATTEND/SNAP instances
• knows only about PB heuristics
• Does not have SNAP-specific heuristics
• trade (possible) performance loss for more
  flexibility
• Goal Learn about how to use general-purpose
  tools to support the negotiation system
• expect that nature of instances and requirements
  might be quite different from those found in
  doing standard optimization

42
Surprise Nature of SNAP Instances

• Problems are very dilute
• relatively large number of variables
• problem is relatively simple
• small number of backtracks needed
• Approach used
• keep standard heuristics, but add new methods to
  account for diluteness

43
Status

• Successes
• Exploit pure/non-occurring literals
• Use history
• Prefer to branch on decision variables
• Partial success
• Learning
• Failures
• Look-ahead by failed literals yielded no gain
  (because instances are so dilute?)

44
Success Using PNO Literals

• A literal is pure or non-occurring (PNO) if it
• does not occur in an unsatisfied clause, or
• only occurs with one sign
• E.g., many variables occur only negatively
• can then be set to false.
• Exploiting PNO structures
• If we enforce x and propagate, we can often find
  another variable y that has become PNO.
• e.g. suppose y becomes pure negative, then
  enforce a rule x y
• Without this, even trivial instances can be very
  slow.
• It is important not to do too much
• just use for preprocessing too expensive to do
  at each node
• only do for selected literals most literals are
  fruitless

45
Success Learn From History

• History remember the previous value of a
  variable, re-use it unless propagation forces
  otherwise Unitwalk, Hirsch et al
• Particularly useful in dilute problems
• usual argument Dont need history since good
  decisions are remembered implicitly by
  remembering bad ones.
• probably true for hard concise problems.
• in dilute easy problems, many decisions may
  happen to have been made correctly, even though
  we do not learn a constraint to force it
• Surprise this is a clear win even though we make
  no effort to remember only good values
• backtracking causes bad values to be changed, but
  leaves good ones alone

46
Success Learn From History

• Example of OPARIS on an ATTEND instance

47
Success Exploit Decision Variables

• Branching rules are made aware of the decision
  variables.
• Encourage system to branch on the switch
  variables that control whether a task is done,
  and at what time it should be started.

48
Partial Success PB Learning

• On conflict, learn a derived constraint.
• Current options are
• learn full PB constraint
• 2 x 2 y w z 2
• learn cardinality constraint
• x y w 2
• Expected full PB would do better, as it learns
  more per conflict.
• in practice, cardinality generally seems better
• Suspicion current learning methods can miss
  important useful constraints.
• Need to explore better focusing mechanisms

49
Overall Success

• Example

• OPARIS now orders of magnitude faster
• Beats local search (WSAT(OIP)) on some ATTEND
  instances

50
Exploit Time Limits?

• Different parameters alter performance curves
• No best set of parameters
• best algorithm depends slightly on time limit

• Possibly should switch depending on context
• have done this for partial restarts on
  tightening
• Need to understand performance curves well enough
  to switch heuristics based on the time limit.
• For demo instances, effects were too weak to be
  useful

51
Lessons

• Nature of problem
• problems tend to be dilute
• i.e. large, but easy for their size
• exploiting opportunities provided by PB for
  learning is hard strongly suspect can do a lot
  better
• need to preserve accidentally good decisions
• dont want to lose them on backtracking
• Can beat local search, WSAT(PB), on instances
  from Cornell encoding.
• we presume/expect that this is because there are
  chains of implications, on which local search is
  known to do badly.

52
Lessons OPARIS on SNAP

• Using a more compact, PB vs. SAT, representation
  does help
• Potential for learning during search to greatly
  reduce search
• Hard to fully exploit this potential
• Complete search is relatively slow, and not
  always needed, so
• Needed to focus effort of complete solver onto a
  reformulation of problem (by ATTEND/Cornell)
• How to select the best reformulation/abstraction
  (granularity or sub-problem) is not understood

53
Lesson? / Program Issue

• Need clearer distinction between
• peer-to-peer horizontal, system-of-systems
  negotiation, e.g.
• CP sensors
• SNAP-MAPLANT
• general-to-specific vertical negotiation and
  reformulation, e.g.
• planner-scheduler,
• scheduler-executor
• SNAP-OPARIS

54
Deliverables

• Final report Publications
• E-Book
• OPARIS webpage download

55
Deliverables Publications

• Generalizing Boolean Satisfiability I Background
  and Existing Work. Heidi E. Dixon, Matthew L.
  Ginsberg, and Andrew J. Parkes submitted to
  JAIR.
• Generalizing Boolean Satisfiability II Theory.
  Heidi E. Dixon, Matthew L. Ginsberg, and Andrew
  J. Parkes in preparation, to be submitted to
  JAIR.
• Scaling Properties of Pure Random Walk on Random
  3-SAT. Andrew J. Parkes. Proceedings of the
  Eighth International Conference on Principles and
  Practice of Constraint Programming (CP2002).
  Published in Lecture Notes in Computer Science,
  LNCS 2470. Pages 708--713.
• Easy Predictions for the Easy-Hard-Easy
  Transition. Andrew J. Parkes. Eighteenth Natl
  Conference on Artificial Intelligence (AAAI-02)
• Likely Near-Term Advances in SAT Solvers.
  Heidi E. Dixon, Matthew L. Ginsberg, Andrew J.
  Parkes, at MTV-02.
• Inference methods for a pseudo-Boolean
  satisfiability solver. Heidi E. Dixon and
  Matthew L. Ginsberg. AAAI-02.
• Exploiting Solution Clusters for Coarse-grained
  Distributed Search Andrew J. Parkes. Proc.
  Distributed Constraint Reasoning, at the
  International Joint Conference on Artificial
  Intelligence (IJCAI-01)
• Distributed Local Search, Phase Transitions, and
  Polylog Time Andrew J. Parkes. Proc. Stochastic
  Search Algorithms, at IJCAI-01.

56
Project Schedule and Milestones
57
Technology Transition/Transfer

• Exploring potential use of PARIS solver in
  microprocessor verification.
• existing SAT solvers are already being used in
  that field and using pseudo-Boolean rather than
  SAT should allow larger problems to be solved.
• paper presented at Microprocessor Testing and
  Verification MTV02.)
• Advised in ATTEND pseudo-Boolean conversion
• Participation of OPARIS in SNAP final demo
• Inclusion of OPARIS in SNAP distribution

58
Program Issues

• N/A

59
Phase Transitions Computational Cliffs
Time to find a solution, or show none exist
A priori probability solution exists
1.0
Easy
Hard
Easier
0.0
Critical region
Solution Quality

• Problem character changes in the critical region
• Computational cliff Improvements become expensive

60
Sequential Search Derived Constraints

• Experiments on detecting precursors of
  computational cliffs
• Artificial domain based on SNAP
• grid of variables mimics resource vs time
• localized constraints to give more realistic
  structure
• switches to activate sets of constraints (e.g.,
  missions/sorties)
• if s(i) then (local constraints on grid)
• Simple greedy search approach
• maximize of switches set ON, while satisfying
  active constraints
• e.g., maximize number of sorties that can be
  scheduled

61
SNAP-like Probe Easy-Hard-Stuck

• The position of the cliff bottom can be predicted
  using phase transition information.

62
Unary-Prime-Implicates (UPIs)

• UPIs are variables with the same value in all
  solutions
• UPIs emerge near easy-hard transition
• multiply rapidly during hard phase

• This suggests using them to predict imminent
  cliffs.
• Too hard for ANTS?

Number sorties/switches turned ON
63
Analyzing Cliffs

• Previous attempts at analysis fail because of
  focus on PT region
• too hard to analyze
• too hard to reach it in real-time anyway
• Complexity analysis is easier at cliff below
  PT.Better chance to
• predict initial progress
• predict when leaving easy region

64
Phase Transitions vs Thresholds

• Phase Transitions
• semantic
• algorithm independent
• often hardexponential timeANTS never reach
  them?
• Thresholds
• algorithm-dependent
• semantics are unclear (not backbones)
• marks boundary of ANTS-accessible region?
• Both indicate regions of high predictability.

65
Achievable Robustness
time
robustness

• initial solutions often brittle

66
Achievable Robustness
time
robustness

• initial solutions often brittle
• more robust solutions available cheaply

67
Achievable Robustness
time
robustness
Complexity arguments identify robustness as a
reasonable computational goal in this environment.