Game Driven Software Development for NPOs

1
Game Driven Software Development for NPOs

• the Scientific Community Game (SCG)

2
NP Optimization Problems (NPOs)

• NPOs are approximated using (ensembles of)
  heuristics.
• Foster development and innovation of heuristics.

3
Fostering Heuristics Development

• Feedback!
• Analyze the performance of heuristics in a niche
  to form better ensembles.
• Parameter tuning.
• Bug fixes.

4
Fostering Heuristics Innovation

• Analyzing the niches within the problem domain.
• Constructing hard problems.
• Hints!

5
Game Driven Software Development

• A number of autonomous teams.
• Each team develops an agent that embodies their
  own heuristics.
• Agents participate in a contest.
• Contest winners get an egoistic boost.
• Teams develop their agents for the next contest.

6
Why It Works?

• Autonomy teams/agents are free to innovate and
  develop their heuristics.
• Mastery getting better supports the
  teams/agents ego.
• Purpose accumulate new shared knowledge about a
  specific NPO.

7
Game Driven Development Has Worked Before!

• Renaissance mathematicians.
• SAT competitions.

8
The SCG(X) Game
9
The SCG(X) Game

• X is a specific predefined NPO problem Domain.
  e.g. Boolean-MAX-CSP.
• In every round, agents must propose new
  hypotheses and oppose other agents hypotheses.
• Agents oppose hypotheses by either strengthening
  or challenging them.

10
The SCG(X) Game cont.

• Agents gain reputation when they strengthen
  hypotheses.
• Agents challenge hypotheses by engaging in a
  discounting protocol.
• Agents gain reputation when they discount
  hypotheses and lose reputation when they fail to
  do so.

11
The SCG(X) Game cont.

• All agents start with the same reputation. The
  sum of all reputations is preserved.
• Agent(s) with the highest reputation win(s).

12
We Didnt Tell You ...

• How do hypotheses look-like?
• What is the discounting protocol?
• How much reputation do agents gain/lose when they
  strengthen/discount hypotheses?

13
All-Conjectures

• Niche lower bound all problems in niche N of X
  can be solved with quality of at least Q.
• Niche upper bound there exists a problem in
  niche N of X cannot be solved with quality more
  than Q.
• For NPO problems, Q ? 0,1. ??

14
Example SCG(MAXCSP) Challenge Language 1 (all)

• Domain MAXCSP maximize fraction of satisfied
  constraints.
• Challenge Alice challenges Bob to discount the
  statement belief(pred, 0.7) There exists a
  problem p in subset pred so that for all
  solutions s to p, quality(p,s) lt 0.7. Confidence
  1.
• Discounting protocol Alice gives Bob p, Bob
  solves it with s quality(p,s) gt 0.7.

10/16/09
Can DM and ML help?
15
All-conjectures Example
16
Opposing Beliefs all
Q is too low Q is too high
for all P exists S Quality(P,S) gt Q Override. Opponent shares a belief with a higher Q. O Challenge. Opponent provides a problem. Belief holder solves it (S). O-
exists P for all S Quality(P,S) lt Q Challenge. Belief holder provides a problem. Opponent solves it (S). O- Override. Opponent shares a belief with a lower Q. O
17
Hypothesis
SQ Quality(P, SAlice)

• Alice Hypothesis There exists a problem P in
  niche N of X s.t. for all solutions SBob searched
  by the opponent Bob in T seconds. Quality(P,
  SBob) lt AR Quality(P, SAlice).
• Hypotheses have an associated confidence 0,1.
• Hypothesis ltN, AR, Confidencegt.

18
Example SCG(MAXCSP) Challenge Language 2
(secret)

• Karl 1 in three example.

10/16/09
Can DM and ML help?
19
Hypothesis Example

• 1in3 example.

20
X Boolean MAXCSP

• Given a sequence of Boolean constraints
  formulated using a set R of Boolean relations,
  find an assignment that maximizes the fraction of
  satisfied constraints.
• Is an NPO for most R. Decision version is
  NP-complete for most R.
• Niche defined by R.

21
1in3 niche
Truth Table 1in3 000 0 001 1 010 1 011
0 100 1 101 0 110 0 111 0

• Only relation 1in3 is used.
• 1in3 problem P

v1 v2 v3 v4 v5 1in3( v1 v2
v3) 1in3( v2 v4 v5) 1in3( v1
v3 v4) 1in3( v3 v4 v5) secret 1 0
0 1 0
Secret quality SQ 3/4
22
1in3 Hypothesis

• 1in3 hypothesis H proposed by Alice exists P in
  1in3 niche so that for all SBob that opponent Bob
  searches in time t (small constant) seconds
  Quality(P,SBob) lt 0.4 Quality(P,SAlice).
• H (niche (1in3), AR 0.4, confidence 0.8)
• Bob has clever knowledge that Alice does not
  have. He opposes the hypothesis H by challenging
  it using his randomized algorithm.

23
Bobs clever knowledge4/9 for 1in3

• 4/9 for 1in3 For all P in 1in3 niche, exists S
  so that Quality(P,S) gt 0.444 SQ.
• Proof la(p)3p(1-p)2 has the maximum 4/9.
• argmax p in 0,1 la(p) 1/3.
• Without search, in PTIME.
• Derandomize
• Bob successfully discounts
• Alice gets a hint
• Was Bob just lucky?

Truth Table 1in3 000 0 001 1 010 1 011
0 100 1 101 0 110 0 111 0
24
1in3 Hypothesis

• Bob does not know whether 4/9 is best possible.
  Should check Semidefinite Programming.
• Bob only knows that the set of 1in3 problems
  having a solution satisfying 4/9 eps, eps gt 0,
  is NP-complete.

25
Opposing Hypotheses
AR is too low AR is too high
exists P for all S that opponent searches Quality(P,S) lt AR SQ Challenge. Hypothesis proposer provides a problem. Opponent solves it. Strengthen. Opponent proposes a hypothesis with a lower AR.
26
Questioning Beliefs secret
Q is too low Q is too high
for all P exists S Quality(P,S) lt Q Discount. Share a belief with a higher Q. Challenge. Ask belief holder to provide a problem, then solve it (S).
exists P for all S Quality(P,S) gt Q Challenge. Provide a problem, then ask challenger to solve it (S). Discount. Share a belief with a lower Q.
27
All Vs. Secret Conjectures
All-conjectures Secret-conjectures
Absolute Certainty confidence 1 Uncertainty confidence lt1
Impossibility Small chance of success
statement about all possible assignments statement about assignments that one specific algorithm searches in a given time.
28
Properties of challenge language

• Doing discounting and supporting requires
  constructive skills. Uncertainty about which
  problem to be delivered.
• Optional mathematical skills
• When agents are perfect, supporting implies the
  statement is a theorem and discounting implies
  the statement is NOT a theorem (a counter example
  was found).

10/16/09
Can DM and ML help?
29
Reputation Gain

• Hypothesis have credibility 0,8. The
  credibility of a hypothesis is proportional to
  agents confidence in the hypothesis and agents
  reputation.
• Reputation gain is proportional to the
  discounting factor and the hypothesis
  credibility.
• The discounting factor -1,1. 1 means the
  hypothesis is completely discounted.

30
Discounting Factor
AR is too low AR is too high
exists P for all S that opponent searches Quality(P,S) lt AR SQ Quality(P,S) - AR SQ strengthens AR - AR.
31
Discounting Factor

• H1 ((1in3), AR 1.0, confidence 1.0)
• H1 proposed by Alice exists P in 1in3 niche so
  that for all S that opponent Bob searches
  Quality(P,S) lt 1.0 SQ.
• This is a reasonable hypothesis if Alice is sure
  that her secret assignment is the maximum
  assignment when she provides a sufficiently big
  problem to Bob.

32
What we did not tell you so far

• A game defines some configuration constants
• a maximum problem size
• For example, all problems in the niche can have
  at most 1 million constraints.
• A maximum time bound for all tasks (propose,
  oppose, provide, solve), e.g. 60 seconds.
• An initial reputation, e.g., 100. When reputation
  becomes negative, agent has lost.

33
Discounting Factor ReputationGain for
Strengthening

• H1 ((1in3), AR 1.0, confidence 1.0)
• H1 proposed by Alice exists P in 1in3 niche so
  that for all S that opponent Bob searches
  Quality(P,S) lt 1.0 SQ.
• Bob thinks he can strengthen H1 to H2 (MAXCSP,
  niche secret ExistsForAll (1in3), AR 0.9,
  confidence 1.0).
• DiscountingFactor 1.0-0.9 0.1.
• ReputationGain for Bob 0.1 1.0
  AliceReputation.
• Alice gets her reputation back if she discounts
  H2.

34
Discounting FactorReputationGain for Discounting

• H ((1in3), AR 0.4, confidence 1.0)
• H proposed by Alice exists P in 1in3 niche so
  that for all S that opponent Bob searches
  Quality(P,S) lt 0.4 SQ.
• Bob knows he can discount H based on this
  knowledge 4/9 for 1in3.
• Lets assume he achieves 0.45 on Alice problem.
• DiscountingFactor 0.45 0.4 0.05 .
• ReputationGain for Bob 0.051.0AliceReputation.

35
Discounting FactorReputationGain for Supporting

• H ((1in3), AR 0.4, confidence 1.0)
• H proposed by Alice exists P in 1in3 niche so
  that for all S that opponent Bob searches
  Quality(P,S) lt 0.4 SQ.
• Bob knows he can discount H based on this
  knowledge 4/9 for 1in3.
• Lets assume he achieves 0.3 on Alice problem.
  Bob has a bug somewhere!
• DiscountingFactor 0.3 0.4 -0.1
• ReputationLoss for Bob -0.11.0AliceReputation
  .

36
Mechanism Design

• The exact SCG(X) mechanism is still a work in
  progress.
• SCG(X) mechanism must be sound
• Encourage productive behavior and discourage
  unproductive behavior of scientists.
• The agent with best heuristics wins.

37
Tools to facilitate use of SCG(X)

• Definition of X.
• Generate a client-server infrastructure for
  playing SCG(X) on the web.
• Administrator enforces SCG(X) rules client.
• Baby agents servers. They can communicate and
  play an uninteresting game.
• Baby agents get improved by their caregivers,
  register with Administrator and the game begins
  at midnight.

38
Properties
39
SCIENTIFIC COMMUNITY
40
SCG a scientific market game

• Domain X (Problem Solving domain such as an NPO
  domain)
• Agents with a reputation offer-accept-deliver-sol
  ve
• Agents offer challenges with a confidence
• Agents accept challenges
• Discounting protocol for challenges
  deliver-solve
• Agent wins reputation
• when it accepts and discounts a challenge of
  another agent (challenge confidence offerer
  reputation at-risk).
• when it supports its own challenge that was
  accepted by an agent (challenge confidence
  acceptor reputation at-risk).

10/16/09
Can DM and ML help?
41
Think of a scientific community about domain X

• Scientists have reputations
• Scientists offer statements with a confidence
• Scientists question statements (accept)
• Scientists use discounting protocol
  (deliver-solve)
• Scientists win and loose reputation

10/16/09
Can DM and ML help?
42
Scientific Market SCG(X)

• Defined by
• Generic SCG game
• Axioms
• A mechanism (game rules) satisfying the axioms
• Description of NPO X
• Feasible solutions, objective function
• Belief language for X
• Predicates for defining niches (subsets of
  problems in X)
• Belief predicates
• Purpose of game Good scientific behavior in
  domain X is rewarded.

10/16/09
Can DM and ML help?
43
Scientists and virtual Scientists!

• Are encouraged to
• offer results that are not easily improved.
• offer results that they can successfully support.
• quote related work and show how they improve on
  previous work.
• prove results if the current state of the art
  allows.
• publish results of an experimental nature with an
  appropriate confidence level.
• stay active and publish new beliefs or oppose
  current beliefs.
• be well-rounded solve posed problems and pose
  difficult problems for others (Like the Four
  Color Conjecture).
• become famous!

44
Productive Scientific Behavior (1)

• The agents propose hypotheses that are difficult
  to strengthen or challenge (i.e. non-trivial yet
  correct). Otherwise, they lose reputation to
  their opponents.
• Offer results that cannot be easily improved.
• Offer results that they can successfully support.

45
Good scientific behavior (2)

• Opposing a belief comes in two flavors.
• The agents should share tight beliefs. Agents
  who share a belief that is not tight lose
  reputation and the agents who tighten a belief
  win reputation unless the tightened belief is
  discounted by some other agent.
• offer results that are not easily improved.
• quote related work and show how they improve on
  previous work.
• The agents should share beliefs that are
  difficult to discount. Agents who share beliefs
  that are discountable lose reputation and the
  challengers who successfully discount win
  reputation.
• offer results that they can successfully support.

46
Productive Scientific Behavior (2)

• Agents are encouraged to propose hypotheses they
  are not sure about. But they need to fairly
  express their confidence in their hypotheses.
• If the confidence is inappropriately high, they
  lose too much reputation if the hypothesis is
  successfully discounted.
• If the confidence is inappropriately low, they
  dont win enough reputation if the hypothesis is
  successfully supported.
• publish results of an experimental nature with an
  appropriate confidence level.

47
Productive Scientific Behavior (3)

• Agents stay active. In each round, they must
  propose new hypotheses and oppose other agents
  hypotheses.
• stay active and publish new hypotheses or oppose
  current hypotheses.
• Agents maximize their reputation.
• become famous!

48
Productive Scientific Behavior (4)

• When Alice loses reputation to Bob, Alice can
  learn from Bob
• Alice has a bug in her software.
• Bob has skills superior to hers. Alice should try
  to acquire Bobs skills.
• Learn from mistakes.
• Be careful how you oppose a Nobel Laureate. The
  risks are high.

49
Unproductive Scientific Behavior

• Cheating is forbidden you can only succeed
  through good scientific behavior (by adding
  useful hypotheses or by successfully opposing
  hypotheses in the knowledge base).

50
Fair Scientific Community

• All agents start with the same initial
  reputation.
• The winner has the best skills in domain X within
  the set of participating agents.

51
Properties

• Agents are penalized for unproductive behaviors.
  A behavior is unproductive if it does not
  possibly lead to the accumulation of new
  knowledge about the specific NPO problem.
• Equilibrium.
• Agent with the best heuristics wins the game. Two
  player games tournament.

52
Applications
53
Improving the research approach

• Problem to be solved Develop the best practical
  algorithms for solving NPO X.
• Standard solution Write hundreds of papers on
  the topic with isolated implementations. What are
  the best practical algorithms?
• Our solution Use the virtual scientific agent
  community SCG(X) with a suitably designed
  hypotheses language to compare the algorithms.
  The winning agent has the best practical
  algorithms.

54

• Game works at the press of a button to determine
  the winner.
• The winner has the best skills in the chosen
  domain. Find the experientially best algorithms
  for solving problems in domain X . Evaluation
  tool.
• The feedback is constructive. Testing and
  Learning Tool. Grading Tool.
• Over time, the market will collect undiscounted
  challenges Belief Maintenance System.
• Agents must be reliable Teaching Software
  Engineering Tool. Grading Tool.

55
What is a scientific virtual market game good
for?

• Market works at the press of a button to
  determine the winner.
• The winner has the best skills in the chosen
  domain. Evaluation tool.
• The feedback is constructive. Testing and
  Learning Tool. Grading Tool.
• Over time, the market will collect undiscounted
  challenges Belief Maintenance System.
• Agents must be reliable Teaching Software
  Engineering Tool. Grading Tool.

10/16/09
Can DM and ML help?
56
Contributing to State of the art knowledge of
domain X
57
Applications of SCG(X)

• Find the experientially best algorithms for
  solving problems in X.

10/16/09
Can DM and ML help?
58
Teaching
59
Agent World for SCG(X)

• Agent Caregiver lives outside SCG(X) world
• World-class experts in domain X.
• Graduate and undergraduate students Studying
  domain X.
• Studying material needed to solve problems in X.
• Learning algorithms based on game histories.
• Agent lives inside SCG(X) world
• Agent win and lose reputation.
• Agent Caregiver prepares agent for next game.

60
Teaching Survival Skills in SCG(X)

• Needed when agent caregiver is human.
• Knowledge about domain X needs to be developed by
  students or taught to them and understood and put
  into algorithms (propose-oppose(strengthen-challen
  ge)-provide-solve) that go into the agent.
• This tests both whether the knowledge about X is
  understood as well as the programming skills.

61
Teaching Survival Skills in SCG(X) cont.

• Scientific Innovation in X Agents get skills
  programmed into them by clever scientists in
  domain X. Scientists use data mining to learn
  from competitions and manually improve the
  agents.
• Machine Learning Innovation in X Agents get
  skills programmed into them by an agent caregiver
  programmed with learning skills and data mining
  skills for domain X. Agent gets updated
  automatically between competitions and they
  improve automatically.

62
Software Development Skills

• Needed when agent caregiver is human.
• Knowledge about domain X needs to be developed by
  students or taught to them and understood and put
  into algorithms (offer-accept-deliver-solve) that
  go into the agent.
• This tests both whether the knowledge about X is
  understood as well as the programming skills.

10/16/09
Can DM and ML help?
63
Athena Lightning Sweet Stepdad Peon
Athena 3 0 3 0
Lightning 0
Sweet 0
Stepdad 3
Peon 1
64
Skills needed to survive in SCG(X)

• Scientific Innovation in X Agents get skills
  programmed into them by clever scientists in
  domain X. Scientists use data mining to learn
  from competitions and manually improve the
  agents.
• Machine Learning Innovation in X Agents get
  skills programmed into them by an agent caregiver
  programmed with learning skills and data mining
  skills for domain X. Agent gets updated
  automatically between competitions and they
  improve automatically.

10/16/09
Can DM and ML help?
65
Possible Application Domain For DM/ML/AI
66
SCG(X) produces history

• Proposers reputation 120
• Hypothesis10 proposer1 opposer2 confidence 1
• Problem delivered
• Solution found discountFactor 1
• Opposer increase in reputation 1 1 120 120

67
Blame assignment

• Where is the proposer to blame?
• Bad hypothesis that is discountable.
• Bug in problem finding algorithm.
• Bug in problem solving algorithm used to check
  proposed hypothesis.

68
Creating Agents
Propose, Oppose, Strengthen, Challenge,
Provide, Solve

• An agent is composed of 6 components Agent
  ltProp, Opp, Str, Cha, Prov, Solgt.
• Components can refer to each other.
• Given a set of agents Agent1 ... Agentn
• Composed agent is a 12-tuple ltPropI, PropO,
  OppI, OppO, StrI, StrO, ChaI, ChaO, ProvI, ProvO,
  SolI, SolOgt.
• ltProp3, (01101),Opp4, (00000), gt

1own 0other
69
Creating Agents cont.

• PropI, OppI, StrI, ChaI, ProvI, SolI ? 1..n.
• PropO consist of 5-bits, each denote one of the
  other components. The first bit describes whether
  to use the opposition component of agent PropI or
  agent OppI.

70
IMPLEMENTATION
71
Tools to facilitate use of SCG(X)

• Definition of X.
• Generate a client-server infrastructure for
  playing SCG(X) on the web.
• Administrator enforces SCG(X) rules client.
• Baby agents servers. They can communicate and
  play an uninteresting game.
• Baby agents get improved by their caregivers,
  register with Administrator and the game begins
  at midnight.

72
Conclusions

• We have shown how a virtual scientific community
  of agents can foster the development and
  innovation of heuristics for approximating NPOs.
• We need your input on how DM and ML could help
  with evolving the agents.

73
Questions?
74
Pending

• When belief is discounted offer complement of
  belief. Belief holder agent that successfully
  discounted.

10/16/09
Can DM and ML help?
75
Discounting

• If Alice offers the belief (FourColorConjecture,
  confidence 1.0), she must be ready to support
  it.
• The opponent Bob gives Alice a planar graph.
• Alice must deliver a 4-coloring.
• If she does not, Bob has successfully discounted
  Alice belief and Alice loses reputation and Bob
  gains.
• If she does, Alice has successfully defended her
  belief and Alice wins reputation and the opponent
  Bob loses.
• Note that discounting is different from finding a
  counterexample. If Alice loses she has a fault
  in her coloring algorithm.

10/16/09
Can DM and ML help?
76
Beliefs Four color conjecture

• FourColorConjecture For all graphs g satisfying
  the predicate planar(g) there exists a 4-coloring
  of the nodes of g such that no two adjacent nodes
  have the same color.
• ForAllExists belief For all problems p
  satisfying predicate pred(p) there exists a
  solution s satisfying a property(p,s).

10/16/09
Can DM and ML help?
77

• Undiscounted beliefs represent the accumulated
  shared knowledge gained from the game. (Requires
  negation and reoffer of discounted beliefs?)

78
Improving the research approach

• Problem to be solved Develop the best practical
  algorithms for solving NPO X.
• Standard solution Write hundreds of papers on
  the topic with isolated implementations. What are
  the best practical algorithms?
• Our solution Use the virtual scientific agent
  community SCG(X) with a suitably designed
  hypotheses language to compare the algorithms.
  The winning agent has the best practical
  algorithms.