Theory and Practice in AI and Law: A Response to Branting

1
Theory and Practice in AI and Law A Response to
Branting

• Katie Atkinson and
• Trevor Bench-Capon
• Department of Computer Science
• The University of Liverpool

2
Overview

• Over the years there has emerged something of an
  academic consensus in the AI and Law field
• Practitioners, however, often find it difficult
  to relate this theory to practice
• Trenchant criticisms from Karl Branting
• This talk attempts to explain why these
  discrepancies exist
• Some remarks on categories of legal AI systems

3
Theoretical Consensus

• Models of reasoning with legal cases
• CBR approaches of Rissland, Ashley, Aleven,
  Brunighaus
• Rule/Logic based approaches of Prakken, Sartor,
  Hage, Roth, Bench-Capon
• Theory (McCarty), Purpose (Berman and Hafner) and
  Dialogue (Gordon)
• A range of work from which some agreement has
  emerged

4
Features of the Consensus

• Representation of cases using factors
• Not facts, but patterns of facts with legal
  significance
• Reasoning based on argument plaintiff and
  defendant exchange arguments based on factors
  present in the case and precedents
• These arguments can be presented as dialectical
  exchanges
• Decisions are based on an evaluation of the
  competing arguments
• Strengths of arguments can be based on the
  purposes served by accepting them.

5
Questions for the Consensus

• Branting has argued that experience with real
  decisions deviates from this consensus
• He has six points of contention

6
Contentions

• The outcomes of disputes are highly predictable
• It is typically hard to find plausible arguments
  for both sides of the case
• Uncertainty arises from characterisation of
  facts. Not interpretation of precedents
• Decisions are justified by precedents, not by
  rules and construction

7
Contentions

• Judicial decisions are not frozen dialectic
• Decisions almost never make reference to purposes
• Factor based argumentation is rare
• So why does the practice theory not reflect the
  consensus theory?

8
Levels of Reasoning

• In previous work we identified three levels at
  which arguments can be presented in a case
• Reasoning about the world to determine how the
  law should be
• Reasoning about legal concepts and their
  applicability to achieve this
• Reasoning about the consequences of decisions
  made at level two

9
Levels of Reasoning
10
Use of these Levels

• Often only Level Three is used
• Applicability is not disputed no need to reason
  at level 2
• Where applicability is in dispute, Level 2 is
  needed.
• But Level 1 is needed only if new law is
  required
• Gaps in understanding
• Conflicts to resolve
• Changes in response to social change
• Many decisions confine themselves to a report of
  the Level 3 argumentation.
• Precedents embody past reasoning from lower
  levels

11
Predictability

• At Level Three all is predictable the
  consequences of applicable legal consequences are
  understood
• At Level 2, applicability of a concept may be
  disputed but precedents are usually clear enough
• Hard cases are relatively rare landmark cases
  producing new law are rarer

12
Characterisation of Facts

• Characterisation of facts takes place at the
  level 2, perhaps based on conclusions from Level
  1.
• Level 1 is where there is little help from
  precedent, and is the least circumscribed, so new
  considerations can be introduced
• Level 1 is thus likely to be the source of
  uncertainty and unpredictability

13
Justification is by Precedent

• A precedent often embodies a rule, and the
  application of that rule.
• It acts as a summary of argumentation from a
  lower level
• If cited at, e.g. Level 3, therefore it can stand
  for arguments that were considered and evaluated
  previously, and so avoid the need to rehearse
  them again.
• Reasoning in a case need not be from first
  principles

14
Decisions are not Frozen Dialectic

• It is important not to confuse the process by
  which a decision is reached with the way in which
  the decision is reported.
• How we choose to summarise the conclusions from a
  dispute need not reflect the process by which we
  reached those conclusions
• If the decision confines itself to Level 3, the
  findings can be stated without rehearsing the
  arguments

15
Reference to Purpose is Rare

• Level 1, which is where the teleological factors
  are considered is required only when
• new law is needed,
• The law needs to be changed
• Very few such landmark cases arise in areas of
  settled law, and so this level is not invoked
• Conclusions as to purpose are embodied in
  precedents to be used at level 2
• Much more common in Supreme Court decisions
  Branting focuses on lower level courts

16
Factor Based Argumentation is Rare

• Again factor based argumentation only becomes
  explicit when Level 1 is reached.
• Typically the precedent will encapsulate a rule
  originally produced in this way, but which can be
  cited to avoid restating such arguments
• Again used in reasoning from first principles

17
Theory Construction

• Bench-Capon and Sartor proposed seeing reasoning
  with legal cases as theory construction
• How does this fit with our levels?

18
Level 1
Level 3
Level 2
Level 1
input
Domain Analysis
19
British Nationality Act/ Expert Systems
input
Domain Analysis
20
HYPO/ CATO
input
Domain Analysis
21
IBP
input
Domain Analysis
22
AGATHA Berman/Hafner 93
input
Domain Analysis
23
Unaddressed?
input
Domain Analysis
24
Summary

• The consensus models reasoning from first
  principles
• In practice decisions do not recapitulate the
  whole story
• Precedents can be used to avoid the need to
  reason from first principles
• Decisions state conclusions, and some level of
  justification
• The consensus assumes a domain analysis, which
  avoids difficulties in characterising facts
• This may be the hardest problem of them all