Learning Entities and Relations

1
Learning Entities and Relations

• Dan Roth Wen-tau Yih
• Department of Computer Science
• University of Illinois at Urbana-Champaign

2
Entity/Relation Recognition

• John was murdered at JFK after his assassin,
  Kevin
• Identify

Kill (X, Y)
person
person
John was murdered at JFK after his assassin,
Kevin
location

• Entity/Relation Recognition is a key task in many
  NLP problems.
• Information Extraction
• Question Answering
• Story Comprehension

• Identify named entities
• Identify relations between entities
• Exploit mutual dependencies between named
  entities and relations to yield a coherent global
  prediction

3
Problem Setting

• Constraints
• (R12 kill) ? (E1 person) ? (E2 person)
• (R12 headquarter) ? (E1 organization) ? (E2
  location)

• The relation between each pair of entities is
  represented by a relation variable most of them
  are null.

• The goal is to assign labels to these E and R
  variables.

4
Pipeline Model

• Train an entity classifier first
• Use the entity predictions along with other
  information to build a relation classifier
• Propagation of errors
• Bi-Directional interactions between stages

5
Global Inference with Classifiers

• Classifiers (for components) are trained or given
  in advance.
• There are constraints on classifiers labels
  (which may be known during training or only known
  during testing).
• The inference procedure attempts to make the best
  global assignment, given the local predictions
  and the constraints

6
Ideal Inference
7
Inference Using Integer Linear Programming

• Integer linear programming (ILP) formulation
• General works on non-sequential constraint
  structure
• Flexible can represent many types of constraints
• Optimal finds the optimal solution
• Fast commercial packages are able to solve it
  quickly
• This framework can be used in many problems.
• e.g., Semantic Role Labeling

8
Experimental Results F1
1,437 sentences 5,336 entities and 19,048
potential relations
Entity Predictions
Relation Predictions

• Improvement compared to the basic (w/o inference)
  and pipeline (entity?relation) models

9
Decision-time Constraint

• Constraints may be known only in decision time.
• Question Answering Who killed JFK?
• Find kill relation in candidate sentences

Find the arguments of the kill relation
10
Summary

• We applied ILP to solve the inference problem in
  Entity/Relation Recognition.
• Linear constraints are very general and can
• Represent any Boolean function
• Handle non-sequential constraint structure
• The current effort is to use this framework for a
  large scale study of named entity and relation
  recognition.