Information Extraction

1
Information Extraction
Ronen Feldman Bar Ilan University Israel

• Junichi Tsujii
• Graduate School of Science
• University of Tokyo
• Japan

2
Application Tasks of NLP
(1)Information Retrieval/Detection
To search and retrieve documents in response to
queries for information
(2)Passage Retrieval
To search and retrieve part of documents in
response to queries for information
(3)Information Extraction
To extract information that fits pre-defined
database schemas or templates, specifying the
output formats
(4) Question/Answering Tasks
To answer general questions by using texts as
knowledge base Fact retrieval, combination of IR
and IE
(5)Text Understanding
To understand texts as people do Artificial
Intelligence
3
Ranges of Queries
(1)Information Retrieval/Detection
(2)Passage Retrieval

Pre-Defined Fixed aspects of information carried
in texts
(3)Information Extraction
(4) Question/Answering Tasks
(5)Text Understanding
4
IE definitions

• Entity an object of interest such as a person or
  organization
• Attribute A property of an entity such as name,
  alias, descriptor or type
• Fact A relationship held between two or more
  entities such as Position of Person in Company
• Event An activity involving several entities
  such as terrorist act, airline crash, product
  information

5
IE accuracy typical figures by information type

• Entity 90-98
• Attribute 80
• Fact 60-70
• Event 50-60

6
MUC conferences

• MUC 1 to MUC 7
• 1987 to 1997
• Topics
• Naval operations (2)
• Terrorist Activity (2)
• Joint venture and microelectronics
• Management changes
• Space Vehicles and Missile launches

7
The ACE Evaluation

• The ACE program challenge of extracting content
  from human language. Research effort directed to
  master
• first the extraction of entities
• Then the extraction of relations among these
  entities
• Finally the extraction of events that are
  causally related sets of relations
• After two years, top systems successfully capture
  well over 50 of the value at the entity level

8
Applications of IE

• Routing of information
• Infrastructure for IR and categorization (higher
  level features)
• Event based summarization
• Automatic creation of databases and knowledge
  bases

9
Where would IE be useful?

• Semi-structured text
• Generic documents like news articles
• Most of the information in the doc is centred
  around a set of easily identifiable entities

10
Example of IE FASTUS(1993)
11
Example of IE FASTUS(1993)
12
Example of IE FASTUS(1993)
13
Example of IE FASTUS(1993)
14
Example of IE FASTUS(1993)
15
FASTUS
Based on finite states automata (FSA)
1.Complex Words Recognition of multi-words and
proper names
set up new Twaiwan dallors
2.Basic Phrases Simple noun groups, verb groups
and particles
a Japanese trading house had set up
3.Complex phrases Complex noun groups and verb
groups
4.Domain Events Patterns for events of interest
to the application Basic templates are to be
built.
5. Merging Structures Templates from different
parts of the texts are merged if they provide
information about the same entity or event.
16
Example of IE FASTUS(1993)
17
Information Extraction
. Jurgen Pfrang, 51, reportedly stumbled upon
the robbers on the second floor of his Nanjing
home early on Sunday. The deputy general manager
of Yaxing Benz, a Sino-German joint venture that
makes buses and bus chassis in nearby
Yangzhou, was hacked to death with 45 cm
watermelon knives. .
Name of the Venture Yaxing Benz Products
buses and bus chassis Location
Yangzhou,China Companies involved
(1)Name X?
Country German
(2)Name Y?
Country China

18
Information Extraction
A German vehicle-firm executive was stabbed to
death . . Jurgen Pfrang, 51, reportedly
stumbled upon the robbers on the second floor of
his Nanjing home early on Sunday. The deputy
general manager of Yaxing Benz, a Sino-German
joint venture that makes buses and bus chassis
in nearby Yangzhou, was hacked to death with 45
cm watermelon knives. .
Crime-Type Murder Type
Stabbing The killed Name Jurgen Pfrang
Age 51
Profession Deputy general
manager Location Nanjing, China

Different template for crimes
19
Interpretation of Texts
(1)Information Retrieval/Detection
(2)Passage Retrieval

(3)Information Extraction
(4) Question/Answering Tasks
(5)Text Understanding
20
IR System
Collection of Texts
21
IR System
Collection of Texts
22
Passage IR System
Collection of Texts
23
Passage IR System
IE System
Collection of Texts
Texts
24
IE System
Templates
Texts
25
IE as compromise NLP
Interpretation
IE System
Templates
Texts
Predefined
26
Performance Evaluation
(1)Information Retrieval/Detection
(2)Passage Retrieval

(3)Information Extraction
(4) Question/Answering Tasks
(5)Text Understanding
27
Collection of Documents
28
Collection of Documents
More complicated due to partially filled
templates
29
Framework of IE
IE as compromise NLP
30
Difficulties of NLP
General Framework of NLP
(1) Robustness Incomplete Knowledge
Morphological and Lexical Processing
Syntactic Analysis
Semantic Analysis
Incomplete Domain Knowledge Interpretation
Rules
Context processing Interpretation
31
Difficulties of NLP
General Framework of NLP
(1) Robustness Incomplete Knowledge
Morphological and Lexical Processing
Syntactic Analysis
Semantic Analysis
Incomplete Domain Knowledge Interpretation
Rules
Context processing Interpretation
32
Approaches for building IE systems

• Knowledge Engineering Approach
• Rules crafted by linguists in cooperation with
  domain experts
• Most of the work done by insoecting a set of
  relevant documents

33
Approaches for building IE systems

• Automatically trainable systems
• Techniques based on statistics and almost no
  linguistic knowledge
• Language independent
• Main input annotated corpus
• Small effort for creating rules, but crating
  annotated corpus laborious

34
Techniques in IE
(1) Domain Specific Partial Knowledge
Knowledge relevant to information to be extracted
(2) Ambiguities Ignoring irrelevant
ambiguities Simpler NLP techniques
(3) Robustness Coping with Incomplete
dictionaries (open
class words) Ignoring irrelevant parts of
sentences
(4) Adaptation Techniques Machine
Learning, Trainable systems
35
General Framework of NLP
Open class words Named entity recognition
(ex) Locations Persons
Companies Organizations
Position names
Morphological and Lexical Processing
Syntactic Analysis
Semantic Anaysis
Domain specific rules ltWordgtltWordgt, Inc.
Mr. ltCpt-Lgt. ltWordgt Machine Learning
HMM, Decision Trees Rules Machine Learning
Context processing Interpretation
36
FASTUS
General Framework of NLP
Based on finite states automata (FSA)
1.Complex Words Recognition of multi-words and
proper names
Morphological and Lexical Processing
2.Basic Phrases Simple noun groups, verb groups
and particles
Syntactic Analysis
3.Complex phrases Complex noun groups and verb
groups
4.Domain Events Patterns for events of interest
to the application Basic templates are to be
built.
Semantic Anaysis
Context processing Interpretation
5. Merging Structures Templates from different
parts of the texts are merged if they provide
information about the same entity or event.
37
FASTUS
General Framework of NLP
Based on finite states automata (FSA)
1.Complex Words Recognition of multi-words and
proper names
Morphological and Lexical Processing
2.Basic Phrases Simple noun groups, verb groups
and particles
Syntactic Analysis
3.Complex phrases Complex noun groups and verb
groups
4.Domain Events Patterns for events of interest
to the application Basic templates are to be
built.
Semantic Anaysis
Context processing Interpretation
5. Merging Structures Templates from different
parts of the texts are merged if they provide
information about the same entity or event.
38
FASTUS
General Framework of NLP
Based on finite states automata (FSA)
1.Complex Words Recognition of multi-words and
proper names
Morphological and Lexical Processing
2.Basic Phrases Simple noun groups, verb groups
and particles
Syntactic Analysis
3.Complex phrases Complex noun groups and verb
groups
4.Domain Events Patterns for events of interest
to the application Basic templates are to be
built.
Semantic Analysis
Context processing Interpretation
5. Merging Structures Templates from different
parts of the texts are merged if they provide
information about the same entity or event.
39
Chomsky Hierarchy Hierarchy of
Grammar of Automata Regular
Grammar Finite State
Automata Context Free Grammar
Push Down Automata Context Sensitive Grammar
Linear Bounded Automata Type 0
Grammar Turing
Machine
40
Chomsky Hierarchy Hierarchy of
Grammar of Automata Regular
Grammar Finite State
Automata Context Free Grammar Push
Down Automata Context Sensitive Grammar
Linear Bounded Automata Type 0 Grammar
Turing Machine
41
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
42
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
43
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
44
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
45
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
46
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
47
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
48
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
49
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
50
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
51
Pattern-maching PN s (ADJ) N P Art (ADJ) N
PN s/ Art(ADJ) N(P Art (ADJ) N)
1
s
PN
Art
2
0
ADJ
N
Art
s
3
Johns interesting book with a nice cover
P
4
PN
52
FASTUS
General Framework of NLP
Based on finite states automata (FSA)
1.Complex Words Recognition of multi-words and
proper names
Morphological and Lexical Processing
2.Basic Phrases Simple noun groups, verb groups
and particles
Syntactic Analysis
3.Complex phrases Complex noun groups and verb
groups
4.Domain Events Patterns for events of interest
to the application Basic templates are to be
built.
Semantic Analysis
Context processing Interpretation
5. Merging Structures Templates from different
parts of the texts are merged if they provide
information about the same entity or event.
53
Example of IE FASTUS(1993)
1.Complex words
2.Basic Phrases Bridgestone Sports Co.
Company name said
Verb Group Friday
Noun Group it
Noun Group had set up
Verb Group a joint venture
Noun Group in
Preposition Taiwan
Location
54
Example of IE FASTUS(1993)


1.Complex words
2.Basic Phrases Bridgestone Sports Co.
Company name said
Verb Group Friday
Noun Group it
Noun Group had set up
Verb Group a joint venture
Noun Group in
Preposition Taiwan
Location
a Japanese tea house a Japanese tea house a
Japanese tea house
55
Example of IE FASTUS(1993)
1.Complex words
2.Basic Phrases Bridgestone Sports Co.
Company name said
Verb Group Friday
Noun Group it
Noun Group had set up
Verb Group a joint venture
Noun Group in
Preposition Taiwan
Location
56
Example of IE FASTUS(1993)
3.Complex Phrases
2.Basic Phrases Bridgestone Sports Co.
Company name said
Verb Group Friday
Noun Group it
Noun Group had set up
Verb Group a joint venture
Noun Group in
Preposition Taiwan
Location
57
Example of IE FASTUS(1993)
3.Complex Phrases
2.Basic Phrases Bridgestone Sports Co.
Company name said
Verb Group Friday
Noun Group it
Noun Group had set up
Verb Group a joint venture
Noun Group in
Preposition Taiwan
Location
Some syntactic structures like
58
Example of IE FASTUS(1993)
3.Complex Phrases
2.Basic Phrases Bridgestone Sports Co.
Company name said
Verb Group Friday
Noun Group it
Noun Group had set up
Verb Group a joint venture
Noun Group in
Preposition Taiwan
Location
Syntactic structures relevant to information to
be extracted are dealt with.
59
Syntactic variations
GM set up a joint venture with Toyota. GM
announced it was setting up a joint venture with
Toyota. GM signed an agreement setting up a joint
venture with Toyota. GM announced it was signing
an agreement to set up a joint venture with
Toyota.
60
Syntactic variations
GM set up a joint venture with Toyota. GM
announced it was setting up a joint venture with
Toyota. GM signed an agreement setting up a joint
venture with Toyota. GM announced it was signing
an agreement to set up a joint venture with
Toyota.
GM plans to set up a joint venture with
Toyota. GM expects to set up a joint venture with
Toyota.
61
Syntactic variations
GM set up a joint venture with Toyota. GM
announced it was setting up a joint venture with
Toyota. GM signed an agreement setting up a joint
venture with Toyota. GM announced it was signing
an agreement to set up a joint venture with
Toyota.
S
NP
VP
GM
V
set up
GM plans to set up a joint venture with
Toyota. GM expects to set up a joint venture with
Toyota.
62
Example of IE FASTUS(1993)
3.Complex Phrases 4.Domain Events COMPANYSET-U
PJOINT-VENTUREwithCOMPNY COMPANYSET-UPJO
INT-VENTURE (others) withCOMPNY
63
Complications caused by syntactic variations
Relative clause The mayor, who was kidnapped
yesterday, was found dead today.
NG Relpro NG/others VG NG/othersVG N
G Relpro NG/others VG
64
Complications caused by syntactic variations
Relative clause The mayor, who was kidnapped
yesterday, was found dead today.
NG Relpro NG/others VG NG/othersVG N
G Relpro NG/others VG
65
Complications caused by syntactic variations
Relative clause The mayor, who was kidnapped
yesterday, was found dead today.
NG Relpro NG/others VG NG/othersVG N
G Relpro NG/others VG
66
FASTUS
Based on finite states automata (FSA)
NP, who was kidnapped, was found.
1.Complex Words
2.Basic Phrases
3.Complex phrases
4.Domain Events Patterns for events of interest
to the application Basic templates are to be
built.
5. Merging Structures Templates from different
parts of the texts are merged if they provide
information about the same entity or event.
67
FASTUS
Based on finite states automata (FSA)
NP, who was kidnapped, was found.
1.Complex Words
2.Basic Phrases
3.Complex phrases
4.Domain Events Patterns for events of interest
to the application Basic templates are to be
built.
Piece-wise recognition of basic templates
5. Merging Structures Templates from different
parts of the texts are merged if they provide
information about the same entity or event.
Reconstructing information carried via syntactic
structures by merging basic templates
68
FASTUS
Based on finite states automata (FSA)
NP, who was kidnapped, was found.
1.Complex Words
2.Basic Phrases
3.Complex phrases
4.Domain Events Patterns for events of interest
to the application Basic templates are to be
built.
Piece-wise recognition of basic templates
5. Merging Structures Templates from different
parts of the texts are merged if they provide
information about the same entity or event.
Reconstructing information carried via syntactic
structures by merging basic templates
69
FASTUS
Based on finite states automata (FSA)
NP, who was kidnapped, was found.
1.Complex Words
2.Basic Phrases
3.Complex phrases
4.Domain Events Patterns for events of interest
to the application Basic templates are to be
built.
Piece-wise recognition of basic templates
5. Merging Structures Templates from different
parts of the texts are merged if they provide
information about the same entity or event.
Reconstructing information carried via syntactic
structures by merging basic templates
70
Current state of the arts of IE

• Carefully constructed IE systems
• F-60 level (interannotater agreement
  60-80)
• Domain telegraphic messages about naval
  operation
• (MUC-187, MUC-289)
• news articles and
  transcriptions of radio broadcasts
• Latin American terrorism
  (MUC-391, MUC-41992)
• News articles about joint
  ventures (MUC-5, 93)
• News articles about
  management changes (MUC-6, 95)
• News articles about space
  vehicle (MUC-7, 97)
• Handcrafted rules (named entity recognition,
  domain events, etc)

Automatic learning from texts Supervised
learning corpus preparation
Non-supervised, or controlled learning