ADC

1
What Do You WantSemantic Understanding?

• (Youve Got to be Kidding)

David W. Embley Brigham Young University
Funded in part by the National Science Foundation
2
Presentation Outline

• Grand Challenge
• Meaning, Knowledge, Information, Data
• Fun and Games with Data
• Information Extraction Ontologies
• Applications
• Limitations and Pragmatics
• Summary and Challenges

3
Grand Challenge
Semantic Understanding
4
Grand Challenge
Semantic Understanding
If ever there were a technology that could
generate trillions of dollars in savings
worldwide , it would be the technology that
makes business information systems
interoperable. (Jeffrey T. Pollock, VP of
Technology Strategy, Modulant Solutions)
5
Grand Challenge
Semantic Understanding
The Semantic Web content that is meaningful
to computers and that will unleash a revolution
of new possibilities Properly designed, the
Semantic Web can assist the evolution of human
knowledge (Tim Berners-Lee, , Weaving the
Web)
6
Grand Challenge
Semantic Understanding
20th Century Data Processing 21st Century
Data Exchange The issue now is mutual
understanding. (Stefano Spaccapietra, Editor in
Chief, Journal on Data Semantics)
7
Grand Challenge
Semantic Understanding
The Grand Challenge of semantic understanding
has become mission critical. Current solutions
wont scale. Businesses need economic growth
dependent on the web working and scaling (cost
1 trillion/year). (Michael Brodie, Chief
Scientist, Verizon Communications)
8
Why Semantic Understanding?

• Because were overwhelmed with data
• Point and click too slow
• Give me what I want when I want it.
• Because its the key to revolutionary progress
• Automated interoperability and knowledge sharing
• Negotiation in e-business
• Large-scale, in-silico experiments in e-science

9
What is Semantic Understanding?
Semantics The meaning or the interpretation of
a word, sentence, or other language form.
Understanding To grasp or comprehend
whats intended or expressed.
- Dictionary.com
10
Can We Achieve Semantic Understanding?
A computer doesnt truly understand anything.

But computers can manipulate terms in ways that
are useful and meaningful to the human user.
- Tim Berners-Lee
Key Point it only has to be good enough. And
thats our challenge and our opportunity!
11
Presentation Outline

• Grand Challenge
• Meaning, Knowledge, Information, Data
• Fun and Games with Data
• Information Extraction Ontologies
• Applications
• Limitations and Pragmatics
• Summary and Challenges

12
Information Value Chain
Translating data into meaning
13
Foundational Definitions

• Meaning knowledge that is relevant or activates
• Knowledge information with a degree of certainty
  or community agreement
• Information data in a conceptual framework
• Data attribute-value pairs

- Adapted from Meadow92
14
Foundational Definitions

• Meaning knowledge that is relevant or activates
• Knowledge information with a degree of certainty
  or community agreement (ontology)
• Information data in a conceptual framework
• Data attribute-value pairs

- Adapted from Meadow92
15
Foundational Definitions

• Meaning knowledge that is relevant or activates
• Knowledge information with a degree of certainty
  or community agreement (ontology)
• Information data in a conceptual framework
• Data attribute-value pairs

- Adapted from Meadow92
16
Foundational Definitions

• Meaning knowledge that is relevant or activates
• Knowledge information with a degree of certainty
  or community agreement (ontology)
• Information data in a conceptual framework
• Data attribute-value pairs

- Adapted from Meadow92
17
Data

• Attribute-Value Pairs
• Fundamental for information
• Thus, fundamental for knowledge meaning

18
Data

• Attribute-Value Pairs
• Fundamental for information
• Thus, fundamental for knowledge meaning
• Data Frame
• Extensive knowledge about a data item
• Everyday data currency, dates, time, weights
  measures
• Textual appearance, units, context, operators,
  I/O conversion
• Abstract data type with an extended framework

19
Presentation Outline

• Grand Challenge
• Meaning, Knowledge, Information, Data
• Fun and Games with Data
• Information Extraction Ontologies
• Applications
• Limitations and Pragmatics
• Summary and Challenges

20
?
Olympus C-750 Ultra Zoom Sensor Resolution 4.2
megapixels Optical Zoom 10 x Digital Zoom 4
x Installed Memory 16 MB Lens Aperture F/8-2.8/3
.7 Focal Length min 6.3 mm Focal Length
max 63.0 mm
21
?
Olympus C-750 Ultra Zoom Sensor Resolution 4.2
megapixels Optical Zoom 10 x Digital Zoom 4
x Installed Memory 16 MB Lens Aperture F/8-2.8/3
.7 Focal Length min 6.3 mm Focal Length
max 63.0 mm
22
?
Olympus C-750 Ultra Zoom Sensor Resolution 4.2
megapixels Optical Zoom 10 x Digital Zoom 4
x Installed Memory 16 MB Lens Aperture F/8-2.8/3
.7 Focal Length min 6.3 mm Focal Length
max 63.0 mm
23
?
Olympus C-750 Ultra Zoom Sensor Resolution 4.2
megapixels Optical Zoom 10 x Digital Zoom 4
x Installed Memory 16 MB Lens Aperture F/8-2.8/3.7
Focal Length min 6.3 mm Focal Length max 63.0 mm
24
Digital Camera
Olympus C-750 Ultra Zoom Sensor Resolution 4.2
megapixels Optical Zoom 10 x Digital Zoom 4
x Installed Memory 16 MB Lens Aperture F/8-2.8/3
.7 Focal Length min 6.3 mm Focal Length
max 63.0 mm
25
?
Year 2002 Make Ford Model Thunderbird Mileage
5,500 miles Features Red ABS 6 CD
changer keyless entry Price 33,000 Phone (916
) 972-9117
26
?
Year 2002 Make Ford Model Thunderbird Mileage
5,500 miles Features Red ABS 6 CD
changer keyless entry Price 33,000 Phone (916
) 972-9117
27
?
Year 2002 Make Ford Model Thunderbird Mileage
5,500 miles Features Red ABS 6 CD
changer keyless entry Price 33,000 Phone (916
) 972-9117
28
?
Year 2002 Make Ford Model Thunderbird Mileage
5,500 miles Features Red ABS 6 CD
changer keyless entry Price 33,000 Phone (916
) 972-9117
29
Car Advertisement
Year 2002 Make Ford Model Thunderbird Mileage
5,500 miles Features Red ABS 6 CD
changer keyless entry Price 33,000 Phone (916
) 972-9117
30
?
Flight Class From Time/Date To
Time/Date Stops Delta 16 Coach JFK
605 pm CDG 735 am 0
02 01 04
03 01 04 Delta 119 Coach CDG
1020 am JFK 100 pm 0
09 01 04
09 01 04
31
?
Flight Class From Time/Date To
Time/Date Stops Delta 16 Coach JFK
605 pm CDG 735 am 0
02 01 04
03 01 04 Delta 119 Coach CDG
1020 am JFK 100 pm 0
09 01 04
09 01 04
32
Airline Itinerary
Flight Class From Time/Date To
Time/Date Stops Delta 16 Coach JFK
605 pm CDG 735 am 0
02 01 04
03 01 04 Delta 119 Coach CDG
1020 am JFK 100 pm 0
09 01 04
09 01 04
33
?
Monday, October 13, 2003 Group
A W L T GF GA Pts. USA 3 0 0 11 1
9 Sweden 2 1 0 5 3 6 North Korea 1 2 0 3
4 3 Nigeria 0 3 0 0 11 0 Group
B W L T GF GA Pts. Brazil 2 0 1 8 2 7
34
?
Monday, October 13, 2003 Group
A W L T GF GA Pts. USA 3 0 0 11 1
9 Sweden 2 1 0 5 3 6 North Korea 1 2 0 3
4 3 Nigeria 0 3 0 0 11 0 Group
B W L T GF GA Pts. Brazil 2 0 1 8 2 7
35
World Cup Soccer
Monday, October 13, 2003 Group
A W L T GF GA Pts. USA 3 0 0 11 1
9 Sweden 2 1 0 5 3 6 North Korea 1 2 0 3
4 3 Nigeria 0 3 0 0 11 0 Group
B W L T GF GA Pts. Brazil 2 0 1 8 2 7
36
?
Calories 250 cal Distance 2.50 miles Time 23.35
minutes Incline 1.5 degrees Speed 5.2 mph Heart
Rate 125 bpm
37
?
Calories 250 cal Distance 2.50 miles Time 23.35
minutes Incline 1.5 degrees Speed 5.2 mph Heart
Rate 125 bpm
38
?
Calories 250 cal Distance 2.50 miles Time 23.35
minutes Incline 1.5 degrees Speed 5.2 mph Heart
Rate 125 bpm
39
Treadmill Workout
Calories 250 cal Distance 2.50 miles Time 23.35
minutes Incline 1.5 degrees Speed 5.2 mph Heart
Rate 125 bpm
40
?
Place Bonnie Lake County Duchesne State Utah Typ
e Lake Elevation 10,000 feet USGS Quad Mirror
Lake Latitude 40.711ºN Longitude 110.876ºW
41
?
Place Bonnie Lake County Duchesne State Utah Typ
e Lake Elevation 10,000 feet USGS Quad Mirror
Lake Latitude 40.711ºN Longitude 110.876ºW
42
?
Place Bonnie Lake County Duchesne State Utah Typ
e Lake Elevation 10,000 feet USGS Quad Mirror
Lake Latitude 40.711ºN Longitude 110.876ºW
43
Maps
Place Bonnie Lake County Duchesne State Utah Typ
e Lake Elevation 10,100 feet USGS Quad Mirror
Lake Latitude 40.711ºN Longitude 110.876ºW
44
Presentation Outline

• Grand Challenge
• Meaning, Knowledge, Information, Data
• Fun and Games with Data
• Information Extraction Ontologies
• Applications
• Limitations and Pragmatics
• Summary and Challenges

45
Information Extraction Ontologies
Source
Target
Information Extraction
Information Exchange
46
What is an Extraction Ontology?

• Augmented Conceptual-Model Instance
• Object relationship sets
• Constraints
• Data frame value recognizers
• Robust Wrapper (Ontology-Based Wrapper)
• Extracts information
• Works even when site changes or when new sites
  come on-line

47
Extraction Ontology Example
Car -gt object Car 01 has Year 1 Car
01 has Make 1 Car 0 has Feature
1 PhoneNr 1 is for Car 01 Year
matches 4 constant extract \d2
context \b4-9\d\b Mileage matches
8 keyword \bmiles\b, \bmi\b.,

48
Extraction OntologiesAn Example ofSemantic
Understanding

• Intelligent Symbol Manipulation
• Gives the Illusion of Understanding
• Obtains Meaningful and Useful Results

49
Presentation Outline

• Grand Challenge
• Meaning, Knowledge, Information, Data
• Fun and Games with Data
• Information Extraction Ontologies
• Applications
• Limitations and Pragmatics
• Summary and Challenges

50
A Variety of Applications

• Information Extraction
• High-Precision Classification
• Schema Mapping
• Semantic Web Creation
• Agent Communication
• Ontology Generation

51
Application 1Information Extraction
52
Constant/Keyword Recognition
'97 CHEVY Cavalier, Red, 5 spd, only 7,000 miles.
Previous owner heart broken! Asking only
11,995. 1415. JERRY SEINER MIDVALE, 566-3800
or 566-3888
Descriptor/String/Position(start/end)
Year9723 MakeCHEV58 MakeCHEVY59 ModelCav
alier1118 FeatureRed2123 Feature5
spd2630 Mileage7,0003842 KEYWORD(Mileage)mil
es4448 Price11,995100105 Mileage11,9951001
05 PhoneNr566-3800136143 PhoneNr566-38881481
55
53
Heuristics

• Keyword proximity
• Subsumed and overlapping constants
• Functional relationships
• Nonfunctional relationships
• First occurrence without constraint violation

54
Keyword Proximity
Year9723 MakeCHEV58 MakeCHEVY59 ModelCav
alier1118 FeatureRed2123 Feature5
spd2630 Mileage7,0003842 KEYWORD(Mileage)mil
es4448 Price11,995100105 Mileage11,9951001
05 PhoneNr566-3800136143 PhoneNr566-38881481
55
D 2
D 52
'97 CHEVY Cavalier, Red, 5 spd, only 7,000 miles
on her. Previous owner heart broken! Asking
only 11,995. 1415. JERRY SEINER MIDVALE,
566-3800 or 566-3888
55
Subsumed/Overlapping Constants
Year9723 MakeCHEV58 MakeCHEVY59 ModelCav
alier1118 FeatureRed2123 Feature5
spd2630 Mileage7,0003842 KEYWORD(Mileage)mil
es4448 Price11,995100105 Mileage11,9951001
05 PhoneNr566-3800136143 PhoneNr566-38881481
55
'97 CHEVY Cavalier, Red, 5 spd, only 7,000 miles.
Previous owner heart broken! Asking only
11,995. 1415. JERRY SEINER MIDVALE, 566-3800
or 566-3888
56
Functional Relationships
Year9723 MakeCHEV58 MakeCHEVY59 ModelCav
alier1118 FeatureRed2123 Feature5
spd2630 Mileage7,0003842 KEYWORD(Mileage)mil
es4448 Price11,995100105 Mileage11,9951001
05 PhoneNr566-3800136143 PhoneNr566-38881481
55
'97 CHEVY Cavalier, Red, 5 spd, only 7,000 miles
on her. Previous owner heart broken! Asking
only 11,995. 1415. JERRY SEINER MIDVALE,
566-3800 or 566-3888
57
Nonfunctional Relationships
Year9723 MakeCHEV58 MakeCHEVY59 ModelCav
alier1118 FeatureRed2123 Feature5
spd2630 Mileage7,0003842 KEYWORD(Mileage)mil
es4448 Price11,995100105 Mileage11,9951001
05 PhoneNr566-3800136143 PhoneNr566-38881481
55
'97 CHEVY Cavalier, Red, 5 spd, only 7,000 miles
on her. Previous owner heart broken! Asking
only 11,995. 1415. JERRY SEINER MIDVALE,
566-3800 or 566-3888
58
First Occurrence without Constraint Violation
Year9723 MakeCHEV58 MakeCHEVY59 ModelCav
alier1118 FeatureRed2123 Feature5
spd2630 Mileage7,0003842 KEYWORD(Mileage)mil
es4448 Price11,995100105 Mileage11,9951001
05 PhoneNr566-3800136143 PhoneNr566-38881481
55
'97 CHEVY Cavalier, Red, 5 spd, only 7,000 miles
on her. Previous owner heart broken! Asking
only 11,995. 1415. JERRY SEINER MIDVALE,
566-3800 or 566-3888
59
Database-Instance Generator
Year9723 MakeCHEV58 MakeCHEVY59 ModelCav
alier1118 FeatureRed2123 Feature5
spd2630 Mileage7,0003842 KEYWORD(Mileage)mil
es4448 Price11,995100105 Mileage11,9951001
05 PhoneNr566-3800136143 PhoneNr566-38881481
55
insert into Car values(1001, 97, CHEVY,
Cavalier, 7,000, 11,995,
556-3800) insert into CarFeature values(1001,
Red) insert into CarFeature values(1001, 5
spd)
60
Application 2High-Precision Classification
61
An Extraction Ontology Solution
62
Density Heuristic
63
Expected Values Heuristic
64
Vector Space of Expected Values
D1

• OV ______ D1 D2
• Year 0.98 16 6
• Make 0.93 10 0
• Model 0.91 12 0
• Mileage 0.45 6 2
• Price 0.80 11 8
• Feature 2.10 29 0
• PhoneNr 1.15 15 11
• D1 0.996
• D2 0.567

ov
D2
65
Grouping Heuristic
66
Grouping
Car Ads ---------------- Year Year Make Model ----
---------- 3 Price Year Model Year ---------------
3 Make Model Mileage Year ---------------4 Model M
ileage Price Year ---------------4 Grouping
0.875
Sale Items ---------------- Year Year Year Mileage
-------------- 2 Mileage Year Price Price -------
--------3 Year Price Price Year ---------------2 P
rice Price Price Price ---------------1 Grouping
0.500
Expected Number in Group floor(? Ave
) 4 (for our example)
1-Max
Sum of Distinct 1-Max Object Sets in each
Group Number of Groups Expected Number in a
Group
67
Application 3Schema Mapping
68
Problem Different Schemas

• Target Database Schema
• Car, Year, Make, Model, Mileage, Price,
  PhoneNr, PhoneNr, Extension, Car, Feature
• Different Source Table Schemas
• Run , Yr, Make, Model, Tran, Color, Dr
• Make, Model, Year, Colour, Price, Auto, Air
  Cond., AM/FM, CD
• Vehicle, Distance, Price, Mileage
• Year, Make, Model, Trim, Invoice/Retail, Engine,
  Fuel Economy

69
Solution Remove Internal Factoring
Discover Nesting Make, (Model, (Year, Colour,
Price, Auto, Air Cond, AM/FM, CD))
70
Solution Replace Boolean Values
ACURA
ACURA
Legend
71
Solution Form Attribute-Value Pairs
ACURA
ACURA
Legend
ltMake, Hondagt, ltModel, Civic EXgt, ltYear, 1995gt,
ltColour, Whitegt, ltPrice, 6300gt, ltAuto,
Autogt, ltAir Cond., Air Cond.gt, ltAM/FM, AM/FMgt,
ltCD, gt
72
Solution Adjust Attribute-Value Pairs
ACURA
ACURA
Legend
ltMake, Hondagt, ltModel, Civic EXgt, ltYear, 1995gt,
ltColour, Whitegt, ltPrice, 6300gt, ltAutogt,
ltAir Condgt, ltAM/FMgt
73
Solution Do Extraction
ACURA
ACURA
Legend
74
Solution Infer Mappings
ACURA
ACURA
Legend
Car, Year, Make, Model, Mileage, Price,
PhoneNr, PhoneNr, Extension, Car, Feature
75
Solution Do Extraction
ACURA
ACURA
Legend
Car, Year, Make, Model, Mileage, Price,
PhoneNr, PhoneNr, Extension, Car, Feature
76
Solution Do Extraction
ACURA
ACURA
Legend
pPriceTable
Car, Year, Make, Model, Mileage, Price,
PhoneNr, PhoneNr, Extension, Car, Feature
77
Solution Do Extraction
ACURA
ACURA
Legend
? Colour?Feature p ColourTable U ? Auto?Feature p
Auto ß AutoTable U ? Air Cond.?Feature p Air
Cond. ß Air Cond.Table U ? AM/FM?Feature p AM/FM
ß AM/FMTable U ? CD?Featurep CDß CDTable
Yes,
Yes,
Yes,
Yes,
Car, Year, Make, Model, Mileage, Price,
PhoneNr, PhoneNr, Extension, Car, Feature
78
Application 4Semantic Web Creation
79
The Semantic Web

• Make web content accessible to machines
• What prevents this from working?
• Lack of content
• Lack of tools to create useful content
• Difficulty of converting the web to the Semantic
  Web

80
Converting Web to Semantic Web
81
Superimposed Information
82
Application 5Agent Communication
83
The Problem
Agents must 1- share ontologies, 2- speak the
same language, 3- pre-agree on message format.

• Requiring these assumptions precludes
• agents from interoperating on the fly

The holy grail of semantic integration in
architectures is to allow two agents to
generate needed mappings between them on the fly
without a priori agreement and without them
having built-in knowledge of any common
ontology. Uschold 02
84
Solution
Agents must 1- share ontologies, 2- speak the
same language, 3- pre-agree on message format.

• Eliminate all assumptions

• This requires

- Translating (developing mutual understanding)

• Dynamically capturing a messages semantics
• Matching a message with a service

85
MatchMaking System (MMS)
86
Application 6Ontology Generation
87
TANGO Table Analysis for Generating Ontologies

• Recognize and normalize table information
• Construct mini-ontologies from tables
• Discover inter-ontology mappings
• Merge mini-ontologies into a growing ontology

88
Recognize Table Information

Religion
Population Albanian
Roman Shia
Sunni Country (July 2001 est.) Orthodox
Muslim Catholic Muslim Muslim
other Afganistan 26,813,057
15
84 1 Albania
3,510,484 20 70 30
89
Construct Mini-Ontology
90
Discover Mappings
91
Merge
92
Presentation Outline

• Grand Challenge
• Meaning, Knowledge, Information, Data
• Fun and Games with Data
• Information Extraction Ontologies
• Applications
• Limitations and Pragmatics
• Summary and Challenges

93
Limitations and Pragmatics

• Data-Rich, Narrow Domain
• Ambiguities Context Assumptions
• Incompleteness Implicit Information
• Common Sense Requirements
• Knowledge Prerequisites

94
Busiest Airport in 2003?
Chicago - 928,735 Landings (Nat. Air Traffic
Controllers Assoc.) - 931,000 Landings
(Federal Aviation Admin.) Atlanta -
58,875,694 Passengers (Sep., latest numbers
available) Memphis - 2,494,190 Metric Tons
(Airports Council Intl.)
95
Busiest Airport in 2003?
Chicago - 928,735 Landings (Nat. Air Traffic
Controllers Assoc.) - 931,000 Landings
(Federal Aviation Admin.) Atlanta -
58,875,694 Passengers (Sep., latest numbers
available) Memphis - 2,494,190 Metric Tons
(Airports Council Intl.)
96
Busiest Airport in 2003?
Chicago - 928,735 Landings (Nat. Air Traffic
Controllers Assoc.) - 931,000 Landings
(Federal Aviation Admin.) Atlanta -
58,875,694 Passengers (Sep., latest numbers
available) Memphis - 2,494,190 Metric Tons
(Airports Council Intl.)
97
Busiest Airport in 2003?
Chicago - 928,735 Landings (Nat. Air Traffic
Controllers Assoc.) - 931,000 Landings
(Federal Aviation Admin.) Atlanta -
58,875,694 Passengers (Sep., latest numbers
available) Memphis - 2,494,190 Metric Tons
(Airports Council Intl.)
Ambiguous Whom do we
trust?
(How do they count?)
98
Busiest Airport in 2003?
Chicago - 928,735 Landings (Nat. Air Traffic
Controllers Assoc.) - 931,000 Landings
(Federal Aviation Admin.) Atlanta -
58,875,694 Passengers (Sep., latest numbers
available) Memphis - 2,494,190 Metric Tons
(Airports Council Intl.)
Important qualification
99
Dow Jones Industrial Average
High Low
Last Chg 30 Indus 10527.03
10321.35 10409.85 85.18 20 Transp
3038.15 2998.60 3008.16 9.83 15
Utils 268.78 264.72 266.45
1.72 66 Stocks 3022.31 2972.94
2993.12 19.65
Graphics, Icons,
100
Dow Jones Industrial Average
High Low
Last Chg 30 Indus 10527.03
10321.35 10409.85 85.18 20 Transp
3038.15 2998.60 3008.16 9.83 15
Utils 268.78 264.72 266.45
1.72 66 Stocks 3022.31 2972.94
2993.12 19.65
101
Mad Cow hurts Utah jobs
Utah stands to lose 1,200 jobs from Asian
countries import bans on beef products, ...
Common sense a cow cant hurt jobs.
102
Mad Cow hurts Utah jobs
Utah stands to lose 1,200 jobs from Asian
countries import bans on beef products, ...
Knowledge required for understanding
Mad Cow disease discovered in Washington.
Humans can get the disease by eating contaminated
beef.
People in Asian countries dont want to get sick.
Washington state (not DC), which is in the
western US.
Utah is in the western US.
Beef cattle are regionally linked
(somehow?)
103
Presentation Outline

• Grand Challenge
• Meaning, Knowledge, Information, Data
• Fun and Games with Data
• Information Extraction Ontologies
• Applications
• Limitations and Pragmatics
• Summary and Challenges

104
Some Key Ideas

• Data, Information, and Knowledge
• Data Frames
• Knowledge about everyday data items
• Recognizers for data in context
• Ontologies
• Resilient Extraction Ontologies
• Shared Conceptualizations
• Limitations and Pragmatics

105
Some Research Issues

• Building a library of open source data
  recognizers
• Creating a corpora of test data for extraction,
  integration, table understanding,
• Precisely finding and gathering relevant
  information
• Subparts of larger data
• Scattered data (linked, factored, implied)
• Data behind forms in the hidden web
• Improving concept matching
• Indirect matching
• Calculations and unit conversions

106
Some Research Challenges

• Automating ontology construction
• Converting web data to Semantic Web data
• Accommodating different views
• Developing effective personal software agents

www.deg.byu.edu