Title: Automatic Extraction From and Reasoning About Genealogical Records: A Prototype
1Automatic Extraction From and Reasoning About
Genealogical Records A Prototype
- By
- Charla J. Woodbury
- A thesis submitted to the faculty of
- Brigham Young University
- in partial fulfillment of the requirements for
the degree of - Master of Science
2Digital Images Human Index
- Large number of competing family history
websites - Digital images
- Human indexes
- Researchers hunting through records and
indexes - to put families together
2
3Problem
- Large amounts of primary genealogical data
- Big projects to index and extract records
- Two independent indexers and adjudication
- Millions of human hours used to index or match
records for names and families
3
4Automated Extraction Solution
- Create a specialized extraction ontology to
interpret and label genealogical data - Add rules and logic that
- Label family roles - husband, daughter, etc.
- Link family relationships
- HUSBAND WIFE
- PARENT CHILD
4
5Outline
- Data Preparation
- Ontology Extraction System (OntoES)
- OWL File and SWRL Rules
- SPARQL Queries
- Experimental Results
- Conclusions
5
61. Data Preparation
- Collect machine-readable records from three
different countries - Format in HTML format for extraction
- Prepare lexicons for names, places, etc.
6
7New England Vital Records Beverly,
Massachusetts 1668-1849
7
8Danish Parish Maglebye, Praesto1646-1813
8
9English Parish South Petherton, Somersetshire
1574-1901
9
10SOUTH PETHERTON MARRIAGES (from genuki)
same day 1576 Nicholas Patch and Christian Denman 26 Jan 1605 Richard Patch and Joan Lavor 25-Sep 1613 John Elliott and Joan Woodbery 7-Aug 1615 Thomas Prime and Maria Parry 29-Jan 1616 William Woodbery and Elizabeth Patch 2-May 1620 William Hillerd and Fortu Patch 17-Sep 1622 Nicholas Patch and Elizabeth Owsley 22-Jan 1627 Richard Patch and Mary White 15-Jan 1630 Andrew Elliott and Joan Patch 12-Feb 1639 Andrew Elliott and Joan Pitts
10
11Transcript of the original record
- 1576/1577 eodem die Nicholaus Patch Christinam
Denman - 26 Jan 1605 Richard Patch et Joanna Lavor
- 1613 Septembris 26 Johannes Elliott et Joanna
Woodbery matrimonis
cominguntur - 1615 Augusti 7 Thoms Prime et Maria Patch
matrimonio cominguntur - 1616/1617 Januarij 29 Wilhelmus Woodbery et
Elizabetha Patch matrimonio cominguntur - 1620 Maij 2 Wilhelmus Hillerd et Fortu Patch
- 1622 Septembris 17 Nicholas Patch et Elizabetha
Owsley matrimonio cominguntur - 1627/1628 Januarij 22 Richardus Patch et Maria
White matrimonio cominguntur - 1630/1631 Januarij 15 Andreas Elliott et Joanna
Patch matrimonio cominguntur - 1639/1640 Februarij 12 Andreas Elliott et Joanna
Pittes matrimonio cominguntur
122. Ontology Extraction System
- OntoES automatically interpret and correctly
label genealogical data using - Data frames
- Regular expressions
- Lexicons
- Date conversion methods
12
13Marriage Ontology
13
14Data Frame Editor
14
15Regular expressions
- MARDATE
- Value expression
- EXAMPLE type 25-Sep 1613
- (0\d1\d2\d3031\d)-Month\.?\s(\d\d\d\d)
- Keyword expression
- (\b(md\.?marrymarriagemarriedmariedwedweddin
g)\b)
16Sample MONTH LEXICON
- 1Ober
- 7ber
- 8ber
- 9ber
- apr
- april
- aprilis
- aug
- august
- augusti
- augustus
- avr
- avril
- avrilis
- dec
- december
- decembr
- decembre
- decembri
- feb
- febr
- februari
- february
- jan
- januarij
- january
- jul
- juli
- julius
- july
- jun
- june
16
17Object Level
17
18CANONICALIZATION METHODSinside the ontology
- Regularize date (Julian format YYYYddd)
- 1620 2-May ? 1620093
- Display stored Julian format as DD MMM YYYY
- 1620093 ? 2 MAY 1620
18
19Feast Dates
- Dates expressed as a holy day
- Fixed Dates
- Christmas 1720 ? 25 DEC 1720
- Moveable Dates around Easter
- (36 possible Easter dates with leap year
variation) - 1723 Dnica Septuagesima ? 24 JAN 1723
- Same day as previous entry
19
20 Run Ontology
- Input
- Ontology (Created with OntoES)
- HTML data (Hypertext Markup Language)
- Output
- RDF database (Resource Description Format)
- OWL file (Ontology Web Language)
20
21Ontology Workbench
21
22Extracted Marriages
Bet Date MarDate NameM NameF NameU
same day 1576 Nicholas Patch Christian Denman
26 JAN 1605 Richard Patch Joan Lavor
26 SEP 1613 John Elliott Joan Woodbery
7 AUG 1615 Thomas Prime Maria Parry
29 JAN 1616 William Woodbery Elizabeth Patch
2 MAY 1620 William Hillerd Fortu Patch
17 SEP 1622 Nicholas Patch Elizabeth Owlsey
22 JAN 1627 Richard Patch Mary White
16 JAN 1630 Andrew Elliott Joan Patch
12 FEB 1639 Andrew Elliott Joan Pitts
22
233. OWL File and SWRL Rules
- OWL HEADER
- ltowlClass rdfID"MarriageRecord"/gt
- ltowlClass rdfID"Person"/gt
- ltowlClass rdfID"NameU"/gt
- ltowlDatatypeProperty rdfID"NameUValue"gt
- ltrdfsdomain rdfresource"NameU"/gt
- ltrdfsrange rdfresource"xsdstring"/gt
- lt/owlDatatypePropertygt
- PERSON - NAMEU
- ltowlObjectProperty rdfID"Person-NameU"gt
- ltrdfsdomain rdfresource"Person"/gt
- ltrdfsrange rdfresource"NameU"/gt
- ltowlinverseOfgt
- ltowlObjectProperty
rdfID"NameU-Person"/gt - lt/owlinverseOfgt
- lt/owlObjectPropertygt
24Sample RDF Triples
- Person_10 sameAs Person_10
- Person_10 type Thing
- Person_10 type Person
- NameU_0 NameUValue Christian Denman
- NameU_0 sameAs NameU_0
- NameU_0 type Thing
- NameU_0 type NameU
- NameM_4 NameMValue Nicholas Patch
- NameM_4 sameAs NameM_4
- NameM_4 type Thing
- NameM_4 type NameM
25SWRL Rules
- Define OWL Class
- Example Husband
- ltowlClass rdfID"Husband"/gt
- Define Rule
- Example Person with male name is a Husband
- Person-NameM(?x,?y) -gt Husband(?x)
?y
?x
25
26Marriage Person_10 to Person_4
- Person_10
- ltPerson rdfID"Person_10"gt
- ltPerson-NameU rdfresource"NameU_0"
/gt - lt/Persongt
- MarriageRecord_7
- ltMarriageRecord rdfID"MarriageRecord_7"gt
- ltMarriageRecord-Person
rdfresource"Person_4" /gt - ltMarriageRecord-Person
rdfresource"Person_10" /gt - lt/rdfMarriageRecordgt
- NameM_4
- ltNameM rdfID"NameM_4"gt
- ltNameMValuegt Nicholas Patchlt/NameMValuegt
- lt/NameMgt
- Person_4
- ltPerson rdfID"Person_4"gt
- ltPerson-NameM rdfresource"NameM_4" /gt
- lt/Persongt
27Rule HEAD in OWL file
- ltswrlImp rdfID"Def-Husband"gt
- ltswrlhead rdfparseTypeCollectiongt
- ltswrlClassAtomgt
- ltswrlargument1 rdfresource"x"/gt
- ltswrlclassPredicate
rdfresource"Husband"/gt - lt/swrlClassAtomgt
- lt/swrlheadgt
28Rule BODY in OWL file
- ltswrlbodygt
- ltswrlIndividualPropertyAtomgt
- ltswrlpropertyPredicate rdfresource"Pers
on-NameM"/gt - ltswrlargument1 rdfresource"x"/gt
- ltswrlargument2 rdfresource"y"/gt
- lt/swrlIndividualPropertyAtomgt
- lt/swrlbodygt
- lt/swrlImpgt
29Related Rules
- NameF is populated then value in NameU is Husband
- Person-NameF(?w,?v) ? MarriageRecord-Person(?z,?w)
? - MarriageRecord-Person(?z,?x) ? Person-NameU(?x,?y)
- -gt Husband(?x)
?z
?v
?w
?x
?y
29
30HusbandOf Rule
- Husband(?x) ? Wife(?y) ? MarriageRecord-Person(?z,
?x) - ? MarriageRecord-Person(?z,?y)
- -gt HusbandOf(?x,?y)
30
31Auxiliary Name Rules
- NameM(?x) -gt Name(?x)
- NameF(?x) -gt Name(?x)
- NameU(?x) -gt Name(?x)
- NameMValue(?x) -gt NameValue(?x)
- NameFValue(?x) -gt NameValue(?x)
- NameUValue(?x) -gt NameValue(?x)
- Person-NameM(?x,?y) -gt Person-Name(?x,?y)
- Person-NameF(?x,?y) -gt Person-Name(?x,?y)
- Person-NameU(?x,?y) -gt Person-Name(?x,?y)
31
32SPARQL Query Who is Husband of Christian Denman?
- PREFIX http//www.deg.byu.edu/ontology/Marriage
- SELECT ?Husband
- WHERE
-
- ?X NameValue "Christian Denman" .
- ?Y Person-Name ?X .
- ?W HusbandOf ?Y .
- ?W Person-Name ?V .
- ?V NameValue ?Husband
32
33Query Results
- Husband
-
- "Nicholas Patch"http//www.w3.org/2001/XMLSchema
string
33
34Query Results
- Husband
-
- "Nicholas Patch"http//www.w3.org/2001/XMLSchema
string
South Petherton Marriages same day 1576 Nicholas
Patch and Christian Denman 26 Jan 1605 Richard
Patch and Joan Lavor 25-Sep 1613 John Elliott
and Joan Woodbery 7-Aug 1615 Thomas Prime and
Maria Parry 29-Jan 1616 William Woodbery and
Elizabeth Patch 2-May 1620 William Hillerd and
Fortu Patch 17-Sep 1622 Nicholas Patch and
Elizabeth Owsley 22-Jan 1627 Richard Patch and
Mary White 15-Jan 1630 Andrew Elliott and Joan
Patch 12-Feb 1639 Andrew Elliott and Joan Pitts
Nicholas Patch because
NameValue(Nicholas Patch) and
Name-NameValue(n1, Nicholas Patch)
and Name(n1) is NameM(n1) and Person-NameM(p1,
n1) NameValue(Christian Denman) and
Name-NameValue(n2, Christian Denman)
and Name(n2) is NameU(n2) and Person-NameU(p2,
n2) Husband(p1) because Person-NameM(p1,
n1) Wife(p2) because Person-NameU(p2, n2)
and Person-MarriageRecord(p2, r1) and
MarriageRecord-Person(r1, p1) and
Person-NameM(p1, n1) HusbandOf(p1, p2) because
Husband(p1) and Wife(p1) and
MarriageRecord-Person(r1, p1) and
MarriageRecord-Person(r1, p1)
34
355. Experimental Results
- Extraction Results
- American Extraction Problem
- Rule Results
35
36Extraction Results
MARRIAGES ENTITIES RECALL ERRORS PRECISION
English 188 594 588 99.0 8 98.7
American 608 1824 1630 89.4 34 98.0
Danish 171 543 538 99.1 10 98.2
BIRTHS
English 3153 9489 9394 99.0 61 99.4
American 675 2055 1809 88.0 33 98.2
Danish 677 2061 2042 99.1 15 99.3
DEATHS
English 3458 8675 8589 99.0 83 99.0
American 510 1305 1148 88.0 28 97.6
Danish 833 2113 2093 99.1 19 99.1
36
37American Difficulty
- BIRTH
- WOODBURY, Charles Henry Charles William, P. R.
4., s. Henry housewright. dup. and Henrietta
(Galloup), Dec. 4, 1845. - Extra information inside brackets parentheses
- Charles William twin of Charles Henry
- Henry housewright identified as NAME
- Henrietta (Galloup) identified as NAME
37
38Rules Results
- 100 Precision and Recall
- (Once rules are well-defined, the results are
perfect.) - Database Size
- (The RDF database is much larger when rule
triples are added.) - NEW PROPERTIES husband, wife, parent, child
- NEW LINKS
38
39Size Impact of Adding Rules
MARRIAGE 21 rules EVENT 30 rules
Triples OWL ( lines) OWL File (kilobytes) Triples OWL ( lines) OWL File (kilobytes)
OWL File 814 498 14 2232 1405 15
W/Rules 1009 785 31 2983 1873 75
Difference 195 287 17 751 468 60
Increase 23.96 57.63 121.43 33.65 33.31 400.00
406. Conclusions
- Speed up data indexing
- Make production of a full index easier
- Ground the index in original documents
- Provide for inferred facts
- Simplify as well as augment record search
- Help link records and form family groups and
ancestral lines
40