The KnowledgeMap Project: Development of a Concept-Based Medical School Curriculum Database

1
The KnowledgeMap Project Development of a
Concept-Based Medical School Curriculum Database

• Joshua C. Denny, MD
• Plomarz R. Irani
• Firas H. Wehbe, MD
• Jeffrey D. Smithers, MD
• Anderson Spickard, III, MD, MS

2
Setting

• Vanderbilt School of Medicine
• 104 Medical students in each class
• 4 local hospitals
• No electronic repository or course schedule

3
Goals for KM

• Provide a set of tools to help improve the
  curriculum and students access to it
• Accommodate a variety of presentation styles
• Automate document conversion
• Provide a secure repository of documents that
  protects intellectual property

4
KM Structure

• Web application
• Apache web server, MySQL database
• Written in Perl, VisualC, and Visual Basic
• Multiple servers
• All documents mapped to UMLS concepts via KM
  Concept Identifier

5
Document Corpus

• Manually converted 2001-2002 preclinical lecture
  handouts
• legacy documents
• New handouts/presentations uploaded by faculty

6
Pilot

• Anatomy (Fall) and Cell Biology (Spring) for
  2002-2003
• 4th year elective later came online

7
KM Concept Identifier

• Uses NLP techniques
• Abbreviation and acronym extraction
• Semantic regularization
• Score-based
• Derivational forms (stenosis ? stenotic, lungs ?
  pulmonary)
• Document-based disambiguation
• Word and concept clustering
• Performs favorably with MetaMap on educational
  documents (82 Recall, 89 Precision)

8
Document Processing
Document Conversion Server pulls next document
off queue, converts to HTML and Text
Document uploaded by lecturer, placed in queue
HTML PDF versions
Text version placed in queue
Identified concepts indexed for searching
9
Search Processing
C0019202 -- Hepatolenticular Degeneration
User enters a search query, example Wilsons
disease
C0019202 found in index of curriculum documents
10
Content Coverage Query

• Created to answer questions such as Where is
  Womens Health taught?
• metaconcepts
• Uses relationships defined in the UMLS to expand
  queries with related child and child-like
  concepts

11
Content Coverage Query
KM finds related UMLS concepts
User enters a metaconcept query Womens
Health
C0080339 -- Womens Health
These concepts found in index of curriculum
documents
12
Other features

• Relevant PubMed searches
• Based on the document title and the most frequent
  MeSH concepts in the document
• Definition searching
• Based on UMLS SRDEF file and MedlinePLUS
• Course management
• Lecture calendar
• Organized by semester and student year

13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
Analysis

• Primary data source was Log Files
• All events in Apache
• Key events in KM, including
• Logins/Logoffs
• Searches
• Documents viewed (by browse or by search)
• PubMed searches
• Content Coverage queries (available only to
  Course Directors and Administrators)
• Removed all events generated by a developer or
  researcher
• Downtime measured by a separate server that
  logged any time the system (or a component) was
  unavailable

19
Calculations

• Browsed document any documents accessed via a
  course home page or via the browse function on
  the toolbar
• Searched document any document accessed via a
  search

20
Results

• 3271 searches
• 84 completed with a concept search
• 15 definition searches (since 5/03)
• Total of 526 users logged in 15,885 times and
  viewed 1,143 documents a total of 32,113 times
• All members of the first, second, and third year
  classes have logged on

21
Results

• 1264 active documents (1489 total)
• 722 uploaded by 28 faculty members
• 135 documents uploaded by authors
• 407 legacy documents
• Total downtime was about 20 hours, including
  scheduled downtime
• No true downtime since 1/03

22
Start of new academic year
23




24
(No Transcript)
25
Conclusions

• KM is being adopted by more classes
• Predominant class and student use still first
  year courses
• Students are using KM more
• Student use precedes classes coming online
• All students with courses online have used KM
• Initial reactions seem positive
• Heaviest use by 1st and 2nd year students

26
Future Directions

• Automation of content coverage queries
• Expand to more courses and another site
• More types of media
• Expansions of search algorithms to include spell
  checker
• Support for a PocketPC/Palm-compatible site
• Student tracking

27
Acknowledgements

• Randy Miller, M.D.
• Michel Décary of Cogilex R D, Inc
• Deans Office
• Art Dalley, Ph.D.
• Cathleen Pettepher, Ph.D.