Quality Labelling of Medical Web Content: the MedIEQ proposal

1
Quality Labelling of Medical Web Content the
MedIEQ proposal
Vangelis Karkaletsis, Miquel Angel Mayer
National Centre for Scientific Research
Demokritos, Athens, Greece, vangelis_at_iit.demokri
tos.gr Web Médica Acreditada, Medical
Association of Barcelona, Spain,
mmayer.wma_at_comb.es
eHealth 2006, Global Trends and Perspectives
Session Malaga, 10-12 May 2006
2
Contents

• Problem Description
• Existing labeling processes
• Previous projects
• MedIEQ proposal for a labeling platform

3
Problem Description - I

• The number of health web sites and online
  services is increasing day by day
• 70-80 of Internet users seeks health information
  for them or for their relatives
• More than 4 out of 10 health information seekers
  say the material they find affect their health
  decisions

4
Problem Description - II

• Quality of health related web content is
  extremely variable
• from evidence-based healthcare to widespread
  practice of fraud and potentially-dangerous
  claims
• Increase in consumer knowledge changes how
  patient, professionals and providers interact
• Patients are becoming more proactive in their
  care management
• What are the effects in terms of Public Health?
• An example Vaccines

5
(No Transcript)
6
(No Transcript)
7
Problem Description - III

• Organisations around the world are working on
  establishing quality standards for health sites
• European Commission
• eEurope 2002 Quality criteria for health related
  web sites
• American Medical Association
• Guidelines for medical health information sites
  on the Internet
• Internet Healthcare Coalition
• eHealth Code of Ethics
• .
• Are they enough?
• Self-adherence to codes of conduct or ethics,
  nothing more than a claim with little
  enforceability

8
Problem Description - IV

• Necessary the establishment of labeling
  mechanisms
• by third party accreditation
• by creating portals where medical web sites are
  organized and characterized against certain
  labeling criteria
• How such mechanisms can be effective?

9
Existing labeling processes - I
Codes of Conduct are defined as sets of quality
criteria that provide a list of recommendations
for the development and content of websites
Quality Label (logo) is diplayed on screen and
represents a commitment by a provider to
implement or adhere to a code of conduct
User Guidance enables users to check if a site
complies with certain standards by accessing a
series of questions from a displayed logo
Filtering Tools applied manually or
automatically, accept or reject web resources -
resources are selected for their quality and
relevance to a particular audience
Third Parties certification quality and
accreditation labels are awarded by a third party
to inform consumers that a site provides
information meeting current standards for content
and form
10
Existing labeling processes IIAnother example
Web Medica Acreditada (WMA) criteria

• Identification
• Content
• Confidentiality
• Advertising and Sponsoring
• Virtual Consultation
• Non compliance

11
Existing Labeling Processes - III

• (A.) A health web site issues a request, for a
  label, to a third party (labeling operator)
• Site checked and if OK, a label is generated by
  the operator
• the label is either stored locally at the web
  sites server, or stored in the operators
  database (a link to the label is added in the web
  site)
• sites content is examined periodically and if an
  unacceptable change occurs, the label is either
  removed or replaced with a relevant message

12
Existing Labeling Processes - IV

• (B.) Location of unlabeled health web sites in
  specific thematic areas
• Characterization of the located sites, with
  respect to certain criteria
• Filtering of some of the web sites based on their
  characterization
• Organizing the rest of the web sites into web
  directories to facilitate access by health
  information consumers

13
Existing Labeling Processes Vlimitations and
needs

• High costs to offer the service
• Huge amount of information to assess (too many
  sites)
• Information changes quickly
• Broken links to accredited websites
• Not standardised rating criteria
• Dishonest use of the label

14
Existing Labeling Processes VIlimitations and
needs

• Most of the work in labeling processes is
  currently performed manually
• A site may have hundreds of pages (static/dynamic
  ones)
• Probably all or most of them have to be checked
• Label for the whole site or different labels for
  sites pages
• Need for technologies that enable the automation
  of the labeling process this involves
• Technology for creating machine processable
  labels
• establishing common vocabularies, and
• exploiting semantic web technologies to enable
  the labels parsing by web browsers or search
  engines
• Technology for automated web content analysis

15
Previous projects MedCERTAIN

• MedCERTAIN MedPICS Certification and Rating of
  Trustworthy and Assessed Health Information on
  the Net
• EC-funded Safer Internet Action Plan, ended
  2002
• http//www.medcertain.org/
• MedCERTAIN developed the standardized metadata
  vocabulary HIDDEL (Health Information Disclosure,
  Description and Evaluation Language) to evaluate
  and annotate health related web sites
• MedCERTAIN consortium involved
• the University of Heidelberg, Dept. of Clinical
  Social Medicine
• the University of Bristol, Institute for Learning
  and Research Technology (ILRT)
• Finnish National Research and Development Centre
  for Welfare and Health (STAKES) / The Finnish
  Office for Health Care Technology Assessment
  (FinOHTA)

16
Previous projects MedCIRCLE

• MedCIRCLE Collaboration for Internet Rating,
  Certification, Labeling and Evaluation of Health
  Information
• EC-funded Safer Internet Action Plan, ended
  2003
• http//www.medcircle.org/
• MedCIRCLE further developed and refined
  MedCERTAINs HIDDEL vocabulary, and developed
  technologies and networks to guide consumers to
  trustworthy health information on the Internet
• MedCIRCLE consortium involved
• University of Heidelberg, Dept. of Clinical
  Social Medicine
• Catalog and Index of French-language health
  resources (CISMeF),
• Medical Association of Barcelona COMB
  Department of Web Medica Acreditada (WMA),
• Agency for Quality in Medicine (AQuMed)

17
Previous projects WRAPIN

• WRAPIN Worldwide online Reliable Advice to
  Patients and Individuals
• EC-funded, FP5-IST, 09/2001-10/2003
• http//www.wrapin.org/
• WRAPIN aimed at helping in the formulation of
  more efficient medical queries and facilitating
  access to multiple knowledge sources.
• WRAPIN consortium involved
• Division dinformatique Medical, Geneva,
  Switzerland.
• Health On the Net, Geneva, Switzerland.
• Laboratoire d'Enseignement et de Recherche sur
  le Traitement de l'Information Médical,
  Marseille, France.
• "Mission pour linformatisation du système de
  santé, Paris, France.
• NICE Computing, Lausanne, Switzerland.
• Thalès Information System, Geneva, Switzerland (
  Paris, France).
• XRParner, Paris, France.

18
Previous projects QUATRO

• QUATRO Quality Assurance and Content
  Description
• EC-funded Safer Internet programme, ends
  November 2006
• http//www.quatro-project.org/
• QUATRO developed a common vocabulary and machine
  processable schema for quality labeling, tools to
  support the exploitation of QUATRO labels by
  search engines or web browsers
• QUATRO consortium involves
• Pira International, UK
• Internet Content Rating Association, UK
• Web Medica Acreditada, Spain
• Agencia de Calidad de Internet (IQUA), Spain
• National Centre for Scientific Research
  Demokritos, Greece
• Università degli Studi di Milano, Italy
• Coolwave Ltd, UK
• CEIE ERCIM, France
• ECP-NL, the Netherlands

19
The MedIEQ Proposal - I

• MedCERTAIN and MedCIRCLE developed a common
  vocabulary (HIDDEL)
• WRAPIN developed technology for supporting the
  end-user in the retrieval of medical content
  from trusted sources
• QUATRO develops technology for creating machine
  processable labels
• QUATRO addresses the needs of both the labeling
  operator and the end-user (sites visitor)
• QUATRO work represents the first step towards a
  platform that will support the work of the
  labeling operator
• Technology for automated web content analysis is
  required to support the work of labeling operators

20
The MedIEQ Proposal - II

• Web content analysis must involve
• in labeled web sites
• monitoring the labels validity with respect to
  the sites content and informing the labeling
  operator
• exploiting technology for spidering web sites, to
  locate interesting pages, and extracting
  information from them
• in unlabeled web sites
• locating these sites first, characterizing and
  storing them into a web directory
• exploiting technology for web crawling, site
  spidering, and information extraction

21
The MedIEQ Proposal - III
22
The MedIEQ Proposal IV

• MedIEQ Quality Labeling of Medical Web content
  using Multilingual Information Extraction
• EC-funded project
• DG SANCO Health Consumer protection,
  Directorate C Public Health and Risk Assessment
• Public Health Programme, Priority Area 1.
  Health Information, Action 1.5 eHealth
• Duration 01/01/2006-01/01/2009

23
The MedIEQ Proposal VThe project consortium

• 2 medical quality labeling agencies
• Medical Association of Barcelona COMB Web Medica
  Acreditada (WMA),
• Agency for Quality in Medicine (AQuMed)
• 4 research groups
• NCSR Demokritos - Software Knowledge
  Engineering Laboratory (Coord.),
• Universidad Nacional de Education a Distancia-
  Natural Language processing and Information
  Retrieval group (UNED),
• University of Economics in Prague - Department of
  Information and Knowledge Engineering (UEP),
• Helsinki University of Technology - Neural
  Networks Research Centre (HUT)
• 1 spin-off company with expertise on content
  analysis I-sieve Technologies Ltd.
• 1 collaborating partner Geneva University
  Hospitals Service of Medical Informatics, HUG

24
The MedIEQ Proposal - VI Project Objectives

• develop a scheme for the quality labelling of
  medical web content and provide the tools
  supporting the creation, maintenance and access
  of labelling data according to this scheme
• specify a methodology for the content analysis of
  medical web sites according to the MedIEQ scheme
  and develop the tools that will implement it
• integrate these technologies into a prototype
  labelling system
• demonstrate the resulting prototype in 7
  different languages (Spanish, Catalan, German,
  English, Greek, Czech, and Finnish) and two
  labelling applications (third party
  accreditation, classification)

25
The MedIEQ Proposal - VII Indicators for
measuring the achievement of objectives

• Reduction of the manual labelling time
• Labeling unlabeled sites, monitoring labeled
  sites, ..
• Effective extraction from large collections of
  medical web content
• Processing time, precision of extracted data,
• Effort required to customize the system into new
  languages
• 7 languages to be supported
• Implementation of an open architecture
• Effort required to integrate new techniques and
  tools,

26
Concluding

• MedIEQ technology is expected to have a
  significant impact on medical quality labeling
• assisting the work of labelling experts,
• increasing the number of labelled medical sites
  across Europe and their effective monitoring, and
  thus
• improving the quality health knowledge
  disseminated through the Web.

27
Quality Labelling of Medical Web Content the
MedIEQ proposal
Vangelis Karkaletsis, Miquel Angel Mayer
Thank you !
eHealth 2006, Global Trends and Perspectives
Session Malaga, 10-12 May 2006