Semantically Enhanced Model Experiment Evaluation Process SeMEEP within the Atmospheric Chemistry Co

1
Semantically Enhanced Model Experiment Evaluation
Process (SeMEEP)within the Atmospheric Chemistry
Community

• Chris Martin 1,2, Mo Haji 2, Peter Dew 2, Peter
  Jimack 2, Mike Pilling 1
• 1 School of Chemistry, University of Leeds
• 2 School of Computing, University of Leeds

2
Outline of the Presentation

• Introduction
• Atmospheric community
• SeMEEP
• ELN Provenance capture
• Conclusion and next stage

3
Section 1 Overview

• Application domain atmospheric community
• Reliance on computational models to evaluate
  data
• Motivation
• Study how to transition from today's ad-hoc
  processes practises
• Sustainable process of
• Gathering, community evaluation and sharing data
  models between scientists
• Minimising changes to proven working practises
  of the scientist
• Within world-wide co-laboratories

4
Related projects

• CombeChem
• Experimental organic chemistry
• From source to long term data
• perseveration (knowledge)
• Semantically-enabled ELN
• Data-driven workflow
• Collaboratory for Multi-Scale Chemical Science
• Multi-layer chemical model
• myGrid
• Bio-informatics and related areas (semantic
  pattern matching
• Reusable semantic workflow using SMD (semantic
  metadata)
• Data Quality
• Karama2
• Weather forecasting computation modelling

5
Section 2 Atmospheric Chemistry

• Seeks to understand the chemical processes
  (reactions) taking place in the lower atmosphere
  (e.g. smoke)
• It has significant implication for both
• Air Quality
• Climate Change

6
The Master Chemical Mechanism (MCM)

• Data repository of elementary chemical reactions
  rate constants
• The mechanism is described by a computational
  model that is evaluated against experimental data
• Chamber experiments
• Field experiments

7
Section 3 SeMEEP

• Today
• Typically within the atmospheric chemistry
  community the provenance is recorded in an
  ad-hoc, unstructured fashion, using a combination
  of traditional lab-book, word processing
  documents and spreadsheet.
• Move to more sustainable evaluation process
  supports the gathering, evaluation and sharing of
  data and models
• Using semantic metadata

8
SeMEEP Vision

• SeMEEP semantically-enabled MEEP
• Supports the organisation of information but
  critically, records its provenance (say to
  recover secondary data)

Mike Pilling SeMEEP approach will radically
enhance the effectiveness of a research community
to deliver new science
9
Requirements with metadata for elementary
chemical reactions
10
Requirements for metadata capture for elementary
reactions

• Only published data
• Rate constants from several labs
• No access to the raw data
• No access to secondary data
• SeMEEP will provide this.

11
Current Evaluation Processes for the MCM
12
Envisioned Evaluation Processes
13
Section 4 Electronic Lab-Books (ELNs)

• ELNs address the limitations of the current
  methods of provenance capture.
• Southampton ELN for organic chemistry
  experiments.
• Benefits to the modeller
• Modelling process can be automatically captured
• Searchable
• Remote access is possible
• Provenance is structured
• Possible to use resolvable references to
  resources

14
Will User attach quality metadata?

• Motivate users
• By demonstrating the value of provenance in their
  day-to-day work
• Writing publication
• Managing their data
• Reinterpretting the data.
• Management
• Publishers

15
Recoding provenance for modelling

• 3-level process
• Experimental modelling plan
• Modelling iterations
• Modelling layer where provenance is capture has
  the modelling process proceeds using data-driven
  workflows where data is a first class object

16
The Modelling Process - A Three Layer Mapping
17
MCM Mechanism being investigated
18
ELN Process
19
ELN Screenshots

• Prompts displayed when changing the changing the
  chemical mechanism
• Editing a reaction
• Adding a new reaction

20
ELN Screenshots
21
ELN Modelling SMD Architecture
22
Evaluation Methodology

• In-depth interviews with members of the
  atmospheric chemistry model group here at Leeds,
  covering
• Demonstration of the prototype
• User testing of the prototype
• Discussion of scenarios involving the use of the
  prototype (e.g. )
• Analysis
• Interviews recorded and transcribed
• Analysed using techniques from grounded theory

23
Evaluation

• Barriers to adoption
• Effort required at modelling time for provenance
  capture
• in your lab book you can write down what ever
  you want but with an ELN it is going to take
  time to go through the different protocol steps.
  
• When asked if they would use an ELN requiring a
  similar amount of user input to the prototype the
  response was positive
• Yeah, I think it would be a good thing. I dont
  think it is too much extra work.
• Rather than viewing the prompts for user
  annotation as interruption to their normal work
  the user recognised the value of being prompted
• is a good way to do it because otherwise you
  wont record the provenance.

24
Evaluation

• Users intuitively grasped the benefits of
  recording provenance with an ELN and that the
  benefits would be realised after the time of
  modelling by a number of stakeholders
• if someone else wants to look at your
  provenance, thats great because the person can
  see exactly what you have done, where you have
  been and where to go next. And for yourself, if
  you are writing up a PhD ... you can see
  exactly what youve done whereas currently you
  have to rifle through lab-books to see exactly
  what you have done.

25
Section 5 Conclusions and future work

• Outlined SeMEEP and ELN
• User evaluated proposed modelling ELN
• Addressed case studies
• IUPAC
• MCM
• Developing a case study with the Geomagnetic
  community
• User and System issues
• Application of actively theory to capture
  requirements and user evaluation
• Querying and inference
• Address QoS issues (e.g. security, scalabilty,
  dynamic roles-based access control)

26
Questions