Embedding e-Science Applications: Designing and Managing for Usability

1
Embedding e-Science Applications Designing and
Managing for Usability

• Marina Jirotka and Sharon Lloyd
• Oxford University
• NeSC The Changing Landscape
• 16th April 2009

2
Embedding e-Science Applications Designing and
Managing for Usability

• A 3-year project funded by the EPSRC
• Project staff Marina Jirotka (PI) Anne Trefethen
  (CI) Sharon Lloyd (Advisor) Dimitrina Spencer
  (Researcher) Ralph Schroeder (Researcher) and
  Grace de la Flor (DPhil student) (Andrew Warr
  -researcher Year 1)
• Working closely with the Oxford e-Social Science
  (OeSS) project, the e-Horizons Institute and the
  UK e-Science Usability Task Force (UTF)

3
Objectives

• To develop an online toolkit defining processes
  and practices for managing collaboration to
  facilitate usability in e-Science projects
• Through engagement with project collaborators, to
  investigate approaches, tools and techniques that
  may enable the development of shared
  understanding of e-Science project expectations,
  management and implementation
• To develop recommendations, guidelines and
  procedures that facilitate the effective
  integration of e-Science technologies with
  existing work practices whilst also allowing
  potentially new ways of working
• To consider usability of nationally provided
  services through a broker, and to draw upon case
  studies to provide insights into the use of the
  NGS and large-scale resources like it
• To consider specific tools and technologies that
  allow user engagement with projects such as
  personal Access Grids.
• To develop a set of case studies leading to
  recommendations for managing usability on
  e-Science projects

4
Embedding
Use of large scale infrastructure
Use of services/ applications
Use of collaborative tools and approaches
Task Forces STC, ETF, UTF OGF NGS HPCx CSAR
Research Projects dealing with specific
problem Dame, e-DiaMoND, Integrative Biology,
NeuroGrid, Carmen
Access Grids, PIGS Commercial Solutions
OMII - Product development
5
What is Usability?

• As defined by the ISO standard ISO 9241 Part 11,
  usability can be measured only by taking into
  account the context of use of the system
• who is using the system, what they are using it
  for, and the environment in which they are using
  it
• Furthermore, measurements of usability have
  several different aspects
• effectiveness (can users successfully achieve
  their objectives)
• efficiency (how much effort and resource is
  expended in achieving those objectives)
• satisfaction (was the experience satisfactory)
• Who is the system built for? Management? End
  users?

6
Who are the users?

Typically
Applications
Portals
Middleware Developers
Middleware/Core infrastructure development
Users of
Standards, Products and Trends
7
Infrastructure/Services - challenges

• to 'productise' the outputs from e-science
  projects/initiatives and to ensure outputs were
  developed in a scalable and robust fashion.
• how to ensure that what is developed is usable
  for everyone?
• What do users expect of an infrastructure? -
  robust and sustainable egs HPCx vs CSAR - user
  forums very technical
• Less uptake than expected
• Inadequate understanding of kinds of services
• Insufficient resources to make it happen
• National vs local solution
• Whose responsibility is to ensure usability?

8
Infrastructure - lessons learned

• Mode 1 Provision of national services and
  infrastructure. Seamless and sustainable
  provision - different mode of engagement with
  users training - handholding - documentation..
• Mode 2 Use of standards and services to develop
  own infrastructure for a specific scientific
  problem - need visible/transparent infrastructure
  where users can see what it is doing and modify
  it
• Applications and infrastructure co-evolve
• Gap exercises with users
• Localisation - local staff, system
  administrators, groups of users
• But many projects involve cross-institutional
  work
• No large scale data sets to work across
  institutions
• No one solution as an institution to investigate
• Researchers have to work out own mechanisms for
  long-term collaboration

9
Users and Applications - challenges identified

• In depth qualitative studies of several key
  e-Science projects reveal lack of impact -
  applications not being used beyond lifetime of
  the project
• User requirements not clearly understood - little
  expertise in elicitation or how they fit into the
  development cycle
• Different types of users - middleware
  developers, end users..
• Stakeholder requirements often poorly
  conceptualised - who is a stakeholder?
• Embedding of applications seen as an additional
  requirement once system developed

10
Users - lessons learned

• Early strong user/stakeholder buy in and feedback
  - engaging people who are v busy - communication
  across different groups translation exercise
• Engage in project vision - recalibrate throughout
  project lifetime
• Showcase technical possibilities - milestones in
  project plan
• Understanding current activities tools and
  techniques
• Develop stakeholder analysis to ensure right
  partners and people in the community
• Ensure enough time and resources for engaging
  users efficiently
• Focus on exploitation and impact - is this
  research? and who funds?

11
Collaboration and Communication - Challenges

• Instantaneous methods for communicating are
  important in dynamic teams e.g. use of video
  conferencing requiring a weeks notice is
  problematic - or limited skill set
• Partners may have favourite audio/video services
  - either initiate change, or ensure
  interoperability of services - applies to
  visualisation tools also
• Methods for recording and recollecting
  information are vitally important for wider
  dissemination of project knowledge
• Translation of information to communities
  requires consideration of target audience
  constantly and often may require external
  development - good for public engagement
• Support for training people and institutional buy
  in to provide tools such as AG

12
Collaboration and Communication - lessons learned

• Methods and tools need to be inclusive - so
  select an approach that interoperates with all
  platforms - compliant technologies - Neurogrid -
  Cancergrid commercial solution (coffee room -
  open activities could have been intrusive)
• Using shared repositories and Wikis to collate
  decision making processes - who does this and
  maintains these repositories even after the
  project is over
• Certain tools such as Crewe and Memetic
  potentially interesting for recording - but
  seemingly no active embedding

13
Project Management - challenges

• Scientific vs operational management
• Imperative that there is clear role definition
  and identification of who takes the scientific
  lead and who takes the operational lead or one
  and the same person? Neutral PM beneficial.
• Project initiation key to ensuring project
  progresses with clear understanding by all
  project members of what is expected of them.
• Communication methods
• Developing and maintaining shared visions and
  objectives
• Closedown and Sustainability
• How do you assess the delivery of a project and
  what legacy do they leave behind?

14
Project Management - lessons learned (1)

• Taught methods not wholly applicable
• Process skills needs to be coupled with social
  /management skills (e.g. NLP)
• Executing plans , but if cannot motivate the
  team ?
• Strategic vs operational management
• Scientific vs operational - is PI both?
• Different skill sets - may not be for entire
  duration of project
• Project initiation
• Defining roles and activities, key to maintaining
  the project teams
• Stakeholder questionnaires, cheap means of
  monitoring project team health
• Communication methods - for different purposes
• Teleconferences, wikis, maillist, websites,
  showcases, newsletters
• Development and exploitation plan
• Can be used to communicate how individuals fit
  into the bigger picture
• Consider sustainable routes - users involved to
  sustain activity
• For every deliverable what can be done to push
  beyond project

15
Project Management - lessons learned (2)

• Management Styles
• - Use of language important We Our
• - Empowerment drives problem ownership
• - Embarrassment drives delivery!
• Project closedown
• Needs time
• Must indicate what will happen to output
• Delivering exploitation plan
• Consider what to do with website, wikis, reports,
  documents etc and whether others can benefit from
  them
• Lessons Learned exercises - useful for all
  contributors
• Blueprint documents useful to document what could
  not be achieved - content often results in new
  proposals for follow on projects and enables
  projects to publish knowledge that is not
  research.

16
Consider

• Scientific and operational concerns
• Involving users from project conception to
  closedown and beyond - strategies
• Initiation activities
• Showcasing technical potential
• Closedown activities
• Exploitation plans
• Lessons learned activities
• Blueprint
• Open, modifiable and transparent infrastructure
  (not only by SA)
• Ongoing agreements between users, users and
  developers, and other stakeholders

• Build it and they will come
• Technical decision making in isolation from users
  
• Users determining requirements
• Disciplinary silos
• PM as requirements engineer
• Rigid inflexible technical vision
• No stakeholder analysis at project inception
  and/or throughout project lifetime
• Fixed waterfall development
• No management of user expectations

17
Suggestions

• Provide training and support in operational
  management of large scale multi-disciplinary
  projects?
• Can we learn from the EU Network of Excellence
  approach bringing together communities of
  interest?
• Perhaps OMII open tools knowledge base
  training toolkit for communities of
  interest?