Virtual Research Environments: The Sakai VRE Demonstrator

1
Virtual Research EnvironmentsThe Sakai VRE
Demonstrator

• Rob Allan, Xiaobo Yang, Rob Crouchley, Adrian
  Fish and Miguel Gonzalez

2
What is a Virtual Research Environment?

• A VRE is defined as a distributed way of working
  using a Web-based portal and for linking into
  users' desktop applications to access a wide and
  growing range of on-line tools. These include
  access to Grid based computing and data
  management systems as well as collaboration
  tools, some based on Web 2.0. A VRE is both a
  one-stop shop'' for academic users and a
  turnkey solution'' for commercial users.
• These emerging characteristics of a VRE are
  increasingly overlaid with a requirement to
  provide support for the creation, further
  development, or enhancement of a research
  community in virtual space - a Virtual Research
  Community''. The OST (UK Office of Science and
  Technology) report of March 2006 indicated that
  VRCs have the potential to open exciting new
  opportunities to collaborate in research and thus
  realise significant gains at institutional,
  national and international levels.
• In this talk we only consider Web based VREs
  using portal technology.

3
JISC VRE-1 Sakai Demonstrator

• JISC VRE-1 Programme 2005-2007
• 4 partner sites Daresbury, Lancaster, Oxford,
  Portsmouth (now Reading)
• Framework extensions
• Security Shibboleth
• WSRP
• JSR-168
• New tools, DMS, Agora, WSRP Consumer, Grid
  portlets, Blogger, Shared Whiteboard, Bridging
  tools, Semantic search tool
• Production portal for e-Research projects
  currently some 400 users and 25 projects hosted.

http//rhine.dl.ac.uk8080/portal
4
Classes of User

• In observing usage patterns we have seen the
  following
• Expert HPC user is happy to log on and develop
  applications
• Semi-expert users like remote scripting
  interfaces
• Novice users like generic portals to test the
  functionality
• Application-based communities develop rich
  clients, e.g. desktop GUI
• There is no single solution that will satisfy all
  the diverse requirements, but exposing a common
  set of underlying services and using standards to
  promote inter-operability can help. This is the
  key to rapid and agile application development,
  using and combining remote resources.
• We are trying to use Sakai to combine a rich set
  of well-integrated internal services with more
  loosely integratedremote services.

5
Classification of Grid User(adapted from Foster
and Kesselman)
6
Some Questions about VRE Usage

• Deployment and evaluation of such a VRE tests and
  extends our understanding of practical IT-based
  support for research in the following areas
• How can such frameworks be configured to best
  suit the expectations and work practices of
  different research user communities and
  institutional or organisational contexts?
• Can tools from multiple institutions and
  organisations be brought together coherently to
  enable sharing of information, processes and
  collaboration?
• Can community-specific tools be integrated
  meaningfully alongside generic and
  remotely-hosted Web tools?
• Can a portal-like approach provide the
  flexibility to enable effective use by both
  researchers and administrators?
• At what points are rich desktop tools or those
  provided by a mobile platform, more effective?
• How might these be best integrated to provide a
  meaningful user experience?

7
Portals and VREs

• The idea of portals has been around for a number
  of years. We organised the Portals and Portlets
  2003 Workshop in Edinburgh just at the time when
  two significant pieces of technology, the JSR-168
  portlet standard and WSRP 1.0, Web Services for
  Remote Portlets standard, were being agreed. We
  organised a second Portals and Portlets Workshop
  in 2006.
• Since 2003, a number of open-source and
  commercial portal projects have been launched and
  are in use for a variety of purposes. One example
  in the UK is the portal for the National Grid
  Service. This evolved from HPC Portal which was
  initially a Perl/ C based environment for
  launching and monitoring Grid jobs similar to the
  US GridPort and HotPage portals from San Diego
  Supercomputer Centre. After briefly using PHP
  technology we have now evolved to using JSR-168
  portlets firstly in the GridSphere and
  StringBeans frameworks and more recently in
  uPortal and Sakai.

See Dave Merediths talk
8
Example NGS Portal

• HPC Portal v4.0 provides a set of tried and
  tested portlets which will work in a variety of
  frameworks, and can be distributed as MyNGS.
• Built-in help system offers guidance for new
  users. Training and documentation is also
  available.
• Open pages show non-authenticated users what is
  on offer.
• Authentication via modified JAAS layer offering,
  local portal id, Grid certificate authentication,
  Shibboleth.
• JSDL editing and sharing portlets use an
  underlying database. Sharing JSDL job
  descriptions encourages developments of
  communities of practice around specific Grid
  applications.
• Job submission across NGS, NW-GRID, SCARF and
  local resources can be configured.
• Data management also included.

http//portal.ngs.ac.uk
9
NGS Portal Application Registry
10
Science Gateways I

• A VRE is however more than just a portal. Whilst
  NGS Portal has a number of tools to encourage
  people to share artefacts, e.g. descriptions of
  computational tasks or workflows, it has very
  little built-in community support. It is
  important to address this if e-Science
  technologies and the Grid are going to be taken
  up more widely.
• In the USA this is done through the concept of
  Science Gateways such as NEESit. A number of
  these science gateways are listed on the TeraGrid
  Web site.
• Scientific gateways can have varying goals and
  implementations. Some expose specific sets of
  community codes so that anonymous scientists can
  run them. Others may serve as a "metaportal," a
  community portal that brings a broad range of new
  services and applications to the community. A
  common trait of all three types is their
  interaction with the TeraGrid through the various
  service interfaces that TeraGrid provides.
  Although the gateways may be instantiated on
  TeraGrid resources, it is expected that many will
  be instantiated on community resources and be
  administered by the community itself.

11
Science Gateways II

• Science Gateways signal a paradigm shift in
  traditional high performance computing use.
  Gateways enable entire communities of users
  associated with a common scientific goal to use
  national resources through a common interface.
  Science gateways are enabled by a community
  allocation whose goal is to delegate account
  management, accounting, certificates management,
  and user support to the gateway developers.
• Science Gateways take three common forms
• A gateway that is packaged as a Web portal
  with users in front and TeraGrid services in
  back
• Grid-bridging Gateways often communities run
  their own Grids devoted to their areas of
  science. In these cases the Science gateway is a
  mechanism to extend the reach of the community
  Grid so it may use the resources of the TeraGrid
• A gateway that involves application programs
  running on users' machines (i.e. workstations and
  desktops) and accesses services in TeraGrid (and
  elsewhere).

12
VREs and CWEs

• According to Wikipedia a Collaborative Working
  Environment (CWE) supports people (e.g.
  E-professionals) in their individual and
  cooperative work. Research in CWE involves
  organisational, technical, and social issues. It
  lists tools or services which may be considered
  elements of a CWE including e-Mail, instant
  messaging, application sharing, video
  conferencing, collaborative workspace, document
  management, task and workflow management, Wiki
  and Blog. Access Grid is mentioned as being a
  particular type of CWE. It will be seen below
  that many of these tools have also been
  recognised as being important in our VRE
  development and are now available in Sakai. Not
  all this work is described here, in particular
  the important work on the Agora conferencing and
  desktop sharing tool from Lancaster University,
  was initially funded as part of the VRE
  Demonstrator and CQeSS project. This tool
  addresses the requirements of desktop-based video
  conferencing!

http//agora.lancs.ac.uk
13
Agora

• Agora is an easy to use online meeting tool. With
  Agora you can take your workplace with your
  laptop.
• Video-conference "many to many" Organised into
  virtual meeting rooms, you can video-conference
  with an unlimited number of participants().
• Shared desktop You can broadcast what you are
  watching on your desktop.
• Whiteboard Collaborative whiteboard on which
  anybody can sketch.
• Chat Instant messaging application.
• Moviecasting Broadcast movies.
• Session recording Record your sessions for
  further analysis.

Demos on Lancaster stand
14
Half Way House?

• Sakai is not a portal, but has many portal-like
  characteristics and similar look-and-feel.
• Sakai supports a Tool Portability Profile
  enabling close integration within the Sakai
  framework
• Sakai uses many underlying standards
• Sakai was designed as a Collaborative Learning
  Environment, so also shares many aspects of CWEs
• It is designed to be scalable, supporting 10,000s
  of users
• Works with Oracle 10g
• To enable interoperability with portal
  technologies we added a WSRP Consumer tool to
  Sakai (there was already a Producer)
• More recently a native JSR-168 interface has been
  added, based on Pluto 1.1
• Sakai tools can also be exposed in portals, such
  as uPortal, so Sakai could be viewed as a Service
  Hosting Environment.
• We think this is required for a VRE

15
Sakai as a VO Management Tool

• In the terminology of Sakai, a VO maps onto a
  worksite''. Through their worksites, bespoke
  tools can be made available to the VOs that
  require them. Each worksite can be customised to
  have a specific look-and-feel and configured to
  contain just the tools that are required by its
  members. This can include Web interfaces to
  distributed services managed by a particular
  project or hosted as part of a Grid resource.
• Sakai's internal VO management is through
  role-based policies. Users can be allocated roles
  within each worksite. Roles can be extended by
  administrative users from the small number of
  defaults like admin' and maintain''.
• Certain users can configure new sites, and
  resources can be shared between sites.
• Other concepts include permissions, types,
  realms, skins, properties, groups, aliases.
• Sites can be public, private or joinable.

16
Roles and Permissions
Users see only what they have access to. Some
additional worksites are joinable.
Worksite Role
Front page(not logged in) view only
My site maintain
Site 1 create
Site 2 maintain
Site 3 access
Site x created from Site 1 maintain
Each worksite provides a list of tools and view
of underlying content depending on the users role
17
Managing Users and Tools
18
Customisation and Personalisation
19
Built-in Web 2.0 Services

• For the end users, Web 2.0 typically provides
  hosted services enabling them to use a Web
  browser to interact with, contribute content to
  the Web and invoke remote operations.
• There is a growing list of such tools which are
  being hosted in the Sakai server and database.
  They can be rendered as stand-alone pages or
  tiled in various combinations as required.
• Blog
• Wiki
• Calendar
• Chat client
• Message Center
• Shared Resources
• Announcements
• Workshop paper management

• RSS News reader
• Glossary
• Threaded Discussion Forum
• Other tools mashup remote Web 2.0 services from
  Google, Yahoo, FaceBook, etc.

20
Wiki and Discussion Forum
21
Calendar and RSS News
22
My Workspace
Each user has a workspace which aggregates views
of all the sites they belong to.
23
Web 2.0 Map Mashup

• Recently we have investigated how to augment the
  built-in Web 2.0 services by making use of the
  Yahoo! Maps Web Service. Such a Web API greatly
  alleviates the entry level of developing Web 2.0
  for geo-spatial research applications. The
  services provide a set of APIs (AJAX or Flash)
  through which developers can easily access online
  maps around the world and overlay their own
  information (mashup).
• What we have tested is to display a map of the
  Sakai Community similar to the one located at the
  Sakai Web site inside our VRE.
• We expect this kind of mashup technology to be of
  use in a number of research fields such as
  archeology, flood monitoring and prediction,
  climate simulation and urban decision making in
  addition to supporting other forms of
  collaborative working, such as locating Access
  Grid rooms. Users will upload their data into
  Resources/MapData and then select which to
  overlay on the map.

24
Screenshot
25
Rollout, Sustainability and Community

• Sakai is running on a fully-supported IBM
  BladeCenter at Daresbury Laboratory currently
  with 28 dual-processor Xeon blades. The content
  is hosted in the Oracle 10g database on the UK
  National Grid Service (RAL node).
• We are currently deploying fully-operational and
  supported Sakai-based VREs for the following
  communities
• NW-GRID a community of computational scientists,
  both academic and commercial, using compute
  clusters in the North West of England.
• ESRC e-Infrastructure a community of
  multi-disciplinary social scientists thoughout
  the UK building a common infrastructure and
  adopting e-Science technology through the work of
  NCeSS and ReDReSS http//www.ncess.ac.uk and
  http//redress.lancs.ac.uk
• Diamond e-Infrastructure a community of
  experimental scientists using the new Diamond
  Light Source http//www.diamond.ac.uk the
  largest investment in science in the UK for 30
  years.