AstroWise meeting

1
Astro-Wise meeting
Tony Linde AstroGrid Project Manager University
of Leicester

• 06-07 April 2004
• OmegaCEN,
• Kapteyn Institute,
• Groningen

2
AstroGrid

• Overview
• Approach
• Architecture
• Usage
• Itn05/06
• Astro-Wise VObs
• Follow-up

3
Overview of AstroGrid/-2

• Duration Sept 2001 Dec 2007Funding 7.7M
  (PPARC)Personnel 26 (23.4 FTE)Goal Develop
  VObs Infrastructure Deploy UK VObsScope Astroph
  ysics, Solar, STP, Optical, X-Ray, Radio,
• Phase A Sept 2001 Dec 2002 Analysis, RD,
  Architecture
• Phase B Jan 2003 Dec 2004 Build, test
  deliver
• AstroGrid-2 Jan 2005 Dec 2007 Structure tbd

4
AstroGrid consortium

• Edinburgh
• Leicester
• Cambridge
• MSSL
• JBO
• RAL
• QUB

• AstroGrid-2
• Bristol
• Exeter
• Leeds
• Portsmouth

5
Lead Personnel

• Project Lead
• Andy Lawrence (Edinburgh)
• al_at_roe.ac.uk
• Project Scientist
• Nic Walton (Cambridge)
• naw_at_ast.cam.ac.uk
• Project Manager
• Tony Linde (Leicester)
• ael_at_star.le.ac.uk
• Technical Lead
• Keith Noddle (Leicester)
• ktn_at_star.le.ac.uk

6
Links

• FP5 AVO, FP6 Euro-VO (UK rep)
• US-NVO
• IVOA
• Standards workgroups
• GGF, Globus, OGSA-DAI
• Other e-Science projects in UK
• GridPP, myGrid, EGSO,

7
Project Approach

• Agile
• iterative incremental development
• Open project
• Wiki http//wiki.astrogrid.org/bin/view/Astrogrid/
  WebHome
• Forum http//forum.astrogrid.org/index.php
• News http//news.astrogrid.org/index.php
• Jabber via jabber.astrogrid.org
• Open Source code
• http//www.astrogrid.org/maven/build/index.html

8
Development Approach

• Iterative
• full cycle each iteration
• 3 month(-ish) iterations
• Incremental
• add functionality each iteration
• refactor wherever necessary
• Use-case driven
• from key 10 science cases
• http//wiki.astrogrid.org/bin/view/Astrogrid/Scien
  ceProblems
• Architecture led
• http//wiki.astrogrid.org/bin/view/Agdoc/ArchVisio
  n
• http//wiki.astrogrid.org/bin/view/Agdoc/ArchOverv
  iew
• http//wiki.astrogrid.org/bin/view/AG2/Architectur
  eNotes

9
Technology choices

• design UML, TogetherSoft ControlCenter
• development language Java
• Web services
• source control CVS
• IDE/build/integrate Eclipse/Ant/Maven
• Java libraries Apache
• Portal Apache Cocoon

10
VObs Structure

• Component-based
• can install any/all components
• Decentralised
• no single product nor centre
• Service-based
• web services

11
Components

• Portal
• Registry
• Security
• Community
• Workflow
• Data Access
• MySpace
• Applications

12
AstroGrid-2

• Aspects
• Extension
• Infrastructure
• Grid (GT4) integration
• Intelligent resource discovery (semantic grid)
• Additional components
• Data exploration
• Agents eStar integration
• Client-based access VO Workbench
• ? Technical Scope
• http//wiki.astrogrid.org/bin/view/AG2/TechnicalSc
  ope

13
AstroGrid-2 Layer Diagram
14
AstroGrid use

• Portal ? Workflow
• Workflow is key to AstroGrid

15
Workflow

• Sequence
• Sequential steps
• Flow
• Steps executed in parallel
• Step
• Application/Tool, Data query
• Configured via XForm specification
• To be enhanced later
• or Query form
• Workflow definition
• Subset of BPEL

16
Create Workflow
17
Add tools to steps
18
Configure step data query
19
Configure step tool
20
Submit workflow, Check status
21
Data query step

• Insert valid ADQL (v0.7.?)
• Assisted query builder this iteration

22
MySpace virtual storage

• Operations
• Copy, Delete, Move
• Upload from file, URL
• End-point
• Multiple servers per manager
• Usually hidden from user

23
eg Copy file, Upload file
24
Registry Query
25
Registry details

• Conforms to IVOA std
• Extended
• ApplicationDefinition
• AstroGrid CEA

26
Registry harvest

• Admin function, so usually invisible

27
Itn05 Complex workflow AVO Demo

• Canned images
• ACE
• AstroGrid wrapping of SExtractor
• VOTMerge
• AstroGrid VOTable federation
• HyperZ
• AstroGrid wrapping of HyperZ
• CEA

28
Itn06 Jan 2005 AVO Demo

• Build on the Itn05 tool
• Image access (SIAP)
• Add in extra redshift applications
• Bpz
• ANNz
• Access wider data resources
• WFS
• GOODS plus IR
• Deploy on 'real' machines
• Disk
• Compute

29
Itn06 Solar science case CMEs
30
Astro-Wise VObs

• Key points
• Make data available
• Publish metadata
• Rapid dissemination (push harvesting)
• Versioning / describing / retiring of products
• Other
• Pipeline services available
• Custom UIs
• Re-process data
• User-derived products
• Agent interfaces

31
Astro-Wise links

• Data model
• IVOA DM group (Jonathan McDowell)
• AstroGrid Interop group (Anita Richards)
• Beta testing
• AstroGrid software
• AstroGrid deployment
• AVO demos

32
Follow-up

• Beta testing
• Contact Nic Walton
• Component Deployment
• Data Centre
• MySpace
• Local Registry
• Community
• Contact Nic Walton
• Software cooperation
• MySpace
• Workflow
• Data Centre
• Registry metadata updating
• Contact Tony Linde, Keith Noddle