An Astronomical Image Mosaic Service for the National Virtual Observatory http://montage.ipac.caltech.edu/

1
An Astronomical Image Mosaic Service for
theNational Virtual Observatoryhttp//montage.ip
ac.caltech.edu/
2
Contributors

• Attila Bergou - JPL
• Bruce Berriman - IPAC
• Ewa Deelman - ISI
• John Good - IPAC
• Joseph C. Jacob - JPL
• Daniel S. Katz - JPL

• Carl Kesselman - ISI
• Anastasia Laity - IPAC
• Thomas Prince - Caltech
• Gurmeet Singh - ISI
• Mei-Hui Su - ISI
• Roy Williams - CACR

3
What is Montage?

• Delivers custom, science grade image mosaics
• User specifies projection, coordinates, spatial
  sampling, mosaic size, image rotation
• Preserve astrometry flux
• Modular toolbox design
• Loosely-coupled Engines for Image Reprojection,
  Background Removal, Co-addition
• Control testing and maintenance costs
• Flexibility e.g., custom background algorithm
  use as a reprojection and co-registration engine
• Implemented in ANSI C for portability
• Public service will be deployed on the Teragrid
• Order mosaics through web portal

4
Public Release of Montage

• Version 1.7.1 and Users Guide available for
  download at http//montage.ipac.caltech.edu/
• Emphasizes accuracy in photometry and astrometry
• Images processed serially
• Tested and validated on 2MASS 2IDR images on
  Linux Red Hat 8.0 (Kernel release 2.4.18-14) on
  a 32-bit processor
• Tested on 10 WCS projections with mosaics smaller
  than 2 x 2 degrees and coordinate transformations
  Equ J2000 to Galactic and Ecliptic
• Extensively tested
• 2,595 test cases executed
• 119 defects reported and 116 corrected
• 3 remaining defects to be corrected in future
  Montage release

5
Applications of Montage

• Large scale processing of the sky e.g.,
  Atlasmaker
• Mosaics of the Infrared Sky
• This is the age of Infrared Astronomy!
• Infrared astronomers study regions much larger
  than covered by individual cameras gt need to
  make mosaics to investigate star formation,
  redshift distribution of galaxies
• Mosaics of the far infrared sky a primary data
  product of the SIRTF mission
• Two SIRTF Legacy teams using Montage as a
  co-registration and mosaic engine to generate
  science mosaics, perform image simulations,
  mission planning pipeline testing
• Public Outreach - the wow factor!
• Combine single color images in Photoshop
• Example back-lit display in NASA booth
• See next image - Rho Ophiuchi

6
Rho Ophiuchi 324 2MASS images in each band gt
972 imagesOn a 1 GHz Sun, mosaicking takes about
15 hours
7
Montage The Grid Years

• Re-projection is slow (100 seconds for one 1024 x
  512 pixel 2MASS image on a single processor 1.4
  GHz Linux box), so use parallelization inherent
  in design
• Grid is an abstraction - array of processors,
  grid of clusters,
• Montage has modular design - run on any
  environment
• Prototype architecture for ordering a mosaic
  through a web portal
• Request processed on a computing grid
• Prototype uses the Distributed Terascale Facility
  (Teragrid)
• This is one instance of how Montage could run on
  a grid
• Atlasmaker is another example of Montage
  parallelization

8
Montage The Grid Years (cont.)

• Prototype version of a methodology for running on
  any grid environment
• Many parts of the process can be parallelized
• Build a script to enable parallelization
• Called a Directed Acyclical Graph (DAG)
• Describes flow of data and processing
• Describes which data are needed by which part of
  the job
• Describes what is to be run and when
• Standard tools can execute a DAG

9
Using Montage Grid Prototype

• Web service at JPL creates an abstract workflow
  description of Montage run (in XML)
• Workflow description run through Pegasus to
  create concrete DAG
• Pegasus includes transfer nodes in the concrete
  DAG for staging in the input image files and
  transferring out the generated mosaic
• Concrete DAG submitted to Condor

Region Name, Degrees
JPL
Abstract DAG
Pegasus Concrete DAG Condor DAGMAN
ISI Condor Pool
10
Montage Workflow

• Described as abstract DAG - specifies
• Input, output, and intermediate files
• Processing jobs
• Dependencies between them

11
Montage Concrete DAG (single pool)
Example DAG for 10 input files
mProject
mDiff
mFitPlane
mConcatFit
mBgModel
mBackground
mAdd
12
Montage Runs on the Teragrid

• Test runs were done on the 1.5 degree x 1.5
  degree area including M42 (Orion Nebula)
• Required 113 input image files
• Single pool DAG for SDSC consisted of 951 jobs
• 117 were data transfer jobs
• 113 for transferring the input image files
• 3 for transferring other header files
• 1 for transferring the final output mosaic
• Run took 94 minutes

13
Montage Runs on the Teragrid (2)

• Using same abstract DAG for multi pool DAG at
  SDSC and NCSA created 1202 jobs
• 367 were data transfer jobs
• Some of these jobs transferred multiple files
• 249 of these 367 were inter pool data transfer
  jobs

14
Montage Computations

• Building a mosaic from N 1024 x 512 pixel 2MASS
  images on a single processor 1.4 GHz Linux box
  takes roughly (N x 100) seconds
• 98-99 of this time is in the reprojection, which
  can be perfectly parallelized (this doesnt
  embarrass us)

Dataset of images Size of each image Sky coverage Total number of pixels (x 1012) Storage size (TB) Processing time for all data in 1.4 GHz IA32 processor hours (x 1,000)
2MASS 4 million 17 x 8.5 at 1 100 2.1 8 111
DPOSS 2,600 6.6 x 6.6 at 1 50 1.4 3 74
SDSS (DR1) 50,000 13.6 x 9 at 0.4 25 1.2 2.4 65
15
Summary

• Montage is a custom astronomical image mosaicking
  service that emphasizes astrometric and
  photometric accuracy
• First public release, Montage_v1.7.1, available
  for download at the Montage website
• A prototype Montage service has been deployed on
  the Teragrid ties together distributed services
  at JPL, Caltech IPAC, and ISI
• More on Montage at SC2003
• See ISI at ANL booth for demo of Pegasus portal
  running Montage on the Teragrid
• See back-lit display of Montage images at NASA
  booth
• Montage website http//montage.ipac.caltech.edu/