Databases, software, etc. for X-ray astronomy

1
Databases, software, etc. for X-ray astronomy

• Keith Arnaud
• NASA Goddard
• University of Maryland

2
What I will cover
Archives CALDB Data reduction and analysis
software
3
Databases
HEASARC
heasarc.gsfc.nasa.gov/docs/archive.html Chandra
Data Archive
asc.harvard.edu/cda XMM Science Archive
xmm.vilspa.esa.es/xsa/ Mirror
sites
4
HEASARC
Provides access to all the publically available
X- and gamma-ray datasets. Also many general
catalogs and datasets held at other data centers
(eg Chandra and XMM-Newton). Powerful search
engine (Browse) with a uniform interface across
missions. Batch processing and cross-correlation
capabilities. Includes bibliographic links to
datasets where available. If you havent used
browse before read the introduction at
heasarc.gsfc.nasa.gov/W3Browse/w3browse-help.html
5
Chandra Data Archive
Provides access to all the Chandra datasets using
an interface tailored for Chandra. ChaSeR (java)
and WebChaSeR (web page) provide search and
retrieval of datasets. A fast image retrieval
option with a simplified interface is intended
for the general public but can also be used to
get quick looks at data before deciding whether
to download it. Bibliography search for Chandra
papers on a given object or observation.
6
XMM Science Archive
Java-based interface provides access to all
XMM-Newton datasets. Many XMM-specific search
options including on attributes in the EPIC
source catalog. Starting to include optical
images from the source ID program. New users
should take the guided tour at xmm.vilspa.esa.es/e
xternal/xmm_data_acc/xsa/docs/tour.html
7
Mirror sites
Several sites mirror some datasets. You should
use these if you are physically located near
them. LEDAS at Leicester University in the UK has
mirrors of the Chandra, ASCA, ROSAT archives and
the XMM source catalog. DARTS at ISAS in Japan
has several archival datasets including a mirror
of the ROSAT All-sky survey. ASDC in Italy has
BeppoSAX, ASCA, and ROSAT datasets.
8
CALDB
One of the problems in X-ray astronomy is the
large number of files that are usually required
to store all the calibration information for an
instrument. Keeping track of these and making
these available to the data analysis software is
challenging. The CALDB is a directory structure
and indexing system for FITS files which enables
software to read the correct calibration data
without needing to know the filename or
directory. Users can either install a CALDB
locally or for occasional use access the
calibration files from the HEASARC site. CALDB is
used for HEASARC archival and current missions
and for Chandra but not (alas) for XMM.
9
Software packages
HEAsoft - heasarc.gsfc.nasa.gov/docs/software/lh
easoft CIAO - asc.harvard.edu/ciao SAS -
xmm.vilspa.esa.es/external/xmm_sw_cal/sas_frame.sh
tml All these packages operate on FITS files
with standard formats defined for each type of
dataset. We have attempted to make it possible to
mix and match tools from different packages. At
present it may not be possible to run CIAO in the
same window as HEAsoft due to the presence of a
compiler library in the CIAO distribution.
10
HEAsoft
Large and diverse package distributed by NASA
Goddard. Latest release is 6.0.2. Runs under
Solaris, OSF, OS-X, Cygwin, and Linux. Can be
downloaded as either source code or binaries. The
latter is usually adequate unless you want to
apply source patches (usually for
xspec). Contains generic multi-mission tools as
well as packages for current missions RXTE,
Swift, Suzaku and a number of archival
missions. For help heasarc.gsfc.nasa.gov/cgi-bin
/ftoolshelp
11
HEAsoft components

• XANADU - High-level, multi-mission tasks for
  X-ray astronomical spectral, timing, and imaging
  data analysis
• FTOOLS - General and mission-specific tools to
  manipulate FITS files
• FITSIO - Core library responsible for reading
  and writing FITS files (distributed with FTOOLS)
• fv - General FITS file browser/editor/plotter
  with a graphical user interface (distributed with
  FTOOLS)
• XSTAR - Tool for calculating the physical
  conditions and emission spectra of photoionized
  gases

12
CIAO
Written for Chandra by the CXC but contains many
tools that can be used for other
purposes. Current version is 3.2.2 which is
supported on Solaris 8, Redhat 6.2, Redhat 8, and
Mac OS X 10.2.8. The binary distribution will
also run under Redhat 7.x, Redhat 9.0, SuSE 7.x,
Mac OS X 10.3 and 10.4. The CIAO source is
available but it is recommended that you download
the binaries and not try to compile it yourself.
For help asc.harvard.edu/helpdesk
13
CIAO components
Chandra-specific tools Multi-mission DM
(DataModel) tools ds9 image display ChIPS
plotting package Sherpa multidimensional fitting
engine GUIDE tool for identifying atomic features
in spectra (uses the APED atomic physics
database)
14
SAS
Tools to be used on data from XMM-Newton. Current
version is 6.1.0 and is supported for Redhat 7.1,
9.0, RHEL 3, SuSE 7.3, 8.2, Solaris 6, 8, 9, Mac
OS X 10.2, 10.3.4, OSF 5.1. Binary versions only
are available. Requires the user to have
installed ds9, XPA, Grace, HEAsoft For help
xmm.vilspa.esa.es/external/xmm_user_support/xmmhel
p_frame.shtml
15
Software overview

• Divide X-ray astronomy software into several
  classes
• Calculating event attributes - sky position,
  PI, grade,
• Cleaning and filtering events
• Extracting products - images, spectra,
  lightcurves,
• Creating observation calibration files
• High-level analysis - source detection,
  spectral fitting

16
Filling event columns
The first step is to calculate various event
attributes such as the sky position, the energy,
and, for CCDs, the grade. Usually this will be
performed automatically for you in pipeline
processing. However, if calibration files change
you might want to reprocess. For Chandra this is
performed using the CIAO routines
acis_process_events or hrc_process_events.
Reprocessing involves setting the input
parameters correctly and making sure that you
have the latest calibration files. XMM
reprocessing is performed using emproc, epproc,
rgsproc for the MOS, PN, and RGS respectively.
17
Cleaning and filtering
The next step is to remove events that do not
meet some criterion for validity, remove events
during any times of particularly bad background.
You might also want to select only events within
a particular region on the sky or on the
detector. There are many tools to select rows
out of FITS files however you do need to be a bit
careful about which ones are used. Other software
may need to know what selections were made. For
instance suppose you select events within some
time interval. You cant reconstruct this
selection just from the events left in the file
because there might be real gaps in the data.
18
Cleaning and filtering II
Selections are tracked by storing additional
information in the file either in keywords or
extra extensions. In CIAO this is done using the
DataModel filtering. See ahelp dmfiltering for
details. The SAS has either a command line option
called evselect or a GUI called xmmselect. See eg
xmm.vilspa.esa.es/sas/current/doc/evselect HEASOFT
has a program xselect which can be used to
perform filtering (through the filter command).
xselect actually runs an ftool called extractor,
which is available for the sophisticated
user. All these methods should be equivalent.
19
Extracting products
Images, energy spectra, and lightcurves are
histograms on event attributes so again one could
use general software to create these but in many
cases subsequent software requires particular
keywords or extensions so it is better to use our
standard tools. The HEAsoft program xselect has a
command extract which can be used to make these
products. heasarc.gsfc.nasa.gov/docs/software/lhea
soft/ftools/xselect/xselect.html evselect and
xmmselect in SAS have options to create these
products. In CIAO dmextract is used to make
energy spectra and lightcurves and dmcopy to make
images.
20
Creating observation calibration files
Once you have your products you may need to
create other files in order to continue with
scientific analysis. For instance, analysis of
images may need exposure maps and spectra require
responses. These tools are always
mission-specific since they depend on details of
the instruments in use. For Chandra you will want
to look at the appropriate analysis threads in
asc.harvard.edu/ciao/threads/index.html For SAS a
good place to start is the XMM ABC guide at
xmm.gsfc.nasa.gov/docs/xmm/abc/abc.html
21
Analysis of products
Finally, you can get to the scientific analysis
you wanted to do in the first place. Source
detection in images, fitting models to energy
spectra, searching for periods in lightcurves,