2G Instrumentation Pipelines

1
2G Instrumentation Pipelines

• Sandra Castro

2
Pipeline Products
1. Product FITS header
2. QC parameters (PAF file)
3. Naming products
4. Saving products
3
Pipeline Products 1

• All product files created by the pipeline must be
  in FITS format

4
Product FITS header
creating a
product header
empty or raw header
cpl_dfs_setup_product_header()
5
Setup Header

• cpl_error_code
• cpl_dfs_setup_product_header(
• cpl_propertylist header,
• const cpl_frame proframe,
• const cpl_frameset set,
• const cpl_parameterlist pars,
• const char recid
• const char pipeid
• const char dicid
• )
• recid ? ocam_mbias
• pipeid ? PACKAGE/PACKAGE_VERSION
• dicid ? PRO-1.15

6
Setup Header

• Copy to header ORIGIN, TELESCOPE, INSTRUME,
  OBJECT, RA, DEC, EPOCH, EQUINOX, RADECSYS,
  DATE-OBS, MJD-OBS, UTC, LST, PI-COI, OBSERVER.
• Copy all the HIERARCH.ESO._ keywords, with the
  exception of the HIERARCH.ESO.DPR._, and of the
  .PRO._ and .DRS._ keywords.
• If found, remove the HIERARCH.ESO.DPR._ keywords
  from header.
• If found, remove the ARCFILE and ORIGFILE
  keywords from header.
• Add to header the PRO keywords PIPEFILE,
  PRO.DID, PRO.REC1.ID, PRO.REC1.DRS.ID,
  PRO.REC1.PIPE.ID, PRO.DATANCOM, PRO.CATG, and
  PRO.TYPE.

http//archive.eso.org/Tools/DidRep/DidRepWebQuery
ESO-VLT-DIC.PRO
7
Multi-HDU Headers
Common keywords to all extensions
INSTRUME PIPEFILE HIERARCH ESO PRO CATG
PRO CATG
Keywords for extension 1
EXTNAME DET1.CHIP1 All other HIERARCH ESO
PRO HIERARCH ESO QC
Keywords for extension 2
8
Multi-HDU Headers
/ Plist is my primary header /
cpl_dfs_setup_product_header(Plist, proframe,
set, pars, recid, pipeid, dicid)
/ Remove undesired keywords /
cpl_propertylist_erase_regex(Plist, ESO PRO, 0)
/ Xlist is my extension header /
cpl_dfs_setup_product_header(Xlist, proframe,
set, pars, recid, pipeid, dicid)
/ Remove undesired keywords /
const char delkeys (ORIGINTELESCOPEINSTRUM
EOBJECTRADECEPOCH PIPEFILEEQUINOXRADECSYSD
ATE-OBSMJD-OBSUTCLSTPICOIOBSERVER)
cpl_propertylist_erase_regex (Xlist, delkeys, 0)
/ Add PRO.CATG to primary header /
cpl_propertylist_copy_property (Plist, Xlist,
ESO PRO CATG)
/ Save product /
cpl_image_save(NULL, name, bitpix, Plist,
CPL_IO_DEFAULT)
cpl_image_save(image, name, bitpix, Xlist,
CPL_IO_EXTEND)
9
Pipeline Products 2

• All QC parameters must be also saved
• on disk on a log file. One file per FITS extension

10
QC parameters PAF file

• PAF.HDR.START start of header
• PAF.TYPE pipeline product
• PAF.ID ocam_mbias
• PAF.NAME ocam_masterbias.paf
• PAF.DESC QC file
• PAF.CHCK.CHECKSUM
• PAF.HDR.END end of header
• ARCFILE OCAM_IMG_BIAS076_0003.fits
• EXTNAME DET1.CHIP1 extension name
• DATE-OBS 2005-03-17T121644632 Date
• TPL.ID OCAM_img_cal_bias Template id
• MJD-OBS 53446.5 Obs start
• QC.MBIAS.MEAN 120.2234
• QC.MBIAS.STDEV 15.23445

11
Pipeline Products 3

• All output FITS files must be named in a unique
  and predictable way by the recipe

12
Naming Products
OCAM_IMG_OBS076_0001.fits OCAM_IMG_OBS076_0002.fit
s OCAM_IMG_OBS076_0003.fits
ocam_reduce_sci
ocam_JITTERED_IMAGE_1.fits ocam_JITTERED_IMAGE_2.f
its ocam_JITTERED_IMAGE_3.fits ocam_COADD.fits

• no matter what input files
• no matter what input parameters

TEXT
TEXT
TEXT
predictable
13
Pipeline Products 4

• All product files must be saved on the working
  directory

14
Good news!
CPL function to save PAF files
CPL function to save products
Planned for CPL-4.0
15
Pipeline Products
1. Products must be FITS files
2. QCs are written on headers and PAF files
3. Names must be unique and predictable
4. Products are saved on the working directory
16
(No Transcript)