Skip to main content

What Calibration and Imaging products will be delivered to me? - Knowledgebase / Archive & Data Retrieval - ALMA Science

What Calibration and Imaging products will be delivered to me?

Authors list

What you see in your data package when you download it from the archive will depend on whether the imaging and calibration have been performed by the pipeline or with manually produced scripts.

In this article we give a quick guide to the data packages, and more detailed information on the imaging pipeline products for data processed after October 1 2016.

Full description of delivered products can be found at: 
Cycle 4 - 6:

Cycle 7:

Cycle 8:

Cycle 9:

Cycle 10:

Other relevant references: 

In all cases, there are five sub-directories at the lowest level of the directory tree (corresponding to the member observation unit set, MOUS), plus a README file:

calibration  log  product  qa  script README

The calibration directory contains calibration and flagging tables that can be applied to the raw data by in the script directory. In the case of pipeline imaging, the calibration directory also contains the file used to define the continuum for continuum subtraction, <MOUS UID>.cont.dat, within *auxproducts.tgz.

The log directory contains the logs of the CASA runs (manual or pipeline) used to calibrate the data and make the image products. For pipeline runs it also contains a file <MOUS UID>.casa_commands.log that contains a list of CASA commands that can be used to rerun (parts of) the pipeline process.

The qa directory contains Quality Assurance information. For pipeline calibrated data, a weblog is included that can be unzipped and untarred and read by a browser. If the imaging pipeline has been run, the weblog includes the imaging steps. For manual calibrations, the QA information is in several files that together constitute a QA summary for the data.

The script directory contains a set of scripts that can be used to recreate the delivered projects. Details on how to run the scripts to reproduce the pipeline products are given in the User's Guide in the ALMA Pipeline Documentation available at that, in the case of pipeline imaging, the is a null script. For manually calibrated data, scripts for each execution can be found in the files <Execution Block UID>

The product directory contains the image products. Note that this directory is only present if the imaging products are selected for download from the archive. The number of products supplied depends on the process used. Fully manual calibration and imaging makes a subset of products tuned to the PI request, usually at the "bandwidth for sensitivity" picked by the PI in the OT. Similarly, datasets that have been imaged manually contain these tuned products, and also images of the phase and bandpass calibrators made by the pipeline. Data that have been imaged by the pipeline contain a standard set of image products, including continuum and a cube for each spectral window, made at the native resolution in the LSRK velocity frame. Details of the image products are given below:

Pipeline imaging products

Pipeline products

By default, the pipeline will produce a full set of products (aggregated continuum, plus continuum and cube images for every science spectral window) for every source.  However, in practice only a subset of data may be imaged if the computational resources required to make a full set of products are too large. The images are released once a QA2 assessment (Pass or SemiPass) has been made. In the Cycle 10 pipeline, self-calibration was implemented for some cases. Self-calibrated imaging products contain a .selfcal. extension whereas regular non-self-calibrated products contain a .regcal. in the file name. As the pipeline heuristics are still being improved, in some cases you will notice problems with, for example, the continuum subtraction, insufficient clean depth and/or incomplete masking, The CASAGuide ALMA Imaging Pipeline Reprocessing outlines how you can improve upon the imaging from the pipeline.

For a comprehensive guide to the imaging pipeline, please refer to the ALMA Pipeline User's Guide. Bear in mind that most ARCs and ARC nodes encourage data reduction visits if you would like help with improving your images.

Naming convention for pipeline products

Image products are named according to the following convention:

<MOUS UID>.<Source Name>_<intent>.<spectral window(s)>.<dimensionality>.<calibration type>.<stokes>.<comment>.<imagetype>.fits

MOUS UID is a string that uniquely identifies the dataset (e.g. uid___A001_X88f_X270)

<intent> is the observation intent of the source, e.g. sci for science target, ph for phase calibrator, bp for bandpass calibrator.

The spectral window list contains the spectral window numbers used in the product (e.g. spw17, spw17_19_21_23). The quickest way to identify which spectral window corresponds to which frequency/resolution combination is to click on the link to the measurement set on the Home page of the weblog, then to click on the "LISTOBS OUTPUT" button.

<dimensionality>  is either mfs for multifrequency synthesis (resulting in an image with two spatial dimensions), cont for continuum aggregated over all spectral windows (two spatial dimensions), or cube for a cube with two spatial axes and a frequency/velocity axis. You may also see tt0 and tt1 for mfs images made using the the zeroth and first Taylor terms, respectively. The tt0 image corresponds to the regular image, the tt1 image is related to the spectral index image.

<calibration type> indicates if the data were self-calibrated with .selfcal. or not self-calibrated with . Only data imaged with the Cycle 10 pipeline and later will have this designation.

<stokes> indicates the Stokes parameter in the image (e.g., I).

<comment> can be explanatory text, usually added by a manual reducer.

<imagetype> is pbcor for a primary beam corrected image, pb for the primary beam image, mask for the mask image, or sd for single dish.