Introduction

As the ALMA data reduction process has matured, the naming conventions of MS files have been updated several times and new cases were added. Expanded pipeline capabilities such as self-calibration have also changed the final contents of these files. This article aims to help users differentiate between the MS files they obtain, understand their contents, and clarify which is best to use for imaging.

For details on archived data products, see here:
https://almascience.org/processing/qa2-data-products (Section 5.6)

For details on Pipeline processing, see here:
https://almascience.org/processing/science-pipeline

For help restoring calibrated data from the Archive or ARC-specific services, see here:
https://help.almascience.org/kb/articles/how-do-i-obtain-a-file-of-calibrated-visibilities-measurement-set-for-alma-data

Data columns

Every MS has at least a DATA column. A CORRECTED column may or may not exist depending on if the MS has been split off from a prior MS or if new calibrations have been applied. Certain CASA tasks operate on a specific data column by default or it can be specified. In particular:

• applycal: applies the specified calibration tables to the DATA column and writes the output to the CORRECTED column

• split/mstransform: by default splits out the CORRECTED column of the input MS, placing it in the DATA column of the output MS.

• tclean: creates images by reading from the CORRECTED column, if present, and will fall back to the DATA column if not.

To override default behaviors, the data column can be specified with the parameter datacolumn=’’, with the arguments ‘data’, ‘corrected’, ‘model’, or ‘all’.

Pipeline calibrated IF data

(1) If casa_piperestorescript.py is run on uncalibrated data it restores the calibration stages of the pipeline only. Imaging stages which include splitting out the science target, continuum subtraction, and self-calibration are not run.

(2) The pipeline implemented self-calibration in Cycle 10. Self-calibrated data will be present per-target field only if self-calibration was successful for that field. If no self-calibration was applied for any field, the CORRECTED column will not be created at all.

(3) https://help.almascience.org/kb/articles/what-global-variables-are-available-in-scriptforpi-py

(4) DOCONTSUB=True is automatically triggered if scriptForPI.py detects that the data was successfully self-calibrated during QA2, and the self-cal solutions will be applied.

(5) The CORRECTED column will not be present for this MS directly after restoring the data with scriptForPI. However, running an imaging script that includes a successful self-calibration step would initialize the CORRECTED column and place the self-calibrated data there.

(6) DOSPLIT=True is automatically triggered if scriptForPI.py detects that manual imaging was performed at QA2.

Manually calibrated IF data

(7) a priori calibration includes WVR, system temperature, and antenna positions. Standard calibrations including bandpass calibration, gain calibration, and flux scaling have not yet been performed.

(8) Pipeline imaging tasks require a corrected column and *.ms.flagversions file, so uid*.ms.split.cal cannot be used.

(9) scriptForPI.py will restore this by default for all manually calibrated data.

(10) # indicates session number (1, 2, 3, etc).

(11) Older versions may not include all calibrators or the check source.

Pipeline calibrated SD data

(12) includes Tsys, sky ON-OFF subtraction, and Jy/K factor scaling, applied to the science targets and science spws only

(13) type may be 1, 2, 3, or 4

(14) Cycle 7 and prior do not include ATM correction

Concatenated data

(15) Some other products you may come across include calibrated.ms, calibrated_source.ms, calibrated_final_cont.ms, calibrated_final_line.ms, etc. Ask the helpdesk for clarification on specific data sets.

(16) The IF pipeline began science target imaging in Cycle 4. Manual imaging has tapered off since then.