How do I handle the visibility flagging issues potentially present in the ALMA datasets delivered to PIs up to November 2014?
Posted by Dirk Petry, Last modified by Sarah Wood on 19 February 2020 04:41 PM
The following two visibility flagging issues were found in the ALMA data analysis done with the CASA package. The issues do not invalidate science results but they may have reduced the sensitivity of your observation. Since delivered datasets did pass ALMA quality assurance (likely QA2), achievement of the science goals was not harmed.
Issue 1: Potentially extensive flagging due to bug in the storage manager of the FLAG_ROW column
This bug was present in all CASA releases before CASA 4.2.2. For large datasets (ca. >33 antennas), it may have led to the unintentional flagging of visibilities thereby reducing the sensitivity. However, all intended flagging took place.
In the current CASA release, you will find a new table tool method which enables you to test your measurement set (as it comes out of calibration with scriptForPI.py) as to where it is affected by the bug. The new tool method is used as follows:
tb.open(<name of your MS here>)
If your dataset is not affected, the testincrstman method should return the message
and return value "True." If the dataset is affected, you will see the message
and the return value "False." In this case, your dataset was potentially flagged too much and you should consider contacting your ARC (or ARC node in Europe) for help with reprocessing. The reprocessing involves simply upgrading your calibration script to the current CASA release and repeating the calibration.
This issue affects datasets where the science spectral windows (SPWs) have overlap with the edge channels (first and last 4 channels) of the SPWs used for the Tsys measurement. In general, this will be datasets in low-resolution SPWs in TDM. For all science SPWs which have overlap with the edge channels of the Tsys SPWs, the imaging WEIGHT may have been underestimated in CASA 4.2.2 since the Tsys SPW edge channels were not flagged before applycal. The affected science results are continuum images combining two or more SPWs at least one of which has the overlap described above.
The effect consists of lower sensitivity since the relative WEIGHT between SPWs might not be quite correct, especially when TDM SPWs were combined with FDM (high-resolution) SPWs. In that case, the TDM SPWs were given too low weight.
To achieve maximum sensitivity, you can reprocess your data introducing a command to flag the edge channels of your Tsys solutions before the first call to applycal in your calibration script. e.g.
If you think your data is affected by this issue, consider contacting your ARC (or ARC node in Europe) for help with reprocessing.