This site requires javascript to be enabled to fuction correctly


Knowledgebase: ALMA Observing Tool (OT)
How can I reduce my data rates for an observation (and reduce the size of my dataset)?
Posted by Brenda Matthews, Last modified by Rein Warmels on 28 February 2018 07:08 AM

The total data rate in an observation is an estimate of the total instantaneous data rate summed over all basebands based on the current parameter settings. Please see the OT Reference Manual and the OT User Manual, available from the Science Portal under 'Documents & Tools' page inALMA Science Portal, regarding baseband configuration. Basically it is a measure of how much data can be sent to the archive per second, a number which can be considerably less than ALMA is capable of producing. When proposing an FDM (Frequency Division Mode, see OT User Manual) observation in the OT, you may therefore encounter a warning about the data rates your observation will require. The FDM raw data rate (in Mbytes/sec) is approximately

NA(NA - 1)/2 * Nbb * 8192 channels * (4/10)Mbytes/IT

where NA is the number of antennas, Nbb is the number of basebands (normally 4) and IT is the spectral integration time (see ALMA Technical Handbook).

While the maximum data rate is 62.5 MB/s, if the spectral setup ends up with a data rate that is higher than 40 MB/s, you will need to technically justify this (see the Proposer's Guide). If you can lower the date rate (using the methods below) without compromising the science, then you should do so since the final size of your dataset will be reduced and more manageable.

In Phase 1 (proposal submission), the proposer is able to select spectral averaging of channels within a subband (independently for different lines in the same subband provided they all start with the same spectral resolution). This factor is the number of adjacent channels to average together and must be a power of 2, e.g., 1, 2, 4, 8 or 16.  The degree of averaging can be different for different lines. This is useful if you are interested in a relatively small bandwidth but do not require the best spectral resolution that bandwidth can offer or you know you will need to average the data for a significant detection. In this case, opting for the narrower band width combined with spectral averaging will reduce your data rate.

If you need a lot of bandwidth but find you can spectral average your data over many channels, you may be able to use a TDM (Time Division Mode) observation instead. Note that there are constraints on some TDM observations in Cycle 4 (e.g., no option for TDM on the Total Power antennas).

(0 vote(s))
Helpful
Not helpful

Comments (0)