ms - Tool

NRAO Home > CASA > CASA Toolkit Reference Manual

1.3.1 ms - Tool

Operations on measurement sets
Requires: table Synopsis

Description

The ms tool provides functions to manipulate the contents of measurement set tables. The functions can be categorised as shown below.

Attaching to a Measurement Set

The simplest and most common way to attach an ms tool to a measurement set is to use the ms.open function which requires that you specify the name of the measurement set table.

The function ms.fromfits converts a uvfits file to a measurement set table prior to attaching the ms tool to the newly created measurement set. The conversion step may take some time if the fits file is large. However it only needs to be done once. The measurement set table is not deleted when you close the ms tool, using the close function, or exit casa. And once the measurement set table is created it is much faster to attach an ms tool to it using the ms.open function.

Getting summary information

The summary function will display, in the logger, an overview of the measurement set. This will include listings of the fields, spectral windows and polarization setups used in the measurement set.

The range function will provide more quantitative information on the minimum, maximum or used values of specified parameters. When using this function you may need to do an initial selection, as described below, depending on whether the parameters you ask for change their shape. The lister function provides a concise listing of the data in the measurement set.

The listhistory function lists the contents of the measurement set history table. The history table contains a record of changes made to the measurement set by flagger, calibrater, imager and other tools.

Selecting data

As described in the ms module documentation a measurement set can contain data with a variety of different shapes. Some of the functions in this tool require the data to be in a fixed shape. Before you can use these functions you need to select a subset of the data in the measurement set where all the data has a fixed shape. There are two functions which can be used to do this. These are the selectinit and command functions.

The select function can be used to further refine which subset of the data will be used by the data access functions. This function allows you to select specific rows in a measurement set using a wide range of criteria.

The select function can only select whole rows in a measurement set. To select specific channels within a row you use the selectchannel function. Similarly to select specific polarizations you should use the selectpolarization function.

Reading and writing data

The getdata function is used to read data from the measurement set into casapy variables. You can select which columns of the measurement set main table you are interested in and only the subset of data specified using the selection functions described above will be retreived. Any frequency averaging (see the selectfrequency function) and polarization conversion (see the selectpolarization function) will be done when you retreive the data. The full power of casapy and other casa tasks and tools, like plotxy, can then be used for adhoc inspection and calculations involving the data.

If the measurement set was opened for writing then the putdata function can be used to write the data back into the measurement set. When writing data back into the measurement set you cannot change the data shape or the coordinates of the data, only the numerical values. This means that you cannot write data that has been averaged in frequency or converted to different polarizations.

When using the getdata function with a large measurement set you need to be careful to not request too much data. The measurement set is stored on disk but casapy variables are stored in memory. To allow you to access large amounts of data in an ordered way the ms tool provides functions that allow you to iterate through the data in a convienient way.

If you need to step through the data in an orderly fashion, you can use the iteration functions. These allow you to set up an iteration order (iterinit), obtain the first iteration (iterorigin), go to the next iteration (iternext) and end the iteration prematurely (iterend). The iterorigin and iternext function set the currently selected table (as used by getdata and others) to the current iteration. At the end of the iteration, the original selection is restored.

You can iterate through a measurement set you have previously selected using select, but if you use select while iterating, you cannot get back the unselected iteration (without reiterating through the table until the current point).

The writehistory function allows messages to be appended to the measurement set history table should you wish to do so. The listhistory function lists your messages and those created by flagger, calibrater, imager and other tools.

Conversions to FITS

Just as the fromfits function will convert a uvfits file to a measurement set the tofits function will convert a measurement set to a uvfits file. Similarly a single dish measurement set ie., one with a FLOAT_DATA column rather than a DATA column, can be converted to a single dish fits file using the tosdfits function.

You cannot read a uvfits file into a measurement set and write it out as a single dish fits file or vice-versa.

Concatenation

The concatenate function can be used to append the data from one measurement set to the end of another. As all the data is copied this function may take some time if the measurement set to be copied is large. The measurement set needs to be opened for writing for this to work.

The virtconcatenate function enables virtual concatenation, i.e. the data is not rewritten but just reindexed such that the two input MSs have the same subtables. They can then be turned into a multi-MS.

Sorting the main table by time

The timesort function permits you to sort the MS main table by time in ascending order. This can be useful after a concatenation.

Sorting the main table by a custom set of columns

The sort function permits you to sort the MS main table by a custom set of columns in ascending order. This can be useful to compare tables generated in different ways (e.g.: cvel and mastransform)

Splitting The split function allows you to make a new ms from a subset of the actual ms.

Table 1.6:

Items recognized by the range, select, getdata and putdata functions.
Items marked with a † are only available in interferometric measurement sets.
Items marked with a ‡ are only available in interferometric measurement sets
that have been processed with calibrator or imager. Items marked with a * do
not require all the data in the selected measurement set to have the same shape.

& range

select

getdata

putdata

comment

amplitude†

amplitude of observed data

corrected_amplitude‡

amplitude of corrected data

model_amplitude‡

amplitude of model data

ratio_amplitude‡

amplitude of corrected/model

residual_amplitude‡

amplitude of residual data

obs_residual_amplitude‡

amplitude of obs residual data

antenna1

1st antenna id

antenna2

2nd antenna id

antennas

list of antenna names

array_id

axis_info

description of data axes

chan_freq

channel frequencies

corr_names

list of polarization strings

corr_types

list of polarization enum values

data†

complex observed data

corrected_data‡

complex corrected data

model_data‡

complex model data

ratio_data‡

complex corrected/model

residual_data‡

complex residual data

obs_residual_data‡

complex observed residual data

feed1

1st feed in correlation

feed2

2nd feed in correlation

field_id

field number

fields

list of field names

flag

data flags

flag_row

MS row flags

flag_sum

flag summary

add hour angle to axis_info

ifr_number

1000*antenna1+antenna2

imaginary†

imag part of observed data

corrected_imaginary‡

imag part of corrected data

model_imaginary‡

imag part of model data

ratio_imaginary‡

imag part of corrected/model

residual_imaginary‡

imag part of residual data

obs_residual_imaginary‡

imag part of obs residual data

last

add LAST to axis_info

num_corr

number of polarizations

num_chan

number of freq channels

phase†

phase of observed data

corrected_phase‡

phase of corrected data

model_phase‡

phase of model data

ratio_phase‡

phase of corrected/model

residual_phase‡

phase of residual data

obs_residual_phase‡

phase of observed residual data

phase_dir

list of phase centers & epoch

real†

real part of observed data

corrected_real‡

real part of corrected data

model_real‡

real part of model data

ratio_real‡

real part of corrected/model

residual_real‡

real part of residual data

obs_residual_real‡

real part of observed res. data

ref_frequency

reference frequency

rows

row numbers in original table

scan_number

sigma

sigma of the data

data_desc_id

time

MJD time range in seconds

times

list of MJD timeslots

add UT to axis_info

uvw

uvw vector

u coordinate

v coordinate

w coordinate

uvdist

uv distance

weight

weight of the data

Example:

ms.open("3C273XC1.MS",nomodify=False)
ms.selectinit(datadescid=0)
ms.select({’antenna1’:[1,3,5],’uvdist’:[1200.,1900.]})
rec=ms.getdata(["weight","data"])
# modify rec[’weight’] and rec[’data’] values as desired
ms.putdata(rec)
ms.close()

We open the MS for writing, select an array and spectral window and then select a few antennas and a uv range. We then get out the weight values and the data. We change these values in casapy and then write them back to the measurement set. Finally, we close the ms, causing the values to be written back to disk.

Methods

ms	Create an ms tool attached to a measurement set
fromfits	Create a measurement set from a uvfits file
fromfitsidi	Create a measurement set from a fits-idi file
listfits
createmultims
ismultims
getreferencedtables
getfielddirmeas	Returns the direction measure from the given FIELD table column and row
nrow	Returns the number of rows in the measurement set
nrow2	Returns the number of rows in the measurement set
iswritable	Returns True is the underlying Table is writable
open	Attach the ms tool to a measurement set table
reset	Re-attach the tool to the original MS.
close	Detach the ms tool from the measurement set table
name	Name of the measurement set table the tool is attached to.
statwt2	THIS APPLICATION IS CURRENTLY FOR TESTING ONLY! USE AT YOUR OWN RISK! Compute and set weights based on variance of data.
tofits	Convert a measurement set to a uvfits file
summary	(PARTIALLY IMPLEMENTED!!!) Summarize the measurement set
getscansummary	Get the summary of the ms
getspectralwindowinfo	Get a summary of the spectral windows

listhistory	List history of the measurement set
writehistory	Add a row of arbitrary information to the measurement set history table
??	Get statistics on the selected measurement set
??	Get statistics on the selected measurement set
??	Get the range of values in the measurement set
??	Get the range of values in the measurement set
??	List measurement set visibilities
??	Get the MS metadata associated with this MS.
??	Initialize the selection of an ms
??	Initialize the selection of an ms
??	Use the MSSelection module for data selection.
??	Return the selected indices of the MS database. The keys in the record are the same as those used in msselect function (i.e. ’spw’, ’time’, ’field’, ’baseline’, ’scan’, ’scanintent’, ’polarization’ and ’uvdist’).
??	Select a subset of the measurement set.
??	Select a subset of the measurement set.
??	Select a subset of the measurement set.
??	Select a subset of the measurement set.
??	Select and average frequency channels
??	Select and average frequency channels
??	Selection and convertion of polarizations
??	Selection and convertion of polarizations

??	transform spectral data to different reference frame and/or regrid the frequency channels
??	transform spectral data to different reference frame and/or regrid the frequency channels
??	calculate the transformed grid of the SPW obtained by combining a given set of SPWs (MS is not modified)
??	Read values from the measurement set.
??	Read values from the measurement set.
??	Write new values into the measurement set
??	Write new values into the measurement set
??	Concatenate two measurement sets
??	Concatenate only the subtables of two measurement sets excluding the POINTING table (resulting MAIN and POINTING table not useful)
??	Concatenate two measurement sets virtually
??	Sort the main table of an MS by time
??	Sort the main table of an MS using a custom set of columns
??	Subtract the continuum from the visibilities
??	Set WEIGHT and SIGMA from the scatter of the visibilities
??	make a new ms from a subset of an existing ms, adjusting subtables and indices
??	make a new ms from a subset of an existing ms, without changing any subtables
??	Initialize for iteration over an ms
??	Initialize for iteration over an ms
??	Set the iterator to the start of the data.
??	Set the iterator to the start of the data.

??	Advance the iterator to the next lot of data
??	Advance the iterator to the next lot of data
??	End the iteration and reset the selected table
??	End the iteration and reset the selected table
??	Test if the MS was imported with option lazy=True in importasdm and optionally change the ASDM reference
??	Continuum fitting and subtraction in uv plane
??	Continuum fitting and subtraction in uv plane
??	Closes the ms tool
??	Returns ids of the selection used
??	Hanning smooth the frequency channels to remove Gibbs ringing.
??	Subtract model from the corrected visibility data.
??	Connect an ephemeris table with the MS FIELD table
??	Read values from the measurement set. Use this method instead of the older getdata() method which is marked for deprecation.
??	Initialize for iteration over an ms. Use this method instead of the older iterinit() method which is marked for deprecation.
??	Set the iterator to the start of the data. Use this method instead of the older iterorigin() method which is marked for deprecation.
??	Advance the iterator to the next lot of data. Use this method instead of the older iternext() method which is marked for deprecation.
??	Query if there are more iterations left in the iterator. Use this method instead of the older iterend() method which is marked for deprecation.