This is an old revision of the document!
Raw OLAP data formats
OLAP produces several data formats, which are intended to be replaced by their final format, such as HDF5. The formats below are not officially supported and subject to change without notice.
Beamformed Data
Beamformed data can be recorded as either complex voltages (yielding X and Y polarisations) or one or more stokes. In either case, a sequence of blocks will be stored, each of which consists of a header and data. The header is defined as:
struct header { uint32_t sequence_number; /* big endian */ char padding[508]; };
in which sequence_number starts at 0, and is increased by 1 for every block. Missing sequence numbers implies missing data. The padding can have any value and is to be ignored.
Complex Voltages
Each (pencil) beam produces two files: one containing the X polarisation, and one containing the Y polarisation. The names of these files adhere to the following scheme:
Lxxxxx_Byyy_S0-bf.raw | X polarisations of beam yyy of observation xxxxx |
Lxxxxx_Byyy_S1-bf.raw | Y polarisations of beam yyy of observation xxxxx |
Each file is a sequence of blocks of the following structure:
struct block { struct header header; /* big endian */ fcomplex voltages[SAMPLES|2][SUBBANDS][CHANNELS]; }
Below is a list of the constants used, and which key they represent in the parset file as produced by OLAP:
SAMPLES | The number of time samples in a block | OLAP.CNProc.integrationSteps |
SUBBANDS | The number of subbands (beamlets) specified | len(Oberservation.subbandList) |
CHANNELS | The number of channels per subband | Observation.channelsPerSubband |
The number of polarisations | 2 |
Stokes
Each (pencil) beam produces one or four files: one containing the Stokes I (power) values, and optionally three files for Stokes Q, U, and V, respectively. The names of these files adhere to the following scheme:
Lxxxxx_Byyy_S0-bf.raw | Stokes I of beam yyy of observation xxxxx |
Lxxxxx_Byyy_S1-bf.raw | Stokes Q of beam yyy of observation xxxxx |
Lxxxxx_Byyy_S2-bf.raw | Stokes U of beam yyy of observation xxxxx |
Lxxxxx_Byyy_S3-bf.raw | Stokes V of beam yyy of observation xxxxx |
Currently (release 2010-09-20), the Stokes U and V are multiplied by a factor of 1/2, but that will be changed in a subsequent release.
Each file is a sequence of blocks of the following structure:
struct block { struct header header; /* big endian */ float stokes[SAMPLES|2][SUBBANDS][CHANNELS]; }
Below is a list of the constants used, and which key they represent in the parset file as produced by OLAP:
SAMPLES | The number of time samples in a block | OLAP.CNProc.integrationSteps |
SUBBANDS | The number of subbands (beamlets) specified | len(Oberservation.subbandList) |
CHANNELS | The number of channels per subband | Observation.channelsPerSubband |
The number of files per beam | len(OLAP.Stokes.which) |
Types
A 'float' is a 32-bit IEEE floating point number. An 'fcomplex' is a complex number defined as
struct fcomplex { float real; float imag; };