Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision | |||
public:user_software:dynspec [2017-03-08 15:27] – external edit 127.0.0.1 | public:user_software:dynspec [2020-11-04 08:46] (current) – Bernard Asabere | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== Dynspec Toolkit Container (DTC) ====== | + | ====== Dynspec Toolkit Container (DTC) ====== |
- | This page is be the central information point for the Dynspec Toolkit Container (DTC). | + | This page is be the central information point for the Dynspec Toolkit Container (DTC). Author: vilchez@astron.nl |
- | Author: vilchez@astron.nl | + | |
+ | ==== What's DTC ==== | ||
- | ====What' | + | DTC (for Dynamic spectrum Toolkit Container) is a complete package of tools to reduce, process and convert Beam formed data (ICD3 format) to Dynspec data (ICD6 format) and, finally, visualize LOFAR dynspec data. LOFAR observations can be recorded in datafiles in the Beam Formed data format (ICD3 format, see below). To convert files from the Beam formed data format to Dynspec data format (ICD6 format, see below), the DTC is required. Dynspec data are written in hdf5 format. |
- | DTC (for Dynamic spectrum Toolkit Container) is a complete package of tools to reduce, process and convert Beam formed data (ICD3 format) to Dynspec data (ICD6 format) and, finally, | + | |
- | LOFAR observations can be recorded in datafiles in the Beam Formed data format (ICD3 format, see below). To convert files from the Beam formed data format to Dynspec data format (ICD6 format, see below), the DTC is required. Dynspec data are written in hdf5 format. | + | |
+ | ==== ICD3 Format ==== | ||
+ | |||
+ | A beam formed observation has (or can have) several SAP (Sub-Array Pointing, i.e. multiple direction, for example one on-source and one off-source), | ||
- | ====ICD3 Format==== | ||
- | A beam formed observation has (or can have) several SAP (Sub-Array Pointing, i.e. multiple direction, for example one on-source and one off-source), | ||
- | To process a given datafile, one has to know the SAP, the Beam, and the corresponding Part. | ||
- | These can be be easily identified through the ICD3 nomenclature: | ||
- | For each observation, | ||
* Lxxxxxx_SAP002_BEAM006_S1_P000.h5 | * Lxxxxxx_SAP002_BEAM006_S1_P000.h5 | ||
* Lxxxxxx_SAP002_BEAM006_S1_P000.raw | * Lxxxxxx_SAP002_BEAM006_S1_P000.raw | ||
+ | |||
Within the .raw file, the data are stored in a 2 dimensional matrix (time and frequency). | Within the .raw file, the data are stored in a 2 dimensional matrix (time and frequency). | ||
+ | ==== ICD6 Format ==== | ||
+ | The DTC package converts ICD3 files to ICD6 format. Here, all Beams and all Stokes are grouped in the same dynspec.h5 file. You note that data are not separated in a raw file anymore, and we obtain one dynspec for one SAP (i.e one dynspec per source). Dynspec nomenclature will be: | ||
+ | * Dynspec_(rebinned or complete)_Lxxxxxx_SAPxxx.h5, | ||
- | ====ICD6 Format==== | ||
- | The DTC package converts ICD3 files to ICD6 format. Here, all Beams and all Stokes are grouped in the same dynspec.h5 file. You note that data are not separated in a raw file anymore, and we obtain one dynspec for one SAP (i.e one dynspec per source). | ||
- | Dynspec nomenclature will be: | ||
- | * Dynspec_(rebinned or complete)_Lxxxxxx_SAPxxx.h5, | ||
Within the .h5 file, the data are stored in a 3 dimensional matrix (time, | Within the .h5 file, the data are stored in a 3 dimensional matrix (time, | ||
+ | ==== Visualization with hdfview ==== | ||
+ | On CEP, you can use << [[http:// | ||
+ | < | ||
+ | use hdfview | ||
- | ====Visualization with hdfview==== | + | </ |
- | On CEP, you can use << [[http:// | + | |
- | <file>use hdfview</ | + | |
- | Below, just a screen shot using hdfview of a dynspec file and a Beam formed file. | + | |
- | Differences in the file organization between ICD3 and ICD6 format are evident, as are the different data dimensions. | + | |
- | {{: | + | Below, just a screen shot using hdfview |
+ | {{: | ||
+ | ==== DTC capabilities ==== | ||
- | |||
- | ====DTC capabilities==== | ||
DTC provides a number of possibilites (example of data processing using DTC): | DTC provides a number of possibilites (example of data processing using DTC): | ||
+ | |||
* ICD3 observations (h5+raw data) are copied to a directory | * ICD3 observations (h5+raw data) are copied to a directory | ||
- | * Use the << | + | * Use the << |
- | * If you are interested in a particular event in your dynamic spectrum, you can extractr it. Select the frequency and time range you want to isolate. Because Beam Formed data are usually very voluminous, it is recommended to select only a part of the dynspec and also rebin this part in frequency and in time. | + | * If you are interested in a particular event in your dynamic spectrum, you can extractr it. Select the frequency and time range you want to isolate. Because Beam Formed data are usually very voluminous, it is recommended to select only a part of the dynspec and also rebin this part in frequency and in time. |
* To convert the Beam formed (ICD3) data (and/or select, rebin your data) to Dynspec (ICD6) data, it is required to use the << | * To convert the Beam formed (ICD3) data (and/or select, rebin your data) to Dynspec (ICD6) data, it is required to use the << | ||
* Use the << | * Use the << | ||
Extra tools (with optional functionality): | Extra tools (with optional functionality): | ||
- | | + | |
+ | | ||
* Use the << | * Use the << | ||
- | * Use the << | + | * Use the << |
+ | |||
+ | Note: :!: The << | ||
- | Note: :!: The << | + | ==== INSTALL DTC ==== |
+ | DTC dependencies, | ||
- | ====INSTALL | + | ==== How to use the DTC package: details of each tool ==== |
- | DTC dependencies, | + | |
- | [[dysnpec_install | DTC install]] | + | |
+ | DTC usage and tool functionalities are described on this web page: [[: | ||
- | ====How to use the DTC package: details of each tool==== | + | ==== How to use the DTC package |
- | DTC usage and tool functionalities are described on this web page: | + | |
- | [[dynspec_tools | DTC usage]] | + | |
- | ====How | + | If you want to use DTC on CEP, continue reading here: [[: |
- | If you want to use DTC on CEP, continue reading here: [[dysnpec_cep|dynspec_cep]] | + | ==== LOFAR Radio Observatory: |
- | ====LOFAR Radio Observatory: | + | DTC usage and tool functionalities for ASTRON. are described on this web page: [[:operations: |
- | DTC usage and tool functionalities | + | |
- | [[operations: | + | |
- | ====DTC Tutorials==== | + | ==== DTC Tutorials ==== |
- | Several tutorials are avaible here on different targets: | + | |
- | * Sun tutorial: [[dynspec_tutorial | The Sun]]. | + | |
- | * Cas A tutorial: [[dynspec_tutorial_casa | Cas A]]. | + | |
- | * Jupiter tutorial: [[dynspec_tutorial_jupiter| Jupiter]]. | + | |
+ | Several tutorials are avaible here on different targets: | ||
+ | * Sun tutorial: [[: | ||
+ | * Cas A tutorial: [[: | ||
+ | * Jupiter tutorial: [[: | ||