public:user_software:documentation:makesourcedb

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
public:user_software:documentation:makesourcedb [2016-10-11 09:12] – Arno Schoenmakerspublic:user_software:documentation:makesourcedb [2022-10-21 14:25] (current) – Fix non logarithmic si definition Tammo Jan Dijkema
Line 28: Line 28:
  
 ==== Format string ==== ==== Format string ====
-The format string defines which fields are present in the ASCII file and which separators are used. + 
-For example: +The format string defines which fields are present in the ASCII file and which separators are used. For example: 
-  Name, Ra, Dec + 
-tells that the lines in the input file contain source name, ra, and dec. It also tells that the values on each line are separated by commas. +<code> 
-\\ A field name can consist of alphanumeric characters, underscores and colons. A colon cannot be the first character. +Name, Ra, Dec 
-\\ All other characters (including whitespace) can be used as separator. A colon can also be used as field separator as long as it is surrounded by whitespace in the format string (to distinguish it from a colon in a field name).+ 
 +</code> 
 + 
 +tells that the lines in the input file contain source name, ra, and dec. It also tells that the values on each line are separated by commas. A field name can consist of alphanumeric characters, underscores and colons. A colon cannot be the first character. All other characters (including whitespace) can be used as separator. A colon can also be used as field separator as long as it is surrounded by whitespace in the format string (to distinguish it from a colon in a field name).
  
 A few special options are available in the format string. A few special options are available in the format string.
-  * Default values are used for missing or empty field values in input lines. Standard default values (as listed in the table below) exist, but it is possible to specify an explicit default value in the format string. It can be done by giving a value (enclosed in quotes) for a field like: <code> Name, Ra, Dec, I='1.5' </code> + 
-  * It is also possible to define a field with a fixed value that does not appear in the input lines like: <code> Name, Ra, Dec, Category=fixed'2' </code> In this way the ''Category'' parameter will be defined for all sources, while the input lines do not contain the ''Category'' field. +  * Default values are used for missing or empty field values in input lines. Standard default values (as listed in the table below) exist, but it is possible to specify an explicit default value in the format string. It can be done by giving a value (enclosed in quotes) for a field like: 
-  * If the input lines contain fields that have to be ignored, they can be skipped by giving no field name in the format line or by using the field name ''dummy''. For example: <code> Name, Ra,,Dec </code> means that the third field is skipped. \\ Note that the name ''dummy'' has to be used if whitespace is used as separator like: <code> Name Ra dummy Dec </code>+ 
 +<code> 
 + Name, Ra, Dec, I='1.5' 
 + 
 +</code> 
 + 
 +  * It is also possible to define a field with a fixed value that does not appear in the input lines like: 
 + 
 +<code> 
 + Name, Ra, Dec, Category=fixed'2' 
 + 
 +</code> 
 + 
 +In this way the ''Category''  parameter will be defined for all sources, while the input lines do not contain the ''Category''  field. 
 + 
 +  * If the input lines contain fields that have to be ignored, they can be skipped by giving no field name in the format line or by using the field name ''dummy''. For example: 
 + 
 +<code> 
 + Name, Ra,,Dec 
 + 
 +</code> 
 + 
 +means that the third field is skipped. Note that the name ''dummy''  has to be used if whitespace is used as separator like: 
 + 
 +<code> 
 + Name Ra dummy Dec 
 + 
 +</code>
  
 The format strings can be given in a few ways: The format strings can be given in a few ways:
-  * Directly in the program ''format'' keyword. + 
-  * As a format line in the input file (before the first value line) or optionally in a separate text file. Such a format line has to start with ''format='' (whitespace is allowed). \\ For backward compatibility the format line can also be given as a special comment line like ''#(...)=format'', where ... is the format string. Whitespace can be used.+  * Directly in the program ''format''  keyword. 
 +  * As a format line in the input file (before the first value line) or optionally in a separate text file. Such a format line has to start with ''format=''  (whitespace is allowed). For backward compatibility the format line can also be given as a special comment line like ''#(…)=format'', where … is the format string. Whitespace can be used. 
 === Field types === === Field types ===
-makesourcedb distinguishes two types of fields: standard fields and other fields. + 
-\\ The standard fields have a predefined (case-insensitive) name. However, the name is case-sensitive in the sense that the name given in the table below is used as the parameter name in the parm table. +makesourcedb distinguishes two types of fields: standard fields and other fields. The standard fields have a predefined (case-insensitive) name. However, the name is case-sensitive in the sense that the name given in the table below is used as the parameter name in the parm table. The following standard fields are known: 
-\\ The following standard fields are known: + 
-^ Field ^ Type ^ Unit ^ Default ^ Description ^ +^Field^Type^Unit^Default^Description| 
-| Name | string | | | Source Name | +|Name|string| | |Source Name| 
-| Type | string | | | Source type. Currently only ''point'', ''gaussian'', or ''shapelet'' is possible. | +|Type|string| | |Source type. Currently only ''point'', ''gaussian'', or ''shapelet'' is possible.| 
-| Ra | angle | | | Right ascension in MVAngle format (see below) | +|Ra|angle| | |Right ascension in MVAngle format (see below)| 
-| Dec | angle | | | Declination in MVAngle format (see below) | +|Dec|angle| | |Declination in MVAngle format (see below)| 
-| I | double | Jy | 1 | Stokes I flux (for Gaussian sources this the //total//' flux) | +|I|double|Jy|1|Stokes I flux (for Gaussian sources this the //total//' flux)| 
-| Q | double | Jy | 0 | Stokes Q flux | +|Q|double|Jy|0|Stokes Q flux| 
-| U | double | Jy | 0 | Stokes U flux | +|U|double|Jy|0|Stokes U flux| 
-| V | double | Jy | 0 | Stokes V flux | +|V|double|Jy|0|Stokes V flux| 
-| MajorAxis | double | arcsec | 1 | FWHM of major axis for a Gaussian source | +|MajorAxis|double|arcsec|1|FWHM of major axis for a Gaussian source| 
-| MinorAxis | double | arcsec | 1 | FWHM of minor axis for a Gaussian source | +|MinorAxis|double|arcsec|1|FWHM of minor axis for a Gaussian source| 
-| Orientation | double | deg | 1 | Orientation of the major axis in degrees North over East | +|Orientation|double|deg|1|Orientation of the major axis in degrees North over East| 
-| IShapelet | string | | | Name of the ASCII file defining the shapelet coefficients and scale of the I flux in the 'Sarod' format. Note that field ''I'' gives the total flux, while ''IShapelet'' defines the shape. | +|OrientationIsAbsolute|boolean| |false|Whether the Orientation is absolute (w.r.t. declination axis) or relative (w.r.t. m axis in not-specified field). See the note below.| 
-| QShapelet | string | | "" | Idem for Q flux. An empty name means that no Q flux is present. Note that the Q (and U) flux can still be given using RotationMeasure. | +|IShapelet|string| | |Name of the ASCII file defining the shapelet coefficients and scale of the I flux in the 'Sarod' format. Note that field ''I'' gives the total flux, while ''IShapelet'' defines the shape.| 
-| UShapelet | string | | "" | Idem for U flux. | +|QShapelet|string| | |Idem for Q flux. An empty name means that no Q flux is present. Note that the Q (and U) flux can still be given using RotationMeasure.| 
-| VShapelet | string | | "" | Idem for V flux. | +|UShapelet|string| | |Idem for U flux.| 
-| SpectralIndex | doubles | | [] | Spectral index. Multiple values can be given enclosed in square brackets and separated by commas. The n-th value defines the n-th order term. \\ Note that the old way of specifying the spectral index (using SpectralIndexDegree and SpectralIndex:i) is not supported anymore. If used, an error is given. | +|VShapelet|string| | |Idem for V flux.| 
-| ReferenceFrequency | double | Hz | | Reference frequency (in Hz) for the spectral index | +|SpectralIndex|doubles| |[]|Spectral index. Multiple values can be given enclosed in square brackets and separated by commas. The n-th value defines the n-th order term. \\ Note that the old way of specifying the spectral index (using SpectralIndexDegree and SpectralIndex:i) is not supported anymore. If used, an error is given.| 
-| RotationMeasure | double | rad/m%%^%%2 | | Rotation measure for the polarized flux Q and U | +|LogarithmicSI|bool| true |Determines the meaning of the spectral index. See Logarithmic SI below| 
-| PolarizationAngle | double | rad | 0 | Polarization angle (in radians) for the polarized flux | +|ReferenceFrequency|double|Hz| |Reference frequency (in Hz) for the spectral index| 
-| PolarizedFraction | double | | 0 | Fraction of I that is polarized | +|RotationMeasure|double|rad/m<nowiki>^</nowiki>2| |Rotation measure for the polarized flux Q and U| 
-| ReferenceWavelength | double | m | c/ReferenceFrequency | Reference wavelength (in m) if RM is specified using Q and U | +|PolarizationAngle|double|rad|0|Polarization angle (in radians) for the polarized flux| 
-| Category | integer | | 2 | Source category (must be 1, 2, or 3) | +|PolarizedFraction|double| |0|Fraction of I that is polarized| 
-| Patch | string | | | Patch name | +|ReferenceWavelength|double|m||Reference wavelength (in m) if RM is specified using Q and U| 
-| rah | double | hr | 0 | Special way to specify RA (see below) | +|Category|integer| |2|Source category (must be 1, 2, or 3)| 
-| rad | double | deg | 0 | | +|Patch|string| | |Patch name| 
-| ram | double | min | 0 | | +|rah|double|hr|0|Special way to specify RA (see below)| 
-| ras | double | sec | 0 | | +|rad|double|deg|0| | 
-| dech | double | hr | 0 | Special way to specify DEC (see below) | +|ram|double|min|0| | 
-| decd | double | deg | 0 | | +|ras|double|sec|0| | 
-| decm | double | min | 0 | | +|dech|double|hr|0|Special way to specify DEC (see below)| 
-| decs | double | sec | 0 | |+|decd|double|deg|0| | 
 +|decm|double|min|0| | 
 +|decs|double|sec|0| |
  
 All non-standard fields have type 'double' and a default of 0. The name as given in the format string is used as the parameter name in the parm table. All non-standard fields have type 'double' and a default of 0. The name as given in the format string is used as the parameter name in the parm table.
  
 Note that the polarized flux can be given directly (as Q and U) or by means of the rotation measure, polarization angle, and polarized fraction. An error is given if both are given for the same source. Note that the polarized flux can be given directly (as Q and U) or by means of the rotation measure, polarization angle, and polarized fraction. An error is given if both are given for the same source.
 +
 +=== Logarithmic Spectral Index ===
 +If ''LogarithmicSI'' is ''true'' (the default), the source intensity 'I' is computed as 
 +
 +''I = I0 * (v/v0) ^ (c0 + c1 * log10(v/v0) + c2 * log10(v/v0)^2 + ...)'', where 'v' is the frequency and 'v0' is the reference frequency.
 +
 +If ''LogarithmicSI'' is ''false'', the spectral index is interpreted in a way compatible with wsclean, namely as 
 +
 +''I = I0 + c0 * (v/v0 - 1) + c1 * (v /v0 - 1)^2 + c2 * (v/v0 -1)^3 + ...''
 +
 +This definition is also described in the [[https://wsclean.readthedocs.io/en/latest/component_list.html|WSClean documentation]].
  
 === Using Rotation Measure === === Using Rotation Measure ===
 +
 The flux of a source can be defined using a rotation measure by means of the field name ''RotationMeasure''. The required polarization info can be given in two ways: The flux of a source can be defined using a rotation measure by means of the field name ''RotationMeasure''. The required polarization info can be given in two ways:
-  * Directly by means of the fields ''PolarizationAngle'' and ''PolarizedFraction''. They define the angle and fraction for wavelength 0. If the field ReferenceWavelength is also given, it must be 0. + 
-  * Indirectly by means of the Q and U flux. ''ReferenceWavelength'' should be given as well to define the wavelength they are valid for. If not given, it will be derived from the spectral index ''ReferenceFrequency''. The polarization angle and fraction will be calculated taking the spectral index into account like: \\ ''  polang  = atan2(U,Q) / 2 - refwvl^2 * rm'' \\ ''  polfrac = sqrt(Q^2 + U^2) / I(refwvl)'' \\ ReferenceWavelength must be > 0 if a spectral index is given.+  * Directly by means of the fields ''PolarizationAngle''  and ''PolarizedFraction''. They define the angle and fraction for wavelength 0. If the field ReferenceWavelength is also given, it must be 0. 
 +  * Indirectly by means of the Q and U flux. ''ReferenceWavelength''  should be given as well to define the wavelength they are valid for. If not given, it will be derived from the spectral index ''ReferenceFrequency''. The polarization angle and fraction will be calculated taking the spectral index into account like: \\ ''polang = atan2(U,Q) / 2 - refwvl^2 * rm'' \\ ''polfrac = sqrt(Q^2 + U^2) / I(refwvl)'' \\ ReferenceWavelength must be > 0 if a spectral index is given.
  
 === Defining Patches === === Defining Patches ===
-A patch is defined when an input line contains a patch name and no source name. For a patch the fields Category, Ra, Dec, and I are also used which define its category, center, and total flux. 
-In a future version of ''makesourcedb'' the center and total flux can be calculated from the sources that are part of the patch. 
  
-A source is attached to a patch if its input line contains a patch name. A line defining that patch should have been processed before this line. +A patch is defined when an input line contains a patch name and no source name. For a patch the fields Category, Ra, Dec, and I are also used which define its category, center, and total flux. In a future version of ''makesourcedb''  the center and total flux can be calculated from the sources that are part of the patch. 
-\\ If no patch name is given for a source (as is usually the case), the source forms a patch in itself and the source name is used as the patch name (with an optional prefix/suffix). In this way BBS can always deal with patches.+ 
 +A source is attached to a patch if its input line contains a patch name. A line defining that patch should have been processed before this line. \\ If no patch name is given for a source (as is usually the case), the source forms a patch in itself and the source name is used as the patch name (with an optional prefix/suffix). In this way BBS can always deal with patches. 
 + 
 +=== Orientation of Gaussian sources === 
 + 
 +Until 2022, the orientation of Gaussian sources has been interpreted somewhat wrong. It was interpreted (both by DP3 and pybdsf) as the angle with respect to the m axis of a certain image. The correct (implicit) definition is the orientation w.r.t. the declination axis. For fields far away from the NCP, and/or sources not too far from the field center, these two definitions are almost identical. 
 + 
 +In DP3 v5.3 a fix was implemented to take the projection effects into account properly. This fix is only called when OrientationIsAbsolute is true. A script will be added to LSMTool to convert from 'relative' orientation to absolute ones (this script needs the phase center of the original image as an input). 
  
 ==== Angle specification ==== ==== Angle specification ====
Line 146: Line 199:
  
 ====== showsourcedb ====== ====== showsourcedb ======
 +
 This program can be used to show the contents of a SourceDB data set. This program can be used to show the contents of a SourceDB data set.
  
-It can be run like: +It can be run like: It can be run like: 
-It can be run like: + 
-  showsourcedb key=value key=value ...+<code> 
 +showsourcedb key=value key=value ... 
 + 
 +</code> 
 The possible keywords and values are described in the following table. The possible keywords and values are described in the following table.
-^ Keyword ^ Type ^ Default ^ Description ^ + 
-| in | string| | Name of the SourceDB; this must be given | +^Keyword^Type^Default^Description| 
-| mode | string | all | patch = show all patches \\ source = show all sources \\ all = show patches and sources \\ make = show all in makesourcedb format | +|in|string| |Name of the SourceDB; this must be given| 
-| patches | string | * | Pattern telling the names of the patches to take |+|mode|string|all|patch = show all patches \\ source = show all sources \\ all = show patches and sources \\ skymodel = show all in skymodel format| 
 +|patches|string|*|Pattern telling the names of the patches to take| 
 + 
  • Last modified: 2016-10-11 09:12
  • by Arno Schoenmakers