Next Previous Up Contents
Next: VOTable
Up: Supplied Output Handlers
Previous: Supplied Output Handlers

#### 3.7.1 FITS

FITS is a very well-established format for storage of astronomical table or image data (see https://fits.gsfc.nasa.gov/). This writer stores tables in BINTABLE extensions of a FITS file.

There are a number of variations in exactly how the table data is written to FITS. These can be configured with `name=value` options in brackets as described below, but for most purposes this isn't required; you can just choose `fits` or one of the standard aliases for commonly-used combinations like `colfits` or `fits-basic`.

In all cases the output from this handler is legal FITS, but some non-standard conventions are used:

fits-plus
  In "fits-plus" format, the primary HDU contains an array of bytes which stores the full table metadata as the text of a VOTable document, along with headers that indicate this has been done. Most FITS table readers will ignore this altogether and treat the file just as if it contained only the table. When it is re-read by this or compatible applications however, they can read out the metadata and make it available for use. In this way you can store your data in the efficient and widely portable FITS format without losing the additional metadata such as table parameters, column UCDs, lengthy column descriptions etc that may be attached to the table. This variant, which is the default, can be explicitly selected with the `primary=votable` option or `fits-plus` alias (if you don't want it, use `primary=basic` or `fits-basic`). This convention is described in more detail in Section 3.8.1.
colfits
  In Column-Oriented FITS output, the HDU containing the table data, instead of containing a multi-row table, contains a single-row table in which each cell is an (nrow-element) array containing the data for an entire column. The point of this is to keep all the data for a single row localised on the disk rather than scattered through the whole file. This can be more efficient for certain applications, especially when the table is larger than physical memory, and has many columns of which only a few are needed for a particular task, for instance plotting two columns against each other. The overhead for writing this format is somewhat higher than for normal (row-oriented) FITS however, and other FITS table applications may not be able to work with it, so in most cases normal FITS is a better choice. This variant can be selected with the `col=true` option or the `colfits-plus/colfits-basic` aliases. If you write to a file with the "`.colfits`" extension it is used by default.
wide
  A private convention is used where required to support encoding of tables with more than 999 columns, which is not possible in standard FITS. If software unaware of this convention (e.g. CFITSIO) is used to read such tables, it will only see the first 998 columns written as intended, plus a column 999 containing an undescribed byte buffer where the rest of the column data is stored. This convention is described in more detail in Section 3.8.2.

For convenience, and compatibility with earlier versions, these standard aliases are provided:

fits-plus
  Alias for `fits` or `fits(primary=votable)`.
fits-basic
  Alias for `fits(primary=basic)`.
fits-var
  Alias for `fits(primary=basic,var=true)`.
colfits-plus
  Alias for `fits(col=true)`.
colfits-basic
  Alias for `fits(col=true,primary=basic)`.
fits-healpix
  This is a special case. It is used for storing HEALPix pixel data in a way that conforms to the HEALPix-FITS serialization convention. In most ways it behaves the same as `fits-basic`, but it will rearrange and rename columns as required to follow the convention, and it will fail if the table does not contain the required HEALPix metadata (`STIL_HPX_*` parameters).

The handler behaviour may be modified by specifying one or more comma-separated name=value configuration options in parentheses after the handler name, e.g. "`fits(primary=basic,col=false)`". The following options are available:

`primary = basic|votable[n.n]|none`
  Determines what is written into the Primary HDU. The Primary HDU (PHDU) of a FITS file cannot contain a table; the following options are available.
  `basic`
    A minimal PHDU is written with no interesting content
  `votable[n.n]`
    The PHDU contains the full table metadata as the text of a VOTable document, along with headers to indicate that this has been done. This corresponds to the "fits-plus" format. The "`[n.n]`" part is optional, but if included (e.g. "`votable1.5`") indicates the version of the VOTable format to use.
  `none`
    No PHDU is written. The output is therefore not a legal FITS file, but it can be appended to an existing FITS file that already has a PHDU and perhaps other extension HDUs.
  (Default: `votable`)
`col = true|false`
  If true, writes data in column-oriented format. In this case, the output is a single-row table in which each cell is an array value holding the data for an entire column. All the arrays in the row have the same length, which is the row count of the table being represented. This corresponds to the "colfits" format. (Default: `false`)
`var = FALSE|TRUE|P|Q`
  Determines how variable-length array-valued columns will be stored. `True` stores variable-length array values after the main part of the table in the heap, while `false` stores all arrays as fixed-length (with a length equal to that of the longest array in the column) in the body of the table.The options `P` or `Q` can be used to force 32-bit or 64-bit pointers for indexing into the heap, but it's not usually necessary since a suitable choice is otherwise made from the data. (Default: `FALSE`)
`date = true|false`
  If true, the DATE-HDU header is filled in with the current date; otherwise it is not included. (Default: `true`)

Multiple tables may be written to a single output file using this format.

If no output format is explicitly chosen, writing to a filename with the extension "`.fits`", "`.fit`" or "`.fts`" (case insensitive) will select `fits` format for output.

The handler class you should usually use for the various variants of the FITS format is `UnifiedFitsTableWriter`, though `FitsTableWriter` may be used instead if fits-plus is not required, and `HealpixFitsTableWriter` is used for the HEALPix variant. In earlier versions of STIL other `*FitsTableWriter` classes were recommended, but these are now deprecated in favour of the above (which have configuration methods to replicate the behaviour of those legacy classes).

To write the FITS header for the table extension, certain things need to be known which may not be available from the `StarTable` object being written; in particular the number of rows and the size of any variable-sized arrays (including variable-length strings) in the table. This may necessitate two passes through the data to do the write.

To fix the value of the `TNULLn` magic value for a given column on output, set the value of the `Tables.NULL_VALUE_INFO` auxiliary metadata value, e.g.:

        colInfo.setAuxDatum(new DescribedValue(Tables.NULL_VALUE_INFO, new Integer(-99)));

Writing columns containing (scalar or array) unsigned byte values (`TFORMnn = 'B'`) cannot be done simply by providing `byte` data to write, since the java byte type is signed. To do it, you must provide a column of java `short` (16-bit) integers, and set the value of the `Tables.UBYTE_FLAG_INFO` auxiliary metadata item to `Boolean.TRUE`, e.g.:

        colInfo.setAuxDatum(new DescribedValue(Tables.UBYTE_FLAG_INFO, Boolean.TRUE));

Writing columns containing (scalar or array) unsigned long values cannot be done straightforwardly, since, like FITS, the Java long type is signed. Instead, you can provide a column of java `String` values giving the integer representation of the numbers required, and set the value of the `BintableStarTable.LONGOFF_INFO` auxiliary metadata item to the string representation of the offset 2**63, e.g.:

       colInfo.setAuxDatum(new DescribedValue(BintableStarTable.LONGOFF_INFO, "9223372036854775808263"));

This will result in the values being written, where in range, with FITS headers `TFORMnn = 'K'`, `TZEROnn = '9223372036854775808263'`. The same mechanism can be used for other long offsets if required (though not for other integer types).

See the "Binary Table Extension" section of the FITS standard for more details of the FITS BINTABLE format. These handlers can write tables with more than the BINTABLE limit of 999 columns, as discussed in Section 3.8.2.

There is some support for the semi-standard HEALPix-FITS serialization convention. If some of the `HPX_*_INFO` metadata items defined by the class `HealpixTableInfo` are present in the output table's parameter list, the corresponding HEALPix-specific FITS headers will be written on a best-efforts basis into the output BINTABLE HDU. This may or may not be good enough to make that FITS file readable by external HEALPix-FITS-aware applications; one of the requirements of the convention is that the HEALPix pixel index, if present, appears in the first column of the table under the name "`PIXEL`". An alternative is to use the handler `HealpixFitsTableWriter`, which tries harder to write tables using the HEALPix convention. This will fail unless the required `HPX_*_INFO` metadata items mentioned above are present, and will reorder and rename columns as required for maximum compatibility.

Next Previous Up Contents
Next: VOTable
Up: Supplied Output Handlers
Previous: Supplied Output Handlers

STIL - Starlink Tables Infrastructure Library
Starlink User Note252
STIL web page: http://www.starlink.ac.uk/stil/
Author email: m.b.taylor@bristol.ac.uk