Many functions are available for use within your expressions, covering standard mathematical and trigonometric functions, arithmetic utility functions, type conversions, and some more specialised astronomical ones. You can use them in just the way you'd expect, by using the function name (unlike column names, this is case-sensitive) followed by comma-separated arguments in brackets, so
max(IMAG,JMAG)
will give you the larger of the values in the columns IMAG and JMAG,
and so on.
The functions are grouped into the following classes:
If you want to calculate aggregating functions like sum, min, max etc
on multiple values which are not part of an array,
it's easier to use the functions from the Lists class.
Note that none of these functions will calculate statistical functions over a whole column of a table.
The functions fall into a number of categories:
size,
count,
countTrue,
maximum,
minimum,
sum,
mean,
median,
quantile,
stdev,
variance,
join.
add,
subtract,
multiply,
divide,
reciprocal,
condition,
slice,
pick.
Mostly these work on any numeric array type and return
floating point (double precision) values,
but some of them (slice, pick)
have variants for different array types.
array,
which lets you assemble a floating point array value from
a list of scalar numbers.
There are variants (intArray, stringArray)
for some different array types.
Note that for bitwise AND, OR, XOR of integer values etc you can use
the java bitwise operators
"&", "|", "^".
One coverage standard is Multi-Order Coverage maps, described at http://www.ivoa.net/Documents/MOC/. MOC positions are always defined in ICRS equatorial coordinates.
MOC locations may be given as either the filename or the URL of
a MOC FITS file. Alternatively, they may be the identifier of a VizieR
table, for instance "V/139/sdss9" (SDSS DR9).
A list of all the MOCs available from VizieR can
currently be found at
http://alasky.u-strasbg.fr/footprints/tables/vizier/.
You can search for VizieR table identifiers from the
VizieR web page
(http://vizier.u-strasbg.fr/);
note you must use
the table identifier (like "V/139/sdss9")
and not the catalogue identifier (like "V/139").
The following parameters are used:
For a flat universe, omegaM+omegaLambda=1
The terms and formulae used here are taken from the paper by D.W.Hogg, Distance measures in cosmology, astro-ph/9905116 v4 (2000).
Some constants for approximate conversions between different magnitude scales are also provided:
JOHNSON_AB_*, for Johnson <-> AB magnitude
conversions, from
Frei and Gunn, Astronomical Journal 108, 1476 (1994),
Table 2
(1994AJ....108.1476F).
VEGA_AB_*, for Vega <-> AB magnitude
conversions, from
Blanton et al., Astronomical Journal 129, 2562 (2005),
Eqs. (5)
(2005AJ....129.2562B).
The methods here are not specific to the Gaia mission,
but the parameters of the functions and their units are specified
in a form that is convenient for use with Gaia data,
in particular the gaia_source
catalogue available from
http://gea.esac.esa.int/archive/
and copies or mirrors.
There are currently three main sets of functions here:
Position and velocity vectors
Functions are provided for converting the astrometric parameters contained in the Gaia catalogue to ICRS Cartesian position (XYZ) and velocity (UVW) vectors. Functions are also provided to convert these vectors between ICRS and Galactic or Ecliptic coordinates. The calculations are fairly straightforward, and follow the equations laid out in section 1.5.6 of The Hipparcos and Tycho Catalogues, ESA SP-1200 (1997) and also section 3.1.7 of the Gaia DR2 documentation (2018).
These functions will often be combined; for instance to calculate the position and velocity in galactic coordinates from Gaia catalogue values, the following expressions may be useful:
xyz_gal = icrsToGal(astromXYZ(ra,dec,parallax))
uvw_gal = icrsToGal(astromUVW(array(ra,dec,parallax,pmra,pmdec,radial_velocity)))
though note that these particular examples simply invert
parallax to provide distance estimates, which is not generally valid.
Note also that these functions do not attempt to correct for
solar motion. Such adjustments should be carried out by hand
on the results of these functions if they are required.
Functions for calculating errors on the Cartesian components based on the error and correlation quantities from the Gaia catalogue are not currently provided. They would require fairly complicated invocations. If there is demand they may be implemented in the future.
Distance estimation
Gaia measures parallaxes, but some scientific use cases require the radial distance instead. While distance in parsec is in principle the reciprocal of parallax in arcsec, in the presence of non-negligable errors on measured parallax, this inversion does not give a good estimate of distance. A thorough discussion of this topic and approaches to estimating distances for Gaia-like data can be found in the papers
The functions provided here correspond to calculations from Astraatmadja & Bailer-Jones, "Estimating Distances from Parallaxes. III. Distances of Two Million Stars in the Gaia DR1 Catalogue", ApJ 833, a119 (2016) 2016ApJ...833..119A based on the Exponentially Decreasing Space Density prior defined therein. This implementation was written with reference to the Java implementation by Enrique Utrilla (DPAC).
These functions are parameterised by a length scale L
that defines the exponential decay (the mode of the prior PDF is at
r=2L).
Some value for this length scale, specified in parsec, must be supplied
to the functions as the lpc parameter.
Note that the values provided by these functions do not match those from the paper Bailer-Jones et al. "Estimating Distances from Parallaxes IV: Distances to 1.33 Billion stars in Gaia Data Release 2", accepted for AJ (2018) arXiv:1804.10121. The calculations of that paper differ from the ones presented here in several ways: it uses a galactic model for the direction-dependent length scale not currently available here, it pre-applies a parallax correction of -0.029mas, and it uses different uncertainty measures and in some cases (bimodal PDF) a different best distance estimator.
Epoch Propagation
The Gaia source catalogue provides, for at least some sources, the six-parameter astrometric solution (Right Ascension, Declination, Parallax, Proper motion in RA and Dec, and Radial Velocity), along with errors on these values and correlations between these errors. While a crude estimate of the position at an earlier or later epoch than that of the measurement can be made by multiplying the proper motion components by epoch difference and adding to the measured position, a more careful treatment is required for accurate propagation between epochs of the astrometric parameters, and if required their errors and correlations. The expressions for this are set out in section 1.5.5 (Volume 1) of The Hipparcos and Tycho Catalogues, ESA SP-1200 (1997) (but see below), and the code is based on an implementation by Alexey Butkevich and Daniel Michalik (DPAC). A correction is applied to the SP-1200 treatment of radial velocity uncertainty following Michalik et al. 2014 2014A&A...571A..85M because of their better handling of small radial velocities or parallaxes.
The calculations give the same results, though not exactly in the same form, as the epoch propagation functions available in the Gaia archive service.
Some of these resemble similar functions in the Arrays class,
and in some cases are interchangeable, but these are easier to use
on non-array values because you don't have to explicitly wrap up
lists of arguments as an array.
However, for implementation reasons, most of the functions defined here
can be used on values which are already double[] arrays
(for instance array-valued columns) rather than as comma-separated
lists of floating point values.
There are two flavours of functions here:
index-based (random*) and sequential (nextRandom*).
Briefly, the index-based ones are safer to use, but provide poorer
random statistics, while the sequential ones provide decent randomness
but are not suitable for use in some/most contexts.
They are documented separately below.
Index-based functions
The functions named random* all take an index
parameter which determines the value of the result;
the same index always leads to the same output,
but there is not supposed to be any obvious relationship between index
and output.
An explicit index is required to ensure that a given cell always has
the same value, since cell values are in general calculated on demand.
The quality of the randomness for these functions may not be that good.
In most cases, the table row index, available as the special token
$0, is a suitable value for the index parameter.
If several different random values are required in the same table row,
one way is to supply a different row-based index value for each one,
e.g. random(2*$0) and random(2*$0+1).
However, this tends to introduce a correlation between the random
values in the same row, so a better (though in some cases slower) solution
is to use one of the array-generating functions, e.g.
randomArray($0,2)[0] and randomArray($0,2)[1].
The output is deterministic, in the sense that the same invocation will always generate the same "random" number, even across different machines. However, in view of the comments in the implementation note below, the output may be subject to change in the future if some improved algorithm can be found, so this guarantee does not necessarily hold across software versions.
Implementation Note:
The requirement for mapping a given input index deterministically
to a pseudo-random number constrains the way that the random number
generation is done; most well-studied RNGs generate sequences of
random numbers, but that approach cannot be used here, since these
sequences do not admit of random-access.
What we do instead is to scramble the input index somewhat and use that
as the seed for an instance of Java's Random class,
which is then used to produce one or more random numbers per input index.
Some thought and experimentation has gone into the current implementation
(I bought a copy of Knuth Vol. 2 specially!)
and an eyeball check of the results doesn't look all that bad,
but it's still probably not very good, and is not likely to pass
random number quality tests (though I haven't tried).
A more respectable approach might be to use a cryptographic-grade
hash function on the supplied index, but that's likely to be much slower.
If there is demand, something like that could be added as an alternative
option. In the mean time, beware if you use these random numbers for
scientifically sensitive output.
Sequential functions
The functions named nextRandom* have no arguments,
and supply the next value in a global sequence when they are evaluated.
These can be used if scanning through a table once (for instance when
writing a table using STILTS), but they are not suitable for contexts
that should supply a fixed value.
For instance if you use them to define the value of a table cell in TOPCAT,
that cell may have a different value every time you look at it,
which may have disconcerting results.
These use the java.util.Random class in a more standard way
than the index-based functions
and should provide random numbers of reasonable quality.
The k parameter for the HEALPix functions is the
HEALPix order, which can be in the range 0<=k<=29.
This is the logarithm to base 2 of the HEALPix NSIDE parameter.
At order k, there are 12*4^k pixels on the sphere.
yyyy-mm-ddThh:mm:ss.s, where the T
is a literal character (a space character may be used instead).
Based on UTC.
Therefore midday on the 25th of October 2004 is
2004-10-25T12:00:00 in ISO 8601 format,
53303.5 as an MJD value,
2004.81588 as a Julian Epoch and
2004.81726 as a Besselian Epoch.
Currently this implementation cannot be relied upon to better than a millisecond.
The UCD parsing functions are based on Grégory Mantelet's library Ucidy corresponding to UCD1+ 1.4, and the VOUnit parsing functions are based on Norman Gray's library Unity corresponding to VOUnits 1.0.
Full documentation of the functions in these classes is given in Appendix B.1, and is also available within TOPCAT from the Available Functions Window.