Next Previous Up Contents
Next: votcopy: VOTable Encoding Translator
Up: tpipe: Generic Table Pipeline Utility
Previous: Other Flags

A.2.2 Examples

Here are some examples of tpipe in use with explanations of what's going on. For simplicity these examples assume that you have the tpipe script installed and are using a Unix-like shell; see Section 2 for an explanation of how to invoke the command if you just have the Java classes.

The examples are arranged with one step (input, filter or destination) per line, to make it easier to see what's going on.

tpipe cat.fits
Writes a table to standard output in human-readable form. Since no mode specifier is given, -write is assumed, and output is to standard output in text format.
tpipe -head 5 cat.fits.gz
Does the same as the last example, but with one processing step: only the first five rows of the table are output. In this case, the input file is compressed using gzip - this is automatically detected.
tpipe -ifmt csv xxx.csv \
      -keepcols "index ra dec" \
      -write -ofmt fits xxx.fits
Reads from a comma-separated values file, writes to a FITS file, and discards all columns in the input table apart from INDEX, RA and DEC.
cat tab1.vot | tpipe - -addcol IV_SUM "(IMAG+VMAG)" \
                       -addcol IV_DIFF "(IMAG-VMAG)" \
                       -delcols "IMAG VMAG" \
                       -write -ofmt votable -o - \
             > tab2.vot
Replaces two columns by their sum and difference in a VOTable. The input and output are actually byte streams on standard input and standard output of the tpipe command in this case. The processing steps first add a column representing the sum, then add column representing the difference, then delete the original columns.
tpipe 2dfgrs_ngp.fits \
      -disk \
      -keepcols "SEQNUM AREA ECCENT" \
      -sort -down AREA \
      -head 20
Displays selected columns for the 20 rows with largest values in the AREA column of a FITS table. First the columns of interest are selected, then the rows are sorted into descending order by the value of the AREA column, then the first 20 rows of the resulting table are selected, and the result is written to standard output. Since a sort is being performed here, it's not possible to do all the processing a row at a time, since all the AREA values must be available for comparison during the sort. Two things are done here to accommodate this fact: first the column selection is done before the sort, so that it's only a 3-column table which needs to be available for random access, reducing the temporary storage required. Secondly the -disk flag is supplied, which means that temporary disk files rather than memory will be used for caching table data.
tpipe http://archive.org/data/survey.vot -meta
Outputs column and table metadata about a table. In this case the table is a VOTable at the end of a URL.
tpipe survey.fits -select "skyDistance(hmsToRadians(RA),dmsToRadians(DEC), \
                                       0.6457,-0.1190) < 5 * ARC_MINUTE" \
                  -count
Counts the number of rows within a given 5 arcmin cone of sky in a FITS table. The skyDistance function is an expression which calculates the distance between the position specified in a row (as given by its RA and DEC columns) and a given point on the sky (RA=0.6457 radians, DEC=-0.1190 radians). Since skyDistance's arguments and return value are in radians, some conversions are required: the RA and DEC columns are sexagesimal strings which are converted using the hmsToRadians and dmsToRadians functions respectively. The result is compared to a multiple of the ARC_MINUTE constant, which is the size of an arcminute in radians. Any rows of the input table for which this comparison is true are included in the output. The functions and constants used here are described in detail in Section 4.4.3.
tpipe -ifmt ascii survey.txt \
      -select "OBJTYPE == 3 && Z > 0.15" \
      -keepcols "IMAG JMAG KMAG" \
      -stats
Calculate statistics on the I, J and K magnitudes of selected objects from a catalogue. Only those rows with the given OBJTYPE and in the given Z range are included. The minimum, maximum, mean, standard deviation etc of the IMAG, JMAG and KMAG columns will be written to standard output.
tpipe -classpath lib/drivers/mysql-connector-java.jar \
      -Djdbc.drivers=com.mysql.jdbc.Driver \
      x.fits \
      -explode \
      -tosql jdbc:mysql://localhost/ASTRO1#TABLEX -user mbt
Writes a FITS table to an SQL table, converting array-valued columns to scalar ones. To make the SQL connection work properly, the classpath is augmented to include the path of the MySQL JDBC driver and the jdbc.drivers system property is set to the JDBC driver class name. The output will be written as a new table TABLEX in the MySQL database called ASTRO1 on a MySQL server on the local host, using the priveliges of MySQL user mbt. If a password is required, it will be prompted for on the terminal (the -password flag could be used to specify it instead). Any existing table in ASTRO1 with the name TABLEX is overwritten. The only processing done here is by the -explode flag, which takes any columns which have fixed-size array values and replaces them in the output with multiple scalar columns.
tpipe USNOB.FITS -every 1000000 -stats
Calculates statistics on a selection of the rows in a catalogue, and writes the result to the terminal. In this example, every millionth row is sampled.


Next Previous Up Contents
Next: votcopy: VOTable Encoding Translator
Up: tpipe: Generic Table Pipeline Utility
Previous: Other Flags

STILTS - Starlink Tables Infrastructure Library Tool Set
Starlink User Note 256
STILTS web page: http://www.starlink.ac.uk/stilts/
Author email: m.b.taylor@bristol.ac.uk