Starlink User Note 256
Mark Taylor
6 November 2024
stilts
command
cmd_*
)
mode_*
)
fits
colfits
votable
cdf
csv
ecsv
ascii
ipac
pds4
mrt
parquet
hapi
feather
gbin
tst
wdc
fits
votable
csv
ecsv
ascii
ipac
parquet
feather
text
html
latex
tst
mirage
addcol
addpixsample
addresolve
addskycoords
assert
badval
cache
check
clearparams
collapsecols
colmeta
constcol
delcols
every
explodeall
explodecols
fixcolnames
group
head
healpixmeta
keepcols
meta
progress
random
randomview
repeat
replacecol
replaceval
rowrange
select
seqview
setparam
sort
sorthead
stats
tablename
tail
transpose
uniq
sky
: Sky Matching
skyerr
:
Sky Matching with Per-Object Errors
skyellipse
:
Sky Matching of Elliptical Regions
sky3d
: Spherical Polar Matching
exact
: Exact Matching
1d
, 2d
, ...:
Isotropic Cartesian Matching
2d_anisotropic
, ...:
Anisotropic Cartesian Matching
2d_cuboid
, ...:
Cuboid Cartesian Matching
1d_err
, 2d_err
, ...:
Cartesian Matching with Per-Object Errors
2d_ellipse
:
Cartesian Matching of Elliptical Regions
mark
size
sizexy
xyvector
xyerror
xyellipse
xycorr
link2
mark2
poly4
mark4
polygon
area
central
lines
marks
handles
yerrors
xyerrors
statline
statmark
arrayquantile
line
linearfit
label
arealabel
contour
grid
fill
quantile
histogram
kde
knn
densogram
gaussian
function
skyvector
skyellipse
skycorr
skydensity
healpix
skygrid
xyzvector
xyzerror
line3d
spheregrid
yerror
spectrogram
arrayjoin
: Adds table-per-row data as array-valued columns
calc
: Evaluates expressions
cdsskymatch
:
Crossmatches table on sky position against VizieR/SIMBAD table
cone
: Executes a Cone Search-like query
coneskymatch
:
Crossmatches table on sky position against remote cone service
datalinklint
:
Validates DataLink documents
funcs
: Browses functions used by algebraic expression language
pixfoot
: Generates Multi-Order Coverage maps
pixsample
: Samples from a HEALPix pixel data file
plot2plane
: Draws a plane plot
plot2sky
: Draws a sky plot
plot2cube
: Draws a cube plot
plot2sphere
:
Draws a sphere plot
plot2corner
: Draws a matrix of plane plots
plot2time
: Draws a time plot
plot2d
: Old-style 2D Scatter Plot
plot3d
: Old-style 3D Scatter Plot
plothist
: Old-style Histogram
regquery
: Queries the VO registry
server
: Runs an HTTP server to perform STILTS commands
sqlclient
:
Executes SQL statements
sqlskymatch
:
Crossmatches table on sky position against SQL table
sqlupdate
: Updates values in an SQL table
taplint
: Tests TAP services
tapquery
: Queries a Table Access Protocol server
tapresume
: Resumes a previous query to a Table Access Protocol server
tapskymatch
:
Crossmatches table on sky position against TAP table
tcat
: Concatenates multiple similar tables
tcatn
: Concatenates multiple tables
tcopy
: Converts between table formats
tcube
: Calculates N-dimensional histograms
tloop
: Generates a single-column table from a loop variable
tgridmap
: Calculates N-dimensional density maps
tgroup
: Calculates aggregate functions on groups of rows
tjoin
: Joins multiple tables side-to-side
tmatch1
: Performs a crossmatch internal to a single table
tmatch2
: Crossmatches 2 tables using flexible criteria
tmatchn
: Crossmatches multiple tables using flexible criteria
tmulti
: Writes multiple tables to a single container file
tmultin
: Writes multiple processed tables to single container file
tpipe
: Performs pipeline processing on a table
tskymap
: Calculates sky density maps
tskymatch2
: Crossmatches 2 tables on sky position
votcopy
: Transforms between VOTable encodings
votlint
: Validates VOTable documents
xsdvalidate
:
Validates against XML Schema
STILTS is a set of command-line tools for processing tabular data. It has been designed for, but is not restricted to, use on astronomical data such as source catalogues. It contains both generic (format-independent) table processing tools and tools for processing VOTable documents. Facilities offered include crossmatching, format conversion, format validation, column calculation and rearrangement, row selection, sorting, plotting, statistical calculations and metadata display. Calculations on cell data can be performed using a powerful and extensible expression language.
The package is written in pure Java and based on STIL, the Starlink Tables Infrastructure Library. This gives it high portability, support for many data formats (including FITS, VOTable, text-based formats and SQL databases), extensibility and scalability. Where possible the tools are written to accept streamed data so the size of tables which can be processed is not limited by available memory. As well as the tutorial and reference information in this document, detailed on-line help is available from the tools themselves.
The STILTS application is available under the GNU General Public License (GPL) though most parts of the library code may alternatively be used under the GNU Lesser General Public License (LGPL).
STILTS provides a number of command-line applications which can be used for manipulating tabular data. Conceptually it sits between, and uses many of the same classes as, the packages STIL, which is a set of Java APIs providing table-related functionality, and TOPCAT, which is a graphical application providing the user with an interactive platform for exploring one or more tables. This document is mostly self-contained - it covers some of the same ground as the STIL and TOPCAT user documents (SUN/252 and SUN/253 respectively).
Currently, this package consists of commands in the following categories:
tcopy
,
tpipe
,
tmulti
,
tmultin
,
tcat
,
tcatn
,
tloop
,
tjoin
,
arrayjoin
,
tgridmap
,
tgroup
, and
tcube
(see Section 6).
tmatch1
,
tmatch2
,
tmatchn
and
tskymatch2
(see Section 7).
plot2plane
,
plot2sky
,
plot2cube
,
plot2sphere
,
plot2corner
and
plot2time
(also deprecated old-style plot commands
plot2d
,
plot3d
and
plothist
)
(see Section 8).
tskymap
,
pixfoot
and
pixsample
.
votcopy
and
votlint
.
cdsskymatch
,
cone
,
coneskymatch
,
tapquery
,
tapresume
,
tapskymatch
,
taplint
,
datalinklint
and
regquery
.
sqlclient
,
sqlupdate
and
sqlskymatch
.
calc
,
funcs
,
server
and
xsdvalidate
.
There are many ways you might want to use these tools; here are a few possibilities:
server
command may help,
but is not required, for use in these situations.
stilts
command
All the functions available in this package can be used from
a single command, which is usually referred to in this document
simply as "stilts
". Depending on how you have installed
the package, you may just type "stilts
",
or something like
java -jar some/path/stilts.jaror
java -classpath topcat-full.jar uk.ac.starlink.ttools.Stiltsor something else - this is covered in detail in Section 3.
In general, the form of a command is
stilts <stilts-flags> <task-name> <task-args>The forms of the parts of this command are described in the following subsections, and details of each of the available tasks along with their arguments are listed in the command reference at the end of this document. Some of the commands are highly configurable and have a variety of parameters to define their operation. In many cases however, it's not complicated to use them. For instance, to convert the data in a FITS table to VOTable format you might write:
stilts tcopy cat.fits cat.vot
Some flags are common to all the tasks in the STILTS package,
and these are specified after the stilts
invocation itself
and before the task name. They generally have the same effect
regardless of which task is running. These generic flags are as
follows:
-help
stilts
command
itself and exits. The message contains a listing of all the
known tasks.
-version
-verbose
+verbose
can be used to do the opposite
(reduce the logging level by one notch).
-allowunused
-prompt
-prompt
flag,
then you will be prompted for every parameter you have not
explicitly specified to give you an opportunity to enter a value
other than the default.
-bench
-debug
-batch
-batch
flag,
then you won't be prompted at all.
-memory
-Dstartable.storage=memory
.
-disk
-Dstartable.storage=disk
.
-memgui
-checkversion <vers>
<vers>
. If it is not, STILTS will exit with
an error. This can be useful when executing in certain controlled
environments to ensure that the correct version of the application
is being picked up.
-stdout <file>
-
" for <file>
will restore this default behaviour.
-stderr <file>
-
" for <file>
will restore this default behaviour.
If you are submitting an error report, please include the result of
running stilts -version
and the output of the troublesome
command with the -debug
flag specified.
The <task-name>
part of the command line is the
name of one of the tasks listed in Appendix B - currently
the available tasks are:
arrayjoin
calc
cdsskymatch
cone
coneskymatch
datalinklint
funcs
pixfoot
pixsample
plot2corner
plot2cube
plot2plane
plot2sphere
plot2sky
plot2time
plot2d
plot3d
plothist
regquery
server
sqlclient
sqlskymatch
sqlupdate
taplint
tapquery
tapresume
tapskymatch
tcat
tcatn
tcopy
tcube
tgridmap
tgroup
tjoin
tloop
tmatch1
tmatch2
tmatchn
tmulti
tmultin
tpipe
tskymap
tskymatch2
votcopy
votlint
xsdvalidate
The <task-args>
part of the command line is a
list of parameter assignments,
each giving the value of one of the named parameters belonging to
the task which is specified in the <task-name>
part.
The general form of each parameter assignment is
<param-name>=<param-value>If you want to set the parameter to the null value, which is legal for some but not all parameters, use the special string "
null
",
or just leave the value blank ("<param-name>=
").
In some cases you can optionally leave out the <param-name>
part of the assignment (i.e. the parameter is positionally determined);
this is indicated in the task's usage description if the parameter
is described like [<param-name>=]<param-value>
rather than <param-name>=<param-value>
.
If the <param-value>
contains spaces or other special
characters, then in most cases, such as from the Unix shell, you will
have to quote it somehow. How this is done depends on your platform,
but usually surrounding the whole value in single quotes will do the trick.
Tasks may have many parameters, and you don't have to set all of them explicitly on the comand line. For a parameter which you don't set, two things can happen. In many cases, it will default to some sensible value. Sometimes however, you may be prompted for the value to use. In the latter case, a line like this will be written to the terminal:
matcher - Name of matching algorithm [sky]:This is prompting you for the value of the parameter named
matcher
. "Name of matching algorithm" is a short
description of what that parameter does. "sky
" is
the default value (if there is no default, no value will appear
in square brackets).
At this point you can do one of four things:
null
".null
" means the null value,
which is legal for some, but not all parameters.
If the value you enter is not legal, you will see an error
message and you will be invited to try again. help
" or a question mark "?
".
This will output a message
giving a detailed description of the parameter
and prompt you again.stilts
command itself (see Section 2.1).
If you supply the -prompt
flag, then you will be prompted
for every parameter you have not explicitly set. If you supply
-batch
on the other hand, you won't be prompted for
any parameters (and if you fail to set any without legal default
values, the task will fail).
If you want to see the actual values of the parameters for a task
as it runs,
including prompted values and defaulted ones
which you haven't specified explicitly,
you can use the -verbose
flag after the stilts
command:
% stilts -verbose tcopy cat.fits cat.vot ifmt=fits INFO: tcopy in=cat.fits out=cat.vot ifmt=fits ofmt=(auto)
If you make a parameter assignment on the command line for a
parameter which is not used by the task in question, STILTS
will issue an error message and the task will fail.
Note some parameters are only used dependent on the presence or
values of other parameters, so even supplying a parameter which is
documented in the task's usage can have this effect.
This is done on the assumption that if you have supplied a spurious
parameter it's probably a mistake and you should be given the
opportunity to correct it.
But if you want to be free to make these mistakes without the
task failing, you can supply the -allowunused
flag
as described in Section 2.1, in which case they will
just result in a warning.
Note that when running STILTS from the shell, it may be necessary to quote some parameter values, in case they contain spaces or other characters which the shell may try to interpret. This can typically be done by writing assignments of the form
<param-name>='<param-value>'but things can get more hairy; see Section 10.6 for more detail.
Extensive help is available from stilts
itself about task and its parameters, as described in the next section.
As well as the command descriptions in this document (especially the reference section Appendix B) you can get help for STILTS usage from the command itself. Typing
stilts -helpresults in this output:
Usage: stilts [-help] [-version] [-verbose] [-allowunused] [-prompt] [-bench] [-debug] [-batch] [-memory] [-disk] [-memgui] [-checkversion <vers>] [-stdout <file>] [-stderr <file>] <task-name> <task-args> stilts <task-name> help[=<param-name>|*] Known tasks: arrayjoin calc cdsskymatch cone coneskymatch datalinklint funcs pixfoot pixsample plot2d plot3d plothist regquery server sqlclient sqlskymatch sqlupdate taplint tapquery tapresume tapskymatch tcat tcatn tcopy tcube tgridmap tgroup tjoin tloop tmatch1 tmatch2 tmatchn tmulti tmultin tpipe tskymap tskymatch2 votcopy votlint xsdvalidate plot2plane plot2sky plot2cube plot2sphere plot2corner plot2time
For help on the individual tasks, including their parameter lists,
you can supply the word help
after the task name, so for instance
stilts tcopy helpresults in
Usage: tcopy ifmt=<in-format> ofmt=<out-format> [in=]<table> [out=]<out-table>
Finally, you can get help on any of the parameters of a task
by writing help=<param-name>
, like this:
stilts tcopy help=ingives
Help for parameter IN in task TCOPY ----------------------------------- Name: in Usage: [in=]<table> Summary: Location of input table Description: The location of the input table. This may take one of the following forms: * A filename. * A URL. * The special value "-", meaning standard input. In this case the input format must be given explicitly using the ifmt parameter. Note that not all formats can be streamed in this way. * A scheme specification of the form :<scheme-name>:<scheme-args>. * A system command line with either a "<" character at the start, or a "|" character at the end ("<syscmd" or "syscmd|"). This executes the given pipeline and reads from its standard output. This will probably only work on unix-like systems. In any case, compressed data in one of the supported compression formats (gzip, Unix compress or bzip2) will be decompressed transparently. Type: uk.ac.starlink.table.StarTableIf you use "
*
" instead of a parameter name in this usage,
help for all the parameters will be printed. Note that in most shells
you will probably need to quote the asterisk, so you should write
stilts tcopy help='*'
In some cases, as described in Section 2.3, you will be prompted for the value of a parameter with a line something like this:
matcher - Name of matching algorithm [sky]:In this case, if you enter "
help
" or a question mark,
then the parameter help entry will be printed to the screen, and
the prompt will be repeated.
For more detailed descriptions of the tasks, which includes explanatory comments and examples as well as the information above, see the full task descriptions in the Command Reference.
There are a number of ways of invoking commands
in the stilts
application,
depending on how you have installed the package.
This section describes how to invoke it from the command line.
Other options are using it from Jython (the Java implementation of
the Python language) as described in Section 4,
invoking it over HTTP as described in Section 11,
and invoking it from within a Java application
as described in Section 12.
If you're using a Unix-like operating system,
the easiest way is to use the stilts
script.
It is a simple shell script which just invokes java with the
right classpath and the supplied arguments.
If you have a full starjava installation the stilts
script
is in the starjava/bin
directory.
Otherwise you can download it separately from wherever you got your
STILTS installation in the first place, or find it
at the top of the stilts.jar
or topcat-*.jar
file
that contains your STILTS installation, so do something like
unzip stilts.jar stilts chmod +x stiltsto extract it (if you don't have
unzip
,
try jar xvf stilts.jar stilts
).
If you have mounted the topcat-all.dmg
file on MacOS
(hdiutil attach topcat-all.dmg
) it will probably be
present at a location like
/Volumes/topcat/TOPCAT.app/Contents/Resources/app/stilts
.
To run using the stilts
script, first make sure that
both the java
executable and the stilts
script itself are on your path,
and that the stilts.jar
or topcat-*.jar
jar file is in the same directory as stilts
.
Then the form of invocation is:
stilts <java-flags> <stilts-flags> <task-name> <task-args>A simple example would be:
stilts votcopy format=binary t1.xml t2.xmlin this case, as often, there are no
<java-flags>
or
<stilts-flags>
.
If you use the -classpath
argument or have a CLASSPATH environment variable set,
then classpath elements thus specified will be added to the classpath
required to run the command.
The examples in the
command descriptions below use this form for convenience.
If you don't have a Unix-like shell available however,
you will need to invoke
Java directly with the appropriate classes on your classpath.
If you have the file stilts.jar
, in most cases you can
just write:
java <java-flags> -jar stilts.jar <stilts-flags> <task-name> <task-args>which in practice would look something like
java -jar /some/where/stilts.jar votcopy format=binary t1.xml t2.xml
In the most general case, Java's -jar
flag might be
no good, for one of the following reasons:
stilts.jar
file (such as topcat-full.jar
)java <java-flags> -classpath <class-path> uk.ac.starlink.ttools.Stilts <stilts-flags> <task-name> <task-args>The example above in this case would look something like:
java -classpath /some/where/topcat-full.jar uk.ac.starlink.ttools.Stilts votcopy format=binary t1.xml t2.xml
Finally, as a convenience, it is possible to run STILTS from a
TOPCAT
installation by using its -stilts
flag, like this:
topcat <java-flags> -stilts <stilts-flags> <task-name> <task-args>This is possible because TOPCAT is built on top of STILTS, so contains a superset of its code.
The
<stilts-flags>
,
<task-name>
and
<task-args>
parts of these invocations are explained in Section 2,
and the
<class-path>
and
<java-flags>
parts are explained in the following subsections.
The classpath is the list of places that Java looks to find
the bits of compiled code that it uses to run an application.
Depending on how you have done your installation the core STILTS
classes could be in various places, but they are probably in a
file with one of the names
stilts.jar
,
topcat-full.jar
or
topcat-extra.jar
.
The full pathname of one of these files can therefore be used as
your classpath. In some cases these files are self-contained and
in some cases they reference other jar files in the filesystem -
this means that they may or may not continue to work if you
move them from their original location.
Under certain circumstances the tools might need additional classes, for instance:
In most cases it is not necessary to specify any additional arguments to the Java runtime, but it can be useful in certain circumstances. The two main kinds of options you might want to specify directly to Java are these:
-Dname=value
.
So for instance to ensure that temporary files are written to
the /home/scratch
directory, you could use the flag
-Djava.io.tmpdir=/home/scratch
-Xmx
flag. To set the heap
memory size to 256 megabytes, use the flag
-Xmx256M(don't forget the 'M' for megabyte). You will probably find performance is dreadful if you specify a heap size larger than the physical memory of the machine you're running on.
You can specify other options to Java such as tuning and profiling flags etc, but if you want to do that sort of thing you probably don't need me to tell you about it.
If you are using the stilts
command-line script,
any flags to it starting -D
or -X
are passed
directly to the java
executable.
You can pass other flags to Java with the stilts
script's
-J
flag; for instance:
stilts -Xmx4M -J-verbose:gc calc 'mjdToIso(0)'is equivalent to
java -Xmx4M -verbose:gc -jar stilts.jar calc 'mjdToIso(0)'
System properties are a way of getting information into the
Java runtime - they are a bit like environment variables.
There are two ways to set them when using STILTS: either
on the command line using arguments of the form
-Dname=value
(see Section 3.2)
or in a file in your home directory named
.starjava.properties
, in the form of a
name=value
line.
Thus submitting the flag
-Dvotable.strict=falseon the command line is equivalent to having the following in your
.starjava.properties
file:
# Force strict interpretation of the VOTable standard. votable.strict=false
The following system properties have special significance to STILTS:
http.proxyHost
java.awt.headless
true
" if running the
plotting tasks on a headless server.
You only need to worry about this if you see error messages
complaining about headlessness.
java.io.tmpdir
/tmp
on Unix),
so if working with large unmapped (e.g. CSV) tables
on a machine with limited space on the default disk,
it may be necessary to change it.
java.util.concurrent.ForkJoinPool.common.parallelism
jdbc.drivers
jel.classes
mark.workaround
mark()
/reset()
methods of some java
InputStream
classes. These are rather common,
including in Sun's J2SE system libraries.
Use this if you are seeing errors that say something like
"Resetting to invalid mark
".
Currently defaults to "false".service.maxparallel
auth.username
auth.password
@<filename>
",
in which case the value is read from the first line of the named file.
This replaces the normal behaviour of asking for a username and
password on the console; see the
section on Authentication
for more details.
Since this setting will pass the username and password information to any
protected resource without checking it is the intended destination,
this can potentially leak secret information to third parties,
so these properties should be set with care.
auth.schemes
AuthScheme
implementation classnames may be provided.
startable.readers
uk.ac.starlink.table.TableBuilder
interface,
and must have a no-arg constructor.
The readers thus named will be available
alongside the standard ones listed in Section 5.1.1.
startable.schemes
uk.ac.starlink.table.TableScheme
interface,
and must have a no-arg constructor.
The schemes thus named will be available
alongside the standard ones listed in Section 5.3.
startable.storage
disk
" has basically the same effect as
supplying the "-disk
" argument on the command line
(see Section 2.1).
Other possible values are "adaptive
", "memory
",
"sideways
" and "discard
";
see SUN/252.
The default is "adaptive
", which means storing smaller
tables in memory, and larger ones on disk.
startable.unmap
sun
" (the default),
"cleaner
", "unsafe
" or "none
".
In most cases you are advised to leave this alone, but in the event of
unmapping-related JVM crashes (not expected!), setting it to
none
may help.
startable.writers
uk.ac.starlink.table.StarTableWriter
interface,
and must have a no-arg constructor.
The writers thus named will be available
alongside the standard ones listed in Section 5.1.2.
votable.namespacing
none
" (no namespacing, xmlns declarations
in VOTable document will probably confuse parser),
"lax
" (anything that looks like it is probably a VOTable
element will be treated as a VOTable element) and
"strict
" (VOTable elements must be properly declared in one
of the correct VOTable namespaces).
May also be set to the classname of a
uk.ac.starlink.votable.Namespacing
implementation.
The default is "lax
".
votable.strict
FIELD
or PARAM
element with
a datatype
attribute of
char
/unicodeChar
,
and no arraysize
attribute.
The VOTable standard says this indicates a single character,
but some VOTables omit arraysize specification by accident when
they intend arraysize="*"
.
If votable.strict
is set true
,
a missing arraysize will be interpreted as meaning a single character,
and if false
, it will be interpreted as a variable-length
array of characters (a string).
The default is true
.
votable.version
1.0
", "1.1
",
"1.2
", "1.3
" or "1.4
".
By default, version 1.4 VOTables are written.
This section describes additional configuration which must be done to allow the commands to access SQL-compatible relational databases for reading or writing tables. If you don't need to talk to SQL-type databases, you can ignore the rest of this section. The steps described here are the standard ones for configuring JDBC (which sort-of stands for Java Database Connectivity); you can find more information on that on the web. The best place to look may be within the documentation of the RDBMS you are using.
To use STILTS with SQL-compatible databases you must:
jdbc.drivers
system property to the name of the
driver class as described in Section 3.3
Here is an example of using tpipe
to write the results
of an SQL query on a table in a MySQL database as a VOTable:
stilts -classpath /usr/local/jars/mysql-connector-java.jar \ -Djdbc.drivers=com.mysql.jdbc.Driver \ tpipe \ in="jdbc:mysql://localhost/db1#SELECT id, ra, dec FROM gsc WHERE mag < 9" \ ofmt=votable gsc.votor invoking Java directly:
java -classpath stilts.jar:/usr/local/jars/mysql-connect-java.jar \ -Djdbc.drivers=com.mysql.jdbc.Driver \ uk.ac.starlink.ttools.Stilts tpipe \ in="jdbc:mysql://localhost/db1#SELECT id, ra, dec FROM gsc WHERE mag < 9" \ ofmt=votable out=gsc.votYou have to exercise some care to get the arguments in the right order here - see Section 3.
Alternatively, you can set some of this up beforehand to make the invocation easier. If you set your CLASSPATH environment variable to include the driver jar file (and the STILTS classes if you're invoking Java directly rather than using the scripts), and if you put the line
jdbc.drivers=com.mysql.jdbc.Driverin the
.starjava.properties
file in your home directory,
then you could avoid having to give the -classpath
and
-Djdbc.drivers
flags respectively.
Below are presented the results of some experiments with JDBC drivers. Note however that this information may be be incomplete and out of date. If you have updates, feel free to pass them on and they may be incorporated here.
To the author's knowledge, STILTS has successfully been used with the following RDBMSs and corresponding JDBC drivers:
useUnicode=true&characterEncoding=UTF8
" may be required
to handle some non-ASCII characters.
jdbc:sybase:Tds:hostname:port/dbname?user=XXX&password=XXX#SELECT...
".
An earlier attempt using Sybase ASE 11.9.2 failed.
Here are some example command lines that at least have at some point got STILTS running with databases:
stilts -classpath pg73jdbc3.jar \ -Djdbc.drivers=org.postgresql.Driver ...
stilts -classpath mysql-connector-java-3.0.8-bin.jar \ -Djdbc.drivers=com.mysql.jdbc.Driver ...
stilts -classpath ojdbc14.jar \ -Djdbc.drivers=oracle.jdbc.driver.OracleDriver ...
stilts -classpath jtds-1.1.jar \ -Djdbc.drivers=net.sourceforge.jtds.jdbc.Driver ...
Most of the discussions and examples in this document describe using STILTS as a standalone java application from the command line; in this case, scripting can be achieved by executing one STILTS command, followed by another, followed by another, perhaps controlled from a shell script, with intermediate results stored in files.
However, it is also possible to invoke STILTS commands from within the Jython environment. Jython is a pure-java implementation of the widely-used Python scripting language. Using Jython is almost exactly the same as using the more usual C-based Python, except that it is not possible to use extensions which use C code. This means that if you are familiar with Python programming, it is very easy to string STILTS commands together in Jython.
This approach has several advantages over the conventional command-line usage:
Usage from jython has syntax which is similar to command-line STILTS, but with a few changes. The following functions are defined by JyStilts:
tread
, which reads a table from a
file or URL and turns it into a table object in jython
write
which takes a table object and
writes it to file
cmd_head
, cmd_select
,
cmd_addcol
)
mode_out
, mode_meta
,
mode_samp
),
tmatch2
, tcat
, plot2sky
)
help
" command, however
for full documentation and examples you should refer to this document.
In JyStilts the input, processing, filtering and output are done in separate steps, unlike in command-line STILTS where they all have to be combined into a single line. This can make the flow of execution easier to follow. A typical sequence will involve:
tread
functioncmd_*
filter methodscmd_*
filter methodsmode_*
output modeswrite
methodHere is an example command line invocation for crossmatching two tables:
stilts tskymatch2 in1=survey.fits \ icmd1='addskycoords fk4 fk5 RA1950 DEC1950 RA2000 DEC2000' \ in2=mycat.csv ifmt2=csv \ icmd2='select VMAG>18' \ ra1=ALPHA dec1=DELTA ra2=RA2000 dec2=DEC2000 \ error=10 join=2not1 \ out=matched.fitsand here is what it might look like in JyStilts:
>>> import stilts >>> t1 = stilts.tread('survey.fits') >>> t1 = t1.cmd_addskycoords(t1, 'fk4', 'fk5', 'RA1950', 'DEC1950', 'RA2000', 'DEC2000') >>> t2 = stilts.tread('mycat.csv', 'csv') >>> t2 = t2.cmd_select('VMAG>18') >>> tm = stilts.tskymatch2(in1=t1, in2=t2, ra1='ALPHA', dec1='DELTA', ... error=10, join='2not1') >>> tm.write('matched.fits')
When running interactively, it can be convenient to examine the intermediate results before processing or writing as well, for instance:
>>> tm.mode_count() columns: 19 rows: 2102 >>> tm.cmd_keepcols('ID ALPHA DELTA').cmd_head(4).write() +--------+---------------+-----------+ | ID | ALPHA | DELTA | +--------+---------------+-----------+ | 262 | 149.82439 | -0.11249 | | 263 | 150.14438 | -0.11785 | | 265 | 149.92944 | -0.11667 | | 273 | 149.93185 | -0.12566 | +--------+---------------+-----------+
More detail about how to run JyStilts and its usage is given in the following subsections.
The easiest way to run JyStilts is to download the standalone
jystilts.jar
file from the STILTS web page,
and simply run
java -jar jystilts.jarThis file includes jython itself and all the STILTS and JyStilts classes. To use the JyStilts commands, you will need to import the stilts module using a line like "
import stilts
"
from Jython in the usual Python way.
Alternatively, you can run JyStilts from an existing Jython installation
using just the stilts.jar
file.
First, make sure that Jython is installed;
it is available from http://www.jython.org/,
and comes as a self-installing jar file.
JyStilts has been tested, and appears to work, with jython version 2.7.2.
It also works with jython 2.5.* under Java 8 and Java 11,
but jystilts with jython 2.5.* and Java 17 can fail with security problems.
Some earlier versions of JyStilts worked with jython 2.2.1,
but that no longer seems to be the case; it might be possible
to reinstate this if there is some pressing need.
To use JyStilts, you then just need to
start jython with the stilts.jar
file on your classpath,
for instance like this:
jython -J-classpath /some/where/stilts.jaror (C-shell):
setenv CLASSPATH /some/where/stilts.jar jython
Optionally, you can extract the stilts.py
module
from the stilts.jar file
(using a command like "unzip stilts.jar stilts.py
")
and put it in a directory on your jython
sys.path
(e.g. jythondir/Lib
);
this may cause jython to compile it to bytecode (stilts$py.class
)
and thus improve startup time.
Note that in this case you will still need the stilts.jar
file on your classpath as above.
The tread
function reads tables from an external location
into JyStilts. Its arguments are as follows:
tread(location, fmt='(auto)', random=False)and its return value is a table object, which can be interrogated directly, or used in other JyStilts commands. Usually, the location argument should be a string which gives the filename or URL at which a table can be found. You can alternatively use a readable python file (or file-like) object for the location, but be aware that this may be less efficient on memory. As with command-line STILTS, the
fmt
argument
is one of the options in Section 5.1.1, but may be
left as the default if the format auto-detectable,
which currently means if the file is in
VOTable, FITS, CDF, ECSV, PDS4, Parquet, Feather or GBIN format.
The random
argument can be used to ensure that the returned file
has random (i.e. not sequential-only) access;
for some table formats the default way of reading them in means that
their rows can only be accessed in sequence.
Depending on what processing you are doing, that may or may not be
satisfactory.
Examples of reading a table are:
>>> import stilts >>> t1 = stilts.tread('cat.fits') >>> t2 = stilts.tread(open('cat.fits', 'rb')) # less efficient >>> t3 = stilts.tread('data.csv', fmt='csv', random=True)
The most straightforward way to write a table
(presumably the result of one or a sequence of JyStilts commands)
is using the write
table method:
write(self, location=None, fmt='(auto)')The
location
gives either a string which is a filename,
or a writable python file (or file-like) object.
Again, use of a filename is preferred as it may(?) be more efficient.
If no location is supplied, the table will be written to standard output
(useful for inspection, but a bad idea for binary formats or very large tables).
The fmt
argument is one of the output formats in
Section 5.1.2, but may be left as the default if the
format can be guessed from the filename.
Examples of writing a table are:
>>> table.write('out.fits') >>> table.write(open('out.fits', 'wb')) # less efficient? >>> table.write('catalogue.dat', fmt='csv') >>> table.write() # display to stdout
Often it's convenient to combine examining the table with filtering steps, for instance:
>>> table.every(100).write()would write only every hundredth row, and
>>> (table.cmd_sorthead(10, 'BMAG') ... .cmd_select('!NULL_VMAG') ... .cmd_keepcols('BMAG VMAG') ... .write())would write only the BMAG and VMAG columns for the ten rows in which VMAG is non-null with the lowest BMAG values.
You can also read and write multiple tables, if you use a table
format for which that is appropriate.
This generally means FITS (which can store tables in multiple extensions)
or VOTable (which can store multiple TABLE elements in one document).
This is done using the treads
and twrites
functions.
The functions look like this:
treads(location, fmt='(auto)', random=False) twrites(tables, location=None, fmt='(auto)')These are similar to the
tread
and twrite
functions,
except that treads
returns a list of tables rather than
a single table, and twrites
's tables
argument is
an iterable over tables rather than a single table.
Here is an example of reading multiple tables from a multi-extension FITS
file, counting the rows in each, and then writing them out to a multi-TABLE
VOTable file:
import stilts tables = stilts.treads('multi.fits') print([t.getRowCount() for t in tables]) stilts.twrites(tables, 'multi.vot', fmt='votable')
The tables read by the tread
function and produced
by operating on them within JyStilts have a number of methods
defined on them.
These are explained below.
First, a number of special methods are defined which allow a table to behave in python like a sequence of rows:
__iter__
for row in table:
" will iterate over all rows.
__len__
(random-access tables only)
len(table)
" to count the number of rows.
This method is not available for tables with sequential access only.
__getitem__
(random-access tables only)
table[3]
" or table[0:10]
to obtain the
row or rows corresponding to a given row index or slice.
This method is not available for tables with sequential access only.
__add__
, __mul__
, __rmul__
+
" and and "*
" to be used with
the sense of concatenation.
Thus "table1+table2
" will produce a new table with the
rows of table1
followed by the rows of table2
.
Note this will only work if both tables have compatible columns.
Similarly "table*3
" would produce a table like
table
but with all its rows repeated three times.
columns()
.
Sometimes, the result of a table operation will be a table which
does not have random access. For such tables you can iterate over
the rows, but not get their row values by indexing.
Non-random-access tables are also peculiar in that getRowCount
returns a negative value.
To take a table which may not have random access and make it capable
of random access, use the random
filter: "table=table.cmd_random()
".
To a large extent it is possible to duplicate the functions of the
various STILTS commands by writing your own python code based on these
python-friendly table access methods.
Note however that such python-based processing is likely to be
much slower than the STILTS equivalents.
If performance is important to you, you should try in most cases
to use the various cmd_*
commands etc for table processing.
Second, some additional utility methods are defined:
count_rows()
columns()
getName()
,
getUnitString()
, getUCD()
.
str(column)
will return its name.
coldata(key)
key
argument may be either an integer
column index (if negative, counts backwards from the end),
or the column name or info object.
The returned value will always be iterable (has __iter__
),
but will only be indexable
(has __len__
and __getitem__
) if the table
is random access.
parameters()
StarTable
methods.
Note that as currently implemented, changing the values in the
returned mapping will not change the actual table parameter values.
write(location=None, fmt=None)
location
argument gives a filename
or writable file object,
and the optional fmt
argument gives a format, one of
the options listed in Section 5.1.1.
If location
is not supplied, output is to standard output,
so in an interactive session it will be printed to the terminal.
If fmt
is not supplied, an attempt will be made to guess
a suitable format based on the location.
Third, a set of cmd_*
methods corresponding to the
STILTS filters are available;
these are described in Section 4.4.
Fourth, a set of mode_*
methods corresponding to the
STILTS output modes are available;
these are described in Section 4.5.
Finally, tables are also instances of the StarTable interface defined by STIL, which is the table I/O layer underlying STILTS. The full documentation can be found in the user manual and javadocs on the STIL page, and all the java methods can be used from JyStilts, but in most cases there are more pythonic equivalents provided, as described above.
Here are some examples of these methods in use:
>>> import stilts >>> xsc = stilts.tread('/data/table/2mass_xsc.xml') # read table >>> xsc.mode_count() # show rows/column count columns: 6 rows: 1646844 >>> print xsc.columns() # full info on columns (id(String), ra(Double)/degrees, dec(Double)/degrees, jmag(Double)/mag, hmag(Double)/mag, kmag(Double)/mag) >>> print [str(col) for col in xsc.columns()] # column names only ['id', 'ra', 'dec', 'jmag', 'hmag', 'kmag'] >>> row = xsc[1000000] # examine millionth row >>> print row (u'19433000+4003190', 295.875, 40.055286, 14.449, 13.906, 13.374) >>> print row[0] # cell by index 19433000+4003190 >>> print row['ra'], row['dec'] # cells by col name 295.875 40.055286 >>> print len(xsc) # count rows, maybe slow 1646844 >>> print xsc.count_rows() # count rows efficiently 1646844L >>> print (xsc+xsc).count_rows() # concatenate 3293688L >>> print (xsc*10000).count_rows() 16468440000L >>> for row in xsc: # select rows using python commands ... if row[4] - row[3] > 3.0: ... print row[0] ... 11165243+2925509 20491597+5119089 04330238+0858101 01182715-1013248 11244075+5218078 >>> # same thing using stilts (50x faster) >>> (xsc.cmd_select('hmag - jmag > 3.0') ... .cmd_keepcols('id') ... .write()) +------------------+ | id | +------------------+ | 11165243+2925509 | | 20491597+5119089 | | 04330238+0858101 | | 01182715-1013248 | | 11244075+5218078 | +------------------+
The following are all ways to obtain the value of a given cell in the table from the previous example.
xsc.getCell(99, 0) xsc[99][0] xsc[99]['id'] xsc.coldata(0)[99] xsc.coldata('id')[99]Some of these methods may be more efficient than others. Note that none of these methods will work if the table has sequential-only access.
cmd_*
)
The STILTS table filters documented in Section 6.1
are available in JyStilts as table methods
which start with the "cmd_
" prefix.
The return value when calling the method on a table object is
another table object.
The arguments, which are the same as those required for the command-line
version, are supplied as a list of unnamed arguments of the
cmd_*
function. In general the arguments are strings,
but numbers are accepted where appropriate.
Use the python help
command to see the usage of each method.
So, to use the tail
filter to
select only the last ten lines of a table, you can write:
table.cmd_tail(10)To set units of "Hz" for some columns using the
colmeta
filter write:
table.cmd_colmeta('-units', 'Hz', 'AFREQ BFREQ CFREQ')Note that where a filter argument is a space-separated list it must appear as a single argument in the filter invocation, just as in command-line STILTS.
The filter commands are also available as module functions. This means that
stilts.cmd_head(table, 10)and
table.cmd_head(10)have exactly the same meaning. It's a matter of taste which you prefer.
mode_*
)
The STILTS table output modes documented in Section 6.4
are available in JyStilts as table methods
which start with the "mode_
" prefix.
These methods have no return value, but cause something to happen,
in some cases output to be written to standard output.
Some of these methods have named arguments, others have no arguments.
Use the python help
command to see the usage of each method.
These methods are straightforward to use. The following example calculates statistics for a table and writes the results to standard output:
>>> table.mode_stats()and this one attempts to send the table via the SAMP communications protocol to a running instance of TOPCAT:
>>> table.mode_samp(client='topcat')
The output modes are also available as module functions. This means that
stilts.mode_samp(table, client='topcat')and
table.mode_samp(client='topcat)have exactly the same meaning. It's a matter of taste which you prefer.
The STILTS tasks documented in Appendix B
can be used under their usual names if they are imported from the
stilts
module.
STILTS parameters as are supplied as named arguments of the python
functions. In general they are either table objects for table input
parameters or strings, but in some cases python arrays are accepted,
and numbers may be used where appropriate.
The STILTS input format (ifmt
, istream
),
filter (cmd
/icmd
/ocmd
)
and output mode (omode
) parameters are not used however;
instead perform filtering directly on the table inputs and outputs
using the python cmd_*
and mode_*
table methods or functions.
Here is an example of concatenating two similar tables together and writing the result:
>>> from stilts import tread, tcat >>> t1 = tread('data1.csv', fmt='csv') >>> t2 = tread('data2.csv', fmt='csv') >>> t12 = tcat([t1,t2], seqcol='seq') >>> t12.write('t12.csv', fmt='csv')
Note that for those tasks which have a parameter named "in
"
in command-line STILTS, it has been renamed as "in_
" for
the python version, to avoid a name clash with the python reserved word.
In most cases, the in
parameter is the first, mandatory
parameter in any case, and so can be referenced by position as in the
previous example (we could have written "tcat(in_=[t1,t2])
"
instead).
The various functions from the expression language
listed in Section 10.7 are available directly from JyStilts.
Each of the subsections in that section is a class in the stilts
module namespace, with unbound functions representing the functions.
This means you can use them like this:
>>> import stilts >>> print stilts.Times.mjdToIso(54292) 2007-07-11T00:00:00or like this:
>>> from stilts import CoordsDegrees >>> dist = CoordsDegrees.skyDistanceDegrees(ra1, dec1, ra2, dec2)
Most of the tools in this package either read one or more tables as input, or write one or more tables as output, or both. This section explains what kind of tables the tools can read and write, and how you tell them where to find the tables to operate on.
In most cases input and output table specifications are given by parameters with the following names (or similar ones):
in
ifmt
out
-
"/blank for standard output
ofmt
The generic table commands in STILTS
(currently tpipe
,
tcopy
,
tmulti
,
tmultin
,
tcat
,
tcatn
,
tloop
,
tjoin
,
tgridmap
,
tgroup
,
tcube
,
tmatch1
,
tmatch2
,
tmatchn
,
tskymap
,
tskymatch2
,
pixfoot
,
pixsample
,
plot2corner
,
plot2cube
,
plot2plane
,
plot2sky
,
plot2sphere
,
plot2time
,
plot2d
,
plot3d
,
plothist
,
cdsskymatch
,
cone
,
coneskymatch
,
sqlskymatch
,
tapquery
,
tapresume
,
tapskymatch
and
regquery
)
have no native format for table storage, they can process
data in a number of formats equally well.
STIL has its own model of what a table
consists of, which is basically:
The formats the package knows about are dependent on the input and output handlers currently installed. The ones installed by default are listed in the following subsections. More may be added in the future, and it is possible to install new ones at runtime - see the STIL documentation for details.
Some formats can be used to hold multiple tables in a single file, and others can only hold a single table per file.
Some of the tools in this package ask you to specify the format
of input tables using the ifmt
(or a similarly named)
parameter.
For some file formats (e.g. FITS, VOTable, CDF),
the format can be automatically determined by
looking at the file content, regardless of filename;
for others (e.g. CSV files with a ".csv
" extension),
STILTS may be able to use the filename as a hint to guess the format
(the details of these rules are given in the format-specific
subsections below).
Otherwise, you have to supply the format using the ifmt
parameter.
It is always safe to specify the format explicitly;
this will be slightly more efficient than auto-determination,
and may lead to more helpful error messages in the case that the
table can't be read correctly.
The available input formats are described in the following subsections.
It is also possible to add new formats at runtime using the
startable.readers
system property,
or by setting the format to the classname of a
uk.ac.starlink.table.TableBuilder
class.
fits
FITS is a very well-established format for storage of
astronomical table or image data
(see https://fits.gsfc.nasa.gov/).
This reader can read tables stored in
binary (XTENSION='BINTABLE'
) and
ASCII (XTENSION='TABLE'
) table extensions;
any image data is ignored.
Currently, binary table extensions are read much more efficiently
than ASCII ones.
When a table is stored in a BINTABLE extension in an uncompressed FITS file on disk, the table is 'mapped' into memory; this generally means very fast loading and low memory usage. FITS tables are thus usually efficient to use.
Limited support is provided for the semi-standard HEALPix-FITS convention; such information about HEALPix level and coordinate system is read and made available for application usage and user examination.
A private convention is used to support encoding of tables with more than 999 columns (not possible in standard FITS); see Section 5.1.3.2.
Header cards in the table's HDU header will be made available as table parameters. Only header cards which are not used to specify the table format itself are visible as parameters (e.g. NAXIS, TTYPE* etc cards are not). HISTORY and COMMENT cards are run together as one multi-line value.
Any 64-bit integer column with a non-zero integer offset
(TFORMn='K'
, TSCALn=1
, TZEROn<>0
)
is represented in the read table as Strings giving the decimal integer value,
since no numeric type in Java is capable of representing the whole range of
possible inputs. Such columns are most commonly seen representing
unsigned long values.
Where a multi-extension FITS file contains more than one table, a single table may be specified using the position indicator, which may take one of the following forms:
spec23.fits.gz
"
with one primary HDU and two BINTABLE extensions,
you would view the first one using the name "spec23.fits.gz
"
or "spec23.fits.gz#1
"
and the second one using the name "spec23.fits.gz#2
".
The suffix "#0
" is never used for a legal
FITS file, since the primary HDU cannot contain a table.
EXTNAME
header in the HDU,
or alternatively the value of EXTNAME
followed by "-
" followed by the value of EXTVER
.
This follows the recommendation in
the FITS standard that EXTNAME
and EXTVER
headers can be used to identify an HDU.
So in a multi-extension FITS file "cat.fits
"
where a table extension
has EXTNAME='UV_DATA'
and EXTVER=3
,
it could be referenced as
"cat.fits#UV_DATA
" or "cat.fits#UV_DATA-3
".
Matching of these names is case-insensitive.
Files in this format may contain multiple tables;
depending on the context, either one or all tables
will be read.
Where only one table is required,
either the first one in the file is used,
or the required one can be specified after the
"#
" character at the end of the filename.
This format can be automatically identified by its content so you do not need to specify the format explicitly when reading FITS tables, regardless of the filename.
There are actually two FITS input handlers,
fits-basic
and fits-plus
.
The fits-basic
handler extracts standard column metadata
from FITS headers of the HDU in which the table is found,
while the fits-plus
handler reads column and table metadata
from VOTable content stored in the primary HDU of the multi-extension
FITS file.
FITS-plus is a private convention effectively defined by the
corresponding output handler; it allows de/serialization of
much richer metadata than can be stored in standard FITS headers
when the FITS file is read by fits-plus-aware readers,
though other readers can understand the unenhanced FITS file perfectly well.
It is normally not necessary to worry about this distinction;
STILTS will determine whether a FITS file is FITS-plus or not based on its
content and use the appropriate handler, but if you want to force the
reader to use or ignore the enriched header, you can explicitly specify
an input format of "fits-plus
" or "fits-basic
".
The details of the FITS-plus convention are described in Section 5.1.3.1.
colfits
As well as normal binary and ASCII FITS tables, STIL supports
FITS files which contain tabular data stored in column-oriented format.
This means that the table is stored in a BINTABLE extension HDU,
but that BINTABLE has a single row, with each cell of that row
holding a whole column's worth of data. The final (slowest-varying)
dimension of each of these cells (declared via the TDIMn
headers)
is the same for every column, namely,
the number of rows in the table that is represented.
The point of this is that all the cells for each column are stored
contiguously, which for very large, and especially very wide tables
means that certain access patterns (basically, ones which access
only a small proportion of the columns in a table) can be much more
efficient since they require less I/O overhead in reading data blocks.
Such tables are perfectly legal FITS files, but general-purpose FITS software may not recognise them as multi-row tables in the usual way. This format is mostly intended for the case where you have a large table in some other format (possibly the result of an SQL query) and you wish to cache it in a way which can be read efficiently by a STIL-based application.
For performance reasons, it is advisable to access colfits files uncompressed on disk. Reading them from a remote URL, or in gzipped form, may be rather slow (in earlier versions it was not supported at all).
This format can be automatically identified by its content so you do not need to specify the format explicitly when reading colfits-basic tables, regardless of the filename.
Like the normal (row-oriented) FITS handler,
two variants are supported:
with (colfits-plus
) or without (colfits-basic
)
metadata stored as a VOTable byte array in the primary HDU.
For details of the FITS-plus convention, see Section 5.1.3.1.
votable
VOTable is an XML-based format for tabular data endorsed by the International Virtual Observatory Alliance; while the tabular data which can be encoded is by design close to what FITS allows, it provides for much richer encoding of structure and metadata. Most of the table data exchanged by VO services is in VOTable format, and it can be used for local table storage as well.
Any table which conforms to the VOTable 1.0, 1.1, 1.2, 1.3 or 1.4 specifications can be read. This includes all the defined cell data serializations; cell data may be included in-line as XML elements (TABLEDATA serialization), included/referenced as a FITS table (FITS serialization), or included/referenced as a raw binary stream (BINARY or BINARY2 serialization). The handler does not attempt to be fussy about input VOTable documents, and it will have a good go at reading VOTables which violate the standards in various ways.
Much, but not all, of the metadata contained in a VOTable
document is retained when the table is read in.
The attributes
unit
, ucd
, xtype
and utype
,
and the elements
COOSYS
, TIMESYS
and DESCRIPTION
attached to table columns or parameters,
are read and may be used by the application as appropriate
or examined by the user.
However, information encoded in the hierarchical structure
of the VOTable document, including GROUP
structure, is not
currently retained when a VOTable is read.
VOTable documents may contain more than one actual table
(TABLE
element).
To specify a specific single table,
the table position indicator is given by the
zero-based index of the TABLE
element in a breadth-first search.
Here is an example VOTable document:
<VOTABLE> <RESOURCE> <TABLE name="Star Catalogue"> ... </TABLE> <TABLE name="Galaxy Catalogue"> ... </TABLE> </RESOURCE> </VOTABLE>If this is available in a file named "cats.xml" then the two tables could be named as "cats.xml#0" and "cats.xml#1" respectively.
Files in this format may contain multiple tables;
depending on the context, either one or all tables
will be read.
Where only one table is required,
either the first one in the file is used,
or the required one can be specified after the
"#
" character at the end of the filename.
This format can be automatically identified by its content so you do not need to specify the format explicitly when reading VOTable tables, regardless of the filename.
cdf
NASA's Common Data Format, described at https://cdf.gsfc.nasa.gov/, is a binary format for storing self-described data. It is typically used to store tabular data for subject areas like space and solar physics.
This format can be automatically identified by its content so you do not need to specify the format explicitly when reading CDF tables, regardless of the filename.
csv
Comma-separated value ("CSV") format is a common semi-standard text-based format in which fields are delimited by commas. Spreadsheets and databases are often able to export data in some variant of it. The intention is to read tables in the version of the format spoken by MS Excel amongst other applications, though the documentation on which it was based was not obtained directly from Microsoft.
The rules for data which it understands are as follows:
#
" character
(or anything else) to introduce "comment" lines.
Because the CSV format contains no metadata beyond column names,
the handler is forced to guess the datatype of the values in each column.
It does this by reading the whole file through once and guessing
on the basis of what it has seen (though see the maxSample
configuration option). This has the disadvantages:
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"csv(header=true,maxSample=100000)
".
The following options are available:
header = true|false|null
true
: the first line is a header line containing column namesfalse
: all lines are data lines, and column names will be assigned automaticallynull
: a guess will be made about whether the first line is a header or not depending on what it looks likenull
(auto-determination).
This usually works OK, but can get into trouble if
all the columns look like string values.
(Default: null
)
maxSample = <int>
0
)
This format cannot be automatically identified
by its content, so in general it is necessary
to specify that a table is in
CSV
format when reading it.
However, if the input file has
the extension ".csv
" (case insensitive)
an attempt will be made to read it using this format.
An example looks like this:
RECNO,SPECIES,NAME,LEGS,HEIGHT,MAMMAL 1,pig,Pigling Bland,4,0.8,true 2,cow,Daisy,4,2.0,true 3,goldfish,Dobbin,,0.05,false 4,ant,,6,0.001,false 5,ant,,6,0.001,false 6,queen ant,Ma'am,6,0.002,false 7,human,Mark,2,1.8,true
See also ECSV as a format which is similar and capable of storing more metadata.
ecsv
The Enhanced Character Separated Values format was developed within the Astropy project and is described in Astropy APE6 (DOI). It is composed of a YAML header followed by a CSV-like body, and is intended to be a human-readable and maybe even human-writable format with rich metadata. Most of the useful per-column and per-table metadata is preserved when de/serializing to this format. The version supported by this reader is currently ECSV 1.0.
There are various ways to format the YAML header, but a simple example of an ECSV file looks like this:
# %ECSV 1.0 # --- # delimiter: ',' # datatype: [ # { name: index, datatype: int32 }, # { name: Species, datatype: string }, # { name: Name, datatype: string }, # { name: Legs, datatype: int32 }, # { name: Height, datatype: float64, unit: m }, # { name: Mammal, datatype: bool }, # ] index,Species,Name,Legs,Height,Mammal 1,pig,Bland,4,,True 2,cow,Daisy,4,2,True 3,goldfish,Dobbin,,0.05,False 4,ant,,6,0.001,False 5,ant,,6,0.001,False 6,human,Mark,2,1.9,TrueIf you follow this pattern, it's possible to write your own ECSV files by taking an existing CSV file and decorating it with a header that gives column datatypes, and possibly other metadata such as units. This allows you to force the datatype of given columns (the CSV reader guesses datatype based on content, but can get it wrong) and it can also be read much more efficiently than a CSV file and its format can be detected automatically.
The header information can be provided either in the ECSV file itself,
or alongside a plain CSV file from a separate source
referenced using the header
configuration option.
In Gaia EDR3 for instance, the ECSV headers are supplied alongside
the CSV files available for raw download of all tables in the
Gaia source catalogue, so e.g. STILTS can read
one of the gaia_source CSV files with full metadata
as follows:
stilts tpipe ifmt='ecsv(header=http://cdn.gea.esac.esa.int/Gaia/gedr3/ECSV_headers/gaia_source.header)' in=http://cdn.gea.esac.esa.int/Gaia/gedr3/gaia_source/GaiaSource_000000-003111.csv.gz
The ECSV datatypes that work well with this reader are
bool
,
int8
, int16
, int32
, int64
,
float32
, float64
and
string
.
Array-valued columns are also supported with some restrictions.
Following the ECSV 1.0 specification,
columns representing arrays of the supported datatypes can be read,
as columns with datatype: string
and a suitable
subtype
, e.g.
"int32[<dims>]
" or "float64[<dims>]
".
Fixed-length arrays (e.g. subtype: int32[3,10]
)
and 1-dimensional variable-length arrays
(e.g. subtype: float64[null]
) are supported;
however variable-length arrays with more than one dimension
(e.g. subtype: int32[4,null]
) cannot be represented,
and are read in as string values.
Null elements of array-valued cells are not supported;
they are read as NaNs for floating point data, and as zero/false for
integer/boolean data.
ECSV 1.0, required to work with array-valued columns,
is supported by Astropy v4.3 and later.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"ecsv(header=http://cdn.gea.esac.esa.int/Gaia/gedr3/ECSV_headers/gaia_source.header,colcheck=FAIL)
".
The following options are available:
header = <filename-or-url>
null
)
colcheck = IGNORE|WARN|FAIL
WARN
)
This format can be automatically identified by its content so you do not need to specify the format explicitly when reading ECSV tables, regardless of the filename.
ascii
In many cases tables are stored in some sort of unstructured plain text format, with cells separated by spaces or some other delimiters. There is a wide variety of such formats depending on what delimiters are used, how columns are identified, whether blank values are permitted and so on. It is impossible to cope with them all, but the ASCII handler attempts to make a good guess about how to interpret a given ASCII file as a table, which in many cases is successful. In particular, if you just have columns of numbers separated by something that looks like spaces, you should be just fine.
Here are the detailed rules for how the ASCII-format tables are interpreted:
null
" (unquoted) represents
the null valueBoolean
,
Short
Integer
,
Long
,
Float
,
Double
,
String
NaN
for not-a-number, which is treated the same as a
null value for most purposes, and Infinity
or inf
for infinity (with or without a preceding +/- sign).
These values are matched case-insensitively.If the list of rules above looks frightening, don't worry, in many cases it ought to make sense of a table without you having to read the small print. Here is an example of a suitable ASCII-format table:
# # Here is a list of some animals. # # RECNO SPECIES NAME LEGS HEIGHT/m 1 pig "Pigling Bland" 4 0.8 2 cow Daisy 4 2 3 goldfish Dobbin "" 0.05 4 ant "" 6 0.001 5 ant "" 6 0.001 6 ant '' 6 0.001 7 "queen ant" 'Ma\'am' 6 2e-3 8 human "Mark" 2 1.8In this case it will identify the following columns:
Name Type ---- ---- RECNO Short SPECIES String NAME String LEGS Short HEIGHT/m FloatIt will also use the text "
Here is a list of some animals
"
as the Description parameter of the table.
Without any of the comment lines, it would still interpret the table,
but the columns would be given the names col1
..col5
.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"ascii(maxSample=5000)
".
The following options are available:
maxSample = <int>
0
)
This format cannot be automatically identified
by its content, so in general it is necessary
to specify that a table is in
ASCII
format when reading it.
However, if the input file has
the extension ".txt
" (case insensitive)
an attempt will be made to read it using this format.
ipac
CalTech's Infrared Processing and Analysis Center use a text-based format for storage of tabular data, defined at http://irsa.ipac.caltech.edu/applications/DDGEN/Doc/ipac_tbl.html. Tables can store column name, type, units and null values, as well as table parameters.
This format cannot be automatically identified
by its content, so in general it is necessary
to specify that a table is in
IPAC
format when reading it.
However, if the input file has
the extension ".tbl
" or ".ipac
" (case insensitive)
an attempt will be made to read it using this format.
An example looks like this:
\Table name = "animals.vot" \Description = "Some animals" \Author = "Mark Taylor" | RECNO | SPECIES | NAME | LEGS | HEIGHT | MAMMAL | | int | char | char | int | double | char | | | | | | m | | | null | null | null | null | null | null | 1 pig Pigling Bland 4 0.8 true 2 cow Daisy 4 2.0 true 3 goldfish Dobbin null 0.05 false 4 ant null 6 0.001 false 5 ant null 6 0.001 false 6 queen ant Ma'am 6 0.002 false 7 human Mark 2 1.8 true
pds4
NASA's Planetary Data System version 4 format is described at https://pds.nasa.gov/datastandards/. This implementation is based on v1.16.0 of PDS4.
PDS4 files consist of an XML Label file which
provides detailed metadata, and which may also contain references
to external data files stored alongside it.
This input handler looks for (binary, character or delimited)
tables in the Label;
depending on the configuration it may restrict them to those
in the File_Area_Observational
area.
The Label is the file which has to be presented to this
input handler to read the table data.
Because of the relationship between the label and the data files,
it is usually necessary to move them around together.
If there are multiple tables in the label,
you can refer to an individual one using the "#
"
specifier after the label file name by table name
,
local_identifier
, or 1-based index
(e.g. "label.xml#1
" refers to the first table).
If there are Special_Constants
defined
in the label, they are in most cases interpreted as blank values
in the output table data.
At present, the following special values are interpreted
as blanks:
saturated_constant
,
missing_constant
,
error_constant
,
invalid_constant
,
unknown_constant
,
not_applicable_constant
,
high_instrument_saturation
,
high_representation_saturation
,
low_instrument_saturation
,
low_representation_saturation
.
Fields within top-level Groups are interpreted as array values. Any fields in nested groups are ignored. For these array values only limited null-value substitution can be done (since array elements are primitives and so cannot take null values).
This input handler is somewhat experimental, and the author is not a PDS expert. If it behaves strangely or you have suggestions for how it could work better, please contact the author.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"pds4(checkmagic=false,observational=true)
".
The following options are available:
checkmagic = true|false
true
)
observational = true|false
<File_Area_Observational>
element
of the PDS4 label should be included.
If true, only observational tables are found,
if false, other tables will be found as well.
(Default: false
)
Files in this format may contain multiple tables;
depending on the context, either one or all tables
will be read.
Where only one table is required,
either the first one in the file is used,
or the required one can be specified after the
"#
" character at the end of the filename.
This format can be automatically identified by its content so you do not need to specify the format explicitly when reading PDS4 tables, regardless of the filename.
mrt
The so-called "Machine-Readable Table" format is used by AAS journals, and based on the format of readMe files used by the CDS. There is some documentation at https://journals.aas.org/mrt-standards/, which mostly builds on documentation at http://vizier.u-strasbg.fr/doc/catstd.htx, but the format is in fact quite poorly specified, so this input handler was largely developed on a best-efforts basis by looking at MRT tables actually in use by AAS, and with assistance from AAS staff. As such, it's not guaranteed to succeed in reading all MRT files out there, but it will try its best.
It only attempts to read MRT files themselves, there is currently no capability to read VizieR data tables which provide the header and formatted data in separate files; however, if a table is present in VizieR, there will be options to download it in more widely used formats that can be used instead.
An example looks like this:
Title: A search for multi-planet systems with TESS using a Bayesian N-body retrieval and machine learning Author: Pearson K.A. Table: Stellar Parameters ================================================================================ Byte-by-byte Description of file: ajab4e1ct2_mrt.txt -------------------------------------------------------------------------------- Bytes Format Units Label Explanations -------------------------------------------------------------------------------- 1- 9 I9 --- ID TESS Input Catalog identifier 11- 15 F5.2 mag Tmag Apparent TESS band magnitude 17- 21 F5.3 solRad R* Stellar radius 23- 26 I4 K Teff Effective temperature 28- 32 F5.3 [cm/s2] log(g) log surface gravity 34- 38 F5.2 [Sun] [Fe/H] Metallicity 40- 44 F5.3 --- u1 Linear Limb Darkening 46- 50 F5.3 --- u2 Quadratic Limb Darkening -------------------------------------------------------------------------------- 231663901 12.35 0.860 5600 4.489 0.00 0.439 0.138 149603524 9.72 1.280 6280 4.321 0.24 0.409 0.140 336732616 11.46 1.400 6351 4.229 0.00 0.398 0.140 231670397 9.85 2.070 6036 3.934 0.00 0.438 0.117 ...
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"mrt(checkmagic=false,errmode=FAIL)
".
The following options are available:
checkmagic = true|false
Title:
".
Setting this true is generally a good idea
to avoid attempting to parse non-MRT files,
but you can set it false to attempt to read an MRT file
that starts with the wrong sequence.
(Default: true
)
errmode = IGNORE|WARN|FAIL
warn
)
usefloat = true|false
If it is set true,
then encountering values outside the representable range
will be reported in accordance with the current ErrorMode.
(Default: false
)
This format can be automatically identified by its content so you do not need to specify the format explicitly when reading MRT tables, regardless of the filename.
parquet
Parquet is a columnar format developed within the Apache project. Data is compressed on disk and read into memory before use.
This input handler will read columns representing scalars, strings and one-dimensional arrays of the same. It is not capable of reading multi-dimensional arrays, more complex nested data structures, or some more exotic data types like 96-bit integers. If such columns are encountered in an input file, a warning will be emitted through the logging system and the column will not appear in the read table. Support may be introduced for some additional types if there is demand.
At present, only very limited metadata is read. Parquet does not seem(?) to have any standard format for per-column metadata, so the only information read about each column apart from its datatype is its name.
Depending on the way that the table is accessed, the reader tries to take advantage of the column and row block structure of parquet files to read the data in parallel where possible.
Parquet support is currently somewhat experimental.
Note:
The parquet I/O handlers require large external libraries, which are not always bundled with the library/application software because of their size. In some configurations, parquet support may not be present, and attempts to read or write parquet files will result in a message like:Parquet-mr libraries not availableIf you can supply the relevant libaries on the classpath at runtime, the parquet support will work. At time of writing, the required libraries are included in thetopcat-extra.jar
monolithic jar file (though nottopcat-full.jar
), and are included if you have thetopcat-all.dmg
file. They can also be found in the starjava github repository (https://github.com/Starlink/starjava/tree/master/parquet/src/lib or you can acquire them from the Parquet MR package. These arrangements may be revised in future releases, for instance if parquet usage becomes more mainstream. The required dependencies are a minimal subset of those required by the Parquet MR submoduleparquet-cli
, in particular the filesaircompressor-0.21.jar
commons-collections-3.2.2.jar
commons-configuration2-2.1.1.jar
commons-lang3-3.9.jar
failureaccess-1.0.1.jar
guava-27.0.1-jre.jar
hadoop-auth-3.2.3.jar
hadoop-common-3.2.3.jar
hadoop-mapreduce-client-core-3.2.3.jar
htrace-core4-4.1.0-incubating.jar
parquet-cli-1.13.1.jar
parquet-column-1.13.1.jar
parquet-common-1.13.1.jar
parquet-encoding-1.13.1.jar
parquet-format-structures-1.13.1.jar
parquet-hadoop-1.13.1.jar
parquet-jackson-1.13.1.jar
slf4j-api-1.7.22.jar
slf4j-nop-1.7.22.jar
snappy-java-1.1.8.3.jar
stax2-api-4.2.1.jar
woodstox-core-5.3.0.jar
.
These libraries support some, but not all, of the compression formats defined for parquet, currentlyuncompressed
,gzip
,snappy
andlz4_raw
. Supplying more of the parquet-mr dependencies at runtime would extend this list.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"parquet(cachecols=true,nThread=4)
".
The following options are available:
cachecols = true|false|null
true
, then when the table is loaded,
all data is read by column into local scratch disk files,
which is generally the fastest way to ingest all the data.
If false
, the table rows are read as required,
and possibly cached using the normal STIL mechanisms.
If null
(the default), the decision is taken
automatically based on available information.
(Default: null
)
nThread = <int>
cachecols
option.
If the value is <=0 (the default), a value is chosen
based on the number of apparently available processors.
(Default: 0
)
tryUrl = true|false
false
)
This format can be automatically identified by its content so you do not need to specify the format explicitly when reading parquet tables, regardless of the filename.
hapi
HAPI, the
Heliophysics Data Application Programmer’s Interface
is a protocol for serving streamed time series data.
This reader can read HAPI CSV and binary tables
if they include header information
(the include=header
request parameter
must be present).
An example HAPI URL is
https://vires.services/hapi/data?dataset=GRACE_A_MAG&start=2009-01-01&stop=2009-01-02&include=header
While HAPI data is normally accessed directly from the service, it is possible to download a HAPI stream to a local file and use this handler to read it from disk.
This format cannot be automatically identified
by its content, so in general it is necessary
to specify that a table is in
HAPI
format when reading it.
However, if the input file has
the extension ".hapi
" (case insensitive)
an attempt will be made to read it using this format.
feather
The Feather file format is a column-oriented binary disk-based format based on Apache Arrow and supported by (at least) Python, R and Julia. Some description of it is available at https://github.com/wesm/feather and https://blog.rstudio.com/2016/03/29/feather/. It can be used for large datasets, but it does not support array-valued columns. It can be a useful format to use for exchanging data with R, for which FITS I/O is reported to be slow.
At present CATEGORY type columns are not supported, and metadata associated with TIME, DATE and TIMESTAMP columns is not retrieved.
This format can be automatically identified by its content so you do not need to specify the format explicitly when reading feather tables, regardless of the filename.
gbin
GBIN format is a special-interest file format used within DPAC, the Data Processing and Analysis Consortium working on data from the Gaia astrometry satellite. It is based on java serialization, and in all of its various forms has the peculiarity that you only stand any chance of decoding it if you have the Gaia data model classes on your java classpath at runtime. Since the set of relevant classes is very large, and also depends on what version of the data model your GBIN file corresponds to, those classes will not be packaged with this software, so some additional setup is required to read GBIN files.
As well as the data model classes, you must provide on the runtime classpath the GaiaTools classes required for GBIN reading. The table input handler accesses these by reflection, to avoid an additional large library dependency for a rather niche requirement. It is likely that since you have to supply the required data model classes you will also have the required GaiaTools classes to hand as well, so this shouldn't constitute much of an additional burden for usage.
In practice, if you have a jar file or files for pretty much any
java library or application which is capable of reading a given
GBIN file, just adding it or them to the classpath at runtime
when using this input handler ought to do the trick.
Examples of such jar files are
the
MDBExplorerStandalone.jar
file available from
https://gaia.esac.esa.int/mdbexp/,
or the gbcat.jar
file you can build from the
CU9/software/gbcat/
directory in the DPAC subversion repository.
The GBIN format doesn't really store tables, it stores arrays of java objects, so the input handler has to make some decisions about how to flatten these into table rows.
In its simplest form, the handler basically looks for public instance
methods of the form getXxx()
and uses the Xxx
as column names.
If the corresponding values are themselves objects with suitable getter
methods, those objects are added as new columns instead.
This more or less follows the practice of the gbcat
(gaia.cu1.tools.util.GbinInterogator
) tool.
Method names are sorted alphabetically.
Arrays of complex objects are not handled well,
and various other things may trip it up.
See the source code (e.g. uk.ac.starlink.gbin.GbinTableProfile
)
for more details.
If the object types stored in the GBIN file are known to the
special metadata-bearing class
gaia.cu9.tools.documentationexport.MetadataReader
and its dependencies, and if that class is on the runtime classpath,
then the handler will be able to extract additional metadata as available,
including standardised column names,
table and column descriptions, and UCDs.
An example of a jar file containing this metadata class alongside
data model classes is GaiaDataLibs-18.3.1-r515078.jar
.
Note however at time of writing there are some deficiencies with this
metadata extraction functionality related to unresolved issues
in the upstream gaia class libraries and the relevant
interface control document
(GAIA-C9-SP-UB-XL-034-01, "External Data Centres ICD").
Currently columns appear in the output table in a more or less
random order, units and Utypes are not extracted,
and using the GBIN reader tends to cause a 700kbyte file "temp.xml"
to be written in the current directory.
If the upstream issues are fixed, this behaviour may improve.
Note: support for GBIN files is somewhat experimental. Please contact the author (who is not a GBIN expert) if it doesn't seem to be working properly or you think it should do things differently.
Note: there is a known bug in some versions of
GaiaTools (caused by a bug in its dependency library zStd-jni)
which in rare cases can fail to read all the rows in a GBIN input file.
If this bug is encountered by the reader, it will by default
fail with an error mentioning zStd-jni.
In this case, the best thing to do is to put a fixed version of zStd-jni
or GaiaTools on the classpath.
However, if instead you set the config option readMeta=false
the read will complete without error, though the missing rows will not
be recovered.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"gbin(readMeta=false,hierarchicalNames=true)
".
The following options are available:
readMeta = true|false
Setting this false can prevent failing on an error related to a broken version of the zStd-jni library in GaiaTools.
Note however that in this case the data read, though not reporting an error, will silently be missing some rows from the GBIN file.
(Default: true
)
hierarchicalNames = true|false
Astrometry_Alpha
", if false they may just be called "Alpha
".
In case of name duplication however, the hierarchical form is always used.
(Default: false
)
This format can be automatically identified by its content so you do not need to specify the format explicitly when reading GBIN tables, regardless of the filename.
Example:
Suppose you have the MDBExplorerStandalone.jar
file
containing the data model classes, you can read GBIN files by
starting STILTS like this:
stilts -classpath MDBExplorerStandalone.jar ...or like this:
java -classpath stilts.jar:MDBExplorerStandalone.jar uk.ac.starlink.ttools.Stilts ...
tst
Tab-Separated Table, or TST, is a text-based table format used
by a number of astronomical tools including Starlink's
GAIA
and
ESO's SkyCat
on which it is based.
A definition of the format can be found in
Starlink Software Note 75.
The implementation here ignores all comment lines: special comments
such as the "#column-units:
" are not processed.
An example looks like this:
Simple TST example; stellar photometry catalogue. A.C. Davenhall (Edinburgh) 26/7/00. Catalogue of U,B,V colours. UBV photometry from Mount Pumpkin Observatory, see Sage, Rosemary and Thyme (1988). # Start of parameter definitions. EQUINOX: J2000.0 EPOCH: J1996.35 id_col: -1 ra_col: 0 dec_col: 1 # End of parameter definitions. ra<tab>dec<tab>V<tab>B_V<tab>U_B --<tab>---<tab>-<tab>---<tab>--- 5:09:08.7<tab> -8:45:15<tab> 4.27<tab> -0.19<tab> -0.90 5:07:50.9<tab> -5:05:11<tab> 2.79<tab> +0.13<tab> +0.10 5:01:26.3<tab> -7:10:26<tab> 4.81<tab> -0.19<tab> -0.74 5:17:36.3<tab> -6:50:40<tab> 3.60<tab> -0.11<tab> -0.47 [EOD]
This format cannot be automatically identified by its content, so in general it is necessary to specify that a table is in TST format when reading it.
wdc
Some support is provided for files produced by the World Data Centre for Solar Terrestrial Physics. The format itself apparently has no name, but files in this format look something like the following:
Column formats and units - (Fixed format columns which are single space separated.) ------------------------ Datetime (YYYY mm dd HHMMSS) %4d %2d %2d %6d - %1s aa index - 3-HOURLY (Provisional) %3d nT 2000 01 01 000000 67 2000 01 01 030000 32 ...Support for this (obsolete?) format may not be very complete or robust.
This format cannot be automatically identified by its content, so in general it is necessary to specify that a table is in WDC format when reading it.
Some of the tools ask you to specify the format of output tables
using the ofmt
parameter.
The output formats described below are supported; in some cases there
are variants or options for the basic formats as documented.
If you don't specify an output format explicitly,
STILTS will try to guess what format to write based on the
output filename; the details of those rules are also documented below.
It is also possible to add new formats at runtime using the
startable.writers
system property,
or by setting the format to the classname of a
uk.ac.starlink.table.StarTableWriter
class.
fits
FITS is a very well-established format for storage of astronomical table or image data (see https://fits.gsfc.nasa.gov/). This writer stores tables in BINTABLE extensions of a FITS file.
There are a number of variations in exactly how the table data is
written to FITS.
These can be configured with name=value
options in brackets
as described below, but for most purposes this isn't required;
you can just choose fits
or one of the standard
aliases for commonly-used combinations like colfits
or fits-basic
.
In all cases the output from this handler is legal FITS, but some non-standard conventions are used:
primary=votable
option or fits-plus
alias
(if you don't want it,
use primary=basic
or fits-basic
).
This convention is described in more detail in Section 5.1.3.1.
col=true
option or the colfits-plus/colfits-basic
aliases.
If you write to a file with the ".colfits
" extension
it is used by default.
For convenience, and compatibility with earlier versions, these standard aliases are provided:
fits
or fits(primary=votable)
.
fits(primary=basic)
.
fits(primary=basic,var=true)
.
fits(col=true)
.
fits(col=true,primary=basic)
.
fits-basic
,
but it will rearrange and rename columns as required to follow
the convention, and it will fail if the table does not contain
the required HEALPix metadata (STIL_HPX_*
parameters).
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"fits(primary=basic,col=false)
".
The following options are available:
primary = basic|votable[n.n]|none
basic
votable[n.n]
[n.n]
" part
is optional, but if included
(e.g. "votable1.5
")
indicates the version of the VOTable format to use.
none
votable
)
col = true|false
false
)
var = FALSE|TRUE|P|Q
True
stores variable-length array values
after the main part of the table in the heap,
while false
stores all arrays as fixed-length
(with a length equal to that of the longest array
in the column) in the body of the table.The options P
or Q
can be used
to force 32-bit or 64-bit pointers for indexing into the heap,
but it's not usually necessary since a suitable choice
is otherwise made from the data.
(Default: FALSE
)
date = true|false
true
)
Multiple tables may be written to a single output file using this format.
If no output format is explicitly chosen,
writing to a filename with
the extension ".fits
", ".fit
" or ".fts
" (case insensitive)
will select fits
format for output.
votable
VOTable is an XML-based format for tabular data endorsed by the International Virtual Observatory Alliance and defined in the VOTable Recommendation. While the tabular data which can be encoded is by design close to what FITS allows, it provides for much richer encoding of structure and metadata. Most of the table data exchanged by VO services is in VOTable format, but it can be used for local table storage as well.
When a table is saved to VOTable format, a document conforming to the
VOTable specification containing a single TABLE element within
a single RESOURCE element is written.
Where the table contains such information
(often obtained by reading an input VOTable),
column and table metadata will be written out as appropriate to
the attributes
unit
, ucd
, xtype
and utype
,
and the elements
COOSYS
, TIMESYS
and DESCRIPTION
attached to table columns or parameters.
There are various ways that a VOTable can be written;
by default the output serialization format is TABLEDATA
and the VOTable format version is 1.4, or a value controlled
by the votable.version
system property.
However, configuration options are available to adjust these defaults.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"votable(format=BINARY2,version=V13)
".
The following options are available:
format = TABLEDATA|BINARY|BINARY2|FITS
TABLEDATA
)
version = V10|V11|V12|V13|V14|V15
V10
" is version 1.0 etc.inline = true|false
true
)
compact = true|false|null
null
)
encoding = UTF-8|UTF-16|...
UTF-8
)
date = true|false
true
)
Multiple tables may be written to a single output file using this format.
If no output format is explicitly chosen,
writing to a filename with
the extension ".vot
", ".votable
" or ".xml
" (case insensitive)
will select votable
format for output.
csv
Writes tables in the semi-standard Comma-Separated Values format. This does not preserve any metadata apart from column names, and is generally inefficient to read, but it can be useful for importing into certain external applications, such as some databases or spreadsheets.
By default, the first line is a header line giving the column names,
but this can be inhibited using the header=false
configuration option.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"csv(header=true,maxCell=160)
".
The following options are available:
header = true|false
true
)
maxCell = <int>
2147483647
)
If no output format is explicitly chosen,
writing to a filename with
the extension ".csv
" (case insensitive)
will select CSV
format for output.
An example looks like this:
RECNO,SPECIES,NAME,LEGS,HEIGHT,MAMMAL 1,pig,Pigling Bland,4,0.8,true 2,cow,Daisy,4,2.0,true 3,goldfish,Dobbin,,0.05,false 4,ant,,6,0.001,false 5,ant,,6,0.001,false 6,queen ant,Ma'am,6,0.002,false 7,human,Mark,2,1.8,true
ecsv
The Enhanced Character Separated Values format was developed within the Astropy project and is described in Astropy APE6 (DOI). It is composed of a YAML header followed by a CSV-like body, and is intended to be a human-readable and maybe even human-writable format with rich metadata. Most of the useful per-column and per-table metadata is preserved when de/serializing to this format. The version supported by this writer is currently ECSV 1.0.
ECSV allows either a space or a comma for delimiting values,
controlled by the delimiter
configuration option.
If ecsv(delimiter=comma)
is used, then removing
the YAML header will leave a CSV file that can be interpreted
by the CSV inputhandler or imported into other
CSV-capable applications.
Following the ECSV 1.0 specification, array-valued columns are supported. ECSV 1.0, required for working with array-valued columns, is supported by Astropy v4.3 and later.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"ecsv(delimiter=comma)
".
The following options are available:
delimiter = comma|space
space
" or "comma
".If no output format is explicitly chosen,
writing to a filename with
the extension ".ecsv
" (case insensitive)
will select ECSV
format for output.
An example looks like this:
# %ECSV 1.0 # --- # datatype: # - # name: RECNO # datatype: int32 # - # name: SPECIES # datatype: string # - # name: NAME # datatype: string # description: How one should address the animal in public & private. # - # name: LEGS # datatype: int32 # meta: # utype: anatomy:limb # - # name: HEIGHT # datatype: float64 # unit: m # meta: # VOTable precision: 2 # - # name: MAMMAL # datatype: bool # meta: # name: animals.vot # Description: Some animals # Author: Mark Taylor RECNO SPECIES NAME LEGS HEIGHT MAMMAL 1 pig "Pigling Bland" 4 0.8 True 2 cow Daisy 4 2.0 True 3 goldfish Dobbin "" 0.05 False 4 ant "" 6 0.001 False 5 ant "" 6 0.001 False 6 "queen ant" Ma'am 6 0.002 False 7 human Mark 2 1.8 True
ascii
Writes to a simple plain-text format intended to be comprehensible by humans or machines.
The first line is a comment, starting with a "#
" character,
naming the columns, and an attempt is made to line up data in columns
using spaces. No metadata apart from column names is written.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"ascii(maxCell=158,maxParam=160)
".
The following options are available:
maxCell = <int>
158
)
maxParam = <int>
160
)
params = true|false
false
)
sampledRows = <int>
0
)
If no output format is explicitly chosen,
writing to a filename with
the extension ".txt
" (case insensitive)
will select ascii
format for output.
An example looks like this:
# RECNO SPECIES NAME LEGS HEIGHT MAMMAL 1 pig "Pigling Bland" 4 0.8 true 2 cow Daisy 4 2.0 true 3 goldfish Dobbin "" 0.05 false 4 ant "" 6 0.001 false 5 ant "" 6 0.001 false 6 "queen ant" "Ma\'am" 6 0.002 false 7 human Mark 2 1.8 true
ipac
Writes output in the format used by CalTech's Infrared Processing and Analysis Center, and defined at http://irsa.ipac.caltech.edu/applications/DDGEN/Doc/ipac_tbl.html. Column name, type, units and null values are written, as well as table parameters.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"ipac(maxCell=1000,maxParam=100000)
".
The following options are available:
maxCell = <int>
1000
)
maxParam = <int>
100000
)
params = true|false
true
)
sampledRows = <int>
0
)
If no output format is explicitly chosen,
writing to a filename with
the extension ".tbl
" or ".ipac
" (case insensitive)
will select IPAC
format for output.
An example looks like this:
\Table name = "animals.vot" \Description = "Some animals" \Author = "Mark Taylor" | RECNO | SPECIES | NAME | LEGS | HEIGHT | MAMMAL | | int | char | char | int | double | char | | | | | | m | | | null | null | null | null | null | null | 1 pig Pigling Bland 4 0.8 true 2 cow Daisy 4 2.0 true 3 goldfish Dobbin null 0.05 false 4 ant null 6 0.001 false 5 ant null 6 0.001 false 6 queen ant Ma'am 6 0.002 false 7 human Mark 2 1.8 true
parquet
Parquet is a columnar format developed within the Apache project. Data is compressed on disk and read into memory before use.
At present, only very limited metadata is written. Parquet does not seem(?) to have any standard format for per-column metadata, so the only information written about each column apart from its datatype is its name.
Parquet support is currently somewhat experimental.
Note:
The parquet I/O handlers require large external libraries, which are not always bundled with the library/application software because of their size. In some configurations, parquet support may not be present, and attempts to read or write parquet files will result in a message like:Parquet-mr libraries not availableIf you can supply the relevant libaries on the classpath at runtime, the parquet support will work. At time of writing, the required libraries are included in thetopcat-extra.jar
monolithic jar file (though nottopcat-full.jar
), and are included if you have thetopcat-all.dmg
file. They can also be found in the starjava github repository (https://github.com/Starlink/starjava/tree/master/parquet/src/lib or you can acquire them from the Parquet MR package. These arrangements may be revised in future releases, for instance if parquet usage becomes more mainstream. The required dependencies are a minimal subset of those required by the Parquet MR submoduleparquet-cli
, in particular the filesaircompressor-0.21.jar
commons-collections-3.2.2.jar
commons-configuration2-2.1.1.jar
commons-lang3-3.9.jar
failureaccess-1.0.1.jar
guava-27.0.1-jre.jar
hadoop-auth-3.2.3.jar
hadoop-common-3.2.3.jar
hadoop-mapreduce-client-core-3.2.3.jar
htrace-core4-4.1.0-incubating.jar
parquet-cli-1.13.1.jar
parquet-column-1.13.1.jar
parquet-common-1.13.1.jar
parquet-encoding-1.13.1.jar
parquet-format-structures-1.13.1.jar
parquet-hadoop-1.13.1.jar
parquet-jackson-1.13.1.jar
slf4j-api-1.7.22.jar
slf4j-nop-1.7.22.jar
snappy-java-1.1.8.3.jar
stax2-api-4.2.1.jar
woodstox-core-5.3.0.jar
.
These libraries support some, but not all, of the compression formats defined for parquet, currentlyuncompressed
,gzip
,snappy
andlz4_raw
. Supplying more of the parquet-mr dependencies at runtime would extend this list.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"parquet(compression=gzip,groupArray=false)
".
The following options are available:
compression = uncompressed|snappy|gzip|lz4_raw
uncompressed
, snappy
,
gzip
and lz4_raw
.
Others may be available if the relevant codecs are on the
classpath at runtime.
If no value is specified, the parquet-mr library default
is used, which is probably uncompressed
.
(Default: null
)
groupArray = true|false
groupArray=false
will write it as
"repeated int32 IVAL
"
while groupArray=true
will write it as
"optional group IVAL (LIST) { repeated group list
{ optional int32 item} }
".
I don't know why you'd want to do it the latter way,
but some other parquet writers seem to do that by default,
so there must be some good reason.
(Default: false
)
usedict = true|false|null
true
.
(Default: null
)
If no output format is explicitly chosen,
writing to a filename with
the extension ".parquet
" or ".parq
" (case insensitive)
will select parquet
format for output.
feather
The Feather file format is a column-oriented binary disk-based format based on Apache Arrow and supported by (at least) Python, R and Julia. Some description of it is available at https://github.com/wesm/feather and https://blog.rstudio.com/2016/03/29/feather/. It can be used for large datasets, but it does not support array-valued columns. It can be a useful format to use for exchanging data with R, for which FITS I/O is reported to be slow.
This writer is somewhat experimental; please report problems if you encounter them.
If no output format is explicitly chosen,
writing to a filename with
the extension ".fea
" or ".feather
" (case insensitive)
will select feather
format for output.
text
Writes tables in a simple text-based format designed to be read by humans. No reader exists for this format.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"text(maxCell=40,maxParam=160)
".
The following options are available:
maxCell = <int>
40
)
maxParam = <int>
160
)
params = true|false
true
)
sampledRows = <int>
0
)
Multiple tables may be written to a single output file using this format.
An example looks like this:
Table name: animals.vot Description: Some animals Author: Mark Taylor +-------+-----------+---------------+------+--------+--------+ | RECNO | SPECIES | NAME | LEGS | HEIGHT | MAMMAL | +-------+-----------+---------------+------+--------+--------+ | 1 | pig | Pigling Bland | 4 | 0.8 | true | | 2 | cow | Daisy | 4 | 2.0 | true | | 3 | goldfish | Dobbin | | 0.05 | false | | 4 | ant | | 6 | 0.001 | false | | 5 | ant | | 6 | 0.001 | false | | 6 | queen ant | Ma'am | 6 | 0.002 | false | | 7 | human | Mark | 2 | 1.8 | true | +-------+-----------+---------------+------+--------+--------+
html
Writes a basic HTML TABLE
element
suitable for use as a web page or for insertion into one.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"html(maxCell=200,standalone=false)
".
The following options are available:
maxCell = <int>
200
)
standalone = true|false
false
)
Multiple tables may be written to a single output file using this format.
If no output format is explicitly chosen,
writing to a filename with
the extension ".html
" or ".htm
" (case insensitive)
will select HTML
format for output.
An example looks like this:
<TABLE BORDER='1'> <CAPTION><STRONG>animals.vot</STRONG></CAPTION> <THEAD> <TR> <TH>RECNO</TH> <TH>SPECIES</TH> <TH>NAME</TH> <TH>LEGS</TH> <TH>HEIGHT</TH> <TH>MAMMAL</TH></TR> <TR> <TH> </TH> <TH> </TH> <TH> </TH> <TH> </TH> <TH>(m)</TH> <TH> </TH></TR> <TR><TD colspan='6'></TD></TR> </THEAD> <TBODY> <TR> <TD>1</TD> <TD>pig</TD> <TD>Pigling Bland</TD> <TD>4</TD> <TD>0.8</TD> <TD>true</TD></TR> <TR> <TD>2</TD> <TD>cow</TD> <TD>Daisy</TD> <TD>4</TD> <TD>2.0</TD> <TD>true</TD></TR> <TR> <TD>3</TD> <TD>goldfish</TD> <TD>Dobbin</TD> <TD> </TD> <TD>0.05</TD> <TD>false</TD></TR> <TR> <TD>4</TD> <TD>ant</TD> <TD> </TD> <TD>6</TD> <TD>0.001</TD> <TD>false</TD></TR> <TR> <TD>5</TD> <TD>ant</TD> <TD> </TD> <TD>6</TD> <TD>0.001</TD> <TD>false</TD></TR> <TR> <TD>6</TD> <TD>queen ant</TD> <TD>Ma'am</TD> <TD>6</TD> <TD>0.002</TD> <TD>false</TD></TR> <TR> <TD>7</TD> <TD>human</TD> <TD>Mark</TD> <TD>2</TD> <TD>1.8</TD> <TD>true</TD></TR> </TBODY> </TABLE>
latex
Writes a table as a LaTeX tabular
environment,
suitable for insertion into a document intended for publication.
This is only likely to be useful for fairly small tables.
The handler behaviour may be modified by specifying
one or more comma-separated name=value configuration options
in parentheses after the handler name, e.g.
"latex(standalone=false)
".
The following options are available:
standalone = true|false
tabular
environment within a table
within a document
. If false, the output is just a tabular
environment.
(Default: false
)
If no output format is explicitly chosen,
writing to a filename with
the extension ".tex
" (case insensitive)
will select LaTeX
format for output.
An example looks like this:
\begin{tabular}{|r|l|l|r|r|l|} \hline \multicolumn{1}{|c|}{RECNO} & \multicolumn{1}{c|}{SPECIES} & \multicolumn{1}{c|}{NAME} & \multicolumn{1}{c|}{LEGS} & \multicolumn{1}{c|}{HEIGHT} & \multicolumn{1}{c|}{MAMMAL} \\ \hline 1 & pig & Pigling Bland & 4 & 0.8 & true\\ 2 & cow & Daisy & 4 & 2.0 & true\\ 3 & goldfish & Dobbin & & 0.05 & false\\ 4 & ant & & 6 & 0.001 & false\\ 5 & ant & & 6 & 0.001 & false\\ 6 & queen ant & Ma'am & 6 & 0.002 & false\\ 7 & human & Mark & 2 & 1.8 & true\\ \hline\end{tabular}
tst
Tab-Separated Table, or TST, is a text-based table format used by a number of astronomical tools including Starlink's GAIA and ESO's SkyCat on which it is based. A definition of the format can be found in Starlink Software Note 75.
If no output format is explicitly chosen,
writing to a filename with
the extension ".tst
" (case insensitive)
will select TST
format for output.
An example looks like this:
animals.vot # Table parameters Description: Some animals Author: Mark Taylor # Attempted guesses about identity of columns in the table. # These have been inferred from column UCDs and/or names # in the original table data. # The algorithm which identifies these columns is not particularly reliable, # so it is possible that these are incorrect. id_col: 2 ra_col: -1 dec_col: -1 # This TST file generated by STIL v4.3-1 RECNO SPECIES NAME LEGS HEIGHT MAMMAL ----- ------- ---- ---- ------ ------ 1 pig Pigling Bland 4 0.8 true 2 cow Daisy 4 2.0 true 3 goldfish Dobbin 0.05 false 4 ant 6 0.001 false 5 ant 6 0.001 false 6 queen ant Ma'am 6 0.002 false 7 human Mark 2 1.8 true [EOD]
mirage
Mirage was a nice standalone tool for analysis of multidimensional data, from which TOPCAT took some inspiration. It was described in a 2007 paper 2007ASPC..371..391H, but no significant development seems to have taken place since then. This format is therefore probably obsolete, but you can still write table output in Mirage-compatible format if you like.
If no output format is explicitly chosen,
writing to a filename with
the extension ".mirage
" (case insensitive)
will select mirage
format for output.
An example looks like this:
# # Written by uk.ac.starlink.mirage.MirageFormatter # Omitted column 5: MAMMAL(Boolean) # # Column names format var RECNO SPECIES NAME LEGS HEIGHT # # Text columns format text SPECIES format text NAME # # Table data 1 pig Pigling_Bland 4 0.8 2 cow Daisy 4 2.0 3 goldfish Dobbin <blank> 0.05 4 ant <blank> 6 0.001 5 ant <blank> 6 0.001 6 queen_ant Ma'am 6 0.002 7 human Mark 2 1.8
STIL, the I/O library underlying STILTS, uses a few private conventions when writing and reading FITS files. These are not private in the sense that non-STIL code is prevented from cooperating with them, but STIL does not assume that other code, or FITS tables it encounters, will use these conventions. Instead, they offer (in some cases) added value for tables that were written by STIL and are subsequently re-read by STIL, while causing the minimum of trouble for non-STIL readers.
When writing tables to FITS BINTABLE format, STIL can optionally store additional metadata in the FITS file using a private convention known as "FITS-plus". The table is written exactly as usual in a BINTABLE extension HDU, but the primary HDU (HDU#0) contains a sequence of characters, stored as a 1-d array of bytes using UTF-8 encoding, which forms the text of a DATA-less VOTable document. Note that the Primary HDU cannot be used to store table data, so under normal circumstances has no interesting content in a FITS file used just for table storage. The FITS tables in the subsequent extensions are understood to contain the data.
The point of this is that the VOTable can contain all the rich metadata about the table(s), but the bulk data are in a form which can be read efficiently. Crucially, the resulting FITS file is a perfectly good FITS table on its own, so non-VOTable-aware readers can read it in just the usual way, though of course they do not benefit from the additional metadata stored in the VOTable header.
In practice, STIL normally writes FITS files using this convention (it writes the VOTable metadata into the Primary HDU) and when reading a FITS files it looks for use of this convention (examines the Primary HDU for VOTable metadata and uses it if present). But if an input file does not follow this convention, the metadata is taken directly from the BINTABLE header as normal. Non-FITS-plus-aware (i.e. non-STIL) readers will ignore the Primary HDU, since it has no purpose in a standard FITS file containing only a table, and it doesn't look like anything else that such readers are usually expecting. The upshot is that for nearly all purposes you can forget about use of this convention when writing and reading FITS tables using STIL and other libraries, but STIL may be able to recover rich metadata from files that it has written itself.
To be recognised as a FITS-plus file, the Primary HDU (and hence the FITS file) must begin like this:
SIMPLE = T BITPIX = 8 NAXIS = 1 NAXIS1 = ??? VOTMETA = TThe sequence and values of the given header cards must be as shown, except for
NAXIS1
which contains the number of bytes
in the data block; any comments are ignored.
The content of the Primary HDU must be a VOTable document
containing zero or more TABLE
elements, one for each
BINTABLE extension appearing later in the FITS file.
Each such TABLE must not contain a DATA
child;
the table content is taken from the BINTABLE in the next unused
table HDU.
For instance the Primary HDU content annotating a single table
might look like this:
<?xml version='1.0'?> <VOTABLE version="1.3" xmlns="http://www.ivoa.net/xml/VOTable/v1.3"> <RESOURCE> <TABLE nrows="1000"> <FIELD datatype="double" name="RA" ucd="pos.eq.ra;meta.main"/> <FIELD datatype="double" name="Dec" ucd="pos.eq.dec;meta.main"/> <!-- Dummy VOTable - no DATA element here --> </TABLE> </RESOURCE> </VOTABLE>The first extension HDU would then contain the two-column BINTABLE corresponding to the given metadata.
The VOTable metadata MUST be compatible with the structure of the annotated BINTABLE(s) in terms of number and datatypes of columns.
Note: This arrangement bears some similarity to VOTable/FITS encoding, in which the output file is a VOTable which references an inline or external FITS file containing the bulk data. However, the VOTable/FITS format is inconvenient in that either (for in-line data) the FITS file is base64-encoded and so hard to read efficiently especially for random access, or (for referenced data) the table is split across two files.
The FITS BINTABLE standard (FITS Standard v4.0, section 7.3) permits a maximum of 999 columns in a binary table extension. Up to version 3.2 of STIL, attempting to write a table with more than 999 columns using one of the supported FITS-based writers failed with an error. In later versions, a non-standard convention is used which can store wider tables in a FITS BINTABLE extension. The various STIL FITS-based readers can (in their default configurations) read these tables transparently, allowing round-tripping of arbitrarily wide tables to FITS files. Note however that other FITS-compliant software is not in general aware of this convention, and will see a 999-column table. The first 998 columns will appear as intended, but subsequent ones will effectively be hidden.
The rest of this section describes the convention that is used to store tables with more than 999 columns in FITS BINTABLE extensions.
The BINTABLE extension type requires table column metadata
to be described using 8-character keywords of the form TXXXXnnn
,
where TXXXX
represents one of an open set of mandatory, reserved
or user-defined root keywords up to five characters in length,
for instance TFORM
(mandatory), TUNIT
(reserved),
TUCD
(user-defined).
The nnn
part is an integer between 1 and 999 indicating the
index of the column to which the keyword in question refers.
Since the header syntax confines this indexed part of the keyword
to three digits, there is an upper limit of 999 columns in
BINTABLE extensions.
Note that the FITS/BINTABLE format does not entail any restriction on the storage of column data beyond the 999 column limit in the data part of the HDU, the problem is just that client software cannot be informed about the layout of this data using the header cards in the usual way.
The following convention is used by STIL FITS-based I/O handlers to accommodate wide tables in FITS files:
N_TOT
is the total number of data columns to be stored
N_TOT
inclusive are known as extended columns.
Their data is stored within the container column.
N_TOT
)
is laid out in the data part of the HDU in exactly the same way
as if there were no 999-column limit.
TFIELDS
header is declared with the value 999.
TFORM999
value corresponding to the total field
length required by all the extended columns
('B
' is the obvious data type,
but any legal TFORM
value that gives the right
width MAY be used).
The byte count implied by TFORM999
MUST be equal to the
total byte count implied by all extended columns.
TXXXX999
headers MAY optionally
be declared to describe
the container column in accordance with the usual rules,
e.g. TTYPE999
to give it a name.
NAXIS1
header is declared
in the usual way to give the width
of a table row in bytes. This is equal to the sum of
all the BINTABLE column widths as usual. It is also equal to
the sum of all the data column widths, which has the same value.
XT_ICOL
indicates the index
of the container column.
It MUST be present with the integer value 999 to indicate
that this convention is in use.
XT_NCOL
indicates the total number
of data columns encoded.
It MUST be present with an integer value equal to N_TOT
.
HIERARCH XT TXXXXnnnnn
',
where TXXXX
are the same keyword roots as used for normal
BINTABLE extensions,
and nnnnn
is a decimal number written as usual
(no leading zeros, as many digits as are required).
Thus the formats for data columns 999, 1000, 1001 etc
are declared with the keywords
HIERARCH XT TFORM999
,
HIERARCH XT TFORM1000
,
HIERARCH XT TFORM1001
, etc.
Note this uses the ESO HIERARCH convention described at
https://fits.gsfc.nasa.gov/registry/hierarch_keyword.html.
The name space token has been chosen as
'XT
' (extended table).
N_TOT
<=999.
The resulting HDU is a completely legal FITS BINTABLE extension. Readers aware of this convention may use it to extract column data and metadata beyond the 999-column limit. Readers unaware of this convention will see 998 columns in their intended form, and an additional (possibly large) column 999 which contains byte data but which cannot be easily interpreted.
An example header might look like this:
XTENSION= 'BINTABLE' / binary table extension BITPIX = 8 / 8-bit bytes NAXIS = 2 / 2-dimensional table NAXIS1 = 9229 / width of table in bytes NAXIS2 = 26 / number of rows in table PCOUNT = 0 / size of special data area GCOUNT = 1 / one data group TFIELDS = 999 / number of columns XT_ICOL = 999 / index of container column XT_NCOL = 1204 / total columns including extended TTYPE1 = 'posid_1 ' / label for column 1 TFORM1 = 'J ' / format for column 1 TTYPE2 = 'instrument_1' / label for column 2 TFORM2 = '4A ' / format for column 2 TTYPE3 = 'edge_code_1' / label for column 3 TFORM3 = 'I ' / format for column 3 TUCD3 = 'meta.code.qual' ... TTYPE998= 'var_min_s_2' / label for column 998 TFORM998= 'D ' / format for column 998 TUNIT998= 'counts/s' / units for column 998 TTYPE999= 'XT_MORECOLS' / label for column 999 TFORM999= '813I ' / format for column 999 HIERARCH XT TTYPE999 = 'var_min_u_2' / label for column 999 HIERARCH XT TFORM999 = 'D' / format for column 999 HIERARCH XT TUNIT999 = 'counts/s' / units for column 999 HIERARCH XT TTYPE1000 = 'var_prob_h_2' / label for column 1000 HIERARCH XT TFORM1000 = 'D' / format for column 1000 ... HIERARCH XT TTYPE1203 = 'var_prob_w_2' / label for column 1203 HIERARCH XT TFORM1203 = 'D' / format for column 1203 HIERARCH XT TTYPE1204 = 'var_sigma_w_2' / label for column 1204 HIERARCH XT TFORM1204 = 'D' / format for column 1204 HIERARCH XT TUNIT1204 = 'counts/s' / units for column 1204 END
This general approach was suggested by William Pence on the FITSBITS list in June 2012, and by François-Xavier Pineau (CDS) in private conversation in 2016. The details have been filled in by Mark Taylor (Bristol), and discussed in some detail on the FITSBITS list in July 2017.
The location of a serialized input table,
usually given using the in
parameter or similar,
may be given in one of the forms listed below.
http:
https:
ftp:
file:
file:///path/to/file
.
This is similar to specifying the filename directly, but
there is a difference: using this form forces reads to be sequential
rather than random access, which may allow you to experience
a different set of performance characteristics and bugs.jar:
myspace:
myspace:/survey/iras_psc.xml
",
and can access files in the myspace are that the user is currently
logged into.
These URLs can be used for both input and output of tables.
To use them you must have an AstroGrid account and the AstroGrid
WorkBench or similar must be running; if you're not currently
logged in a dialogue will pop up to ask you for name and
password.ivo:
ivo://uk.ac.le.star/filemanager#node-2583
".
These URLs can be used for both input and output of tables.
To use them you must have an AstroGrid account and the AstroGrid
WorkBench or similar must be running; if you're not currently
logged in a dialogue will pop up to ask you for name and
password.jdbc:
-
")
<
syscmd" or
"syscmd|
")
<
" character
or ends with a "|
" character,
the rest of the string is taken as a command line to be executed
by the system shell.
For instance a location like
"<cat header.txt data.txt
"
(or equivalently
"cat header.txt data.txt|
")
could be used to prepend a header line to an ASCII data file
before it is passed to the STILTS ASCII-format input handler.
Note this syntax will probably only work on Unix-like systems.
In any of these cases, for input locations compression is taken care
of automatically. That means that you can give the filename or URL
of a file which is compressed using gzip
, bzip2
or Unix compress
and the program will uncompress it on the fly.
For file formats that can contain multiple tables,
the one required, if it's not the first in the file,
can generally be specified by a position indicator string
appended to the basic location following a "#
".
For instance "cat.fits#3" references HDU 3 in a multi-extension FITS file.
The details of this syntax depend on the file format,
and are given in the relevant subsection of
Section 5.1.1.
Note that tables can also be supplied from non-serialized sources, as described in Input Schemes.
As well as being able to load tables from external data streams, STILTS offers a way to specify tables that do not correspond to a stream of bytes. These may be defined programmatically or interact with external services in some way that is not as straightforward as decoding a stream of bytes.
Such tables are defined using different schemes,
and scheme specifications may be used in the same places as
input table names, for instance as the value of the in
parameter in tpipe
and other commands.
If an input location parameter (in
) uses a scheme specification,
the corresponding input format parameter (ifmt
) is ignored.
The form of a scheme specification is:
:<scheme-name>:<scheme-specific-part>so that for instance "
in=:loop:10
" specifies
a 10-row single-column table,
as described by the loop scheme documentation
below.
For an example of using such tables, you can try running for instance
stilts plot2plane in=:attractor:1e7,clifford layer1=mark shading1=density densemap1=plasma x1=x y1=y
The following subsections describe all the schemes that are
available by default.
It is also possible to add new schemes at runtime by using the
startable.schemes
system property.
skysim
Usage: :skysim:<nrow>
Generates a simulated all-sky star catalogue
with a specified number of rows.
This is intended to provide crude test catalogues
when no suitable real dataset of the required size
is available.
In the current implementation the row count,
which may be given in integer or exponential notation,
is the only parameter,
so the specification
":skysim:5e6
"
would give a 5 million-row simulated catalogue.
The current implementation provides somewhat realistic position-dependent star densities and distributions of magnitudes and colours based on positionally averaged values from Gaia EDR3. The source positions do not correspond to actual stars. The columns and the statistics on which the output is based may change in future releases.
Example:
:skysim:6 +-----------+------------+------------+------------+-----------+-----------+------------+ | ra | dec | l | b | gmag | rmag | b_r | +-----------+------------+------------+------------+-----------+-----------+------------+ | 266.7702 | -32.689117 | -3.044958 | -2.217145 | 18.168278 | 16.03555 | 1.046624 | | 276.83398 | 8.132022 | 37.491447 | 9.031909 | 18.331224 | 18.786451 | 1.4425725 | | 92.04118 | 23.79857 | -173.02219 | 1.7854756 | 16.743847 | 15.623316 | 1.690048 | | 271.08215 | -5.2012086 | 22.848532 | 8.022958 | 21.538874 | 17.782997 | 2.1645386 | | 298.83368 | 31.401922 | 67.75605 | 1.6348709 | 20.145718 | 17.728764 | 1.1521724 | | 204.9299 | -77.07571 | -54.29949 | -14.464215 | 19.044079 | 20.277771 | 0.92987275 | +-----------+------------+------------+------------+-----------+-----------+------------+
attractor
Usage: :attractor:<nrow>[,(clifford[,a,b,c,d]|rampe[,a,b,c,d,e,f]|henon[,a,b,c])]
Generates tables listing points sampled from one of a specified family of strange attractors. These can provide tables with (X,Y) or (X,Y,Z) columns and arbitrarily many rows. They can be used, for instance, to make (beautiful) example large-scale scatter plots in 2-d or 3-d space.
The specification syntax is of the form
:attractor:<nrow>,<family-name>[,<args>]
where
<nrow>
is the number of rows required,
<family-name>
is the name of one of
the supported families of attractors, and
<args>
is an optional comma-separated list
of numeric arguments specifying the family-specific parameters
of the required attractor.
If the <args>
part is omitted,
an example attractor from the family is used.
Note that picking <args>
values at random
will often result in rather boring (non-strange) attractors.
The following families are currently supported:
The iteration is defined by the equations:
x' = sin(a*y) + c * cos(a*x) y' = sin(b*x) + d * cos(b*y)
Examples:
:attractor:9999,clifford
:attractor:1e6,clifford,1.32,-1.44,-1.7,-1.58
:attractor:65536,clifford,1.27,-1.35,0.82,1.8
:attractor:1e7,clifford,-1.9,1.18,-1.21,1.07
:attractor:400,clifford,-1.27,-1.28,1.0,-1.26
:attractor:4e6,clifford,1.8,0.9,-1.8,0.8
The iteration is defined by the equations:
x' = x * z * sin(a*x) - cos(b*y) y' = y * x * sin(c*y) - cos(d*z) z' = z * y * sin(e*z) - cos(f*x)
Examples:
:attractor:10e6,rampe
:attractor:4,rampe,-1.81,1.35,-0.85,0.32,1.68,-1.62
:attractor:5.5e5,rampe,0.23,-1.77,1.32,-1.44,-1.7,-1.58
:attractor:9999,rampe,-0.3,1.78,-0.87,1.69,1.42,1.21
:attractor:1e6,rampe,1.42,-1.98,0.39,1.32,1.79,-0.37
The iteration is defined by the equations:
x' = y + a + b*x*x y' = c*x
Examples:
:attractor:65536,henon
:attractor:1e7,henon,-0.68,1.64,0.36
:attractor:400,henon,1.73,0.29,-0.99
:attractor:4e6,henon,0.88,-0.9,-0.93
:attractor:10e6,henon,1.4,-1.13,-0.01
Example:
:attractor:6,rampe +----------------------+---------------------+----------------------+ | x | y | z | +----------------------+---------------------+----------------------+ | -0.5759098296568739 | 0.09844750286352466 | -0.6712534741282851 | | -1.3295344852011892 | -0.9829776649068059 | -0.7814409891660122 | | -1.1910376215054008 | 0.04335596646295736 | -1.0308958690758545 | | -2.0144704755218514 | -0.9699626185329038 | -0.35169532148364757 | | -0.16145296509226564 | 0.5245428249077974 | 0.17929370340580017 | | -0.8409807675257591 | -0.9598486078341374 | -0.955769158222801 | +----------------------+---------------------+----------------------+
jdbc
Usage: :jdbc:<jdbc-part>
Interacts with the JDBC system
(JDBC sort-of stands for Java DataBase Connectivity)
to execute an SQL query on a connected database.
The jdbc:...
specification is the JDBC URL.
For historical compatibility reasons,
specifications of this scheme
may omit the leading colon character,
so that the following are both legal, and are equivalent:
jdbc:mysql://localhost/dbl#SELECT TOP 10 ra, dec FROM gsc :jdbc:mysql://localhost/dbl#SELECT TOP 10 ra, dec FROM gsc
In order for this to work, you must have access to a suitable database with a JDBC driver, and some standard JDBC configuration is required to set the driver up. The following steps are necessary:
jdbc.drivers
system property must be set
to the driver classname
More detailed information about how to set up the JDBC system to connect with an available database, and of how to construct JDBC URLs, is provided elsewhere in the documentation.
loop
Usage: :loop:<count>|<start>,<end>[,<step>]
Generates a table whose single column increments over a given range.
The specification may either be a single value N giving the number of rows, which yields values in the range 0..N-1, or two or three comma-separated values giving the start, end and optionally step corresponding to the conventional specification of a loop variable.
The supplied numeric parameters are interpreted as floating point values, but the output column type will be 32- or 64-bit integer or 64-bit floating point, depending on the values that it has to take.
Examples:
:loop:5
:
a 5-row table whose integer column has values
0, 1, 2, 3, 4
:loop:10,20
:
a 10-row table whose integer column has values
10, 11, ... 19
:loop:1,2,0.25
:
a 10-row table whose floating point column has values
1.00, 1.25, 1.50, 1.75
:loop:1e10
:
a ten billion row table, with 64-bit integer values
Example:
:loop:6 +---+ | i | +---+ | 0 | | 1 | | 2 | | 3 | | 4 | | 5 | +---+
test
Usage: :test:[<nrow>[,<opts-ibsfgvwmk*>]]
Generates a table containing test data. The idea is to include columns of different data types, for instance to provide an example for testing I/O handler implementations. The columns will contain some variety of more or less meaningless values, but the content is reproducible between runs, so the same specification will produce the same output each time. Updates of the implementation might change the output however, so the output is not guaranteed to be the same for all time.
The table specification has two comma-separated parameters:
<nrow>
: row count<opts>
:
a string of letter options specifying what types of data
will be included; options are:
<opts>
and/or <nrow>
are omitted, some default values are used.
Example:
:test:10,is +---------+--------+---------+-------+--------+---------+----------+----------------+-----------+ | i_index | s_byte | s_short | s_int | s_long | s_float | s_double | s_string | s_boolean | +---------+--------+---------+-------+--------+---------+----------+----------------+-----------+ | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | zero | false | | 1 | | 1 | 1 | 1 | 1.0 | 1.0 | one | true | | 2 | 2 | | 2 | 2 | 2.0 | 2.0 | two | false | | 3 | 3 | 3 | | 3 | 3.0 | 3.0 | three | true | | 4 | 4 | 4 | 4 | | 4.0 | 4.0 | four | false | | 5 | 5 | 5 | 5 | 5 | | 5.0 | five | true | | 6 | 6 | 6 | 6 | 6 | 6.0 | | six | false | | 7 | 7 | 7 | 7 | 7 | 7.0 | 7.0 | | true | | 8 | 8 | 8 | 8 | 8 | 8.0 | 8.0 | ' "\""' ; '&<> | | | 9 | 9 | 9 | 9 | 9 | | | | true | +---------+--------+---------+-------+--------+---------+----------+----------------+-----------+
class
Usage: :class:<TableScheme-classname>:<scheme-spec>
Uses an instance of a named class that implements
the uk.ac.starlink.table.TableScheme
interface
and that has a no-arg constructor.
Arguments to be passed to an instance of the named class
are appended after a colon following the classname.
For example, the specification
":class:uk.ac.starlink.table.LoopTableScheme:10
"
would return a table constructed by the code
new uk.ac.starlink.table.LoopTableScheme().createTable("10")
.
Example:
:class:uk.ac.starlink.table.LoopTableScheme:5 +---+ | i | +---+ | 0 | | 1 | | 2 | | 3 | | 4 | +---+
hapi
Usage: :hapi:<server-url>;<dataset>;start=<start>;stop=<stop>[;maxChunk=<n>][;failOnLimit=<true|false>][;<key>=<value>...]
Generates a table by interacting with a HAPI service. HAPI, the Heliophysics Data Application Programmer’s Interface is a protocol for serving streamed time series data.
In most cases it is not essential to use this scheme, since pointing the HAPI table input handler at a URL with suitable parameters will be able to read the data, but this scheme provides some added value by negotiating with the server to make sure that the correct version-sensitive request parameter names and the most efficient data stream format are used, and can split the request into multiple chunks if the service rejects the whole query as too large.
The first token in the specification is
the base URL of the HAPI service,
the second is the dataset identifier,
and others, as defined by the HAPI protocol, are supplied as
<name>=<value>
pairs,
separated by a semicolon (";
")
or an ampersand ("&
").
The start
and stop
parameters,
giving ISO-8601-like bounds for the interval requested,
are required.
Additionally, some parameters may be supplied which affect load behaviour but are not transmitted to the HAPI service. These are:
maxChunk=<n>
<n>
smaller chunks
if the server refuses to supply the whole range at once.
failOnLimit=<true|false>
Some variant syntax is permitted;
an ampersand ("&
") may be used instead of
a semicolon to separate tokens,
and the names "time.min
" and
"time.max
" may be used in place of
"start
" and "stop
".
Note that since semicolons and/or ampersands form part of the syntax, and these characters have special meaning in some contexts, it may be necessary to quote the scheme specification on the command line.
Example:
:hapi:https://vires.services/hapi;GRACE_A_MAG;start=2009-01-01T00:00:00;stop=2009-01-01T00:00:10;parameters=Latitude,Longitude +--------------------------+---------------+---------------+ | Timestamp | Latitude | Longitude | +--------------------------+---------------+---------------+ | 2009-01-01T00:00:03.607Z | -74.136357526 | -78.905620222 | | 2009-01-01T00:00:05.607Z | -74.009378676 | -78.884853931 | | 2009-01-01T00:00:06.607Z | -73.945887793 | -78.874590667 | | 2009-01-01T00:00:07.607Z | -73.882397005 | -78.864406236 | | 2009-01-01T00:00:08.607Z | -73.818903534 | -78.854396448 | +--------------------------+---------------+---------------+
Some external data services restrict access to registered users, meaning that you have to log in to use them. If STILTS encounters such a restriction and knows how to try to authenticate to the service in question, it will report on the console the URL for which the access is blocked (and possibly some additional information about the way authentication is being carried out), and ask for entry of a username and password. If authentication is successful, the resource can be retrieved, and so can any other resources from the same place, so if multiple contacts to the same service are required from the same STILTS command/session, only one login attempt should be required.
If user interaction during the command is not suitable,
it is possible to supply a username and password using the
system properties
auth.username
and auth.password
.
If both of these are set, then instead of asking on the console
for login credentials, they will be taken from the property values,
for instance
stilts -Dauth.username=foo -Dauth.password=@~/passwd.txt tpipe in=https://secret.com/data.votwould access the named resource, and if challenged by the service for authentication would supply "
foo
" for user name
and the contents of the file at ~/passwd.txt
as the password.
Note however that this feature should be used with care,
since it passes private information indiscriminately to any
service that asks for it.
Some TAP services offer optional authentication;
anonymous access is permitted, but users who log in may
benefit from restricted data or enhanced resource limits.
By default, the TAP access commands
(tapquery
,
taplint
,
tapskymatch
and
tapresume
)
will use anonymous access by default in this case.
But if you prefer to use the service in authenticated mode,
you can supply the auth=true
parameter and an attempt will be made to log in before use.
There is currently no way to log in to non-TAP VO services that provide optional authentication; however at time of writing I'm not aware of any.
Note: These authentication arrangements in STILTS are new at version 3.4-9, and rely on VO standards that are still under discussion. The behaviour and user interface may change in future releases, and at time of writing not all data services that provide authentication advertise it in a way that STILTS can work with. It is hoped that authentication interoperability will improve in future versions of STILTS and of server-side software.
Several of the tasks available in STILTS take one or more input tables, do something or other with them, and produce one or more output tables. This is a pretty obvious way to go about things, and in the most straightforward case that's exactly what happens: you name one or more input tables, specify the processing parameters, and name an output table; the task then reads the input tables from disk, does the processing and writes the output table to disk.
However, many of the tasks in STILTS allow you to do pre-processing of the input tables before the main job, post-processing of the output table after the main job, and to decide what happens to the final tabular result, without any intermediate storage of the data. Examples of the kind of pre-processing you might want to do are to rearrange the columns so that they have the right units for the main task, or replace 'magic' values such as -999 with genuine blank values; the kind of post-processing you might want to do is to sort the rows in the output table or delete some of the columns you're not interested in. As for the destination of the final table, you might want to write it to disk, but equally you might not want to store it anywhere, but only be interested in counting the number of rows, or seeing the minima/maxima of a few of the columns, or you might want to send it straight to TOPCAT or some other table viewing application for interactive analysis.
Clearly, you could achieve the same effect by running multiple applications: preprocess your original input tables to write intermediate files on disk, run the main processing application which reads those files from disk and writes a new output file, run another application to postprocess the output file and write a new final output file, and finally do something with this such as counting the rows in it or viewing it in TOPCAT. However, by doing it all within a single task instead, no intermediate results have to be stored, and the whole sequence can be very much more efficient. You can think of this (if it helps) like a Unix pipeline, except what is being streamed from the start to the end of the pipe is not bytes, but table metadata and data. In most cases, the table data is streamed through the pipeline a row at a time, meaning that the amount of memory required is small (though in some cases, for instance row sorting and crossmatching, this is not possible).
Tasks which allow this pre/post-processing, or "filtering",
have parameters with names like "cmd
" which you
use to specify processing steps.
Tasks with multiple input tables
(tmatch2
,
tskymatch2
,
tcatn
,
tjoin
)
may have parameters named icmd1
, icmd2
, ...
for preprocessing the different input tables and
ocmd
for postprocessing the output table.
tpipe
does nothing except
filtering, so there is no distinction between pre- and post-processing,
and its filter parameter is just named cmd
.
tpipe
additionally has a script
parameter which allows you to use a text file to write the
commands in, to prevent the command line getting too long.
In both cases there is a parameter named omode
which defines the "output mode", that is, what happens to the
post-processed output table that comes out of the end of the pipeline.
Section 6.1 lists the processing steps available,
and explains how to use them,
Section 6.2 and Section 6.3 describe the syntax
used in some of these filter commands for specifying columns,
and Section 6.4 describes the available output modes.
See the examples in the
command reference,
and particularly the
tpipe
examples,
for some examples putting all this together.
This section lists the filter commands which can be used for
table pipeline processing, in conjunction with cmd
-
or script
-type parameters.
You can string as many of these together as you like.
On the command line, you can repeat the cmd
(or icmd1
, or ocmd
...) parameter
multiple times, or use one cmd
parameter and
separate different filter specifiers with semicolons (";
").
The effect is the same.
It's important to note that each command in the sequence of processing steps acts on the table at that point in the sequence. Thus either of the two identical invocations:
stilts tpipe cmd='delcols 1; delcols 1; delcols 1' stilts tpipe cmd='delcols 1' cmd='delcols 1' cmd='delcols 1'has the same effect as
stilts tpipe cmd='delcols "1 2 3"'since in the first case the columns are shifted left after each one is deleted, so the table seen by each step has one fewer column than the one before. Note also the use of quotes in the latter of the examples above, which is necessary so that the
<colid-list>
of the delcols
command is interpreted as one argument not
three separate words.
The available filters are described in the following subsections.
addcol
Usage:
addcol [-after <col-id> | -before <col-id>] [-units <units>] [-ucd <ucd>] [-utype <utype>] [-xtype <xtype>] [-desc <descrip>] [-shape <n>[,<n>...][,*]] [-elsize <n>] <col-name> <expr>
Add a new column called <col-name>
defined
by the algebraic expression <expr>
.
By default the new column appears after the last column
of the table, but you can position it either before or
after a specified column using the -before
or -after
flags respectively.
The -units
, -ucd
,
-utype
, -xtype
and -desc
flags can be used
to define textual metadata values for the new column.
The -shape
flag can also be used,
but is intended only for array-valued columns,
e.g. -shape 3,3
to declare a 3x3 array.
The final entry only in the shape list
may be a "*
" character
to indicate unknown extent.
Array values with no specified shape effectively have a
shape of "*
".
The -elsize
flag may be used to specify the length
of fixed length strings; use with non-string columns
is not recommended.
Syntax for the <expr>
and <col-id>
arguments is described in the manual.
addpixsample
Usage:
addpixsample [-radius <expr-rad>] [-systems <in-sys> <pix-sys>] <expr-lon> <expr-lat> <healpix-file>
Samples pixel data from an all-sky image file
in HEALPix format.
The <healpix-file>
argument must be
the filename of a table containing HEALPix pixel data.
The URL of such a file can be used instead, but local files
are likely to be more efficient.
The <expr-lon>
and <expr-lat>
arguments
give expressions for the longitude and latitude in degrees
for each row of the input table;
this is usually just the column names.
The long/lat must usually be in the same coordinate system
as that used for the HEALPix data, so if the one is in
galactic coordinates the other must be as well.
If this is not the case, use the -systems
flag
to give the input long/lat and healpix data coordinate system
names respectively.
The available coordinate system names are:
icrs
: ICRS (Right Ascension, Declination)fk5
: FK5 J2000.0 (Right Ascension, Declination)fk4
: FK4 B1950.0 (Right Ascension, Declination)galactic
: IAU 1958 Galactic (Longitude, Latitude)supergalactic
: de Vaucouleurs Supergalactic (Longitude, Latitude)ecliptic
: Ecliptic (Longitude, Latitude)The <expr-rad>
, if present,
is a constant or expression
giving the radius in degrees over which
pixels will be averaged to obtain the result values.
Note that this averaging is somewhat approximate;
pixels partly covered by the specified disc are weighted
the same as those fully covered.
If no radius is specified, the value of the pixel covering
the central position will be used.
The <healpix-file>
file is a table
with one row per HEALPix pixel and one or more columns
representing pixel data.
A new column will be added to the output table
corresponding to each of these pixel columns.
This type of data is available in FITS tables for a number of
all-sky data sets, particularly from the
LAMBDA archive;
see for instance the page on
foreground products
(including dust emission, reddening etc)
or
WMAP 7 year data.
If the filename given does not appear to point to a file
of the appropriate format, an error will result.
Note the LAMBDA files mostly (all?) use galactic coordinates,
so coordinate conversion using the -systems
flag
may be appropriate, see above.
Syntax for the <expr-lon>
, <expr-lat>
and <expr-rad>
arguments is described in the manual.
This filter is somewhat experimental, and its usage may be changed or replaced in a future version.
Note: you may prefer to use the
pixsample
command instead.
addresolve
Usage:
addresolve <col-id-objname> <col-name-ra> <col-name-dec>
Performs name resolution on the string-valued column
<col-id-objname>
and appends two new columns
<col-name-ra>
and
<col-name-dec>
containing the resolved Right Ascension and Declination
in degrees.
Syntax for the <col-id-objname>
argument is described in Section 6.2.
UCDs are added to the new columns in a way which tries to be consistent with any UCDs already existing in the table.
Since this filter works by interrogating a remote service, it will obviously be slow. The current implementation is experimental; it may be replaced in a future release by some way of doing the same thing (perhaps a new STILTS task) which is able to work more efficiently by dispatching multiple concurrent requests.
This is currently implemented using the Simbad service operated by CDS.
addskycoords
Usage:
addskycoords [-epoch <expr>] [-inunit deg|rad|sex] [-outunit deg|rad|sex] <insys> <outsys> <col-id1> <col-id2> <col-name1> <col-name2>
Add new columns to the table representing position on the sky.
The values are determined by converting a sky position
whose coordinates are contained in existing columns.
The <col-id>
arguments give identifiers for
the two input coordinate columns
in the coordinate system named by
<insys>
, and
the <col-name>
arguments name
the two new columns,
which will be in the coordinate system named by
<outsys>
.
The <insys>
and <outsys>
coordinate system specifiers are one of
icrs
: ICRS (Right Ascension, Declination)fk5
: FK5 J2000.0 (Right Ascension, Declination)fk4
: FK4 B1950.0 (Right Ascension, Declination)galactic
: IAU 1958 Galactic (Longitude, Latitude)supergalactic
: de Vaucouleurs Supergalactic (Longitude, Latitude)ecliptic
: Ecliptic (Longitude, Latitude)The -inunit
and -outunit
flags
may be used to indicate the units of the existing coordinates
and the units for the new coordinates respectively;
use one of
degrees
, radians
or
sexagesimal
(may be abbreviated),
otherwise degrees will be assumed.
For sexagesimal, the two corresponding columns must be
string-valued in forms like hh:mm:ss.s and dd:mm:ss.s
respectively.
For certain conversions, the value specified by the
-epoch
flag is of significance.
Where significant its value defaults to 2000.0.
Syntax for the <expr>
, <col-id1>
and <col-id2>
arguments is described in the manual.
assert
Usage:
assert <test-expr> [<msg-expr>]
Check that a boolean expression is true for each row.
If the expression <test-expr>
does not
evaluate true for any row of the table, execution terminates
with an error.
As long as no error occurs, the output table is identical
to the input one.
If the <msg-expr>
parameter is supplied,
then on failure it will be evaluated and its value presented
in the error message.
The exception generated by an assertion violation is of class
uk.ac.starlink.ttools.filter.AssertException
although that is not usually obvious if you are running from
the shell in the usual way.
Syntax for the <test-expr>
and <msg-expr>
arguments is described in the manual.
badval
Usage:
badval <bad-val> <colid-list>
For each column specified in <colid-list>
any occurrence of the value <bad-val>
is replaced by a blank entry.
Syntax for the <colid-list>
argument is described in Section 6.3.
cache
Usage:
cache
Stores in memory or on disk a temporary copy of the table at this point in the pipeline. This can provide improvements in efficiency if there is an expensive step upstream and a step which requires more than one read of the data downstream. If you see an error like "Can't re-read data from stream" then adding this step near the start of the filters might help.
The output table contains no code-level reference to the input table, so this filter can also be useful when managing tables that have become deeply nested as the result of successively applying many STILTS operations.
The result of this filter is guaranteed to be random-access.
See also the random
filter, which caches only when the input table is not
random-access.
check
Usage:
check
Runs checks on the table at the indicated point in the processing pipeline. This is strictly a debugging measure, and may be time-consuming for large tables.
clearparams
Usage:
clearparams <pname> ...
Clears the value of one or more named parameters.
Each of the <pname>
values supplied may be
either a parameter name or a simple wildcard expression
matching parameter names. Currently the only wildcarding
is a "*
" to match any sequence of characters.
clearparams *
will clear all the parameters
in the table.
It is not an error to supply <pname>
s
which do not exist in the table - these have no effect.
collapsecols
Usage:
collapsecols [-[no]keepscalars] <array-colname> <col-id0> <ncol>
Adds a new array-valued column
by using the values from a specified range of scalar columns
as array elements.
The new column is named <array-colname>
,
and produced from the sequence of <ncol>
scalar columns starting with <col-id0>
.
The array type of the output column is determined by the type
of the first input column (<col-id0>
).
If it is of type Double
,
the output array column will be a double[]
array,
and similarly for types Long
, Integer
,
Float
and Boolean
.
Other integer types are currently mapped to int[]
,
and object types, e.g. String
,
to the corresponding array type.
Array elements for null or mistyped input values
are mapped to NaN for floating point types, but
note that they currently
just turn into zeros for integer array types
and false
for boolean.
By default the scalar columns that have been used are removed
from the output table and the new column replaces them
at the same position.
However, if you supply the -keepscalars
flag
they will be retained alongside the new array column
(the new column will appear just after the run of scalar
columns).
This filter does the opposite of
explodecols
.
Syntax for the <col-id0>
argument is described in Section 6.2.
colmeta
Usage:
colmeta [-name <name>] [-units <units>] [-ucd <ucd>] [-utype <utype>] [-xtype <xtype>] [-desc <descrip>] [-shape <n>[,<n>...][,*]] [-elsize <n>] <colid-list>
Modifies the metadata of one or more columns.
Some or all of the name, units, ucd, utype, xtype, description,
shape and elementsize of
the column(s), identified by <colid-list>
can be set by using some or all of the listed flags.
Typically, <colid-list>
will simply be
the name of a single column.
The -name
, -units
,
-ucd
, -utype
, -xtype
and -desc
flags just take textual arguments.
The -shape
flag can also be used,
but is intended only for array-valued columns,
e.g. -shape 3,3
to declare a 3x3 array.
The final entry only in the shape list
may be a "*
" character
to indicate unknown extent.
Array values with no specified shape effectively have a
shape of "*
".
The -elsize
flag may be used to specify the length
of fixed length strings; use with non-string columns
is not recommended.
Syntax for the <colid-list>
argument is described in Section 6.3.
constcol
Usage:
constcol [-noparam] [-acceptnull] [-[no]parallel] [<colid-list>]
Identifies columns with constant values. Such columns are removed from the table and by default their fixed value is added to the table as a table parameter with the same name as the removed column. Such columns may have scalar or array values.
The -noparam
flag
controls whether constant columns identified are recorded instead
as table parameters (per-table metadata items).
By default they are, but supplying -noparam
means these values will just be discarded.
The -acceptnull
flag
controls how blank values in candidate columns are treated.
By default, all values in a column must be strictly the same
for a column to be identified as constant value,
but if -acceptnull
is supplied
then a column will be treated as constant if all its entries
are either a single fixed value or blank.
The -[no]parallel
flag
controls whether processing is done using multithreading
for large tables.
The <colid-list>
gives the columns to be assessed by this filter;
if not supplied, all columns will be examined.
Syntax for the <colid-list>
argument is described in Section 6.3.
delcols
Usage:
delcols <colid-list>
Delete the specified columns. The same column may harmlessly be specified more than once.
Syntax for the <colid-list>
argument is described in Section 6.3.
every
Usage:
every [-exact|-approx] <step>
Include only every <step>
'th row in the
result, starting with the first row.
The optional -approx
/-exact
argument
controls whether the selection needs to be exact;
in some cases an approximate calculation can take advantage
of parallelism where an exact one cannot.
explodeall
Usage:
explodeall [-ifndim <ndim>] [-ifshape <dims>]
Replaces any columns which is an N-element arrays with N scalar columns. Only columns with fixed array sizes are affected. The action can be restricted to only columns of a certain shape using the flags.
If the -ifndim
flag is used, then only columns
of dimensionality <ndim>
will be exploded.
<ndim>
may be 1, 2, ....
If the -ifshape
flag is used, then only columns
with a specific shape will be exploded;
<dims>
is a space- or comma-separated list
of dimension extents, with the most rapidly-varying first,
e.g. '2 5
' to explode all 2 x 5 element array
columns.
explodecols
Usage:
explodecols <colid-list>
Takes a list of specified columns which represent N-element
arrays and replaces each one with N scalar columns.
Each of the columns specified by <colid-list>
must have a fixed-length array type,
though not all the arrays need to have the same number
of elements.
This filter does the opposite of
collapsecols
.
Syntax for the <colid-list>
argument is described in Section 6.3.
fixcolnames
Usage:
fixcolnames
Renames all columns and parameters in the input table so that they have names which have convenient syntax for STILTS. For the most part this means replacing spaces and other non-alphanumeric characters with underscores. This is a convenience which lets you use column names in algebraic expressions and other STILTS syntax.
Additionally, column names are adjusted if necessary to ensure
that they are all unique when compared case-insensitively.
If the names are all unique to start with
then no changes are made,
but if for instance two columns exist with names
gMag
and GMag
,
one of them will be altered
(for instance to GMag_1
).
group
Usage:
group [-[no]parallel] <key> [<key> ...] [<aggcol> ...]
Calculates aggregate functions on groups of rows.
This does the same job as a
SELECT ... GROUP BY
statement
with aggregate functions in ADQL/SQL.
The functionality is identical to that of the
tgroup
command,
and the meaning and syntax of the <key>
and
<aggcol>
words are identical too,
except that the <aggcol>
delimiter character
is an at sign ("@
")
rather than a semicolon (";
"),
so a group entry count can be added for instance using
an <aggcol>
like
"null@count
".
The <aggcol>
arguments
are distinguished by the fact that they contain
at least one delimiter ("@
").
See the tgroup
documentation
for a full explanation of the syntax and functionality.
The syntax here is rather cramped,
so in many cases it will be more comfortable to use
tgroup
instead,
but this filter is provided for cases where that may be
more convenient.
head
Usage:
head <nrows>
Include only the first <nrows>
rows of
the table.
If the table has fewer than <nrows>
rows
then it will be unchanged.
healpixmeta
Usage:
healpixmeta [-level <n>] [-implicit|-column <col-id>] [-csys C|G|E] [-nested|-ring]
Adjusts the table metadata items that describe how HEALPix pixel data is encoded in the table.
Zero or more of the following flags may be supplied:
-level <n>
:
Defines the HEALPix level;
the sky is split into 12*4^n pixels.
This quantity is equal to logarithm base 2 of NSIDE.-implicit
:
Declares that pixel indices are implicit, so that row
i represents HEALPix pixel index i.
The table should have 12*4^level rows in this case.
Not to be used with -column
.-column <col-id>
:
Declares that the column identified
contains the (0-based) HEALPix pixel index.
Not to be used with -implicit
.-csys C|G|E
:
Declares the sky coordinate system to which the HEALPix
pixels apply:
Celestial(=equatorial),
Galactic or
Ecliptic.
Some applications assume Galactic
if this is not specified.-nested
:
Declares that the NESTED ordering scheme is in use.
Not to be used with -ring
.-ring
:
Declares that the RING ordering scheme is in use.
Not to be used with -nested
.The effect of this filter is to write, or overwrite, certain special table parameters (per-table metadata) that STIL uses to describe how HEALPix pixel information is encoded in a table, specifically the HEALPix level, the column containing pixel index, the ordering scheme, and the sky coordinate system. Adding these parameters doesn't do anything on its own, but some of the STIL I/O handlers recognise these parameters, and they affect how the table will be formatted for output. In particular, if you set these parameters and then output to FITS format, the output table will contain headers defined by the HEALPix-FITS serialization format which is understood by several other applications to describe HEALPix maps. If you write to VOTable format, the metadata will only be recognised by other STIL-based applications but it means that if you, e.g., load the table into TOPCAT and then write it out again as FITS, the HEALPix information should be preserved.
When writing tables marked up like this to FITS,
you have two options.
If you write to one of the "normal" FITS formats
(e.g. fits
, fits-basic
)
then suitable headers will be added;
in this case if an explicit pixel index column is used it must be
the first column, and should be named "PIXEL".
This may be enough for other applications to recognise
the HEALPix metadata.
However, if you use the special fits-healpix
format
more efforts will be made to conform to the HEALPix-FITS
convention, for instance moving and renaming
the explicit pixel index column if required.
The table parameters affected by this filter are:
STIL_HPX_LEVEL
,
STIL_HPX_ISNEST
,
STIL_HPX_COLNAME
,
STIL_HPX_CSYS
.
Note these are not defined by any standard,
they are defined and used only by STILTS and related
applications (TOPCAT).
Syntax for the <col-id>
argument is described in Section 6.2.
keepcols
Usage:
keepcols <colid-list>
Select the columns from the input table which will be
included in the output table.
The output table will include only those columns listed in
<colid-list>
, in that order.
The same column may be listed more than once,
in which case it will appear in the output table more than once.
Syntax for the <colid-list>
argument is described in Section 6.3.
meta
Usage:
meta [<item> ...]
Provides information about the metadata for each column. This filter turns the table sideways, so that each row of the output corresponds to a column of the input. The columns of the output table contain metadata items such as column name, units, UCD etc corresponding to each column of the input table.
By default the output table contains columns for all metadata items for which any of the columns have non-blank values.
However, the output may be customised by supplying
one or more <item>
headings,
in which case exactly those columns will appear,
regardless of whether they have entries.
It is not an error
to specify an item for which no metadata exists in any of
the columns (such entries will result in empty columns).
Some of the metadata items commonly found are:
Index
: Position of column in table
Name
: Column name
Class
: Data type of objects in column
Shape
: Shape of array values
ElSize
: Size of each element in column (mostly useful for strings)
Units
: Unit string
Description
: Description of data in the column
UCD
: Unified Content Descriptor
Utype
: Type in data model
Xtype
: Extended data type
CoosysSystem
: Sky coordinate system name from COOSYS
CoosysEpoch
: Sky epoch from COOSYS
CoosysRefposition
: Reference position from COOSYS
CoosysEquinox
: Sky equinox from COOSYS
TimesysTimeorigin
: Time origin from TIMESYS
TimesysTimescale
: Timescale from TIMESYS
TimesysRefposition
: Ref position from TIMESYS
STIL_HPX_LEVEL
: Level of HEALPix pixels contained in the table (nside=2^level)
STIL_HPX_ISNEST
: True for NEST indexation scheme, False for RING
STIL_HPX_COLNAME
: Name of the table column containing HEALPix index; null value or empty string indicates implicit
STIL_HPX_CSYS
: 'C'=celestial/equatorial, 'G'=galactic, 'E'=ecliptic
Any table parameters of the input table are propagated to the output one.
progress
Usage:
progress
Monitors progress by displaying the number of rows processed so far on the terminal (standard error). This number is updated every second or thereabouts; if all the processing is done in under a second you may not see any output. If the total number of rows in the table is known, an ASCII-art progress bar is updated, otherwise just the number of rows seen so far is written.
Note under some circumstances progress may appear to
complete before the actual work of the task is done
since part of the processing involves slurping up the whole
table to provide random access on it.
In this case, applying the
cache
upstream may help.
random
Usage:
random
Ensures that random access is available on this table. If the table currently has random access, it has no effect. If only sequential access is available, the table is cached so that downstream steps will see the cached, hence random-access, copy.
randomview
Usage:
randomview
Ensures that steps downstream only use random access methods for table access. If the table is sequential only, this will result in an error. Only useful for debugging.
repeat
Usage:
repeat [-row|-table] <count>
Repeats the rows of a table multiple times to produce
a longer table.
The output table will have <count>
times
as many rows as the input table.
The optional flag determines the sequence of the output rows.
If <count>
=2 and there are three rows,
the output sequence will be 112233 for -row
and 123123 for -table
.
The default behaviour is currently -table
.
The <count>
value will usually
be a constant integer value, but it can be an expression
evaluated in the context of the table.
replacecol
Usage:
replacecol [-name <name>] [-units <units>] [-ucd <ucd>] [-utype <utype>] [-xtype <xtype>] [-desc <descrip>] <col-id> <expr>
Replaces the content of a column with the value of an
algebraic expression.
The old values are discarded in favour of the result of
evaluating <expr>
.
You can specify the metadata for the new column using the
-name
, -units
, -ucd
,
-utype
, -xtype
and -desc
flags; for any of these items which you
do not specify, they will take the values from the column
being replaced.
It is legal to reference the replaced column in the
expression,
so for example "replacecol pixsize pixsize*2
"
just multiplies the values in column pixsize
by 2.
Syntax for the <col-id>
and <expr>
arguments is described in the manual.
replaceval
Usage:
replaceval <old-val> <new-val> <colid-list>
For each column specified in <colid-list>
any instance of <old-val>
is replaced by
<new-val>
.
The value string 'null
' can be used for either
<old-value>
or <new-value>
to indicate a blank value
(but see also the badval
filter).
Syntax for the <colid-list>
argument is described in Section 6.3.
rowrange
Usage:
rowrange <first> <last>|+<count>
Includes only rows in a given range.
The range can either be supplied as
"<first> <last>
",
where row indices are inclusive, or
"<first> +<count>
".
In either case, the first row is numbered 1.
Thus, to get the first hundred rows, use either
"rowrange 1 100
" or
"rowrange 1 +100
"
and to get the second hundred, either
"rowrange 101 200
" or
"rowrange 101 +100
"
select
Usage:
select <expr>
Include in the output table only rows for which the
expression <expr>
evaluates to true.
<expr>
must be an expression which
evaluates to a boolean value (true/false).
Syntax for the <expr>
argument is described in Section 10.
seqview
Usage:
seqview
Ensures that steps downstream see the table as sequential access. Any attempts at random access will fail. Only useful for debugging.
setparam
Usage:
setparam [-type byte|short|int|long|float|double|boolean|string] [-desc <descrip>] [-unit <units>] [-ucd <ucd>] [-utype <utype>] [-xtype <xtype>] <pname> <pexpr>
Sets a named parameter in the table to a given value.
The parameter named <pname>
is set
to the value <pexpr>
,
which may be a literal value or an expression involving
mathematical operations and other parameter names
(using the param$<name>
syntax).
By default, the data type of the parameter is determined
by the type of the supplied expression,
but this can be overridden using the -type
flag.
The parameter description, units, UCD, Utype and Xtype attributes
may optionally be set using the other flags.
sort
Usage:
sort [-down] [-nullsfirst] [-[no]parallel] <key-list>
Sorts the table according to the value of one or more
algebraic expressions.
The sort key expressions appear,
as separate (space-separated) words,
in <key-list>
; sorting is done on the
first expression first, but if that results in a tie then
the second one is used, and so on.
Each expression must evaluate to a type that
it makes sense to sort, for instance numeric.
If the -down
flag is used, the sort order is
descending rather than ascending.
Blank entries are by default considered to come at the end
of the collation sequence, but if the -nullsfirst
flag is given then they are considered to come at the start
instead.
By default, sorting is done sequentially for small tables
and in parallel for large tables, but this can be controlled
with the -parallel
or -noparallel
flag.
Syntax for the <key-list>
argument is described in Section 10.
sorthead
Usage:
sorthead [-tail] [-down] [-nullsfirst] <nrows> <key-list>
Performs a sort on the table according to the value of
one or more algebraic expressions, retaining only
<nrows>
rows at the head
of the resulting sorted table.
The sort key expressions appear,
as separate (space-separated) words,
in <key-list>
; sorting is done on the
first expression first, but if that results in a tie then
the second one is used, and so on.
Each expression must evaluate to a type that
it makes sense to sort, for instance numeric.
If the -tail
flag is used, then the
last <nrows>
rows rather than the first
ones are retained.
If the -down
flag is used the sort order is
descending rather than ascending.
Blank entries are by default considered to come at the end
of the collation sequence, but if the -nullsfirst
flag is given then they are considered to come at the start
instead.
This filter is functionally equivalent to using
sort
followed by head
,
but it can be done in one pass and is usually cheaper
on memory and faster, as long as <nrows>
is significantly lower than the size of the table.
Syntax for the <key-list>
argument is described in Section 10.
stats
Usage:
stats [-[no]parallel] [-qapprox|-qexact] [<item> ...]
Calculates statistics on the data in the table. This filter turns the table sideways, so that each row of the output corresponds to a column of the input. The columns of the output table contain statistical items such as mean, standard deviation etc corresponding to each column of the input table.
By default the output table contains columns for the following items:
Name
: Column name
Mean
: Average
StDev
: Population Standard deviation
Minimum
: Numeric minimum
Maximum
: Numeric maximum
NGood
: Number of non-blank cells
However, the output may be customised by supplying one or more
<item>
headings. These may be selected
from the above as well as the following:
NBad
: Number of blank cells
Variance
: Population Variance
SampStDev
: Sample Standard Deviation
SampVariance
: Sample Variance
MedAbsDev
: Median Absolute Deviation
ScMedAbsDev
: Median Absolute Deviation * 1.4826
Skew
: Gamma 1 skewness measure
Kurtosis
: Gamma 2 peakedness measure
Sum
: Sum of values
MinPos
: Row index of numeric minimum
MaxPos
: Row index of numeric maximum
Cardinality
: Number of distinct values in column; values >100 ignored
Median
: Middle value in sequence
Quartile1
: First quartile
Quartile2
: Second quartile
Quartile3
: Third quartile
ArrayNGood
: Per-element non-blank counts for fixed-length array columns
ArraySum
: Per-element sums for fixed-length array columns
ArrayMean
: Per-element means for fixed-length array columns
ArrayStDev
: Per-element population standard deviation for fixed-length array columns
Q.25
: First quartile
Q.625
: Fifth octile
Any parameters of the input table are propagated to the output one.
The -qapprox
or -qexact
flag controls how quantiles are calculated.
With -qexact
they are calculated exactly,
but this requires memory usage scaling with the number of rows.
If the -qapprox
flag is supplied,
an method is used which is typically slower and produces only
approximate values, but which will work in fixed memory
and so can be used for arbitrarily large tables.
By default, exact calculation is used.
These flags are ignored if neither quantiles nor the MAD
are being calculated
The -noparallel
flag may be supplied to inhibit
multi-threaded statistics accumulation.
Calculation is done in parallel by default if multi-threaded
hardware is available, and it's usually faster.
tablename
Usage:
tablename <name>
Sets the table's name attribute to the given string.
tail
Usage:
tail <nrows>
Include only the last <nrows>
rows
of the table.
If the table has fewer than <nrows>
rows
then it will be unchanged.
transpose
Usage:
transpose [-namecol <col-id>]
Transposes the input table so that columns become rows
and vice versa.
The -namecol
flag can be used to specify a column
in the input table which will provide the column names for
the output table.
The first column of the output table will contain the
column names of the input table.
Syntax for the <col-id>
argument is described in Section 6.2.
uniq
Usage:
uniq [-count] [<colid-list>]
Eliminates adjacent rows which have the same values. If used with no arguments, then any row which has identical values to its predecessor is removed.
If the <colid-list>
parameter is given
then only the values in the specified columns must be equal
in order for the row to be removed.
If the -count
flag is given, then an additional
column with the name DupCount will be
prepended to the table giving a count of the number of duplicated
input rows represented by each output row. A unique row
has a DupCount value of 1.
Syntax for the <colid-list>
argument is described in Section 6.3.
If an argument is specified in the help text for a
command with the symbol <col-id>
it means you must give a string which identifies one of the
existing columns in a table.
There are several ways you can specify a column in this context:
Tip: if counting which column has which index is giving you a
headache, running tpipe
with omode=meta
or
omode=stats
on the table may help.
ucd$<ucd-spec>
". Depending on the version of
UCD scheme used, UCDs can contain various punctuation marks such
as underscores, semicolons and dots; for the purpose of this syntax
these should all be represented as underscores ("_
").
So to identify a column which has the UCD "phot.mag;em.opt.R
",
you should use the identifier "ucd$phot_mag_em_opt_r
".
Matching is not case-sensitive. Futhermore, a trailing underscore
acts as a wildcard, so that the above column could also be referenced
using the identifier "ucd$phot_mag_
". If multiple
columns have UCDs which match the given identifer, the first one
will be used.
utype$<utype-spec>
".
Utypes may contain various punctuation marks such as colons and dots;
for the purpose of this syntax these should all be represented as
underscores ("_
").
So to identify a column which has the Utype
"ssa:Access.Format
",
you should use the identifier
"utype$ssa_Access_format
".
Matching is not case-sensitive.
If multiple columns have Utypes which match the given identifier,
the first one will be used.
If an argument is specified in the help text for a command
with the symbol <colid-list>
it means you
must give a string which identifies a list of zero, one or more
of the existing columns in a table.
The string you specify is a separated into separate tokens by
whitespace, which means that you will normally
have to surround it in single or double quotes to ensure
that it is treated as a single argument and not several of them.
Each token in the <colid-list>
string may be one of
the following:
<name>
,
<index>
,
$<index>
,
ucd$<ucd-expr>
or
utype$<utype-expr>
.
*
'
which matches any sequence of characters. To match an unknown
sequence at the start or end of the string an asterisk must be
given explicitly. Other than that, matching is usually
case insensitive. The order of the expanded list is the same
as the order in which the columns appear in the table.
Thus "col*
" will match columns named
col1
, Column2
and COL_1024
,
but not decOld
.
"*MAG*
" will match columns named
magnitude
, ABS_MAG_U
and JMAG
.
"*
" on its own
expands to a list of all the columns of the table in order.
<first-colid>-<last-colid>
,
where the syntax for
<first-colid>
and <last-colid>
is a Single Column Identifier as above
(as long as it doesn't contain a "-
" character).
The range is inclusive, so the first and last column are both included.
The <first-colid>
or <last-colid>
part (but not both) may be omitted, to indicate all the columns from the
start or all the columns to the end, respectively.
So "RA-PARALLAX
" means all the columns starting with
the one named RA
and ending with
the one named PARALLAX
(inclusive);
"1-100
" or "$1-$100
"
means the first hundred columns in the table;
"101-
" or "$101-
"
means all the columns apart from the first hundred,
"PARALLAX-
" means PARALLAX
and all
subsequent columns, etc.
Specifying a list which contains a given column more than once is not usually an error, but what effect it has depends on the function you are executing.
This section lists the output modes which can be used as
the value of the omode
parameter of
tpipe
and other commands.
Typically, having produced a result table by pipeline processing
an input one, you will write it out by specifying
omode=out
(or not using the omode
parameter at all -
out
is the default). However, you can do other things
such as calculate statistics, display metadata, etc. In some of
these cases, additional parameters are required. The different
output modes, with their associated parameters, are described in
the following subsections.
cgi
Usage:
omode=cgi ofmt=<out-format>
Writes a table to standard output in a way suitable for
use as output from a CGI (Common Gateway Interface) program.
This is very much like out
mode
but a short CGI header giving the MIME Content-Type
is prepended to the output
Additional parameters for this output mode are:
ofmt = <out-format>
(String)
[Default: votable
]
checksum
Usage:
omode=checksum
Calculates a checksum from all the data in the table. The checksum is written to standard output in hexadecimal; row and column counts are also written.
If two tables have the same checksum it is extremely likely that they contain the same cell data. If they have a different checksum, their cell data differs. By default, the checksum implementation uses Adler32, which is fast but not cryptographically secure.
count
Usage:
omode=count
Counts the number of rows and columns and writes the result to standard output.
discard
Usage:
omode=discard
Reads all the data in the table in sequential mode
and discards it.
May be useful in conjunction with
the assert
filter.
gui
Usage:
omode=gui
Displays the table in a scrollable window.
meta
Usage:
omode=meta
Prints the table metadata to standard output. The name and type etc of each column is tabulated, and table parameters are also shown.
See the meta
filter for more flexible output of table metadata.
out
Usage:
omode=out out=<out-table> ofmt=<out-format>
Writes a new table.
Additional parameters for this output mode are:
out = <out-table>
(TableConsumer)
[Default: -
]
ofmt = <out-format>
(String)
(auto)
"
(the default),
then the output filename will be
examined to try to guess what sort of file is required
usually by looking at the extension.
If it's not obvious from the filename what output format is
intended, an error will result.
[Default: (auto)
]
plastic
Usage:
omode=plastic transport=string|file client=<app-name>
Broadcasts the table to any registered Plastic-aware applications. PLASTIC, the PLatform for AStronomical Tool InterConnection, is a tool interoperability protocol. A Plastic hub must be running in order for this to work.
Additional parameters for this output mode are:
transport = string|file
(String)
string
:
VOTable serialized as a string and passed as a call parameter
(ivo://votech.org/votable/load
).
Not suitable for very large files.file
:
VOTable written to a temporary file and the filename passed as
a call parameter
(ivo://votech.org/votable/loadFromURL
).
The file ought to be deleted once it has been loaded.
Not suitable for inter-machine communication.null
) then a decision will
be taken based on the apparent size of the table.
client = <app-name>
(String)
samp
Usage:
omode=samp format=<value> client=<name-or-id>
Sends the table to registered SAMP-aware applications subscribed to a suitable table load MType. SAMP, the Simple Application Messaging Protocol, is a tool interoperability protocol. A SAMP Hub must be running for this to work.
Additional parameters for this output mode are:
format = <value>
(String[])
votable
:
use MType table.load.votable
fits
:
use MType table.load.fits
table.load.format
.
[Default: votable fits
]
client = <name-or-id>
(String)
stats
Usage:
omode=stats
Calculates and displays univariate statistics for each of the numeric columns in the table. The following entries are shown for each column as appropriate:
See the stats
filter
for more flexible statistical calculations.
topcat
Usage:
omode=topcat
Attempts to display the output table directly in TOPCAT. If a TOPCAT instance is already running on the local host, an attempt will be made to open the table in that. A variety of mechanisms are used to attempt communication with an existing TOPCAT instance. In order:
-soap
flag,
may be limitations on table size)topcat -exthub
)If large tables are involved, starting TOPCAT with the
-disk
flag is probably a good idea.
tosql
Usage:
omode=tosql protocol=<jdbc-protocol> host=<value> db=<db-name> dbtable=<table-name> write=create|dropcreate|append user=<username> password=<passwd>
Writes a new table to an SQL database.
You need the appropriate JDBC drivers and
-Djdbc.drivers
set as usual
(see Section 3.4).
Additional parameters for this output mode are:
protocol = <jdbc-protocol>
(String)
mysql
,
and for PostgreSQL's driver it is postgresql
.
For other drivers, you may have to consult the driver
documentation.
host = <value>
(String)
[Default: localhost
]
db = <db-name>
(String)
The value of this parameter forms the last part
of the JDBC connection URL.
That means that for some JDBC drivers,
you can append parameter specifications to it
to affect the way the connection is made to the database, e.g.
"db=some_db?useSSL=false
"
for MySQL's Connector/J.
dbtable = <table-name>
(String)
write = create|dropcreate|append
(WriteMode)
create
: Creates a new table before writing. It is an error if a table of the same name already exists.dropcreate
: Creates a new database table before writing. If a table of the same name already exists, it is dropped first.append
: Appends to an existing table. An error results if the named table has the wrong structure (number or types of columns) for the data being written.[Default: create
]
user = <username>
(String)
[Default: mbt
]
password = <passwd>
(String)
STILTS offers flexible and efficient facilities for crossmatching tables. Crossmatching is identifying different rows, which may be in the same or different tables, that refer to the same item. In an astronomical context such an item is usually, though not necessarily, an astronomical source or object. This operation corresponds to what in database terminology is called a join.
There are various complexities to specifying such a match. In the first place you have to define what is the condition that must be satisfied for two rows to be considered matching. In the second place you must decide what happens if, for a given row, more than one match can be found. Finally, you have to decide what to do having worked out what the matched rows are; the result will generally be presented as a new output table, but there are various choices about what columns and rows it will consist of. Some of these issues are discussed in this section, and others in the reference sections on the tools themselves in Appendix B.
Matching can in general be a computationally intensive process.
The algorithm used by the tmatch*
tasks in STILTS,
except in pathological cases, scales as O(N log(N)) or thereabouts,
where N is the total number of rows in all the tables being matched.
No preparation (such as sorting) is required on the tables prior to
invoking the matching operation.
It is reasonably fast; for instance an RA, Dec positional match
of two 105-row catalogues takes of the order of 60 seconds
on current (2005 laptop) hardware. Attempting matches with large tables can
lead to running out of memory; the calculation just mentioned required a
java heap size of around 200Mb (-Xmx200M
).
In the current release of STILTS the following tasks are provided for crossmatching between local tables:
tmatch2
tskymatch2
tmatch2
provided for convenience when the
full generality is not required.
tmatch1
tmatchn
tjoin
Determining whether one row represents the same item as another is done by comparing the values in certain of their columns to see if they are the same or similar. The most common astronomical case is to say that two rows match if their celestial coordinates (right ascension and declination) are within a given small radius of each other on the sky. There are other possibilities; for instance the coordinates to compare may be in a Cartesian space, or have a higher (or lower) dimensionality than two, or the match may be exact rather than within an error radius....
If you just need to match two tables according to sky position
with fixed errors you are recommended to use the simplified
tskymatch2
task.
For other cases, this section describes how to specify much more
flexible match criteria for use with
tmatch1
, tmatch2
or tmatchn
by setting the following parameters:
matcher
params
values*
tmatch2
you must specify both
values1
and values2
.
tuning
For example, suppose we wish to locate objects in two tables which are
within 3 arcseconds of each other on the sky. One table has columns
RA and DEC which give coordinates in degrees, and the other has columns
RArad and DECrad which give coordinates in radians. These are the
arguments which would be used to tell tmatch2
what the match
criteria are:
matcher=sky params=3 values1='RA DEC' values2='radiansToDegrees(RArad) radiansToDegrees(DECrad)'It is clearly important that corresponding values are comparable (in the same units) between the tables being matched, and in geometrically sensitive cases such as matching on the sky, it's important that they are the units expected by the matcher as well. To determine what those units are, either consult the roster below, or run the following command:
stilts tmatch2 help=matcherwhich will tell you about all the known matchers and their associated
params
, values*
and tuning
parameters.
The following subsections list the basic matcher
types and the
requirements of their associated params
,
values*
and tuning
parameters.
The units of the required values are given where significant.
sky
: Sky Matching
matcher=sky values*='<ra/deg> <dec/deg>' params='<max-error/arcsec>' tuning='<healpix-k>'
values*
:
ra/deg
: Right Ascensiondec/deg
: Declinationparams
:
max-error/arcsec
: Maximum separation along a great circletuning
:
healpix-k
: Controls sky pixel size. Legal range 0 - 29. 0 is 60deg, 20 is 0.2".
The sky
matcher compares positions on the celestial sphere
with a fixed error radius.
Rows are considered to match when the two (ra
, dec
)
positions are within max-error
arcseconds of each other
along a great circle.
In fact this matching is not restricted to equatorial coordinates -
the ra
and dec
parameters may represent
any longitude-like and latitude-like coordinates in degrees,
since the spherical geometry for the matching is unchanged under
such transformations.
skyerr
:
Sky Matching with Per-Object Errors
matcher=skyerr values*='<ra/deg> <dec/deg> <error/arcsec>' params='<scale/arcsec>' tuning='<healpix-k>'
values*
:
ra/deg
: Right Ascensiondec/deg
: Declinationerror/arcsec
: Per-object error radius along a great circleparams
:
scale/arcsec
: Rough average of per-object error distance; just used for tuning to set default pixel sizetuning
:
healpix-k
: Controls sky pixel size. Legal range 0 - 29. 0 is 60deg, 20 is 0.2".
The skyerr
matcher compares positions on the celestial
sphere using error radii which can be different for each row.
Rows are considered to match when the separation between the
two ra
, dec
positions is no larger than
the sum of the two per-row error
values.
The scale
parameter should be a rough average value
of the error distances. It is used only to set a sensible default for
healpix-k
tuning parameter, and its value does not affect
the result. If you set healpix-k
directly, its value is ignored.
As with sky
matching,
other longitude/latitude coordinate pairs may be used in place of
right ascension and declination.
A variant form skyerr_q
does the same thing but combines the two per-row error
values
in quadrature rather than by summation.
In that case the separation between the two positions
must be no larger than sqrt(error12+error22).
Note: the semantics of this matcher have changed
slightly at version 2.4 of STILTS.
In earlier versions the single parameter was named max-error
and provided an additional constraint on the maximum accepted separation
between matched objects. For most uses, the old and new behaviours are
expected to give the same results, but in cases of difference, the new
behaviour is more likely what you want.
skyellipse
:
Sky Matching of Elliptical Regions
matcher=skyellipse values*='<ra/deg> <dec/deg> <primary-radius/arcsec> <secondary-radius/arcsec> <position-angle/deg>' params='<scale/arcsec>' tuning='<healpix-k>'
values*
:
ra/deg
: Right ascension of centredec/deg
: Declination of centreprimary-radius/arcsec
: Length of ellipse semi-major axissecondary-radius/arcsec
: Length of ellipse semi-minor axisposition-angle/deg
: Position angle - measured from north pole to primary axis, in direction of positive RAparams
:
scale/arcsec
: Rough average of ellipse major radius; just used for tuning to set default pixel sizetuning
:
healpix-k
: Controls sky pixel size. Legal range 0 - 29. 0 is 60deg, 20 is 0.2".
The skyellipse
matcher compares elliptical regions
on the sky for overlap. Each row has to provide five values,
giving the centre, the major and minor radii, and the position angle
of an ellipse.
Rows are considered to match if there is any overlap between the ellipses.
The goodness of match is a normalised generalisation of the symmetrical case
used by the skyerr
matcher, in which the best possible match
is two concentric ellipses, and the worst allowable match is when the
circumferences just touch.
The calculations are approximate since in some cases they rely on projecting the ellipses onto a Cartesian tangent plane before evaluating the match, so for larger ellipses the criterion will be less exact. For objects the size of most observed stars or galaxies, this approximation is not expected to be problematic.
The scale
parameter must be supplied, and should be
a rough average value of the major radii. it is used only to set
a sensible default for the healpix-k
tuning parameter,
and its value does not affect the result.
If you set healpix-k
directly,
the value of scale
is ignored.
sky3d
: Spherical Polar Matching
matcher=sky3d values*='<ra/deg> <dec/deg> <distance>' params='<error/units of distance>' tuning='<bin-factor>'
values*
:
ra/deg
: Right Ascensiondec/deg
: Declinationdistance
: Distance from originparams
:
error/units of distance
: Maximum Cartesian separation for matchtuning
:
bin-factor
: Scaling factor to adjust bin size; larger values mean larger bins
The sky3d
matcher compares positions in the volume of
the sky taking account of distance from the observer.
The position in three-dimensional space is calculated for each
row using the ra
, dec
and distance
as spherical polar coordinates, where distance
is the
distance from the observer along the line of sight.
Rows are considered to match when their positions in this space are
within error
units of each other.
The units of error
are the same as those of
distance
.
As with sky
matching,
other longitude/latitude coordinate pairs may be used in place of
right ascension and declination.
exact
: Exact Matching
matcher=exact values*='<matched-value>'
values*
:
matched-value
: Value for exact match
The exact
matcher compares arbitrary key values
for exact equality.
Rows are considered to match only if the values in their
matched-value
columns are exactly the same.
These values can be strings, numbers, or anything else.
A blank value never matches, not even with another blank one.
Since the params
parameter holds no values,
it does not have to be specified.
Note that the values must also be of the same type, so for instance
a Long (64-bit) integer value will not match an Integer (32-bit) value.
1d
, 2d
, ...:
Isotropic Cartesian Matching
matcher=1d values*='<x>' params='<error>' tuning='<bin-factor>'
values*
:
x
: Cartesian co-ordinate #1params
:
error
: Maximum Cartesian separation for matchtuning
:
bin-factor
: Scaling factor to adjust bin size; larger values mean larger bins
matcher=2d values*='<x> <y>' params='<error>' tuning='<bin-factor>'
values*
:
x
: Cartesian co-ordinate #1y
: Cartesian co-ordinate #2params
:
error
: Maximum Cartesian separation for matchtuning
:
bin-factor
: Scaling factor to adjust bin size; larger values mean larger bins
The 1d
matcher compares positions in 1-dimensional
Cartesian space.
Rows are considered to match if their x
column
values differ by no more than error
.
The 2d
matcher compares postions in 2-dimensional
Cartesian space.
Rows are considered to match if the difference in their
(x
,y
) positions reckoned using
Pythagoras is less than error
.
Matching in any number of Cartesian dimensions can be done by extending this syntax in the obvious way.
2d_anisotropic
, ...:
Anisotropic Cartesian Matching
matcher=2d_anisotropic values*='<x> <y>' params='<error-in-x> <error-in-y>' tuning='<bin-factor>'
values*
:
x
: Cartesian co-ordinate #1y
: Cartesian co-ordinate #2params
:
error-in-x
: Axis length of error ellipse in Cartesian co-ordinate #1 directionerror-in-y
: Axis length of error ellipse in Cartesian co-ordinate #2 directiontuning
:
bin-factor
: Scaling factor to adjust bin size; larger values mean larger bins
The 2d_anisotropic
matcher compares positions in
2-dimensional Cartesian space using an anisotropic metric.
Rows are considered to match if their (x
,y
)
positions fall within an error ellipse with axis lengths
error-in-x
, error-in-y
of each other.
This kind of match will typically be used for non-'spatial' spaces,
for instance (magnitude,redshift) space, in which the metrics along
different axes are not related to each other.
Matching in any number of dimensions of Cartesian space using an anisotropic metric can be done by extending this syntax in the obvious way.
2d_cuboid
, ...:
Cuboid Cartesian Matching
matcher=2d_cuboid values*='<x> <y>' params='<error-in-x> <error-in-y>' tuning='<bin-factor>'
values*
:
x
: Cartesian co-ordinate #1y
: Cartesian co-ordinate #2params
:
error-in-x
: Half length of cuboid in Cartesian co-ordinate #1 directionerror-in-y
: Half length of cuboid in Cartesian co-ordinate #2 directiontuning
:
bin-factor
: Scaling factor to adjust bin size; larger values mean larger bins
The 2d_cuboid
matcher compares positions
in 2-dimensional Cartesian space in cuboidal cells.
Rows are considered to match if their (x
,y
)
positions fall within an error cuboid with half-axis lengths
error-in-x
, error-in-y
of each other.
This kind of match is suitable for grouping items into pixels,
though it's not a very efficient way of doing that.
Matching in any number of dimensions using N-dimensional hyper-cuboids can be done by extending this syntax in the obvious way.
1d_err
, 2d_err
, ...:
Cartesian Matching with Per-Object Errors
matcher=2d_err values*='<x> <y> <error>' params='<scale>' tuning='<bin-factor>'
values*
:
x
: Cartesian co-ordinate #1y
: Cartesian co-ordinate #2error
: Per-object error radiusparams
:
scale
: Rough average of per-object error distance; just used for tuning in conjunction with bin factortuning
:
bin-factor
: Scaling factor to adjust bin size; larger values mean larger bins
The 1d_err
, 2d_err
, ... matchers compare
positions in N-dimensional Cartesian space like the
1d
, 2d
matchers described in
Section 7.1.6,
except that the match radius can be different for each row.
Rows are considered to match when the separation reckoned by
Pythagoras between the x
, y
, ... positions
is no larger than the sum of the two per-row error
values.
Matching in any number of Cartesian dimensions can be done by extending
this syntax in the obvious way.
A variant form 1d_err_q
, 2d_err_q
, ...
does the same thing but combines the two per-row error
values
in quadrature rather than by summation.
In that case the separation beween the two positions
must be no larger than sqrt(error12+error22).
The scale
parameter must be supplied, and should be
approximately the characteristic size of the per-object error values.
In conjunction with the bin-factor
tuning parameter
its value affects the performance of the match, but not the result.
2d_ellipse
:
Cartesian Matching of Elliptical Regions
matcher=2d_ellipse values*='<x> <y> <primary-radius> <secondary-radius> <orientation-angle/deg>' params='<scale>' tuning='<bin-factor>'
values*
:
x
: X coordinate of centrey
: Y coordinate of centreprimary-radius
: Length of ellipse semi-major axissecondary-radius
: Length of ellipse semi-minor axisorientation-angle/deg
: Angle from X axis towards Y axis of semi-major axisparams
:
scale
: Rough average of per-object error distance; just used for tuning in conjunction with bin factortuning
:
bin-factor
: Scaling factor to adjust bin size; larger values mean larger bins
The 2d_ellipse
matcher compares elliptical regions
in a 2d plane for overlap. Each row has to specify five values,
giving the centre, the major and minor radii, and the orientation angle
of an ellipse.
Rows are considered to match if there is any overlap between the ellipses.
The goodness of match is a normalised generalisation of the symmetrical case
used by the isotropic matcher, in which the best possible match is
two concentric ellipses, and the worst allowable match is when
the circumferences just touch.
Note the orientation angle is measured anticlockwise from the horizontal,
unlike the position angle used by the
skyellipse
matcher.
The scale
parameter must be supplied, and should be
approximately the characteristic size of the per-object major radius.
In conjunction with the bin-factor
tuning parameter
its value affects the performance of the match, but not the result.
For advanced users, it is possible to supply the name of a class
on the classpath which implements the
uk.ac.starlink.table.join.MatchEngine
interface
and which has a no-arg constructor.
This allows java programmers to write their own matchers using any
match criteria and binning algorithms they choose.
In addition to the matching criteria listed in the previous subsections,
you can build your own by combining any of these.
To do this, take the two (or more)
matchers that you want to use, and separate their names with a
"+
" character. The values*
parameters
of the combined matcher should then hold the concatenation of the
values*
entries of the constituent matchers, and the
same for the params
parameter.
Two rows are then considered to match if the the match is successful
for all of their constituent matchers.
A variant form where the names are separated with a
"*
" character instead of "+
" may also be used.
In this case an additional constraint is applied requiring that
the distance measure (see below) is less than or equal to unity,
thus requiring the points in the notional scaled parameter coordinate space
to be within a unit hypersphere rather than a unit hypercube.
So for instance the matcher "sky+1d
" could be used
with the following syntax:
matcher=sky+1d values*='<ra/deg> <dec/deg> <x>' params='<max-error/arcsec> <error>' tuning='<healpix-k> <bin-factor>'
This would compare positions on the sky with an additional scalar constraint. Rows are considered to match if both their
values*
:
ra/deg
: Right Ascensiondec/deg
: Declinationx
: Cartesian co-ordinate #1params
:
max-error/arcsec
: Maximum separation along a great circleerror
: Maximum Cartesian separation for matchtuning
:
healpix-k
: Controls sky pixel size. Legal range 0 - 29. 0 is 60deg, 20 is 0.2".bin-factor
: Scaling factor to adjust bin size; larger values mean larger bins
ra
, dec
positions are within
max-error
arcseconds of each other along a great circle
(as for matcher=sky
)
and
their x
values differ by no more than error
(as for matcher=1d
).
Using matcher=sky*1d
instead would work the same way
but restrict the matches a bit further.
This example might be used for instance to identify objects from two catalogues which are within a couple of arcseconds and also 0.5 blue magnitudes of each other. Rolling your own matchers in this way can give you quite flexible match constraints.
When identifying the closest match
(e.g. find=best1
in tmatch2
)
the "distance" measure is obtained by
scaling the distances from each of the constituent matchers
and adding these scaled distances in quadrature,
so that each element of the matcher has approximately equal weight.
Scaling is generally done using the maximum permissible match
radius (or equivalent), so the distance measure looks something like
d = sqrt([dA/max(dA)]2
+ [dB/max(dB)]2).
However the details are a bit dependent on which matchers you are combining.
If the "*
" separator is used instead of "+
"
in the matcher specification as described above,
this distance will always be <=1 for successful matches.
Note that in STILTS v3.0-9 and earlier, a linear unscaled distance measure was used here instead, which did not give very meaningful Best match results.
The generic matching in STILTS is determined by specified
match criteria, as described in Section 7.1.
These criteria give conditions for whether two items (table rows)
count as matched with each other. In the case of a pair match,
as provided by tmatch2
,
it is clear how this is to be interpreted.
However, some of the matching tasks
(tmatchn
in group mode and
tmatch1
)
search for match groups which may have more than two members.
This section explains precisely how STILTS applies the pair-wise
matching criteria it is given to identifying multi-object groups.
In a multi-object match context, the matcher identifies a matched group as the largest possible group of objects in which each is linked by a pair match to any other object in the group - it is a group of "friends of friends". Formally, the set of matched groups is a set of disjoint graphs whose nodes are input table rows and whose edges are successful pair matches, where no successful pair match exists between nodes in different elements of that set. Thus the set has a minimal number of elements, and each of its elements is a matched group of maximal size. The important point to note is that for any particular pair in a matched group, there is no guarantee that the two objects match each other, only that you can hop from one to the other via pairs which do match.
So in the case of a multi-object sky match on a field which is very crowded compared to the specified error radius, it is quite possible for all the objects in the input table(s) to end up as part of the same large matching group. Results at or near this percolation threshold are (a) probably not useful and (b) likely to take a long time to run. Some care should therefore be exercised when specifying match criteria in multi-object match contexts.
As of version 3.0 (October 2014), STILTS offers plotting commands corresponding to the new-style plots in version 4 of the TOPCAT application. The commands are currently:
plot2plane
:
Draws a plane plot
plot2sky
:
Draws a sky plot
plot2cube
:
Draws a cube plot
plot2sphere
:
Draws a sphere plot
plot2corner
:
Draws a matrix of plane plots
plot2time
:
Draws a time plot
plot2d
,
plot3d
and
plothist
were available -
these are now deprecated, but described in Section 9).
These commands all have a similar structure.
The plot surface, or geometry of the plot,
is defined by which command you use
(for instance, if you want to plot longitude/latitude data on
the celestial sphere, use plot2sky
).
Content is added to the plot by specifying zero or more
plot layers,
as described in Section 8.3 below.
Section 8.4 describes the shading modes which affect
how colouring is performed for some of the layer types.
Once a plot has been specified, it can be displayed on
the screen or exported in some way according to
a selected output mode (Section 8.5)
and perhaps export format (Section 8.6).
Plots displayed to the screen are by default "live" - they can be
resized and navigated around (pan, zoom, rotate, ...) using the mouse
in the same way as in a TOPCAT window.
These commands allow you to make all the plots that can be produced with TOPCAT, in some cases with more flexibility in configuration. Unlike TOPCAT, the size of table you can plot is not limited by the size of table you can load into the application. In most cases, STILTS will generate plots from arbitrarily large data sets with fixed (and modest) memory requirements. Performance is of course highly dependent on the details of the plot, but for instance an all-sky density plot for 2 billion points can be produced in the order of 30 minutes.
The plotting commands offer a great deal of control over what is plotted and how it is represented, and thus unavoidably have lots of parameters. When looking at the command documentation in Appendix B the Usage sections may look rather daunting. However, the discussion below and the Examples sections should help. Generating a simple plot is straightforward and can be done with only four or five parameters; if you want to represent more complicated data or have specific preferences for appearance then you can consult the documentation for the additional options.
As a simple example, if a file "cat.fits" contains the columns RMAG and BMAG for red and blue magnitudes, you can draw a two-dimensional colour-magnitude scatter plot with the command:
stilts plot2plane layer_1=mark in_1=cat.fits x_1=BMAG-RMAG y_1=BMAGSince an output file is not specified, the plot is shown in a window on the screen. This plot window is "live" - you can resize the window, or pan and zoom around it using the same mouse controls as in TOPCAT. To send the output to a PNG file, do instead:
stilts plot2plane layer_1=mark in_1=cat.fits x_1=BMAG-RMAG y_1=BMAG out=fig.png
We can adjust the plot by inverting the Y axis so it increases downwards instead of upwards:
stilts plot2plane yflip=true layer_1=mark in_1=cat.fits x_1=BMAG-RMAG y_1=BMAGThe parameters of the plot now fall into two groups. Global parameters, without suffixes, make global adjustments to the plot. In this example
yflip=true
inverts the Y axis.
Layer parameters, with suffixes,
are introduced by a layer
parameter
and grouped together by a given suffix.
Each layer group defines a plot layer with content
to be drawn on the plot surface.
In this case the layer is of type mark
(draw markers)
and the suffix is "_1
".
Global and Layer parameters are described separately in the following
subsections.
The global plot parameters are documented in the usage sections of the various plot commands (e.g. Appendix B.10.1). They deal with things like positioning the plot axes, fixing the data bounds, selecting font types and sizes, adjusting grids and tickmarks, configuring how interactive navigation works, managing data storage, and so on. They are all optional, since they all have sensible defaults, for instance data bounds will be determined from the supplied data if they are not given explicitly.
The layer parameters come in groups, each specifying the details of one plot layer. Each layer type has its own list of parameters. A plot layer is introduced on the command line with a parameter of the form
layer<suffix>=<layer-type>and any other parameters with the same
<suffix>
are considered to apply to the same layer.
In the basic example we considered:
stilts plot2plane layer_1=mark in_1=cat.fits x_1=BMAG-RMAG y_1=BMAGthe suffix is "
_1
" and the layer type associated with it is
mark
(plotting markers to make a scatter plot).
The different layer types are documented in Section 8.3,
and each has its own set of parameters, some of which are mandatory
and some which are optional with sensible defaults.
In the documentation, the suffix is represented as "N
".
For instance the mark
layer type
requires you to specify an input table (inN
) and
point positions (xN
and yN
).
Since the suffix we have used in the example for the
layerN
parameter is "_1
",
we have written in_1
, x_1
and y_1
.
The mark
layer has some optional style parameters as well,
so we could adjust the plot's appearance by adding
shape_1=cross size_1=4 color_1=blue
.
You can have as many layers as you like (even none), so we could overplot two datasets from different input files like this:
stilts plot2plane layer_1=mark in_1=cat1.fits x_1=BMAG-RMAG y_1=BMAG color_1=magenta size_1=5 layer_2=mark in_2=cat2.fits x_2=mag_b-mag_r y_2=mag_b color_2=cyan size_2=5We have assigned different colours to the different layers and boosted the marker size to 5 pixels.
As a convenience, if the same value is used for all the layers, you can omit the suffix. So to avoid having to specify the same markers size for both layers, you can write instead:
stilts plot2plane size=5 layer_1=mark in_1=cat1.fits x_1=BMAG-RMAG y_1=BMAG color_1=magenta layer_2=mark in_2=cat2.fits x_2=mag_b-mag_r y_2=mag_b color_2=tealAlthough the
size
parameter no longer has an explicit suffix,
it's still a layer parameter, it just applies to multiple layers.
This shorthand works for all layer parameters.
Here is another example which also shows how you can use the
icmdN
parameter to pre-process
input data prior to performing the plot.
Here, we make two different selections of the input rows to plot
two different data sets.
stilts plot2plane in=cat.fits x=BMAG-RMAG y=BMAG layer_1=mark icmd_1='select vel<1000' color_1=blue layer_2=mark icmd_2='select vel>=1000' color_2=redThe input tables and data values are the same for both datasets, so we can just supply the parameters
in
, x
and y
,
rather than
in_1
, in_2
etc.
Any string can be used as a suffix, including the empty string
(though an empty string can cause confusion if there are multiple layers).
The suffixing is also slightly more sophisticated than described above;
to find parameters relating to a layer with a given suffix, the
parameter looks first using the whole suffix, and strips single
characters off it until it has none left.
So if a layer is introduced with the parameter layer_ab
,
you can give the marker shape using any of the parameters
shape_ab
, shape_a
, shape_
or shape
. If more than one of these is present,
the first one in that list will be used
(the order in which they appear on the command line is not significant).
This can be used to group sets of layers.
By default, if multiple layers are specified, they are plotted
in the order in which the introducing layerN
parameters
appear on the command line. This may be relevant, since layers
plotted later sometimes obscure ones plotted earlier.
You can alter the order of plotting with the seq
(global)
parameter, which is a comma-separated list of layer suffixes giving
the sequence in which layers should be plotted.
So adding "seq=_2,_1
" would cause layer _2 to be plotted
before layer _1, instead of the other way round.
By default, if more than one layer is plotted, a legend will
appear labelling the datasets.
The dataset labels appearing in the legend are by default
the layer suffixes specified on the command line.
However, the labels can be given explicitly with the legendN
parameter, so for instance in the example above
leglabel_1=Slow leglabel_2=Fast
would adjust the
legend accordingly.
Legend appearance and positioning can be adjusted by various
leg*
global parameters.
The plotting commands can be used to produce animations.
This is done by supplying an animation control table
using the animate
parameter
(which has associated afmt
and acmd
parameters
for specifying its file format and applying filters).
One output image is produced for each row of the control table.
The columns of the table have names which correspond to plot command
parameters, and for each row, the basic plot command is executed
with the parameters on the command line supplied or replaced by
those from the table.
This is most commonly used for providing a movie of the kind of
navigation you can do interactively with the mouse,
but other applications are possible.
For instance, given the following animation control table with the name "bounds.txt", in ASCII format:
# xmax ymax 4.0 2.0 3.0 1.5 2.0 1.0 1.0 0.5then this command:
stilts plot2plane xmin=0 ymin=0 layer_1=mark in_1=gums_smc.fits x_1=ag y_1=av animate=bounds.txt afmt=asciiwould produce a 4-frame animation zooming in towards the origin.
If output is to the screen
(omode=swing
, the default)
the animation can be seen directly.
If it is to an output file
(omode=out
)
then a number of output files is written with sequence numbers,
so adding the parameter "out=x.png
" to the above command
would produce 4 files,
x-1.png
,
x-2.png
,
x-3.png
and
x-4.png
.
Padding zeros are used to keep the files in alphanumeric sequence,
so for instance in a 500-frame animation the first one would be named
x-001.png
.
STILTS does not actually turn these files into a single animated output file,
but you can use other tools to do this, for instance using ImageMagick:
convert x-*.png xmovie.gifor ffmpeg:
ffmpeg -i 'x-%03d.png' -framerate 15 -pix_fmt yuv420p xmovie.webm
You can create the animation control table any way you like,
but you may find the tloop
command convenient. For instance the above table can be
written like this:
stilts tloop xmax 4 0 -1 ocmd='addcol ymax xmax*0.5' ofmt=ascii out=bounds.txtAlternatively, you can generate a table like this inline using the loop scheme. A common requirement is to produce an animation of rotating a 3-d plot, here's an example of how to do that:
stilts plot2sphere layer_1=mark in_1=hip_main.fits lon_1=radeg lat_1=dedeg r_1=plx \ animate=:loop:15,375,2 acmd='colmeta -name phi $1'The
phi
parameter controls the angle from which the
3D plot is viewed, and here it is incremented by 2 degrees
for each frame.
The same thing would work for plot2cube
as well as plot2sphere
.
Note that producing animations in this way is usually much more efficient than writing a shell script which invokes STILTS multiple times. The plot commands also employ multi-threading when animating to output files, so should make efficient use of multi-core machines (though currently animations to the screen are not multi-threaded).
The different plot2*
commands correspond to
different plot surface geometries.
The different commands come with their own specific
axis configuration parameters.
Some of the plot layer types are specific to certain surface types.
When supplying data from input tables to plot layers,
the coordinate values you need to supply
(and hence the corresponding parameter names)
are determined not by the layer type, but by the surface type.
For instance, point positions for layer N on
a 2-d Cartesian surface (plot2plane
command)
are given using parameters xN
and yN
,
but when plotting to the celestial sphere
(plot2sky
command) you supply
lonN
and latN
).
The following list summarises the available surface types and their corresponding positional coordinates.
plot2plane
)
x
, y
pairs.
Note that this command can also be used to draw histograms.
plot2sky
)
lon
, lat
pairs,
giving longitude and latitude in decimal degrees.
A number of different projections are available,
and conversion between different celestial coordinate systems can
also be performed.
You could use it for other spherical coordinate systems too
(like the surface of a planet).
plot2cube
)
x
, y
, z
triples.
plot2sphere
)
lon
, lat
, r
triples,
giving longitude and latitude in decimal degrees, and radius in
an arbitrary unit.
The plotting surface (space) is similar to Cube,
except that the unit distance is always
the same in all three directions.
plot2corner
)
x1
, x2
, x3
, ... xN
,
up to the number N specified by the nvar
parameter.
plot2time
)
t
, y
pairs.
Time can be represented in input data in various ways;
if sufficient metadata is provided in the input format
the epoch can be determined automatically,
otherwise it may be necessary to specify the time representation
being used.
The different plot layers and how to configure them with parameters is given in the following subsections. The layers which may be plotted on a particular surface depend on the plot geometry, so not all of these are available for every plot command.
mark
Plots a marker of fixed size and shape at each position.
Usage Overview:
layerN=mark shapeN=filled_circle|open_circle|... sizeN=<pixels> shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> <pos-coord-paramsN> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2plane layer1=mark in1=rrlyrae.fits x1=p1 y1=peak_to_peak_g
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
shapeN = filled_circle|open_circle|...
(MarkerShape)
The available options are:
filled_circle
open_circle
cross
x
open_square
open_diamond
open_triangle_up
open_triangle_down
fat_circle
fat_cross
fat_x
fat_square
fat_diamond
fat_triangle_up
fat_triangle_down
filled_square
filled_diamond
filled_triangle_up
filled_triangle_down
[Default: filled_circle
]
sizeN = <pixels>
(Integer)
[Default: 1
]
size
Plots a marker of fixed shape but variable size at each position. The size is determined by an additional input data value.
The actual size of the markers depends on the setting of the
autoscale
parameter.
If autoscaling is off, then the basic size of each marker
is the input data value in units of pixels.
If autoscaling is on, then the data values are gathered
for all the currently visible points, and a scaling factor
is applied so that the largest ones will be a sensible size
(a few tens of pixels).
This basic size can be further adjusted with the
scale
factor.
Currently data values of zero always correspond to marker size of zero, negative data values are not represented, and the mapping is linear. An absolute maximum of 100 pixels is also imposed on marker sizes. Other options may be introduced in future.
Note: for marker sizes that correspond to data values in data coordinates, you may find Error plotting more appropriate.
Usage Overview:
layerN=size shapeN=filled_circle|open_circle|... scaleN=<factor> autoscaleN=true|false shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> <pos-coord-paramsN> sizeN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2sky projection=aitoff xpix=500 ypix=250 layer1=size in1=messier.xml shading1=transparent lon1=RA lat1=DEC size1=Radius
autoscaleN = true|false
(Boolean)
If auto-scaling is off, then markers will keep exactly the same screen size during pan and zoom operations; if it's on, then the visible sizes will change according to what other points are currently plotted.
Marker size is also affected by the
scale
parameter.
[Default: true
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
scaleN = <factor>
(Double)
[Default: 1
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
shapeN = filled_circle|open_circle|...
(MarkerShape)
The available options are:
filled_circle
open_circle
cross
x
open_square
open_diamond
open_triangle_up
open_triangle_down
fat_circle
fat_cross
fat_x
fat_square
fat_diamond
fat_triangle_up
fat_triangle_down
filled_square
filled_diamond
filled_triangle_up
filled_triangle_down
[Default: filled_circle
]
sizeN = <num-expr>
(String)
scale
value.
The value is a numeric algebraic expression based on column names as described in Section 10.
sizexy
Plots a shaped marker with variable horizontal and vertical extents at each position. The X and Y dimensions are determined by two additional input data values.
The actual size of the markers depends on the setting of the
autoscale
parameter.
If autoscaling is off, the basic dimensions of each marker
are given by the input data values in units of pixels.
If autoscaling is on, the data values are gathered
for all the currently visible points, and scaling factors
are applied so that the largest ones will be a sensible size
(a few tens of pixels).
This autoscaling happens independently for
the X and Y directions.
The basic sizes can be further adjusted with the
scale
factor.
Currently data values of zero always correspond to marker dimension of zero, negative data values are not represented, and the mapping is linear. An absolute maximum of 100 pixels is also imposed on marker sizes. Other options may be introduced in future.
Note: for marker sizes that correspond to data values in data coordinates, you may find Error plotting more appropriate.
Usage Overview:
layerN=sizexy shapeN=open_rectangle|open_triangle|... thickN=<int-value> scaleN=<factor> autoscaleN=true|false shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> <pos-coord-paramsN> xsizeN=<num-expr> ysizeN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2plane layer1=sizexy in1=dr5qso.fits shape1=filled_rectangle x1=psfmag_u-psfmag_g y1=psfmag_r-psfmag_z xsize1=exp(psfmag_g) ysize1=exp(psfmag_r) xmin=-3 xmax=1 ymin=1 ymax=3.2
autoscaleN = true|false
(Boolean)
If auto-scaling is off, then markers will keep exactly the same screen size during pan and zoom operations; if it's on, then the visible sizes will change according to what other points are currently plotted.
Marker size is also affected by the
scale
parameter.
[Default: true
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
scaleN = <factor>
(Double)
[Default: 1
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
shapeN = open_rectangle|open_triangle|...
(BasicXYShape)
open_rectangle
open_triangle
open_triangle_down
open_diamond
open_ellipse
filled_rectangle
filled_triangle
filled_triangle_down
filled_diamond
filled_ellipse
[Default: open_rectangle
]
thickN = <int-value>
(Integer)
[Default: 0
]
xsizeN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
ysizeN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xyvector
Plots directed lines from the data position given delta values for the coordinates. The plotted markers are typically little arrows, but there are other options.
In some cases the supplied data values
give the actual extents in data coordinates
for the plotted vectors
but sometimes the data is on a different scale
or in different units to the positional coordinates.
As a convenience for this case, the plotter can optionally
scale the magnitudes of all the vectors
to make them a reasonable size on the plot,
so by default the largest ones are a few tens of pixels long.
This auto-scaling is turned off by default,
but it can be activated with the
autoscale
option.
Whether autoscaling is on or off, the
scale
option can be used to apply a fixed scaling factor.
Usage Overview:
layerN=xyvector arrowN=small_arrow|medium_arrow|... thickN=<int-value> scaleN=<factor> autoscaleN=true|false shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> xN=<num-expr> yN=<num-expr> xdeltaN=<num-expr> ydeltaN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=xyvector in1=gavo_g2.fits x1=x y1=y xdelta1=velX ydelta1=velY autoscale1=true thick1=1 xmin=9 xmax=11 ymin=12 ymax=13.5
arrowN = small_arrow|medium_arrow|...
(MultiPointShape)
The available options are:
small_arrow
medium_arrow
large_arrow
small_open_dart
medium_open_dart
large_open_dart
small_filled_dart
medium_filled_dart
large_filled_dart
lines
capped_lines
[Default: small_arrow
]
autoscaleN = true|false
(Boolean)
If auto-scaling is on, then markers will keep approximately the same screen size during zoom operations; if it's off, they will keep the same size in data coordinates.
Marker size is also affected by the
scale
parameter.
[Default: false
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
scaleN = <factor>
(Double)
[Default: 1
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xdeltaN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
ydeltaN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xyerror
Plots symmetric or asymmetric error bars in some or all of the plot dimensions. The shape of the error "bars" is quite configurable, including (for 2-d and 3-d errors) ellipses, rectangles etc aligned with the axes.
Usage Overview:
layerN=xyerror errorbarN=none|lines|capped_lines|... thickN=<int-value> shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> xN=<num-expr> yN=<num-expr> xerrhiN=<num-expr> xerrloN=<num-expr> yerrhiN=<num-expr> yerrloN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=J_MNRAS_440_1571.vot x=S500 y=S160 layer1=mark size1=5 shape1=fat_circle layer2=xyerror xerrhi2=e_S500 yerrhi2=e_S160 errorbar2=capped_lines thick2=1 xlog=true ylog=true shading=flat xmin=0.012 xmax=1 ymin=0.01 ymax=10
errorbarN = none|lines|capped_lines|...
(MultiPointShape)
The available options are:
none
lines
capped_lines
caps
arrows
ellipse
crosshair_ellipse
rectangle
crosshair_rectangle
filled_ellipse
filled_rectangle
[Default: lines
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xerrhiN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xerrloN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yerrhiN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yerrloN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xyellipse
Plots an ellipse (or rectangle, triangle, or other similar figure) defined by two principal radii and an optional angle of rotation, the so-called position angle. This angle, if specified, is in degrees and gives the angle counterclockwise from the horizontal axis to the first principal radius.
In some cases the supplied data values
give the actual extents in data coordinates
for the plotted ellipses
but sometimes the data is on a different scale
or in different units to the positional coordinates.
As a convenience for this case, the plotter can optionally
scale the magnitudes of all the ellipses
to make them a reasonable size on the plot,
so by default the largest ones are a few tens of pixels long.
This auto-scaling is turned off by default,
but it can be activated with the
autoscale
option.
Whether autoscaling is on or off, the
scale
option can be used to apply a fixed scaling factor.
Usage Overview:
layerN=xyellipse ellipseN=ellipse|crosshair_ellipse|... thickN=<int-value> scaleN=<factor> autoscaleN=true|false shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> xN=<num-expr> yN=<num-expr> raN=<num-expr> rbN=<num-expr> posangN=<deg-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=mgc_ok.fits x=mgc_alpha_j2000 y=mgc_delta_j2000 ra=bulge_re/3600. rb=bulge_re*bulge_e/3600. posang=bulge_pa autoscale=false scale=10 color=blue layer1=xyellipse ellipse1=filled_ellipse shading1=transparent opaque1=4 layer2=xyellipse ellipse2=crosshair_ellipse aspect=1 xmin=181.3 xmax=181.9
autoscaleN = true|false
(Boolean)
If auto-scaling is on, then markers will keep approximately the same screen size during zoom operations; if it's off, they will keep the same size in data coordinates.
Marker size is also affected by the
scale
parameter.
[Default: false
]
ellipseN = ellipse|crosshair_ellipse|...
(MultiPointShape)
The available options are:
ellipse
crosshair_ellipse
filled_ellipse
rectangle
crosshair_rectangle
filled_rectangle
open_triangle
filled_triangle
lines
capped_lines
arrows
[Default: ellipse
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
posangN = <deg-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
raN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
rbN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
scaleN = <factor>
(Double)
[Default: 1
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xycorr
Plots an error ellipse (or rectangle or other similar figure) defined by errors in the X and Y directions, and a correlation between the two errors.
The supplied correlation is a dimensionless value in the range -1..+1 and is equal to the covariance divided by the product of the X and Y errors. The covariance matrix is thus:
[ xerr*xerr xerr*yerr*xycorr ] [ xerr*yerr*xycorr yerr*yerr ]
In some cases the supplied data values
give the actual extents in data coordinates
for the plotted ellipses
but sometimes the data is on a different scale
or in different units to the positional coordinates.
As a convenience for this case, the plotter can optionally
scale the magnitudes of all the ellipses
to make them a reasonable size on the plot,
so by default the largest ones are a few tens of pixels long.
This auto-scaling is turned off by default,
but it can be activated with the
autoscale
option.
Whether autoscaling is on or off, the
scale
option can be used to apply a fixed scaling factor.
This plot type is suitable for use with the
<x>_error
and
<x>_<y>_corr
columns
in the Gaia source catalogue.
Usage Overview:
layerN=xycorr ellipseN=ellipse|crosshair_ellipse|... thickN=<int-value> scaleN=<factor> autoscaleN=true|false shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> xN=<num-expr> yN=<num-expr> xerrN=<num-expr> yerrN=<num-expr> xycorrN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=tgas_source.fits icmd='select skyDistanceDegrees(ra,dec,56.9,23.9)<0.4' x=pmra y=pmdec layer1=mark xerrhi2=pmra_error yerrhi2=pmdec_error color2=cyan shading2=transparent layer2a=xyerror errorbar2a=filled_rectangle opaque2a=10 layer2b=xyerror errorbar2b=crosshair_rectangle opaque2b=4 layer3=xycorr autoscale3=false xerr3=pmra_error yerr3=pmdec_error xycorr3=pmra_pmdec_corr ellipse3=crosshair_ellipse aspect=1 xmin=17 xmax=24 ymin=-48 ymax=-42
autoscaleN = true|false
(Boolean)
If auto-scaling is on, then markers will keep approximately the same screen size during zoom operations; if it's off, they will keep the same size in data coordinates.
Marker size is also affected by the
scale
parameter.
[Default: false
]
ellipseN = ellipse|crosshair_ellipse|...
(MultiPointShape)
The available options are:
ellipse
crosshair_ellipse
filled_ellipse
rectangle
crosshair_rectangle
filled_rectangle
open_triangle
filled_triangle
lines
capped_lines
arrows
[Default: ellipse
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
scaleN = <factor>
(Double)
[Default: 1
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xerrN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xycorrN = <num-expr>
(String)
*_corr
values
supplied in the Gaia source catalogue.
The value is a numeric algebraic expression based on column names as described in Section 10.
yN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yerrN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
link2
Plots a line linking two positions from the same input table row.
Usage Overview:
layerN=link2 thickN=<int-value> shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> <pos-coord-params1N> <pos-coord-params2N> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-params1N>
, <pos-coord-params2N>
give 2 positions for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
x1N
, y1N
,
x2N
and y2N
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2sky clon=14.78 clat=-72.1525 radius=0.0015 sex=false layer_h=mark in_h=ngc346.fits lon_h=_RAJ2000 lat_h=_DEJ2000 color_h=red layer_g=mark in_g=ngc346xGaiadr1.fits lon_g=ra lat_g=dec color_g=blue shading_g=flat size_g=3 in_x=ngc346xGaiadr1.fits lon1_x=_RAJ2000 lat1_x=_DEJ2000 lon2_x=ra lat2_x=dec shading_x=flat layer_xl=link2 color_xl=forestgreen layer_xm=mark2 color_xm=greenyellow size_xm=4 shape_xm=open_circle seq=_xm,_xl,_h,_g leglabel_h=HST leglabel_g='Gaia DR1' legseq=_h,_g legpos=0.95,0.95
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
mark2
Plots 2 similar markers of fixed size and shape representing 2 separate positions from the same input table row. This is a convenience option that can be used with other plot layers based on 2 positions.
Usage Overview:
layerN=mark2 shapeN=filled_circle|open_circle|... sizeN=<pixels> shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> <pos-coord-params1N> <pos-coord-params2N> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-params1N>
, <pos-coord-params2N>
give 2 positions for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
x1N
, y1N
,
x2N
and y2N
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2sky clon=14.78 clat=-72.1525 radius=0.0015 sex=false layer_h=mark in_h=ngc346.fits lon_h=_RAJ2000 lat_h=_DEJ2000 color_h=red layer_g=mark in_g=ngc346xGaiadr1.fits lon_g=ra lat_g=dec color_g=blue shading_g=flat size_g=3 in_x=ngc346xGaiadr1.fits lon1_x=_RAJ2000 lat1_x=_DEJ2000 lon2_x=ra lat2_x=dec shading_x=flat layer_xl=link2 color_xl=greenyellow layer_xm=mark2 color_xm=forestgreen size_xm=4 shape_xm=open_circle seq=_xm,_xl,_h,_g leglabel_h=HST leglabel_g='Gaia DR1' legseq=_h,_g legpos=0.95,0.95
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
shapeN = filled_circle|open_circle|...
(MarkerShape)
The available options are:
filled_circle
open_circle
cross
x
open_square
open_diamond
open_triangle_up
open_triangle_down
fat_circle
fat_cross
fat_x
fat_square
fat_diamond
fat_triangle_up
fat_triangle_down
filled_square
filled_diamond
filled_triangle_up
filled_triangle_down
[Default: filled_circle
]
sizeN = <pixels>
(Integer)
[Default: 1
]
poly4
Draws a closed quadrilateral
given the coordinates of its vertices
supplied as 4 separate positions.
The way that the polygon is drawn (outline, fill etc)
is determined using the
polymode
option.
Polygons smaller than a configurable threshold size in pixels are by default represented by a replacement marker, so the position of even a very small polygon is still visible on the screen.
Usage Overview:
layerN=poly4 polymodeN=outline|border|fill|cross|star thickN=<int-value> minsizeN=<pixels> minshapeN=filled_circle|open_circle|... shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> <pos-coord-params1N> <pos-coord-params2N> <pos-coord-params3N> <pos-coord-params4N> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-params1N>
, <pos-coord-params2N>
, <pos-coord-params3N>
, <pos-coord-params4N>
give 4 positions for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
x1N
, y1N
,
x2N
, y2N
,
x3N
, y3N
,
x4N
and y4N
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2sky in=big_tab_VIR_VIS_CSA_public.fits icmd='every 32' lon1=LON_CORNER_1 lat1=LAT_CORNER_1 lon2=LON_CORNER_2 lat2=LAT_CORNER_2 lon3=LON_CORNER_3 lat3=LAT_CORNER_3 lon4=LON_CORNER_4 lat4=LAT_CORNER_4 aux=RADIUS layer_o=poly4 polymode_o=outline shading_o=aux layer_f=poly4 polymode_f=fill shading_f=aux opaque_f=4 auxmap=rainbow auxvisible=false xpix=300 ypix=300 labelpos=none
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
minshapeN = filled_circle|open_circle|...
(MarkerShape)
minsize
.
The available options are:
filled_circle
open_circle
cross
x
open_square
open_diamond
open_triangle_up
open_triangle_down
fat_circle
fat_cross
fat_x
fat_square
fat_diamond
fat_triangle_up
fat_triangle_down
filled_square
filled_diamond
filled_triangle_up
filled_triangle_down
[Default: x
]
minsizeN = <pixels>
(Integer)
[Default: 1
]
polymodeN = outline|border|fill|cross|star
(PolygonShape)
The available options are:
outline
: draws a line round the outside of the polygonborder
: draws a line butting up to the outside of the polygon; may look better for adjacent shapes, but more expensive to drawfill
: fills the interior of the polygoncross
: draws a line round the outside of the polygon and lines between all the verticesstar
: draws a line round the outside of the polygon and lines from the nominal center to each vertex[Default: outline
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
mark4
Plots 4 similar markers of fixed size and shape representing 4 separate positions from the same input table row. This is a convenience option that can be used with other plot layers based on 4 positions.
Usage Overview:
layerN=mark4 shapeN=filled_circle|open_circle|... sizeN=<pixels> shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> <pos-coord-params1N> <pos-coord-params2N> <pos-coord-params3N> <pos-coord-params4N> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-params1N>
, <pos-coord-params2N>
, <pos-coord-params3N>
, <pos-coord-params4N>
give 4 positions for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
x1N
, y1N
,
x2N
, y2N
,
x3N
, y3N
,
x4N
and y4N
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2plane in=big_tab_VIR_VIS_CSA_public.fits icmd='select IOF_055<0.005' icmd='select lon_center>250&&lon_center<300&&lat_center>-65&&lat_center<-16' x1=LON_CORNER_1 y1=LAT_CORNER_1 x2=LON_CORNER_2 y2=LAT_CORNER_2 x3=LON_CORNER_3 y3=LAT_CORNER_3 x4=LON_CORNER_4 y4=LAT_CORNER_4 layer_q=poly4 polymode_q=fill shading_q=transparent opaque_q=4 layer_m=mark4 color_m=404040 shape_m=open_circle size_m=3
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
shapeN = filled_circle|open_circle|...
(MarkerShape)
The available options are:
filled_circle
open_circle
cross
x
open_square
open_diamond
open_triangle_up
open_triangle_down
fat_circle
fat_cross
fat_x
fat_square
fat_diamond
fat_triangle_up
fat_triangle_down
filled_square
filled_diamond
filled_triangle_up
filled_triangle_down
[Default: filled_circle
]
sizeN = <pixels>
(Integer)
[Default: 1
]
polygon
Draws a closed polygon given an array of coordinates
that define its vertices.
In fact this plot requires the position of the first vertex
supplied as a positional value in the usual way
(e.g. X
and Y
coordinates)
and the second, third etc vertices supplied as an array
using the otherpoints
parameter.
Invocation might therefore look like
"xN=x1 yN=y1 otherpointsN=array(x2,y2, x3,y3, x4,y4)
".
Usage Overview:
layerN=polygon useposN=true|false polymodeN=outline|border|fill|cross|star thickN=<int-value> shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> <pos-coord-paramsN> otherpointsN=<array-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2sky in=big_tab_VIR_VIS_CSA_public.fits icmd='select ALTITUDE>4e4&&ALTITUDE<4.3e4' layer=polygon polymode=fill lon=LON_CENTER lat=LAT_CENTER otherpoints=array(lon_corner_1,lat_corner_1,lon_corner_2,lat_corner_2) shading=weighted weight=IR_TEMPERATURE auxmap=plasma texttype=latex fontsize=14 auxlabel=T_{IR} clon=83 clat=34 radius=11
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
otherpointsN = <array-expr>
(String)
usepos
parameter.
Some expression language functions that can be useful
when specifying this parameter are
array()
and parseDoubles()
.
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
polymodeN = outline|border|fill|cross|star
(PolygonShape)
The available options are:
outline
: draws a line round the outside of the polygonborder
: draws a line butting up to the outside of the polygon; may look better for adjacent shapes, but more expensive to drawfill
: fills the interior of the polygoncross
: draws a line round the outside of the polygon and lines between all the verticesstar
: draws a line round the outside of the polygon and lines from the nominal center to each vertex[Default: outline
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
useposN = true|false
(Boolean)
[Default: true
]
area
Plots a region on the plotting surface
specified by a string or array of numbers.
The area may be specified as an STC-S string
(as for example in an ObsCore or EPN-TAP
s_region
column)
or using an array of numbers representing a
polygon, circle or point as flagged using the DALI/VOTable
extended type (xtype) marker,
or as an ASCII-encoded MOC.
Areas smaller than a configurable threshold size in pixels are by default represented by a replacement marker, so the position of even a very small area is still visible on the screen.
This plot type is generally intended for displaying relatively small shapes such as instrument footprints. It can be used for areas that are larger as well, but there may be issues with use, for instance auto-determination of the initial plot region may not work so well, and the rendering of shapes that are large relative to the sky may be inaccurate. These issues may be addressed in future releases.
Usage Overview:
layerN=area polymodeN=outline|border|fill|cross|star thickN=<int-value> minsizeN=<pixels> minshapeN=filled_circle|open_circle|... shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> areaN=<area-expr> areatypeN=STC-S|POLYGON|CIRCLE|POINT|MOC-ASCII|UNIQ|TFCAT inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2sky reflectlon=false sex=false clon=348.9 clat=79.8 radius=1.0 in=crism.fits icmd='select sensor_id==0x4c' area_p=s_region areatype_p=stc-s layer_pf=area polymode_pf=fill color_pf=1199ff shading_pf=transparent layer_pl=area polymode_pl=outline color_pl=grey
areaN = <area-expr>
(String)
areatype
parameter.
The value is a Area value algebraic expression based on column names as described in Section 10.
areatypeN = STC-S|POLYGON|CIRCLE|POINT|MOC-ASCII|UNIQ|TFCAT
(AreaMapper)
areaN
is supplied.
Options are:
STC-S
: Region description using STC-S syntax;
see TAP 1.0, section 6.
Note there are some restrictions:
<frame>
, <refpos>
and <flavor>
metadata are ignored,
polygon winding direction is ignored (small polygons are assumed)
and the INTERSECTION
and NOT
constructions are not supported.
The non-standard MOC
construction is supported.POLYGON
: 2n-element array (x1
,y1
, x2
,y2
,...);
a NaN
,NaN
pair can be used to delimit distinct polygons.CIRCLE
: 3-element array (x
, y
,r
)POINT
: 2-element array (x
,y
)MOC-ASCII
: Region description using ASCII MOC syntax;
see MOC 1.1 2.3.2.
Note there are currently a few issues
with MOC plotting, especially for large pixels.UNIQ
: Region description representing a single HEALPix cell
as defined by an UNIQ value, see
MOC 1.1
sec 2.3.1.TFCAT
: Time-Frequency region defined by the
TFCat standard.
Support is currently incomplete;
holes in Polygons and MultiPolygons are not displayed correctly,
single Points may not be displayed,
and Coordinate Reference System information is ignored.
areaN
value.
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
minshapeN = filled_circle|open_circle|...
(MarkerShape)
minsize
.
The available options are:
filled_circle
open_circle
cross
x
open_square
open_diamond
open_triangle_up
open_triangle_down
fat_circle
fat_cross
fat_x
fat_square
fat_diamond
fat_triangle_up
fat_triangle_down
filled_square
filled_diamond
filled_triangle_up
filled_triangle_down
[Default: x
]
minsizeN = <pixels>
(Integer)
[Default: 1
]
polymodeN = outline|border|fill|cross|star
(PolygonShape)
The available options are:
outline
: draws a line round the outside of the polygonborder
: draws a line butting up to the outside of the polygon; may look better for adjacent shapes, but more expensive to drawfill
: fills the interior of the polygoncross
: draws a line round the outside of the polygon and lines between all the verticesstar
: draws a line round the outside of the polygon and lines from the nominal center to each vertex[Default: outline
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
central
Plots the nominal central point of an area. This appears just like a normal marker plot, but can be used when the available geometry information is an area description (such as an STC-S string or an array of polygon vertices) rather than coordinate values such as an X,Y pair. The position plotted is the nominal center of the shape as determined by the plotting code; that may or may not correspond to the actual center.
Usage Overview:
layerN=central shapeN=filled_circle|open_circle|... sizeN=<pixels> shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> areaN=<area-expr> areatypeN=STC-S|POLYGON|CIRCLE|POINT|MOC-ASCII|UNIQ|TFCAT inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane xmin=136.7 xmax=138.5 ymin=-5.7 ymax=-4.2 in=crism.fits icmd='select sensor_id==0x53' area=s_region areatype=STC-S layer1=area polymode1=fill shading1=density densemap1=heat layer2=central shape2=fat_circle size2=3 color2=black
areaN = <area-expr>
(String)
areatype
parameter.
The value is a Area value algebraic expression based on column names as described in Section 10.
areatypeN = STC-S|POLYGON|CIRCLE|POINT|MOC-ASCII|UNIQ|TFCAT
(AreaMapper)
areaN
is supplied.
Options are:
STC-S
: Region description using STC-S syntax;
see TAP 1.0, section 6.
Note there are some restrictions:
<frame>
, <refpos>
and <flavor>
metadata are ignored,
polygon winding direction is ignored (small polygons are assumed)
and the INTERSECTION
and NOT
constructions are not supported.
The non-standard MOC
construction is supported.POLYGON
: 2n-element array (x1
,y1
, x2
,y2
,...);
a NaN
,NaN
pair can be used to delimit distinct polygons.CIRCLE
: 3-element array (x
, y
,r
)POINT
: 2-element array (x
,y
)MOC-ASCII
: Region description using ASCII MOC syntax;
see MOC 1.1 2.3.2.
Note there are currently a few issues
with MOC plotting, especially for large pixels.UNIQ
: Region description representing a single HEALPix cell
as defined by an UNIQ value, see
MOC 1.1
sec 2.3.1.TFCAT
: Time-Frequency region defined by the
TFCat standard.
Support is currently incomplete;
holes in Polygons and MultiPolygons are not displayed correctly,
single Points may not be displayed,
and Coordinate Reference System information is ignored.
areaN
value.
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
shapeN = filled_circle|open_circle|...
(MarkerShape)
The available options are:
filled_circle
open_circle
cross
x
open_square
open_diamond
open_triangle_up
open_triangle_down
fat_circle
fat_cross
fat_x
fat_square
fat_diamond
fat_triangle_up
fat_triangle_down
filled_square
filled_diamond
filled_triangle_up
filled_triangle_down
[Default: filled_circle
]
sizeN = <pixels>
(Integer)
[Default: 1
]
lines
Plots an N-segment line for each input row, with the X and Y coordinate arrays each supplied by an N-element array value.
Usage Overview:
layerN=lines thickN=<pixels> dashN=dot|dash|...|<a,b,...> sortaxisN=[X|Y] shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> xsN=<array-expr> ysN=<array-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in1=LRS_NPW_V010_20071101.cdf layer1=lines xs=multiply(param$frequency,1e-6) xlabel=f/MHz ys=RX1 thick=2 shading=aux aux=Epoch auxmap=sron icmd='every 100' xmin=13 xmax=16 xpix=660 auxvisible=false
dashN = dot|dash|...|<a,b,...>
(float[])
Possible values for dashed lines are
dot
, dash
, longdash
, dotdash
.
You can alternatively supply a comma-separated list
of on/off length values such as
"4,2,8,2
".
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
sortaxisN = [X|Y]
(AxisOpt)
X
" or
"Y
"
to ensure that the points for each line
are plotted in ascending order
of the corresponding coordinate.
This will ensure that the plotted line resembles a
function of the corresponding coordinate rather than
a scribble.
The default (null) value causes the points for each line
to be joined
in the sequence in which they appear in the arrays.
If the points already appear in the arrays sorted
according to the corresponding coordinate,
this option has no visible effect,
though it may slow things down.
[Default: None
]
thickN = <pixels>
(Integer)
[Default: 1
]
xsN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
ysN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
marks
Plots N markers for each input row, with the X and Y coordinate values each supplied by an N-element array value.
Usage Overview:
layerN=marks shapeN=filled_circle|open_circle|... sizeN=<pixels> shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> xsN=<array-expr> ysN=<array-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=LRS_NPW_V010_20071101.cdf xs=param$frequency ys=RX2 layer1=lines shading1=density densemap1=greyscale denseclip1=0.2,0.7 layer2=marks shading2=weighted weight2=epoch shape2=filled_triangle_down size2=4 xmin=13e6 xmax=16e6 xpix=660 icmd='head 50' auxmap=sron auxvisible=false
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
shapeN = filled_circle|open_circle|...
(MarkerShape)
The available options are:
filled_circle
open_circle
cross
x
open_square
open_diamond
open_triangle_up
open_triangle_down
fat_circle
fat_cross
fat_x
fat_square
fat_diamond
fat_triangle_up
fat_triangle_down
filled_square
filled_diamond
filled_triangle_up
filled_triangle_down
[Default: filled_circle
]
sizeN = <pixels>
(Integer)
[Default: 1
]
xsN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
ysN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
handles
Draws a symbol representing the position of an X/Y array plot. Although this may not do a good job of showing the position for a whole X/Y array, which is line-like rather than point-like, it provides a visible reference position for the plotted row.
This plot type is therefore mostly useful in interactive environments like TOPCAT, where the plotted marker can be used for activating or identifying the corresponding table row.
Usage Overview:
layerN=handles placementN=index|ymax|ymin|xmax|xmin|xymean fractionN=<0..1> sizeN=<pixels> shapeN=filled_circle|open_circle|... shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> xsN=<array-expr> ysN=<array-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=LRS_NPW_V010_20071101.cdf ys=add(RX1,20*$index) shading=aux auxmap=sron aux=$index icmd='head 8' auxvisible=false legend=false layer0=lines opaque0=2 layer1=handles placement1=index fraction1=0.85
fractionN = <0..1>
(Double)
placement=index
,
where it indicates how far through the array
the reference (X,Y) position should be taken
(0.0 means the first element, 1.0 means the last).
For other values of
placement
it is ignored.
[Default: 0.5
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
placementN = index|ymax|ymin|xmax|xmin|xymean
(XYArrayPlacement)
The available options are:
index
: (X,Y) point at a certain fraction of the way through the arrays, as given by the fraction
value; fraction=0.0
is the first element, fraction=1.0
is the last.ymax
: (X,Y) position at which the maximum Y value is located (fraction
is ignored)ymin
: (X,Y) position at which the minimum Y value is located (fraction
is ignored)xmax
: (X,Y) position at which the maximum X value is located (fraction
is ignored)xmin
: (X,Y) position at which the minimum X value is located (fraction
is ignored)xymean
: center of gravity of all the (X,Y) points (fraction
is ignored)[Default: index
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
shapeN = filled_circle|open_circle|...
(MarkerShape)
The available options are:
filled_circle
open_circle
cross
x
open_square
open_diamond
open_triangle_up
open_triangle_down
fat_circle
fat_cross
fat_x
fat_square
fat_diamond
fat_triangle_up
fat_triangle_down
filled_square
filled_diamond
filled_triangle_up
filled_triangle_down
[Default: fat_square
]
sizeN = <pixels>
(Integer)
[Default: 4
]
xsN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
ysN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
yerrors
Plots N error bars in the Y direction for each input row, with the X, Y and error bar extents each supplied by N-element array values.
Usage Overview:
layerN=yerrors errorbarN=none|lines|capped_lines|caps|arrows thickN=<int-value> shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> xsN=<array-expr> ysN=<array-expr> yerrhisN=<array-expr> yerrlosN=<array-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=LRS_NPW_V010_20071101.cdf shading=aux aux=epoch xs=divide(2.998e8,param$Frequency) ys=multiply(add(RX1,RX2),0.5) layer_l=lines thick_l=2 layer_e=yerrors yerrhis_e=arrayFunc("abs(x)",subtract(RX1,RX2)) errorbar_e=capped_lines auxmap=paired auxvisible=false xmin=116 xmax=161 ymin=-184 ymax=-148 xpix=660 ypix=300 icmd='every 1000' xlabel=lambda ylabel=Intensity
errorbarN = none|lines|capped_lines|caps|arrows
(MultiPointShape)
The available options are:
none
lines
capped_lines
caps
arrows
[Default: lines
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
xsN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
yerrhisN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
yerrlosN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
ysN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
xyerrors
Plots N error bars in the X and Y directions for each input row, with the X, Y and error bar extents each supplied by N-element array values.
Usage Overview:
layerN=xyerrors errorbarN=none|lines|capped_lines|... thickN=<int-value> shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> xsN=<array-expr> ysN=<array-expr> xerrhisN=<array-expr> xerrlosN=<array-expr> yerrhisN=<array-expr> yerrlosN=<array-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=LRS_NPW_V010_20071101.cdf ys=RX1 shading=aux aux=epoch layer_l=lines opaque_l=1 layer_xy=xyerrors opaque_xy=3.3 errorbar_xy=crosshair_ellipse xerrhis_xy=constant(512,0.5) yerrhis_xy=arrayFunc("abs(x)",subtract(RX1,RX2)) xmin=125 xmax=144 ymin=-184 ymax=-145 xpix=660 icmd='every 1000' auxmap=viridis auxvisible=false
errorbarN = none|lines|capped_lines|...
(MultiPointShape)
The available options are:
none
lines
capped_lines
caps
arrows
ellipse
crosshair_ellipse
rectangle
crosshair_rectangle
filled_ellipse
filled_rectangle
[Default: lines
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
xerrhisN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
xerrlosN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
xsN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
yerrhisN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
yerrlosN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
ysN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
statline
Plots a single line based on a combination (typically the mean) of input array-valued coordinates. The input X and Y coordinates must be fixed-length arrays of length N; a line with N points is plotted, each point representing the mean (or median, minimum, maximum, ...) of all the input array elements at the corresponding position.
Note that because the X and Y arrays must be of a fixed size for all rows, and because combination is performed in both X and Y directions, this is typically only suitable for plotting combined spectra if they all share a common horizontal axis, e.g. are all sampled into the same wavelength bins. To visually combine spectra with non-uniform sampling, the arrayquantile plotter may be more useful.
Usage Overview:
layerN=statline xcombineN=mean|median|min|max|q.01|... ycombineN=mean|median|min|max|q.01|... colorN=<rrggbb>|red|blue|... thickN=<pixels> antialiasN=true|false xsN=<array-expr> ysN=<array-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=LRS_NPW_V010_20071101.cdf xs=multiply(param$frequency,1e-6) ys=RX2 xlabel=f/MHz ylabel=RX2/dB xmin=0.7 xmax=2.0 icmd='select rx2[71]<-170' xpix=700 layer1=lines color1=cyan layer2=statline color2=red thick2=3
antialiasN = true|false
(Boolean)
[Default: false
]
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
thickN = <pixels>
(Integer)
[Default: 3
]
xcombineN = mean|median|min|max|q.01|...
(Combiner)
The available options are:
mean
: the mean of the combined valuesmedian
: the medianmin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuesq.01
: 1st percentileq1
: first quartileq3
: third quartileq.99
: 99th percentilestdev
: the sample standard deviation of the combined valuessum
: the sum of all the combined values per bincount
: the number of non-blank values per bin (weight is ignored)[Default: mean
]
xsN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
ycombineN = mean|median|min|max|q.01|...
(Combiner)
The available options are:
mean
: the mean of the combined valuesmedian
: the medianmin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuesq.01
: 1st percentileq1
: first quartileq3
: third quartileq.99
: 99th percentilestdev
: the sample standard deviation of the combined valuessum
: the sum of all the combined values per bincount
: the number of non-blank values per bin (weight is ignored)[Default: mean
]
ysN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
statmark
Plots a set of markers based on a combination (typically the mean) of input array-valued coordinates. The input X and Y coordinates must be fixed-length arrays of length N; N markers are plotted, each one representing the mean (or median, minimum, maximum, ...) of all the input array elements at the corresponding position.
Note that because the X and Y arrays must be of a fixed size for all rows, and because combination is performed in both X and Y directions, this is typically only suitable for plotting combined spectra if they all share a common horizontal axis, e.g. are all sampled into the same wavelength bins. To visually combine spectra with non-uniform sampling, the arrayquantile plotter may be more useful.
Usage Overview:
layerN=statmark xcombineN=mean|median|min|max|q.01|... ycombineN=mean|median|min|max|q.01|... colorN=<rrggbb>|red|blue|... shapeN=filled_circle|open_circle|... sizeN=<pixels> xsN=<array-expr> ysN=<array-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=LRS_NPW_V010_20071101.cdf ys=RX1 xmin=115 xmax=145 ymin=-183 ymax=-149 xpix=700 xcrowd=0.8 layer-d=lines color-d=wheat layer-m=statline ycombine-m=median color-m=LimeGreen thick-m=3 color-q=DodgerBlue size-q=4 layer-q1=statmark ycombine-q1=Q1 shape-q1=filled_triangle_up layer-q3=statmark ycombine-q3=Q3 shape-q3=filled_triangle_down leglabel-m=Median leglabel-q1='First Quartile' leglabel-q3='Third Quartile' legseq=-q3,-m,-q1 legpos=0.98,0.93
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shapeN = filled_circle|open_circle|...
(MarkerShape)
The available options are:
filled_circle
open_circle
cross
x
open_square
open_diamond
open_triangle_up
open_triangle_down
fat_circle
fat_cross
fat_x
fat_square
fat_diamond
fat_triangle_up
fat_triangle_down
filled_square
filled_diamond
filled_triangle_up
filled_triangle_down
[Default: filled_circle
]
sizeN = <pixels>
(Integer)
[Default: 4
]
xcombineN = mean|median|min|max|q.01|...
(Combiner)
The available options are:
mean
: the mean of the combined valuesmedian
: the medianmin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuesq.01
: 1st percentileq1
: first quartileq3
: third quartileq.99
: 99th percentilestdev
: the sample standard deviation of the combined valuessum
: the sum of all the combined values per bincount
: the number of non-blank values per bin (weight is ignored)[Default: mean
]
xsN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
ycombineN = mean|median|min|max|q.01|...
(Combiner)
The available options are:
mean
: the mean of the combined valuesmedian
: the medianmin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuesq.01
: 1st percentileq1
: first quartileq3
: third quartileq.99
: 99th percentilestdev
: the sample standard deviation of the combined valuessum
: the sum of all the combined values per bincount
: the number of non-blank values per bin (weight is ignored)[Default: mean
]
ysN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
arrayquantile
Displays a quantile or quantile range for a set of plotted X/Y array pairs. If a table contains one spectrum per row in array-valued wavelength and flux columns, this plotter can be used to display a median of all the spectra, or a range between two quantiles. Smoothing options are available to even out noise arising from the pixel binning.
For each row, the
xs
and
ys
arrays
must be the same length as each other,
but this plot type does not require all the arrays
to be sampled into the same bins.
The algorithm calculates quantiles for all the X,Y points plotted in each column of pixels. This means that more densely sampled spectra have more influence on the output than sparser ones.
Note: in the current implementation, depending on the details of the configuration and the data, there may be some distortions or missing graphics near the edges of the plot. This may be improved in future releases, depending on feedback.
Usage Overview:
layerN=arrayquantile colorN=<rrggbb>|red|blue|... transparencyN=0..1 quantilesN=<low-frac>[,<high-frac>] thickN=<pixels> smoothN=+<width>|-<count> kernelN=square|linear|epanechnikov|cos|cos2|gauss3|gauss6 joinN=none|polygon|lines horizontalN=true|false xsN=<array-expr> ysN=<array-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=xq100sub.fits xs=subWave ys=multiply(subFlux,1./mean(subFlux)) xlabel=Wavelength/nm ylabel='Normalised Flux' x2func=SPEED_OF_LIGHT*1E9*1E-12/x x2label=Frequency/THz layer1=lines shading1=density densemap1=greyscale denseclip1=0.2,1 densefunc1=linear leglabel1=Spectra layer_q13=ArrayQuantile color_q13=DodgerBlue transparency_q13=0.5 quantiles_q13=0.25,0.75 leglabel_q13='Quartile Range' layer_med=ArrayQuantile color_med=blue join_med=lines leglabel_med=Median legend=true legpos=0.95,0.95 xpix=600 ypix=380 xmin=1348 xmax=1358 ymin=-0.2 ymax=2.2
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
horizontalN = true|false
(Boolean)
true
, y quantiles are calculated
for each pixel column, and
if false
, x quantiles are calculated
for each pixel row.
[Default: true
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
joinN = none|polygon|lines
(QJoin)
The available options are:
none
: displayed quantile ranges are not joinedpolygon
: the area between a line connecting the upper quantiles and a line connecting the lower quantiles is filledlines
: a line of thickness given by thick
is drawn from the center of each quantile range to the next[Default: polygon
]
kernelN = square|linear|epanechnikov|cos|cos2|gauss3|gauss6
(Kernel1dShape)
The available options are:
square
: Uniform value: f(x)=1, |x|=0..1linear
: Triangle: f(x)=1-|x|, |x|=0..1epanechnikov
: Parabola: f(x)=1-x*x, |x|=0..1cos
: Cosine: f(x)=cos(x*pi/2), |x|=0..1cos2
: Cosine squared: f(x)=cos^2(x*pi/2), |x|=0..1gauss3
: Gaussian truncated at 3.0 sigma: f(x)=exp(-x*x/2), |x|=0..3gauss6
: Gaussian truncated at 6.0 sigma: f(x)=exp(-x*x/2), |x|=0..6[Default: epanechnikov
]
quantilesN = <low-frac>[,<high-frac>]
(Subrange)
<lo>,<hi>
)
indicating two quantile lines bounding an area to be filled.
A pair of equal values "a,a
"
is equivalent to the single value "a
".
The default is 0.5
,
which means to mark the median value in each column,
and could equivalently be specified 0.5,0.5
.
[Default: 0.5
]
smoothN = +<width>|-<count>
(BinSizer)
If the supplied value is a positive number it is interpreted as a fixed width in the data coordinates of the X axis (if the X axis is logarithmic, the value is a fixed factor). If it is a negative number, then it will be interpreted as the approximate number of smooothing widths that fit in the width of the visible plot (i.e. plot width / smoothing width). If the value is zero, no smoothing is applied.
When setting this value graphically, you can use either the slider to adjust the bin count or the numeric entry field to fix the bin width.
[Default: 0
]
thickN = <pixels>
(Integer)
quantiles
specifies a single value rather than a pair)
this will give the actual thickness of the plotted line.
If the range is non-zero however, the line may be thicker
than this in places according to the quantile positions.
[Default: 3
]
transparencyN = 0..1
(Double)
[Default: 0
]
xsN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
ysN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
line
Plots a point-to-point line joining
up the positions of data points.
There are additional options to pre-sort the points
according to their order on the X or Y axis (using the
sortaxis
value),
and to vary the colour of the line along its length (using the
aux
value).
The options for controlling the Aux colour map are controlled at the level of the plot itself, rather than by per-layer configuration.
Usage Overview:
layerN=line colorN=<rrggbb>|red|blue|... thickN=<pixels> dashN=dot|dash|...|<a,b,...> sortaxisN=[X|Y] antialiasN=true|false auxnullcolorN=<rrggbb>|red|blue|... <pos-coord-paramsN> auxN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2time in=ACE_data.vot t=epoch layer1=line y1=Br zone1=A layer2=line y2=Bt zone2=B layer3=line y3=Bn zone3=C
antialiasN = true|false
(Boolean)
[Default: false
]
auxN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
auxnullcolorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
If the value is null, then points with a null Aux value will not be plotted at all.
[Default: grey
]
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
dashN = dot|dash|...|<a,b,...>
(float[])
Possible values for dashed lines are
dot
, dash
, longdash
, dotdash
.
You can alternatively supply a comma-separated list
of on/off length values such as
"4,2,8,2
".
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
sortaxisN = [X|Y]
(AxisOpt)
X
" or
"Y
"
to ensure that the points are plotted in ascending order
of the corresponding coordinate.
This will ensure that the plotted line resembles a
function of the corresponding coordinate rather than
a scribble.
The default (null) value causes the points to be joined
in the sequence in which they appear in the table.
If the points already appear in the table sorted
according to the corresponding coordinate,
this option has no visible effect,
though it may slow things down.
[Default: None
]
thickN = <pixels>
(Integer)
[Default: 1
]
linearfit
Plots a line of best fit for the data points.
Usage Overview:
layerN=linearfit colorN=<rrggbb>|red|blue|... thickN=<pixels> dashN=dot|dash|...|<a,b,...> antialiasN=true|false <pos-coord-paramsN> weightN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2plane in=6dfgs_mini.xml x=RMAG y=BMAG layer1=mark layer2=linearfit
antialiasN = true|false
(Boolean)
[Default: false
]
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
dashN = dot|dash|...|<a,b,...>
(float[])
Possible values for dashed lines are
dot
, dash
, longdash
, dotdash
.
You can alternatively supply a comma-separated list
of on/off length values such as
"4,2,8,2
".
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
thickN = <pixels>
(Integer)
[Default: 1
]
weightN = <num-expr>
(String)
Given certain assumptions about independence of samples,
a suitable value for the weight may be
1/(err*err)
, if err
is the
measurement error for each Y value.
The value is a numeric algebraic expression based on column names as described in Section 10.
label
Draws a text label at each position. You can select the font, where the labels appear in relation to the point positions, and how crowded the points have to get before they are suppressed.
Usage Overview:
layerN=label texttypeN=plain|antialias|latex fontsizeN=<int-value> fontstyleN=standard|serif|mono fontweightN=plain|bold|italic|bold_italic anchorN=west|east|north|south|center colorN=<rrggbb>|red|blue|... xoffN=<pixels> yoffN=<pixels> spacingN=<pixels> crowdlimitN=<n> <pos-coord-paramsN> labelN=<txt-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2sky in=messier.xml lon=RA lat=DEC layer1=mark size1=3 layer2=label label2=NAME color2=black
anchorN = west|east|north|south|center
(Anchor)
The available options are:
west
east
north
south
center
[Default: west
]
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
crowdlimitN = <n>
(Integer)
It is useful for instance if you are looking at pairs of points, which will always be close together; if you set this value to 2, an isolated pair of labels can be seen, but if it's 1 then they will only be plotted when they are distant from each other, which may only happen at very high magnifications.
[Default: 2
]
fontsizeN = <int-value>
(Integer)
[Default: 12
]
fontstyleN = standard|serif|mono
(FontType)
The available options are:
standard
serif
mono
[Default: standard
]
fontweightN = plain|bold|italic|bold_italic
(FontWeight)
The available options are:
plain
bold
italic
bold_italic
[Default: plain
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
labelN = <txt-expr>
(String)
The value is a string algebraic expression based on column names as described in Section 10.
spacingN = <pixels>
(Integer)
[Default: 12
]
texttypeN = plain|antialias|latex
(TextSyntax)
Plain
and
Antialias
both take the text at face value,
but Antialias
smooths the characters.
LaTeX
interprets the text as LaTeX source code
and typesets it accordingly.
When not using LaTeX, antialiased text usually looks nicer, but can be perceptibly slower to plot. At time of writing, on MacOS antialiased text seems to be required to stop the writing coming out upside-down for non-horizontal text (MacOS java bug).
[Default: plain
]
xoffN = <pixels>
(Integer)
[Default: 0
]
yoffN = <pixels>
(Integer)
[Default: 0
]
arealabel
Draws a text label near the center of each area.You can select the font, where the labels appear in relation to the point positions, and how crowded the points have to get before they are suppressed.
This is just like a normal Label plot, but the positions are taken from an Area coordinate rather than normal positional coordinates.
Usage Overview:
layerN=arealabel texttypeN=plain|antialias|latex fontsizeN=<int-value> fontstyleN=standard|serif|mono fontweightN=plain|bold|italic|bold_italic anchorN=west|east|north|south|center colorN=<rrggbb>|red|blue|... xoffN=<pixels> yoffN=<pixels> spacingN=<pixels> crowdlimitN=<n> areaN=<area-expr> areatypeN=STC-S|POLYGON|CIRCLE|POINT|MOC-ASCII|UNIQ|TFCAT labelN=<txt-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2sky reflectlon=false sex=false clon=18 clat=0 radius=36 xpix=550 ypix=600 in=countries.vot area=shape areatype=STC-S layer_1=area polymode_1=fill shading_1=aux aux_1=index opaque_1=2 layer_2=area polymode_2=outline shading_2=flat color_2=grey auxmap=paired auxvisible=false layer_3=arealabel label_3=name anchor_3=center color_3=black
anchorN = west|east|north|south|center
(Anchor)
The available options are:
west
east
north
south
center
[Default: west
]
areaN = <area-expr>
(String)
areatype
parameter.
The value is a Area value algebraic expression based on column names as described in Section 10.
areatypeN = STC-S|POLYGON|CIRCLE|POINT|MOC-ASCII|UNIQ|TFCAT
(AreaMapper)
areaN
is supplied.
Options are:
STC-S
: Region description using STC-S syntax;
see TAP 1.0, section 6.
Note there are some restrictions:
<frame>
, <refpos>
and <flavor>
metadata are ignored,
polygon winding direction is ignored (small polygons are assumed)
and the INTERSECTION
and NOT
constructions are not supported.
The non-standard MOC
construction is supported.POLYGON
: 2n-element array (x1
,y1
, x2
,y2
,...);
a NaN
,NaN
pair can be used to delimit distinct polygons.CIRCLE
: 3-element array (x
, y
,r
)POINT
: 2-element array (x
,y
)MOC-ASCII
: Region description using ASCII MOC syntax;
see MOC 1.1 2.3.2.
Note there are currently a few issues
with MOC plotting, especially for large pixels.UNIQ
: Region description representing a single HEALPix cell
as defined by an UNIQ value, see
MOC 1.1
sec 2.3.1.TFCAT
: Time-Frequency region defined by the
TFCat standard.
Support is currently incomplete;
holes in Polygons and MultiPolygons are not displayed correctly,
single Points may not be displayed,
and Coordinate Reference System information is ignored.
areaN
value.
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
crowdlimitN = <n>
(Integer)
It is useful for instance if you are looking at pairs of points, which will always be close together; if you set this value to 2, an isolated pair of labels can be seen, but if it's 1 then they will only be plotted when they are distant from each other, which may only happen at very high magnifications.
[Default: 2
]
fontsizeN = <int-value>
(Integer)
[Default: 12
]
fontstyleN = standard|serif|mono
(FontType)
The available options are:
standard
serif
mono
[Default: standard
]
fontweightN = plain|bold|italic|bold_italic
(FontWeight)
The available options are:
plain
bold
italic
bold_italic
[Default: plain
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
labelN = <txt-expr>
(String)
The value is a string algebraic expression based on column names as described in Section 10.
spacingN = <pixels>
(Integer)
[Default: 12
]
texttypeN = plain|antialias|latex
(TextSyntax)
Plain
and
Antialias
both take the text at face value,
but Antialias
smooths the characters.
LaTeX
interprets the text as LaTeX source code
and typesets it accordingly.
When not using LaTeX, antialiased text usually looks nicer, but can be perceptibly slower to plot. At time of writing, on MacOS antialiased text seems to be required to stop the writing coming out upside-down for non-horizontal text (MacOS java bug).
[Default: plain
]
xoffN = <pixels>
(Integer)
[Default: 0
]
yoffN = <pixels>
(Integer)
[Default: 0
]
contour
Plots position density contours. This provides another way (alongside the auto, density and weighted shading modes) to visualise the characteristics of overdense regions in a crowded plot. It's not very useful if you just have a few points.
A weighting may optionally be applied to the quantity
being contoured.
To do this, provide a non-blank value for the
weight
coordinate, and use the
combine
parameter to define how the weights are combined
(sum
,
mean
, etc).
The contours are currently drawn as pixels rather than lines so they don't look very beautiful in exported vector output formats (PDF, PostScript). This may be improved in the future.
Usage Overview:
layerN=contour colorN=<rrggbb>|red|blue|... combineN=sum|mean|median|stdev|min|max|count nlevelN=<int-value> smoothN=<pixels> thickN=<pixels> scalingN=linear|log|equal zeroN=<number> <pos-coord-paramsN> weightN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2plane in=tgas_source.fits x=phot_g_mean_mag y=phot_g_mean_flux_error ylog=true xmax=14 ymin=10 layer1=mark shading1=density densemap1=greyscale layer2=contour scaling2=log nlevel=6
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
combineN = sum|mean|median|stdev|min|max|count
(Combiner)
mean
which traces the
mean values of a quantity and
sum
which traces the
weighted sum.
Other values such as
median
are of dubious validity because of the way that the
smoothing is done.
This value is ignored if the weighting coordinate
weight
is not set.
The available options are:
sum
: the sum of all the combined values per binmean
: the mean of the combined valuesmedian
: the medianstdev
: the sample standard deviation of the combined valuesmin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuescount
: the number of non-blank values per bin (weight is ignored)[Default: sum
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
nlevelN = <int-value>
(Integer)
[Default: 5
]
scalingN = linear|log|equal
(LevelMode)
The available options are:
linear
: levels are equally spacedlog
: level logarithms are equally spacedequal
: levels are spaced to provide equal-area inter-contour regions[Default: linear
]
smoothN = <pixels>
(Integer)
[Default: 5
]
thickN = <pixels>
(Integer)
[Default: 1
]
weightN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
zeroN = <number>
(Double)
[Default: 1
]
grid
Plots 2-d data aggregated into rectangular cells. You can optionally use a weighting for the points, and you can configure how the values are combined to produce the output pixel values (colours). You can use this plotter in various ways, including as a 2-d histogram or weighted density map, or to plot gridded data.
The X and Y dimensions of the grid cells (or equivalently histogram bins) can be configured either in terms of the data coordinates or relative to the plot dimensions.
The way that data values are mapped to colours is usually controlled by options at the level of the plot itself, rather than by per-layer configuration.
Usage Overview:
layerN=grid xbinsizeN=+<extent>|-<count> ybinsizeN=+<extent>|-<count> combineN=sum|sum-per-unit|count|... transparencyN=0..1 xphaseN=<number> yphaseN=<number> <pos-coord-paramsN> weightN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2plane layer1=grid in1=gk_hess.fits x1=g_min_ks y1=g_mag_abs weight1=n combine1=sum xbinsize1=0.2 ybinsize1=0.2 xphase1=0.5 yphase1=0.5 yflip=true auxfunc=log auxmap=viridis
combineN = sum|sum-per-unit|count|...
(Combiner)
weight
coordinate
is left blank, a weighting of unity is assumed.
The available options are:
sum
: the sum of all the combined values per binsum-per-unit
: the sum of all the combined values per unit of bin sizecount
: the number of non-blank values per bin (weight is ignored)count-per-unit
: the number of non-blank values per unit of bin size (weight is ignored)mean
: the mean of the combined valuesmedian
: the medianq1
: first quartileq3
: third quartilemin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuesstdev
: the sample standard deviation of the combined valuesstdev_pop
: the population standard deviation of the combined valueshit
: 1 if any values present, NaN otherwise (weight is ignored)[Default: mean
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
transparencyN = 0..1
(Double)
[Default: 0
]
weightN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xbinsizeN = +<extent>|-<count>
(BinSizer)
If the supplied value is a positive number it is interpreted as a fixed size in data coordinates (if the X axis is logarithmic, the value is a fixed factor). If it is a negative number, then it will be interpreted as the approximate number of bins to display across the plot in the X direction (though an attempt is made to use only round numbers for bin sizes).
When setting this value graphically, you can use either the slider to adjust the bin count or the numeric entry field to fix the bin size.
[Default: -30
]
xphaseN = <number>
(Double)
A value of 0 (or any integer) will result in a bin boundary at X=0 (linear X axis) or X=1 (logarithmic X axis). A fractional value will give a bin boundary at that value multiplied by the bin width.
[Default: 0
]
ybinsizeN = +<extent>|-<count>
(BinSizer)
If the supplied value is a positive number it is interpreted as a fixed size in data coordinates (if the Y axis is logarithmic, the value is a fixed factor). If it is a negative number, then it will be interpreted as the approximate number of bins to display across the plot in the Y direction (though an attempt is made to use only round numbers for bin sizes).
When setting this value graphically, you can use either the slider to adjust the bin count or the numeric entry field to fix the bin size.
[Default: -30
]
yphaseN = <number>
(Double)
A value of 0 (or any integer) will result in a bin boundary at X=0 (linear X axis) or X=1 (logarithmic X axis). A fractional value will give a bin boundary at that value multiplied by the bin width.
[Default: 0
]
fill
If a two-dimensional dataset represents a single-valued function, this plotter will fill the area underneath the function's curve with a solid colour. Parts of the surface which would only be partially covered (because of rapid function variation within the width of a single pixel) are represented using appropriate alpha-blending. The filled area may alternatively be that above the curve or to its left or right.
One example of its use is to reconstruct the appearance of a histogram plot from a set of histogram bins. For X,Y data which is not single-valued, the result may not be very useful.
Usage Overview:
layerN=fill colorN=<rrggbb>|red|blue|... transparencyN=0..1 horizontalN=true|false positiveN=true|false <pos-coord-paramsN> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2time layer1=fill in1=iers.fits t1=decYear y1=lodErr ylog=true texttype=latex fontsize=16
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
horizontalN = true|false
(Boolean)
[Default: false
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
positiveN = true|false
(Boolean)
[Default: false
]
transparencyN = 0..1
(Double)
[Default: 0
]
quantile
Plots a line through a given quantile of the values binned within each pixel column (or row) of a plot. The line is optionally smoothed using a configurable kernel and width, to even out noise arising from the pixel binning. Instead of a simple line through a given quantile, it is also possible to fill the region between two quantiles.
One way to use this is to draw a line estimating a function y=f(x) (or x=g(y)) sampled by a noisy set of data points in two dimensions.
Note: in the current implementation, depending on the details of the configuration and the data, there may be some distortions or missing graphics near the edges of the plot. This may be improved in future releases, depending on feedback.
Usage Overview:
layerN=quantile colorN=<rrggbb>|red|blue|... transparencyN=0..1 quantilesN=<low-frac>[,<high-frac>] thickN=<pixels> smoothN=+<width>|-<count> kernelN=square|linear|epanechnikov|cos|cos2|gauss3|gauss6 joinN=none|polygon|lines horizontalN=true|false <pos-coord-paramsN> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2plane in=tgas_source.fits x=phot_g_mean_mag y=phot_g_mean_flux_error ylog=true xmax=15 ymin=10 layer.d=mark color.d=99ff99 layer.q4=quantile quantiles.q4=0.25,0.75 color.q4=magenta transparency.q4=0.35 layer.q2=quantile quantiles.q2=0.5 color.q2=SkyBlue thick.q2=4 smooth.q=0.05 leglabel.q4=Quartiles leglabel.q2=Median legseq=.q4,.q2 legpos=0.95,0.95
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
horizontalN = true|false
(Boolean)
true
, y quantiles are calculated
for each pixel column, and
if false
, x quantiles are calculated
for each pixel row.
[Default: true
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
joinN = none|polygon|lines
(QJoin)
The available options are:
none
: displayed quantile ranges are not joinedpolygon
: the area between a line connecting the upper quantiles and a line connecting the lower quantiles is filledlines
: a line of thickness given by thick
is drawn from the center of each quantile range to the next[Default: none
]
kernelN = square|linear|epanechnikov|cos|cos2|gauss3|gauss6
(Kernel1dShape)
The available options are:
square
: Uniform value: f(x)=1, |x|=0..1linear
: Triangle: f(x)=1-|x|, |x|=0..1epanechnikov
: Parabola: f(x)=1-x*x, |x|=0..1cos
: Cosine: f(x)=cos(x*pi/2), |x|=0..1cos2
: Cosine squared: f(x)=cos^2(x*pi/2), |x|=0..1gauss3
: Gaussian truncated at 3.0 sigma: f(x)=exp(-x*x/2), |x|=0..3gauss6
: Gaussian truncated at 6.0 sigma: f(x)=exp(-x*x/2), |x|=0..6[Default: epanechnikov
]
quantilesN = <low-frac>[,<high-frac>]
(Subrange)
<lo>,<hi>
)
indicating two quantile lines bounding an area to be filled.
A pair of equal values "a,a
"
is equivalent to the single value "a
".
The default is 0.5
,
which means to mark the median value in each column,
and could equivalently be specified 0.5,0.5
.
[Default: 0.5
]
smoothN = +<width>|-<count>
(BinSizer)
If the supplied value is a positive number it is interpreted as a fixed width in the data coordinates of the X axis (if the X axis is logarithmic, the value is a fixed factor). If it is a negative number, then it will be interpreted as the approximate number of smooothing widths that fit in the width of the visible plot (i.e. plot width / smoothing width). If the value is zero, no smoothing is applied.
When setting this value graphically, you can use either the slider to adjust the bin count or the numeric entry field to fix the bin width.
[Default: 0
]
thickN = <pixels>
(Integer)
quantiles
specifies a single value rather than a pair)
this will give the actual thickness of the plotted line.
If the range is non-zero however, the line may be thicker
than this in places according to the quantile positions.
[Default: 3
]
transparencyN = 0..1
(Double)
[Default: 0
]
histogram
Plots a histogram.
Bin heights may optionally be weighted by the
values of some additional coordinate,
supplied using the
weight
parameter.
In this case you can choose how these weights are combined
in each bin using the
combine
parameter.
Various options are provided for configuring how the bar heights are calculated, but note that not all combinations of the available parameters will necessarily lead to meaningful visualisations.
Usage Overview:
layerN=histogram colorN=<rrggbb>|red|blue|... transparencyN=0..1 binsizeN=+<width>|-<count> phaseN=<number> combineN=sum|sum-per-unit|count|... sidewaysN=true|false cumulativeN=none|forward|reverse normaliseN=none|area|unit|maximum|height barformN=open|filled|semi_filled|steps|semi_steps|spikes thickN=<pixels> dashN=dot|dash|...|<a,b,...> xN=<num-expr> weightN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=histogram in1=rrlyrae.fits x1=p1
barformN = open|filled|semi_filled|steps|semi_steps|spikes
(Form)
The available options are:
open
filled
semi_filled
steps
semi_steps
spikes
[Default: semi_filled
]
binsizeN = +<width>|-<count>
(BinSizer)
When setting this value graphically, you can use either the slider to adjust the bin count or the numeric entry field to fix the bin width.
[Default: -30
]
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
combineN = sum|sum-per-unit|count|...
(Combiner)
weight
coordinate,
but if no weight is supplied,
a weighting of unity is assumed.
The available options are:
sum
: the sum of all the combined values per binsum-per-unit
: the sum of all the combined values per unit of bin sizecount
: the number of non-blank values per bin (weight is ignored)count-per-unit
: the number of non-blank values per unit of bin size (weight is ignored)mean
: the mean of the combined valuesmedian
: the medianq1
: first quartileq3
: third quartilemin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuesstdev
: the sample standard deviation of the combined valuesstdev_pop
: the population standard deviation of the combined valueshit
: 1 if any values present, NaN otherwise (weight is ignored)[Default: sum
]
cumulativeN = none|forward|reverse
(Cumulation)
forward
/reverse
the histogram bars plotted are calculated
cumulatively;
each bin includes the counts from all previous bins
working up/down the independent axis.
Note that setting cumulative plotting may not make much sense with some other parameter values, for instance averaging aggregation modes.
For reasons of backward compatibility,
the values true
and false
may be used as aliases for
forward
and none
.
The available options are:
none
: The value plotted for each bin uses the samples accumulated in that bin.forward
: The value plotted for each bin uses the samples accumulated all the way from negative infinity to that bin.reverse
: The value plotted for each bin uses the samples accumulated all the way from positive infinity to that bin.[Default: none
]
dashN = dot|dash|...|<a,b,...>
(float[])
Possible values for dashed lines are
dot
, dash
, longdash
, dotdash
.
You can alternatively supply a comma-separated list
of on/off length values such as
"4,2,8,2
".
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
normaliseN = none|area|unit|maximum|height
(Normalisation)
Note that some of the normalisation options may not make much sense with some other parameter values, for instance averaging aggregation modes.
The available options are:
none
: No normalisation is performed.area
: The total area of histogram bars is normalised to unity. For cumulative plots, this behaves like height
.unit
: Histogram bars are scaled by the inverse of the bin width in data units. For cumulative plots, this behaves like none
.maximum
: The height of the tallest histogram bar is normalised to unity. For cumulative plots, this behaves like height
.height
: The total height of histogram bars is normalised to unity.[Default: none
]
phaseN = <number>
(Double)
A value of 0 (or any integer) will result in a bin boundary at X=0 (linear X axis) or X=1 (logarithmic X axis). A fractional value will give a bin boundary at that value multiplied by the bin width.
[Default: 0
]
sidewaysN = true|false
(Boolean)
false
,
the quantity being accumulated is on the the horizontal axis
and the frequency is represented vertically as usual.
If set true
the quantity accumulated
is on the vertical axis,
and the frequency is represented horizontally,
so that the chart is displayed reflected in the X=Y line.
[Default: false
]
thickN = <pixels>
(Integer)
[Default: 2
]
transparencyN = 0..1
(Double)
[Default: 0
]
weightN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
kde
Plots a Discrete Kernel Density Estimate giving a smoothed frequency of data values along the horizontal axis, using a fixed-width smoothing kernel. This is a generalisation of a histogram in which the bins are always 1 pixel wide, and a smoothing kernel is applied to each bin. The width and shape of the kernel may be varied.
This is suitable for cases where the division into discrete bins done by a normal histogram is unnecessary or troublesome.
Note this is not a true Kernel Density Estimate, since, for performance reasons, the smoothing is applied to the (pixel-width) bins rather than to each data sample. The deviation from a true KDE caused by this quantisation will be at the pixel level, hence in most cases not visually apparent.
A weighting may be applied to the calculated levels
by supplying the
weight
coordinate.
In this case you can choose how these weights are aggregated
in each pixel bin using the
combine
parameter.
The result is something like a smoothed version of the
corresponding weighted histogram.
Note that some combinations of the available parameters
(e.g. a normalised cumulative median-aggregated KDE)
may not make much visual sense.
Usage Overview:
layerN=kde colorN=<rrggbb>|red|blue|... transparencyN=0..1 sidewaysN=true|false smoothN=+<width>|-<count> combineN=sum|sum-per-unit|count|... kernelN=square|linear|epanechnikov|cos|cos2|gauss3|gauss6 cumulativeN=none|forward|reverse normaliseN=none|area|unit|maximum|height fillN=solid|line|semi thickN=<pixels> xN=<num-expr> weightN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane ymin=0 layer1=kde in1=rrlyrae.fits x1=p1
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
combineN = sum|sum-per-unit|count|...
(Combiner)
sum-per-unit
and
mean
make sense,
others such as
sum
do not.
The combined values are those given by the
weight
coordinate,
but if no weight is supplied,
a weighting of unity is assumed.
The available options are:
sum
: the sum of all the combined values per binsum-per-unit
: the sum of all the combined values per unit of bin sizecount
: the number of non-blank values per bin (weight is ignored)count-per-unit
: the number of non-blank values per unit of bin size (weight is ignored)mean
: the mean of the combined valuesmedian
: the medianq1
: first quartileq3
: third quartilemin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuesstdev
: the sample standard deviation of the combined valuesstdev_pop
: the population standard deviation of the combined valueshit
: 1 if any values present, NaN otherwise (weight is ignored)[Default: sum-per-unit
]
cumulativeN = none|forward|reverse
(Cumulation)
forward
/reverse
the histogram bars plotted are calculated
cumulatively;
each bin includes the counts from all previous bins
working up/down the independent axis.
Note that setting cumulative plotting may not make much sense with some other parameter values, for instance averaging aggregation modes.
For reasons of backward compatibility,
the values true
and false
may be used as aliases for
forward
and none
.
The available options are:
none
: The value plotted for each bin uses the samples accumulated in that bin.forward
: The value plotted for each bin uses the samples accumulated all the way from negative infinity to that bin.reverse
: The value plotted for each bin uses the samples accumulated all the way from positive infinity to that bin.[Default: none
]
fillN = solid|line|semi
(FillMode)
The available options are:
solid
: area between level and axis is filled with solid colourline
: level is marked by a wiggly linesemi
: level is marked by a wiggly line, and area below it is filled with a transparent colour[Default: semi
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
kernelN = square|linear|epanechnikov|cos|cos2|gauss3|gauss6
(Kernel1dShape)
The available options are:
square
: Uniform value: f(x)=1, |x|=0..1linear
: Triangle: f(x)=1-|x|, |x|=0..1epanechnikov
: Parabola: f(x)=1-x*x, |x|=0..1cos
: Cosine: f(x)=cos(x*pi/2), |x|=0..1cos2
: Cosine squared: f(x)=cos^2(x*pi/2), |x|=0..1gauss3
: Gaussian truncated at 3.0 sigma: f(x)=exp(-x*x/2), |x|=0..3gauss6
: Gaussian truncated at 6.0 sigma: f(x)=exp(-x*x/2), |x|=0..6[Default: epanechnikov
]
normaliseN = none|area|unit|maximum|height
(Normalisation)
Note that some of the normalisation options may not make much sense with some other parameter values, for instance averaging aggregation modes.
The available options are:
none
: No normalisation is performed.area
: The total area of histogram bars is normalised to unity. For cumulative plots, this behaves like height
.unit
: Histogram bars are scaled by the inverse of the bin width in data units. For cumulative plots, this behaves like none
.maximum
: The height of the tallest histogram bar is normalised to unity. For cumulative plots, this behaves like height
.height
: The total height of histogram bars is normalised to unity.[Default: none
]
sidewaysN = true|false
(Boolean)
false
,
the quantity being accumulated is on the the horizontal axis
and the frequency is represented vertically as usual.
If set true
the quantity accumulated
is on the vertical axis,
and the frequency is represented horizontally,
so that the chart is displayed reflected in the X=Y line.
[Default: false
]
smoothN = +<width>|-<count>
(BinSizer)
If the supplied value is a positive number it is interpreted as a fixed width in the data coordinates of the X axis (if the X axis is logarithmic, the value is a fixed factor). If it is a negative number, then it will be interpreted as the approximate number of smooothing widths that fit in the width of the visible plot (i.e. plot width / smoothing width). If the value is zero, no smoothing is applied.
When setting this value graphically, you can use either the slider to adjust the bin count or the numeric entry field to fix the bin width.
[Default: -100
]
thickN = <pixels>
(Integer)
[Default: 2
]
transparencyN = 0..1
(Double)
[Default: 0
]
weightN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
knn
Plots a Discrete Kernel Density Estimate giving a smoothed frequency of data values along the horizontal axis, using an adaptive (K-Nearest-Neighbours) smoothing kernel. This is a generalisation of a histogram in which the bins are always 1 pixel wide, and a smoothing kernel is applied to each bin. The width and shape of the kernel may be varied.
The K-Nearest-Neighbour figure gives the number of points in each direction to determine the width of the smoothing kernel for smoothing each bin. Upper and lower limits for the kernel width are also supplied; if the upper and lower limits are equal, this is equivalent to a fixed-width kernel.
Note this is not a true Kernel Density Estimate, since, for performance reasons, the smoothing is applied to the (pixel-width) bins rather than to each data sample. The deviation from a true KDE caused by this quantisation will be at the pixel level, hence in most cases not visually apparent.
Usage Overview:
layerN=knn colorN=<rrggbb>|red|blue|... transparencyN=0..1 sidewaysN=true|false knnN=<number> symmetricN=true|false minsmoothN=+<width>|-<count> maxsmoothN=+<width>|-<count> kernelN=square|linear|epanechnikov|cos|cos2|gauss3|gauss6 cumulativeN=none|forward|reverse normaliseN=none|area|unit|maximum|height fillN=solid|line|semi thickN=<pixels> xN=<num-expr> weightN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=knn in1=rrlyrae.fits x1=p1
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
cumulativeN = none|forward|reverse
(Cumulation)
forward
/reverse
the histogram bars plotted are calculated
cumulatively;
each bin includes the counts from all previous bins
working up/down the independent axis.
Note that setting cumulative plotting may not make much sense with some other parameter values, for instance averaging aggregation modes.
For reasons of backward compatibility,
the values true
and false
may be used as aliases for
forward
and none
.
The available options are:
none
: The value plotted for each bin uses the samples accumulated in that bin.forward
: The value plotted for each bin uses the samples accumulated all the way from negative infinity to that bin.reverse
: The value plotted for each bin uses the samples accumulated all the way from positive infinity to that bin.[Default: none
]
fillN = solid|line|semi
(FillMode)
The available options are:
solid
: area between level and axis is filled with solid colourline
: level is marked by a wiggly linesemi
: level is marked by a wiggly line, and area below it is filled with a transparent colour[Default: semi
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
kernelN = square|linear|epanechnikov|cos|cos2|gauss3|gauss6
(Kernel1dShape)
The available options are:
square
: Uniform value: f(x)=1, |x|=0..1linear
: Triangle: f(x)=1-|x|, |x|=0..1epanechnikov
: Parabola: f(x)=1-x*x, |x|=0..1cos
: Cosine: f(x)=cos(x*pi/2), |x|=0..1cos2
: Cosine squared: f(x)=cos^2(x*pi/2), |x|=0..1gauss3
: Gaussian truncated at 3.0 sigma: f(x)=exp(-x*x/2), |x|=0..3gauss6
: Gaussian truncated at 6.0 sigma: f(x)=exp(-x*x/2), |x|=0..6[Default: epanechnikov
]
knnN = <number>
(Double)
The threshold is actually the weighted total of samples;
for unweighted (weight=1
) bins
that is equivalent to the number of samples.
[Default: 100
]
maxsmoothN = +<width>|-<count>
(BinSizer)
If the supplied value is a positive number it is interpreted as a fixed width in the data coordinates of the X axis (if the X axis is logarithmic, the value is a fixed factor). If it is a negative number, then it will be interpreted as the approximate number of smooothing widths that fit in the width of the visible plot (i.e. plot width / smoothing width). If the value is zero, no smoothing is applied.
When setting this value graphically, you can use either the slider to adjust the bin count or the numeric entry field to fix the bin width.
[Default: -100
]
minsmoothN = +<width>|-<count>
(BinSizer)
If the supplied value is a positive number it is interpreted as a fixed width in the data coordinates of the X axis (if the X axis is logarithmic, the value is a fixed factor). If it is a negative number, then it will be interpreted as the approximate number of smooothing widths that fit in the width of the visible plot (i.e. plot width / smoothing width). If the value is zero, no smoothing is applied.
When setting this value graphically, you can use either the slider to adjust the bin count or the numeric entry field to fix the bin width.
[Default: 0
]
normaliseN = none|area|unit|maximum|height
(Normalisation)
Note that some of the normalisation options may not make much sense with some other parameter values, for instance averaging aggregation modes.
The available options are:
none
: No normalisation is performed.area
: The total area of histogram bars is normalised to unity. For cumulative plots, this behaves like height
.unit
: Histogram bars are scaled by the inverse of the bin width in data units. For cumulative plots, this behaves like none
.maximum
: The height of the tallest histogram bar is normalised to unity. For cumulative plots, this behaves like height
.height
: The total height of histogram bars is normalised to unity.[Default: none
]
sidewaysN = true|false
(Boolean)
false
,
the quantity being accumulated is on the the horizontal axis
and the frequency is represented vertically as usual.
If set true
the quantity accumulated
is on the vertical axis,
and the frequency is represented horizontally,
so that the chart is displayed reflected in the X=Y line.
[Default: false
]
symmetricN = true|false
(Boolean)
[Default: true
]
thickN = <pixels>
(Integer)
[Default: 2
]
transparencyN = 0..1
(Double)
[Default: 0
]
weightN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
densogram
Represents smoothed density of data values along the horizontal axis using a colourmap. This is like a Kernel Density Estimate (smoothed histogram with bins 1 pixel wide), but instead of representing the data extent vertically as bars or a line, values are represented by a fixed-size pixel-width column of a colour from a colour map. A smoothing kernel, whose width and shape may be varied, is applied to each data point.
A weighting may be applied to the calculated levels
by supplying the
weight
coordinate.
In this case you can choose how these weights are aggregated
in each pixel bin using the
combine
parameter.
The result is something like a smoothed version of the
corresponding weighted histogram.
Note that some combinations of the available parameters
(e.g. a normalised cumulative median-aggregated KDE)
may not make much visual sense.
This is a rather unconventional way to represent density data, and this plotting mode is probably not very useful. But hey, nobody's forcing you to use it.
Usage Overview:
layerN=densogram colorN=<rrggbb>|red|blue|... smoothN=+<width>|-<count> sidewaysN=true|false kernelN=square|linear|epanechnikov|cos|cos2|gauss3|gauss6 densemapN=<map-name>|<color>-<color>[-<color>...] denseclipN=<lo>,<hi> denseflipN=true|false densequantN=<number> densesubN=<lo>,<hi> densefuncN=log|linear|histogram|histolog|sqrt|square|acos|cos cumulativeN=none|forward|reverse sizeN=<pixels> posN=<fraction> xN=<num-expr> weightN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=tgas_source.fits x=hypot(pmra_error,pmdec_error) xlog=true normalise=maximum color=grey layer1=histogram layer2=kde layer3=densogram densemap3=skyblue-yellow-hotpink densefunc3=log size3=50 pos3=0.5
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
cumulativeN = none|forward|reverse
(Cumulation)
forward
/reverse
the histogram bars plotted are calculated
cumulatively;
each bin includes the counts from all previous bins
working up/down the independent axis.
Note that setting cumulative plotting may not make much sense with some other parameter values, for instance averaging aggregation modes.
For reasons of backward compatibility,
the values true
and false
may be used as aliases for
forward
and none
.
The available options are:
none
: The value plotted for each bin uses the samples accumulated in that bin.forward
: The value plotted for each bin uses the samples accumulated all the way from negative infinity to that bin.reverse
: The value plotted for each bin uses the samples accumulated all the way from positive infinity to that bin.[Default: none
]
denseclipN = <lo>,<hi>
(Subrange)
If the full range 0,1
is used,
the whole range of colours specified by the selected
shader will be used.
But if for instance a value of 0,0.5
is given,
only those colours at the left hand end of the ramp
will be seen.
If the null (default) value is chosen,
a default clip will be used.
This generally covers most or all of the range 0-1
but for colour maps which fade to white,
a small proportion of the lower end may be excluded,
to ensure that all the colours are visually distinguishable
from a white background.
This default is usually a good idea if the colour map
is being used with something like a scatter plot,
where markers are plotted against a white background.
However, for something like a density map when the whole
plotting area is tiled with colours from the map,
it may be better to supply the whole range
0,1
explicitly.
denseflipN = true|false
(Boolean)
[Default: false
]
densefuncN = log|linear|histogram|histolog|sqrt|square|acos|cos
(Scaling)
The available options are:
log
: Logarithmic scalinglinear
: Linear scalinghistogram
: Scaling follows data distribution, with linear axishistolog
: Scaling follows data distribution, with logarithmic axissqrt
: Square root scalingsquare
: Square scalingacos
: Arccos Scalingcos
: Cos ScalingFor all these options,
the full range of data values is used,
and displayed on the colour bar
if applicable (though it can be restricted using the densesub
option)
The Linear
,
Log
,
Square
and
Sqrt
options
just apply the named function to the full data range.
The histogram options on the other hand use a scaling function
that corresponds to the actual distribution of the data, so that
there are about the same number of points (or pixels, or whatever
is being scaled) of each colour.
The histogram options are somewhat more expensive,
but can be a good choice if you are exploring data whose
distribution is unknown or not well-behaved over
its min-max range.
The Histogram
and
HistoLog
options both
assign the colours in the same way, but they display the colour
ramp with linear or logarithmic annotation respectively;
the HistoLog
option also ignores
non-positive values.
[Default: linear
]
densemapN = <map-name>|<color>-<color>[-<color>...]
(Shader)
A mixed bag of colour ramps are available
as listed in Section 8.7:
inferno
,
magma
,
plasma
,
viridis
,
cividis
,
cubehelix
,
sron
,
rainbow
,
rainbow2
,
rainbow3
,
pastel
,
cosmic
,
ember
,
gothic
,
rainforest
,
voltage
,
bubblegum
,
gem
,
chroma
,
sunset
,
neon
,
tropical
,
accent
,
gnuplot
,
gnuplot2
,
specxby
,
set1
,
paired
,
hotcold
,
guppy
,
iceburn
,
redshift
,
pride
,
rdbu
,
piyg
,
brbg
,
cyan-magenta
,
red-blue
,
brg
,
heat
,
cold
,
light
,
greyscale
,
colour
,
standard
,
bugn
,
bupu
,
orrd
,
pubu
,
purd
,
painbow
,
huecl
,
infinity
,
hue
,
intensity
,
rgb_red
,
rgb_green
,
rgb_blue
,
hsv_h
,
hsv_s
,
hsv_v
,
yuv_y
,
yuv_u
,
yuv_v
,
scale_hsv_s
,
scale_hsv_v
,
scale_yuv_y
,
mask
,
blacker
,
whiter
,
transparency
.
Note:
many of these, including rainbow-like ones,
are frowned upon by the visualisation community.
You can also construct your own custom colour map
by giving a sequence of colour names separated by
minus sign ("-
") characters.
In this case the ramp is a linear interpolation
between each pair of colours named,
using the same syntax as when specifying
a colour value.
So for instance
"yellow-hotpink-#0000ff
"
would shade from yellow via hot pink to blue.
[Default: inferno
]
densequantN = <number>
(Double)
If left blank, the colour map is nominally continuous (though in practice it may be quantised to a medium-sized number like 256).
densesubN = <lo>,<hi>
(Subrange)
The default value "0,1
" therefore has
no effect.
The range could be restricted to its lower half
with the value 0,0.5
.
[Default: 0,1
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
kernelN = square|linear|epanechnikov|cos|cos2|gauss3|gauss6
(Kernel1dShape)
The available options are:
square
: Uniform value: f(x)=1, |x|=0..1linear
: Triangle: f(x)=1-|x|, |x|=0..1epanechnikov
: Parabola: f(x)=1-x*x, |x|=0..1cos
: Cosine: f(x)=cos(x*pi/2), |x|=0..1cos2
: Cosine squared: f(x)=cos^2(x*pi/2), |x|=0..1gauss3
: Gaussian truncated at 3.0 sigma: f(x)=exp(-x*x/2), |x|=0..3gauss6
: Gaussian truncated at 6.0 sigma: f(x)=exp(-x*x/2), |x|=0..6[Default: epanechnikov
]
posN = <fraction>
(Double)
[Default: 0.05
]
sidewaysN = true|false
(Boolean)
false
,
the quantity being accumulated is on the the horizontal axis
and the frequency is represented vertically as usual.
If set true
the quantity accumulated
is on the vertical axis,
and the frequency is represented horizontally,
so that the chart is displayed reflected in the X=Y line.
[Default: false
]
sizeN = <pixels>
(Integer)
[Default: 12
]
smoothN = +<width>|-<count>
(BinSizer)
If the supplied value is a positive number it is interpreted as a fixed width in the data coordinates of the X axis (if the X axis is logarithmic, the value is a fixed factor). If it is a negative number, then it will be interpreted as the approximate number of smooothing widths that fit in the width of the visible plot (i.e. plot width / smoothing width). If the value is zero, no smoothing is applied.
When setting this value graphically, you can use either the slider to adjust the bin count or the numeric entry field to fix the bin width.
[Default: -100
]
weightN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
gaussian
Plots a best fit Gaussian to the histogram of a sample of data. In fact, all this plotter does is to calculate the mean and standard deviation of the sample, and plot the corresponding Gaussian curve. The mean and standard deviation values are reported by the plot.
The normalise
config option,
perhaps in conjunction with binsize
,
can be used to scale the height of the plotted curve
in data units.
In this case, binsize
just describes the bar width of a notional histogram
whose outline the plotted Gaussian should try to fit,
and is only relevant for some of the normalisation options.
Usage Overview:
layerN=gaussian colorN=<rrggbb>|red|blue|... showmeanN=true|false sidewaysN=true|false thickN=<pixels> dashN=dot|dash|...|<a,b,...> antialiasN=true|false normaliseN=none|area|unit|maximum|height binsizeN=+<width>|-<count> xN=<num-expr> weightN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane in=mgc_ok.fits x=mgc_dc_sb layer1=histogram color1=green layer2=gaussian color2=grey thick2=3 ymax=1200 shadow=false
antialiasN = true|false
(Boolean)
[Default: false
]
binsizeN = +<width>|-<count>
(BinSizer)
When setting this value graphically, you can use either the slider to adjust the bin count or the numeric entry field to fix the bin width.
[Default: -30
]
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
dashN = dot|dash|...|<a,b,...>
(float[])
Possible values for dashed lines are
dot
, dash
, longdash
, dotdash
.
You can alternatively supply a comma-separated list
of on/off length values such as
"4,2,8,2
".
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
normaliseN = none|area|unit|maximum|height
(Normalisation)
Note that some of the normalisation options may not make much sense with some other parameter values, for instance averaging aggregation modes.
The available options are:
none
: No normalisation is performed.area
: The total area of histogram bars is normalised to unity. For cumulative plots, this behaves like height
.unit
: Histogram bars are scaled by the inverse of the bin width in data units. For cumulative plots, this behaves like none
.maximum
: The height of the tallest histogram bar is normalised to unity. For cumulative plots, this behaves like height
.height
: The total height of histogram bars is normalised to unity.[Default: none
]
showmeanN = true|false
(Boolean)
[Default: true
]
sidewaysN = true|false
(Boolean)
false
,
the quantity being accumulated is on the the horizontal axis
and the frequency is represented vertically as usual.
If set true
the quantity accumulated
is on the vertical axis,
and the frequency is represented horizontally,
so that the chart is displayed reflected in the X=Y line.
[Default: false
]
thickN = <pixels>
(Integer)
[Default: 1
]
weightN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
function
Plots an analytic function. This layer is currently only available for the Plane plots (including histogram).
Usage Overview:
layerN=function axisN=Horizontal|Vertical xnameN=<name> fexprN=<expr> colorN=<rrggbb>|red|blue|... thickN=<pixels> dashN=dot|dash|...|<a,b,...> antialiasN=true|false
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=function fexpr1=sin(x)/x thick1=3 xmin=0 xmax=30 ymin=-0.25 ymax=0.25
antialiasN = true|false
(Boolean)
[Default: false
]
axisN = Horizontal|Vertical
(FuncAxis)
Horizontal
and
Vertical
.
[Default: Horizontal
]
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
dashN = dot|dash|...|<a,b,...>
(float[])
Possible values for dashed lines are
dot
, dash
, longdash
, dotdash
.
You can alternatively supply a comma-separated list
of on/off length values such as
"4,2,8,2
".
fexprN = <expr>
(String)
thickN = <pixels>
(Integer)
[Default: 1
]
xnameN = <name>
(String)
x
for a horizontal independent variable and
y
for a vertical independent variable,
but any string that is a legal expression language identifier
(starts with a letter, continues with letters, numbers,
underscores) can be used.
[Default: x
]
skyvector
Plots directed lines from the data position given delta values for the coordinates The plotted markers are typically little arrows, but there are other options.
The dimensions of the plotted vectors
are given by the
dlon
and dlat
coordinates.
The units of these values are specified using the
unit
option.
If only the relative rather than the absolute sizes
are required on the plot,
or if the units are not known,
the special value
unit=scaled
may be used;
this applies a non-physical scaling factor
to make the vectors appear at some reasonable size
in the plot.
When unit=scaled
vectors will keep approximately the same screen size
during zoom operations;
when one of the angular units is chosen, they will keep
the same size in data coordinates.
Additionally, the
scale
option
may be used to scale all the plotted vectors
by a given factor to make them all larger or smaller.
Usage Overview:
layerN=skyvector arrowN=small_arrow|medium_arrow|... thickN=<int-value> scaleN=<number> unitN=scaled|radian|degree|minute|arcsec|mas|uas shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> lonN=<deg-expr> latN=<deg-expr> dlonN=<num-expr> dlatN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2sky in=tgas_source.fits lon=ra lat=dec layer1=mark layer2=skyvector dlon2=pmra dlat2=pmdec unit2=scaled scale2=6 arrow2=medium_arrow clon=56.75 clat=24.10 radius=1.5
arrowN = small_arrow|medium_arrow|...
(MultiPointShape)
The available options are:
small_arrow
medium_arrow
large_arrow
small_open_dart
medium_open_dart
large_open_dart
small_filled_dart
medium_filled_dart
large_filled_dart
lines
capped_lines
[Default: small_arrow
]
dlatN = <num-expr>
(String)
unit
option.
The value is a numeric algebraic expression based on column names as described in Section 10.
dlonN = <num-expr>
(String)
unit
option.
The value is a numeric algebraic expression based on column names as described in Section 10.
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
latN = <deg-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
lonN = <deg-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
scaleN = <number>
(Double)
The main purpose of this option is to tweak
the visible sizes of the plotted markers for better visibility.
The unit
option
should be used to account for the units in which the
angular extent coordinates are supplied.
If the markers are supposed to be plotted with their
absolute angular extents visible, this option should be set
to its default value of 1.
[Default: 1
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
unitN = scaled|radian|degree|minute|arcsec|mas|uas
(AngleUnit)
scaled
is given
then a non-physical scaling is applied to the
input values to make the the largest markers
appear at a reasonable size (a few tens of pixels)
in the plot.
Note that the actual plotted size of the markers
can also be scaled using the
scale
option;
these two work together to determine the actual plotted sizes.
The available options are:
scaled
: a non-physical scaling is applied
based on the size of values present
radian
: radiansdegree
: degreesminute
: arcminutesarcsec
: arcsecondsmas
: milli-arcsecondsuas
: micro-arcseconds[Default: degree
]
skyellipse
Plots an ellipse (or rectangle, triangle, or other similar figure) defined by two principal radii and an optional angle of rotation, the so-called position angle. This angle, if specified, is in degrees and gives the angle from the North pole towards the direction of increasing longitude on the longitude axis.
The dimensions of the plotted ellipses
are given by the
ra
and rb
coordinates.
The units of these values are specified using the
unit
option.
If only the relative rather than the absolute sizes
are required on the plot,
or if the units are not known,
the special value
unit=scaled
may be used;
this applies a non-physical scaling factor
to make the ellipses appear at some reasonable size
in the plot.
When unit=scaled
ellipses will keep approximately the same screen size
during zoom operations;
when one of the angular units is chosen, they will keep
the same size in data coordinates.
Additionally, the
scale
option
may be used to scale all the plotted ellipses
by a given factor to make them all larger or smaller.
Usage Overview:
layerN=skyellipse ellipseN=ellipse|crosshair_ellipse|... thickN=<int-value> scaleN=<number> unitN=scaled|radian|degree|minute|arcsec|mas|uas shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> lonN=<deg-expr> latN=<deg-expr> raN=<num-expr> rbN=<num-expr> posangN=<deg-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2sky in=mgc_ok.fits lon=mgc_alpha_j2000 lat=mgc_delta_j2000 ra=bulge_re rb=bulge_re*bulge_e unit=arcsec posang=bulge_pa scale=10 color=#cc00ff layer1=skyellipse ellipse1=filled_ellipse shading1=transparent opaque1=4 layer2=skyellipse ellipse2=crosshair_ellipse clon=180.1 clat=0 radius=0.25
ellipseN = ellipse|crosshair_ellipse|...
(MultiPointShape)
The available options are:
ellipse
crosshair_ellipse
filled_ellipse
rectangle
crosshair_rectangle
filled_rectangle
open_triangle
filled_triangle
lines
capped_lines
arrows
[Default: ellipse
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
latN = <deg-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
lonN = <deg-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
posangN = <deg-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
raN = <num-expr>
(String)
unit
option.
The value is a numeric algebraic expression based on column names as described in Section 10.
rbN = <num-expr>
(String)
unit
option.
If this value is blank, the two radii will be assumed equal,
i.e. the ellipses will be circles.
The value is a numeric algebraic expression based on column names as described in Section 10.
scaleN = <number>
(Double)
The main purpose of this option is to tweak
the visible sizes of the plotted markers for better visibility.
The unit
option
should be used to account for the units in which the
angular extent coordinates are supplied.
If the markers are supposed to be plotted with their
absolute angular extents visible, this option should be set
to its default value of 1.
[Default: 1
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
unitN = scaled|radian|degree|minute|arcsec|mas|uas
(AngleUnit)
scaled
is given
then a non-physical scaling is applied to the
input values to make the the largest markers
appear at a reasonable size (a few tens of pixels)
in the plot.
Note that the actual plotted size of the markers
can also be scaled using the
scale
option;
these two work together to determine the actual plotted sizes.
The available options are:
scaled
: a non-physical scaling is applied
based on the size of values present
radian
: radiansdegree
: degreesminute
: arcminutesarcsec
: arcsecondsmas
: milli-arcsecondsuas
: micro-arcseconds[Default: degree
]
skycorr
Plots an error ellipse (or rectangle or other similar figure) on the sky defined by errors in the longitude and latitude directions, and a correlation between the two errors.
The error in longitude is considered to be premultiplied by the cosine of the latitude, i.e. both errors correspond to angular distances along a great circle.
The supplied correlation is a dimensionless value in the range -1..+1 and is equal to the covariance divided by the product of the lon and lat errors. The covariance matrix is thus:
[ lonerr*lonerr lonerr*laterr*corr ] [ lonerr*laterr*corr laterr*laterr ]
The dimensions of the plotted ellipses
are given by the
lonerr
and laterr
coordinates.
The units of these values are specified using the
unit
option.
If only the relative rather than the absolute sizes
are required on the plot,
or if the units are not known,
the special value
unit=scaled
may be used;
this applies a non-physical scaling factor
to make the ellipses appear at some reasonable size
in the plot.
When unit=scaled
ellipses will keep approximately the same screen size
during zoom operations;
when one of the angular units is chosen, they will keep
the same size in data coordinates.
Additionally, the
scale
option
may be used to scale all the plotted ellipses
by a given factor to make them all larger or smaller.
This plot type is suitable for use with the
ra_error
, dec_error
and
ra_dec_corr
columns
in the Gaia source catalogue.
Note that Gaia positional errors are generally quoted
in milli-arcseconds, so you should set
unit=mas
.
Note also that in most plots Gaia positional errors
are much too small to see!
Usage Overview:
layerN=skycorr ellipseN=ellipse|crosshair_ellipse|... thickN=<int-value> scaleN=<number> unitN=scaled|radian|degree|minute|arcsec|mas|uas shadingN=auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN> lonN=<deg-expr> latN=<deg-expr> lonerrN=<num-expr> laterrN=<num-expr> corrN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2sky in=tgas_source.fits lon=ra lat=dec icmd='select ra>245.1&&ra<245.9&&dec>-17.8&&dec<-17.2' color=blue layer1=mark unit=mas scale=2e5 ra2=ra_error rb2=dec_error posang2=90 color2=orange shading2=transparent layer2a=skyellipse ellipse2a=filled_rectangle opaque2a=6 layer2b=skyellipse ellipse2b=crosshair_rectangle opaque2b=2 layer3=skycorr lonerr3=ra_error laterr3=dec_error corr3=ra_dec_corr ellipse3=crosshair_ellipse
corrN = <num-expr>
(String)
ra_dec_corr
value
supplied in the Gaia source catalogue.
The value is a numeric algebraic expression based on column names as described in Section 10.
ellipseN = ellipse|crosshair_ellipse|...
(MultiPointShape)
The available options are:
ellipse
crosshair_ellipse
filled_ellipse
rectangle
crosshair_rectangle
filled_rectangle
open_triangle
filled_triangle
lines
capped_lines
arrows
[Default: ellipse
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
latN = <deg-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
laterrN = <num-expr>
(String)
unit
option.
The value is a numeric algebraic expression based on column names as described in Section 10.
lonN = <deg-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
lonerrN = <num-expr>
(String)
unit
option.
The value is a numeric algebraic expression based on column names as described in Section 10.
scaleN = <number>
(Double)
The main purpose of this option is to tweak
the visible sizes of the plotted markers for better visibility.
The unit
option
should be used to account for the units in which the
angular extent coordinates are supplied.
If the markers are supposed to be plotted with their
absolute angular extents visible, this option should be set
to its default value of 1.
[Default: 1
]
shadingN = auto|flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: auto
]
thickN = <int-value>
(Integer)
[Default: 0
]
unitN = scaled|radian|degree|minute|arcsec|mas|uas
(AngleUnit)
scaled
is given
then a non-physical scaling is applied to the
input values to make the the largest markers
appear at a reasonable size (a few tens of pixels)
in the plot.
Note that the actual plotted size of the markers
can also be scaled using the
scale
option;
these two work together to determine the actual plotted sizes.
The available options are:
scaled
: a non-physical scaling is applied
based on the size of values present
radian
: radiansdegree
: degreesminute
: arcminutesarcsec
: arcsecondsmas
: milli-arcsecondsuas
: micro-arcseconds[Default: degree
]
skydensity
Plots a density map on the sky. The grid on which the values are drawn uses the HEALPix tesselation, with a configurable resolution. You can optionally use a weighting for the points, and you can configure how the points are combined to produce the output pixel values.
The way that data values are mapped to colours is usually controlled by options at the level of the plot itself, rather than by per-layer configuration.
Usage Overview:
layerN=skydensity levelN=<-rel-level|+abs-level> combineN=sum|sum-per-unit|count|... perunitN=steradian|degree2|arcmin2|arcsec2|mas2|uas2 transparencyN=0..1 lonN=<deg-expr> latN=<deg-expr> weightN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2sky in=tgas_source.fits lon=l lat=b layer1=skydensity weight1=parallax combine1=mean level1=4 projection=aitoff auxmap=PuRd auxfunc=histogram xpix=540 ypix=250
combineN = sum|sum-per-unit|count|...
(Combiner)
weight
is left blank, a weighting of unity is assumed.
For density-like values
(count-per-unit
,
sum-per-unit
)
the scaling is additionally influenced by the
perunit
parameter.
The available options are:
sum
: the sum of all the combined values per binsum-per-unit
: the sum of all the combined values per unit of bin sizecount
: the number of non-blank values per bin (weight is ignored)count-per-unit
: the number of non-blank values per unit of bin size (weight is ignored)mean
: the mean of the combined valuesmedian
: the medianq1
: first quartileq3
: third quartilemin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuesstdev
: the sample standard deviation of the combined valuesstdev_pop
: the population standard deviation of the combined valueshit
: 1 if any values present, NaN otherwise (weight is ignored)[Default: sum-per-unit
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
latN = <deg-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
levelN = <-rel-level|+abs-level>
(Integer)
If the supplied value is a non-negative integer, it gives the absolute level to use; at level 0 there are 12 pixels on the sky, and the count multiplies by 4 for each increment.
If the value is negative, it represents a relative level; it is approximately the (negative) number of screen pixels along one side of a HEALPix sky pixel. In this case the actual HEALPix level will depend on the current zoom.
[Default: -3
]
lonN = <deg-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
perunitN = steradian|degree2|arcmin2|arcsec2|mas2|uas2
(SolidAngleUnit)
combine
=count-per-unit
or
sum-per-unit
).
If the Combination mode is calculating values per unit area,
this configures the area scale in question.
For non-density-like combination modes
(e.g. combine
=sum
or
mean
)
it has no effect.
The available options are:
steradian
: (180/Pi)^2 deg^2degree2
: square degreesarcmin2
: square arcminute, (1/60)^2 deg^2arcsec2
: square arcsecond, (1/3600)^2 deg^2mas2
: square milliarcsecond, (.001/3600)^2 deg^2uas2
: square microarcsecond, (1e-6/3600)^2 deg^2[Default: degree2
]
transparencyN = 0..1
(Double)
[Default: 0
]
weightN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
healpix
Plots a table representing HEALPix pixels on the sky. Each row represents a single HEALPix tile, and a value from that row is used to colour the corresponding region of the sky plot.
The way that data values are mapped to colours is usually controlled by options at the level of the plot itself, rather than by per-layer configuration.
Usage Overview:
layerN=healpix datalevelN=<int-value> datasysN=<value> degradeN=<int-value> combineN=sum|sum-per-unit|count|... perunitN=steradian|degree2|arcmin2|arcsec2|mas2|uas2 transparencyN=0..1 healpixN=<num-expr> valueN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2sky layer1=healpix in1=simbad-hpx8.fits healpix1=HPX8 value1=NBREF datalevel1=8 degrade1=2 combine=sum-per-unit perunit=arcmin2 projection=aitoff datasys1=equatorial viewsys=galactic labelpos=none gridcolor=beige auxfunc=log auxmap=cold auxflip=true auxclip=0,1 xpix=600 ypix=280
combineN = sum|sum-per-unit|count|...
(Combiner)
degrade
>0.
For density-like values
(count-per-unit
,
sum-per-unit
)
the scaling is additionally influenced by the
perunit
parameter.
The available options are:
sum
: the sum of all the combined values per binsum-per-unit
: the sum of all the combined values per unit of bin sizecount
: the number of non-blank values per bin (weight is ignored)count-per-unit
: the number of non-blank values per unit of bin size (weight is ignored)mean
: the mean of the combined valuesmedian
: the medianq1
: first quartileq3
: third quartilemin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuesstdev
: the sample standard deviation of the combined valuesstdev_pop
: the population standard deviation of the combined valueshit
: 1 if any values present, NaN otherwise (weight is ignored)[Default: mean
]
datalevelN = <int-value>
(Integer)
[Default: -1
]
datasysN = <value>
(SkySys)
[Default: equatorial
]
degradeN = <int-value>
(Integer)
degrade
input tiles.
The way that values are combined within each painted tile
is controlled by the
combine
value.
[Default: 0
]
healpixN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
perunitN = steradian|degree2|arcmin2|arcsec2|mas2|uas2
(SolidAngleUnit)
combine
=count-per-unit
or
sum-per-unit
).
If the Combination mode is calculating values per unit area,
this configures the area scale in question.
For non-density-like combination modes
(e.g. combine
=sum
or
mean
)
it has no effect.
The available options are:
steradian
: (180/Pi)^2 deg^2degree2
: square degreesarcmin2
: square arcminute, (1/60)^2 deg^2arcsec2
: square arcsecond, (1/3600)^2 deg^2mas2
: square milliarcsecond, (.001/3600)^2 deg^2uas2
: square microarcsecond, (1e-6/3600)^2 deg^2[Default: degree2
]
transparencyN = 0..1
(Double)
[Default: 0
]
valueN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
skygrid
Plots an additional axis grid on the celestial sphere.
This can be overlaid on the default sky axis grid
so that axes for multiple sky coordinate systems
are simultaneously visible.
The plots are done relative to the View sky system
(viewsys
parameter)
defined for the plot as a whole.
Note that some of the configuration items for this plotter, such as grid line antialiasing and the decimal/sexagesimal flag, are inherited from the values set for the main sky plot grid.
Usage Overview:
layerN=skygrid gridsysN=equatorial|galactic|supergalactic|ecliptic gridcolorN=<rrggbb>|red|blue|... transparencyN=0..1 labelposN=Internal|None loncrowdN=<number> latcrowdN=<number>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2sky xpix=500 ypix=250 projection=aitoff viewsys=equatorial labelpos=none sex=false layer1=skygrid gridsys1=ecliptic gridcolor1=HotPink transparency1=0.7 labelpos1=internal
gridcolorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: grey
]
gridsysN = equatorial|galactic|supergalactic|ecliptic
(SkySys)
viewsys
parameter
defined for the plot as a whole
to determine what grid lines to plot.
The available options are:
equatorial
: J2000 equatorial systemgalactic
: IAU 1958 galactic systemsupergalactic
: De Vaucouleurs supergalactic systemecliptic
: ecliptic system based on conversion at 2000.0[Default: equatorial
]
labelposN = Internal|None
(SkyAxisLabeller)
The available options are:
Internal
: Labels are drawn inside the plot boundsNone
: Axes are not labelled[Default: Internal
]
latcrowdN = <number>
(Double)
[Default: 1
]
loncrowdN = <number>
(Double)
[Default: 1
]
transparencyN = 0..1
(Double)
[Default: 0
]
xyzvector
Plots directed lines from the data position given delta values for the coordinates. The plotted markers are typically little arrows, but there are other options.
In some cases the supplied data values
give the actual extents in data coordinates
for the plotted vectors
but sometimes the data is on a different scale
or in different units to the positional coordinates.
As a convenience for this case, the plotter can optionally
scale the magnitudes of all the vectors
to make them a reasonable size on the plot,
so by default the largest ones are a few tens of pixels long.
This auto-scaling is turned off by default,
but it can be activated with the
autoscale
option.
Whether autoscaling is on or off, the
scale
option can be used to apply a fixed scaling factor.
Usage Overview:
layerN=xyzvector arrowN=small_arrow|medium_arrow|... thickN=<int-value> scaleN=<factor> autoscaleN=true|false shadingN=flat|translucent|transparent|density|aux|weighted <shade-paramsN> xN=<num-expr> yN=<num-expr> zN=<num-expr> xdeltaN=<num-expr> ydeltaN=<num-expr> zdeltaN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2cube in=gavo_g2.fits x=x y=y z=z xdelta=velX ydelta=velY zdelta=velZ autoscale=true color=BlueViolet scale=1.5 layer1=xyzvector shading1=transparent opaque1=5 arrow1=medium_filled_dart layer2=xyzvector shading2=flat arrow2=medium_open_dart xmin=6 xmax=7.5 ymin=12.5 ymax=13.5 zmin=19 zmax=21.5
arrowN = small_arrow|medium_arrow|...
(MultiPointShape)
The available options are:
small_arrow
medium_arrow
large_arrow
small_open_dart
medium_open_dart
large_open_dart
small_filled_dart
medium_filled_dart
large_filled_dart
lines
capped_lines
[Default: small_arrow
]
autoscaleN = true|false
(Boolean)
If auto-scaling is on, then markers will keep approximately the same screen size during zoom operations; if it's off, they will keep the same size in data coordinates.
Marker size is also affected by the
scale
parameter.
[Default: false
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
scaleN = <factor>
(Double)
[Default: 1
]
shadingN = flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: flat
]
thickN = <int-value>
(Integer)
[Default: 0
]
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xdeltaN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
ydeltaN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
zN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
zdeltaN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xyzerror
Plots symmetric or asymmetric error bars in some or all of the plot dimensions. The shape of the error "bars" is quite configurable, including (for 2-d and 3-d errors) ellipses, rectangles etc aligned with the axes.
Usage Overview:
layerN=xyzerror errorbarN=none|lines|capped_lines|... thickN=<int-value> shadingN=flat|translucent|transparent|density|aux|weighted <shade-paramsN> xN=<num-expr> yN=<num-expr> zN=<num-expr> xerrhiN=<num-expr> xerrloN=<num-expr> yerrhiN=<num-expr> yerrloN=<num-expr> zerrhiN=<num-expr> zerrloN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2cube in=dr5qso.fits icmd='select morphology==1' x=psfmag_g xerrhi=psfmagerr_g y=psfmag_r yerrhi=psfmagerr_r z=psfmag_u zerrhi=psfmagerr_u layer1=mark layer2=xyzerror errorbar2=cuboid shading=transparent opaque=3 xmin=17.5 xmax=18 ymin=17.3 ymax=17.7 zmin=17.4 zmax=18.2
errorbarN = none|lines|capped_lines|...
(MultiPointShape)
The available options are:
none
lines
capped_lines
caps
arrows
cuboid
ellipse
crosshair_ellipse
rectangle
crosshair_rectangle
filled_ellipse
filled_rectangle
[Default: lines
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = flat|translucent|transparent|density|aux|weighted <shade-paramsN>
(ShapeMode)
[Default: flat
]
thickN = <int-value>
(Integer)
[Default: 0
]
xN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xerrhiN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
xerrloN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yerrhiN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yerrloN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
zN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
zerrhiN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
zerrloN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
line3d
Plots a point-to-point line joining
up the positions of data points in three dimensions.
There are additional options to pre-sort the points
by a given quantity before drawing the lines
(using the sort
value),
and to vary the colour of the line along its length
(using the aux
value).
The options for controlling the Aux colour map
are controlled at the level of the plot itself,
rather than by per-layer configuration.
Note that the line positioning in 3d and the line segment aux colouring is somewhat approximate. In most cases it is good enough for visual inspection, but pixel-level examination may reveal discrepancies.
Usage Overview:
layerN=line3d colorN=<rrggbb>|red|blue|... thickN=<pixels> <pos-coord-paramsN> auxN=<num-expr> sortN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
<pos-coord-paramsN>
give a position for each row of the input table.
Their form depends on the plot geometry,
i.e. which plotting command is used.
For a plane plot (plot2plane
)
the parameters would be
xN
and yN
.
The coordinate parameter values are in all cases strings
interpreted as numeric expressions based on column names.
These can be column names, fixed values or algebraic
expressions as described in Section 10.
Example:
stilts plot2cube in=iers.fits x=x y=y z=LOD layer1=line3d icmd1='select decYear>1963&&decYear<1964.5' thick1=3 aux1=LOD layer2=mark shading2=translucent color2=cccc00 translevel2=0.35 auxmap=cyan-magenta auxvisible=false legend=false phi=-150 theta=25 psi=180 xpix=400 ypix=400
auxN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
sortN = <num-expr>
(String)
Note that if the required order is in fact the natural order of the table, it is better to leave this value blank, since sorting is a potentially expensive step.
The value is a numeric algebraic expression based on column names as described in Section 10.
thickN = <pixels>
(Integer)
[Default: 1
]
spheregrid
Plots a spherical grid around the origin of a 3-d plot. The radius of the sphere can be configured explicitly, otherwise a suitable default value (that should make at least some of the grid visible) will be chosen.
Usage Overview:
layerN=spheregrid radiusN=<number> gridcolorN=<rrggbb>|red|blue|... thickN=<pixels> nlonN=<int-value> nlatN=<int-value>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2sphere legend=false xpix=350 ypix=350 layer1=mark in1=tgas_source.fits lon1=ra lat1=dec r1=1 shading1=transparent opaque1=850 color1=orange layer2=spheregrid gridcolor2=green thick2=2
gridcolorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: grey
]
nlatN = <int-value>
(Integer)
[Default: 2
]
nlonN = <int-value>
(Integer)
[Default: 3
]
radiusN = <number>
(Double)
[Default: -1
]
thickN = <pixels>
(Integer)
[Default: 1
]
yerror
Plots symmetric or asymmetric error bars in the Y direction.
Usage Overview:
layerN=yerror errorbarN=none|lines|capped_lines|caps|arrows thickN=<int-value> shadingN=flat <shade-paramsN> tN=<time-expr> ttypeN=DecYear|MJD|JD|Unix|Iso8601 yN=<num-expr> yerrhiN=<num-expr> yerrloN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2time in=ACE_data.vot t=epoch y=Bmag layer1=yerror yerrhi1=sigma_B errorbar1=capped_lines layer2=mark shape2=open_circle size2=3 layer3=line color3=a0a0a0 tmin=2001-08-17T07 tmax=2001-08-17T10 ypix=250
errorbarN = none|lines|capped_lines|caps|arrows
(MultiPointShape)
The available options are:
none
lines
capped_lines
caps
arrows
[Default: lines
]
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
shadingN = flat <shade-paramsN>
(ShapeMode)
[Default: flat
]
tN = <time-expr>
(String)
The value is a Time value algebraic expression based on column names as described in Section 10.
thickN = <int-value>
(Integer)
[Default: 0
]
ttypeN = DecYear|MJD|JD|Unix|Iso8601
(TimeMapper)
tN
is supplied.
Options are:
DecYear
: Years since 0 ADMJD
: Modified Julian DateJD
: Julian DayUnix
: Seconds since midnight 1 Jan 1970Iso8601
: ISO 8601 stringtN
value.
yN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yerrhiN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
yerrloN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
spectrogram
Plots spectrograms. A spectrogram is a sequence of spectra plotted as vertical 1-d images, each one plotted at a different horizontal coordinate.
This specialised layer is only available for
time
plots.
The way that data values are mapped to colours is usually controlled by options at the level of the plot itself, rather than by per-layer configuration.
Usage Overview:
layerN=spectrogram scalespecN=true|false tN=<time-expr> ttypeN=DecYear|MJD|JD|Unix|Iso8601 spectrumN=<array-expr> twidthN=<num-expr> inN=<table> ifmtN=<in-format> istreamN=true|false icmdN=<cmds>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2time layer1=spectrogram in1=LRS_NPW_V010_20071101.cdf t1=epoch spectrum1=RX2 t2func=mjd t2label=MJD auxfunc=linear auxmap=plasma auxclip=0,1 xpix=600 ypix=320 tmin=2007-11-01T00 tmax=2007-11-01T12 ylog=true ylabel=Frequency/Hz ymin=8e4 ymax=2e7
icmdN = <cmds>
(ProcessingStep[])
inN
.
The value of this parameter is one or more of the filter
commands described in Section 6.1.
If more than one is given, they must be separated by
semicolon characters (";").
This parameter can be repeated multiple times on the same
command line to build up a list of processing steps.
The sequence of commands given in this way
defines the processing pipeline which is performed on the table.
Commands may alternatively be supplied in an external file,
by using the indirection character '@
'.
Thus a value of "@filename
"
causes the file filename
to be read for a list
of filter commands to execute. The commands in the file
may be separated by newline characters and/or semicolons,
and lines which are blank or which start with a
'#
' character are ignored.
A backslash character '\
' at the end of a line
joins it with the following line.
ifmtN = <in-format>
(String)
inN
.
The known formats are listed in Section 5.1.1.
This flag can be used if you know what format your
table is in.
If it has the special value
(auto)
(the default),
then an attempt will be
made to detect the format of the table automatically.
This cannot always be done correctly however, in which case
the program will exit with an error explaining which
formats were attempted.
This parameter is ignored for scheme-specified tables.
[Default: (auto)
]
inN = <table>
(StarTable)
-
",
meaning standard input.
In this case the input format must be given explicitly
using the ifmtN
parameter.
Note that not all formats can be streamed in this way.:<scheme-name>:<scheme-args>
.<
" character at the start,
or a "|
" character at the end
("<syscmd
" or
"syscmd|
").
This executes the given pipeline and reads from its
standard output.
This will probably only work on unix-like systems.istreamN = true|false
(Boolean)
inN
parameter
will be read as a stream.
It is necessary to give the
ifmtN
parameter
in this case.
Depending on the required operations and processing mode,
this may cause the read to fail (sometimes it is necessary
to read the table more than once).
It is not normally necessary to set this flag;
in most cases the data will be streamed automatically
if that is the best thing to do.
However it can sometimes result in less resource usage when
processing large files in certain formats (such as VOTable).
This parameter is ignored for scheme-specified tables.
[Default: false
]
scalespecN = true|false
(Boolean)
If this flag is set false, or if no suitable array can be found, the vertical axis just represents channel indices and so is labelled from 0 to the number of channels per spectrum.
This configuration item is somewhat experimental; the details of how the spectral axis is configured may change in future releases.
[Default: true
]
spectrumN = <array-expr>
(String)
The value is an array-valued algebraic expression based on column names as described in Section 10. Some of the functions in the Arrays class may be useful here.
tN = <time-expr>
(String)
The value is a Time value algebraic expression based on column names as described in Section 10.
ttypeN = DecYear|MJD|JD|Unix|Iso8601
(TimeMapper)
tN
is supplied.
Options are:
DecYear
: Years since 0 ADMJD
: Modified Julian DateJD
: Julian DayUnix
: Seconds since midnight 1 Jan 1970Iso8601
: ISO 8601 stringtN
value.
twidthN = <num-expr>
(String)
The value is a numeric algebraic expression based on column names as described in Section 10.
Some plot layer types
have an associated shading
parameter
which determines how plotted markers are coloured.
This is independent of the marker shapes
(which may be points, vectors, ellipses, ...)
but may be affected by how many markers are plotted on top of each other,
additional input table values, selected colour maps etc.
For the simplest shading types
(e.g. flat
)
it's just a case of choosing a colour,
but the more complex ones have several associated parameters.
The various shading types and their usages are described in the following subsections.
auto
Paints isolated points in their selected colour but where multiple points in the same layer overlap it adjusts the clour by darkening it. This means that for isolated points (most or all points in a non-crowded plot, or outliers in a crowded plot) it behaves just like flat mode, but it's easy to see where overdense regions lie.
This is like density mode, but with no user-configurable options.
Usage:
shadingN=auto colorN=<rrggbb>|red|blue|...
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=mark in1=dr5qso.fits shading1=auto x1=psfmag_g-psfmag_r y1=psfmag_u-psfmag_g size1=2 xmin=-0.5 xmax=2.5 ymin=-1 ymax=6
Associated parameters are as follows:
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
flat
Paints markers in a single fixed colour.
Usage:
shadingN=flat colorN=<rrggbb>|red|blue|...
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=mark in1=dr5qso.fits shading1=flat x1=psfmag_g-psfmag_r y1=psfmag_u-psfmag_g size1=2 xmin=-0.5 xmax=2.5 ymin=-1 ymax=6
Associated parameters are as follows:
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
translucent
Paints markers in a transparent version of their selected colour. The degree of transparency is determined by how many points are plotted on top of each other and by the transparency level. Unlike transparent mode, the transparency varies according to the average point density in the plot, so leaving the setting the same as you zoom in and out usually has a sensible effect.
Usage:
shadingN=translucent colorN=<rrggbb>|red|blue|... translevelN=<number>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=mark in1=dr5qso.fits shading1=translucent x1=psfmag_g-psfmag_r y1=psfmag_u-psfmag_g size1=2 xmin=-0.5 xmax=2.5 ymin=-1 ymax=6
Associated parameters are as follows:
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
translevelN = <number>
(Double)
[Default: 0.1
]
transparent
Paints markers in a transparent version of their selected colour. The degree of transparency is determined by how many points are plotted on top of each other and by the opaque limit. The opaque limit fixes how many points must be plotted on top of each other to completely obscure the background. This is set to a fixed value, so a transparent level that works well for a crowded region (or low magnification) may not work so well for a sparse region (or when zoomed in).
Usage:
shadingN=transparent colorN=<rrggbb>|red|blue|... opaqueN=<number>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=mark in1=dr5qso.fits shading1=transparent x1=psfmag_g-psfmag_r y1=psfmag_u-psfmag_g size1=2 xmin=-0.5 xmax=2.5 ymin=-1 ymax=6
Associated parameters are as follows:
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
opaqueN = <number>
(Double)
[Default: 4
]
density
Paints markers using a configurable colour map to indicate how many points are plotted over each other. Specifically, it colours each pixel according to how many times that pixel has has been covered by one of the markers plotted by the layer in question. To put it another way, it generates a false-colour density map with pixel granularity using a smoothing kernel of the form of the markers plotted by the layer. The upshot is that you can see the plot density of points or other markers plotted.
This is like auto mode, but with more user-configurable options.
Usage:
shadingN=density colorN=<rrggbb>|red|blue|... densemapN=<map-name>|<color>-<color>[-<color>...] denseclipN=<lo>,<hi> denseflipN=true|false densequantN=<number> densesubN=<lo>,<hi> densefuncN=log|linear|histogram|histolog|sqrt|square|acos|cos
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=mark in1=dr5qso.fits shading1=density densemap1=viridis x1=psfmag_g-psfmag_r y1=psfmag_u-psfmag_g size1=2 xmin=-0.5 xmax=2.5 ymin=-1 ymax=6
Associated parameters are as follows:
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
denseclipN = <lo>,<hi>
(Subrange)
If the full range 0,1
is used,
the whole range of colours specified by the selected
shader will be used.
But if for instance a value of 0,0.5
is given,
only those colours at the left hand end of the ramp
will be seen.
If the null (default) value is chosen,
a default clip will be used.
This generally covers most or all of the range 0-1
but for colour maps which fade to white,
a small proportion of the lower end may be excluded,
to ensure that all the colours are visually distinguishable
from a white background.
This default is usually a good idea if the colour map
is being used with something like a scatter plot,
where markers are plotted against a white background.
However, for something like a density map when the whole
plotting area is tiled with colours from the map,
it may be better to supply the whole range
0,1
explicitly.
denseflipN = true|false
(Boolean)
[Default: false
]
densefuncN = log|linear|histogram|histolog|sqrt|square|acos|cos
(Scaling)
The available options are:
log
: Logarithmic scalinglinear
: Linear scalinghistogram
: Scaling follows data distribution, with linear axishistolog
: Scaling follows data distribution, with logarithmic axissqrt
: Square root scalingsquare
: Square scalingacos
: Arccos Scalingcos
: Cos ScalingFor all these options,
the full range of data values is used,
and displayed on the colour bar
if applicable (though it can be restricted using the densesub
option)
The Linear
,
Log
,
Square
and
Sqrt
options
just apply the named function to the full data range.
The histogram options on the other hand use a scaling function
that corresponds to the actual distribution of the data, so that
there are about the same number of points (or pixels, or whatever
is being scaled) of each colour.
The histogram options are somewhat more expensive,
but can be a good choice if you are exploring data whose
distribution is unknown or not well-behaved over
its min-max range.
The Histogram
and
HistoLog
options both
assign the colours in the same way, but they display the colour
ramp with linear or logarithmic annotation respectively;
the HistoLog
option also ignores
non-positive values.
[Default: log
]
densemapN = <map-name>|<color>-<color>[-<color>...]
(Shader)
A mixed bag of colour ramps are available
as listed in Section 8.7:
blacker
,
whiter
,
inferno
,
magma
,
plasma
,
viridis
,
cividis
,
cubehelix
,
sron
,
rainbow
,
rainbow2
,
rainbow3
,
pastel
,
cosmic
,
ember
,
gothic
,
rainforest
,
voltage
,
bubblegum
,
gem
,
chroma
,
sunset
,
neon
,
tropical
,
accent
,
gnuplot
,
gnuplot2
,
specxby
,
set1
,
paired
,
hotcold
,
guppy
,
iceburn
,
redshift
,
pride
,
rdbu
,
piyg
,
brbg
,
cyan-magenta
,
red-blue
,
brg
,
heat
,
cold
,
light
,
greyscale
,
colour
,
standard
,
bugn
,
bupu
,
orrd
,
pubu
,
purd
,
painbow
,
huecl
,
infinity
,
hue
,
intensity
,
rgb_red
,
rgb_green
,
rgb_blue
,
hsv_h
,
hsv_s
,
hsv_v
,
yuv_y
,
yuv_u
,
yuv_v
,
scale_hsv_s
,
scale_hsv_v
,
scale_yuv_y
.
Note:
many of these, including rainbow-like ones,
are frowned upon by the visualisation community.
You can also construct your own custom colour map
by giving a sequence of colour names separated by
minus sign ("-
") characters.
In this case the ramp is a linear interpolation
between each pair of colours named,
using the same syntax as when specifying
a colour value.
So for instance
"yellow-hotpink-#0000ff
"
would shade from yellow via hot pink to blue.
[Default: blacker
]
densequantN = <number>
(Double)
If left blank, the colour map is nominally continuous (though in practice it may be quantised to a medium-sized number like 256).
densesubN = <lo>,<hi>
(Subrange)
The default value "0,1
" therefore has
no effect.
The range could be restricted to its lower half
with the value 0,0.5
.
[Default: 0,1
]
aux
Paints markers in a colour determined by the value of an additional data coordinate. The marker colours then represent an additional dimension of the plot. You can also adjust the transparency of the colours used. The way that data values are mapped to colours is usually controlled by options at the level of the plot itself, rather than by per-layer configuration.
Usage:
shadingN=aux auxN=<num-expr> auxnullcolorN=<rrggbb>|red|blue|... opaqueN=<number>
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=mark in1=dr5qso.fits shading1=aux aux1=z auxmap=plasma x1=psfmag_g-psfmag_r y1=psfmag_u-psfmag_g size1=2 xmin=-0.5 xmax=2.5 ymin=-1 ymax=6
Associated parameters are as follows:
auxN = <num-expr>
(String)
This parameter gives a column name, fixed value, or algebraic expression for the
aux
coordinate
for layer N
.
The value is a numeric algebraic expression based on column names
as described in Section 10.
auxnullcolorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
If the value is null, then points with a null Aux value will not be plotted at all.
[Default: grey
]
opaqueN = <number>
(Double)
[Default: 1
]
weighted
Paints markers like the Density mode, but with optional weighting by an additional coordinate. You can configure how the weighted coordinates are combined to give the final weighted result. The way that data values are mapped to colours is usually controlled by options at the level of the plot itself, rather than by per-layer configuration.
Usage:
shadingN=weighted weightN=<num-expr> colorN=<rrggbb>|red|blue|... combineN=sum|count|mean|median|...
All the parameters listed here
affect only the relevant layer,
identified by the suffix
N
.
Example:
stilts plot2plane layer1=mark in1=dr5qso.fits shading1=weighted weight1=z auxmap=plasma x1=psfmag_g-psfmag_r y1=psfmag_u-psfmag_g size1=2 xmin=-0.5 xmax=2.5 ymin=-1 ymax=6
Associated parameters are as follows:
colorN = <rrggbb>|red|blue|...
(Color)
The standard plotting colour names are
red
, blue
, green
, grey
, magenta
, cyan
, orange
, pink
, yellow
, black
, light_grey
, white
.
However, many other common colour names (too many to list here)
are also understood.
The list currently contains those colour names understood
by most web browsers,
from AliceBlue
to YellowGreen
,
listed e.g. in the
Extended color keywords section of
the CSS3 standard.
Alternatively, a six-digit hexadecimal number RRGGBB
may be supplied,
optionally prefixed by "#
" or "0x
",
giving red, green and blue intensities,
e.g. "ff00ff
", "#ff00ff
"
or "0xff00ff
" for magenta.
[Default: red
]
combineN = sum|count|mean|median|...
(Combiner)
When a weight is in use,
mean
or
sum
are typically sensible choices.
If there is no weight (a pure density map)
then count
is usually better,
but in that case it may make more sense
(it is more efficient)
to use one of the other shading modes instead.
The available options are:
sum
: the sum of all the combined values per bincount
: the number of non-blank values per bin (weight is ignored)mean
: the mean of the combined valuesmedian
: the medianq1
: first quartileq3
: third quartilemin
: the minimum of all the combined valuesmax
: the maximum of all the combined valuesstdev
: the sample standard deviation of the combined valuesstdev_pop
: the population standard deviation of the combined valueshit
: 1 if any values present, NaN otherwise (weight is ignored)[Default: mean
]
weightN = <num-expr>
(String)
This parameter gives a column name, fixed value, or algebraic expression for the
weight
coordinate
for layer N
.
The value is a numeric algebraic expression based on column names
as described in Section 10.
The plots generated by the plotting commands can be used in various
different ways. One thing you might want to do is to write the
output to a file in a given graphics format
(out
);
another is to preview it directly on the screen
(swing
).
By default one or other of these will happen depending on whether you
specify an output file. However there are other possibilities;
these are listed in the following subsections.
Except for display to the screen, these modes should work happily
on a headless machine (one with no graphics display, as may be the case
for a web server).
When running headless, you may find it necessary to
set the java system property
"java.awt.headless
" to true
- see Section 3.3.
The default output mode is auto
,
which means that output is to a file if an output file is specified,
or to the screen if it is not. So in most cases you don't need to specify the
omode
parameter explicitly.
swing
Usage:
omode=swing
Plot will be displayed in a window on the screen. This plot is "live"; it can be resized and (except for old-style plots) navigated around with mouse actions in the same way as plots in TOPCAT.
out
Usage:
omode=out out=<out-file> ofmt=png|png-transp|gif|jpeg|pdf|svg|eps|eps-gzip
Plot will be written to a file given by out
using the graphics format given by ofmt
.
cgi
Usage:
omode=cgi ofmt=png|png-transp|gif|jpeg|pdf|svg|eps|eps-gzip
Plot will be written in a way suitable for CGI use direct from a web server.
The output is in the graphics format given by ofmt
,
preceded by a suitable "Content-type" declaration.
discard
Usage:
omode=discard
Plot is drawn, but discarded. There is no output.
auto
Usage:
omode=auto [out=<out-file>]
Behaves as swing
or out
mode depending on presence of out
parameter
Several of the plot output modes write the plot in some graphics format or other. When selecting an output format it is important to understand the distinction between bitmapped and vector formats; basically bitmapped formats represent the image as a grid of finite-sized pixels while vector formats notionally draw smooth lines. Bitmapped formats are fine for a computer screen, but for high quality paper printouts you will want a vector format. You can convert from vector to bitmapped but not (usefully) in the other direction. There are a couple of subtleties to this distinction specific to STILTS graphical output as discussed below.
The following formats are the available values for the
ofmt
parameter of the various plot commands:
png
gif
jpeg
pdf
eps
output (see below).
svg
eps
eps-gzip
or pdf
may be
a better choice.
eps-gzip
eps
format above except that the output
is automatically compressed using the GZIP format as it is written.
Postscript compresses well (typically a factor of 5-10).
A number of colour maps are available, and used for instance
with the shading modes
density
,
aux
and
weighted
.
Not all colour maps are suitable/available in all contexts,
and in some cases the maps are by default clipped at one end to avoid
for instance white-on-white plotting,
but the lists below give an overview of which named colourmaps can be used.
Note that custom colourmaps can also be used by supplying a specification
like "red-yellow-blue
".
The absolute colour maps are listed below: these do not depend on the underlying colour of the plotted symbols, so are suitable when only one dataset is being plotted.
Absolute colour maps
The non-absolute colour maps are listed below: these modify an underlying colour, so are suitable for applying to several different datasets with different underlying colours. The representation here shows how they affect several different colours; for each row of pixels the unmodified (value=0) colour is at the left of the image and the most modified (value=1) is at the right.
Non-absolute colour maps
These colour maps have been derived from several sources, including SkyCat/GAIA, MatPlotLib 1.5, Gnuplot, Daniel Michalik, Paul Tol, CMasher, Color Brewer, HCL Wizard, Dave Green, xkcd, and maybe some others I forgot.
This section describes deprecated commands. For recommended plotting commands, see Section 8.
From version 2.0 (October 2008), STILTS incorporated three table plotting commands:
These provided command-line access to some, though not all, of the plotting capabilities offered by TOPCAT.Since version 3.0 (October 2014), these commands are deprecated in favour of the more powerful ones described in Section 8. The rest of this section describes some aspects of the deprecated commands for the benefit of legacy code. The output modes and formats are the same in old- and new-style plots, and are discussed in Section 8.5 and Section 8.6. The handling of parameters and suffixes for these commands is not quite the same as for new-style plots, and is documented in the next subsection.
As a simple example, if a file "cat.fits" contains the columns RMAG and BMAG for red and blue magnitudes, you can draw a two-dimensional colour-magnitude scatter plot with the command:
stilts plot2d in=cat.fits xdata=BMAG-RMAG ydata=BMAGSince an output file is not specified, the plot is shown on the screen for convenience. To send the output to a PNG file, do instead:
stilts plot2d in=cat.fits xdata=BMAG-RMAG ydata=BMAG out=plot.png ofmt=pngin some cases (including the above), the
ofmt
parameter is not
required since STILTS may be able to guess the format from the output
file name.
Various other options for output and graphics formats are described in
Section 8.5 and Section 8.6
Some of the parameters use suffixes to define data sets and therefore behave a bit differently from the parameters elsewhere in STILTS - a discussion of these is given in the following subsection. Some other plotting-specific topics are also discussed below.
This section describes deprecated commands. For recommended plotting commands, see Section 8.
Some of the parameters for the plotting tasks behave a little bit
differently to other parameters in STILTS, in order to accommodate
related sets of values. If you look at the usage of one of the
plotting commands, for instance in Appendix B.16.1,
you will see that a number of the parameters have the
suffixes "N
" or "NS
".
These suffixes can be substituted with any convenient string to identify
parameters which relate to the same input datasets or subsets.
Specifically:
N
":
inN
parameter
must be given to identify the source of the data; any other parameters
with the same value of the N
suffix relate to that dataset.
A dataset here refers to a particular set of plot data from a
table; in most cases each input table corresponds to a different dataset,
though two datasets may correspond to different sets of columns from
the same table.
NS
":
N
.
At least the subsetNS
parameter must be given to
identify the expression by which the subset is defined;
any other parameters with the same value of the NS
suffix
relate to that subset.
Some examples will help to illustrate. The following will generate a Cartesian plot of catalogue position from a single dataset:
stilts plot2d in=gals.fits xdata=RA ydata=DECIn this case the
N
suffix is present on each of the parameters
in
, xdata
and ydata
, but is
equal to the empty string, hence invisible. This is perfectly legal,
and convenient when only a single table is in use.
If we wish to overplot two datasets however, the dataset suffixes
(or one of them at least) have to be made explicit so that different ones
can be used, for instance:
stilts plot2d in1=gals.fits xdata1=RA ydata1=DEC in2=stars.fits xdata2=RAJ2000 ydata2=DEJ2000The suffix values "
1
" and "2
" are quite arbitrary
and can be chosen as convenient, so the following would do exactly the
same as the previous example:
stilts plot2d in_GAL=gals.fits xdata_GAL=RA ydata_GAL=DEC in_STAR=stars.fits xdata_STAR=RAJ2000 ydata_STAR=DEJ2000The other parameters which have the
N
suffix apply only
to the matching dataset, so for instance the following:
stilts plot2d in1=gals.fits xdata1=RA ydata1=DEC txtlabel1=NGC_ID in2=stars.fits xdata2=RAJ2000 ydata2=DEJ2000would draw text labels adjacent to the points from only the gals.fits file giving the contents of its NGC_ID column.
The NS
suffix identifies distinct row subsets
within the same or different datasets. A subset is defined by supplying
a boolean inclusion expression (each row is included only if the
expression evaluates true for that row) as the value of a
subsetNS
parameter.
If, as in all the examples we have seen so far,
no subsetNS
parameter is supplied for a given dataset,
then it is treated as a special case, as if a single subset with a name
equal to the empty string (S=""
)
containing all rows has been specified.
So our earlier simple example:
stilts plot2d in=gals.fits xdata=RA ydata=DECis equivalent to
stilts plot2d in=gals.fits xdata=RA ydata=DEC subset=trueIf we wish to split the plotted points into two sets based on their R-B colours, we can write something like:
stilts plot2d in=gals.fits xdata=RA ydata=DEC subsetX='RMAG-BMAG>0' subsetY='RMAG-BMAG<=0'This will generate a plot with two subsets shown using different colours and/or plotting symbols. These colours and symbols are selected automatically. More control over the appearance can be exercised by setting values for some of the other parameters with
NS
suffixes, for instance
stilts plot2d in=gals.fits xdata=RA ydata=DEC subset_A='RMAG-BMAG>0' colour_A=blue subset_B='RMAG-BMAG<=0' colour_B=redAgain, the suffix strings can be chosen to have any value as convenient.
The dataset- and subset-specific parameters must be put together if there are multiple datasets with multiple subsets to plot simultaneously, for instance:
stilts plot2d in_1=gals.fits xdata_1=RA ydata_1=DEC subset_1_A='RMAG-BMAG>0' colour_1_A=blue subset_1_B='RMAG-BMAG<=0' colour_1_B=red in_2=stars.fits xdata_2=RAJ2000 ydata_2=DEJ2000 colour_2=green
Finally, it's not quite true that the suffixes chosen have no effect
on the plot; they may influence the order in which sets are plotted.
Markers drawn for sets plotted earlier may be obscured by the markers
drawn for sets plotted later, so this can affect the appearance of the plot.
If you want to control this, use the sequence
parameter.
For instance, to ensure that star data appears on top of galaxy data
in the plot, do the following:
stilts plot2d in_GAL=gals.fits xdata_GAL=RA ydata_GAL=DEC in_STAR=stars.fits xdata_STAR=RAJ2000 ydata_STAR=DEJ2000 sequence=_GAL,_STAR
More examples can be found in the Examples subsections of the individual plotting command descriptions in Appendix B.
Many of the STILTS commands allow you to use algebraic expressions based on table columns when doing things like making row selections, defining new columns, selecting values to plot or match, and so on. In these cases you are defining an expression which has a value in each row as a function of the values in the existing columns in that row. This is a powerful feature which permits you to manipulate and select table data in very flexible ways. The syntax for entering these expressions is explained in this section.
What you write are actually expressions in the Java language, which are compiled into Java bytecode before evaluation. However, this does not mean that you need to be a Java programmer to write them. The syntax is pretty similar to C, but even if you've never programmed in C most simple things, and many complicated ones, are quite intutitive.
The following explanation gives some guidance and examples for writing these expressions. Unfortunately a complete tutorial on writing Java expressions is beyond the scope of this document, but it should provide enough information for even a novice to write useful expressions.
The expressions that you can write are basically any function of all the column values which apply to a given row; the function result can then be used where STILTS needs a per-row value, for instance to define a new column. If the built-in operators and functions are not sufficient, or it's unwieldy to express your function in one line of code, it is possible to add new functions by writing your own classes - see Section 10.9.3.
Note that since these algebraic expressions often contain spaces, you may need to enclose them in single or double quotes so that they don't get confused with other parts of the command string.
Note: if Java is running in an environment with certain security restrictions (a security manager which does not permit creation of custom class loaders) then algebraic expressions won't work at all. It's not particularly likely that security restrictions will be in place if you are running from the command line though.
To create a useful expression which can be evaluated for each row in a table, you will have to refer to cells in different columns of that row. You can do this in several ways:
As a special case, if an expression contains just a single column name, rather than some more complicated expression, then any column name may be used, even one containing non-alphanumeric characters.
Column names are treated case-insensitively.
ucd$<ucd-spec>
". Depending on the version of
UCD scheme used, UCDs can contain various punctuation marks such
as underscores, semicolons and dots; for the purpose of this syntax
these should all be represented as underscores ("_
").
So to identify a column which has the UCD "phot.mag;em.opt.R
",
you should use the identifier "ucd$phot_mag_em_opt_r
".
Matching is not case-sensitive. Futhermore, a trailing underscore
acts as a wildcard, so that the above column could also be referenced
using the identifier "ucd$phot_mag_
". If multiple
columns have UCDs which match the given identifer, the first one
will be used.
Note that the same syntax can be used for referencing table parameters (see the next section); columns take preference so if a column and a parameter both match the requested UCD, the column value will be used.
utype$<utype-spec>
".
Utypes can contain various punctuation marks such as colons and dots;
for the purpose of this syntax
these should all be represented as underscores ("_
").
So to identify a column which has the Utype
"ssa:Access.Format
",
you should use the identifier
"utype$ssa_Access_Format
".
Matching is not case-sensitive.
If multiple columns have Utypes which match the given identifier,
the first one will be used.
Note that the same syntax can be used for referencing table parameters (see the next section); columns take preference so if a column and a parameter both match the requested Utype, the column value will be used.
valueDouble
, valueInt
, valueLong
,
valueString
and valueObject
to obtain the typed value of a column with a given name.
The argument of the function is a string giving the exact
(case-sensitive) column name, for instance
valueDouble("b_E(BP-RP)")
will yield the value of the column named "b_E(BP-RP)
"
as a double-precision floating point value.
These functions are not the generally recommended way
to get column values,
since they are slower and provide less type-checking than
the other options listed above,
and can occasionally lead to some other esoteric problems.
However, if you need to refer by name to strangely-named columns
they are sometimes a convenient option.
Object$
"
before its identifier
(e.g. "Object$BMAG
" for a column named BMAG
)
the result will be the column value as a java Object.
Without that prefix, numeric columns are evaluated as java primitives.
In most cases, you don't want to do this,
since it means that you can't use the value in arithmetic expressions.
However, if you need the value to be passed to a
(possibly user-defined) method,
and you need that method to be invoked even when the value is null,
you have to do it like this. Null-valued primitives
otherwise cause expression evaluation to abort.
The value of the variables so referenced will be a primitive
(boolean, byte, short, char, int, long, float, double) if the
column contains one of the corresponding types. Otherwise it will
be an Object of the type held by the column, for instance a String.
In practice this means: you can write the name of a column, and it will
evaluate to the numeric (or string) value that that column contains
in each row. You can then use this in normal algebraic expressions
such as "B_MAG-U_MAG
" as you'd expect.
Some tables have constant values associated with them; these may represent such things as the epoch at which observations were taken, the name of the catalogue, an angular resolution associated with all observations, or any number of other things. Such constants are known as table parameters (not to be confused with parameters passed to STILTS commands) and can be thought of as extra columns which have the same value for every row. The values of such parameters can be referenced in STILTS algebraic expressions as follows:
param$
.
ucd$
. Any punctuation marks in the
UCD should be replaced by underscores, and a trailing underscore
is interpreted as a wildcard. See Section 10.1 for
more discussion.
utype$
. Any punctuation marks in the
Utype should be replaced by underscores. See Section 10.1 for
more discussion.
fixcolnames
filter.
There are a few pseudo-variables which have special functions in the expression language. The following specials are column-like, in that they have a different value for each row:
long
(8-byte integer);
when using it in certain expressions you may find it necessary to convert
it to an int
(4-byte integer) using the
toInteger()
function.
The deprecated alias "INDEX
" may also be used.
0<=x<1
.
NOTE: this token is deprecated since it can behave
unpredictably (the same cell does not always yield the same result).
Use instead the random()
function in class
Maths.
The following specials are parameter-like, in that their value is not sensitive to the row:
long
(8-byte integer);
when using it in certain expressions you may find it necessary to convert
it to an int
(4-byte integer) using the
toInteger()
function.
When no special steps are taken, if a null value (blank cell) is encountered in evaluating an expression (usually because one of the columns it relies on has a null value in the row in question) then the result of the expression is also null.
It is possible to exercise more control than this, but it
requires a little bit of care,
because the expressions work in terms of primitive values
(numeric or boolean ones) which don't in general have a defined null
value. The name "null
"
in expressions gives you the java null
reference, but this cannot be matched against a primitive value
or used as the return value of a primitive expression.
For most purposes, the following two tips should enable you to work with null values:
NULL_
"
(use upper case) to the column name or $ID. This
will yield a boolean value which is true if the column contains
a blank or a floating point NaN (not-a-number) value,
and false otherwise.
Note that if combined with other boolean expressions,
this null test should come first, i.e. write
"NULL_i || i==999
" rather than
"i==999 || NULL_i
",
though this is only essential for integer or boolean variables.
NULL
"
(upper case). To return a null value from a non-numeric expression
(e.g. a String column) use the name "null
" (lower case).
Null values are often used in conjunction with the conditional
operator, "? :
"; the expression
test ? tval : fvalreturns the value
tval
if the boolean expression test
evaluates true, or fval
if test
evaluates false.
So for instance the following expression:
Vmag == -99 ? NULL : Vmagcan be used to define a new column which has the same value as the
Vmag
column for most values, but if Vmag
has the "magic" value -99 the new column will contain a blank.
The opposite trick (substituting a blank value with a magic one) can
be done like this:
NULL_Vmag ? -99 : VmagSome more examples are given in Section 10.8.
Note that for floating point data,
STILTS treats null
and NaN (Not-a-Number) values
somewhat interchangeably.
Blank values arising either from an input file format that can represent
missing values, or from processing that fails to provide a definite value,
are in most cases represented internally as null
for integer-type values and NaN for floating point values.
However in general users should not rely on distinguishing between
null
and NaN.
The operators are pretty much the same as in the C language. The common ones are:
+
(add)
-
(subtract)
*
(multiply)
/
(divide)
%
(modulus)
!
(not)
&&
(and)
||
(or)
^
(exclusive-or)
==
(numeric identity)
!=
(numeric non-identity)
<
(less than)
>
(greater than)
<=
(less than or equal)
>=
(greater than or equal)
&
(and)
|
(or)
^
(exclusive-or)
<<
(left shift)
>>
(right shift)
>>>
(logical right shift)
(byte)
(numeric -> signed byte)
(short)
(numeric -> 2-byte integer)
(int)
(numeric -> 4-byte integer)
(long)
(numeric -> 8-byte integer)
(float)
(numeric -> 4-type floating point)
(double)
(numeric -> 8-byte floating point)
+
(string concatenation)
[]
(array dereferencing - first element is zero)
?:
(conditional switch)
instanceof
(class membership)
Sometimes in an algebraic expression you will want to
use a literal string value,
for instance if you want to test a string-valued
column for equality with some fixed string.
Literal strings must always be delimited by double quote characters
("
).
This can be problematic, since (single or double) quote characters
are interpreted within STILTS parameter values as grouping text
that may contain spaces into a single token.
And, depending how you are invoking STILTS,
the shell may do a similar thing:
interpret (single or double) quote characters
on the command line as grouping text that may contain spaces
or shell magic characters into a single string
(such as a parameter value assignment) to pass to STILTS.
To complicate matters further, literals of the char
type
(single characters) in the expression language are delimited using
single-quote characters ('
),
though it's not very often necessary to deal with
char
values in STILTS.
And in a few cases specific parameters might have their own
requirements for quotes (like the adql
parameter of
tapquery
;
in ADQL string literals need single quotes and
double quotes delimit identifiers).
To summarise the common uses of quotes that you might have to make:
"
)
'
or "
)
'
or "
)
Unfortunately, all this can lead to a kind of quoting hell when trying to write STILTS commands, especially if they have to contain String literals, for which the author apologises :-(. Working round these requirements can be quite messy, but here are some tips:
ofmt=votable(format=BINARY)
will fail because the
shell tries to interpret the parenthesis characters, but
ofmt='votable(format=BINARY)'
, or
'ofmt=votable(format=BINARY)'
, will succeed.
cmd='select rmag<12'
instead of cmd='select "rmag < 12"'
.
cmd='keepcols "id ra dec"'
or cmd="keepcols 'id ra dec'"
.
Note however that different types of quote have somewhat
different meanings to the shell (single quotes are generally safer).
\
)
to prevent them being interpreted as grouping constructs.
So although cmd='select equals(release,"DR4")'
fails
(STILTS interprets the double quotes as grouping characters
rather than string literal delimiters),
cmd='select equals(release,\"DR4\")'
succeeds.
\
)
to prevent them being interpreted as token separators.
So cmd='keepcols id\ ra\ dec'
can be used to mean the same
as cmd='keepcols "id ra dec"'
.
Armed with this information it is usually possible to phrase a STILTS command on the command line that does what you want. In cases where that seems to be untrue or too painful, there are a couple of ways to avoid use of the shell, removing one layer of quote (mis)interpretation:
cmd
-like) parameters
can be specified using file indirection;
cmd=@filename
reads the filter value(s) from the line(s) of the named file
The comments above concerning the Unix shell
are in principle dependent on
which shell is in use, but they should apply to
sh
, bash
and csh
on any common Unix-like OS including MacOS.
Something similar probably applies to other OSes like MS Windows,
but the details may be different.
Many functions are available for use within your expressions, covering standard mathematical and trigonometric functions, arithmetic utility functions, type conversions, and some more specialised astronomical ones. You can use them in just the way you'd expect, by using the function name (unlike column names, this is case-sensitive) followed by comma-separated arguments in brackets, so
max(IMAG,JMAG)will give you the larger of the values in the columns IMAG and JMAG, and so on.
The functions available for use by default
are listed by class in the following subsections
with their arguments and short descriptions.
The funcs
command provides another
way to browse these function descriptions online.
Standard arithmetic functions including things like rounding, sign manipulation, and maximum/minimum functions. Phase folding operations, and a convenient form of the modulus operation on which they are based, are also provided.
roundUp( x )
x
(floating point): a value.x
rounded uproundDown( x )
x
(floating point): a valuex
rounded downround( x )
x
(floating point): a floating point value.x
rounded to the nearest integerroundDecimal( x, dp )
float
(32-bit floating point value),
so this is only suitable for relatively low-precision values.
It's intended for truncating the number of apparent significant
figures represented by a value which you know has been obtained
by combining other values of limited precision.
For more control, see the functions in the Formats
class.
x
(floating point): a floating point valuedp
(integer): number of decimal places (digits after the decimal point)
to retainx
but with a
limited apparent precisionroundDecimal(PI,2) = 3.14f
abs( x )
x
(integer): the argument whose absolute value is to be determinedabs( x )
x
(floating point): the argument whose absolute value is to be determinedmax( a, b )
Multiple-argument maximum functions are also provided in the
Arrays
and Lists
packages.
a
(integer): an argument.b
(integer): another argument.a
and b
.maxNaN( a, b )
a
(floating point): an argument.b
(floating point): another argument.a
and b
.maxReal( a, b )
Multiple-argument maximum functions are also provided in the
Arrays
and Lists
packages.
a
(floating point): an argumentb
(floating point): another argumenta
and b
min( a, b )
Multiple-argument minimum functions are also provided in the
Arrays
and Lists
packages.
a
(integer): an argument.b
(integer): another argument.a
and b
.minNaN( a, b )
a
(floating point): an argument.b
(floating point): another argument.a
and b
.minReal( a, b )
Multiple-argument minimum functions are also provided in the
Arrays
and Lists
packages.
a
(floating point): an argumentb
(floating point): another argumenta
and b
mod( a, b )
a/b
.
This is a modulo operation, but differs from the expression
a%b
in that the answer is always >=0
(as long as b
is not zero).
a
(floating point): dividendb
(floating point): divisora
by b
modulo(14, 5) = 4
modulo(-14, 5) = 1
modulo(2.75, 0.5) = 0.25
phase( t, period )
For positive period, the returned value is in the range [0,1).
t
(floating point): valueperiod
(floating point): folding periodphase(7, 4) = 0.75
phase(-1000.5, 2.5) = 0.8
phase(-3300, 33) = 0
phase( t, period, t0 )
t0
corresponds to phase zero.
For positive period, the returned value is in the range [0,1).
t
(floating point): valueperiod
(floating point): folding periodt0
(floating point): reference value, corresponding to phase zerophase(5003,100,0) = 0.03
phase(5003,100,2) = 0.01
phase(5003,100,4) = 0.99
phase( t, period, t0, phase0 )
t0
corresponds to integer phase
value, and the phase offset phase0
determines the
starting value for the phase range.
For positive period, the returned value is in the range
[phase0
,phase0+1
).
t
(floating point): valueperiod
(floating point): folding periodt0
(floating point): reference value, corresponding to phase zerophase0
(floating point): offset for phasephase(23,10,1,99) = 99.2
phase(8.6125,0.2,0.0125,-0.3) = 0
phase(8.6125,0.2,0.1125,-0.7) = -0.5
Functions which operate on array-valued cells. The array parameters of these functions can only be used on values which are already arrays (usually, numeric arrays). In most cases that means on values in table columns which are declared as array-valued. FITS and VOTable tables can have columns which contain array values, but other formats such as CSV cannot.
If you want to calculate aggregating functions like sum, min, max etc
on multiple values which are not part of an array,
it's easier to use the functions from the Lists
class.
Note that none of these functions will calculate statistical functions over a whole column of a table.
The functions fall into a number of categories:
size
,
count
,
countTrue
,
maximum
,
minimum
,
sum
,
mean
,
median
,
quantile
,
stdev
,
variance
,
join
.
add
,
subtract
,
multiply
,
divide
,
reciprocal
,
condition
,
slice
,
pick
.
Mostly these work on any numeric array type and return
floating point (double precision) values,
but some of them (slice
, pick
)
have variants for different array types.
array
,
which lets you assemble a floating point array value from
a list of scalar numbers.
There are variants (intArray
, stringArray
)
for some different array types.
sum( array )
array
is not a numeric array, null
is returned.
array
(Object): array of numbersarray
mean( array )
array
is not a numeric array, null
is returned.
array
(Object): array of numbersarray
variance( array )
array
is not a numeric array,
null
is returned.
array
(Object): array of numbersarray
stdev( array )
array
is not a numeric array,
null
is returned.
array
(Object): array of numbersarray
minimum( array )
array
is not a numeric array, null
is returned.
array
(Object): array of numbersarray
maximum( array )
array
is not a numeric array, null
is returned.
array
(Object): array of numbersarray
median( array )
array
is not a numeric array, null
is returned.
array
(Object): array of numbersarray
quantile( array, quant )
quant
value;
values of 0, 0.5 and 1 give the minimum, median and maximum
respectively. A value of 0.99 would give the 99th percentile.
array
(Object): array of numbersquant
(floating point): number in the range 0-1 deterining which quantile
to calculatequant
size( array )
array
is not an array, zero is returned.
array
(Object): arrayarray
count( array )
array
is not an array, zero is returned.
array
(Object): array (may or may not be numeric)array
countTrue( array )
array
(array of boolean): array of true/false valuesarray
join( array, joiner )
array
is not an array, null is returned.
array
(Object): array of numbers or stringsjoiner
(String): text string to interpose between adjacent elementsarray
elements separated by
joiner
stringsjoin(array(1.5,2.1,-3.9), "; ") = "1.5; 2.1; -3.9"
dotProduct( array1, array2 )
array1
(Object): first arrayarray2
(Object): second arraydotProduct(array(3,4,5), array(1,2,3)) = 26
add( arrayOrScalar1, arrayOrScalar2 )
If the arguments are not as expected (e.g. arrays of different lengths, both scalars, not numeric) then null is returned.
arrayOrScalar1
(Object): first numeric array/scalararrayOrScalar2
(Object): second numeric array/scalararrayOrScalar1 + arrayOrScalar2
,
the same length as the input array(s)add(array(1,2,3), array(0.1,0.2,0.3))
= [1.1, 2.2, 3.3]
add(array(1,2,3), 10) = [11,12,13]
subtract( arrayOrScalar1, arrayOrScalar2 )
If the arguments are not as expected (e.g. arrays of different lengths, both scalars, not numeric) then null is returned.
arrayOrScalar1
(Object): first numeric array/scalararrayOrScalar2
(Object): second numeric array/scalararrayOrScalar1 - arrayOrScalar2
,
the same length as the input array(s)subtract(array(1,2,3), array(0.1,0.2,0.3))
= [0.9, 1.8, 2.7]
subtract(array(1,2,3), 1.0)
= [0, 1, 2]
multiply( arrayOrScalar1, arrayOrScalar2 )
If the arguments are not as expected (e.g. arrays of different lengths, both scalars, not numeric) then null is returned.
arrayOrScalar1
(Object): first numeric array/scalararrayOrScalar2
(Object): second numeric array/scalararrayOrScalar1 * arrayOrScalar2
,
the same length as the input array(s)multiply(array(1,2,3), array(2,4,6)) = [2, 8, 18]
multiply(2, array(1,2,3)) = [2, 4, 6]
divide( arrayOrScalar1, arrayOrScalar2 )
If the arguments are not as expected (e.g. arrays of different lengths, both scalars, not numeric) then null is returned.
arrayOrScalar1
(Object): first numeric array/scalararrayOrScalar2
(Object): second numeric array/scalararrayOrScalar1 / arrayOrScalar2
,
the same length as the input array(s)divide(array(0,9,4), array(1,3,8)) = [0, 3, 0.5]
divide(array(50,60,70), 10) = [5, 6, 7]
reciprocal( array )
array
argument is not a numeric array,
null
is returned.
array
(Object): array inputarray
parameterreciprocal(array(1,2,0.25) = [1, 0.5, 4]
condition( flagArray, trueValue, falseValue )
This has the same effect as applying the expression
outArray[i] = flagArray[i] ? trueValue : falseValue
.
flagArray
(array of boolean): array of boolean valuestrueValue
(floating point): output value corresponding to an input true valuefalseValue
(floating point): output value corresponding to an input false valueflagArray
condition([true, false, true], 1, 0) = [1, 0, 1]
constant( n, value )
Note:
This documents the double-precision version of the routine.
Corresponding routines exist for other data types
(float
, long
, int
,
short
, byte
).
n
(integer): size of output arrayvalue
(floating point): value of every element in the output arrayn
-element array with every element set to
value
constant(5, 23.5) = [23.5, 23.5, 23.5, 23.5, 23.5]
slice( array, i0, i1 )
The semantics are like python array slicing, though both limits
have to be specified: the output array contains the sequence of
elements in the input array from i0
(inclusive)
to i1
(exclusive). If a negative value is given
in either case, it is added to the length of the input array,
so that -1 indicates the last element of the input array.
The indices are capped at 0 and the input array length respectively,
so a large positive value may be used to indicate the end of the array.
If the end index is less than or equal to the start index,
a zero-length array is returned.
Note:
This documents the double-precision version of the routine.
Corresponding routines exist for other data types
(float
, long
, int
,
short
, byte
, String
,
Object
).
array
(array of floating point): input arrayi0
(integer): index of first element, inclusive
(may be negative to count back from the end)i1
(integer): index of the last element, exclusive
(may be negative to count back from the end)i0
and i1
slice(array(10,11,12,13), 0, 3) = [10, 11, 12]
slice(array(10,11,12,13), -2, 999) = [12, 13]
pick( array, indices, ... )
The output array consists of one element selected from the input array for each of the supplied index values. If a negative value is supplied for an index value, it is added to the input array length, so that -1 indicates the last element of the input array. If the input array is null, null is returned. If any of the index values is out of the range of the extent of the input array, an error results.
Note:
This documents the double-precision version of the routine.
Corresponding routines exist for other data types
(float
, long
, int
,
short
, byte
, String
,
Object
).
array
(array of floating point): input arrayindices
(integer, one or more): one or more index into the input array
(may be negative to count back from the end)indices
pick(array(10,11,12,13), 0, 3) = [10, 13]
pick(array(10,11,12,13), -1, -2, -3)
= [13, 12, 11]
arrayFunc( expr, inArray )
The supplied expression can use the variable "x
"
to refer to the corresponding element of the input array, and
"i
"
to refer to its (zero-based) index.
The various functions and operators from the expression language
can all be used, but it is currently not possible
to reference other table column values.
If there is an error in the expression, a blank value (not an array) will be returned.
expr
(String): expression mapping input to output array valuesinArray
(Object): input arrayinArray
, or null for a bad expr
arrayFunc("3*x",array(0,1,2,3,NaN))
= [0, 3, 6, 9, NaN]
arrayFunc("pow(2,i)+x", array(0.5,0.5,0.5,0.5))
= [1.5, 2.5, 4.5, 8.5]
intArrayFunc( expr, inArray )
The supplied expression can use the variable "x
"
to refer to the corresponding element of the input array, and
"i
"
to refer to its (zero-based) index.
The various functions and operators from the expression language
can all be used, but it is currently not possible
to reference other table column values.
If there is an error in the expression, a blank value (not an array) will be returned.
expr
(String): expression mapping input to output array valuesinArray
(Object): input arrayinArray
, or null for a bad expr
intArrayFunc("-x",sequence(5))
= [0, -1, -2, -3, -4]
indexOf( array, item )
item
is the first entry in the array
, the return value
will be zero.
If the item does not appear in the array, -1 is returned. If it appears multiple times, the index of its first appearance is returned.
If indexOf(array, item)==n
, then
array[n]
is equal to item
.
Note:
This documents the Object
version of the routine.
Corresponding routines exist for other data types
(double
, float
, long
,
int
, short
).
array
(array of Object): array which may contain the supplied itemitem
(Object): entry to look for in the arrayitem
in array
,
or -1indexOf(stringArray("QSO", "BCG", "SNR"), "BCG")
= 1
indexOf(stringArray("QSO", "BCG", "SNR"), "TLA")
= -1
sequence( n )
See also the loop
functions, which provide
similar functionality.
n
(integer): length of arrayn
-element array,
(0, 1, 2, ... n
-1)sequence(4) = (0, 1, 2, 3)
sequence( n, start, step )
See also the loop
functions, which provide
similar functionality.
n
(integer): length of arraystart
(floating point): value of first elementstep
(floating point): increment to apply to each elementn
-element array,
(start, start+step, start+2*step, ... start+(n-1)*step)sequence(4, 100, 0.1) =
(100.0, 100.1, 100.2, 100.3)
loop( start, end )
for (int x = start; x < end; x++)
If you want a floating point array, or one with a non-unit step,
you can use the three-parameter version of
the loop
function.
See also the sequence
functions.
start
(integer): value for first element of output arrayend
(integer): value one greater than last element of output arrayend-start
(or 0) elements
(start, start+1, start+2, ..., end-1)
loop(0, 5) = (0, 1, 2, 3, 4)
loop(5, 0) = ()
loop( start, end, step )
for (double x = start; x < end; x += step)
Note that numerical precision issues may result in the output
array differing from its expected length by 1
(it is generally risky to rely on exact comparison of
floating point values).
If you want to be sure of the number of elements you can use
the sequence
function instead.
start
(floating point): value for first element of output arrayend
(floating point): first value beyond last element of output arraystep
(floating point): increment between output array values; may not be zero(end-start)/step
(or 0) elementsloop(10, 12, 0.5) = (10.0, 10.5, 11.0, 11.5)
loop(0, 10, 3) = (0., 3., 6., 9.)
loop(5, 0, -1) = (5., 4., 3., 2., 1.)
array( values, ... )
values
(floating point, one or more): one or more array elementsintArray( values, ... )
values
(integer, one or more): one or more array elementsstringArray( values, ... )
values
(String, one or more): one or more array elementsBit manipulation functions.
Note that for bitwise AND, OR, XOR of integer values etc you can use
the java bitwise operators
"&
", "|
", "^
".
hasBit( value, bitIndex )
value
(long integer): integer whose bits are to be testedbitIndex
(integer): index of bit to be tested in range 0..63,
where 0 is the least significant bit(value & 1L<<bitIndex) != 0
hasBit(64, 6) = true
hasBit(63, 6) = false
bitCount( i )
i
(long integer): integer valuei
bitCount(64) = 1
bitCount(3) = 2
toBinary( value )
value
(long integer): integer valuevalue
toBinary(42) = "101010"
toBinary(255^7) = "11111000"
fromBinary( binVal )
binVal
(String): binary representation of valuebinVal
fromBinary("101010") = 42
Functions for converting between strings and numeric values.
toString( fpVal )
fpVal
(floating point): floating point numeric valuefpVal
toString( intVal )
intVal
(long integer): integer numeric valueintVal
toString( charVal )
charVal
(char): character numeric valuecharVal
toString( byteVal )
byteVal
(byte): byte numeric valuebyteVal
toString( booleanVal )
booleanVal
(boolean): boolean value (true or false)booleanVal
("true
" or "false
")toString( objVal )
toString
to any object value without knowing its type
and get a useful return from it.
objVal
(Object): non-primitive valueobjVal
parseByte( str )
str
(String): string containing numeric representationstr
parseShort( str )
str
(String): string containing numeric representationstr
parseInt( str )
str
(String): string containing numeric representationstr
parseLong( str )
str
(String): string containing numeric representationstr
parseFloat( str )
str
(String): string containing numeric representationstr
parseDouble( str )
str
(String): string containing numeric representationstr
parseBigInteger( str )
The result is a BigInteger
object,
which can't be used in normal numeric expressions, but has a number
of methods defined on it for comparison, arithmetic,
bit manipulation etc.
See the
java.math.BigInteger javadocs for details.
str
(String): string containing numeric representationstr
parseBigInteger("-20000000000000000023").doubleValue()
= -2e19
parseBigInteger("18446744073709551616").testBit(64)
= true
parseBigDecimal( str )
The result is a BigDecimal
object,
which can't be used in normal numeric expressions, but has a number
of methods defined on it for comparison, arithmetic,
bit manipulation etc.
See the
java.math.BigDecimal javadocs for details.
str
(String): string contining numeric representationstr
parseBigDecimal("101").compareTo(parseBigDecimal("102"))
= -1
parseInts( str )
The details of this function's behaviour may change in future releases.
str
(String): string containing a list of integer valuesparseInts("9 8 -23") = [9, 8, -23]
parseInts("tiddly-pom") = []
parseDoubles( str )
This function can be used as a hacky way to extract the
numeric values from an STC-S
(for instance ObsCore/EPNcore s_region
) string.
The details of this function's behaviour may change in future releases.
str
(String): string containing a list of floating point valuesparseDoubles("1.3, 99e1, NaN, -23")
= [1.3, 990.0, NaN, -23.0]
parseDoubles("Polygon ICRS 0.8 2.1 9.0 2.1 6.2 8.6")
= [0.8, 2.1, 9.0, 2.1, 6.2, 8.6]
parseDoubles("La la la") = []
toByte( value )
value
(floating point): numeric value for conversionvalue
converted to type bytetoShort( value )
value
(floating point): numeric value for conversionvalue
converted to type shorttoInteger( value )
value
(floating point): numeric value for conversionvalue
converted to type inttoLong( value )
value
(floating point): numeric value for conversionvalue
converted to type longtoFloat( value )
value
(floating point): numeric value for conversionvalue
converted to type floattoDouble( value )
value
(floating point): numeric value for conversionvalue
converted to type doubletoHex( value )
value
(long integer): integer valuevalue
toHex(42) = "2a"
fromHex( hexVal )
hexVal
(String): hexadecimal representation of valuehexVal
fromHex("2a") = 42
Functions for angle transformations and manipulations, with angles generally in degrees. In particular, methods for translating between degrees and HH:MM:SS.S or DDD:MM:SS.S type sexagesimal representations are provided.
degreesToDms( deg )
deg
(floating point): angle in degreesdeg
degreesToDms( deg, secFig )
deg
(floating point): angle in degreessecFig
(integer): number of decimal places in the seconds fielddeg
degreesToHms( deg )
deg
(floating point): angle in degreesdeg
degreesToHms( deg, secFig )
deg
(floating point): angle in degreessecFig
(integer): number of decimal places in the seconds fielddeg
dmsToDegrees( dms )
dm[s]
, or some others.
Additional spaces and leading +/- are permitted.
The :seconds part is optional.
dms
(String): formatted DMS stringdms
hmsToDegrees( hms )
hm[s]
, or some others.
Additional spaces and leading +/- are permitted.
The :seconds part is optional.
hms
(String): formatted HMS stringhms
dmsToDegrees( deg, min, sec )
In conversions of this type, one has to be careful to get the
sign right in converting angles which are between 0 and -1 degrees.
This routine uses the sign bit of the deg
argument,
taking care to distinguish between +0 and -0 (their internal
representations are different for floating point values).
It is illegal for the min
or sec
arguments
to be negative.
deg
(floating point): degrees part of anglemin
(floating point): minutes part of anglesec
(floating point): seconds part of anglehmsToDegrees( hour, min, sec )
In conversions of this type, one has to be careful to get the
sign right in converting angles which are between 0 and -1 hours.
This routine uses the sign bit of the hour
argument,
taking care to distinguish between +0 and -0 (their internal
representations are different for floating point values).
hour
(floating point): degrees part of anglemin
(floating point): minutes part of anglesec
(floating point): seconds part of angleskyDistanceDegrees( ra1, dec1, ra2, dec2 )
ra1
(floating point): right ascension of point 1 in degreesdec1
(floating point): declination of point 1 in degreesra2
(floating point): right ascension of point 2 in degreesdec2
(floating point): declination of point 2 in degreesposAngDegrees( ra1, dec1, ra2, dec2 )
ra1
(floating point): right ascension of point 1 in degreesdec1
(floating point): declination of point 1 in degreesra2
(floating point): right ascension of point 2 in degreesdec2
(floating point): declination of point 2 in degreespolarDistanceDegrees( ra1, dec1, radius1, ra2, dec2, radius2 )
ra1
(floating point): right ascension of point 1 in degreesdec1
(floating point): declination of point1 in degreesradius1
(floating point): distance from origin of point1ra2
(floating point): right ascension of point 2 in degreesdec2
(floating point): declination of point2 in degreesradius2
(floating point): distance from origin of point2radius1
and radius2
Functions for angle transformations and manipulations, based on radians rather than degrees. In particular, methods for translating between radians and HH:MM:SS.S or DDD:MM:SS.S type sexagesimal representations are provided.
radiansToDms( rad )
rad
(floating point): angle in radiansrad
radiansToDms( rad, secFig )
rad
(floating point): angle in radianssecFig
(integer): number of decimal places in the seconds fieldrad
radiansToHms( rad )
rad
(floating point): angle in radiansrad
radiansToHms( rad, secFig )
rad
(floating point): angle in radianssecFig
(integer): number of decimal places in the seconds fieldrad
dmsToRadians( dms )
dm[s]
, or some others.
Additional spaces and leading +/- are permitted.
The :seconds part is optional.
dms
(String): formatted DMS stringdms
hmsToRadians( hms )
hm[s]
, or some others.
Additional spaces and leading +/- are permitted.
The :seconds part is optional.
hms
(String): formatted HMS stringhms
dmsToRadians( deg, min, sec )
In conversions of this type, one has to be careful to get the
sign right in converting angles which are between 0 and -1 degrees.
This routine uses the sign bit of the deg
argument,
taking care to distinguish between +0 and -0 (their internal
representations are different for floating point values).
It is illegal for the min
or sec
arguments
to be negative.
deg
(floating point): degrees part of anglemin
(floating point): minutes part of anglesec
(floating point): seconds part of anglehmsToRadians( hour, min, sec )
In conversions of this type, one has to be careful to get the
sign right in converting angles which are between 0 and -1 hours.
This routine uses the sign bit of the hour
argument,
taking care to distinguish between +0 and -0 (their internal
representations are different for floating point values).
hour
(floating point): degrees part of anglemin
(floating point): minutes part of anglesec
(floating point): seconds part of angleskyDistanceRadians( ra1, dec1, ra2, dec2 )
ra1
(floating point): right ascension of point 1 in radiansdec1
(floating point): declination of point 1 in radiansra2
(floating point): right ascension of point 2 in radiansdec2
(floating point): declination of point 2 in radiansposAngRadians( ra1, dec1, ra2, dec2 )
ra1
(floating point): right ascension of point 1 in radiansdec1
(floating point): declination of point 1 in radiansra2
(floating point): right ascension of point 2 in radiansdec2
(floating point): declination of point 2 in radianspolarDistanceRadians( ra1, dec1, radius1, ra2, dec2, radius2 )
ra1
(floating point): right ascension of point 1 in radiansdec1
(floating point): declination of point1 in radiansradius1
(floating point): distance from origin of point1ra2
(floating point): right ascension of point 2 in radiansdec2
(floating point): declination of point2 in radiansradius2
(floating point): distance from origin of point2radius1
and radius2
hoursToRadians( hours )
hours
(floating point): angle in hoursdegreesToRadians( deg )
deg
(floating point): angle in degreesradiansToDegrees( rad )
rad
(floating point): angle in radiansraFK4toFK5radians( raFK4, decFK4 )
raFK4
(floating point): right ascension in B1950.0 FK4 system (radians)decFK4
(floating point): declination in B1950.0 FK4 system (radians)decFK4toFK5radians( raFK4, decFK4 )
raFK4
(floating point): right ascension in B1950.0 FK4 system (radians)decFK4
(floating point): declination in B1950.0 FK4 system (radians)raFK5toFK4radians( raFK5, decFK5 )
raFK5
(floating point): right ascension in J2000.0 FK5 system (radians)decFK5
(floating point): declination in J2000.0 FK5 system (radians)decFK5toFK4radians( raFK5, decFK5 )