Next Previous Up Contents
Next: Running Jobs tab
Up: Table Access Protocol (TAP) Query
Previous: Select Service tab
TAP window with Use Service tab visible
The Use Service tab of the
TAP load dialogue
displays information about the service you have selected to query,
including what tables are available and what columns they contain,
and allows you to enter the query text and some additional options
about how the query is to be executed.
Once you have entered suitable query text,
click the Run Query button at the bottom of the window
to submit the job.
The various parts of the window are described in detail below.
Metadata Panel
The Metadata panel displays information about
the tables held by the selected service.
On the left of the panel is a searchable tree
summarising the schemas and tables in the service,
and on the right are some tabs containing more detail.
Note all this information has to be loaded from the server,
so in some cases you may have to wait before it is visible.
The tree on the left contains a listing of the service's Schemas,
and under them the Tables they contain.
A Schema in this context is just a subject grouping that
contains one or more tables.
Each schema notes in brackets the number of tables it contains;
you can reveal or hide the tables by clicking on the schema node's handle.
If the service contains only a small number of tables it may be
convenient just to scroll up and down to see them all.
But for services with hundreds or thousands of tables,
you might need some help to find the tables that you're interested in
by using the controls above the tree:
-
Find
- You can restrict the displayed tables to those that you're
interested in.
To do this, type one or more search terms into the Find
field. The nodes displayed in the tree will be filtered to include only
those that match the search terms as you type them.
The Name and Descrip checkboxes
below the field indicate
whether you want your search terms to match tables/schemas by name
and/or by description.
The And/Or toggle button determines
whether each displayed table/schema has to match all of the search
terms or just one.
So for instance if you want to list just two tables, you can type in
both table full names, uncheck the Descrip control,
and set the toggle button to "Or".
-
Sort
- You can choose between two options to determine the order
in which entries are displayed in the tree.
Alphabetic means that the schemas are listed
in alphabetical order, and within each schema the tables are listed
in alphabetical order.
Service attempts to use information provided by
the data service about how to order the schemas and tables.
Depending on how the data provider has organised its metadata
this may list more important schemas and tables first,
or it may show them in a more or less random order,
or it may be alphabetical as with the other option.
To the right of the tree is a tabbed panel giving detail relating
to the currently selected tree node.
Select a tree node by clicking on it.
Each tab has a little circle next to the title which may be empty
or filled, according to whether it contains information.
If it's empty it's either because that information doesn't make sense
for the selected node in the tree
(e.g. if a schema is selected, there is no column information)
or because the information has not been retrieved from the service;
either it's on its way and will be filled in later, or there's an error.
The tabs are as follows:
-
Service tab
- Provides various items of information about the TAP service itself,
gathered from the service and its registry record if available.
This includes the service's name, Service URL, Reference URL,
curation and contact information, detailed description,
declared data models, and a summary of optional and custom features
of the ADQL implementation
(these are listed with more detail in the next tab).
It also displays your currently authenticated identity for
service interaction, if any.
If a Reference URL or Examples URL are provided,
clicking on the URLs or the little link icon ()
in the panel will try to open the relevant pages in
an external web browser (or you can copy and paste them yourself).
The content of this tab is not sensitive to which node of the tree
is currently selected.
-
ADQL tab
- Lists language features associated with the service.
This includes User-Defined Functions (UDFs) alongside all other
standard mandatory and optional functions the service provides,
as well as supported optional and custom language features
such as Common Table Expressions.
These functions and features are displayed in a tree
which can be explored by clicking on node handles ().
The content of this tab is not sensitive to which node of the tree
is currently selected, but it may change according to which
ADQL version is selected.
-
Schema tab
- Provides a textual description of the currently selected schema,
or the schema of the currently selected table.
A schema is just a higher-level grouping of tables within the TAP service.
Depending on how the service has arranged its tables this may or may
not provide useful information that's not available at the table level.
-
Table tab
- Provides a textual description of the currently selected table (if any).
Some additional information like the number of columns, rows and
foreign keys may also be displayed if available.
-
Columns tab
- Tabulates a list of all the columns in the currently selected table;
column names, data types, units, descriptions, UCDs, and possibly other
metadata are shown. This is generally the information you will need
to know about a table before you can write queries.
You can sort the entries by clicking on the column header
(clicking cycles between ascending, descending,
and unsorted),
so for instance to list the columns in alphabetical order click on
the Name column header,
or to group together all the columns with units of
mag
click on the Unit column header.
If no table is selected in the tree, this tab will be empty.
-
FKeys tab
- If the service provides information about foreign keys
(links between fields in different tables in the database)
they will be tabulated here for the currently selected table.
If any are present, they may help you to formulate efficient queries.
If no table is selected in the tree, this tab will be empty.
-
Hints tab
- This panel gives you a very basic "Cheat Sheet" for writing ADQL queries.
There are just a few hints to jog your memory about the required syntax
for some common operations. But the very best advice it gives
you is to use the Examples menu,
and the service-provided examples (from the Examples menu or
via a link) if available.
The content of this panel is somewhat sensitive to the
capabilities of the service, for instance the ADQL version selected
and the availability of service-provided examples.
Service Capabilities Panel
The Service Capabilities panel shows capability
metadata that the service provides about itself. This has to
be loaded from the server and may not appear immediately.
It contains the following information:
-
Query Language
- Shows what query languages, and what versions, are supported
by the service; the default selection is the most recent version
of ADQL on offer.
If there is more than one option listed, you can choose which language
you want to submit your query in, but if you choose something
which is not a variant of ADQL it may not work.
The ADQL version selected here will affect the syntax
highlighting in the ADQL Text panel below,
as well as the content of the ADQL and Hints tabs,
and the exact form of the example queries in
some of the Examples menu entries.
It's usually best to stick with the default language selection,
but experimenting with earlier ADQL versions if available
can help to understand differences between the ADQL versions.
-
Max Rows
- Selector which shows the maximum number of output rows
that the service is
willing to deliver as the result of a query under normal circumstances.
Entries in this list value may be marked "(default)" or "(max)".
You can change this value by typing in a different number, as long as
it does not exceed the server's maximum.
This value is here to protect the user, as well as the service, from
inadvertently requesting an unduly large output table.
Note that if the construction "
TOP nnn
" is used in the ADQL,
the nnn
limit may override the value supplied here.
-
Uploads
- Indicates whether table uploads are permitted, and perhaps what is the
largest upload size (in terms of rows and/or bytes) which will be
accepted.
See below for more about table uploads.
-
Log In/Out
- This button is used to log in or out of the TAP service
in cases where this is permitted or required.
If the service has no authentication, this button will be disabled.
See the Authentication
item below for more detail.
ADQL Text Panel
The ADQL Text panel is where you can enter the actual
query text for the request.
It has the following parts:
-
Query Mode selector
- Provides the following options to control what happens when you
hit the Run Query button:
-
Synchronous means that TOPCAT sends the query
to the server and
waits for the result as the response of the same HTTP request.
It's simpler and faster than async, but if the job takes a long time,
the request may time out and the results be lost.
When the query is submitted,
the Load Window will appear
with a load progress indicator,
and once it's complete
the table will be loaded into TOPCAT in the usual way.
-
Asynchronous means that TOPCAT sends the query
to the server and gets
a message back telling it where to look for information about the
job's progress; the application then keeps polling the server
to find out when it has completed.
When the query is submitted,
you will be taken to the
Running Jobs tab
where you can see the progress of your submitted job;
when it's ready it will be loaded into TOPCAT.
-
Quick Look runs the query as for Synchronous mode
above, but when the result comes back the data is just displayed in
a new popup window rather than loading it into TOPCAT for
further analysis.
Synchronous mode is suitable for short jobs (it may typically execute
with a few seconds less delay), but for longer jobs aysnchronous is
more reliable.
Some TAP services permit longer queries to be executed
in asynchronous than in synchronous mode.
It is possible to submit an asynchronous job
on one day from one machine, and pick up the results on a different
day from another machine, using another invocation of TOPCAT or of
a different application altogether.
Quick look is good if you just want to see the result but don't want
to do any detailed analysis of it. One example is if you just want
to know the number of rows in a table
(SELECT COUNT(*) FROM table
);
the result of the query is the row count in a one-column, one-row table,
and there's not much point loading that table into TOPCAT.
-
Edit toolbar
- This contains a number of actions that affect the
Text Entry panel:
-
Add Tab
- Adds a new blank editing tab, so you can start editing a new query
without affecting any existing ones.
-
Copy Tab
- Adds a new editing tab with text initially copied from that
in the currently visible one.
-
Remove Tab
- Deletes the currently visible tab.
-
Title Tab
- Sets the title of the currently visible tab.
By default the titles are consecutive integers (the first is "1",
the second "2" etc), but you can give them more meaningful names
if you like.
-
Clear
- Erases all the text in the currently visible tab.
The text can be retrieved using the Undo action.
-
Undo
- Undoes the most recent editing action in the currently visible tab.
You can do the same thing from the keyboard using
Ctrl-Z.
-
Redo
- Reverses the most recent Undo action in the currently visible tab.
You can do the same thing from the keyboard using
Ctrl-Shift-Z or Ctrl-Y.
-
Insert Table
- Inserts into the text at the current cursor position
the name of the table (if any) that is currently selected
in the metadata tree above.
This is a convenience that may save some typing,
especially for long or complicated table names.
-
Insert Columns
- Inserts into the text at the current cursor position
a comma-separated list of the names of any columns that
are currently selected in the Columns
tab of the Metadata panel above.
In order to select columns, click on the rows in the Columns panel;
you may need to hold down the Ctrl key while clicking
or something similar (depends on your OS) to select more than one.
Columns are inserted in the text in the order they were selected.
This is a convenience that may save some typing,
but note that you may still need to edit the inserted text,
for instance to add table name/alias prefixes.
-
Parse Errors
- If this button is enabled, it means that the parser has detected
an error in the currently visible ADQL text.
The error will usually be highlighted in pink in the editing panel.
Clicking this button pops up the actual parser error message,
which may give you more idea what's wrong.
-
Fix Errors
- If this button is enabled, it can fix some common errors in the
currently visible ADQL. This includes things like putting quotes
around table or column names that non-alphanumeric or that clash
with ADQL reserved words. There are plenty of problems it can't fix!
These actions are also available from the Edit menu.
-
Text Entry area
- This panel is where you type ADQL text that forms the query to send
to the TAP service for execution.
The Edit actions described above, along with
Undo/Redo actions from the keyboard, can also help.
You can have multiple queries on the go at once in different tabs;
add/copy/remove/retitle tabs with the toolbar actions, and move between
them by clicking on the tab headers at the top.
The tab you can see is the one that will be executed if you hit
the Run Query button.
As you enter the query text, it will be checked for syntax errors.
If an error is found, the place in the text which appears
to have caused it
will be highlighted in pink (as in the figure above).
To see more detail on the error,
click the Parse Errors () button.
In some cases, the parser can guess how to fix such errors;
if the Fix Errors () button is enabled,
you can try clicking that.
The checking itself is reasonably reliable, but the position of the
highlighted error, and the text of the error message, can sometimes
be a bit misleading, so don't take the details too seriously.
The error highlighting is just used as a warning, you are not prevented
from submitting a query marked as erroneous, since
in some cases TAP services may accept ADQL which is not strictly correct,
and in some cases the error checking may not be perfect.
Note also that for various reasons the service may not be able to
accept all queries that count as syntactically valid ADQL,
so even if TOPCAT doesn't report any errors, the service may still
respond with an error message.
The details of the syntax checking depend on the
table metadata, ADQL version and optional language features
declared by the service.
If multiple ADQL versions are available in the Query Language selector,
it is also be possible to change the language version checked
by making a different selection.
-
Examples line
- Clicking on the Examples button
at the bottom left of the panel will pop up a menu
that allows you to select from a number of example ADQL queries.
If you select one, the relevant ADQL text will be pasted into
the editing panel; you should be able to hit the Run Query
button straight away and execute it to get a result.
In some cases the example text is influenced by the
table you're currently looking at in the Metadata panel,
and by the ADQL version of the Query Language currently selected
in the Service Capabilities panel.
Until you are experienced with ADQL,
starting from one of these examples and editing it to taste
is often a good way to write a query of your own.
The Examples menu is divided into a number of sub-menus,
though not all will be enabled for all services.
The sub-menus are:
-
Basic:
Basic examples that should be applicable to most services.
-
Upload:
Examples that involve uploading a tables from TOPCAT to the TAP
server and executing queries using it.
Only enabled if the service supports table uploads and
at least one table is currently loaded into TOPCAT.
-
Service-Provided:
Examples provided by the TAP service.
These are likely to provide some insight into the kinds of query
that make sense and make good use of the particular tables
available from the service,
so it is often instructive to look at these.
Only enabled if the service actually provides some examples
(at time of writing, not many do, but hopefully this will improve).
Note to TAP service implementers:
at time of writing, there are two variants of the standard way
for services to provide examples
(DALI
sec 2.3 and
TAPNotes sec 4.2.1),
and it's not clear which is preferred.
For now, TOPCAT will recognise either.
-
TAP_SCHEMA:
Examples using the tables in the TAP_SCHEMA schema,
which describes metadata about the tables in the service.
You can use these to do things like count tables or columns
in the current service,
or locate tables with columns having particular physical meaning.
-
ObsTAP:
Examples using the IVOA
ObsCore data model
(
ivoa.obscore
table),
which describes astronomical observations in a standardised way.
You can use these to do things like search for all observations
at a particular sky position, or observation time, or waveband etc.
Only available if the service declares support for the ObsCore
data model.
-
RegTAP:
Examples using the IVOA
RegTAP data model
(tables in the
rr
schema).
You can use these to do things like search for
all TAP services containing tables having columns with particular
physical meanings,
or all cone search services relating to particular wavebands, etc.
You can use queries like this to perform more sophisticated searches
for TAP services than you can do using the
Select Service tab.
Only available if the TAP service declares support for the RegTAP
data model (i.e. if it hosts an IVOA searchable registry).
When you select an example from one of the sub-menus,
its name and title will be displayed to the right of the Examples button.
Little left and right arrow buttons
allow you to cycle through the examples in the current submenu
so you can browse what's available.
Some examples come with additional explanation on the web.
If the example you're currently looking at has such additional
documentation then the Info button
to the right will be enabled, and clicking on that should open
the relevant page in a browser on your desktop.
Some TAP services permit Table Uploads.
What this means is that you can upload tables from TOPCAT into the
remote database and perform queries on your table directly.
In this way you can perform joins between your own tables
(anything loaded into TOPCAT) and any of the tables in the remote
database. This is an extremely powerful feature.
To take advantage of it, you just need to use a special form for
the table name to reference a TOPCAT-based table:
you write "TAP_UPLOAD.t<n>
", where
<n>
is the ID number of the table in TOPCAT,
that is the number before the colon in the
Control Window Table List.
If the table's label has a suitable form
(roughly, if it's alphanumeric and unique)
you can use that instead of "t<n>
".
So, if a table is identified as "1: messier
"
in the table list, you can reference it in ADQL as
"TAP_UPLOAD.t1
" or as "TAP_UPLOAD.messier
"
- see the Upload
sub-menu of the Examples menu described above for
some examples.
Note the table uploaded in this way is the
Apparent Table corresponding to the
given ID number, i.e. current subset and column selections apply.
It's a good idea to ensure that any table you are uploading has
columns with sensible names, otherwise the service may rename the
columns or otherwise have trouble handling it.
Authentication
Some TAP services (at time of writing, a minority) permit or require
you to log in before using the service,
as discussed in Section 6.3.
The behaviour of this window depends on whether the service provides
mandatory, optional or no authentication,
and can be controlled using the
Log In/Out button on the right of the
Service Capabilities panel in the middle of the window.
-
Mandatory Authentication
- For services that require authentication,
when you open the Use Service panel
an authentication dialogue
asking you for your username
and password will appear before you can do anything else.
-
Optional Authentication
- For services that allow optional authentication,
the panel will be displayed initially in anonymous mode,
but you can use the Log In/Out button ()
to supply your username and password.
The advantage that authenticating confers on your usage
depends on the service;
in some cases you will have access to more of the hosted data,
in other cases it may entitle you to higher resource limits or
enable protection of your asynchronous query results.
TOPCAT can't tell you what the difference will be if you authenticate,
but following successful login it will reload and display
the service metadata in case it has changed for the
new authenticated context.
-
No Authentication
- For services with no possibility of authentication,
the Log In/Out button will be disabled.
The Log In/Out button's appearance changes depending
on whether you are logged in () or not ().
To check your authenticated identity, you can hover over the button
or look under the Authentication item in the
Service tab of the Metadata Panel above.
If you have already logged in, clicking this button will pop up
the authentication dialogue again
and you can enter a different
username and password, or select Anonymous to log out.
It is also possible to log out of all services you are currently
authenticated with by using the Reset Authentication
option from the Authentication menu.
Note:
These authentication arrangements in TOPCAT are new at version 4.9,
and rely on VO standards that are still under discussion.
The behaviour and user interface may change in future releases,
and at time of writing not all TAP services that provide authentication
advertise it in a way that TOPCAT can work with.
It is hoped that authentication interoperability
will improve in future versions of TOPCAT and of server-side software.
Next Previous Up Contents
Next: Running Jobs tab
Up: Table Access Protocol (TAP) Query
Previous: Select Service tab
TOPCAT - Tool for OPerations on Catalogues And Tables
Starlink User Note253
TOPCAT web page:
http://www.starlink.ac.uk/topcat/
Author email:
m.b.taylor@bristol.ac.uk
Mailing list:
topcat-user@jiscmail.ac.uk