Q1.1 What is TOPCAT?
TOPCAT is an interactive graphical tool for analysis and manipulation of tabular data. Its manifesto is:
Does what you want with tables
Q1.2 What is a table?
A table, for TOPCAT's purposes, is a data structure with columns, so that all the values in a given column are of the same kind. The types of value in each column are usually, but not necessarily, numbers, strings, or arrays of these. The most important example in astronomy of a table is an object or source catalogue. Two- or multi-dimensional arrays, such as images or data cubes are not tables - TOPCAT will not be able to help you with these.
Q1.3 What are TOPCAT's strengths?
Q1.4 What are its weaknesses?
Some of these capabilities (limited scriptability, large tables) are however provided by TOPCAT's sister package, STILTS.
Q1.5 Is TOPCAT just for astronomy?
The large majority of TOPCAT's user base is currently in astronomy. However many of its facilities are not astronomy-specific; it is quite suitable for use in other contexts, and has been so used. I'm very open to approaches from other scientific domains who are interested in using it, and possibly customising it accordingly, but the amount of effort to publicise the software effectively in another user community would be considerable, and I'm not currently doing that.
Q1.6 Is TOPCAT just for the Virtual Observatory?
No. In many ways, TOPCAT is a traditional data analysis tool, in that it will perform most of its functions quite happily with no network connection. It does however provide a number of VO-friendly facilities.
Q1.7 Why is TOPCAT written in Java?
Several reasons, some of the important ones being:
Q1.8 But isn't Java slow?
Q1.9 Gosh what a terrific bit of software. How can I express my appreciation?
Why thank you. The best thing you can do is to help publicise its use by telling your colleagues how you've found it useful. Acknowledging its use in published papers and talks is also very helpful. Telling me what you've used it for can (a) give me a warm fuzzy feeling inside and (b) come in useful when it comes to justifying development effort, e.g. to funding agencies.
Q1.10 How should I acknowledge TOPCAT?
It is much appreciated if you can acknowledge TOPCAT in your publications and presentations. The preferred method is to cite one or more of these papers:
Alternatively, you can cite one or both of the following permanent URLs:
If you publish papers or preprints acknowledging this software, it would be both interesting and helpful for me if you could let me know where to find them.
Q1.11 Where can I find old/new versions?
The most recent public release is available from the web page. Previous versions can be found in the release archive. New releases are notified on both the mailing lists.
There will sometimes be a bleeding-edge version in the pre-release directory - this may contain fixes and new features not yet available in the public release which you can experiment with at your own risk.
Q1.12 How do I get the source code to look at?
The source code for TOPCAT and friends is available in the Starlink/starjava repository on Github.
If you just want all the source code just to look at,
an alternative may be to grab the
topcat_src.zip file from the web page,
which contains all the java source code for TOPCAT, STILTS and the
uk.ac.starlink.* classes it uses.
Third party library code is not included.
See also Q1.13.
Q1.13 How can I compile TOPCAT from source code?
The proper way to build the application from scratch is to check out the starjava build system from the starjava git repository. No password is required for read access. Do something like:
% git clone git://github.com/Starlink/starjava.git starjava/sourceYou can then read the file
starjava/source/READMEfor instructions on what to do next. Your mileage may vary somewhat, but the sequence will be along these lines:
% cd starjava/source % setenv STAR_JAVA /usr/java/jdk1.6.0/bin/java .. or wherever % cd ant % bin/ant build install % cd .. % ../bin/ant build % ../bin/ant installTo build the standalone TOPCAT jar files
topcat-lite.jaryou can do:
% cd topcat % ../../bin/ant build-standalone installYour JDK may be any version >=1.6, and must have JAI (Java Advanced Imaging) installed. Note that this process builds the whole starjava suite of software, which includes more than just TOPCAT.
Since the standalone jar file
contains all the library classes, a quick and dirty alternative
is to compile from the sources in the
topcat_src.zip file from the web page
topcat-full.jar on your classpath at compile time.
Q1.14 I think I've found a bug.
If there's something wrong which is not noted in Section 4, please mail me and let me know. I may be able to give you a workaround or updated copy, and more importantly it gives me a chance to fix it in future versions.
Q1.15 Why doesn't TOPCAT display sky images/show movies/speak my weight?
I have a long list of features that I'd like to add eventually, some of which will probably get done and some of which will probably not. However, I'm always interested in new suggestions. The existing feature set is very largely due to suggestions and requests from existing users.
Having said that, there are some things which I am unlikely to add, in particular things which are not basically to do with manipulation of tables, and things which would require non-Java code to be introduced.
Q1.16 How can I find out when new versions are released?
Subscribe to the
topcat-announce mailing list - see
list information here.
Q1.17 What license is TOPCAT released under?
The application is releaed under the GNU General Public Licence (GPL). However, many of the underlying libraries are available under more permissive licenses such as the LGPL. There is some more information in the STILTS LICENSE.txt file.
Q1.18 What related software is there?
The following items are related to TOPCAT:
Q1.19 Who has developed TOPCAT?
Most of the code has been written by Mark Taylor. Much help has been supplied by other members of various user and developer communities, and some external libraries are used. More details are available in the Acknowledgements section of the manual.
Q1.20 What project(s) support(s) TOPCAT?
TOPCAT has been supported by a number of grants and projects over its lifetime, nearly all ultimately funded by UK or European astronomy research organisations. In chronological order:
The work has mostly been done in the Astrophysics Group of the School of Physics at the University of Bristol in the UK (though some was done in various senses at the Institute of Astronomy in Cambridge and ARI in Heidelberg).
Q1.21 What are its future prospects?
TOPCAT has no assured long term funding, but has managed for the last decade or so to hop from one funding source to another without falling between the cracks. At time of writing things are looking positive for the near future: funding is secured until around the mid 2018 by grants from STFC, the UK's main funding body for astronomical research.
If because of terminated funding or for some other reason I cease work on it, another project or individual could take on some level of maintenance, support and/or development in my absence. I like to think that the code is sufficiently well organised and documented that this is a realistic possibility. Feel free to take a look to see whether you agree.
Q2.1 What do I need to run TOPCAT?
You need a Java SE (Java 2 Standard Edition) runtime environment version 1.6 or above (see Q2.2) and, usually, one of the TOPCAT jar files (topcat-lite.jar or topcat-full.jar, though it's available in other forms too). Since Java is available for almost all modern machines, TOPCAT will run on almost any computer with a screen, for instance Linux, Solaris, Mac OSX and MS Windows.
It will not run on a phone, iPad, or within a web browser.
Q2.2 What version of Java do I need?
The short answer is that any non-ancient version of the Java 2 Standard Edition should be OK - specifically Java SE 6 or later. On Mac OS X any known version (Apple Java 6 or Oracle Java 7 or 8) should work (though there are one or two known issues). On other platforms, Oracle's JRE is recommended. If you do not already have this, you can obtain it from Oracle's Java SE page "Java Runtime Environment" is what you want. If for some strange reason you need to run using Java 1.4, you can use a TOPCAT version older than version 3.6, or a version older than 4.3-3 for Java 1.5.
Recent versions of OpenJDK have been reported to work tolerably well, though some people have also experienced problems with them. For the present I will continue to develop using Oracle's Java, so it remains the recommended platform.
At the time of writing GNU's
does not appear to work well.
gij is distributed as
/usr/bin/java in some Linux distributions.
Q2.3 I'm running Ubuntu and my TOPCAT windows look ugly
The implementation of Java that runs by default on current versions of Ubuntu at time of writing is based on OpenJDK. This works to some extent, but various things, including the size and shape of some graphical elements, is substandard (see the previous question - but this may only apply to older Ubuntu versions?). You are advised to use Sun's java instead. This can be done on Ubuntu as follows:
% sudo update-java-alternatives -s java-6-sunIf that doesn't work, try "
update-java-alternatives -l" to see what options are available, and try to find one with "
sun" in the name.
If no such java version is offered, you might need to install it, using the Adept package installer. From the Universe repository. Or something. (Note: I'm not really up to speed on Ubuntu workings - I'm happy to accept more authoritative text from somebody who is better informed).
Q2.4 What's the best way to download and run TOPCAT?
The recommended method is to grab
TOPCAT web page.
On a Un*x system you can also get (and chmod +x) the
shell script and put it in the same directory.
Then just invoke "
topcat" (if you have the shell script)"
java -jar topcat-full.jar" and you're off.
If you are tight for space, you can use
instead - it is missing only a few less important facilities.
If you have a Mac instead, you may prefer to get the topcat-full.dmg file instead, but note items Q4.3 and Q2.5 in this case.
Q2.5 How do I set command-line flags from Mac OS X?
If you have downloaded the
topcat.dmg file from the
web page and installed it on your Mac, you will be able to run it in a
Mac-like way by pointing and clicking. However, this does not permit
you to specify important command-line flags such as
-Xmx256M, not to mention specifying tables on the
command line to load at startup etc
(see the manual for
more information on command-line arguments).
To do that, you will have to run it from the command line.
Depending where you have installed the
topcat script should be present at a location like:
/Applications/TOPCAT.app/bin/topcat <flags>or putting
/Applications/TOPCAT.app/binon your path and invoking
topcat <flags>should do the trick.
Another possibility if there are command-line flags that you want
to use on a regular basis is to edit the
file associated with the application bundle. You should find this
Apple provides documentation on the
Info.plist file format.
In particular if you want to
increase the heap memory
you can fill in the
<string> value of the
Java section with a value like "
Thanks to Frank Primini, Juan de Dios Santander Vela and Klaus Rübke for help with this question.
Q2.6 I have a Retina hi-res display on my Mac and TOPCAT looks blurry
Thanks to Christophe Deil for supplying this answer.
TOPCAT does work on Macbook Pro with Retina display, although some plots are drawn as bitmaps by the application and then scaled by 2x by the java implementation before being rendered to the Retina screen, resulting in unsharp image and text. Everything else (text, buttons, other plots) will look Retina-sharp if you start TOPCAT in high-resolution mode.
From the command line TOPCAT will start in high-resolution mode automatically. To make TOPCAT start in high-resolution mode when double-clicked on the icon in Applications or the Dock, you have to manually declare TOPCAT to be high-resolution capable. Add
<key>NSHighResolutionCapable</key> <true/>toward the end of the
/Applications/TOPCAT.app/Contents/Info.plistfile with the text editor of your choice, right before the closing
</dict></plist>lines (see also the previous question). Because Mac OS X caches the Info.plist content you have do a little trick to make it reload the Info.plist content change. Go to Applications in Finder and select TOPCAT. Type CMD+C, then CMD+V and you'll get a copy called "TOPCAT copy". Delete the original TOPCAT, then rename "TOPCAT copy" to TOPCAT. This seems fishy, but it is the standard suggestion for plist changes.
Finally you can check by right-clicking on TOPCAT and selecting "Get Info" in Applications, you should see the "Open in Low Resolution" checkbox disabled. A simpler check is to just start TOPCAT, if low-resolution mode is enabled you will see it right away because then everything in TOPCAT (text, buttons, plots) will look unsharp.
I may decide to change the default in the future, but for now there seems a question about whether setting hi-res might cause problems. Any feedback about positive or negative experiences with this is very welcome.
Q2.7 How do I set up TOPCAT to run on Microsoft Windows?
The following instructions have kindly been supplied by Clive Page for use with Windows XP:
C:\WINDOWS\System32\java.exeand they should be modified in the obvious way if this is not the case.
topcat-full.jarto a suitable location; the following instructions assume
topcat-full.jarand that you put it in your own "
My Documents" folder, but any folder will do.
topcat-full.jarin your own "
My Documents" folder:
C:\WINDOWS\System32\java.exe -jar "C:\Documents and Settings\your name\My Documents\topcat-full.jar"If you don't get this right first time it can be altered later, using the Properties window as described below. Note that you need quotation marks around the string after the
-jarswitch if it contains spaces, as it does in the example above. If you want to provide a switch for example to specify more memory, such as
-Xmx100Mfor 100 Megabytes, then this switch should be inserted before the
-jarswitch. Click NEXT, then you get prompted for a name: enter "Topcat" or whatever seems appropriate. Click FINISH and a new icon should appear on your desktop (with a Java logo).
java.exefile. You will probably find it convenient to alter this to some other folder, such as the one where your FITS files are generally located, or where your browser downloads them from the web. After editing it, finish by clicking Apply and OK.
Christopher Bishop adds some notes relating to Vista and Firefox downloads:
If you are downloading Topcat using Firefox on Vista it is important that you make sure that you set the Firefox download to the folder you want to save it in. If you don't then the topcat-full.jar will end up by default in a temporary file. If you then follow steps 3 and 4 above a Topcat icon will be created, but you may find Vista will have deleted your download in the temp file. At least it did in mine! The solution I found was to go to Firefox Tools - Options - Main and then look at the downloads window. In the downloads window there is a box that enables you to specify where you want the download to go. So I put
C:\user\yourname\documents(I would check in your documents folder that topcat-full.jar is really there and that Vista hasn't deleted it!)
Q2.8 Can TOPCAT be run as an applet?
No. It is possible that with some changes to the code it could be done, but I don't have any plans to do that. Applets are a bit out of fashion now, and in general TOPCAT seems a bit heavyweight to run inside a web browser (for instance it tends to need a lot of windows at once).
It can be run using Java WebStart, which is a technology that allows you to make one click on a web page and start the application (downloading it automatically first if necessary). You can see an example at topcat-lite.jnlp.
Q2.9 It doesn't seem to work!
Installing a java application such as TOPCAT is usually pretty
straightforward, and there's not too much that can go wrong - at least
compared to applications written in compiled languages such as C.
If it's not starting, or failing to complete even very simple actions,
the most likely explanation is that the version of Java you are using
is broken or incomplete. The most likely cause of this is that you are
using GNU Java, which at time of writing is not nearly complete enough
to run TOPCAT. Unfortunately it is distributed as the default java
in some Linux distributions.
java -version" should enable you
to work out if this is the case. If you are using GNU java, then
download a JRE from java.sun.com
and make sure the
java from that is first on your path.
Apparently on Debian and Ubuntu, you can install Sun's JRE directly
simply by doing "
sudo apt-get install sun-java6-jre".
Other distributions may have their own shortcuts.
If you have a Mac, see also Q4.3.
If neither of those seems to be the problem, please mail me and ask for help.
Q3.1 What table format should I use?
The most efficient table format to use is FITS (either basic or "FITS-plus" - see Q3.2) specifically FITS BINTABLE extensions in uncompressed files on disk. If your tables are large, using FITS rather than some other format will both save memory (enabling you to work with larger datasets) and speed up table loading. If your data is initially in some other format (e.g. an ASCII-based one) it is often a good idea to convert once to FITS and subsequently use the FITS file within TOPCAT.
If your tables are small then the efficiency doesn't matter so much. VOTable and FITS-plus, and to a lesser extent basic FITS, can store rich metadata (units, UCDs, column descriptions etc) which may be something you want.
If you want to do processing of a table in some other language such as IDL or Python or Perl, a number of ASCII-based formats are available. Of these, CSV (Comma-Separated Values) is usually the best choice - the others can sometimes truncate long string values.
Q3.2 What's FITS-plus?
TOPCAT can read and write two variants of FITS binary tables, "FITS-basic" and "FITS-plus" (these are just terms used by STIL/TOPCAT to distinguish the two variants). Both of these are perfectly legal FITS tables, but TOPCAT squirrels away some extra metadata in the primary HDU when using FITS-plus. In most cases, TOPCAT writes FITS-plus by default, and reads either. Nearly all of the time, you shouldn't need to worry about the difference.
There is more detailed information in the manual.
Q3.3 How do I read Multi-Extension FITS files?
If you load a FITS file, any table extensions it contains (BINTABLE or TABLE extensions) will be loaded as separate tables by default.
Q3.4 How do I read multi-table VOTable documents?
If you load a VOTable document, any tables it contains (<TABLE> elements) will be loaded as separate tables by default.
Q3.5 What size tables can TOPCAT cope with?
This depends a bit on one thing and another, but as a rule of thumb a few million rows and a few tens of columns should be quite usable while maintaining quite good performance. If you're having trouble with much smaller tables than this, there is almost certainly something you can do about it. For more details see Q3.6.
Larger tables are not out of the question; there are reports of interactive use with 300 million rows, and 20 thousand columns (not at the same time).
Q3.6 Help! I've run out of memory.
A lot of effort has gone into making TOPCAT scalable. Some data sets may simply be too large to process, but up to a few million rows it should be possible to use the program with good performance and without running out of resources. You may have to take some steps to ensure that this is so however. Some good tips are:
-diskflag (this is less important since v3.5).
Some operations which would run out of memory in TOPCAT can be done using STILTS, since it can stream data in some circumstances.
Q3.7 Can I save a Row Subset?
Since version 3.6 (August 2010), the Session save option in the Save Window can be used to save your entire session (for some or all of the loaded tables). Unlike the Current Table save option, this saves things like lists of subsets, hidden columns, etc, in addition to the usual table data and metadata. It can be used to save a single table or several in the same file. To reload the session data, simply reload the saved file into TOPCAT like a normal table. Note however that this saves information about which rows are in which subsets as flags, rather than the rule for subset inclusion (such as an algebraic expression).
In previous versions, the only way you could do this was to create a new boolean column representing the subset(s) to save, and save the table with that column. You could create the new column from the Subset Window using the Subsets|To Column menu item or equivalent toolbar button; this column then contained true/false according to whether each row is/is not in the subset in question. When the table was reloaded into TOPCAT, new subsets will be created for any such boolean columns.
Q3.8 TOPCAT won't read my VOTable
It is possible that there is a bug in TOPCAT, but it's
much more likely that there is an error in the VOTable.
The best thing to do in this case is to diagnose the problem by using
tool. Do not be afraid, this is very easy to do, something like:
% stilts votlint foo.xmlor
% topcat -stilts votlint foo.xmlThis will print WARNING or ERROR messages to the terminal describing questionable or illegal aspects of the file. You may be able to see what's wrong, and possibly fix it with careful use of a text editor from this information. Either way, if there are errors it would be a good idea to contact the originator of the VOTable with the
votlintoutput and tell them that they are generating broken VOTables.
Q3.9 Where has my VOTable metadata gone?
TOPCAT's data access is format-neutral - it has its own model of what constitutes table data and metadata rather than being tied to a particular data format. This is good in that it means it can work with tables in various different formats. However, it also means that data/metadata structure which is very specific to a particular data format may not be accommodated. Things like VOTable <GROUP> and <RESOURCE> elements fall into this category.
To put it another way, both load and save operations in TOPCAT result in a sort of format translation (to and from TOPCAT's internal data model). Some things can get lost or changed in both of these stages, so unfortunately loading a table into TOPCAT and saving it without making any explicit changes can result in loss of data.
To put it a third way, TOPCAT is (amongst other things) a table editor, but if you think of it as a VOTable editor you may be disappointed.
Q3.10 Can I indicate missing upper/lower limits when plotting error bars?
There is no specific facility for displaying missing limits, but you can probably generate a plot that looks like you want it to. You can represent missing limits using the error bar style which resembles little arrow heads. Set up two subsets in the plot, one for points with error limits and one for points without. You can set up two data sets in the plot with the same point data but different error data, one containing the subset of points with limits and one containing the points without. For the points without limits, provide fake error values and use a suitable error plotting style, such as the error bars with the little arrow heads.
Q3.11 How can I join two tables with the same row numbers side by side?
TOPCAT's crossmatching facilities are very smart and very flexible.
So much so that they can do stupid, easy things as well as clever, hard ones.
If you have tables that want joining together without reordering the
rows, choose the Exact Value algorithm and use the
magic value "
$0" for the Matched Value Column
for each input table.
$0 evaluates to the row index in
each case, so that all the row 1's will be joined together, as will all
the row 2's etc.
In STILTS it's easier, there is a command that does just this:
Q3.12 The crossmatching is giving the wrong results!
The crossmatching code in TOPCAT/STILTS is quite well tested, and reports of actual bugs are rare. However, I take the correctness seriously, and if you think you've found an anomaly, please report it.
The following bugs affecting crossmatching have been reported in recent years:
<n>d_err/ <n>-d Cartesian with Errors
skyerr/ Sky with Errors
2d_ellipse/ 2-d Cartesian Ellipses
skyellipse/ Sky Ellipse
No other crossmatching bug has been reported in recent years.
One thing which can be confusing is matching objects in crowded regions, since in crowded regions (where the typical distance between positions is similar to or less than the match radius) the results depend strongly on the details of the matching criteria.
Note in particular that TOPCAT's "Best Match Symmetric" mode
is symmetric between tables.
If you select "Best Match Symmetric" for the Match Selection toggle
(or equivalently "
find=best" in STILTS
tasks), then each row from either table will only appear a maximum
of once in the output table. That means if object A in table 1 is a
match for object B in table 2, it cannot be a match for any other objects
in table 2. This may lead to fewer matches than you were expecting.
If what you want to do is to find, for each object in table 1, the
closest match in table 2, you should use one of the asymmetric options:
"Best match for each Table 1 row" (or "Table 2 row") in TOPCAT's
Match Selection chooser, or "
find=best1 (or "
Note that these options were only introduced in TOPCAT v3.9/STILTS v2.4,
prior to that only symmetric best and all options were available.
So, if you have an earlier version and want to do this, you should upgrade.
If you've got doubts about the correctness of a crossmatch, it's a very good idea to plot the results to see whether it's doing what you think or hope it is. Since v4.1 this is usually fairly easy; see the manual.
Q3.13 Can I crossmatch more than 2 tables at once?
Yes. The multi-table crossmatch options are available from the Joins menu in the control menu. They don't appear on the toolbar just because they would take up too much space. Currently a 4-way match is the most you can do at once. But that's only because nobody has asked me for more. If you want the option to match more than that, let me know and I will add higher-numbered options.
Somewhat more flexible multi-table crossmatches are available
using the STILTS
Equivalent functionality will be added to TOPCAT at some point.
Q3.14 Can I concatenate more than 2 tables at once?
No. You can do this using the STILTS
Q3.15 Can TOPCAT read SExtractor output?
It cannot read the ASCII files that SExtractor outputs by default. If you want to use TOPCAT with the output from SExtractor, set it to write its output as FITS tables. You can do this by setting SExtractor's CATALOG_TYPE parameter to "FITS_1.0".
Q3.16 Can I script TOPCAT?
No. The only ways you can control TOPCAT itself are
However TOPCAT's sister application STILTS is a set of command-line tools which provide much of the same functionality as TOPCAT. You can write scripts using these applications if you want to automate the kind of table processing that TOPCAT can do.
Q3.17 Can I save/restore the state of a TOPCAT session?
Since version 3.6 (August 2010), yes you can. Use the Session option in the Save Window to save, and load the table that outputs in the normal way to restore.
Note however that this saves (most of) per-table state, and not everything, so for instance currently displayed plots will be lost. This may be improved in a future release.
Q3.18 Can I see a log of the actions TOPCAT has performed during a session?
No, I'm afraid not.
Q3.19 Can TOPCAT talk to other tools/processes?
Yes, by using SAMP or PLASTIC.
Q3.20 What's SAMP and/or PLASTIC all about then?
SAMP is the Simple Application Messaging Protocol.
It allows different tools, usually running on the same desktop, to talk to each other, for instance by sending tables from one to another. This topic is discussed in more detail in the manual.
Briefly, if a SAMP "hub" is running then some of TOPCAT's controls allow you to send instructions or data to other tools. Usually these options are controlled from an Interop menu on the relevant window.
A SAMP hub may already be running as part of another application (such as Aladin). If it is not, TOPCAT usually starts one internally when it starts up.
PLASTIC is the PLatform for AStronomical Tool InterConnection; it was an earlier protocol that did effectively the same thing as SAMP, but is now more or less obsolete.
Q3.21 Can I select the image viewer for TOPCAT to send images to?
Currently it is possible to choose the image viewer only for the View URL as Image option in the Activation Window by using the Image Viewer selector in that panel - SAMP/PLASTIC and non-SAMP/PLASTIC options are available. You cannot currently select the image viewer for the Display Cutout Image option or else where in TOPCAT, a non-SAMP/PLASTIC viewer is chosen automatically.
It is hoped that this situation will be rationalised and improved in a future version.
Q3.22 Can I calculate the difference between column values in adjacent rows?
The expression language used by TOPCAT works entirely within a single row, so doing things like adding a new column B whose value in row n is the difference between the values of another column A in row n and its value in row n+1 is not directly possible.
There is a hack which you can use to achieve this however.
If you join (match) the table to itself so that the same column appears
twice in the resulting table, but offset by one row, you can then
do calculations involving both values. The join can be done
in the Pair Match Window,
by using the Exact Value match algorithm,
using the same table for both inputs, with the
Matched Value Column
set to "
index" and "
index" (or equivalently "
$0") token is
a magic column which evaluates to the row number in the table.
You can then add a column B with an expression like "
to get the difference. If required, you could then match the part of the
table containing the new column back to the original table.
Yes this is a bit of an effort. Here is a STILTS command which effectively does what is described above:
stilts tmatch2 in1=table.fits in2=table.fits \ matcher=exact values1=index+1 values2=index join=1or2 \ ocmd='addcol b a_2-a_1'
Q4.1 Overwriting mapped FITS file causes Java crash
If a FITS file has been mapped for reading (this is the normal way that uncompressed FITS files on disk are read) and it is overwritten, either by TOPCAT or by some other program, it can result in a crash of the Java Virtual Machine (ugly error message to the console and instant death of TOPCAT). TOPCAT will not normally perform such an overwrite, but it can't stop you doing it from outside the program.
The best thing is: don't overwrite FITS files currently in use.
However, you can work around this by deleting the file before you overwrite it,
or by loading it into TOPCAT using a
file:-type URL, e.g
file://localhost/foo/bar/cat.fits" rather than
/foo/bar/cat.fits". This will load without mapping
(and as a result will be much slower for large files).
Q4.2 Expressions containing self-references cause trouble
If an algebraic expression for a synthetic column or row subset
contains a reference to itself via a
the program may hang. It might recover - if you can edit the expression
so it doesn't contain such a reference any more, do so.
In any case, try to avoid doing this.
Q4.3 On MacOS Mountain Lion the .dmg file is reported "damaged" or "can't be opened"
On the Mountain Lion release of Mac OS X, attempting to install the topcat-full.dmg file apparently can result in the message:
"TOPCAT is damaged and can't be opened, you should eject the disk image"or possibly (not so far reported, but looks like it might happen):
"TOPCAT can't be opened because it is from an unidentified developer"
It seems that this is in Mountain Lion and later, security settings (a.k.a. GateKeeper) default to not letting you run anything from anywhere other than the Mac App Store or from signed applications. Unfortunately, this results in it reporting java-based non-signed applications as "damaged".
There are two workarounds for this. One is to eliminate the quarantine flag for the file from the command line. If TOPCAT is in the current path, you can type
xattr -d com.apple.quarantine TOPCAT.app
Alternatively, you can (perhaps temporarily) change one of the security settings by going to the Apple menu | "System Preferences..." | "Personal" section | "Security & Privacy" | "General" pane' and changing the setting Allow applications downloaded from: to Anywhere. You can run TOPCAT, and a dialog will be shown telling you the application is not signed. Once you accept it, you can revert, if you want, your security settings to the default setting, Allow applications downloaded from: Mac App Store and identified developers. Here is a more detailed explanation (for a different application, but you get the idea).
I may be able to fix this one day if I can work out how to sign an
OS X application (I think I have a respectable certificate, but don't
know how to use
codesign). Offers of help welcome.
Thanks to Juan de Dios Santander Vela for help with this question.
Q4.4 Tables are scrambled when scrolling right
In some cases apparently the table viewer window and other JTables displayed can get their pixels scrambled whenever you scroll it to the right. This seems to be some kind of bug in Java or maybe some display drivers - it's discussed for instance here. To fix it try running with one or other of these system properties set:
Q5.1 I can't find a resource in the NVO/VAO registry from the Cone/SIA/SSA/TAP window
Between mid-2010 and mid-2014 searching the NVO (a.k.a. VAO) registry at STScI did not work. This turned out to be the result of bugs in both TOPCAT and the STScI registry, as finally uncovered by the estimable Menelaus Perdikeas. In version 4.1 and later, these searches should work. In version 4.2 and later, the RegTAP protocol is used by default, and the problem ought not to arise.
Q5.2 It crashes immediately on startup, or when I open the Load window
There is a bug which may be seen by people who are set up to use Starlink applications as well as TOPCAT. It may provoke messages like:
no such constant AST__KYCIRor
# # An unexpected error has been detected by HotSpot Virtual Machine: # # SIGSEGV (0xb) at pc=0xf79af28b, pid=5153, tid=3888896928
You should be able to fix it by unsetting the LD_LIBRARY_PATH environment variable, for instance if you use a C-like shell:
% unsetenv LD_LIBRARY_PATHor for bash/sh:
$ unset LD_LIBRARY_PATH $ export LD_LIBRARY_PATHIf you continue to have problems, please contact me and I'll try to provide more help.
Q5.3 3D error bars extending outside the box are invisible
In the Cartesian/Spherical 3D plots, if an error bar would extend outside of the box/sphere then it is simply not plotted, so that even the part which is within the box/sphere is not visible.
Q5.4 Scisoft 7.0 invocation script broken
TOPCAT is included in the ESO
Scisoft software bundle.
topcat" invocation script included with that bundle
does not have the same functionality as the one documented in
so for instance command-line
arguments cannot be supplied to it.
If it's not doing what you want, you can run using
java -jar topcat-full.jar.
This problem should be fixed in Scisoft releases after 7.0.
Q5.5 I get an error about a SizingScrollPane on my Java 1.7 Mac
On some Macs running java 1.7 (Mountain Lion??), doing things like trying to open the table or statistics windows can give you a stack trace like:
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException at uk.ac.starlink.topcat.SizingScrollPane.getConfig(SizingScrollPane.java:81) at uk.ac.starlink.topcat.SizingScrollPane.getPreferredSize(SizingScrollPane.java:40) ...I can't tell if this is my fault or OSX's, but it is fixed in version 4.0 and later.
Q5.6 My plot axis labels come out backwards on OS X!
It seems that in recent versions (java 1.7?) of MacOS X,
the text written sideways up the Y axis of plots sometimes
comes out written backwards, e.g. it appears to read
EDUTINGAM_B" rather than "
In fact any text written non-horizontally is placed wrong
(e.g. sky axis labels).
I am pretty sure this is a Mac Java bug, which I hope will fix itself
at some point (does anybody know how to report this kind of thing?).
The new-style plotting windows in v4.1 can work around this issue (I believe) which seems to go away if you use antialiased text. As well as getting the text the right way round it does look a bit nicer like this, but it is also perceptibly slower. TOPCAT attempts to determine if its running in MacOS, and if so it sets antialiasing on by default. However at version 4.1, it doesn't always guess right. You can turn antialiasing on/off by using the Antialias checkbox in the Grid tab of the Axes control in the various plot windows, so try that if your labels are backwards.
In version 4.2 and later, text antialiasing will be set using the Antialias text type in the Font selector instead. By then, it ought to be better at guessing OS, so you hopefully will not need to worry about it.
If you're using the old-style plot windows or an older topcat version, running on java 1.6 seems to make it work OK. Although apparently Oracle has removed the ability to select Apple's Java 1.6 from it's Java preference pane, you may be able to launch topcat using Java 1.6 by typing:
export JAVA_HOME=`/usr/libexec/java_home -v 1.6` java -jar /path/to/topcat-full.jar
Thanks to Juan de Dios Santander Vela for help with this question.
Q5.7 When I load an LDAC FITS file I can't see my data
Files in the LDAC FITS format are organised in an unusual way with the first (LDAC_IMHEAD) table extension containing a single-cell table where the cell's content is an array of FITS headers. This confuses TOPCAT into thinking that the file is in colfits format, which in turn means it ignores the second (LDAC_OBJECTS) table extension, and that's the one you probably want to look at.
A hack is present in TOPCAT v4.2-2 and later to avoid this,
but there are various ways to work around it anyway.
One is to select "
FITS" (rather than "
as the input file format.
Another is to use the
to examine the file and select the table you want.
You can also select the correct table by name or HDU index as, e.g.,
D1_r_sex.cat#2", either on the command line or
using the Position in file field at the bottom right of the
Q6.1 I still don't know...
Try consulting the manual. Or at least the relevant part - click the Help button in the window you're in for some context-specific documentation.
Q6.2 It's not in/I'm too lazy to read the manual.
The best thing is to subscribe to, then send a message to, the topcat-user mailing list. I am usually monitoring this, and you will probably get a reply quite soon.
If for some reason you don't want to do that, then by all means contact me by mail. I'd rather answer questions on usage than have people assume something doesn't work when it does.