Next Previous Up Contents
Next: I/O using SQL databases
Up: Table I/O
Previous: Mirage

3.6 Streamed Input

The generic table input mechanism described in Section 3.2 provides various ways of reading a table from inputs which can be read multiple times - e.g. files, URLs and DataSources. The reason for this is that in general reading a table may be a multiple-pass process; when reading a CSV table for instance the parser goes through it once to find out what kinds of values are in each column and a second time to populate the data structures. So in the general case it is not possible to read from an InputStream, which can only be read once from start to finish.

However, some of the input handlers only need one pass of the byte stream, and these may offer streamed reading. To make use of this facility, you must identify the TableBuilder corresponding to the format of data in the stream, and then use its streamStarTable method as follows:

    public StarTable streamVOTable( InputStream in ) throws IOException {
        RowStore rowStore = StoragePolicy.getDefaultPolicy().makeRowStore();
        new VOTableBuilder().streamStarTable( in, rowStore, "" );
        return rowStore.getStarTable();
    }

This requires some explanation. The streamStarTable method (if it works) goes through each row of the table it reads from the input stream and writes it to a TableSink. A TableSink is just defined as an object which can accept rows one at a time, and do anything it likes with them. A RowStore is a TableSink which stores each row somewhere or other in such a way that once it's got all the rows it can supply a StarTable object containing the data that it has accepted. So what we are doing here is to make sure the streaming goes somewhere we can pick up the results later in the form of a StarTable. Exactly how the rows are stored (probably in memory or on disk in some format or other) depends on the particular RowStore instance - here we have got a default one which depends on the system configuration, but you can supply your own (see StoragePolicy).


Next Previous Up Contents
Next: I/O using SQL databases
Up: Table I/O
Previous: Mirage

STIL - Starlink Tables Infrastructure Library
Starlink User Note 252
STIL web page: http://www.starlink.ac.uk/stil/
Author email: m.b.taylor@bristol.ac.uk
Starlink: http://www.starlink.ac.uk/