uk.ac.starlink.ttools.cone

Class BlockUploader

java.lang.Object

uk.ac.starlink.ttools.cone.BlockUploader

public class BlockUploader
extends Object

Works with an UploadMatcher dividing the input table into chunks and uploading them separately to produce an arbitrarily large result while each upload/match operation is of a limited size.

Since:

15 May 2014

Author:

Mark Taylor

Constructor Summary

Constructors

Constructor and Description
BlockUploader(UploadMatcher umatcher, int blocksize, long maxrec, String outName, JoinFixAction uploadFixAct, JoinFixAction remoteFixAct, ServiceFindMode serviceMode, boolean oneToOne, boolean uploadEmpty) Constructor.

Method Summary

Modifier and Type	Method and Description
StarTable	runMatch(StarTable inTable, QuerySequenceFactory qsFact, StoragePolicy storage) Performs an upload join in blocks.
void	setTruncationAdvice(String truncationAdvice) Sets a string that can be issued to the user as additional advice if there are truncations in block results.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlockUploader

public BlockUploader(UploadMatcher umatcher,
                     int blocksize,
                     long maxrec,
                     String outName,
                     JoinFixAction uploadFixAct,
                     JoinFixAction remoteFixAct,
                     ServiceFindMode serviceMode,
                     boolean oneToOne,
                     boolean uploadEmpty)

Constructor.

Parameters:

umatcher - upload matcher

blocksize - maximum number of rows per uploaded block

maxrec - maximum number of output rows accepted in total

outName - name of output table

uploadFixAct - name deduplication policy for upload table

remoteFixAct - name deduplication policy for remote table

serviceMode - upload match mode

oneToOne - true iff output rows match 1:1 with input rows

uploadEmpty - determines behaviour if there are no input rows: true means attempt the match anyway, false means throw an IOException

Method Detail

runMatch

public StarTable runMatch(StarTable inTable,
                          QuerySequenceFactory qsFact,
                          StoragePolicy storage)
                   throws IOException

Performs an upload join in blocks.

As currently implemented, the input table needs to have random access, and the output table has random access. It would be possible to stream on input and output to some extent, though each input chunk will still have to be random access.

The row indices given by ConeQueryRowSequence.getIndex calls associated with the supplied qsFact object must correspond to the row indices in the supplied inTable.

Parameters:

inTable - input table, must have random access

qsFact - object to generate positional queries when applied to a table

storage - storage policy for storing raw result table

Throws:

IOException

setTruncationAdvice

public void setTruncationAdvice(String truncationAdvice)

Sets a string that can be issued to the user as additional advice if there are truncations in block results.

Parameters:

truncationAdvice - user-readable advice string