uk.ac.starlink.parquet

Class ParquetStarTable

java.lang.Object

uk.ac.starlink.table.AbstractStarTable

uk.ac.starlink.parquet.ParquetStarTable

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, StarTable

Direct Known Subclasses:

CachedParquetStarTable, SequentialParquetStarTable

public abstract class ParquetStarTable
extends AbstractStarTable

Partial StarTable implementation based on a Parquet file. This class provides metadata handling and preparation for data reading, but does not implement actual data access methods.

Since:

25 Feb 2021

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	ParquetStarTable.Config Defines configuration options for reading parquet tables.

Field Summary

Fields

Modifier and Type	Field and Description
static ValueInfo	CREATEDBY_INFO Parameter metadata for parquet creation info.
static java.lang.String	NAME_KEY Extra metadata key for table name.
static java.lang.String	REQUIRED_VOTMETAVERSION Required value for VOTable metadata version.
static ValueInfo	UNSUPPORTED_INFO Column metadata for dummy/unsupported columns.
static java.lang.String	VOTMETA_KEY Extra metadata key for skeleton VOTable text.
static java.lang.String	VOTMETA_NAMESPACE Namespace used in metadata keys for VOParquet convention things.
static java.lang.String	VOTMETAVERSION_KEY Extra metadata key for VOTable metadata version.

Constructor Summary

Constructors

Constructor and Description
ParquetStarTable(IOSupplier<ParquetFileReader> pfrSupplier, ParquetStarTable.Config config) Constructor.

Method Summary

Modifier and Type	Method and Description
int	getColumnCount() Returns the number of columns in this table.
ColumnInfo	getColumnInfo(int icol) Returns the object describing the data in a given column.
ColumnReadStore	getColumnReadStore(PageReadStore pageStore, MessageType schema) Gets a ColumnReadStore from a PageReadStore.
java.util.Map<java.lang.String,java.lang.String>	getExtraMetadataMap() Returns the parquet key-value file-level metadata map associated with this parquet file.
InputColumn<?>	getInputColumn(int icol) Returns the input column that can be used to read a given column of this table.
ParquetFileReader	getParquetFileReader() Returns a reader for this table's underlying file.
long	getRowCount() Returns the number of rows in this table, if known.
MessageType	getSchema() Returns the schema of the the parquet file.
java.lang.String	getVOTableMetadataText() Returns the text content of a VOTable intended to supply metadata for this table.
void	setVOTableMetadataText(java.lang.String votmeta) Sets the text content of a VOTable intended to supply metadata for this table.

Methods inherited from class uk.ac.starlink.table.AbstractStarTable

checkedLongToInt, close, getCell, getColumnAuxDataInfos, getName, getParameters, getRow, getRowAccess, getRowSequence, getRowSplittable, getURL, isRandom, setName, setParameters, setURL

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface uk.ac.starlink.table.StarTable

getParameterByName, setParameter

Field Detail

CREATEDBY_INFO

public static final ValueInfo CREATEDBY_INFO

Parameter metadata for parquet creation info.

UNSUPPORTED_INFO

public static final ValueInfo UNSUPPORTED_INFO

Column metadata for dummy/unsupported columns.

NAME_KEY

public static final java.lang.String NAME_KEY

Extra metadata key for table name.

See Also:

Constant Field Values

VOTMETA_NAMESPACE

public static final java.lang.String VOTMETA_NAMESPACE

Namespace used in metadata keys for VOParquet convention things.

See Also:

Constant Field Values

VOTMETA_KEY

public static final java.lang.String VOTMETA_KEY

Extra metadata key for skeleton VOTable text.

See Also:

Constant Field Values

VOTMETAVERSION_KEY

public static final java.lang.String VOTMETAVERSION_KEY

Extra metadata key for VOTable metadata version.

See Also:

Constant Field Values

REQUIRED_VOTMETAVERSION

public static final java.lang.String REQUIRED_VOTMETAVERSION

Required value for VOTable metadata version.

See Also:

Constant Field Values

Constructor Detail

ParquetStarTable

public ParquetStarTable(IOSupplier<ParquetFileReader> pfrSupplier,
                        ParquetStarTable.Config config)
                 throws java.io.IOException

Constructor.

Parameters:

pfrSupplier - access to parquet data file

config - table reading options

Throws:

java.io.IOException

Method Detail

getColumnCount

public int getColumnCount()

Description copied from interface: StarTable

Returns the number of columns in this table.

Specified by:

getColumnCount in interface StarTable

Specified by:

getColumnCount in class AbstractStarTable

Returns:

the number of columns

getRowCount

public long getRowCount()

Description copied from interface: StarTable

Returns the number of rows in this table, if known. If the number of rows cannot be (easily) determined, a value of -1 will be returned.

Specified by:

getRowCount in interface StarTable

Specified by:

getRowCount in class AbstractStarTable

Returns:

the number of rows, or -1

getColumnInfo

public ColumnInfo getColumnInfo(int icol)

Description copied from interface: StarTable

Returns the object describing the data in a given column.

Specified by:

getColumnInfo in interface StarTable

Specified by:

getColumnInfo in class AbstractStarTable

Parameters:

icol - the column for which header information is required

Returns:

a ValueInfo object for column icol

getParquetFileReader

public ParquetFileReader getParquetFileReader()
                                       throws java.io.IOException

Returns a reader for this table's underlying file.

Returns:

parquet file reader

Throws:

java.io.IOException

getSchema

public MessageType getSchema()

Returns the schema of the the parquet file.

Returns:

parquet schema

getExtraMetadataMap

public java.util.Map<java.lang.String,java.lang.String> getExtraMetadataMap()

Returns the parquet key-value file-level metadata map associated with this parquet file.

Returns:

extra metadata

getVOTableMetadataText

public java.lang.String getVOTableMetadataText()

Returns the text content of a VOTable intended to supply metadata for this table.

Returns:

metadata VOTable text, or null

setVOTableMetadataText

public void setVOTableMetadataText(java.lang.String votmeta)

Sets the text content of a VOTable intended to supply metadata for this table. This is called at construction time using key-value metadata map in accordance with the VOParquet convention, but it may be called subsequently to modify or reset the associated VOTable.

Parameters:

votmeta - VOTable metadata document

getColumnReadStore

public ColumnReadStore getColumnReadStore(PageReadStore pageStore,
                                          MessageType schema)

Gets a ColumnReadStore from a PageReadStore.

Parameters:

pageStore - page store

schema - schema for required data; this may for instance define only a subset of available columns

Returns:

column store

getInputColumn

public InputColumn<?> getInputColumn(int icol)

Returns the input column that can be used to read a given column of this table.

Parameters:

icol - column index

Returns:

input column