uk.ac.starlink.parquet

Class ParquetDump

java.lang.Object

uk.ac.starlink.parquet.ParquetDump

public class ParquetDump
extends java.lang.Object

Simple utility that can display metadata information from a parquet file to standard output.

This can be invoked using the main(java.lang.String[]) method, which provides a -h option for help. Or its functionality can be wrapped by a more capable utility.

Since:

8 Jan 2024

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ParquetDump.ChunkMeta Bean adaptor class that wraps a ColumnChunkMetaData object to provide BeanStarTable-friendly methods (string- or primitive-valued accessors) that report interesting information about a ColumnChunkMetaData object.

Constructor Summary

Constructors

Constructor and Description
ParquetDump(ParquetStarTable starTable) Constructor.

Method Summary

Modifier and Type	Method and Description
static java.util.Map<java.lang.String,java.util.function.Function<ParquetDump,java.lang.String>>	createDumpFunctionMap() Returns a map of the output functions offered by this utility.
java.lang.String	formatBlocks() Returns a multi-line string showing information about the data blocks in the parquet file.
java.lang.String	formatColumnChunks() Returns a multi-line string showing information about the column chunks in each data block.
java.lang.String	formatKeyValuesCompact(int maxChar) Returns a multi-line string giving an abbreviated representation of the content of the file-level key-value metadata in the parquet footer.
java.lang.String	formatKeyValuesFull() Returns a multi-line string giving the full content of the file-level key-value metadata in the parquet footer.
static java.lang.String	formatLines(java.lang.String heading, java.lang.String lines) Formats a multi-line string with a supplied heading.
java.lang.String	formatSchema() Returns a multi-line string showing the parquet schema.
ParquetStarTable	getTable() Returns the table on which this object can report.
static void	main(java.lang.String[] args) Main method.
static ParquetStarTable	readParquetTable(java.lang.String location) Reads a parquet table from a filename or URL, suitable for use with this utility.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ParquetDump

public ParquetDump(ParquetStarTable starTable)
            throws java.io.IOException

Constructor.

Parameters:

starTable - parquet star table

Throws:

java.io.IOException

Method Detail

getTable

public ParquetStarTable getTable()

Returns the table on which this object can report.

Returns:

parquet table

formatSchema

public java.lang.String formatSchema()

Returns a multi-line string showing the parquet schema.

Returns:

schema string

formatKeyValuesCompact

public java.lang.String formatKeyValuesCompact(int maxChar)

Returns a multi-line string giving an abbreviated representation of the content of the file-level key-value metadata in the parquet footer. If any of the values is longer than the supplied maxChar threshold, or if it contains embedded newlines, it is just represented by a summary of its length.

Parameters:

maxChar - maximum length of displayed value

Returns:

key-value string

formatKeyValuesFull

public java.lang.String formatKeyValuesFull()

Returns a multi-line string giving the full content of the file-level key-value metadata in the parquet footer.

Returns:

key-value string

formatBlocks

public java.lang.String formatBlocks()

Returns a multi-line string showing information about the data blocks in the parquet file.

Returns:

block info string

formatColumnChunks

public java.lang.String formatColumnChunks()

Returns a multi-line string showing information about the column chunks in each data block.

Returns:

column-chunk string

formatLines

public static java.lang.String formatLines(java.lang.String heading,
                                           java.lang.String lines)

Formats a multi-line string with a supplied heading.

Parameters:

heading - heading text

lines - multi-line string giving text

createDumpFunctionMap

public static java.util.Map<java.lang.String,java.util.function.Function<ParquetDump,java.lang.String>> createDumpFunctionMap()

Returns a map of the output functions offered by this utility.

Returns:

map with keys that are function labels, and values that map a dump instance to a multiline output string

readParquetTable

public static ParquetStarTable readParquetTable(java.lang.String location)
                                         throws java.io.IOException

Reads a parquet table from a filename or URL, suitable for use with this utility. Columns unsupported by STIL are included.

Parameters:

location - filename or URL

Returns:

parquet table

Throws:

java.io.IOException

main

public static void main(java.lang.String[] args)
                 throws java.io.IOException

Main method. Use -help.

Throws:

java.io.IOException