Package org.yamcs.yarch

Class TableDefinition

java.lang.Object
org.yamcs.yarch.TableDefinition
public class TableDefinition extends Object
A table definition consists of a (key,value) pair of tuple definitions. A tuple has to contain all the columns from the key while it can contain only a few of the columns from the value (basically it's a sparse table).

The key is encoded as a byte array of all the columns in order. The value is encoded as a byte array of all the columns preceded by the id of their data type (1 byte) and their index (3 bytes).

The secondary index key is encoded as a byte array of all the columns in order preceded by the id of their data type with the first bit set to 1 for the columns present and 0 for the column not present (i.e. null).

A table can also be partitioned according to the partitioningSpec.

  • Field Details

    • CURRENT_FORMAT_VERSION

      public static final int CURRENT_FORMAT_VERSION
      See Also:

  • Constructor Details

  • Method Details

    • setDb

      public void setDb(YarchDatabaseInstance ydb)

    • setPartitioningSpec

      public void setPartitioningSpec(PartitioningSpec pspec) throws StreamSqlException
      time based partitions can be on the first column of the key (which has to be of type timestamp) value based partitions can be on any other mandatory column
      Parameters:
      pspec -
      Throws:
      StreamSqlException

    • getKeyDefinition

      public List<TableColumnDefinition> getKeyDefinition()

    • getValueDefinition

      public List<TableColumnDefinition> getValueDefinition()

    • getName

      public String getName()

    • setName

      public void setName(String name)

    • getTupleDefinition

      public TupleDefinition getTupleDefinition()

    • validate

      public void validate() throws StreamSqlException
      Checks that the table definition is valid: - primary key not string, except for the last in the list (otherwise the binary sorting does not work properly)
      Throws:
      StreamSqlException

    • generateRow

      public Row generateRow(Tuple t) throws YarchException
      Generate a new table row by transforming the key part of the tuple into a byte array to be written to disk. The tuple must contain each column from the key and they are written in order (such that sorting is according to the definition of the primary key).

      In addition, it stores into the returned row all the values for the columns used in histograms or indices

      Parameters:
      t -
      Returns:
      a tuple containing the histogram and secondary index values as well as the generated key
      Throws:
      YarchException

    • addAndGetEnumValue

      public Short addAndGetEnumValue(String columnName, String value)
      get the enum value corresponding to a column, creating it if it does not exist
      Returns:

    • serializeValue

      public void serializeValue(Tuple tuple, Row sertuple, ByteArray byteArray)
      Same as serializeValue(Tuple, Row) but encodes the output in user provided byte array
      Parameters:
      tuple -
      sertuple -
      byteArray -

    • serializeValue

      public byte[] serializeValue(Tuple tuple, Row sertuple)
      Transform the value part of the tuple into a byte array to be written on disk. Each column is preceded by a tag (the column index).

      If there are columns in the tuple which are not in the valueDef, they are added and the TableDefinition is serialized on disk.

      Columns whose values are null are not serialized but their definition is still added to the table definition if not present already.

      Parameters:
      tuple -
      sertuple - - if not null, store all the values of the columns to this tuple as written to the database (possibly after some data casting)
      Returns:
      the serialized version of the value part of the tuple

    • deserialize

      public Tuple deserialize(byte[] k, byte[] v)

    • isCompressed

      public boolean isCompressed()

    • isIndexedByKey

      public boolean isIndexedByKey(String cname)
      Parameters:
      cname - the column name
      Returns:
      true if cname is the first column of the key

    • getColumnDefinition

      public TableColumnDefinition getColumnDefinition(String colName)

    • hasPartitioning

      public boolean hasPartitioning()

    • getPartitioningSpec

      public PartitioningSpec getPartitioningSpec()

    • setCompressed

      public void setCompressed(boolean compressed)

    • setHistogramColumns

      public void setHistogramColumns(List<String> histoColumns) throws StreamSqlException
      Throws:
      StreamSqlException

    • setSecondaryIndex

      public void setSecondaryIndex(List<String> index) throws StreamSqlException
      Throws:
      StreamSqlException

    • hasHistogram

      public boolean hasHistogram()

    • getEnumValues

      public com.google.common.collect.BiMap<String,Short> getEnumValues(String columnName)

    • getHistogramColumns

      public List<String> getHistogramColumns()

    • getSecondaryIndex

      public List<String> getSecondaryIndex()

    • getColumnSerializer

      public <T> ColumnSerializer<T> getColumnSerializer(String columnName)

    • getStorageEngineName

      public String getStorageEngineName()

    • setStorageEngineName

      public void setStorageEngineName(String storageEngineName)

    • getFormatVersion

      public int getFormatVersion()

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • changeDataType

      public void changeDataType(String cname, DataType dataType)
      this method is used during table migration between two versions

      It should not be used for other things.

    • hasKey

      public boolean hasKey(String colName)
      Parameters:
      colName -
      Returns:
      true if the column is part of the primary key

    • isPartitionedByTime

      public boolean isPartitionedByTime()

    • getHistoIdx

      public IndexedList<String,TableColumnDefinition> getHistoIdx()

    • hasSecondaryIndex

      public boolean hasSecondaryIndex()

    • isPartitionedBy

      public boolean isPartitionedBy(String colName)
      Return true if the table is partitioned and the colName is used as partition column (either time or value)
      Parameters:
      colName -
      Returns: