Package org.yamcs.yarch

Class PartitionManager

java.lang.Object
org.yamcs.yarch.PartitionManager
Direct Known Subclasses:
RdbPartitionManager
public abstract class PartitionManager extends Object
Keeps track of partitions and histograms for one table.

The partitioning is organised in a set of time based partitioning intervals, each interval being one day/month/year long, depending on the partitioning schema.

Each interval has associated a set of value based partitions.

In addition each interval has a set of histograms - one for each table column for which histograms have been created.

In the RocksDB implementation (both old and new) each interval corresponds to one rocksdb database directory.

In case there is no time based partitioning, there is only one interval.

In case there is no value based partitioning, there is only one partition in each interval.

  • Field Details

  • Constructor Details

    • PartitionManager

      public PartitionManager(TableDefinition tableDefinition)

  • Method Details

    • iterator

      public Iterator<PartitionManager.Interval> iterator(Set<Object> partitionValueFilter)
      Returns an iterator which at each step gives the list of partition corresponding to a time interval (so when we do a replay those partitions have to be played in parallel). The iterator returns intervals sorted on time.
      Parameters:
      partitionValueFilter - - return only partitions whose value are in the filter. If null, return all partitions;
      Returns:
      iterator going over partitions

    • reverseIterator

      public Iterator<PartitionManager.Interval> reverseIterator(Set<Object> partitionValueFilter)
      same as above, only in reverse direction
      Parameters:
      partitionValueFilter -
      Returns:

    • iterator

      public Iterator<PartitionManager.Interval> iterator(long start, Set<Object> partitionValueFilter)
      See iterator(Set)
      Parameters:
      start -
      partitionValueFilter - values - return only partitions whose value are in the filter. If null, return all partitions;
      Returns:
      an iterator over the partitions starting at the specified start time

    • intervalIterator

      public Iterator<PartitionManager.Interval> intervalIterator(TimeInterval timeInterval)
      Iterates over all intervals overlapping with the timeInterval. Note that the timeInterval is considered closed at both ends (if set): [timeInterval.start, timeInterval.stop] whereas the partition intervals are considered closed at start and open at stop: [Interval.start, Interval.stop)

    • reverseIterator

      public Iterator<PartitionManager.Interval> reverseIterator(long start, Set<Object> partitionValueFilter)

    • createAndGetPartition

      public Partition createAndGetPartition(long instant, Object value) throws IOException
      Creates (if not already existing) and returns the partition in which the instant,value should be written. value can be null (in case of no value partitioning)
      Parameters:
      instant - - time for which the partition has to be created - can be TimeEncoding.INVALID in case value only or no partitioning
      value - - value for which the partition has to be created - can be null in case of time only or no partitioning. For the enum partitions, the value is the index (type Short) rather than the string.
      Returns:
      a Partition
      Throws:
      IOException

    • createAndGetHistogram

      public HistogramInfo createAndGetHistogram(long instant, String columnName)

    • getPartitionForTuple

      public Partition getPartitionForTuple(Tuple t) throws IOException
      Gets partition where tuple has to be written. Creates the partition if necessary.
      Parameters:
      t -
      Returns:
      the partition where the tuple has to be written
      Throws:
      IOException

    • getTableName

      public String getTableName()
      Get the name of the table whose partitions are managed by this object.
      Returns:
      the name of the table.

    • getTableDefinition

      public TableDefinition getTableDefinition()

    • getPartitioningSpec

      public PartitioningSpec getPartitioningSpec()

    • createPartitionByTime

      protected abstract Partition createPartitionByTime(TimePartitionInfo pinfo, Object value) throws IOException
      Create a partition for time (and possible value) based partitioning
      Parameters:
      pinfo -
      value -
      Returns:
      Throws:
      IOException

    • createPartition

      protected abstract Partition createPartition(Object value)
      Create a partition for value based partitioning
      Parameters:
      value -
      Returns:

    • createHistogramByTime

      protected abstract HistogramInfo createHistogramByTime(TimePartitionInfo pinfo, String columnName)

    • createHistogram

      protected abstract HistogramInfo createHistogram(String columnName)

    • getPartitions

      public List<Partition> getPartitions()
      Retrieves the existing partitions
      Returns:
      list of all existing partitions