Package org.yamcs.parameterarchive

Class ParameterArchive

java.lang.Object
com.google.common.util.concurrent.AbstractService
org.yamcs.AbstractYamcsService
org.yamcs.parameterarchive.ParameterArchive
All Implemented Interfaces:
com.google.common.util.concurrent.Service, YamcsService
public class ParameterArchive extends AbstractYamcsService
The parameter archive stores data in partitions(optional) -> intervals -> segments.

A partition covers one year/month/day and each partition has its own RocksDB database.

An interval covers 2^23 millisec (=~ 139 minutes) - so for any timestamp (Yamcs time) we know exactly in which interval it falls.

A segment covers at most maxSegmentSize samples for one parameter. The segments do not cover a fixed period of time; we use them to avoid intervals getting very large; usually (1Hz or less frequency data) there is only one segment in an interval.

Segments cannot span across intervals.

When new data has been received in the past, the whole interval has to be re-created (by doing a replay); that likely means a new split of the respective interval into segments.

  • Field Details

  • Constructor Details

    • ParameterArchive

      public ParameterArchive()

  • Method Details

    • getSpec

      public Spec getSpec()
      Description copied from interface: YamcsService
      Returns the valid configuration options for this service.
      Returns:
      the argument specification, or null if the args should not be validated.

    • init

      public void init(String yamcsInstance, String serviceName, YConfiguration config) throws InitException
      Description copied from interface: YamcsService
      Initialize this service. This is called before the service is started. All operations should finish fast.
      Specified by:
      init in interface YamcsService
      Overrides:
      init in class AbstractYamcsService
      Parameters:
      yamcsInstance - The yamcs instance, or null if this is a global service.
      serviceName - The service name.
      config - The configured arguments for this service. If YamcsService.getSpec() is implemented then this contains the arguments after being validated (including any defaults).
      Throws:
      InitException - When something goes wrong during the execution of this method.

    • getPartitioningSchema

      public TimePartitionSchema getPartitioningSchema()

    • getParameterIdDb

      public ParameterIdDb getParameterIdDb()

    • getParameterGroupIdDb

      public ParameterGroupIdDb getParameterGroupIdDb()

    • updateCoverageEnd

      public void updateCoverageEnd(long t)

    • resetCoverageEnd

      public void resetCoverageEnd()
      Sets the coverageEnd to negative infinity

      This method is used mostly in unit tests

    • writeToArchive

      public void writeToArchive(PGSegment pgs) throws org.rocksdb.RocksDBException, IOException
      Throws:
      org.rocksdb.RocksDBException
      IOException

    • writeToArchive

      public void writeToArchive(long interval, Collection<PGSegment> pgList) throws org.rocksdb.RocksDBException, IOException
      Throws:
      org.rocksdb.RocksDBException
      IOException

    • reprocess

      public Future<?> reprocess(long start, long stop)
      Rebuild the parameter archive between start and stop. The times will be adjusted to build full intervals.

      Both start and stop can be left unspecified (by setting them to TimeEncoding.INVALID_INSTANT) to start from the beginning and/or go to the end of the archive.

      If the realtime parameter filler is enabled, the end will be set by default to the end of the previous interval compared to the interval of the current mission time.

    • getPartitions

      public List<ParameterArchive.Partition> getPartitions(long start, long stop, boolean ascending)
      a copy of the partitions from start to stop inclusive
      Parameters:
      start -
      stop -
      Returns:
      a sorted list of partitions

    • doStart

      protected void doStart()
      Specified by:
      doStart in class com.google.common.util.concurrent.AbstractService

    • doStop

      protected void doStop()
      Specified by:
      doStop in class com.google.common.util.concurrent.AbstractService

    • printKeys

      public void printKeys(PrintStream out) throws DecodingException, org.rocksdb.RocksDBException, IOException
      Throws:
      DecodingException
      org.rocksdb.RocksDBException
      IOException

    • deletePartitions

      public List<ParameterArchive.Partition> deletePartitions(long start, long stop) throws org.rocksdb.RocksDBException
      Delete all partitions that overlap with [start, stop) segment.
      Parameters:
      start -
      stop -
      Returns:
      all the partitions removed
      Throws:
      org.rocksdb.RocksDBException

    • purge

      public void purge() throws org.rocksdb.RocksDBException, InterruptedException, IOException
      Remove all the data and metadata related to the parameter archive and initialize a new database

      Prior to Yamcs 5.9.0 the Parameter Archive was stored on the default RocksDB column family. After the purge operation, the parameter archive will be moved to its own column family

      If the parameter archive is stored in the default column family this operation will remove all the records.

      If the parameter archive is stored into its own column family this operation will simply drop that column family (for all time based partitions)

      Throws:
      org.rocksdb.RocksDBException
      InterruptedException
      IOException

    • getIterator

      public org.rocksdb.RocksIterator getIterator(ParameterArchive.Partition p) throws org.rocksdb.RocksDBException, IOException
      Throws:
      org.rocksdb.RocksDBException
      IOException

    • getIteratorWithOptions

      public RdbIteratorWithOptions getIteratorWithOptions(ParameterArchive.Partition p) throws org.rocksdb.RocksDBException, IOException
      Throws:
      org.rocksdb.RocksDBException
      IOException

    • getTimeSegment

      public SortedTimeSegment getTimeSegment(ParameterArchive.Partition p, long segmentStart, int parameterGroupId, org.rocksdb.ReadOptions opts) throws org.rocksdb.RocksDBException, IOException
      Throws:
      org.rocksdb.RocksDBException
      IOException

    • coverageEnd

      public long coverageEnd()

    • getIntervalStart

      public static long getIntervalStart(long instant)
      returns the interval (instant) where this instant could fit.
      Parameters:
      instant -
      Returns:

    • getInterval

      public static long getInterval(long instant)

    • getIntervalEnd

      public static long getIntervalEnd(long instant)
      returns the end of the interval where the instant fits
      Parameters:
      instant -
      Returns:

    • getIntervalDuration

      public static long getIntervalDuration()
      duration in milliseconds of one segment
      Returns:

    • getTablespace

      public Tablespace getTablespace()

    • getRealtimeFiller

      public RealtimeArchiveFiller getRealtimeFiller()

    • getBackFiller

      public BackFiller getBackFiller()

    • disableAutoCompaction

      public void disableAutoCompaction(long start, long stop)

    • enableAutoCompaction

      public void enableAutoCompaction(long start, long stop)

    • compact

      public void compact()

    • maxCoverageEnd

      public long maxCoverageEnd()

    • getCoverageEnd

      public long getCoverageEnd(long now) throws org.rocksdb.RocksDBException, IOException, DecodingException
      Computes the coverage end as the greatest timestamp of a parameter in the archive, smaller than now

      In order to find that, it iterates over all time segments

      Throws:
      IOException
      org.rocksdb.RocksDBException
      DecodingException

    • getFillerLock

      public FillerLock getFillerLock()