Package org.yamcs.yarch.rocksdb

Class RdbStorageEngine

java.lang.Object

org.yamcs.yarch.rocksdb.RdbStorageEngine

All Implemented Interfaces:

StorageEngine

public class RdbStorageEngine
extends Object
implements StorageEngine

Storage Engine based on RocksDB. Data is stored in multiple Tablespace.

Each table has associated a tablespace, if not set at creation of the table, it is the tablespace associated to the instance.

Field Summary

Fields

Modifier and Type	Field	Description
static final int	TBS_INDEX_SIZE
static final byte[]	ZERO_BYTES

Method Summary

Modifier and Type	Method	Description
void	createTable(YarchDatabaseInstance ydb, TableDefinition def)	Create a new table based on definition.
Tablespace	createTablespace(String tablespaceName)
static final byte[]	dbKey(int tbsIndex)	returns a byte array containing the encoded tbsIndex
void	dropTable(YarchDatabaseInstance ydb, TableDefinition tbl)	Drop the table (removing all data)
void	dropTablespace(String tablespaceName)
RdbBucketDatabase	getBucketDatabase(YarchDatabaseInstance ydb)
HistogramIterator	getHistogramIterator(YarchDatabaseInstance ydb, TableDefinition tblDef, String columnName, TimeInterval interval)
static RdbStorageEngine	getInstance()
RdbPartitionManager	getPartitionManager(YarchDatabaseInstance ydb, TableDefinition tblDef)
ProtobufDatabase	getProtobufDatabase(YarchDatabaseInstance ydb)
Sequence	getSequence(YarchDatabaseInstance ydb, String name, boolean create)	Returns a sequence with the given name if it exists or first create it and returns it if create is true.
List<SequenceInfo>	getSequencesInfo(YarchDatabaseInstance ydb)	Gets the list of sequences togehter with their latest values
Tablespace	getTablespace(String tablespaceName)
Tablespace	getTablespace(YarchDatabaseInstance ydb)	Create and/or get the tablespace for the yarch database instance.
Map<String,Tablespace>	getTablespaces()
List<TableDefinition>	loadTables(YarchDatabaseInstance ydb)	Loads the table definitions from the disk for all the tables belonging to the instance.
void	migrateTableDefinition(YarchDatabaseInstance ydb, TableDefinition tblDef)	In Yamcs version 4 the table definitions were stored in yaml serialized format (in the /storage/yamcs-data/{instance-name}/{table-name}.def)
RocksdbSysParamProducer	newRdbParameterProducer(String yamcsInstance, SystemParametersService sps)	returns a system parameter producer for the databases used by the yamcsInstance
TableWalker	newSecondaryIndexTableWalker(YarchDatabaseInstance ydb, TableDefinition tableDefinition, boolean ascending, boolean follow)
TableWalker	newTableWalker(ExecutionContext ctx, TableDefinition tbl, boolean ascending, boolean follow)	Creates a new table iterator.
RdbTableWriter	newTableWriter(YarchDatabaseInstance ydb, TableDefinition tblDef, TableWriter.InsertMode insertMode)	Creates a new table writer
void	renameTable(YarchDatabaseInstance ydb, TableDefinition tblDef, String newName)	rename the table
void	saveTableDefinition(YarchDatabaseInstance ydb, TableDefinition tblDef, List<TableColumnDefinition> keyColumns, List<TableColumnDefinition> valueColumns)	Persist the table definition to diks (called when the table definition modifies)
void	setIgnoreVersionIncompatibility(boolean b)	set to ignore version incompatibility - only used from the version upgrading functions to allow loading old tables.
void	shutdown()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

TBS_INDEX_SIZE

public static final int TBS_INDEX_SIZE

See Also:

• Constant Field Values

ZERO_BYTES

public static final byte[] ZERO_BYTES

Method Details

dropTable

public void dropTable(YarchDatabaseInstance ydb,
 TableDefinition tbl)
               throws YarchException

Description copied from interface: StorageEngine

Drop the table (removing all data)

Specified by:

dropTable in interface StorageEngine

Throws:

YarchException

newTableWriter

public RdbTableWriter newTableWriter(YarchDatabaseInstance ydb,
 TableDefinition tblDef,
 TableWriter.InsertMode insertMode)

Description copied from interface: StorageEngine

Creates a new table writer

Specified by:

newTableWriter in interface StorageEngine

Returns:

newTableWalker

public TableWalker newTableWalker(ExecutionContext ctx,
 TableDefinition tbl,
 boolean ascending,
 boolean follow)

Description copied from interface: StorageEngine

Creates a new table iterator.

Specified by:

newTableWalker in interface StorageEngine

createTable

public void createTable(YarchDatabaseInstance ydb,
 TableDefinition def)
                 throws YarchException

Description copied from interface: StorageEngine

Create a new table based on definition.

Specified by:

createTable in interface StorageEngine

Throws:

YarchException

getInstance

public static RdbStorageEngine getInstance()

getPartitionManager

public RdbPartitionManager getPartitionManager(YarchDatabaseInstance ydb,
 TableDefinition tblDef)

Specified by:

getPartitionManager in interface StorageEngine

getTablespace

public Tablespace getTablespace(YarchDatabaseInstance ydb)

Create and/or get the tablespace for the yarch database instance.

Parameters:

ydb -

Returns:

getTablespaces

public Map<String,Tablespace> getTablespaces()

createTablespace

public Tablespace createTablespace(String tablespaceName)

setIgnoreVersionIncompatibility

public void setIgnoreVersionIncompatibility(boolean b)

set to ignore version incompatibility - only used from the version upgrading functions to allow loading old tables.

Parameters:

b -

getHistogramIterator

public HistogramIterator getHistogramIterator(YarchDatabaseInstance ydb,
 TableDefinition tblDef,
 String columnName,
 TimeInterval interval)
                                       throws YarchException

Specified by:

getHistogramIterator in interface StorageEngine

Throws:

YarchException

getBucketDatabase

public RdbBucketDatabase getBucketDatabase(YarchDatabaseInstance ydb)
                                    throws YarchException

Specified by:

getBucketDatabase in interface StorageEngine

Throws:

YarchException

getProtobufDatabase

public ProtobufDatabase getProtobufDatabase(YarchDatabaseInstance ydb)
                                     throws YarchException

Specified by:

getProtobufDatabase in interface StorageEngine

Throws:

YarchException

getTablespace

public Tablespace getTablespace(String tablespaceName)

dropTablespace

public void dropTablespace(String tablespaceName)

shutdown

public void shutdown()

loadTables

public List<TableDefinition> loadTables(YarchDatabaseInstance ydb)
                                 throws YarchException

Description copied from interface: StorageEngine

Loads the table definitions from the disk for all the tables belonging to the instance.

called at startup.

Specified by:

loadTables in interface StorageEngine

Throws:

YarchException

renameTable

public void renameTable(YarchDatabaseInstance ydb,
 TableDefinition tblDef,
 String newName)

Description copied from interface: StorageEngine

rename the table

Specified by:

renameTable in interface StorageEngine

migrateTableDefinition

public void migrateTableDefinition(YarchDatabaseInstance ydb,
 TableDefinition tblDef)
                            throws YarchException

Description copied from interface: StorageEngine

In Yamcs version 4 the table definitions were stored in yaml serialized format (in the /storage/yamcs-data/{instance-name}/{table-name}.def)

This function is called to migrate them inside the storage engine where they are stored starting with Yamcs 5

Specified by:

migrateTableDefinition in interface StorageEngine

Throws:

YarchException

saveTableDefinition

public void saveTableDefinition(YarchDatabaseInstance ydb,
 TableDefinition tblDef,
 List<TableColumnDefinition> keyColumns,
 List<TableColumnDefinition> valueColumns)
                         throws YarchException

Description copied from interface: StorageEngine

Persist the table definition to diks (called when the table definition modifies)

The general table properties should be read from the tblDef argument but the column properties should be read from the extra arguments This is because the method is called with modified column content which is not reflected in the table definition until the data is saved in the database.

Specified by:

saveTableDefinition in interface StorageEngine

Throws:

YarchException

getSequence

public Sequence getSequence(YarchDatabaseInstance ydb,
 String name,
 boolean create)
                     throws YarchException

Description copied from interface: StorageEngine

Returns a sequence with the given name if it exists or first create it and returns it if create is true.

If create is false and the sequence does not exist, returns null.

Specified by:

getSequence in interface StorageEngine

Returns:

Throws:

YarchException

newSecondaryIndexTableWalker

public TableWalker newSecondaryIndexTableWalker(YarchDatabaseInstance ydb,
 TableDefinition tableDefinition,
 boolean ascending,
 boolean follow)

Specified by:

newSecondaryIndexTableWalker in interface StorageEngine

getSequencesInfo

public List<SequenceInfo> getSequencesInfo(YarchDatabaseInstance ydb)

Description copied from interface: StorageEngine

Gets the list of sequences togehter with their latest values

Specified by:

getSequencesInfo in interface StorageEngine

dbKey

public static final byte[] dbKey(int tbsIndex)

returns a byte array containing the encoded tbsIndex

newRdbParameterProducer

public RocksdbSysParamProducer newRdbParameterProducer(String yamcsInstance,
 SystemParametersService sps)

returns a system parameter producer for the databases used by the yamcsInstance

the producer will provide statistics with the memory usage