DML and Memory Utilization

Vector supports multi-version read consistency. Committed data in PDTs is efficiently merged with data read from disk to answer queries. Any data that is committed to PDTs is also written to the transaction log (the Vector LOG file) and persisted on disk for recovery purposes before the commit completes.

In Vector, a transaction starts with the first operation, even if that operation is a read query, and read consistency is maintained throughout the transaction.

If one session is performing read-only queries, and another session is performing DML operations, then the read-only session will see the changes made by the DML session only when:

• The DML session commits the changes (which completes the transaction); and

• The read-only session ends its ongoing transaction by issuing a commit or rollback.

Maintaining read-consistency for a read-only session can use a lot of memory if other sessions perform DML operations and complete transactions (for details refer to Knowledge Base Document Why does Committed Transaction Memory Keep Increasing?).

For this reason on a system that performs lots of DML operations, it is important to COMMIT or ROLLBACK a session periodically, or to DISCONNECT a session and re-CONNECT.

Every Vector database reserves a portion of the server’s memory to store PDTs. The amount of memory reserved is controlled by the parameter max_global_update_memory which is the upper limit on total update memory consumption as a percentage of query execution memory (max_memory_size) that can be used for PDTs. The default is 0.25.

When the amount of memory used by PDTs exceeds the maximum target amount then Vector triggers automatic data propagation against all tables with outstanding transactions. Transactions will continue to be executed during the process but more memory may be consumed.