Skip to content
Snippets Groups Projects
Commit 74fdd1c8 authored by Andrey Saranchin's avatar Andrey Saranchin Committed by Serge Petrenko
Browse files

memtx: remove all the space stories right on DDL

Currently, every DDL transaction has such properties:
1. DDL and DML operations can be mixed inside one transaction, order is
   not restricted.
2. DDL has an effect right after the operation, not after it's prepared
   or committed.
3. After the first DDL operation, transaction is not allowed to yield.
4. If DDL yields (format check, index build), operation must be the
   first in its transaction. If DDL yields, space cache is updated
   strictly after yields are over. In other words, transaction cannot
   yield after DDL changes became visible.

Keeping these properties in mind, we can recognize several flows in
the way memtx MVCC handles DDL now. Here is the approach:
1. All transactions concurrent with DDL are aborted when DDL gets
   prepared. Memtx stories are not deleted here.
2. If DDL gets committed, all memtx stories of the old space are
   deleted. It means that only stories created before DDL are deleted.

Such design is bad. Firstly, if transaction does a DDL and than a DML,
stories belonging to the new and the old schema will be mixed in the
indexes. Starting a DML transaction after DDL is prepared but before
it's committed leads to the same problem. If stories of different
schemas are mixed, Tarantool is most likely to crash since MVCC does
not handle this case at all. Secondly, transactions started after DDL is
prepared but before it's committed can read stories belonging to the old
schema. In this case, after DDL is committed and the stories are
deleted, we cannot do proper mvcc for such transactions anymore because
we've just deleted the tuple it has read.

Here is the new approach that current commit implements:
1. Abort all concurrent transactions and delete all memtx stories on
   every replace in space cache (right on DDL operation and on its
   rollback). In this case, every time a space and its schema is
   replaced with a new one, all the mvcc objects belonging to the old
   schema are deleted. In order not to break isolation of concurrent
   transactions, we have to abort them all right before deleting
   stories - it makes sense because we don't support old schema anymore
   and all its readers should be aborted.
2. When DDL gets committed, nothing happens since nothing has been
   changed.
3. Abort all concurrent transactions when DDL is being prepared. It is
   needed for DDL that does not update any space. For example, update in
   space `_cluster` or `_schema`. It worked this way before the commit
   so it is needed to provide the old behavior in such cases. Note that
   if DDL updates any space, all concurrent transactions are aborted
   right on DDL operation, when replace in space cache happens, and
   since DDL cannot yield, we have nothing to abort here so it's noop
   in such cases.

Since `memtx_tx_on_space_delete` was supposed to be called on commit, it
inserts prepared tuples to indexes. Since we want it to be called on
actual DDL, the commit makes it insert all tuples visible to the
transaction doing DDL.

Also, if we delete all the stories on DDL, we should correctly rollback
all non-committed DML statements - DML statements of the transaction
doing DDL if they happened before DDL and all transactions prepared
before DDL - if WAL write fails, we will need to roll them back without
stories. For this purpose, firstly, let's remove `engine_savepoint` only
for statements of aborted transactions in `memtx_tx_history_remove_stmt`
(it is needed not to roll them back because they are already handled) so
that statements of transaction doing DDL can be rolled back. And,
secondly, let's add a new helper `memtx_tx_history_rollback_empty_stmt`
that handles rollback of statements without stories (see helper's
description for elaboration).

Along the way, let's clean read lists of transactions aborted on DDL -
they are not needed anymore and potentially keeping them can lead to
use-after-free in future. Moreover, we should remove transactions
aborted by DDL from `read_view_txs` list so that they won't affect
memtx story GC until they are deleted - cleaning of transactions
does it as well.

Part of #10146
Part of #10474
Closes #10171
Closes #10096
Closes #10097

NO_CHANGELOG=later
NO_DOC=bugfix

(cherry picked from commit 959027de5553078dbe442aacb52e01e9b46c542c)
parent 584a8d9c
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment