Skip to content
Snippets Groups Projects
Commit 4586f2d6 authored by Vladimir Davydov's avatar Vladimir Davydov
Browse files

vinyl: fix snapshot consistency

Commit dbfd515f ("vinyl: fix crash if snapshot is called while dump
is in progress") introduced a bug that can result in statements inserted
after WAL checkpoint being included in a snapshot. This happens, because
vy_begin_checkpoint() doesn't force rotation of in-memory trees anymore:
it bumps checkpoint_generation, but doesn't touch scheduler->generation,
which is used to trigger in-memory tree rotation.

To fix this issue, this patch zaps scheduler->checkpoint_generation and
makes vy_begin_checkpoint() bump scheduler->generation directly as it
used to. To guarantee dump consistency (the issued fixed by commit
dbfd515f), scheduler->dump_generation is introduced - it defines the
generation of in-memory data that are currently being dumped. The
scheduler won't start dumping newer trees until all trees whose
generation equals dump_generation have been dumped. The counter is only
bumped by the scheduler itself when all old in-memory trees have been
dumped. Together, this guarantees that each dump contains data of the
same generation, i.e. is consistent.

While we are at it, let's also remove vy_scheduler->dump_fifo, the list
of all in-memory trees sorted in the chronological order. The scheduler
uses it to keep track of the oldest in-memory tree, which is needed to
invoke lsregion_gc(). However, since we do not remove indexes from the
dump_heap, as we used to not so long ago, we can use the heap for this.
The only problem is indexes that are currently being dumped are moved
off the top of the heap, but we can detect this case by maintaining a
counter of dump tasks in progress: if dump_task_count is > 0 when a dump
task is completed, we must not call lsregion_gc() irrespective of the
generation of the index at the top of the heap. A good thing about
ridding of vy_scheduler->dump_fifo is that it is a step forward towards
making vy_index independent of vy_scheduler so that it can be moved to a
separate source file.

Closes #2541
Needed for #1906
parent 199836db
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment