vinyl: fix snapshot consistency
Commit dbfd515f ("vinyl: fix crash if snapshot is called while dump is in progress") introduced a bug that can result in statements inserted after WAL checkpoint being included in a snapshot. This happens, because vy_begin_checkpoint() doesn't force rotation of in-memory trees anymore: it bumps checkpoint_generation, but doesn't touch scheduler->generation, which is used to trigger in-memory tree rotation. To fix this issue, this patch zaps scheduler->checkpoint_generation and makes vy_begin_checkpoint() bump scheduler->generation directly as it used to. To guarantee dump consistency (the issued fixed by commit dbfd515f), scheduler->dump_generation is introduced - it defines the generation of in-memory data that are currently being dumped. The scheduler won't start dumping newer trees until all trees whose generation equals dump_generation have been dumped. The counter is only bumped by the scheduler itself when all old in-memory trees have been dumped. Together, this guarantees that each dump contains data of the same generation, i.e. is consistent. While we are at it, let's also remove vy_scheduler->dump_fifo, the list of all in-memory trees sorted in the chronological order. The scheduler uses it to keep track of the oldest in-memory tree, which is needed to invoke lsregion_gc(). However, since we do not remove indexes from the dump_heap, as we used to not so long ago, we can use the heap for this. The only problem is indexes that are currently being dumped are moved off the top of the heap, but we can detect this case by maintaining a counter of dump tasks in progress: if dump_task_count is > 0 when a dump task is completed, we must not call lsregion_gc() irrespective of the generation of the index at the top of the heap. A good thing about ridding of vy_scheduler->dump_fifo is that it is a step forward towards making vy_index independent of vy_scheduler so that it can be moved to a separate source file. Closes #2541 Needed for #1906
Loading
Please register or sign in to comment