Skip to content
Snippets Groups Projects
  • Vladimir Davydov's avatar
    29e2931c
    vinyl: fix race between compaction and gc of dropped LSM · 29e2931c
    Vladimir Davydov authored
    An LSM tree (space index, that is) can be dropped while compaction is in
    progress for it. In this case compaction will still commit the new run
    to vylog upon completion. This usually works fine, but not if gc has
    already purged all the information about the dropped LSM tree from vylog
    by that time, in which case an attempt to commit the new run will result
    in permanently broken vylog (because compaction will write vylog records
    for a non-existing object):
    
    ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 13 deleted but not registered
    
    To prevent this from happening, let's make compaction silently drop the
    new run without committing it to vylog if the LSM tree has been dropped.
    This should work just fine - since the LSM tee isn't used anymore we
    don't need to have it compacted, neither do we need to delete the run,
    since gc will eventually clean up all artefacts left from the dropped
    LSM tree.
    
    One thing to be noted is that we also must exclude dropped LSM trees
    from further compaction - if we don't do that, we might end up picking
    the dropped LSM tree for compaction over and over again (because it
    isn't actually compacted).
    
    This patch also drops the gh-5141-invalid-vylog-file test, because the
    latter just ensured that the issue fixed by this patch is there.
    
    Closes #5436
    29e2931c
    History
    vinyl: fix race between compaction and gc of dropped LSM
    Vladimir Davydov authored
    An LSM tree (space index, that is) can be dropped while compaction is in
    progress for it. In this case compaction will still commit the new run
    to vylog upon completion. This usually works fine, but not if gc has
    already purged all the information about the dropped LSM tree from vylog
    by that time, in which case an attempt to commit the new run will result
    in permanently broken vylog (because compaction will write vylog records
    for a non-existing object):
    
    ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 13 deleted but not registered
    
    To prevent this from happening, let's make compaction silently drop the
    new run without committing it to vylog if the LSM tree has been dropped.
    This should work just fine - since the LSM tee isn't used anymore we
    don't need to have it compacted, neither do we need to delete the run,
    since gc will eventually clean up all artefacts left from the dropped
    LSM tree.
    
    One thing to be noted is that we also must exclude dropped LSM trees
    from further compaction - if we don't do that, we might end up picking
    the dropped LSM tree for compaction over and over again (because it
    isn't actually compacted).
    
    This patch also drops the gh-5141-invalid-vylog-file test, because the
    latter just ensured that the issue fixed by this patch is there.
    
    Closes #5436