Skip to content
Snippets Groups Projects
  • Vladimir Davydov's avatar
    353bcdc5
    Rework space truncation · 353bcdc5
    Vladimir Davydov authored
    Space truncation that we have now is not atomic: we recreate all indexes
    of the truncated space one by one. This can result in nasty failures if
    a tuple insertion races with the space truncation and sees some indexes
    truncated and others not.
    
    This patch redesigns space truncation as follows:
    
     - Truncate is now triggered by bumping a counter in a new system space
       called _truncate. As before, space truncation is implemented by
       recreating all of its indexes, but now this is done internally in one
       go, inside the space alter trigger. This makes the operation atomic.
    
     - New indexes are created with Handler::createIndex method, old indexes
       are deleted with Index::~Index. Neither Index::commitCreate nor
       Index::commitDrop are called in case of truncation, in contrast to
       space alter. Since memtx needs to release tuples referenced by old
       indexes, and vinyl needs to log space truncation in the metadata log,
       new Handler methods are introduced, prepareTruncateSpace and
       commitTruncateSpace, which are passed the old and new spaces. They
       are called before and after truncate record is written to WAL,
       respectively.
    
     - Since Handler::commitTruncateSpace must not fail while vylog write
       obviously may, we reuse the technique used by commitCreate and
       commitDrop methods of VinylIndex, namely leave the record we failed
       to write in vylog buffer to be either flushed along with the next
       write or replayed on WAL recovery. To be able to detect if truncation
       was logged while recovering WAL, we introduce a new vylog record
       type, VY_LOG_TRUNCATE_INDEX which takes truncate_count as a key: if
       on WAL recovery index truncate_count happens to be <= space
       truncate_count, then it it means that truncation was not logged and
       we need to log it again.
    
    Closes #618
    Closes #2060
    353bcdc5
    History
    Rework space truncation
    Vladimir Davydov authored
    Space truncation that we have now is not atomic: we recreate all indexes
    of the truncated space one by one. This can result in nasty failures if
    a tuple insertion races with the space truncation and sees some indexes
    truncated and others not.
    
    This patch redesigns space truncation as follows:
    
     - Truncate is now triggered by bumping a counter in a new system space
       called _truncate. As before, space truncation is implemented by
       recreating all of its indexes, but now this is done internally in one
       go, inside the space alter trigger. This makes the operation atomic.
    
     - New indexes are created with Handler::createIndex method, old indexes
       are deleted with Index::~Index. Neither Index::commitCreate nor
       Index::commitDrop are called in case of truncation, in contrast to
       space alter. Since memtx needs to release tuples referenced by old
       indexes, and vinyl needs to log space truncation in the metadata log,
       new Handler methods are introduced, prepareTruncateSpace and
       commitTruncateSpace, which are passed the old and new spaces. They
       are called before and after truncate record is written to WAL,
       respectively.
    
     - Since Handler::commitTruncateSpace must not fail while vylog write
       obviously may, we reuse the technique used by commitCreate and
       commitDrop methods of VinylIndex, namely leave the record we failed
       to write in vylog buffer to be either flushed along with the next
       write or replayed on WAL recovery. To be able to detect if truncation
       was logged while recovering WAL, we introduce a new vylog record
       type, VY_LOG_TRUNCATE_INDEX which takes truncate_count as a key: if
       on WAL recovery index truncate_count happens to be <= space
       truncate_count, then it it means that truncation was not logged and
       we need to log it again.
    
    Closes #618
    Closes #2060