vinyl: use LSN from WAL instead of index_opts->lsn in vylog
An index can be dropped and then recreated with the same space/index id. To discriminate between different incarnations of the same index during recovery, we use index LSN stored in index options, as it is supposed to be unique. However, the uniqueness property doesn't always hold: - If two indexes are created from different fibers, they might receive the same LSN. - If an index is created by inserting a record into _index system space directly, without using the public API, as it is the case in case of logical backup, its LSN might conflict with the LSN of an existing index or a previous incarnation of the same index stored in vylog. These exceptions can result in unrecoverable errors during local recovery, like this one: F> can't initialize storage: Invalid VYLOG file: Duplicate index id 3 Besides, storing LSN in index options is ugly, because LSN isn't a user-defined option - it's a part of the implementation. To fix this issues, let's use the LSN passed to Index::commitCreate, i.e. the actual LSN received by the row that created the index. There's one problem though: snapshot rows don't store LSNs. However, it doesn't mean we can't find the index in vylog corresponding to a snapshot row: we just need to look up the index by space_id/index_id instead of LSN and then compare the snapshot LSN with the LSN of the last index incarnation stored in vylog - if the latter turns out to be less, then we need to load the index, otherwise the index is going to be dropped and we need to load a dummy index. For more details, see the comment to vy_recovery_load_index(). Another issue that needs a clarification is backward compatibility. The thing is the LSN written to the index options lags behind the actual LSN assigned to the row that created the index by 1. So to preserve backward compatibility, we use LSN from index options for legacy indexes that have it, while for indexes created after this patch we don't store LSN in index options (set it to 0), neither do we use it on recovery (use row LSN instead). Closes #2536
Showing
- src/box/lua/schema.lua 0 additions, 2 deletionssrc/box/lua/schema.lua
- src/box/vinyl.c 27 additions, 28 deletionssrc/box/vinyl.c
- src/box/vy_index.c 50 additions, 17 deletionssrc/box/vy_index.c
- src/box/vy_index.h 16 additions, 7 deletionssrc/box/vy_index.h
- src/box/vy_log.c 185 additions, 44 deletionssrc/box/vy_log.c
- src/box/vy_log.h 33 additions, 6 deletionssrc/box/vy_log.h
- test/vinyl/layout.result 8 additions, 8 deletionstest/vinyl/layout.result
Loading
Please register or sign in to comment