vinyl: fix duplicate multikey stmt accounting with deferred deletes
`vy_mem_insert()` and `vy_mem_insert_upsert()` increment the row count statistic of `vy_mem` only if no statement is replaced, which is correct, while `vy_lsm_commit()` increments the row count of `vy_lsm` unconditionally. As a result, `vy_lsm` may report a non-zero statement count (via `index.stat()` or `index.len()`) after a dump. This may happen only with a non-unique multikey index, when the statement has duplicates in the indexed array, and only if the `deferred_deletes` option is enabled, because otherwise we drop duplicates when we form the transaction write set, see `vy_tx_set()`. With `deferred_deletes`, we may create a `txv` for each multikey entry at the time when we prepare to commit the transaction, see `vy_tx_handle_deferred_delete()`. Another problem is that `vy_mem_rollback_stmt()` always decrements the row count, even if it didn't find the rolled back statement in the tree. As a result, if the transaction with duplicate multikey entries is rolled back on WAL error, we'll decrement the row count of `vy_mem` more times than necessary. To fix this issue, let's make the `vy_mem` methods update the in-memory statistic of `vy_lsm`. This way they should always stay in-sync. Also, we make `vy_mem_rollback_stmt()` skip updating the statistics in case the rolled back statement isn't present in the tree. This issue results in `vinyl-luatest/select_consistency_test.lua` flakiness when checking `index.len()` after compaction. Let's make the test more thorough and also check that `index.len()` equals `index.count()`. Closes #10751 Part of #10752 NO_DOC=bug fix (cherry picked from commit e8810c555d4e6ba56e6c798e04216aa11efb5304)
Showing
- changelogs/unreleased/gh-10751-fix-vinyl-memory-stmt-accounting.md 5 additions, 0 deletions...s/unreleased/gh-10751-fix-vinyl-memory-stmt-accounting.md
- src/box/vinyl.c 1 addition, 3 deletionssrc/box/vinyl.c
- src/box/vy_lsm.c 10 additions, 14 deletionssrc/box/vy_lsm.c
- src/box/vy_mem.c 27 additions, 12 deletionssrc/box/vy_mem.c
- src/box/vy_mem.h 9 additions, 3 deletionssrc/box/vy_mem.h
- test/unit/vy_iterators_helper.c 3 additions, 2 deletionstest/unit/vy_iterators_helper.c
- test/unit/vy_iterators_helper.h 1 addition, 0 deletionstest/unit/vy_iterators_helper.h
- test/unit/vy_mem.c 1 addition, 1 deletiontest/unit/vy_mem.c
- test/vinyl-luatest/gh_10751_duplicate_multikey_stmt_accounting_test.lua 80 additions, 0 deletions...test/gh_10751_duplicate_multikey_stmt_accounting_test.lua
- test/vinyl-luatest/select_consistency_test.lua 12 additions, 3 deletionstest/vinyl-luatest/select_consistency_test.lua
Loading
Please register or sign in to comment