vinyl: make read iterator always return newest tuple version
vy_read_iterator_next() first scans all sources to find the newest statement for the next key. If the statement turns out to be UPSERT, it retrieves older statements until a terminal statement is found. While retrieving older statements, it may yield, which opens a window for other fibers to insert newer statements for the same key. Those statements will be ignored by the read iterator even if they are visible from the iterator read view. To be able to use read iterator for building secondary indexes (i.e. iterate over tuples stored in pk and insert them into the new index), we need to lift that limitation, otherwise there's a good chance that while building a new index we will overwrite statements inserted by concurrent transactions. So this patch reworks the read iterator to guarantee that it always returns the newest tuple version. To achieve that, we now retrieve the whole key history instead of only the last statement first time we scan a source. Respectively, we store a vy_history instead of tuple in vy_read_src. All source iterators now return vy_history too. As a result, we don't need to iterate to the next LSN (and possibly yield) to apply UPSERTs - we simply splice all histories corresponding to the next key and call vy_history_apply() to return the resultant tuple. Note, this patch changes the output of vinyl/upsert.test.lua. This is OK as now UPSERTs are applied in the reverse order, from the oldest statement to the newest.
Showing
- src/box/vy_cache.c 19 additions, 13 deletionssrc/box/vy_cache.c
- src/box/vy_cache.h 18 additions, 13 deletionssrc/box/vy_cache.h
- src/box/vy_history.c 7 additions, 4 deletionssrc/box/vy_history.c
- src/box/vy_history.h 29 additions, 2 deletionssrc/box/vy_history.h
- src/box/vy_mem.c 37 additions, 44 deletionssrc/box/vy_mem.c
- src/box/vy_mem.h 12 additions, 21 deletionssrc/box/vy_mem.h
- src/box/vy_point_lookup.c 8 additions, 21 deletionssrc/box/vy_point_lookup.c
- src/box/vy_read_iterator.c 104 additions, 202 deletionssrc/box/vy_read_iterator.c
- src/box/vy_read_iterator.h 0 additions, 2 deletionssrc/box/vy_read_iterator.h
- src/box/vy_run.c 48 additions, 7 deletionssrc/box/vy_run.c
- src/box/vy_run.h 9 additions, 14 deletionssrc/box/vy_run.h
- src/box/vy_tx.c 27 additions, 16 deletionssrc/box/vy_tx.c
- src/box/vy_tx.h 18 additions, 11 deletionssrc/box/vy_tx.h
- test/unit/CMakeLists.txt 2 additions, 0 deletionstest/unit/CMakeLists.txt
- test/unit/vy_cache.c 14 additions, 2 deletionstest/unit/vy_cache.c
- test/unit/vy_mem.c 19 additions, 5 deletionstest/unit/vy_mem.c
- test/vinyl/upsert.result 4 additions, 4 deletionstest/vinyl/upsert.result
- test/vinyl/upsert.test.lua 3 additions, 3 deletionstest/vinyl/upsert.test.lua
Loading
Please register or sign in to comment