vinyl: fix compaction priority calculation
When computing the number of runs that need to be compacted for a range to conform to the target LSM tree shape, we use the newest run size for the size of the first LSM tree level. This isn't quite correct for two reasons. First, the size of the newest run is unstable - it may vary in a relatively wide range from dump to dump. This leads to frequent changes in the target LSM tree shape and, as a result, unpredictable compaction behavior. In particular this breaks compaction randomization, which is supposed to smooth out IO load generated by compaction. Second, this can increase space amplification. We trigger compaction at the last level when there's more than one run, irrespective of the value of run_count_per_level configuration option. We expect this to keep space amplification below 2 provided run_count_per_level is not greater than (run_size_ratio - 1). However, if the newest run happens to have such a size that multiplying it by run_size_ratio several times gives us a value only slightly less than the size of the oldest run, we can accumulate up to run_count_per_level more runs that are approximately as big as the last level run without triggering compaction, thus increasing space amplification by up to run_count_per_level. To fix these problems, let's use the oldest run size for computing the size of the first LSM tree level - simply divide it by run_size_ratio until it exceeds the size of the newest run. Follow-up #3657
Showing
- src/box/vy_range.c 31 additions, 11 deletionssrc/box/vy_range.c
- test/vinyl/errinj.result 2 additions, 2 deletionstest/vinyl/errinj.result
- test/vinyl/errinj.test.lua 2 additions, 1 deletiontest/vinyl/errinj.test.lua
- test/vinyl/gc.result 2 additions, 1 deletiontest/vinyl/gc.result
- test/vinyl/gc.test.lua 2 additions, 1 deletiontest/vinyl/gc.test.lua
- test/vinyl/layout.result 133 additions, 45 deletionstest/vinyl/layout.result
- test/vinyl/layout.test.lua 2 additions, 3 deletionstest/vinyl/layout.test.lua
- test/vinyl/replica_rejoin.result 1 addition, 1 deletiontest/vinyl/replica_rejoin.result
- test/vinyl/replica_rejoin.test.lua 1 addition, 1 deletiontest/vinyl/replica_rejoin.test.lua
- test/vinyl/update_optimize.result 38 additions, 38 deletionstest/vinyl/update_optimize.result
- test/vinyl/update_optimize.test.lua 27 additions, 29 deletionstest/vinyl/update_optimize.test.lua
Loading
Please register or sign in to comment