vinyl: set range size automatically
The key space of a vinyl index consists of multiple ranges that can be compacted independently. This design was initially invented to enable parallel compaction, so the range size is configured statically, by the range_size index option, which equals 1 GB by default. However, it turns out that ranges can also be useful for smoothing IO load: if we compact approximately the same number of ranges after each dump, we will avoid IO bursts, which is good, because IO bursts can distort the LSM tree shape, resulting in increased read amplification. To achieve that, we need to maintain at least as many ranges as the number of dumps it takes to trigger major compaction of a range. With the default range size, this condition will hold only if the index is huge (tens to hundreds gigabytes). If the database isn't that big or consists of many small indexes, the range count will never even approach that number. So this patch makes the range size scale dynamically to satisfy that condition. The range size configuration options, both global and per index, aren't removed though. The patch just changes box.cfg.vinyl_range_size default value to nil, which enables automatic range sizing for all new indexes created without passing range_size explicitly. All existing indexes will still use the range size stored in index options (we don't want to alter the behavior of an existing production setup). We are not planning to drop range_size option altogether - it still can be useful for testing and performance analysis. The actual range size value is now reported in index.stat(). Needed for #3944
Showing
- src/box/alter.cc 2 additions, 6 deletionssrc/box/alter.cc
- src/box/box.cc 1 addition, 5 deletionssrc/box/box.cc
- src/box/index_def.c 1 addition, 1 deletionsrc/box/index_def.c
- src/box/lua/load_cfg.lua 1 addition, 1 deletionsrc/box/lua/load_cfg.lua
- src/box/lua/space.cc 4 additions, 2 deletionssrc/box/lua/space.cc
- src/box/vinyl.c 1 addition, 0 deletionssrc/box/vinyl.c
- src/box/vy_lsm.c 42 additions, 2 deletionssrc/box/vy_lsm.c
- src/box/vy_lsm.h 4 additions, 0 deletionssrc/box/vy_lsm.h
- src/box/vy_range.c 5 additions, 5 deletionssrc/box/vy_range.c
- src/box/vy_range.h 5 additions, 5 deletionssrc/box/vy_range.h
- test/app-tap/init_script.result 10 additions, 11 deletionstest/app-tap/init_script.result
- test/box-tap/cfg.test.lua 1 addition, 2 deletionstest/box-tap/cfg.test.lua
- test/box/admin.result 0 additions, 2 deletionstest/box/admin.result
- test/box/cfg.result 0 additions, 4 deletionstest/box/cfg.result
- test/vinyl/ddl.result 0 additions, 5 deletionstest/vinyl/ddl.result
- test/vinyl/ddl.test.lua 0 additions, 1 deletiontest/vinyl/ddl.test.lua
- test/vinyl/misc.result 78 additions, 0 deletionstest/vinyl/misc.result
- test/vinyl/misc.test.lua 26 additions, 0 deletionstest/vinyl/misc.test.lua
- test/vinyl/stat.result 53 additions, 51 deletionstest/vinyl/stat.result
Loading
Please register or sign in to comment