Skip to content
Snippets Groups Projects
Commit bedb1d94 authored by Vladimir Davydov's avatar Vladimir Davydov
Browse files

vinyl: set range size automatically

The key space of a vinyl index consists of multiple ranges that can be
compacted independently. This design was initially invented to enable
parallel compaction, so the range size is configured statically, by the
range_size index option, which equals 1 GB by default. However, it turns
out that ranges can also be useful for smoothing IO load: if we compact
approximately the same number of ranges after each dump, we will avoid
IO bursts, which is good, because IO bursts can distort the LSM tree
shape, resulting in increased read amplification.

To achieve that, we need to maintain at least as many ranges as the
number of dumps it takes to trigger major compaction of a range. With
the default range size, this condition will hold only if the index is
huge (tens to hundreds gigabytes). If the database isn't that big or
consists of many small indexes, the range count will never even approach
that number. So this patch makes the range size scale dynamically to
satisfy that condition.

The range size configuration options, both global and per index, aren't
removed though. The patch just changes box.cfg.vinyl_range_size default
value to nil, which enables automatic range sizing for all new indexes
created without passing range_size explicitly. All existing indexes will
still use the range size stored in index options (we don't want to alter
the behavior of an existing production setup). We are not planning to
drop range_size option altogether - it still can be useful for testing
and performance analysis.

The actual range size value is now reported in index.stat().

Needed for #3944
parent e4f5476c
No related branches found
No related tags found
Loading
Showing with 234 additions and 103 deletions
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment