Skip to content
Snippets Groups Projects
Commit d5ceb204 authored by Vladimir Davydov's avatar Vladimir Davydov
Browse files

vinyl: fix compaction priority calculation

When computing the number of runs that need to be compacted for a range
to conform to the target LSM tree shape, we use the newest run size for
the size of the first LSM tree level. This isn't quite correct for two
reasons.

First, the size of the newest run is unstable - it may vary in a
relatively wide range from dump to dump. This leads to frequent changes
in the target LSM tree shape and, as a result, unpredictable compaction
behavior. In particular this breaks compaction randomization, which is
supposed to smooth out IO load generated by compaction.

Second, this can increase space amplification. We trigger compaction at
the last level when there's more than one run, irrespective of the value
of run_count_per_level configuration option. We expect this to keep
space amplification below 2 provided run_count_per_level is not greater
than (run_size_ratio - 1). However, if the newest run happens to have
such a size that multiplying it by run_size_ratio several times gives us
a value only slightly less than the size of the oldest run, we can
accumulate up to run_count_per_level more runs that are approximately as
big as the last level run without triggering compaction, thus increasing
space amplification by up to run_count_per_level.

To fix these problems, let's use the oldest run size for computing the
size of the first LSM tree level - simply divide it by run_size_ratio
until it exceeds the size of the newest run.

Follow-up #3657
parent c6743038
No related branches found
No related tags found
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment