Vladimir Davydov
authored
Every byte of data written to a vinyl database eventually gets compacted with data written to the database earlier. The ratio of the size of data actually written to disk to the size of data written to the database is called write amplification. Write amplification depends on the LSM tree configuration and the workload parameters and varies in a wide range, from 2-3 to 10-20 or even higher in some extreme cases. If the database engine doesn't manage to write those extra data, LSM tree shape will get distorted, which will result in increased read and space amplification, which, in turn, will lead to slowing down reads and wasting disk space. That's why it's so important to ensure the database engine has enough compaction power. One way to ensure that is increase the number of compaction threads by tuning box.cfg.vinyl_write_threads configuration knob, but one can't increase it beyond the capacity of the server running the instance. So the database engine must throttle writes if it detects that compaction threads are struggling to keep up. This patch implements a very simple algorithm to achieve that: it keeps track of recently observed write amplification and data compaction speed, use them to calculate the max transaction rate that the database engine can handle while steadily maintaining the current level of write amplification, and sets the rate limit to 0.75 of that so as to give the engine enough room to increase write amplification if needed. The algorithm is obviously pessimistic: it undervalues the transaction rate the database can handle after write amplification has steadied. But this is compensated by its simplicity and stability - there shouldn't be any abrupt drops or peaks in RPS due to its decisions. Besides, it adapts fairly quickly to increase in write amplification when a database is filled up. If one finds that the algorithm is being too cautious by undervaluing the limit, it's easy to fix by simply increasing the number of compaction threads - the rate limit will scale proportionately if the system is underloaded. The current value of the rate limit set by the algorithm is reported by box.stat.vinyl() under regulator.rate_limit section. Thanks to @kostja for the great comment explaining the logic behind the rate limiting algorithm. Closes #3721
Name | Last commit | Last update |
---|