vinyl: be pessimistic about write rate when setting dump watermark
We set the dump watermark using the following formula limit - watermark watermark ---------------- = -------------- write_rate dump_bandwidth This ensures that by the time we run out of memory quota, memory dump will have been completed and we'll be able to proceed. Here the write_rate is the expected rate at which the workload will write to the database while the dump is in progress. Once the dump is started, we throttle the workload in case it exceeds this rate. Currently, we estimate the write rate as a moving average observed for the last 5 seconds. This performs poorly unless the workload write rate is perfectly stable: if the 5 second average turns out to be even slightly less than the max rate, the workload may experience long stalls during memory dump. To avoid that let's use the max write rate multiplied by 1.5 instead of the average when setting the watermark. This means that we will start dump earlier than we probably could, but at the same time this will tolerate write rate fluctuations thus minimizing the probability of stalls. Closes #4166
Loading
Please register or sign in to comment