vinyl: fix read stalled by write
The Vinyl read iterator, which is used for serving range select requests, works as follows: 1. Scan in-memory sources. If found an exact match or a chain in the cache, return it. 2. If not found, scan disk sources. This operation may yield. 3. If any new data was inserted into the active memory tree, go to step 1, effectively restarting the iteration from the same key. Apparently, such an algorithm doesn't guarantee any progress of a read operation at all - when we yield reading disk on step 2 after a restart, even newer data may be inserted into the active memory tree, forcing us to restart again. In other words, in presence of an intensive write workload, read ops rate may drop down to literally 0. It hasn't always been like so. Before commit 83462a5c ("vinyl: restart read iterator in case L0 is changed"), we only restored the memory tree iterator after a yield, without restarting the whole procedure. This makes sense, because only memory tree may change after a yield so there's no point in rescanning other sources, including disk. By restarting iteration after a yield, the above-mentioned commit fixed bug #3395: initially we assumed that statements may never be deleted from a memory tree while actually they can be deleted by rollback after a failed WAL write. Let's revert this commit to fix the performance degradation. We will re-fix bug #3395 in the next commit. Closes #5700 NO_DOC=bug fix NO_TEST=should be checked by performance tests
Loading
Please register or sign in to comment