vinyl: rework dump trigger
Currently, dump is triggered (by bumping the memory generation) by the scheduler fiber while quota consumers just wake it up. As a result, the scheduler depends on the quota - it has to access the quota to check if it needs to trigger dump. In order to move the scheduler to a separate source file, we need to get rid of this dependency. Let's rework this code as follows: - Remove vy_scheduler_trigger_dump() from vy_scheduler_peek_dump(). The scheduler fiber now just dumps all indexes eligible for dump and completes dump by bumping dump_generation. It doesn't trigger dump by bumping generation anymore. As a result, it doesn't need to access the quota. - Make quota consumers call vy_scheduler_trigger_dump() instead of just waking up the scheduler. This function will become a public one once the scheduler is moved out of vinyl.c. The function logic is changed a bit. First, besides bumping generation, it now also wakes up the scheduler fiber. Second, it does nothing if dump is already in progress or can't be scheduled because of concurrent checkpoint. In the latter case it sets a special flag though that will force the scheduler trigger dump upon checkpoint completion. - vy_scheduler_begin_checkpoint() can't use vy_scheduler_trigger_dump() anymore due to additional checks added to the function, so it bumps the generation directly. This looks fine. - Such a design has a subtlety regarding how quota consumers notify the scheduler and how they are notified back about available quota. In extreme cases, quota released by a dump may be not enough to satisfy all consumers, in which case we need to reschedule dump. Since the scheduler doesn't check the quota anymore and doesn't reschedule dump, it has to be done by the left consumers. So consumers has to call the quota_exceeded_cb (which triggers a dump now) callback every time they are woken up and see there's not enough quota. The vy_quota_use() is reworked accordingly. Also, since the quota usage may exceed the limit (because of vy_quota_force_use()), the quota usage may remain higher than the limit after a dump completion, in which case vy_quota_release() doesn't wake up consumers and again there's no one to trigger another dump. So we must wake up all consumers every time vy_quota_release() is called.
Loading
Please register or sign in to comment