Skip to content
Snippets Groups Projects
Commit f927a87b authored by Vladimir Davydov's avatar Vladimir Davydov Committed by Roman Tsisyk
Browse files

vinyl: rework dump trigger

Currently, dump is triggered (by bumping the memory generation) by the
scheduler fiber while quota consumers just wake it up. As a result, the
scheduler depends on the quota - it has to access the quota to check if
it needs to trigger dump. In order to move the scheduler to a separate
source file, we need to get rid of this dependency.

Let's rework this code as follows:

 - Remove vy_scheduler_trigger_dump() from vy_scheduler_peek_dump(). The
   scheduler fiber now just dumps all indexes eligible for dump and
   completes dump by bumping dump_generation. It doesn't trigger dump by
   bumping generation anymore. As a result, it doesn't need to access
   the quota.

 - Make quota consumers call vy_scheduler_trigger_dump() instead of just
   waking up the scheduler. This function will become a public one once
   the scheduler is moved out of vinyl.c. The function logic is changed
   a bit. First, besides bumping generation, it now also wakes up the
   scheduler fiber. Second, it does nothing if dump is already in
   progress or can't be scheduled because of concurrent checkpoint.
   In the latter case it sets a special flag though that will force the
   scheduler trigger dump upon checkpoint completion.

 - vy_scheduler_begin_checkpoint() can't use vy_scheduler_trigger_dump()
   anymore due to additional checks added to the function, so it bumps
   the generation directly. This looks fine.

 - Such a design has a subtlety regarding how quota consumers notify the
   scheduler and how they are notified back about available quota.
   In extreme cases, quota released by a dump may be not enough to
   satisfy all consumers, in which case we need to reschedule dump.
   Since the scheduler doesn't check the quota anymore and doesn't
   reschedule dump, it has to be done by the left consumers. So
   consumers has to call the quota_exceeded_cb (which triggers a dump
   now) callback every time they are woken up and see there's not enough
   quota. The vy_quota_use() is reworked accordingly.

   Also, since the quota usage may exceed the limit (because of
   vy_quota_force_use()), the quota usage may remain higher than the
   limit after a dump completion, in which case vy_quota_release()
   doesn't wake up consumers and again there's no one to trigger another
   dump. So we must wake up all consumers every time vy_quota_release()
   is called.
parent 0d751d84
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment