vinyl: write index dump lsn to metadata log
When replaying local WAL, we filter out statements that were written to disk before restart by checking stmt->lsn against run->max_lsn: if the latter is greater, the statement was dumped. Although it is undoubtedly true, this check isn't quite correct. The thing is run->max_lsn might be less that the actual lsn at the time the run was dumped, because max_lsn is computed as the maximum among all statements present in the run file, which doesn't include deleted statements. If this happens, we might replay some statements for nothing: they will cancel each other anyway. This may be dangerous, because the number of such statements can be huge. Suppose, a whole run consists of deleted statements, i.e. there's no run file at all. Then we replay all statements in-memory, which might result in OOM, because the scheduler isn't started until the local recovery is completed. To avoid that, introduce a new record type in the metadata log, VY_LOG_DUMP_INDEX, which is written on each index dump, even if no file is created, and contains the LSN of the dump. Use this LSN on recovery to detect statements that don't need to be replayed.
Loading
Please register or sign in to comment