- Mar 01, 2018
-
-
Vladimir Davydov authored
This is pointless as worker threads don't use coio.
-
Vladislav Shpilevoy authored
vy_run_writer incapsulates logic of writing statements to a run. It provides API to write statements one by one. It is needed so that we can abort a run writing task before waiting for it to finish writing the file. Edited by @locker: - add region_truncate() wherever necessary - do not reallocate row index ibuf for each page - remove vy_run_write(), use vy_run_writer directly instead Needed for #3166
-
- Feb 20, 2018
-
-
Vladimir Davydov authored
We keep run files corresponding to (at least) the last snapshot, because we need them for backups and replication. Deletion of compacted run files is postponed until the next snapshot. As a consequence, we don't delete run files created on a replica during the join stage. However, in contrast to run files created during normal operation, these are pure garbage and should be deleted right away. Not deleting them can result in depletion of disk space, because vinyl has quite high write amplification by design. We can't write a functional test for this, because there's no way to guarantee that compaction started during join will finish before join completion - if it doesn't, compacted runs won't be removed, because they will be assigned to the snapshot created by join. Closes #3162
-
- Dec 15, 2017
-
-
Vladimir Davydov authored
It's really annoying to pass vy_run_env along with vy_run or vy_slice every time we want to read a run. Let's store a pointer to vy_run_env immediately in vy_run. This is a widely accepted practice throughout vinyl - we already do this in case of vy_index, vy_cache, and vy_mem.
-
- Dec 01, 2017
-
-
Vladimir Davydov authored
- Use diag_log() or error_log() instead of printing error->errmsg whenever possible as they refer to the place where the error was initially raised, which may be useful. - Do not include the description of a transaction we failed to flush and so left in the buffer to be flushed later - this information is generally useless. Just print a warning that we failed to flush the vylog buffer to draw attention of the admin to possible IO problems. Consequently, remove the error code from vy_log_tx_try_commit(). - Use vy_log_tx_try_commit() for writing VY_RUN_DISCARD and FORGET records. Currently, we use vy_log_tx_commit(), because at the time these methods were initially implemented, vy_log_tx_try_commit() simply didn't exist. Let's use vy_log_tx_try_commit() and remove the error messages emitted on vy_log_tx_commit() failure. - If we failed to load the vylog and failed replication or garbage collection or backup, print what exactly we failed to do. The error message is already printed by vy_recovery_new() so don't include it in the output. - Remove error message from gc_run() as engine callbacks already print error messages, no point to duplicate them. Actually, we usually print error messages from engine callbacks rather than engine wrappers. - Improve dump/checkpoint logging. Print when checkpoint started, when it ended, how many bytes were dumped, how long it took. - Print names of files written by worker threads (.run, .index). To differentiate messages from different worker threads, give unique names to them: vinyl.worker.0, 1, and so on. We already name reader threads in the same fashion. - Print names of files removed by the vinyl garbage collection callback. Also, makes messages look exactly like the ones printed by xdir_collect_garbage() (used by memtx, wal). - Use VERBOSE level for printing the message about scheduling a background fiber for squashing UPSERTs. There shouldn't be many of them so this should be fine. - Use INFO level for 'rebuilding index for ...' messages (currently, it's WARN for no reason). Also, print the file name of the index/data file we failed to load (currently, it may be omitted on certain errors, making identification of a missing or corrupted file nearly impossible). - Print the name of the run file we failed to read a page from to the log. Also include the page offset and size to the output. Full file path is not printed, because it's difficult to procure it from the error path, but I guess it's OK as each run file has a unique name and so can be easily found by its name. I guess it addresses #1972.
-
- Nov 16, 2017
-
-
Vladimir Davydov authored
If the index was dropped during dump, we abort the dump task so as not to log information about zombie indexes to the vylog. However, we should still notify the scheduler that its memory can be released by calling vy_scheduler_complete_dump(). If we don't, lsregion won't be truncated until another index is dumped. If there is no other vinyl indexes out there or all other indexes have already been dumped, memory will never be freed. On debug builds, it will result in the following assertion failure: vy_scheduler.c:1319: vy_scheduler_peek_dump: Assertion `scheduler->dump_task_count > 0' failed.
-
- Nov 02, 2017
-
-
Vladimir Davydov authored
The only reason why it was allocated is that struct vy_scheduler was defined after struct vy_env, which is not a problem any more. Embedding it allows us to drop the extra argument to vy_scheduler_need_dump_f().
-
Vladimir Davydov authored
It's a big independent entity, let's isolate its code in a separate file. While we are at it, add missing comments to vy_scheduler struct members.
-