Skip to content
Snippets Groups Projects
  1. Mar 01, 2018
    • Vladimir Davydov's avatar
      vinyl: don't start event loop for worker threads · 8cd06e6a
      Vladimir Davydov authored
      This is pointless as worker threads don't use coio.
      8cd06e6a
    • Vladislav Shpilevoy's avatar
      vinyl: introduce vy_run_writer · 2b7efc1e
      Vladislav Shpilevoy authored
      vy_run_writer incapsulates logic of writing statements to a run.
      It provides API to write statements one by one. It is needed so
      that we can abort a run writing task before waiting for it to
      finish writing the file.
      
      Edited by @locker:
       - add region_truncate() wherever necessary
       - do not reallocate row index ibuf for each page
       - remove vy_run_write(), use vy_run_writer directly instead
      
      Needed for #3166
      2b7efc1e
  2. Feb 20, 2018
    • Vladimir Davydov's avatar
      vinyl: delete runs compacted during join immediately · 9c446a20
      Vladimir Davydov authored
      We keep run files corresponding to (at least) the last snapshot, because
      we need them for backups and replication. Deletion of compacted run
      files is postponed until the next snapshot. As a consequence, we don't
      delete run files created on a replica during the join stage. However, in
      contrast to run files created during normal operation, these are pure
      garbage and should be deleted right away. Not deleting them can result
      in depletion of disk space, because vinyl has quite high write
      amplification by design.
      
      We can't write a functional test for this, because there's no way to
      guarantee that compaction started during join will finish before join
      completion - if it doesn't, compacted runs won't be removed, because
      they will be assigned to the snapshot created by join.
      
      Closes #3162
      9c446a20
  3. Dec 15, 2017
    • Vladimir Davydov's avatar
      vinyl: store pointer to vy_run_env in vy_run · bd6d138b
      Vladimir Davydov authored
      It's really annoying to pass vy_run_env along with vy_run or vy_slice
      every time we want to read a run. Let's store a pointer to vy_run_env
      immediately in vy_run. This is a widely accepted practice throughout
      vinyl - we already do this in case of vy_index, vy_cache, and vy_mem.
      bd6d138b
  4. Dec 01, 2017
    • Vladimir Davydov's avatar
      vinyl: improve logging · f69907db
      Vladimir Davydov authored
       - Use diag_log() or error_log() instead of printing error->errmsg
         whenever possible as they refer to the place where the error was
         initially raised, which may be useful.
      
       - Do not include the description of a transaction we failed to flush
         and so left in the buffer to be flushed later - this information is
         generally useless. Just print a warning that we failed to flush the
         vylog buffer to draw attention of the admin to possible IO problems.
         Consequently, remove the error code from vy_log_tx_try_commit().
      
       - Use vy_log_tx_try_commit() for writing VY_RUN_DISCARD and FORGET
         records. Currently, we use vy_log_tx_commit(), because at the time
         these methods were initially implemented, vy_log_tx_try_commit()
         simply didn't exist. Let's use vy_log_tx_try_commit() and remove the
         error messages emitted on vy_log_tx_commit() failure.
      
       - If we failed to load the vylog and failed replication or garbage
         collection or backup, print what exactly we failed to do. The error
         message is already printed by vy_recovery_new() so don't include it
         in the output.
      
       - Remove error message from gc_run() as engine callbacks already print
         error messages, no point to duplicate them. Actually, we usually
         print error messages from engine callbacks rather than engine
         wrappers.
      
       - Improve dump/checkpoint logging. Print when checkpoint started, when
         it ended, how many bytes were dumped, how long it took.
      
       - Print names of files written by worker threads (.run, .index). To
         differentiate messages from different worker threads, give unique
         names to them: vinyl.worker.0, 1, and so on. We already name reader
         threads in the same fashion.
      
       - Print names of files removed by the vinyl garbage collection
         callback. Also, makes messages look exactly like the ones printed by
         xdir_collect_garbage() (used by memtx, wal).
      
       - Use VERBOSE level for printing the message about scheduling a
         background fiber for squashing UPSERTs. There shouldn't be many of
         them so this should be fine.
      
       - Use INFO level for 'rebuilding index for ...' messages (currently,
         it's WARN for no reason). Also, print the file name of the index/data
         file we failed to load (currently, it may be omitted on certain
         errors, making identification of a missing or corrupted file nearly
         impossible).
      
       - Print the name of the run file we failed to read a page from to the
         log. Also include the page offset and size to the output. Full file
         path is not printed, because it's difficult to procure it from the
         error path, but I guess it's OK as each run file has a unique name
         and so can be easily found by its name. I guess it addresses #1972.
      f69907db
  5. Nov 16, 2017
    • Vladimir Davydov's avatar
      vinyl: complete dump if the index was dropped · ce1a3774
      Vladimir Davydov authored
      If the index was dropped during dump, we abort the dump task so as not
      to log information about zombie indexes to the vylog. However, we should
      still notify the scheduler that its memory can be released by calling
      vy_scheduler_complete_dump(). If we don't, lsregion won't be truncated
      until another index is dumped. If there is no other vinyl indexes out
      there or all other indexes have already been dumped, memory will never
      be freed.
      
      On debug builds, it will result in the following assertion failure:
      
        vy_scheduler.c:1319: vy_scheduler_peek_dump: Assertion `scheduler->dump_task_count > 0' failed.
      ce1a3774
  6. Nov 02, 2017
Loading