Skip to content
Snippets Groups Projects
  1. May 22, 2017
    • Ilya's avatar
      Optimize tuple_hash() for sequential keys · ceaee86a
      Ilya authored
      * Refactor templates in tuple_compare.cc to generate optimized
        versions for various field types
      * Add a function pointer in key_def to select optimized
        version at runtime
      
      Fixes #2084
      ceaee86a
    • Ilya's avatar
      Optimize tuple_extract_key() for sequential keys · 02f0710c
      Ilya authored
      * Implement tuple_extract_key_sequential() for sequential keys
      * Add a function pointer to in key_def to select optimized
        version at runtime
      
      Closes #2048
      02f0710c
  2. May 19, 2017
  3. May 18, 2017
  4. May 17, 2017
    • Vladimir Davydov's avatar
      vinyl: allow optional keys in vylog · ea0d6471
      Vladimir Davydov authored
      Currently, only keys from vy_log_key_mask[type] may be present in a
      record of the given type. Extra keys will result in an error. In
      order to rework space truncate, we will have to add a new key for
      VY_LOG_CREATE_INDEX record (space_version). To be able to start from
      vylog generated by an older version, we need to allow extra keys. So
      this patch deletes vy_log_key_mask[] and makes vy_log_record_encode()
      encode only those keys whose value is different from the default one
      (similarly to request_encode()). Checking for mandatory keys is now up
      to vy_recovery_process_record() (currently there's the only that needs
      to be checked - VY_LOG_KEY_DEF)
      ea0d6471
    • Vladimir Davydov's avatar
      vinyl: don't use data left from previous vylog records · cd4ed98b
      Vladimir Davydov authored
      vy_recovery_iterate() doesn't clean vy_log_record before proceeding to
      the next log entry. As a result, a run record passed to the callback by
      vy_recovery_iterate() contains extra info left from the index this run
      is for: index_lsn, index_id and space_id. We use this in gc and backup
      callbacks to format file name. The problem is vy_recovery_iterate() is
      also used internally for log rotation. Including extra keys in records
      doesn't result in writing them to the log file on rotation, because per
      each record type we have a mask of keys corresponding to the record
      (vy_log_key_mask). In order to allow optional keys in vylog, the
      following patch will change the meaning of the mask so that it only
      contains mandatory keys, while a key will be written to the log only if
      its value differs from the default (similarly to request_encode). Thus,
      to avoid writing keys not relevant to a record type we need to clean
      vy_log_record within vy_recovery_iterate() before jumping to the next
      record. So this patch weans gc and backup from exploiting this feature -
      let them save index_id and space_id in the context, as we do on
      replication and recovery.
      cd4ed98b
    • Vladislav Shpilevoy's avatar
      Implement info_append_double · 24707c0c
      Vladislav Shpilevoy authored
      Info_append_double is needed to print vinyl index.info.bloom_fpr later.
      24707c0c
    • Vladimir Davydov's avatar
      vinyl: zap vy_index::path · 124bac9a
      Vladimir Davydov authored
      It was helpful when a vinyl index could have a custom path. Currently,
      it's forbidden, so we can format index path in place.
      124bac9a
    • Vladimir Davydov's avatar
      vinyl: zap vy_index::name · ae80dbc1
      Vladimir Davydov authored
      We can format it in place when needed.
      ae80dbc1
    • Vladimir Davydov's avatar
      vinyl: introduce quota timeout · 70a610d1
      Vladimir Davydov authored
      There must be a reasonable timeout on quota wait time, otherwise we risk
      throttling a client forever, e.g. in case of a disk error. This patch
      introduces quota timeout. The timeout is configured via vinyl_timeout
      configuration option and set to 60 seconds by default.
      
      Closes #2014
      70a610d1
    • Vladimir Davydov's avatar
      vinyl: split quota callback function · 883f16a7
      Vladimir Davydov authored
      Use separate function for each kind of callback. The next patch will add
      an argument and a return value to the throttle callback, so it isn't apt
      anymore to use the only callback to handle all cases.
      883f16a7
    • Vladimir Davydov's avatar
      vinyl: reserve quota before using memory · c8ea5fc7
      Vladimir Davydov authored
      Currently, we reserve quota after allocating memory, which can result in
      exceeding the memory limit, e.g. after the following script
      
          box.cfg{vinyl_memory = 1024 * 1024}
          s = box.schema.space.create('test', {engine = 'vinyl'})
          s:create_index('pk')
          pad = string.rep('x', 2 * box.cfg.vinyl_memory / 3)
          s:auto_increment{pad}
          s:auto_increment{pad}
      
      is done, box.info.vinyl().memory.used reports that 1447330 bytes are
      allocated. Fix this by reserving quota before allocation. A test is
      added later in the series.
      c8ea5fc7
    • Vladimir Davydov's avatar
      vinyl: add test checking that we don't exceed quota on recovery · c06fc47a
      Vladimir Davydov authored
      Vinyl dump is disabled during local recovery from WAL. This is OK,
      because we shouldn't exceed the quota provided we don't replay
      statements that were dumped to disk before restart. Check that.
      c06fc47a
  5. May 16, 2017
  6. May 15, 2017
    • Vladimir Davydov's avatar
      vinyl: fix index->stmt_count incremented twice on upsert · 13dd8a41
      Vladimir Davydov authored
      Although upsert optimizers (both sync and background) replace the
      last UPSERT with a REPLACE and do not insert new statements, they
      use vy_index_insert_stmt(), which increments index->stmt_count.
      As a result, index->stmt_count is incremented twice.
      
      Closes #2421
      13dd8a41
  7. May 12, 2017
    • Roman Tsisyk's avatar
    • Roman Tsisyk's avatar
      Disable buggy test/unit/guard.test · 808fe987
      Roman Tsisyk authored
      See #2429
      808fe987
    • Vladislav Shpilevoy's avatar
      1dcc60c0
    • Roman Tsisyk's avatar
      Debian: bump tarantool-common dependency · a89cc747
      Roman Tsisyk authored
      Force tarantool-common upgrade to support systemd notifications.
      
      + Add missing "s" suffix to TimeoutStartSec= option.
      
      See #1923
      a89cc747
    • Roman Tsisyk's avatar
      vinyl: add a workaround for cache · 2f8e0204
      Roman Tsisyk authored
      Check that there are no statements between prev_stmt and stmt
      in the cache on vy_cache_add() before trying to build a chain.
      This workaround makes vy_cache_add() more fool-proof for cases
      when vy_read_iterator skips some keys during restoration.
      2f8e0204
    • Vladislav Shpilevoy's avatar
      vinyl: fix error in mem_restore on prepared statement · a0db136e
      Vladislav Shpilevoy authored
      Between prepare and commit of a transaction it is possible
      that some prepared statements are read from mem.
      Prepared statement has abnomal lsn > MAX_LSN.
      
      If version of the index is changed after return of a
      prepared statement and the iterator is placed to read view,
      the restart and restore on a such prepared statement could
      lead to restore on LSN bigger than vlsn.
      
      Example:
      
              FIBER 1                        FIBER 2
             box.begin()
       replace {1}, lsn=MAX_LSN+1
       replace {2}, lsn=MAX_LSN+1
          prepare vy_tx ...
                              ->->->      open iterator
                                    read {1},lsn=MAX_LSN+1 from mem
                                           yield ...
                              <-<-<-
          commit vy_tx
       replace {1, 1},lsn=100
      send iterator to read view
                              ->->->
                                       iterator read view = 100
                                index version changed - restore on last_stmt
                                    last_stmt = {1},lsn=MAX_LSN+1
                                    last_stmt LSN > iterator VLSN
                                       ??????????????????????
      
      Lets return in such situations the next key.
      a0db136e
    • Vladislav Shpilevoy's avatar
      vinyl: fix mem restore in case of not started mem iterator · d21971cd
      Vladislav Shpilevoy authored
      If vy_mem_iterator was not started, the vy_mem_iterator_restore had been
      able to iterate to the same key, as the target of restore, and then call
      vy_mem_iterator_next_lsn_impl to find the older lsn.
      
      But the vy_mem_iterator_start_from did not set 'search_started' flag and
      next_lsn_impl would restart the mem iterator again, to the first statement,
      which could be incorrect result of restore.
      
      Example:
      
      cache statements: {1}, {2}, {3}
      mem statements: {1}, {2}, {3}
      iterator: GE
      key: {1}
      
      - merge_iterator returns all cached data and then tries to restore mem on {3}.
      - then mem_iterator during restore calls vy_mem_iterator_start_from and sees tuple {3}.
      - then mem_iterator tries to find older lsn and calls vy_mem_iterator_next_lsn_impl,
        which sees 'search_started = false' and restarts iterator to {1}.
      - then read_iterator returns {1} after {3} - ERROR!
      
      Lets set 'search_started' flag inside vy_mem_iterator_start_from.
      d21971cd
    • Vladislav Shpilevoy's avatar
      vinyl: fix return of same statements from read_iterator · a03a4a99
      Vladislav Shpilevoy authored
      Under heavy load it is possible, that mem_iterator_restore
      inside read_iterator_next returns statement with the same key,
      as the previous one.
      
      Check keys in vy_read_iterator_merge_next_key.
      a03a4a99
    • Vladislav Shpilevoy's avatar
      vinyl: fix restore of started mem_iterator to not null last_stmt · 5f5b63d0
      Vladislav Shpilevoy authored
      If the mem_iterator is started, but hasn't any curr_stmt (maybe it
      was finished), it could return its first statement regardless of last_stmt.
      
      Lets call start_from() instead of start() in case of not null last_stmt.
      5f5b63d0
    • Alexandr Lyapunov's avatar
      vinyl: fix restoration of mem iterator and avoid unnecessary restore calls. · 9f341871
      Alexandr Lyapunov authored
      Mem iterator restoration in case of LE and LT was completelly wrong.
      Additionally I added more asserts and found that restoration is
      called too frequently.
      Fix'em both.
      
      fix #2207
      9f341871
    • bigbes's avatar
    • bigbes's avatar
      Adding table.deepcopy() · b1d805d3
      bigbes authored
      b1d805d3
    • Vladimir Davydov's avatar
      vinyl: fix quota watermark calculation · a74219d5
      Vladimir Davydov authored
      The code calculating the quota watermark was written long ago, when we
      didn't have the common memory level and didn't use lsregion allocator
      and hence it was possible to free all memory used by any range. Things
      have changed drastically since then: now it's impossible to free memory
      occupied by a range or even an index, because statements are shared
      between indexes, so the scheduler effectively dumps all memory on
      exceeding the quota. Fix the quota calculation accordingly.
      a74219d5
    • Vladimir Davydov's avatar
      vinyl: yield properly · aa0ebf46
      Vladimir Davydov authored
      We should use fiber_sleep(0) to yield periodically, not
      fiber_reschedule() - the latter doesn't advance the event loop.
      aa0ebf46
    • Roman Tsisyk's avatar
      03882982
    • Georgy Kirichenko's avatar
      Unit test for fibers stack guards · 826ccd20
      Georgy Kirichenko authored
      826ccd20
    • Vladimir Davydov's avatar
      vinyl: add test for partial dump · e5f6f0d1
      Vladimir Davydov authored
      Check that we correctly recover in case not all indexes of the same
      space were dumped.
      e5f6f0d1
    • Vladimir Davydov's avatar
      vinyl: increment ref counter of pinned index · 40989528
      Vladimir Davydov authored
      We remove the primary index from the scheduler while we are dumping a
      secondary index of the same space ('pin' it). On dump task completion we
      add it back. By that time, the primary index could have been dropped, in
      which case we may get use-after-free in vy_scheduler_unpin_index(). To
      avoid it we should increment a reference counter of a pinned index.
      40989528
    • Vladimir Davydov's avatar
      errinj: ERRINJ_U64 => ERRINJ_INT · ff064d10
      Vladimir Davydov authored
      I need to set errinj parameter to a negative value (errinj.param < 0 -
      errinj is disabled, errinj.param = X >= 0 - errinj is enabled for X).
      I don't think it's a good idea to introduce int64_t in addtion to
      uint64_t, because int64_t should be enough for error injection. So,
      let's change ERRINJ_U64 (and errinj.u64param) to ERRINJ_INT
      (errinj.iparam) of type int64_t.
      ff064d10
Loading