Skip to content
Snippets Groups Projects
  1. Nov 16, 2017
    • Vladimir Davydov's avatar
      vinyl: decompose read iterator merge procedure · 023ab35f
      Vladimir Davydov authored
      The generic approach trying to build the merge procedure around the
      vy_stmt_iterator interface didn't pan out, because sources are way too
      different: in contrast to other sources, the cache stores intervals;
      run iterators may yield; txw does not preserve statement history.
      
      Let's rewrite vy_read_iterator_next_{key,lsn} in such a way that they
      do not use this generic interface. This results in a quite bit of code
      being duplicated, because loops over sources are unrolled, but this is
      intentional - hopefully it makes the code easier to follow. The patch
      isn't supposed to change the merge algorithm or remove any optimization
      implemented in it.
      023ab35f
    • Vladimir Davydov's avatar
      Partially revert "vinyl: reference iterator keys" · 705ca361
      Vladimir Davydov authored
      This reverts commit be8ee29a.
      
      Taking a reference to the search key in source iterators is pointless -
      it can't go away while we are using them.
      
      The only part of this patch that makes sense is removing the const
      specifier from vy_point_iterator->key.
      705ca361
    • Vladimir Davydov's avatar
      Add test for vinyl statistics · 1456af17
      Vladimir Davydov authored
      Closes #2558
      1456af17
    • Ivan Kosenko's avatar
      8ff671d4
  2. Nov 15, 2017
    • Vladimir Davydov's avatar
      vinyl: log reads that take too long · af63fcbe
      Vladimir Davydov authored
      If read of a single statement from vinyl takes more than the value of
      box.cfg.too_long_threshold, the request will be logged:
      
        512/1: select([1], EQ) => REPLACE([100001, 1], lsn=200006) took too long: 0.626 sec
      
      This is useful for debugging.
      
      While we are at it, let's also remove 'timeout' from the vinyl engine
      constructor arguments and set it with box_set_vinyl_timeout() on box
      initialization instead, similarly to vinyl_max_tuple_size.
      
      Closes #2871
      af63fcbe
    • Vladimir Davydov's avatar
      vinyl: zap vy_key_snprint() and vy_key_str() · 41e08261
      Vladimir Davydov authored
      Use generic tuple_snprint() and tuple_str() instead.
      41e08261
    • Vladimir Davydov's avatar
      vinyl: merge vy_cursor and vinyl_iterator · 54f9d3b7
      Vladimir Davydov authored
      The vinyl_iterator struct was introduced as a C++ wrapper around
      vy_cursor. Since there's no C++ code left in the engine, and both
      structures are defined in the same file, we can merge them now.
      54f9d3b7
    • Vladimir Davydov's avatar
      vinyl: remove engine wrapper functions · f5fca22f
      Vladimir Davydov authored
      The engine infrastructure was initially implemented in C++ so we
      needed the wrappers to provide C++ API to Vinyl. Now everything is
      in C so we don't need them any more. Let's fold them in vinyl.c.
      
      Note, this patch does not touch vinyl_engine, vinyl_index, and
      vinyl_iterator structures, they are still there, it just gets rid
      of the intermediate layer of wrapper functions, which is not needed
      any more.
      f5fca22f
    • Vladimir Davydov's avatar
      vinyl: pass force_recovery on engine initialization · 85fd9907
      Vladimir Davydov authored
      Accessing configuration from inside an engine implementation
      violates encapsulation.
      85fd9907
    • Roman Tsisyk's avatar
  3. Nov 13, 2017
  4. Nov 10, 2017
  5. Nov 06, 2017
  6. Nov 04, 2017
  7. Nov 03, 2017
    • Vladimir Davydov's avatar
      Fix compilation on Mac OS · e9c7a713
      Vladimir Davydov authored
      > src/tarantool/src/box/vinyl.c:2111:33: error: initializer element is not a compile-time constant
      >         static const double weight = 1 - exp(-VY_QUOTA_UPDATE_INTERVAL /
      >                                      ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      
      Remove the "static" qualifier. It is not really needed as any sane
      compiler will pre-calculate the value of 'weight' at compile time
      (checked on gcc 6.3.0 with -O0).
      e9c7a713
    • Vladimir Davydov's avatar
      vinyl: cleanup global statistics · 64a4b538
      Vladimir Davydov authored
       - Remove vy_stat::rmean statistics, which were left from Sophia, as now
         we have per index statistics which are much more verbose than those.
      
       - Move vy_stat::dump_bw to vy_env and remove struct vy_stat as there's
         nothing left in it.
      
       - Move quota statistics from box.info.vinyl().performance.memory to
         box.info.vinyl().quota. Remove 'ratio', which equals used / limit, as
         this kind of calculations should be done by a script aggregating
         statistics. Report 'use_rate' and 'dump_bandwidth' there.
      
       - Report 'limit' in cache statistics to make them consistent with
         'quota' statistics, where 'limit' is reported. Rename 'cache.count'
         to 'cache.tuples'. Remove vy_cache_env::cache_count, use mempool
         stats instead.
      
       - Move 'tx_allocated', 'txv_allocated', 'read_interval', 'read_view'
         from box.info.vinyl().performance to box.info.vinyl().tx and name
         them 'transactions', 'statements', 'gap_locks', and 'read_views',
         respectively. Remove vy_tx_stat::active and 'tx.active' as the same
         value is shown by 'tx.transactions', extracted from the mempool.
      
       - Zap box.info.vinyl().performance - there's nothing left there.
      
      Now global statistics look like:
      
        tarantool> box.info.vinyl()
        ---
        - cache:
            limit: 134217728
            tuples: 32344
            used: 34898794
          tx:
            conflict: 1
            commit: 324
            rollback: 13
            statements: 10
            transactions: 3
            gap_locks: 4
            read_views: 1
          quota:
            dump_bandwidth: 10000000
            watermark: 119488351
            use_rate: 1232703
            limit: 134217728
            used: 34014634
        ...
      
      Closes #2861
      64a4b538
    • Vladimir Davydov's avatar
      vinyl: do not use rmean for calculating quota use rate · 5e28b70a
      Vladimir Davydov authored
      We have a timer for updating watermark every second. Let's reuse it for
      quota use rate calculation. This will allow us to get rid of legacy
      vinyl statistics.
      
      Also, let's use EWMA for calculating the average. It is a more efficient
      and common method, which allows to easily tune the period over which the
      value is averaged.
      5e28b70a
    • Roman Tsisyk's avatar
      box: add box.NULL alias for msgpack.NULL · 043ba278
      Roman Tsisyk authored
      Follow up #1557
      043ba278
  8. Nov 02, 2017
    • Alexandr Lyapunov's avatar
      Reject attempts to create non-string index part with collation · 4c10b711
      Alexandr Lyapunov authored
      Collation was simply ignored for non-string parts, that could
      confuse potential user.
      
      Generate a readable error in this case.
      
      Fix #2862 part 2
      4c10b711
    • Alexandr Lyapunov's avatar
      Make collation work with scalar fields · 2601fcd3
      Alexandr Lyapunov authored
      Now collation is silently ignored for type='scalar' parts.
      
      Use collation for string scalar fields.
      
      Fix #2862 part 1
      2601fcd3
    • Alexandr Lyapunov's avatar
      Show collation in lua index object · 76b6e110
      Alexandr Lyapunov authored
      Show collation name (if present) in space.index.name.parts[no].
      
      Fix #2862 part 4
      76b6e110
    • Alexandr Lyapunov's avatar
      Make collation by name lookup case insensitive · 3f9a73a7
      Alexandr Lyapunov authored
      test:create_index('unicode_s1', {parts = {{1, 'STR', collation =
      'UNICODE'}}}) will work now.
      
      Fix #2862 part 3
      3f9a73a7
    • Vladislav Shpilevoy's avatar
      schema: allow to store smaller field count that specified in format · 2f53308e
      Vladislav Shpilevoy authored
      If a field is not indexed and no more indexed or not nullable
      fields after that, than allow to skip it in insertion. Such field
      value looks like MP_NIL, but MP_NIL is not explicitly stored.
      Named access to this field in lua returns nil.
      
      Example:
      format =
      {{'field1'},
       {'field2'},
       {'field3', is_nullable = true},
       {'field4', is_nullable = true}}
      
      t = space:insert{1, 2} -- ok.
      
      t.field1 == 1, t.field2 == 2, t.field3 == nil, t.field4 == nil
      
      Closes #2880
      2f53308e
    • Vladislav Shpilevoy's avatar
      schema: allow to store custom fields in format's field definition · f688ef36
      Vladislav Shpilevoy authored
      Some users store in format fields their custom keys. But current
      opts parser does not allow to store any unknown keys. Lets allow it.
      
      Example:
      format = {}
      format[1] = {name = 'field1', type = 'unsigned', custom_field = 'custom_value'}
      s = box.schema.create_space('test', {format = format})
      s:format()[1].custom_field == 'custom_value'
      
      Closes #2839
      f688ef36
    • Vladimir Davydov's avatar
      vinyl: forbid DDL/DML if wal is disabled · b75ab0e0
      Vladimir Davydov authored
      Using DML/DDL on a Vinyl index with wal_mode = 'none' is likely to
      result in unrecoverable errors like:
      
        F> can't initialize storage: Invalid VYLOG file: Index 512/0 created twice
      
      To avoid data corruption in case the user tries to use an existing Vinyl
      database in conjunction with wal_mode = 'none', let's explicitly forbid
      it until we figure out how to fix it.
      
      Workaround #2278
      b75ab0e0
Loading