Skip to content
Snippets Groups Projects
  1. Apr 13, 2022
    • Vladimir Davydov's avatar
      box: separate access check and function call in box_process_call · 52fd97ec
      Vladimir Davydov authored
      box_process_call() uses func_call(), which not only calls the given
      function, but also checks that the current user has the right to execute
      it. As a result, we can't add auditing for only those function calls
      that passed the access check (apparently, there's no reason to log
      function calls that failed with an 'access denied' error - we have a
      separate audit event for this).
      
      To fix this, let's introduce func_call_no_access_check() helper, which
      calls a function without checking access rights, and use it along with
      existing func_access_check() in box_process_call(). func_call() is now
      an inline function that calls func_access_check() and then on success
      func_call_no_access_check().
      
      It's probably wrong that func_call() checks access rights, because this
      means that to use a space with a functional index/constraint, the user
      needs not only read/write access to the space itself, but also execute
      access to the function. I think we should check the right to execute
      such function only once - on functional index/constraint creation, not
      on every call, but I'm not going to change this now, because nobody's
      complained so far, and a change like this needs a proper discussion
      anyway.
      
      NO_TEST=refactoring
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      52fd97ec
    • Vladimir Davydov's avatar
      console: add internal on_console_eval trigger · d0ce4c9a
      Vladimir Davydov authored
      The trigger is invoked on console eval. The trigger callback is passed
      the eval expression string. It will be used for auditing console events
      in the EE version.
      
      NO_TEST=ee
      NO_DOC=internal
      NO_CHANGELOG=internal
      d0ce4c9a
    • Nikita Pettik's avatar
      lua: introduce interface for prbuf · c53a45ec
      Nikita Pettik authored
      Simply add Lua wrappers for recently introduced prbuf.
      
      Introduce new buffer (in addition to ibuf) - prbuf. "pr" stands for
      "partitioned ring-". It save all metadata in the same memory chunk
      provided for storage, so it can be completely restored from the 'raw'
      memory. API:
      
      ```
      -- mem is a chunk of raw (char *) memory, of size mem_size.
      -- It is used for data storage. Note that available space is of less
      -- size due to prbuf metadata overhead.
      -- Returns handle to prbuf.
      --
      require('buffer').prbuf_create(mem, mem_size)
      
      -- mem is a chunk of memory, which contains already created prbuf.
      -- It implies that once prbuf_create() was called with the same memory.
      -- If mem does not contain valid buffer - raise an appropriate error.
      require('buffer').prbuf_open(mem)
      
      -- Returns continuous chunk of memory with given size. May return nil
      -- in case if requested chunk is too large. Note that after copying
      -- object to returned chunk, it should be committed with prbuf:commit();
      -- otherwise :prepare() may return the same chunk twice.
      prbuf:prepare(size)
      
      -- Commits the last prepared chunk. Undefined behaviour in case of
      -- committing the same chunk twice.
      prbuf:commit()
      
      -- Create and return prbuf_iterator. Does not fail. Created iterator
      -- points to nowhere - it should be adjusted with :next() call to
      -- the first entry.
      prbuf:iterator_create()
      
      -- Advance iterator position. Returns prbuf_entry or nil in case
      -- iterator has reached the end. Entry consists of two members:
      -- size and ptr. The last one is an array of characters of given size.
      iterator:next()
      ```
      
       Usage examples:
      
      ```
      
      local ibuf = buffer.ibuf()
      local prbuf_size = 100
      local memory = ibuf:alloc(prbuf_size)
      local prbuf = buffer.prbuf_create(memory, prbuf_size)
      
      local sample_size = 4
      local raw = prbuf:prepare(4)
      if raw == nil then
          -- allocation size is too large, try smaller.
      end
      raw[0] = ...
      ...
      raw[3] = ...
      prbuf:commit()
      local prbuf_recovered = buffer.prbuf_open(memory)
      local iter = prbuf_recovered:iterator_create()
      local entry = iter:next()
      assert(tonumber(entry.size) == 4)
      -- Check values stored in the buffer.
      assert(entry.ptr[0] == ...)
      entry = iter:next()
      -- Our buffer has only one entry.
      assert(entry == nil)
      
      ```
      
      NO_DOC=<Feature for internal usage>
      NO_CHANGELOG=<Feature for internal usage>
      c53a45ec
    • Vladimir Davydov's avatar
      cfg: change default value of audit_format to json · 4d14961d
      Vladimir Davydov authored
      Required for backward compatibility.
      
      Follow-up commit a303b53b ("cfg: add audit_format option").
      
      NO_DOC=ee
      NO_CHANGELOG=ee
      4d14961d
    • Nikita Pettik's avatar
      core: introduce prbuf · 686989f8
      Nikita Pettik authored
      prbuf is partitioned ring buffer. The main property of the buffer is
      that it can be recovered from raw memory. To achieve this buffer saves
      metadata before each stored entry. For further details see source code.
      
      NO_DOC=<Private data structure>
      NO_CHANGELOG=<No user visible changes>
      686989f8
  2. Apr 11, 2022
    • Andrey Saranchin's avatar
      txm: introduce memtx mvcc memory monitoring · 5a99429f
      Andrey Saranchin authored
      This patch introduces memtx_tx_region and memtx_tx_mempool:
      engineers must use only these proxies to collect statistics.
      
      Also this patch introduces box.stat.memtx.mvcc - the way to
      get memtx mvcc memory statistics.
      
      Closes #6150
      
      @TarantoolBot document
      Title: Memtx MVCC memory monitoring
      
      Introduce memtx MVCC memory monitoring. One can get it with
      box.stat.memtx.tx() method or use index to access a particular
      statistic. The statistics format:
      
      txn:
        statements:
          max: 0
          avg: 0
          total: 0
        user:
          max: 0
          avg: 0
          total: 0
        system:
          max: 0
          avg: 0
          total: 0
      mvcc:
        trackers:
          max: 0
          avg: 0
          total: 0
        conflicts:
          max: 0
          avg: 0
          total: 0
        tuples:
          tracking:
            stories:
              count: 0
              total: 0
            retained:
              count: 0
              total: 0
          used:
            stories:
              count: 0
              total: 0
            retained:
              count: 0
              total: 0
          read_view:
            stories:
              count: 0
              total: 0
            retained:
              count: 0
              total: 0
      5a99429f
    • Andrey Saranchin's avatar
      txn: collect txn allocation statistics · 5e8981ce
      Andrey Saranchin authored
      Since transaction can allocate objects on its own, we need to track it
      to collect memory statistics of memtx txm. Txn's use only region
      allocator so this patch introduces tx_region methods - txns should use
      it instead of region methods to collect allocation statistics.
      
      Part of #6150
      
      NO_DOC=no visible changes
      NO_CHANGELOG=no visible changes
      NO_TEST=already covered
      5e8981ce
    • Andrey Saranchin's avatar
      txm: relax conditions for memtx_story delete · 2245ef06
      Andrey Saranchin authored
      Since a transaction has been prepared its garbage (produced stories
      and replaced tuples) cannot be deleted because they are recognized
      as used in read-view even if there are no transactions in read-view.
      This problem makes it difficult to test the memory monitoring system,
      so this patch solves the problem.
      
      Close #6635
      Part of #6150
      
      NO_DOC=no visible changes
      NO_CHANGELOG=no visible changes
      2245ef06
    • Andrey Saranchin's avatar
      txm: fix typo in memtx_tx_story_gc_step · 2439aee3
      Andrey Saranchin authored
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      2439aee3
    • Andrey Saranchin's avatar
      txm: do not use regions directly in memtx tx manager · b66f5c15
      Andrey Saranchin authored
      It is difficult to monitor memory consumption for each transcation
      when some functions allocating on txn's region do not have any information
      about owner of the region. This patch solve the problem.
      
      Part of #6150
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      b66f5c15
    • Andrey Saranchin's avatar
      box: add memtx_tx_gc() to box.internal · b4ebfc21
      Andrey Saranchin authored
      An opportunity to call garbage collector of memtx transaction
      manager manually allows to understand which garbage cannot be freed.
      This knowledge can help us to improve garbage collector. Also this
      opportunity makes it easier to test memtx mvcc memory monitoring.
      
      Part of #6150
      
      NO_DOC=internal feature
      NO_CHANGELOG=internal feature
      NO_TEST=internal feature
      b4ebfc21
    • Andrey Saranchin's avatar
      txm: pass txn instead of its region to txn_stmt_new() · 3063e12c
      Andrey Saranchin authored
      There is no information about owner of txn_stmt when it is
      being allocated. It makes difficult to track memory
      consumption for each transaction. That is why we need to
      pass transaction itself to txn_stmt_new() instead of its region.
      
      Part of #6150
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      3063e12c
  3. Apr 08, 2022
    • Vladimir Davydov's avatar
      tx: set session and user for commit/rollback triggers · 3917e472
      Vladimir Davydov authored
      Commit/rollback triggers are run asynchronously, upon receiving the
      write status from WAL. We can't run them in the original fiber that
      submitted the WAL request, because it would open a time window between
      writing a transaction to WAL and committing it in tx, which could lead
      to violating the cascading rolback principles. As a result,
      commit/rollback triggers run with admin privileges.
      
      Let's fix this issue by temporarily setting session and credentials to
      the original fiber's for running commit/rollback triggers.
      
      Closes #7005
      
      NO_DOC=bugfix
      3917e472
  4. Apr 07, 2022
    • Ilya Verbin's avatar
      app: fix crash with multiple -e -l options · bb3ed321
      Ilya Verbin authored
      Tarantool used to crash if launched with multiple -e or -l options without
      a space between the option and the value, e.g.: `tarantool -ea -eb`.
      It happened because optv[] was allocated for argc==3 items, while 4
      options were written to it after parsing (-e, a, -e, b).
      This patch allocates optv[] for the maximum possible number of options:
      (argc - 1) * 2.
      
      Closes #5747
      
      NO_DOC=bugfix
      bb3ed321
    • Vladimir Davydov's avatar
      cfg: add audit_filter option · 0d9344ef
      Vladimir Davydov authored
      The option will be used in the Enterprise version to configure audit
      log event filter.
      
      NO_DOC=ee
      NO_CHANGELOG=ee
      0d9344ef
    • Vladimir Davydov's avatar
      box: drop box_set_log_level and box_set_log_format · 8a9183d9
      Vladimir Davydov authored
      The functions are not used since commit 1a6ad79e ("lua/cfg: drop
      unused log methods").
      
      NO_DOC=drop unused code
      NO_TEST=drop unused code
      NO_CHANGELOG=drop unused code
      8a9183d9
    • Mergen Imeev's avatar
      sql: add support of DECIMAL and INTEGER to ROUND() · 5273b8cb
      Mergen Imeev authored
      This patch fixes an issue with the implicit cast of INTEGER and DECIMAL
      values to DOUBLE when they are passed as the first argument to the
      ROUND() function.
      
      Closes #6988
      
      @TarantoolBot document
      Title: ROUND() now properly supports INTEGER and DECIMAL
      
      INTEGER and DECIMAL values passed as the first argument now will not be
      cast to DOUBLE and the result will be of the same type as the first
      argument. Also, the default type for the ROUND() is now DECIMAL.
      5273b8cb
  5. Apr 05, 2022
  6. Apr 04, 2022
    • Aleksandr Lyapunov's avatar
      txm: expose transaction isolation in iproto · a8eda574
      Aleksandr Lyapunov authored
      Introduce new option IPROTO_TXN_ISOLATION (0x59) in the body of
      IPROTO_BEGIN request, so a user can set isolation level similar
      to box.begin in lua.
      
      The value must be one of the following integers:
      enum txn_isolation_level {
      	/** Take isolation level from global default level. */
      	TXN_ISOLATION_DEFAULT,
      	/** Allow to read committed, but not confirmed changes. */
      	TXN_ISOLATION_READ_COMMITTED,
      	/** Allow to read only confirmed changes. */
      	TXN_ISOLATION_READ_CONFIRMED,
      	/** Determine isolation level automatically. */
      	TXN_ISOLATION_BEST_EFFORT,
      };
      
      Support the new option in net.box.
      
      Part of #6930
      NO_DOC=see later commits
      NO_CHANGELOG=see later commits
      a8eda574
    • Aleksandr Lyapunov's avatar
      txm: introduce transaction isolation levels · ec750af6
      Aleksandr Lyapunov authored
      Now memtx TX manager tries to determine the best isolation level
      by itself. There could be two options:
      * READ_COMMITTED, when the transaction see changes of other tx
      that are committed but not yet confirmed (written to WAL)
      * READ_CONFIRMED, when the transaction see only confirmed changes.
      
      Introduce a simple way to specify the isolation level explicitly:
      box.begin{tx_isolation = 'default'} - the same as box.begin().
      box.begin{tx_isolation = 'read-committed'} - READ_COMMITTED.
      box.begin{tx_isolation = 'read-confirmed'} - READ_CONFIRMED.
      box.begin{tx_isolation = 'best-effort'} - old automatic way.
      
      Intrduce a bit more complex but potentially faster way to set
      isolation level, like that:
      my_level = box.tnx_isolation_level.READ_COMMITTED
      ..
      box.begin{tx_isolation = my_level}
      
      For simplicity of implementation also support symmetric values as
      'READ_COMMITTED' and box.tnx_isolation_level['read-committed'].
      
      Introduce a new box.cfg option - default_tx_isolation, that is
      used as a default when a transaction is started. The option is
      dynamic and possible values are the same as in box.begin, except
      'default' which is meaningless.
      
      In addition to string value the corresponding numeric values can
      be used in both box.begin and box.cfg.
      
      Part of #6930
      NO_DOC=see later commits
      NO_CHANGELOG=see later commits
      ec750af6
    • Aleksandr Lyapunov's avatar
      txm: better detect isolation level in transaction · 4b511eeb
      Aleksandr Lyapunov authored
      When a transaction is started without specifying isolation level
      (which is impossible now) the transactional manager must choose
      the transaction level automatically, that means that is must
      detemine whether the transaction can see other prepared changes
      or not. The best effort that we can made is to check if current
      transaction is read-only or not. For read-only transactions
      there are hope and fear that it will remain read-only, and the
      best choice is not to see prepared changes. But if the transaction
      has DML statements - it must see prepared changed.
      
      Note that a read-only transaction can became read-write if it make
      a DML statement. But if a transaction ignores some other prepared
      change and then makes a DML, there are no other options except
      abort that transaction - it could not be serialized anymore.
      
      Part of #6930
      Closes #6246
      NO_DOC=see later commits
      4b511eeb
    • Aleksandr Lyapunov's avatar
      txm: remove is_prepared_ok argument from memtx_tx_tuple_clarify · 44ae54ae
      Aleksandr Lyapunov authored
      This flag actually describes isolation level:
      is_prepared_ok == true - READ_COMMITTED,
      is_prepared_ok == false - READ_CONFIRMED.
      
      Now it is always calculated as tnx != NULL.
      Let memtx_tx_tuple_clarify to calculate it by itself.
      
      No logical changes.
      
      Part of #6930
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      44ae54ae
    • Aleksandr Lyapunov's avatar
      txm: refactor chain lookup functions · 83f8c5f5
      Aleksandr Lyapunov authored
      No logical changes.
      
      Part of #6930
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      83f8c5f5
    • Aleksandr Lyapunov's avatar
      txm: fix a bug with wrong space:count() result · d54f4ece
      Aleksandr Lyapunov authored
      Internally space's indexes can containt dirty tuples that are
      invisible for user. That's why there's special adjustment in
      space:count() that substracts number of invisible tuple in the
      space.
      
      By a mistake that check thought that all prepared statements are
      visible, which is wrong for autocommit reads, like standalone
      space:count() without explicit transaction.
      
      Fix it by using common for all reads practice: ignore prepared
      statements if current transaction is NULL.
      
      Closes #6421
      NO_DOC=bugfix
      d54f4ece
    • Aleksandr Lyapunov's avatar
      txm: fix a crash in mvcc with secondary index conflict · c2dd8b0a
      Aleksandr Lyapunov authored
      Memtx TX manager stores a pointer to deleting statement in prepared
      story. This pointer is set in two cases:
      1. a statement deletes (or overwrites) a tuple
      2. a story becomes prepared while other inprogress TX overwrites it
      
      By design a tuple can be deleted only by primary index, the case
      when a transaction overwrites somehting in secondary index but does
      not overwrite the same tuple in primary index is prohibited. That's
      why the pointer (to deleting statement) must be set by and only by
      the next statement in the primary index chain.
      
      By mistake the pointer is set also in second index chain analysis
      after reordering which led to unexpected state of a story.
      
      The patch removes the problem.
      
      Closes #6452
      NO_DOC=bugfix
      c2dd8b0a
    • Timur Safin's avatar
      datetime: simplify boundary checks · 64faabe8
      Timur Safin authored
      We used to use very ugly and tricky approach to check that passed years,
      months and days were not exceeding supported range of values. Now we have
      introduced to `c-dt` library the new function `dt_from_ymd_checked` for
      that purpose (i.e. check that values are valid, and construct dt from
      them). So rewrite/simplify Lua code to use that entry as
      `tnt_dt_from_ymd_checked`.
      
      Part of #6731
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      64faabe8
    • Timur Safin's avatar
      datetime: huge dates support in parse functions · 5511dda7
      Timur Safin authored
      * Default parse
        - new c-dt version used which handles extended years range
          while parse relaxed iso8601 gformat strings;
        - family of functions like dt_from_ymd_checked functions
          added to the new c-dt version, now used by conversion code
          to properly handle validation of a 32-bit boundary values;
        - datetime_parse_full() modified to properly handle huge years values;
        - added tests for extended years range.
      
      * strptime-like parse
        - properly handle longer than 4 years values, negative values,
          and handle zulu suffix, which may be generated by Tarantool
          stringization routines;
      
      Part of #6731
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      5511dda7
    • Timur Safin's avatar
      datetime, lua: strptime-like parse format · 02aa8f51
      Timur Safin authored
      To parse date/time strings using format string we use
      `strptime()` implementation from FreeBSD, which is
      modified to use our `struct datetime` data structure.
      
      List of supported format has been extended to include
      `%f` which is flag used whenever you need to process
      nanoseconds part of datetime value.
      
      ```
      tarantool> T = date.parse('Thu Jan  1 03:00:00 1970', {format = '%c'})
      
      tarantool> T
      - 1970-01-01T03:00:00Z
      
      tarantool> T = date.parse('12/31/2020', {format = '%m/%d/%y'})
      
      tarantool> T
      - 2020-12-31T00:00:00Z
      
      tarantool> T = date.parse('1970-01-01T03:00:00.125000000+0300',
                                {format = '%FT%T.%f%z'})
      
      tarantool> T
      - 1970-01-01T03:00:00.125+0300
      ```
      
      Part of #6731
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      02aa8f51
    • Timur Safin's avatar
      datetime, lua: date parsing functions · 3c403661
      Timur Safin authored
      Datetime module provides parse function to create
      datetime object given input string.
      
      `datetime.parse` function expect 1 required argument - which is
      input string, and set of optional parameters passed as table
      in 2nd argument.
      
      Allowed attributes in this optional table are:
      * `format` - should be either 'iso8601', 'rfc3339' or `strptime`-like
        format string. [strptime format will be added as part of next
        commit];
      * `tzoffset` - to redefine offset of input string value, if there
        is no timezone provided.
      * `tz` - human-readable, Olson database, timezone identifier, e.g.
        'Europe/Moscow'. Not yet implemented in this commit.
      
      ```
      tarantool> T = date.parse('1970-01-01T00:00:00Z')
      
      tarantool> T
      - 1970-01-01T00:00:00Z
      
      tarantool> T = date.parse('1970-01-01T00:00:00',
                                {format = 'iso8601', tzoffset = 180})
      
      tarantool> T
      - 1970-01-01T00:00:00+0300
      
      tarantool> T = date.parse('2017-12-27T18:45:32.999999-05:00',
                                {format = 'rfc3339'})
      
      tarantool> T
      - 2017-12-27T18:45:32.999999-0500
      ```
      
      Implemented as per RFC https://hackmd.io/@Mons/S1Vfc_axK#%D0%AD%D1%82%D0%B0%D0%BF-3-%D0%9F%D0%B0%D1%80%D1%81%D0%B8%D0%BD%D0%B3-%D0%B4%D0%B0%D1%82-%D0%BF%D0%BE-%D1%84%D0%BE%D1%80%D0%BC%D0%B0%D1%82%D1%83
      
      Part of #6731
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      3c403661
  7. Apr 01, 2022
    • Serge Petrenko's avatar
      replication: fix bootstrap failing with ER_READONLY · c1c77782
      Serge Petrenko authored
      When the master is just starting up it's possible for replica's JOIN
      request to arrive right in time to bypass ER_LOADING check (after master
      is fully recovered) but still fail due to ER_READONLY: box.cfg.read_only
      is only read and set after box_cfg() (its C part) returns.
      
      In this case the joining replica simply exits with an error and doesn't
      retry JOIN.
      
      Let's fix that. Make ER_READONLY a recoverable error and let replica
      retry joining after receiving ER_READONLY.
      
      Anonymous nodes relied on ER_READONLY to forbid replication from
      anonymous to normal replicas. That check doesn't work anymore.
      So introduce explicit checks banning replication from anonymous nodes.
      
      Note, there were some alternatives to this fix.
      
      First of all, theoretically, we could stop firing ER_LOADING later,
      after box_cfg() is complete. This solution wouldn't work because it
      would lead to deadlocks: the nodes would be stuck in replicaset_sync(),
      because each of them rejects replication with ER_LOADING.
      
      Another solution would be to read the real box.cfg.read_only value
      earlier, in order to allow replication right after the node finishes
      recovery.
      This would also be bad, because we should never let a node become
      writeable before box.cfg is finished. Even after local_recovery is
      complete, the node should stay read-only until it synchronizes with
      other replicas.
      
      That said, neither of the two alternatives fit, so the solution with
      retrying JOIN on ER_READONLY was chosen.
      
      Since the bug is fixed, re-enable the test in which it was discovered:
      replication-py/init_storage.test.py
      
      Also, remove replication/ddl.test.lua from fragile list, since this bug
      was the only reason for its fragility.
      
      Closes #5337
      Closes #6966
      
      NO_DOC=minor bugfix
      c1c77782
  8. Mar 31, 2022
  9. Mar 29, 2022
    • Vladislav Shpilevoy's avatar
      fiber: fix ignorance of flags for reused fibers · 31d27599
      Vladislav Shpilevoy authored
      fiber_new_ex() used to ignore fiber_attr flags when the fiber was
      taken from the cache, not created anew.
      
      It didn't matter much though for the public API, because the only
      public flag in fiber_attr was FIBER_CUSTOM_STACK (which can be
      set via fiber_attr_setstacksize()).
      
      Anyway that was a bug for internal API and would lead to issues in
      the future when more public flags are added. The patch fixes it.
      
      NO_DOC=Bugfix
      NO_CHANGELOG=No reproducer via public API
      31d27599
    • Vladislav Shpilevoy's avatar
      fiber: panic on cancel of a recycled fiber · dbb90274
      Vladislav Shpilevoy authored
      There was a user who complained about this code crashing:
      
          f = fiber_new_ex(...);
          fiber_start(f);
          fiber_cancel(f);
      
      The crash was at cancel. It happened because the fiber finished
      immediately. It was already recycled after fiber_start() return.
      
      Recycled fiber didn't have any flags, so fiber_cancel() didn't
      see the fiber was already dead and tried to wake it up. It crashed
      when the fiber tried to call its 'fiber->f' function which was
      NULL.
      
      In debug build the process fails earlier with an assertion on
      'fiber->fid != 0'.
      
      It can't be really fixed because the problem is the same as with
      use-after-free. The fiber could be not recycled but already freed
      completely, returned back to the mempool.
      
      This patch tries to help the users by a panic with a message
      saying that it wasn't just a crash, it is a bug in user's code.
      
      There is an alternative - make fibers never return to the mempool.
      Then fiber_cancel() could ignore recycled fibers. But it would
      lead to another problem that if the fiber is already reused, then
      fiber_cancel() would hit a totally irrelevant fiber who was
      unlucky to reuse that fiber pointer. It seems worse than panic.
      
      Same problem exists for `fiber_wakeup()`, but I couldn't figure
      out how to add a panic there and not add an `if` on the normal
      execution path (which includes 'ready' and 'running' fibers).
      
      Closes #6837
      
      NO_CHANGELOG=The same crash remains, but happens a bit earlier and
        with a message.
      
      @TarantoolBot document
      Title: `fiber_cancel()` C API clarification
      
      The documentation must warn that the fiber passed to
      `fiber_cancel()` must not be already dead unless it was set to be
      joinable. Same for `fiber_wakeup()` and all the other fiber
      functions. A dead non-joinable fiber could already be freed or
      reused.
      dbb90274
    • Vladislav Shpilevoy's avatar
      fiber: fix fibers with custom stack leak · 4ea29055
      Vladislav Shpilevoy authored
      Fibers with custom stack couldn't be reused via cord->dead list,
      but neither were ever deleted via mempool_free(). They just leaked
      until the cord was destroyed. Their custom stack also leaked.
      
      It happened for all non-joinable custom-stack fibers. That was
      because fiber_destroy() simply skipped the destruction if the
      fiber is the current one.
      
      It didn't affect joinable fibers because their fiber_destroy() is
      done in another fiber. Their stack was deleted, but the fiber
      itself still leaked.
      
      The fix makes so fiber_destroy() is never called for the current
      fiber. Instead, cord uses an approach like in pthread library -
      the fiber who wants to be deleted is saved into cord->garbage
      member. When some other fiber will want to be deleted in the
      future, it will firstly cleanup the previous one and put self into
      its place. And so on - fibers cleanup each other.
      
      The process is optimized for the case when the fiber to delete is
      not the current one - can delete it right away then.
      
      NO_DOC=Bugfix
      4ea29055
    • Igor Munkin's avatar
      lua: rewrite crc32 digest via Lua C API · 6b913198
      Igor Munkin authored
      As a result of recording <crc32:update> method or <digest.crc32>
      function wrong semantics is compiled (strictly saying, the resulting
      trace produces the different result from the one yielded by
      interpreter). The easiest solution is disabling JIT for particular
      functions, however, such approach drops the overall platform
      performance. Hence, the mentioned functions are rewritten line by line
      via Lua C API to avoid JIT misbehaviour.
      
      NO_DOC=no visible changes
      NO_CHANGELOG=no visible changes
      6b913198
    • Georgiy Lebedev's avatar
      core: fix `coro_unwcontext` invalid unwind info · c8ad49f0
      Georgiy Lebedev authored
      During the context switch required for backtracing a suspended fiber,
      unwinders go crazy, as the unwind information they had gets implicitly
      invalidated: provide an annotation for a dummy frame for
      `coro_unwcontext`, as if it were at the bottom of the call-chain — that
      way unwinders can normally proceed further.
      
      We need to know the exact location of the stack pointer: replace the
      16-byte stack alignment instruction on x86_64 macOS by adding the
      `force_align_arg_pointer` attribute to `unw_getcontext_f`.
      
      Needed for #4002
      
      NO_DOC=bug fix
      NO_CHANGELOG=bug fix
      NO_TEST=unwind information annotation in inline assembly
      c8ad49f0
    • cha-cha369's avatar
      docs: fix typos in module.h · 1afde72c
      cha-cha369 authored
      NO_DOC=no behavior changes
      NO_TEST=no behavior changes
      NO_CHANGELOG=no behavior changes
      1afde72c
    • Yan Shtunder's avatar
      net.box: add predefined system events for pub/sub · e1d2f7f0
      Yan Shtunder authored
      Added predefined system events: box.status, box.id, box.election and
      box.schema.
      
      Closes #6260
      
      @TarantoolBot document
      Title: Built-in events for pub/sub
      
      Built-in events are needed, first of all, in order to learn who is the
      master, unless it is defined in an application specific way. Knowing who
      is the master is necessary to send changes to a correct instance, and
      probably make reads of the most actual data if it is important. Also
      defined more built-in events for other mutable properties like leader
      state change, his election role and election term, schema version change
      and instance state.
      
      Built-in events have a special naming schema - their name always starts
      with box.. The prefix is reserved for built-in events. Creating new events
      with this prefix is banned. Below is a list of all the events + their names
      and values:
      
      1. box.id
      Description - identification of the instance. Changes are extra rare. Some
      values never change or change only once. For example, instance UUID never
      changes after the first box.cfg. But is not known before box.cfg is called.
      Replicaset UUID is unknown until the instance joins to a replicaset or
      bootsa new one, but the events are supposed to start working before that -
      right at listen launch. Instance numeric ID is known only after
      registration. On anonymous replicas is 0 until they are registered
      officially.
      Value - {
          MP_STR “id”: MP_UINT; box.info.id,
          MP_STR “instance_uuid”: MP_UUID; box.info.uuid,
          MP_STR “replicaset_uuid”: MP_UUID box.info.cluster.uuid,
      }
      
      2. box.status
      Description - generic blob about instance status. Its most commonly used
      and not frequently changed config options and box.info fields.
      Value - {
          MP_STR “is_ro”: MP_BOOL box.info.ro,
          MP_STR “is_ro_cfg”: MP_BOOL box.cfg.read_only,
          MP_STR “status”: MP_STR box.info.status,
      }
      
      3. box.election
      Description - all the needed parts of box.info.election needed to find who
      is the most recent writable leader.
      Value - {
          MP_STR “term”: MP_UINT box.info.election.term,
          MP_STR “role”: MP_STR box.info.election.state,
          MP_STR “is_ro”: MP_BOOL box.info.ro,
          MP_STR “leader”: MP_UINT box.info.election.leader,
      }
      
      4. box.schema
      Description - schema-related data. Currently it is only version.
      Value - {
          MP_STR “version”: MP_UINT schema_version,
      }
      
      Built-in events can't be override. Meaning, users can't be able to call
      box.broadcast(‘box.id’, any_data) etc.
      
      The events are available from the very beginning as not MP_NIL. It's
      necessary for supported local subscriptions. Otherwise no way to detect
      whether an event is even supported at all by this Tarantool version. If
      events are broadcast before box.cfg{}, then the following values will
      available:
          box.id = {}
          box.schema = {}
          box.status = {}
          box.election = {}
      
      This way the users will be able to distinguish an event being not supported
      at all from box.cfg{} being not called yet. Otherwise they would need to
      parse _TARANTOOL version string locally and peer_version in netbox.
      
      Example usage:
      
       * Client:
         ```lua
         conn = net.box.connect(URI)
         -- Subscribe to updates of key 'box.id'
         w = conn:watch('box.id', function(key, value)
             assert(key == 'box.id')
             -- do something with value
         end)
         -- or to updates of key 'box.status'
         w = conn:watch('box.status', function(key, value)
             assert(key == 'box.status')
             -- do something with value
         end)
         -- or to updates of key 'box.election'
         w = conn:watch('box.election', function(key, value)
             assert(key == 'box.election')
             -- do something with value
         end)
         -- or to updates of key 'box.schema'
         w = conn:watch('box.schema', function(key, value)
             assert(key == 'box.schema')
             -- do something with value
         end)
         -- Unregister the watcher when it's no longer needed.
         w:unregister()
         ```
      e1d2f7f0
  10. Mar 28, 2022
    • Vladimir Davydov's avatar
      memtx: drop UNCHANGED (get = get_raw) index vtab optimization · 5e340b6e
      Vladimir Davydov authored
      We use a special, less efficient index vtab if a space can store
      compressed tuples. The problem is it isn't enough to look at a space
      definition to figure out if there are compressed tuples in the space:
      there may be compressed tuples left from before the alter operation that
      disabled compression, since we don't rebuild tuples on alter. To update
      an index vtab dynamically, we implement some complicated logic, but
      it's buggy (results in a test failure in EE). Fixing it requires some
      non-trivial effort, because a vtab may be changed after index creation
      (when a space format is updated).
      
      Let's drop this optimization altogether for now and use the same vtab
      for both compressed and uncompressed indexes. We might return to this
      issue in future, but first we need to run some benchmarks to check if
      this optimization is worth the complexity. Possible ways how we could
      resurrect this optimization:
       - Call get_raw from get directly (without function pointer), inline
         memtx_prepare_result_tuple, and move is_compressed flag to struct
         tuple for better cache locality.
       - Rebuild all tuples on space alter and use a different vtab for
         compressed indexes.
      
      NO_DOC=bug fix
      NO_TEST=enterprise
      NO_CHANGELOG=unrelased
      5e340b6e
Loading