Skip to content
Snippets Groups Projects
  1. Apr 21, 2021
    • Serge Petrenko's avatar
      txn_limbo: filter rows based on known peer terms · 560d71b4
      Serge Petrenko authored
      
      Start writing the actual leader term together with the PROMOTE request
      and process terms in PROMOTE requests on receiver side.
      
      Make applier only apply synchronous transactions from the instance which
      has the greatest term as received in PROMOTE requests.
      
      Closes #5445
      
      Co-developed-by: default avatarVladislav Shpilevoy <v.shpilevoy@tarantool.org>
      560d71b4
    • Serge Petrenko's avatar
      box: write PROMOTE even for empty limbo · a33dfe07
      Serge Petrenko authored
      PROMOTE entry will be used to mark limbo ownership transition besides
      emptying the limbo. So it has to be written every time
      `box.ctl.clear_synchro_queue()` succeeds. Even when the limbo was
      already empty.
      
      Part of #5445
      a33dfe07
    • Serge Petrenko's avatar
      box: make clear_synchro_queue() write a PROMOTE entry instead of CONFIRM + ROLLBACK · f1746512
      Serge Petrenko authored
      A successful box_clear_synchro_queue() call results in writing
      CONFIRM(N) ROLLBACK(N+1) pair, where N is  the confirmed lsn.
      
      Let's write a single PROMOTE(N) entry instead. It'll have  the same
      meaning as CONFIRM + ROLLBACK and it will give followers some additional
      information regarding leader state change later.
      
      Part of #5445
      f1746512
    • Serge Petrenko's avatar
      box: actualise iproto_key_type array · 588b6930
      Serge Petrenko authored
      iproto_key_type array is used while validating incoming requests, but it
      was only half-filled. The last initialized field was 0x2b, while
      IPROTO_KEY_MAX is currently 0x54.
      
      We got away with it, since the array is only  used in xrow_header_decode(),
      xrow_decode_dml() and xrow_decode_synchro(), and all the keys usually present
      in these requests were present in the array. This is not true anymore,
      so it's time to make array contents up to date with all the IPROTO_KEY_*
      constants we have.
      
      Part of #5445
      588b6930
    • Serge Petrenko's avatar
      xrow: introduce a PROMOTE entry · 23a83352
      Serge Petrenko authored
      A PROMOTE entry combines effect of CONFIRM, ROLLBACK and RAFT_TERM
      entries with some additional semantics on top.
      
      PROMOTE carries the following arguments:
      
      1) former_leader_id - the id of previous limbo owner whose entries we
         want to confirm.
      2) confirm_lsn - the lsn of the last former leader's transaction to be
         confirmed. In this sense PROMOTE(confirm_lsn) replaces
         CONFIRM(confirm_lsn) + ROLLBACK(confirm_lsn + 1).
      3) replica_id - id of the instance issuing
         `box.ctl.clear_synchro_queue()`
      4) term - the new term the instance issuing
         `box.ctl.clear_synchro_queue()` has just entered.
      
      This entry will be written to WAL instead of the usual CONFIRM +
      ROLLBACK pair on a successful `box.ctl.clear_synchro_queue()` call.
      
      Note, the ususal CONFIRM and ROLLBACK occurrences (after a confirmed or
      rolled back synchronous transaction) are here to stay.
      
      Part of #5445
      23a83352
    • Serge Petrenko's avatar
      xrow: enrich row's meta information with sync replication flags · 109502cc
      Serge Petrenko authored
      Introduce two new flags to xrow_header: `wait_ack` and `wait_sync`.
      These flags are set for rows belonging to synchronous transactions in
      addition to `is_commit`.
      
      The new flags help to define whether the rows belong to a synchronous
      transaction or not without parsing them all and checking whether any of
      the rows touches a synchronous space.
      
      This will be used in applier once it is taught to filter synchronous
      transactions based on whether they are coming from a raft leader or not.
      
      P.S. These flags will also be useful once we allow to turn any transaction
      synchronous. Once this is done, the flags in row header will be the only
      source of information on whether the transaction is synchronous or not.
      
      Prerequisite #5445
      
      @TarantoolBot document
      Title: new values for IPROTO_FLAGS field
      
      IPROTO_FLAGS bitfield is enriched with two new constant:
      IPROTO_FLAG_WAIT_SYNC = 0x02
      IPROTO_FLAG_WAIT_ACK = 0x04
      
      IPROTO_FLAG_WAIT_SYNC is set for the last row of a transaction which
      cannot be committed immediately: either because it is synchronous or
      because it waits for other synchronous transactions to complete.
      IPROTO_FLAG_WAIT_ACK is set for the last synchronous transaction row.
      109502cc
    • Serge Petrenko's avatar
      wal: make wal_assign_lsn accept journal entry · 766c6754
      Serge Petrenko authored
      Refactor wal_assign_lsn() to accept a journal entry instead of a pair of
      pointers to the first and last entry rows.
      
      Journal entry will carry additional meta information for the last row
      soon, which will be needed in wal_assign_lsn().
      
      Prerequisite #5445
      766c6754
  2. Apr 20, 2021
  3. Apr 19, 2021
    • Igor Munkin's avatar
      tools: implement toolchain for crash artefacts · e733f08d
      Igor Munkin authored
      
      This patch introduces two scripts to ease crash artefacts collecting and
      loading for postmortem analysis:
      
      * tarabrt.sh - the tool collecting a tarball with the crash artefacts
        the right way: the coredump with the binary, all loaded shared libs,
        Tarantool version (this is a separate exercise to get it from the
        binary built with -O2). Besides, the tarball has a unified layout, so
        it can be easily processed with the second script:
        - /coredump - core dump file on the root level
        - /binary - tarantool executable on the root level
        - /version - plain text file on the root level with
          `tarantool --version` output
        - /checklist - plain text file on the root level with the list of the
          collected entities
        - /etc/os-release - the plain text file containing operating system
          identification data
        - all shared libraries used by the crashed instance - their layout
          respects the one on the host machine, so they can be easily loaded
          with the following gdb command: set sysroot $(realpath .)
      
        The script can be easily used either manually or via
        kernel.core_pattern variable.
      
      * gdb.sh - the auxiliary script originally written by @Totktonada, but
        needed to be adjusted to the crash artefacts layout every time. Since
        there is a unified layout, the original script is enhanced a bit to
        automatically load the coredump via gdb the right way.
      
      Closes #5569
      
      Reviewed-by: default avatarAlexander Turenko <alexander.turenko@tarantool.org>
      Reviewed-by: default avatarSergey Bronnikov <sergeyb@tarantool.org>
      Signed-off-by: default avatarIgor Munkin <imun@tarantool.org>
      e733f08d
    • Igor Munkin's avatar
      luajit: bump new version · be10be8f
      Igor Munkin authored
      LuaJIT submodule is bumped to introduce the following change:
      * test: fix directory detection in lua-Harness suite
      
      This changeset fixes the hidden bug in lua-Harness test suite that was
      revealed by applying 9610c741 ('hotfix:
      update libcurl submodule to 7.76.0'). This was 314-th patch in 2.8.0
      release, so RPM-related tools confuse the testing machinery using the
      patch revision in the building tree names[1].
      
      [1]: https://github.com/tarantool/tarantool/runs/2361010932#step:5:10032
      
      Follows up #5844
      be10be8f
  4. Apr 17, 2021
  5. Apr 16, 2021
    • Nikita Pettik's avatar
      hotfix: update libcurl submodule to 7.76.0 · 9610c741
      Nikita Pettik authored
      It was accidentally reverted by 660686e3.
      9610c741
    • Artem Starshov's avatar
      coverity: add errinj macroses to fix dead code · d15c633e
      Artem Starshov authored
      Our coverity (scan.coverity.com) reports about dead code (CWE-561)
      in cases where variable of type `struct errinj *` is compared with
      NULL at if condition. Dead code appears due to function-like macro
      `errinj()` always returns NULL at non-debug version, so execution
      never goes into if block.
      
      Added macroses ERROR_INJECT_INT and ERROR_INJECT_DOUBLE which solves
      this problem like it does ERROR_INJECT macro with boolean errinj type.
      
      Usage you can see on example of ERRINJ_STDIN_ISATTY fix in this commit.
      
      Follows up #5040
      d15c633e
  6. Apr 15, 2021
    • Nikita Pettik's avatar
      Add changelos for memtx mvcc issues · 7c9ce0b4
      Nikita Pettik authored
      7c9ce0b4
    • Aleksandr Lyapunov's avatar
      txm: fix a small mistake that led to an assertion · 660686e3
      Aleksandr Lyapunov authored
      Follow-ip of #5628
      660686e3
    • Aleksandr Lyapunov's avatar
      txm: implement read gap trackers · 1faa45f9
      Aleksandr Lyapunov authored
      After implementation of point hole trackers only one problem has
      left. When a transaction reads a value from a TREE index by key
      that key may be partial and/or search type may be GE, GT etc. In
      this case point hole hash table doesn't provide track of writes
      into some key interval.
      
      Implement gap trackers for those purposes. The main idea is to
      store those trackers in a story of a closest tuple to the right of
      the lookup (aka successor) or directly in index if there no
      successor (for example if the space in empty).
      
      Closes #5628
      1faa45f9
    • Aleksandr Lyapunov's avatar
      memtx: refactor tree iterator start function · 85b6b46a
      Aleksandr Lyapunov authored
      Reorder some code, rename a bit and and unify branches.
      
      Part of #5628
      85b6b46a
    • Iskander Sagitov's avatar
      test: fix nondeterminism of ER_TUPLE_FOUND error · ef0f2b78
      Iskander Sagitov authored
      ER_TUPLE_FOUND error message depends on the previous tests, so we can
      just check error code in these tests to avoid it.
      ef0f2b78
    • mechanik20051988's avatar
      tuple: do not allow autofill of nested arrays with nulls · e17a5384
      mechanik20051988 authored
      In previous version there was inconsistent/ behaviour for update
      operations to nested arrays. When user try to update non existent
      field of nested array along with previous update existent field,
      it's works, otherwise no. For example we create tuple and two
      update operations:
      `t = box.tuple.new({1, 2, {11, 22}})`
      `op1 = {'=', '[3][1]', 11}`
      `op2 = {'=', '[3][4]', 44}`
      Then in case we apply operations separately `t:update({op1})` and
      then `t:update({op2})`, second operation fails but is case we apply
      them together t:update({op1, op2}) they both complete successfully.
      
      Follow up #3378
      e17a5384
    • Roman Khabibov's avatar
      box: set box.cfg options via environment variables · 1b330121
      Roman Khabibov authored
      
      Add ability to set box.cfg options via environment variables. These
      variables should have name `TT_<OPTION>`. When Tarantool instance is
      started under tarantoolctl utility, environment variables have higher
      priority than tarantoolctl configuration file.
      
      Closes #5602
      
      Co-authored-by: default avatarLeonid Vasiliev <lvasiliev@tarantool.org>
      Co-authored-by: default avatarAlexander Turenko <alexander.turenko@tarantool.org>
      
      @TarantoolBot document
      Title: Set box.cfg options via environment variables
      
      Now, it is possible to set box.cfg options via environment variables.
      The name of variable should correspond the following pattern:
      `TT_<NAME>`, where `<NAME>` is uppercase box.cfg option name. For
      example: `TT_LISTEN`, `TT_READAHEAD`.
      
      Array values are separated by comma. Example:
      
      ```sh
      export TT_REPLICATION=localhost:3301,localhost:3302
      ```
      
      An empty variable is the same as unset one.
      1b330121
    • Cyrill Gorcunov's avatar
      cfg: provide types for logger options · 8d1a4bdb
      Cyrill Gorcunov authored
      
      This needed for fast type check when fetching
      options from environment variable.
      
      Part-of #5602
      
      Signed-off-by: default avatarCyrill Gorcunov <gorcunov@gmail.com>
      8d1a4bdb
    • Mergen Imeev's avatar
      sql: introduce mem_get_agg() · 8cb3b77d
      Mergen Imeev authored
      This patch introduces mem_get_agg(). This function is used to receive
      address of memory allocated for accumulation structure of the aggregate
      function.
      
      Closes #5818
      8cb3b77d
    • Mergen Imeev's avatar
      sql: introduce mem_len() · 718d43ee
      Mergen Imeev authored
      This patch introduces mem_len(). This function is used to receive
      length of string or binary value of MEM. If MEM is not of STRING or
      VARBINARy type this function returns -1.
      
      Part of #5818
      718d43ee
    • Mergen Imeev's avatar
      sql: introduce mem_get_bin() · deabc2ce
      Mergen Imeev authored
      This patch introduces mem_get_bin(). This function is used to receive
      binary value from MEM. If value of MEM is not binary value, it is
      converted to binary value if possible. MEM is not changed.
      
      Part of #5818
      deabc2ce
    • Mergen Imeev's avatar
      sql: introduce mem_get_str0() and mem_as_str0() · 716a33d1
      Mergen Imeev authored
      This patch introduces mem_get_str0() and mem_as_str0(). Function
      mem_get_str0() is used to receive NULL-terminated string from MEM. If
      value of MEM is not NULL-terminated string, it is converted to
      NULL-terminated string if possible. MEM is not changed. Function
      mem_as_str0() is also used to receive NULL-terminated string from MEM,
      however if MEM does not contain NULL-terminated string it converts MEM
      to MEM that contains NULL-terminated string and returns its value.
      
      Part of #5818
      716a33d1
    • Mergen Imeev's avatar
      sql: introduce mem_get_bool() · 953a847c
      Mergen Imeev authored
      This patch introduces mem_get_bool(). This function is used to receive
      boolean value from MEM. If value of MEM is not boolean, it is
      converted to boolean if possible. MEM is not changed.
      
      Part of #5818
      953a847c
    • Mergen Imeev's avatar
      sql: introduce mem_get_double() · d4e55ae8
      Mergen Imeev authored
      This patch introduces mem_get_double(). This function is used to receive
      double value from MEM. If value of MEM is not double, it is converted to
      double if possible. MEM is not changed.
      
      Part of #5818
      d4e55ae8
    • Mergen Imeev's avatar
      sql: introduce mem_get_uint() · bde6d26e
      Mergen Imeev authored
      This patch introduces mem_get_uint() function. This function is used to
      receive unsigned value from MEM. If value of MEM is not unsigned, it is
      converted to unsigned if possible. MEM is not changed.
      
      Part of #5818
      bde6d26e
    • Mergen Imeev's avatar
      sql: introduce mem_get_int() · dbdc4eec
      Mergen Imeev authored
      This patch introduces mem_get_int() function. This function is used to
      receive integer value from MEM. If value of MEM is not integer, it is
      converted to integer if possible. MEM is not changed.
      
      Part of #5818
      dbdc4eec
    • Mergen Imeev's avatar
      sql: introduce mem_cast_implicit() · 655c4df5
      Mergen Imeev authored
      This patch introduces mem_cast_implicit(). This function is used to
      convert a MEM to given type according to implicit cast rules.
      
      Part of #5818
      655c4df5
    • Mergen Imeev's avatar
      sql: introduce mem_cast_explicit() · c8b8fd66
      Mergen Imeev authored
      This patch introduces mem_cast_explicit(). This function is used to
      convert a MEM to a given field type according to explicit cast rules.
      
      Part of #5818
      c8b8fd66
    • Mergen Imeev's avatar
      sql: introduce mem_to_str() and mem_to_str0() · b47715c1
      Mergen Imeev authored
      This patch introduces mem_to_str() and mem_to_str0() functions. These
      functions are used to convert a MEM to a MEM that contains string value.
      These functions defines the rules that are used during convertion from
      values of all other types to STRING.
      
      Part of #5818
      b47715c1
    • Mergen Imeev's avatar
      sql: introduce mem_to_number() · 985eca7c
      Mergen Imeev authored
      This patch introduces mem_to_number(). This function is used to convert
      MEM to MEM contains number.
      
      Part of #5818
      985eca7c
    • Mergen Imeev's avatar
      sql: introduce mem_to_double() · 5b438e24
      Mergen Imeev authored
      This patch intruduces mem_to_double(). This function is used to convert
      a MEM to a MEM that contains double value. This function defines the
      rules that are used during conversion from values of all other types to
      double.
      
      Part of #5818
      5b438e24
    • Mergen Imeev's avatar
      sql: introduce mem_to_int*() functions · 829daeed
      Mergen Imeev authored
      This patch introduces mem_to_int() and mem_to_int_precise() functions.
      These functions are used to convert a MEM to a MEM that contains
      integer value. These functions defines the rules that are used during
      convertion from values of all other types to INTEGER.
      
      Part of #5818
      829daeed
    • Mergen Imeev's avatar
      sql: introduce mem_set_null_clear() · 1581f883
      Mergen Imeev authored
      This patch introduces mem_set_null_clear() function. This function sets
      "cleared" NULL to MEM. This NULL is used only in internal VDBE
      operations.
      
      Part of #5818
      1581f883
    • Mergen Imeev's avatar
      sql: introduce mem_set_agg() · bdb64c1d
      Mergen Imeev authored
      This patch introduces mem_set_agg() function. This function stores
      aggregation function to MEM and allocates enough memory to hold
      accumulation structure for aggregate function.
      
      Part of #5818
      bdb64c1d
    • Mergen Imeev's avatar
      sql: introduce mem_set_frame() · d15f2a72
      Mergen Imeev authored
      This patch introduces mem_set_frame() function. This function clears the
      MEM and sets a frame to MEM. Frames used for internal VDBE operations.
      
      Part of #5818
      d15f2a72
    • Mergen Imeev's avatar
      sql: refactor mem_set_ptr() · f22686dc
      Mergen Imeev authored
      This patch refactors mem_set_ptr() function. Also, it moves the function
      to all others mem_set_*() functions.
      
      Part of #5818
      f22686dc
Loading