Skip to content
Snippets Groups Projects
  1. Aug 12, 2021
    • Serge Petrenko's avatar
      replication: fix flaky gh-3055-election-promote test · 1df99600
      Serge Petrenko authored
      Found the following error in our CI:
      
      [001] Test failed! Result content mismatch:
      [001] --- replication/gh-3055-election-promote.result	Mon Aug  2 17:52:55 2021
      [001] +++ var/rejects/replication/gh-3055-election-promote.reject	Mon Aug  9 10:29:34 2021
      [001] @@ -88,7 +88,7 @@
      [001]   | ...
      [001]  assert(not box.info.ro)
      [001]   | ---
      [001] - | - true
      [001] + | - error: assertion failed!
      [001]   | ...
      [001]  assert(box.info.election.term > term)
      [001]   | ---
      [001]
      
      The problem was the same as in recently fixed election_qsync.test
      (commit 096a0a7d): PROMOTE is written to
      WAL asynchronously, and box.ctl.promote() returns earlier than this
      happens.
      
      Fix the issue by waiting for the instance to become writeable.
      
      Follow-up #6034
      1df99600
    • Aleksandr Lyapunov's avatar
      box: implement compact mode in tuples · 74177dd8
      Aleksandr Lyapunov authored
      Tuple are designed to store (almost) any sizes of msgpack data
      and rather big count of field offsets. That requires data_offsert
      and bsize members of tuples to be rather large - 16 and 32 bits.
      
      That is good, but the problem is that in cases when the majority
      of tuples are small that price is significant.
      
      This patch introduces compact tuples: if tuple data size and its
      offset table are small - both tuple_offset and bsize are stored in
      one 16 bit integer and that saves 4 bytes per tuple.
      
      Compact tuples are used for memtx and runtime tuples. They are not
      implemented for vinyl, because in contrast to memtx vinyl stores
      engine specific fields after struct tuple and thus requires
      different approach for compact tuple.
      
      Part of #5385
      74177dd8
    • Aleksandr Lyapunov's avatar
      box: make a pair of struct tuple's members private · 65c4d37e
      Aleksandr Lyapunov authored
      There's tuple_offset and bsize members in tuple. For better code
      incapsulation in general and for futher changes in particular they
      should be incapsulated within struct tuple. This commit provides
      that.
      
      There's no functional changes in this commit.
      
      Part of #5385
      65c4d37e
    • Aleksandr Lyapunov's avatar
      box: move is_dirty flag in tuple to appropriate section · 24c31bff
      Aleksandr Lyapunov authored
      Now we have a place in tuple for different flags, move is_dirty
      there.
      
      Part of #5385
      24c31bff
    • Aleksandr Lyapunov's avatar
      box: rework tuple reference count · 45269211
      Aleksandr Lyapunov authored
      Tuples are usually have a very low reference counter (I bet the
      majority of tuple have it less than 10), and we may rely on the
      fact in optimization issues. On the other hand it is not actually
      prohibited for a tuple to have a big reference counter, thus the
      code must handle it properly.
      
      The obvious solution is to store narrow reference counter right
      in struct tuple, and store it somewhere else if it hits threshold.
      
      The previous implementation has a 15 bit counter and 1 bit flag the
      that actual counter is stored in separate array. That worked fine
      except 15 bits are still an overkill for real reference counts.
      And that solution introduced unions into struct tuple, which in
      turn, generally speaking, causes an UB since by standard it is
      an UB to access one union part after setting other.
      
      The new solution is to store 8 bit counter and 1 bit flag. The
      external storage is made as hash table to which a portion of the
      counter is uploaded (or acquire) very seldom. That makes the
      counter in tuple more compact, rather fast (and even fastest for
      low reference counter values) and has no limitation such as
      limited count of tuples that can have big reference counts.
      
      Part of #5385
      45269211
    • Aleksandr Lyapunov's avatar
      vinyl: use simple struct member for n_upserts · 60a2e9b9
      Aleksandr Lyapunov authored
      Now it is stored in memory right before struct struct vy_stmt,
      consuming 8 bytes per tuple (due to alignment).
      
      It is much more simple and completely free to store it right in
      member of vy_stmt.
      
      Part of #5385
      60a2e9b9
    • Aleksandr Lyapunov's avatar
      vinyl: save 8 bytes in struct vy_stmt · b9c598ec
      Aleksandr Lyapunov authored
      Due to C/C++ standard layout sizeof(struct vy_stmt) was 32 bytes.
      Is a pity since it has only 20 bytes of payload (10 byte for base
      struct tuple and 10 for lsn (8) + type (1) + flags (1)).
      
      Repack struct vy_stmt to be 24 bytes long.
      
      Part of #5385
      b9c598ec
    • Aleksandr Lyapunov's avatar
      perf: introduce tuple perf test · 3dea259c
      Aleksandr Lyapunov authored
      Part of #5385
      3dea259c
  2. Aug 11, 2021
    • Igor Munkin's avatar
      luajit: bump new version · e42d116f
      Igor Munkin authored
      * ARM64: Fix write barrier in BC_USETS.
      * Linux/ARM64: Make mremap() non-moving due to VA space woes.
      * Add support for full-range 64 bit lightuserdata.
      
      Closes #2712
      Needed for #6154
      Part of #5629
      Unverified
      e42d116f
    • Yan Shtunder's avatar
      replication: fill replicaset.applier.vclock after local recovery · 68851b35
      Yan Shtunder authored
      replicaset.applier.vclock is initialized in replication_init(),
      which happens before local recovery. If some changes are come
      from an instance via replication the applier.vclock will be equal 0.
      This means that if some wild master will send this node already applied
      data, the node will apply the same data twice.
      
      Closes #6028
      68851b35
  3. Aug 10, 2021
    • Mergen Imeev's avatar
      sql: remove unnecessary function initialization · dd7fa342
      Mergen Imeev authored
      After removing the SQL built-in functions from _func, the code used to
      initialize these SQL built-in functions is no longer used and should be
      removed.
      
      Follow-up #6106
      dd7fa342
    • Mergen Imeev's avatar
      alter: disallow creation of SQL built-in function · c49eab90
      Mergen Imeev authored
      This patch prohibits creation of user-defined functions with SQL_BUILTIN
      engine.
      
      Closes #6106
      c49eab90
    • Vladislav Shpilevoy's avatar
      alter: parse data dictionary version · e925537d
      Vladislav Shpilevoy authored
      Version is needed to disallow creation of SQL built-in functions using
      _func starting with 2.9.0.
      
      Needed for #6106
      e925537d
    • Mergen Imeev's avatar
      sql: remove SQL built-in functions from _func · 970062d7
      Mergen Imeev authored
      This patch removes SQL built-in functions from _func. These functions
      could be called directly from Lua, however all they did was returned an
      error. After this patch, no SQL built-in functions can be called
      directly from LUA.
      
      Part of #6106
      970062d7
    • Mergen Imeev's avatar
      sql: introduce sql_func_find() · c8c56b14
      Mergen Imeev authored
      This patch introduces the sql_func_find() function. This function allows
      us to centralize the look up of functions during parsing, which
      simplifies code and fixes some incorrect error messages.
      
      Part of #6106
      c8c56b14
    • Mergen Imeev's avatar
      sql: introduce sql_func_flags() · 2abb5732
      Mergen Imeev authored
      This function returns a set of parameters for the function with the
      given name. This function is used when we do not need to call a
      function, but we need its parameters.
      
      In addition, this function will allow us to split the parameters
      between those that are the same for all implementations, and the
      parameters, the value of which is implementation-dependent.
      
      Needed for #6105
      Part of #6106
      2abb5732
  4. Aug 09, 2021
    • Leonid Vasiliev's avatar
      test: remove unused cases · ed7da7e6
      Leonid Vasiliev authored
      After changing the way symbols are exported, handling several
      cases in the "ssl-cert-paths-discover" test is no longer necessary.
      Let's remove it.
      
      Part of #5932
      ed7da7e6
    • Leonid Vasiliev's avatar
      cmake: wrap the symbols used in the "ssl-cert-paths-discover" test · 64807be3
      Leonid Vasiliev authored
      Wrap the symbols used in the "ssl-cert-paths-discover" test to
      avoid clashes.
      
      Symbols from openssl have been wraped to:
      
          crypto_X509_get_default_cert_dir_env
          crypto_X509_get_default_cert_file_env
      
      Tarantool symbols have been prefixed by "tnt_":
      
          tnt_ssl_cert_paths_discover
          tnt_default_cert_dir_paths
          tnt_default_cert_file_paths
      
      Part of #5932
      64807be3
    • Leonid Vasiliev's avatar
      cmake: wrap the exported readline function · ffdb9f24
      Leonid Vasiliev authored
      Wrap the exported readline function to avoid clash of symbols.
      
      Part of #5932
      ffdb9f24
    • Leonid Vasiliev's avatar
      cmake: hide tarantool symbols back · 5ceabb37
      Leonid Vasiliev authored
      After unhiding all internal symbols([1]) we experience a bunch of
      problems ([2], [3]). The second one (clash of symbols from different version
      of the "small" library) still have no good solution.
      You can find more on the topic [4].
      
      The situation for tarantool executable is the same as for any other library.
      A library should expose only its public API and should not increase probability
      of hard to debug problems due to clash of a user's code with an internal name
      from the library.
      
      Let's hide all symbols by default and create a list of exported symbols.
      (In fact, this patch is a revert of the patch
      03790ac5 ([5]) taking into account the changes
      made to the code)
      
      Explanation of adding some controversial symbols to the export list:
      
      * The following symbols are used in shared libraries used in tests
        ("cfunc*.so", "sql_uuid.so", "gh-6024-funcs-return-bin.so", "function1.so",
        "gh-5938-wrong-string-length.so", "module_api.so")
      
          mp_check
          mp_encode_array
          mp_encode_bin
          mp_encode_bool
          mp_encode_int
          mp_encode_map
          mp_encode_nil
          mp_encode_str
          mp_encode_uint
          mp_decode_array_slowpath
          mp_decode_str
          mp_next_slowpath
          mp_load_u8
          mp_next
          mp_sizeof_array
          mp_sizeof_str
          mp_type_hint
          decimal_from_string
      
      * These symbols are used in "crypto.lua" and, if absent, will lead to the
        failure of the "static_build_cmake_linux" on CI (the crypto prefix was
        used to avoid the clashes of names)
      
          crypto_ERR_error_string
          crypto_ERR_get_error
          crypto_EVP_DigestInit_ex
          crypto_EVP_DigestUpdate
          crypto_EVP_DigestFinal_ex
          crypto_EVP_get_digestbyname
          crypto_HMAC_Init_ex
          crypto_HMAC_Update
          crypto_HMAC_Final
      
      * For correct work of "schema.lua" in the "static_build_cmake_linux"
      
          rl_get_screen_size
      
      * The following symbols are used in "ssl-cert-paths-discover.test.lua"
        (I think these symbols will have to be wrapped in the to avoid clashes
        problems)
      
          X509_get_default_cert_dir_env
          X509_get_default_cert_file_env
          ssl_cert_paths_discover
      
      From "exports.test.lua" have been removed ZSTD symbols checking (see [6]) and
      "tt_uuid_str" (see [7]).
      
      1. https://github.com/tarantool/tarantool/issues/2971
      2. https://github.com/tarantool/tarantool/issues/5001
      3. https://github.com/tarantool/memcached/issues/59
      4. https://lists.tarantool.org/pipermail/tarantool-discussions/2020-September/000095.html
      5. https://github.com/tarantool/tarantool/commit/03790ac5510648d1d9648bb2281857a7992d0593
      6. https://github.com/tarantool/tarantool/issues/4225
      7. https://github.com/tarantool/tarantool/commit/acf8745ed8fef47e6d1f1c31708c7c9d6324d2f3
      
      Part of #5932
      5ceabb37
    • Vladimir Davydov's avatar
      Add changelog entry for gh-6241 · 8ea25027
      Vladimir Davydov authored
      Closes #6241
      8ea25027
    • Vladimir Davydov's avatar
      net.box: add Lua aliases for perform_request and perform_async_request · 3868ca40
      Vladimir Davydov authored
      These two are hot functions. Let's add aliases for them to avoid a
      lookup in a Lua table on each call.
      
      At the same time, internal.decode_greeting is called only when a
      connection is established so we don't need to have an alias for it.
      
      Part of #6241
      3868ca40
    • Vladimir Davydov's avatar
      net.box: do not create request object in Lua for sync requests · bfe79ecc
      Vladimir Davydov authored
      It's not really necessary - we can wait for the request to complete in C
      code, without returning to Lua. Since creating a Lua object puts extra
      pressure on the garbage collector, we'd better avoid it when we can.
      
      Part of #6241
      bfe79ecc
    • Vladimir Davydov's avatar
      net.box: merge new_id, new_request and encode_method · 78248545
      Vladimir Davydov authored
      So as not to call three C functions per each request, let's merge them
      and call the resulting function perform_async_request.
      
      Part of #6241
      78248545
    • Vladimir Davydov's avatar
      net.box: rewrite iproto handlers in C · 20158a95
      Vladimir Davydov authored
      Those are performance-critical paths. Moving them to C should speed up
      overall net.box performance.
      
      Part of #6241
      20158a95
    • Vladimir Davydov's avatar
      net.box: rewrite console handlers in C · e83bf593
      Vladimir Davydov authored
      Strictly speaking, it's not necessary, because console performance is
      not a problem. We do this for consistency with iproto.
      
      Part of #6241
      e83bf593
    • Vladimir Davydov's avatar
      net.box: store next_request_id in C code · 6a8da268
      Vladimir Davydov authored
      Needed to rewrite performance-critical parts of net.box in C, e.g.
      iproto_schema_sm.
      
      Part of #6241
      6a8da268
    • Vladimir Davydov's avatar
      net.box: rewrite request implementation in C · 954194a1
      Vladimir Davydov authored
      This patch turns 'request' object and 'requests' table into userdata and
      rewrites all their methods in C. Needed to rewrite performance-critical
      parts of net.box in C.
      
      Part of #6241
      954194a1
    • Vladimir Davydov's avatar
      net.box: rename netbox_{prepare,encode}_request to {begin,end} · f51b3ffb
      Vladimir Davydov authored
      netbox_encode_request is a confusing name - the function doesn't encode
      a request, it just writes the actual request size to the fix header.
      Let's rename the two functions to netbox_begin_request and
      netbox_end_request to emphasize that they are used to start and finish
      request encoding.
      
      Another reason to do that now is that in the following patches we will
      be moving the request table implementation from Lua to C so having a
      "request" in the function names would be confusing.
      
      Part of #6241
      f51b3ffb
    • Vladimir Davydov's avatar
      net.box: rewrite send_and_recv_{iproto,console} in C · b180ea25
      Vladimir Davydov authored
      Basically, this patch converts the above function line-by-line from Lua
      to C. This is a step towards rewriting performance-critical parts of
      net.box in C.
      
      Note, Lua code expects send_and_recv functions return errno, errmsg on
      error and nil, ... on success while it's more convenient to return an
      error object from C code so we do the conversion in Lua wrappers.
      
      Part of #6241
      b180ea25
    • Vladimir Davydov's avatar
      net.box: remove unused timeout argument from IO methods · 86bd5986
      Vladimir Davydov authored
      Part of #6241
      86bd5986
    • Vladimir Davydov's avatar
      net.box: rewrite error decoder in C · 6aafa697
      Vladimir Davydov authored
      This patch moves response_decoder table (which decodes only errors
      despite its name) from Lua to C. This is a step towards rewriting
      performance-critical parts of net.box in C.
      
      Since error_unpack_unsafe is not used by FFI anymore, remove it from
      exports.h
      
      Part of #6241
      6aafa697
    • Alexander V. Tikhonov's avatar
      github-ci: pull all available tags · 1c76fa51
      Alexander V. Tikhonov authored
      Found that Github actions/checkout@v2, when Github Actions triggers
      a job upon the tag push, it makes the following calls [1]:
      
        git fetch origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*
        git fetch origin +$HASH:refs/tags/$TAG
      
      While the first call is fine, the second one spoils the tag and makes
      it non-annotated.
      
      Later, Tarantool calls git describe without --tags flag, and the
      spoiled tag is ignored. This patch handles it by fetching all the
      tags once again explicitly.
      
      Also explained the next step, where we remove the tag. The Github
      Actions may runs twice the same job on branch push and on new tag
      if it was created on this push. To avoid of fails on saving the
      commonly named packages, we drop a tag that points to a current
      commit (if any) on a job triggered by pushing to a branch (it will
      produce the package based on previous tag) and leave the new tag
      in job triggered by tag.
      
      To check this change was used commands to create dump commit and tag
      the new version. Also made twice push with extra flag for tags:
      
        git commit -m"Dump commit" --allow-empty
        git tag -a 2.4.8 -m ''
        git push && git push --tags
      
      In this way would be run 2 jobs for each package:
      
        1. Build package on testing branch. It will produce package with
           name 2.4.7.x (because of the exception mentioned above).
        2. Build package on tag 2.4.8. It will produce package with name
           2.4.8.x.
      
      Closes tarantool/tarantool-qa#119
      
      [1]: https://github.com/actions/checkout/issues/290
      1c76fa51
    • Aleksandr Lyapunov's avatar
      txm: fix ER_TUPLE_FOUND error message in MVCC · 970f90c9
      Aleksandr Lyapunov authored
      At some point ER_TUPLE_FOUND error message was extended to contain
      old tuple and new tuple. It seems that after rebase this change
      was ignored in MVCC code that used previous format. This patch
      fixes that.
      
      Closes #6247
      970f90c9
    • Mergen Imeev's avatar
      sql: remove unused MEM cast functions · 4d5ec7ba
      Mergen Imeev authored
      This patch removes functions that become unused due to changes of
      implicit and explicit cast rules.
      
      Closes #4470
      4d5ec7ba
    • Mergen Imeev's avatar
      sql: remove implicit cast from OP_MakeRecord · b0ef1a56
      Mergen Imeev authored
      This patch removes deprecated implicit cast from OP_MakeRecord opcode,
      which were used in some rare cases, for example during IN operation
      with subselect as right-value.
      
      Closes #4230
      Part of #4470
      b0ef1a56
    • Mergen Imeev's avatar
      sql: remove unnecessary calls of OP_ApplyType · 145e5423
      Mergen Imeev authored
      Since the OP_Seek* opcodes now work using the new implicit casting
      rules, we don't need to call OP_ApplyType before them. This patch
      removes such calls.
      
      Part of #4230
      Part of #4470
      145e5423
    • Mergen Imeev's avatar
      sql: rework OP_Seek* opcodes · 42cd274c
      Mergen Imeev authored
      This patch changes the Seek* opcodes that are used to search in space
      using index. After the redesign, searches using these opcodes work
      according to the new implicit casting rules. However, currently implicit
      cast in these opcodes is not invoked since there is OP_ApplyType before
      them. Unnecessary OP_ApplyType calls will be removed in next patch.
      
      Part of 4230
      Part of 4470
      42cd274c
    • Mergen Imeev's avatar
      sql: remove implicit cast from comparison opcodes · 078bcf00
      Mergen Imeev authored
      After this patch, the new rules will be applied to implicit cast during
      comparison where index is not used. Essentially it means that implicit
      cast from STRING to number during such comparisons was removed.
      
      Part of #4230
      Part of #4470
      078bcf00
    • Mergen Imeev's avatar
      sql: rework implicit cast fo assignment · ac5405bf
      Mergen Imeev authored
      After this patch, the new rules will be applied to implicit cast during
      assignment. According to these rules, all scalar values can be cast to
      SCALAR, all numeric values can be cast to NUMBER, and any numeric value
      can be cast to another numeric type only if the conversion is exact.
      No other implicit cast is allowed.
      
      Part of #4470
      ac5405bf
Loading