Skip to content
Snippets Groups Projects
  1. Aug 10, 2021
    • Mergen Imeev's avatar
      sql: introduce sql_func_find() · c8c56b14
      Mergen Imeev authored
      This patch introduces the sql_func_find() function. This function allows
      us to centralize the look up of functions during parsing, which
      simplifies code and fixes some incorrect error messages.
      
      Part of #6106
      c8c56b14
    • Mergen Imeev's avatar
      sql: introduce sql_func_flags() · 2abb5732
      Mergen Imeev authored
      This function returns a set of parameters for the function with the
      given name. This function is used when we do not need to call a
      function, but we need its parameters.
      
      In addition, this function will allow us to split the parameters
      between those that are the same for all implementations, and the
      parameters, the value of which is implementation-dependent.
      
      Needed for #6105
      Part of #6106
      2abb5732
  2. Aug 09, 2021
    • Leonid Vasiliev's avatar
      test: remove unused cases · ed7da7e6
      Leonid Vasiliev authored
      After changing the way symbols are exported, handling several
      cases in the "ssl-cert-paths-discover" test is no longer necessary.
      Let's remove it.
      
      Part of #5932
      ed7da7e6
    • Leonid Vasiliev's avatar
      cmake: wrap the symbols used in the "ssl-cert-paths-discover" test · 64807be3
      Leonid Vasiliev authored
      Wrap the symbols used in the "ssl-cert-paths-discover" test to
      avoid clashes.
      
      Symbols from openssl have been wraped to:
      
          crypto_X509_get_default_cert_dir_env
          crypto_X509_get_default_cert_file_env
      
      Tarantool symbols have been prefixed by "tnt_":
      
          tnt_ssl_cert_paths_discover
          tnt_default_cert_dir_paths
          tnt_default_cert_file_paths
      
      Part of #5932
      64807be3
    • Leonid Vasiliev's avatar
      cmake: wrap the exported readline function · ffdb9f24
      Leonid Vasiliev authored
      Wrap the exported readline function to avoid clash of symbols.
      
      Part of #5932
      ffdb9f24
    • Leonid Vasiliev's avatar
      cmake: hide tarantool symbols back · 5ceabb37
      Leonid Vasiliev authored
      After unhiding all internal symbols([1]) we experience a bunch of
      problems ([2], [3]). The second one (clash of symbols from different version
      of the "small" library) still have no good solution.
      You can find more on the topic [4].
      
      The situation for tarantool executable is the same as for any other library.
      A library should expose only its public API and should not increase probability
      of hard to debug problems due to clash of a user's code with an internal name
      from the library.
      
      Let's hide all symbols by default and create a list of exported symbols.
      (In fact, this patch is a revert of the patch
      03790ac5 ([5]) taking into account the changes
      made to the code)
      
      Explanation of adding some controversial symbols to the export list:
      
      * The following symbols are used in shared libraries used in tests
        ("cfunc*.so", "sql_uuid.so", "gh-6024-funcs-return-bin.so", "function1.so",
        "gh-5938-wrong-string-length.so", "module_api.so")
      
          mp_check
          mp_encode_array
          mp_encode_bin
          mp_encode_bool
          mp_encode_int
          mp_encode_map
          mp_encode_nil
          mp_encode_str
          mp_encode_uint
          mp_decode_array_slowpath
          mp_decode_str
          mp_next_slowpath
          mp_load_u8
          mp_next
          mp_sizeof_array
          mp_sizeof_str
          mp_type_hint
          decimal_from_string
      
      * These symbols are used in "crypto.lua" and, if absent, will lead to the
        failure of the "static_build_cmake_linux" on CI (the crypto prefix was
        used to avoid the clashes of names)
      
          crypto_ERR_error_string
          crypto_ERR_get_error
          crypto_EVP_DigestInit_ex
          crypto_EVP_DigestUpdate
          crypto_EVP_DigestFinal_ex
          crypto_EVP_get_digestbyname
          crypto_HMAC_Init_ex
          crypto_HMAC_Update
          crypto_HMAC_Final
      
      * For correct work of "schema.lua" in the "static_build_cmake_linux"
      
          rl_get_screen_size
      
      * The following symbols are used in "ssl-cert-paths-discover.test.lua"
        (I think these symbols will have to be wrapped in the to avoid clashes
        problems)
      
          X509_get_default_cert_dir_env
          X509_get_default_cert_file_env
          ssl_cert_paths_discover
      
      From "exports.test.lua" have been removed ZSTD symbols checking (see [6]) and
      "tt_uuid_str" (see [7]).
      
      1. https://github.com/tarantool/tarantool/issues/2971
      2. https://github.com/tarantool/tarantool/issues/5001
      3. https://github.com/tarantool/memcached/issues/59
      4. https://lists.tarantool.org/pipermail/tarantool-discussions/2020-September/000095.html
      5. https://github.com/tarantool/tarantool/commit/03790ac5510648d1d9648bb2281857a7992d0593
      6. https://github.com/tarantool/tarantool/issues/4225
      7. https://github.com/tarantool/tarantool/commit/acf8745ed8fef47e6d1f1c31708c7c9d6324d2f3
      
      Part of #5932
      5ceabb37
    • Vladimir Davydov's avatar
      Add changelog entry for gh-6241 · 8ea25027
      Vladimir Davydov authored
      Closes #6241
      8ea25027
    • Vladimir Davydov's avatar
      net.box: add Lua aliases for perform_request and perform_async_request · 3868ca40
      Vladimir Davydov authored
      These two are hot functions. Let's add aliases for them to avoid a
      lookup in a Lua table on each call.
      
      At the same time, internal.decode_greeting is called only when a
      connection is established so we don't need to have an alias for it.
      
      Part of #6241
      3868ca40
    • Vladimir Davydov's avatar
      net.box: do not create request object in Lua for sync requests · bfe79ecc
      Vladimir Davydov authored
      It's not really necessary - we can wait for the request to complete in C
      code, without returning to Lua. Since creating a Lua object puts extra
      pressure on the garbage collector, we'd better avoid it when we can.
      
      Part of #6241
      bfe79ecc
    • Vladimir Davydov's avatar
      net.box: merge new_id, new_request and encode_method · 78248545
      Vladimir Davydov authored
      So as not to call three C functions per each request, let's merge them
      and call the resulting function perform_async_request.
      
      Part of #6241
      78248545
    • Vladimir Davydov's avatar
      net.box: rewrite iproto handlers in C · 20158a95
      Vladimir Davydov authored
      Those are performance-critical paths. Moving them to C should speed up
      overall net.box performance.
      
      Part of #6241
      20158a95
    • Vladimir Davydov's avatar
      net.box: rewrite console handlers in C · e83bf593
      Vladimir Davydov authored
      Strictly speaking, it's not necessary, because console performance is
      not a problem. We do this for consistency with iproto.
      
      Part of #6241
      e83bf593
    • Vladimir Davydov's avatar
      net.box: store next_request_id in C code · 6a8da268
      Vladimir Davydov authored
      Needed to rewrite performance-critical parts of net.box in C, e.g.
      iproto_schema_sm.
      
      Part of #6241
      6a8da268
    • Vladimir Davydov's avatar
      net.box: rewrite request implementation in C · 954194a1
      Vladimir Davydov authored
      This patch turns 'request' object and 'requests' table into userdata and
      rewrites all their methods in C. Needed to rewrite performance-critical
      parts of net.box in C.
      
      Part of #6241
      954194a1
    • Vladimir Davydov's avatar
      net.box: rename netbox_{prepare,encode}_request to {begin,end} · f51b3ffb
      Vladimir Davydov authored
      netbox_encode_request is a confusing name - the function doesn't encode
      a request, it just writes the actual request size to the fix header.
      Let's rename the two functions to netbox_begin_request and
      netbox_end_request to emphasize that they are used to start and finish
      request encoding.
      
      Another reason to do that now is that in the following patches we will
      be moving the request table implementation from Lua to C so having a
      "request" in the function names would be confusing.
      
      Part of #6241
      f51b3ffb
    • Vladimir Davydov's avatar
      net.box: rewrite send_and_recv_{iproto,console} in C · b180ea25
      Vladimir Davydov authored
      Basically, this patch converts the above function line-by-line from Lua
      to C. This is a step towards rewriting performance-critical parts of
      net.box in C.
      
      Note, Lua code expects send_and_recv functions return errno, errmsg on
      error and nil, ... on success while it's more convenient to return an
      error object from C code so we do the conversion in Lua wrappers.
      
      Part of #6241
      b180ea25
    • Vladimir Davydov's avatar
      net.box: remove unused timeout argument from IO methods · 86bd5986
      Vladimir Davydov authored
      Part of #6241
      86bd5986
    • Vladimir Davydov's avatar
      net.box: rewrite error decoder in C · 6aafa697
      Vladimir Davydov authored
      This patch moves response_decoder table (which decodes only errors
      despite its name) from Lua to C. This is a step towards rewriting
      performance-critical parts of net.box in C.
      
      Since error_unpack_unsafe is not used by FFI anymore, remove it from
      exports.h
      
      Part of #6241
      6aafa697
    • Alexander V. Tikhonov's avatar
      github-ci: pull all available tags · 1c76fa51
      Alexander V. Tikhonov authored
      Found that Github actions/checkout@v2, when Github Actions triggers
      a job upon the tag push, it makes the following calls [1]:
      
        git fetch origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*
        git fetch origin +$HASH:refs/tags/$TAG
      
      While the first call is fine, the second one spoils the tag and makes
      it non-annotated.
      
      Later, Tarantool calls git describe without --tags flag, and the
      spoiled tag is ignored. This patch handles it by fetching all the
      tags once again explicitly.
      
      Also explained the next step, where we remove the tag. The Github
      Actions may runs twice the same job on branch push and on new tag
      if it was created on this push. To avoid of fails on saving the
      commonly named packages, we drop a tag that points to a current
      commit (if any) on a job triggered by pushing to a branch (it will
      produce the package based on previous tag) and leave the new tag
      in job triggered by tag.
      
      To check this change was used commands to create dump commit and tag
      the new version. Also made twice push with extra flag for tags:
      
        git commit -m"Dump commit" --allow-empty
        git tag -a 2.4.8 -m ''
        git push && git push --tags
      
      In this way would be run 2 jobs for each package:
      
        1. Build package on testing branch. It will produce package with
           name 2.4.7.x (because of the exception mentioned above).
        2. Build package on tag 2.4.8. It will produce package with name
           2.4.8.x.
      
      Closes tarantool/tarantool-qa#119
      
      [1]: https://github.com/actions/checkout/issues/290
      1c76fa51
    • Aleksandr Lyapunov's avatar
      txm: fix ER_TUPLE_FOUND error message in MVCC · 970f90c9
      Aleksandr Lyapunov authored
      At some point ER_TUPLE_FOUND error message was extended to contain
      old tuple and new tuple. It seems that after rebase this change
      was ignored in MVCC code that used previous format. This patch
      fixes that.
      
      Closes #6247
      970f90c9
    • Mergen Imeev's avatar
      sql: remove unused MEM cast functions · 4d5ec7ba
      Mergen Imeev authored
      This patch removes functions that become unused due to changes of
      implicit and explicit cast rules.
      
      Closes #4470
      4d5ec7ba
    • Mergen Imeev's avatar
      sql: remove implicit cast from OP_MakeRecord · b0ef1a56
      Mergen Imeev authored
      This patch removes deprecated implicit cast from OP_MakeRecord opcode,
      which were used in some rare cases, for example during IN operation
      with subselect as right-value.
      
      Closes #4230
      Part of #4470
      b0ef1a56
    • Mergen Imeev's avatar
      sql: remove unnecessary calls of OP_ApplyType · 145e5423
      Mergen Imeev authored
      Since the OP_Seek* opcodes now work using the new implicit casting
      rules, we don't need to call OP_ApplyType before them. This patch
      removes such calls.
      
      Part of #4230
      Part of #4470
      145e5423
    • Mergen Imeev's avatar
      sql: rework OP_Seek* opcodes · 42cd274c
      Mergen Imeev authored
      This patch changes the Seek* opcodes that are used to search in space
      using index. After the redesign, searches using these opcodes work
      according to the new implicit casting rules. However, currently implicit
      cast in these opcodes is not invoked since there is OP_ApplyType before
      them. Unnecessary OP_ApplyType calls will be removed in next patch.
      
      Part of 4230
      Part of 4470
      42cd274c
    • Mergen Imeev's avatar
      sql: remove implicit cast from comparison opcodes · 078bcf00
      Mergen Imeev authored
      After this patch, the new rules will be applied to implicit cast during
      comparison where index is not used. Essentially it means that implicit
      cast from STRING to number during such comparisons was removed.
      
      Part of #4230
      Part of #4470
      078bcf00
    • Mergen Imeev's avatar
      sql: rework implicit cast fo assignment · ac5405bf
      Mergen Imeev authored
      After this patch, the new rules will be applied to implicit cast during
      assignment. According to these rules, all scalar values can be cast to
      SCALAR, all numeric values can be cast to NUMBER, and any numeric value
      can be cast to another numeric type only if the conversion is exact.
      No other implicit cast is allowed.
      
      Part of #4470
      ac5405bf
  3. Aug 06, 2021
    • Serge Petrenko's avatar
      replication: fix flaky election_qsync.test · 096a0a7d
      Serge Petrenko authored
      Fix the test failing occasionally with the following result mismatch:
      
      [001] replication/election_qsync.test.lua             memtx           [ fail ]
      [001]
      [001] Test failed! Result content mismatch:
      [001] --- replication/election_qsync.result	Thu Jul 15 17:15:48 2021
      [001] +++ var/rejects/replication/election_qsync.reject	Thu Jul 15 20:46:51 2021
      [001] @@ -145,8 +145,7 @@
      [001]   | ...
      [001]  box.space.test:select{}
      [001]   | ---
      [001] - | - - [1]
      [001] - |   - [2]
      [001] + | - - [2]
      [001]   | ...
      [001]  box.space.test:drop()
      [001]   | ---
      [001]
      
      The issue happened because row [1] wasn't delivered to the 'default'
      instance from the 'replica' at all. The test does try to wait for [1] to
      be written to WAL and replicated, but sometimes it fails to wait until
      this event happens:
      
      box.ctl.promote() is issued asynchronously once the instance becomes the
      Raft leader. So issuing `box.ctl.wait_rw()` doesn't guarantee that the
      replica has already written the PROMOTE (the limbo is initially
      unclaimed so replica becomes writeable as soon as it becomes the Raft
      leader).
      
      Right after `wait_rw()` we wait for lsn propagation and for 'default'
      instance to reach replica's lsn. It may happen that lsn propagation
      happens due to PROMOTE being written to WAL, and not row [1].
      
      When this is the case, the 'default' instance doesn't receive row [1] at
      all, resulting in the test error shown above.
      
      Fix the issue by waiting for the promotion to happen explicitly.
      
      Part of #5430
      096a0a7d
    • Serge Petrenko's avatar
      box: introduce `box.ctl.demote` · cdb234e1
      Serge Petrenko authored
      Introduce a new journal entry, DEMOTE. The entry has the same meaning as
      PROMOTE, with the only difference that it clears limbo ownership instead
      of transferring it to the issuer.
      
      Introduce `box.ctl.demote`, a counterpart to `box.ctl.promote`, which
      clears the limbo ownership (when elections are off) via writing the DEMOTE, or
      simply makes this instance step down from the leader's role (when elections
      are enabled in any mode).
      
      A new request was necessary instead of simply writing PROMOTE(origin_id
      = 0), because origin_id is deduced from row.replica_id, which cannot be
      0 for replicated rows (it's always equal to instance_id of the row
      originator).
      
      Closes #6034
      
      @TarantoolBot document
      Title: box.ctl.demote
      
      `box.ctl.demote()` is a counterpart to `box.ctl.promote()` which works
      as follows:
       - when `box.cfg.election_mode` is not 'off': make the instance give up
         leadership.
       - when `box.cfg.election_mode` is 'off': write a DEMOTE entry to WAL.
      
      `box.ctl.demote()` may only be issued on the synchronous transaction
      queue owner (i.e. leader when elections are enabled).
      
      A DEMOTE request (DEMOTE = 32) copies PROMOTE behaviour (it clears the
      limbo as well), but clears the synchronous transaction queue  ownership instead
      of assigning it to a new instance.
      cdb234e1
    • Serge Petrenko's avatar
      replication: forbid implicit limbo owner transition · 362e9a66
      Serge Petrenko authored
      Forbid limbo ownership transition without an explicit promote.
      Make it so that synchronous transactions may be committed only after it
      is claimed by some instance via a PROMOTE request.
      
      Make everyone but the limbo owner read-only even when the limbo is
      empty.
      
      Part-of #6034
      
      @TarantoolBot document
      Title: synchronous replication changes
      
      `box.info.synchro.queue` receives a new field: `owner`. It's a replica
      id of the instance owning the synchronous transaction queue.
      
      Once some instance owns the queue, every other instance becomes
      read-only. When the queue is unclaimed, e.g.
      `box.info.synchro.queue.owner` is `0`, everyone may be writeable, but
      cannot create synchronous transactions.
      
      In order to claim or re-claim the queue, you have to issue
      `box.ctl.promote()` on the instance you wish to promote.
      
      When elections are enabled, the instance issues `box.ctl.promote()`
      automatically once it wins the elections, no additional actions are
      required.
      362e9a66
    • Serge Petrenko's avatar
      box: allow calling promote on a candidate · b0192cc1
      Serge Petrenko authored
      Part of #6034
      b0192cc1
    • Serge Petrenko's avatar
      box: split manual and automatic promotion · 8d9326f6
      Serge Petrenko authored
      There are two use cases for box_promote(): it's either invoked manually
      or after a won election round. In the latter case promote() is much
      simpler.
      
      Part-of #6034
      8d9326f6
    • Serge Petrenko's avatar
      box: test the assertion failure after a spurious wakeup in promote · 338e8675
      Serge Petrenko authored
      The failure itself was fixed in 68de8753
      (raft: replace raft_start_candidate with _promote), let's add a
      regression test now.
      
      Follow-up #3055
      338e8675
    • Serge Petrenko's avatar
      box: make promote on the current leader a no-op · ed7fd96d
      Serge Petrenko authored
      It was allowed to call promote on any instance, even when it's already
      the limbo owner (iow Raft leader, when elections are enabled).
      
      This doesn't break anything, when elections are disabled, but is rather
      strange.
      
      When elections are enabled, in contrary, calling promote() should be a
      no-op on the leader. Otherwise it would make the leader read-only until
      it wins the next election round, which's quite inconvenient.
      
      Part-of #6034
      ed7fd96d
    • Serge Petrenko's avatar
      box: make promote always bump the term · d2996fa5
      Serge Petrenko authored
      When called without elections, promote resulted in multiple
      PROMOTE entries for the same term. This is not correct, because all
      the promotions for the same term except the first one would be ignored
      as already seen.
      
      Part-of #6034
      d2996fa5
    • Serge Petrenko's avatar
      box: split promote() into reasonable parts · 4fe5d5db
      Serge Petrenko authored
      box_promote() is a monster. It does a lot of different things based on
      flags: try_wait and run_elections. The flags themselves depend on the
      node's Raft state and the lunar calendar.
      
      Moreover, there are multiple cancellation points and places where
      external state may have changed and needs a re-check.
      
      Things are going to get even worse with the introduction of box.ctl.demote().
      
      So it's time to split up box_promote() into reasonable parts, each doing
      exactly one thing.
      
      This commit mostly addresses the multiple cancellation points issue,
      so that promote() doesn't look like a huge pile of if(something_changed)
      blocks. Some other functions will look like that instead.
      
      Part of #6034
      4fe5d5db
    • Serge Petrenko's avatar
      raft: refactor raft_new_term() · 1dd9bd4f
      Serge Petrenko authored
      Make raft_new_term() always bump the current term, even when Raft is
      disabled.
      
      Part-of #6034
      1dd9bd4f
    • Serge Petrenko's avatar
      replication: send current Raft term in join response · e7f3b3b9
      Serge Petrenko authored
      Make Raft nodes send out their latest persisted term to joining
      replicas.
      
      This is needed to avoid the situation when txn_limbo-managed 'promote
      greatest term' is greater than current Raft term. Otherwise the
      following may happen: replica joins off some instance and receives its
      latest limbo state. The state includes "greatest term seen" and makes
      limbo filter out any data coming from instances with smaller terms.
      Imagine that master this replica has joined from dies before replica has
      a chance to subscribe to it. Then it doesn't receive its current Raft
      term and start elections at smallest term possible, 2 (when there are no
      suitable Raft nodes besides the replica).
      
      Once the elections in a small term number are won, a ton of problems
      arises: starting with filtering out PROMOTE requests for "old" term and
      nop-ifying any data coming from terms smaller than "greatest term seen".
      
      Prerequisite #6034
      e7f3b3b9
    • Serge Petrenko's avatar
      replication: send latest effective promote in initial join · 05ef687f
      Serge Petrenko authored
      A joining instance may never receive the latest PROMOTE request, which
      is the only source of information about the limbo owner. Send out the
      latest limbo state (e.g. the latest applied PROMOTE request) together
      with the initial join snapshot.
      
      Prerequisite #6034
      05ef687f
    • Serge Petrenko's avatar
      replication: add META stage to JOIN · f1c2127d
      Serge Petrenko authored
      The new META stage is part of server's response to a join request.
      It's marked by IPROTO_JOIN_META and IPROTO_JOIN_SNAPSHOT requests and goes
      before the actual snapshot data.
      
      Prerequisite #6034
      
      @TarantoolBot document
      Title: new protocol stage during JOIN
      
      A new stage is added to the stream of JOIN rows coming from master.
      The stage is marked with a bodyless row with type
      IPROTO_JOIN_META = 71
      Once all the rows from the stage are sent out, the JOIN continues as
      before (as a stream of snapshot rows). The end of META stage is marked
      with a row of type IPROTO_JOIN_SNAPSHOT = 72
      
      The stage contains the rows that are necessary for instance
      initialization (current Raft term, current state of synchronous
      transaction queue), but do not belong to any system space.
      f1c2127d
    • Serge Petrenko's avatar
      iproto: make iproto_write_error() blocking in debug · 2b3ca42c
      Serge Petrenko authored
      iproto_write_error() used to be blocking until the commit
      4dac37a6 (iproto: remove
      iproto_write_error_blocking())
      Actually, it should block until the error is written to socket, because
      some tests (vinyl/errinj.test.lua, for example) rely on that.
      
      Do not make iproto_write_error() blocking in release builds for safety
      reasons, as stated in commit above. But make it blocking in debug for
      testing sake.
      
      Part-of #6034
      2b3ca42c
Loading