Skip to content
Snippets Groups Projects
  1. Jan 19, 2023
    • Vladislav Shpilevoy's avatar
      box: rename a few 'cluster's to 'replicaset's · a24714c5
      Vladislav Shpilevoy authored
      In a few places visible to users and in iproto naming the term
      "cluster" really means "replicaset". One of those places is a
      part of public API - box.iproto.key.CLUSTER_UUID - which is not
      yet released.
      
      The commit renames "cluster" in those places as a preparation for
      introduction of actual "cluster", like a set of replicasets. It
      will start from introduction of cluster name in addition to
      replicaset uuid/name.
      
      There are places which still mention 'cluster', but their rename
      would be breaking. It will be addressed in scope of a bigger
      patchset.
      
      Part of #5029
      
      NO_CHANGELOG=Was not released
      
      @TarantoolBot document
      Title: Rename `IPROTO_CLUSTER_UUID` to `IPROTO_REPLICASET_UUID`
      
      This is a name for one of the IProto keys. The key value doesn't
      change and the protocol is still backward compatible. But better
      rename it to `IPROTO_REPLICASET_UUID`, because in future
      `IPROTO_CLUSTER_UUID` will most likely mean a different thing.
      a24714c5
  2. Jan 18, 2023
    • Ilya Verbin's avatar
      log: fix inability to start from the root directory · 6e70fcf0
      Ilya Verbin authored
      The function remove_root_directory, which is used for obtaining module
      names for per-module logging, throws an error when current working
      directory is `/'.
      Rewrite it to fix the bug and rename it to strip_cwd_from_path to make
      the name more clear.
      
      Closes #8158
      
      NO_DOC=unreleased
      NO_CHANGELOG=unreleased
      6e70fcf0
    • Serge Petrenko's avatar
      replication: change bootstrap and replication configuration behaviour · 2f8e2d98
      Serge Petrenko authored
      See the docbot request for details.
      
      Closes #5272
      
      @TarantoolBot document
      Title: new `bootstrap_strategy` configuration option
      
      Default behaviour of replica set bootstrap, replica recovery when
      connecting to remote nodes and replication reconfiguration is changed.
      The new behaviour is controlled by the option `bootstrap_strategy`,
      which has the default value "auto".
      
      Now `replication_connect_quorum` configuration option takes no effect,
      and the effective quorum value for each stage of configuration (quorum
      of established connections, quorum of synced nodes) is determined
      automatically.
      
      On replica set bootstrap, the nodes will refuse to boot, unless a
      majority is reached (this would mean replication_connect_quorum = 3,
      when #box.cfg.repilcation is 4 or 5, for example, or
      replication_connect_quorum = 2, when #box.cfg.replication is 2 or 3).
      Moreover, the bootstrap leader will fail to boot unless it sees that
      every connected node chose it as the bootstrap leader.
      
      On new replica join to an existing cluster, the replica will fail to
      boot only if it couldn't connect to anyone. As long as at least one
      connection is established, the replica will try to join like before.
      
      Moreover, the replica will check that its box.cfg.replication table
      contains every registered node in the cluster, thus ensuring that it has
      tried to connect to everyone and chose the best bootstrap leader
      possible.
      
      On replication reconfiguration on a working instance and recovery from
      local WAL files, the node will try to connect to everyone specified in
      box.cfg.replication. Any number of connections (even no connections)
      will be deemed a success, but the replica will stay in orphan mode until
      it is synced with everyone connected.
      
      If you wish to return to the old behavior, a deprecated setting
      `bootstrap_strategy` = "legacy" is left for now. With
      `bootstrap_strategy` = "legacy", the node behaves exactly like before:
      quorum for both connection and synchronisation is determined by
      `replication_connect_quorum`, and neither bootstrap leader nor joining
      replicas perform any additional checks on bootstrap.
      2f8e2d98
    • Serge Petrenko's avatar
      replication: set default replication_sync_timeout to 0 · 67cb4e4e
      Serge Petrenko authored
      The only observable behaviour of non-zero replication_sync_timeout is
      that it delays box.cfg{replication=...} return until either the node is
      synced with others or the timeout passes.
      
      If the timeout passes without reaching sync, box.cfg{} is exited and the
      node enters "orphan" state, in which it can't write anything until
      either a reconfiguration happens or replicaset is finally synced.
      
      While the previous box.cfg{} call is running (probably waiting for
      replication_sync_timeout), the user can't issue another box.cfg{} call.
      
      So basically, while giving no guarantees that the node exits box.cfg{}
      in fully synced state, the timeout makes reconfiguration harder: even if
      the user knows that the sync won't be achieved, he will have to wait
      until the full timeout passes in order to reconfigure replication.
      
      Let's make the default value of replication_sync_timeout 0 instead of
      300 seconds. The user still may set the timeout to whatever he likes.
      Besides, we have recently introduced box.ctl.on_recovery_state triggers,
      which have a "synced" event, and this is the new recommended way to wait
      until the node is synced with others.
      
      Part-of #5272
      
      @TarantoolBot document
      Title: Changed default value for `box.cfg.replication_sync_timeout
      
      The default value for `replication_sync_timeout` configuration option
      was changed from 300 seconds to 0.
      67cb4e4e
    • Serge Petrenko's avatar
      box: broadcast registered replica uuids in ballot · fd61dc64
      Serge Petrenko authored
      Now the instance appends a list of registered replica set members it
      knows of to its ballot.
      
      Prerequisite #5272
      
      NO_CHANGELOG=not user-visible
      
      @TarantoolBot document
      Title: New fields in instance's ballot.
      
      Instance's ballot (a response to IPROTO_VOTE sent on replica connect)
      receives two new fields:
      1) The uuid of the node this instance considers the bootstrap leader.
         Key: IPROTO_BALLOT_BOOTSTRAP_LEADER_UUID = 0x08
         Value: uuid, encoded as 36-byte string (like
         "bfd2b31c-b740-43e5-bf3c-28538a74c9a6").
      2) An array of registered replica set members uuids.
         Key: IPROTO_BALLOT_REGISTERED_REPLICA_UUIDS = 0x09
         Value: a MP_ARRAY of uuids, each uuid encoded as a 36-byte string
         (like in an example above).
      fd61dc64
    • Serge Petrenko's avatar
      box: broadcast bootstrap leader uuid in ballot · f06825e6
      Serge Petrenko authored
      Note that bootstrap leader uuid is not set when an anonymous replica
      registers, because technically it's not performing a bootstrap.
      
      Prerequisite #5272
      
      NO_DOC=appended to next commit's doc request
      NO_CHANGELOG=not user-visible
      f06825e6
    • Serge Petrenko's avatar
      replication: make replica subscribe to master's ballot · 2a0c4f2b
      Serge Petrenko authored
      Previously replicas chose the remote master to boot from by comparing
      master ballot, which are received in response to IPROTO_VOTE request
      right on connection init.
      
      Such information is not enough in some scenarios. For example, when
      implementing anonymous replicas and retrying relica join, we had to
      restart all connections in order to get the latest ballot information.
      
      Let's change that: make replica subscribe to the built-in
      "internal.ballot" event instead of relying on request-response scheme of
      IPROTO_VOTE. Now replicas always have up-to-date ballot information and
      there is no need to reinitialize connections to update the ballots.
      
      Introduce a new fiber running in tx thread for this purpose: applier
      ballot watcher. The fiber subscribes on "internal.ballot" event and
      watches it all the time while the connection to master is alive.
      
      In case the master isn't aware of IPROTO_WATCH request or of
      "internal.ballot" event, old behaviour is also implemented: ballot
      watcher simply waits for IPROTO_VOTE response and exits.
      
      The ballot watcher is started whenever replica tries to connect or
      reconnect to the remote master and is cancelled whenever its parent
      connection to the master is closed.
      
      We do not put much effort into restarting the fiber and retrying to
      connect in case it fails. For now ballot info is only used during
      bootstrap, and not trying to keep the fiber alive at all costs
      simplifies the code quite a lot.
      
      Later on ballot subscriptions will play a more significant role in
      choosing the bootstrap leader: replicas will re-check remote ballots
      every now and then during the bootstrap leader election.
      
      Part-of #5272
      
      NO_CHANGELOG=internal change
      NO_TEST=tested by existing replication tests
      NO_DOC=internal change
      2a0c4f2b
    • Serge Petrenko's avatar
      box: introduce "internal.ballot" builtin event · e49b9085
      Serge Petrenko authored
      Add a new builtin event carrying instance's ballot information (that is,
      what this instance would normally send in reply to IPROTO_VOTE request).
      
      The event will be watched by connecting replicas to find the bootstrap
      leader.
      
      Prerequisite #5272
      
      NO_DOC=technically user-visible, but not intended for users
      NO_CHANGELOG=see NO_DOC
      e49b9085
    • Serge Petrenko's avatar
      test: rename cluster to replica_set in gh_6260 test · dc635190
      Serge Petrenko authored
      luatest_helpers/cluster module was recently added to luatest trunk and
      renamed to replica_set.
      Let's update its name everywhere in gh_6260_add_builtin_events_test,
      since this test will be amended in the following commits and the new
      module name will be used.
      
      In-scope-of #5272
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      dc635190
    • Georgiy Lebedev's avatar
      lua: fix `print` wrapper to handle errors thrown from `print` · c9986d93
      Georgiy Lebedev authored
      Both of the callbacks in the `print` wrapper are expected to be called, but
      `print` may throw errors, e.g.,
      `print(setmetatable({}, {__tostring = error})`, so we need to call it in a
      protected environment and execute the 'after' callback even if `print`
      throws.
      
      Closes #8136
      
      NO_CHANGELOG=<gh-7186 was not released yet>
      NO_DOC=bugfix
      c9986d93
  3. Jan 16, 2023
    • Vladimir Davydov's avatar
      lua: gracefully fail space on/before replace trigger if txn was aborted · 1a678a5e
      Vladimir Davydov authored
      lbox_push_event_f and lbox_push_event_f callback functions used for
      passing the statement between txn and space on/before replace Lua
      triggers don't assume that the transaction may be aborted by yield
      after the current statement began (this may happen if a trigger callback
      yields). In this case, all statements in txn would be rolled back and
      txn_current_stmt would return NULL, leading to a crash.
      
      Let's fix this by checking if the transaction is still active and
      raising an error immediately if it isn't, thus skipping Lua triggers.
      
      Notes:
       - We merged lbox_pop_txn_stmt_and_check_format into lbox_pop_txn_stmt,
         because the latter is only called by the former.
       - Since lbox_push_event_f callback may now fail, we have to update
         lbox_trigger_run to handle it.
      
      Closes #8027
      
      NO_DOC=bug fix
      1a678a5e
  4. Jan 13, 2023
    • Vladimir Davydov's avatar
      net.box: add info about sequence to remote space object · 457b293c
      Vladimir Davydov authored
      A remote space object presented by a net.box connection mimics the API
      of a local space object presented by box.space. Currently, it misses
      information about sequences. Let's add it.
      
      Note, we have to handle the case when the recently introduced
      _vspace_sequence system space view is missing on the remote host.
      To check that this works correctly, we reuse the 2.10.4 test data
      created by commit 1c33484d ("box: add auth_history and
      last_modified fields to _user space"). We also add the 'gen.lua'
      that can be used to regenerated the data.
      
      Closes #7858
      
      NO_DOC=bug fix
      457b293c
    • Vladimir Davydov's avatar
      schema: create _vspace_sequence system space view · 06ee1134
      Vladimir Davydov authored
      Note, this patch will be backported to 2.10 so we add upgrade function
      for 2.10.5, not for 2.11.0.
      
      Needed for #7858
      
      @TarantoolBot document
      Title: Document `_space_sequence` and `_vspace_sequence` system spaces
      
      The `_space_sequence` system space was added long time ago (in 1.7.5)
      along with the `_sequence` and `_sequence_data` system spaces, but it
      was never documented. The space is used to attach sequences to spaces
      and has the following fields:
       1. 'id', type 'unsigned'. Space id.
       2. 'sequence_id', type 'unsigned'. Id of the attached sequence.
       3. 'is_generated', type 'boolean'. True if the sequence was created
          automatically (`space:create_index('pk', {sequence = true})`)
       4. 'field', type 'unsigned'. Id of the space field that is set using
          the attached sequence.
       5. 'path', type 'string'. Path to the data within the field that is set
          using the attached sequence.
      
      The `_vspace_sequence` is a system space view of the `_space_sequence`
      space that, like any other system space view, shows only rows accessible
      by the current user. It will be introduced in Tarantool 2.10.5.
      06ee1134
    • Aleksandr Lyapunov's avatar
      box: introduce options in box.atomic · 9d1cbda5
      Aleksandr Lyapunov authored
      If the first argument of box.atomic is a non-callable table then
      consider it as options table for box.begin{}.
      
      For test and debug purposes introduce internal getter of current
      transaction isolation level as box.internal.txn_isolation().
      
      Closes #7202
      
      @TarantoolBot document
      Title: Options in box.atomic
      
      Now it's allowed to pass transaction options in the first argument
      of box.atomic(..) call. The options must be a table, exactly as
      in box.begin(..). If options are passed as the first arguments,
      the second and the rest arguments are expected to be a functions
      and its arguments, like in usual box.atomic.
      9d1cbda5
    • Aleksandr Lyapunov's avatar
      txm: carefully handle conflict · 91d6d70f
      Aleksandr Lyapunov authored
      When a transaction is in read-confirmed state it must ignore all
      prepared changes, and if it actually ignores something - it must
      fall to read-view state.
      
      By a mistake the check relied not on actual skipping of a prepared
      statement, but on the fact that there is a deleting statement. That
      leads to excess conflicts for transactions with read-committed
      isolation level.
      
      Fix it by raising a conflict only if a deleting statement is skipped.
      
      Closes #8122
      Needed for #7202
      
      NO_DOC=bugfix
      91d6d70f
    • Aleksandr Lyapunov's avatar
      txm: free read lists of a transaction when it's prepared · ba8e00c0
      Aleksandr Lyapunov authored
      Read lists (read set and other similar lists) are used only for
      detecting a conflict when another transaction is committed.
      Once a transaction is prepared (no matter with success or not)
      those lists are no more needed. Moreover, in some part of code it
      is expected that there can be no read set of already prepared tx.
      
      So let's clean those lists once a transaction is prepared.
      
      Closes #7945
      
      NO_DOC=bugfix
      ba8e00c0
    • Aleksandr Lyapunov's avatar
      memtx: simplify check_dup function · a5dd48ed
      Aleksandr Lyapunov authored
      There ware two functions - check_dup_clean and check_dup_dirty.
      Merge them to one.
      
      Also extract phantom checks from check_dup and call them explicitly.
      That will additionally simplify check_dup and will allow to get rid
      of temporary conflict trackers - memtx_tx_conflict. Note that this
      kind of object will remain in memory monitoring by now. It will be
      removed later.
      
      No logical changes.
      
      Part of #8122
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      a5dd48ed
  5. Jan 12, 2023
    • Vladimir Davydov's avatar
      txn: fail ro stmt if transaction is aborted · 5f1500f4
      Vladimir Davydov authored
      We fail write statements if the current transaction was aborted by yield
      or timeout. We should fail read-only statements in this case, as well.
      Note, we already fail read-only statements if the current transaction
      was aborted by conflict.
      
      Closes #8123
      
      NO_DOC=bug fix
      5f1500f4
  6. Jan 11, 2023
    • Georgiy Lebedev's avatar
      box: overriding IPROTO request handlers · 998964c8
      Georgiy Lebedev authored
      Add `IPROTO_UNKNOWN` command code for overriding the unknown request
      handler.
      
      Change request type variable types related to IPROTO to `uint32_t`.
      
      Add request handler hash table to transaction thread and request handler
      set to IPROTO threads for storing overridden request handlers: TX thread
      notifies IPROTO threads about overridden request handlers using IPROTO
      configuration message.
      
      If a given request handler is overridden, the IPROTO thread does not
      preprocess it and sends the package immediately over a dedicated route. If
      later it is necessary to fallback to the system handler, the message
      decoding and dispatching is done in the TX thread.
      
      Add new `box.iproto.override` method to Lua and `box_iproto_override` to C
      API, which allow setting IPROTO request handler callbacks.
      
      Closes #7901
      
      @TarantoolBot document
      Title: Document overriding IPROTO request handlers feature
      For the API description and usage examples, see:
      * [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#0f84694523214c0e9bf2f3d75cccace4);
      * tarantool/tarantool#7901.
      998964c8
    • Georgiy Lebedev's avatar
      msgpack: add alias dictionary to MsgPack object used during indexing · 96dfce74
      Georgiy Lebedev authored
      Add an optional dictionary field to MsgPack object which can be used during
      indexing for aliasing string keys.
      
      Needed for #7901
      
      NO_CHANGELOG=<internal feature>
      NO_DOC=<internal feature>
      96dfce74
  7. Jan 10, 2023
  8. Dec 27, 2022
    • Mergen Imeev's avatar
      sql: introduce SEQSCAN to SELECT · 77648827
      Mergen Imeev authored
      This patch introduces new keyword SEQSCAN and new restrictions on
      SELECTs. These restrictions are disabled by default.
      
      Closes #7747
      
      @TarantoolBot document
      Title: SEQSCAN
      
      Now scanning SELECT will not run and will throw an error if the new
      SEQSCAN keyword is not used for scanned spaces. This change only affects
      SELECT and does not affect UPDATE and DELETE. A SELECT is recognized as
      a scanning SELECT if `EXPLAIN QUERY PLAN SELECT ...` indicates that the
      SELECT `scans` rather than `searches`.
      
      For example, if we have spaces created with these queries:
      ```
      CREATE TABLE t(i INT PRIMARY KEY, a INT);
      CREATE TABLE s(i INT PRIMARY KEY, a INT);
      ```
      
      Then these queries will throw an error:
      ```
      SELECT * FROM t;
      SELECT * FROM t WHERE a > 1;
      SELECT * FROM t WHERE i + 1 = 5;
      SELECT * FROM t, s;
      SELECT * FROM t JOIN s;
      ```
      
      And these will not:
      ```
      SELECT * FROM t WHERE i > 1;
      SELECT * FROM SEQSCAN t;
      SELECT * FROM SEQSCAN t WHERE i + 1 = 5;
      SELECT * FROM SEQSCAN t, SEQSCAN s;
      SELECT * FROM SEQSCAN t JOIN SEQSCAN s;
      ```
      
      Scanning can be allowed or disallowed by default. To do this, a new
      session setting is introduced: `sql_seq_scan`. The default value for
      setting is `true`, i.e. scanning is allowed. When set to `false`, the
      scanning SELECTs will throw a `scanning is not allowed` error.
      77648827
    • Vladimir Davydov's avatar
      box: add stubs for extra authentication checks · 9422ca70
      Vladimir Davydov authored
      This commit adds a few configuration options, function stubs, and error
      codes that will be used to perform extra security checks in EE:
      
       * box.cfg.auth_delay. Type: double. Default: 0. Unit: seconds.
      
         Description: If authentication of a user fails, the next
         authentication attempt for the same user will fail with
         ER_AUTH_DELAY error if called before box.cfg.auth_delay passes.
      
         Implementation: Error will be raised by security_check_auth_pre()
         called by authenticate() right before checking the challenged
         password. Authentication failures will be accounted per user in
         session_on_auth trigger.
      
       - box.cfg.disable_guest. Type: boolean. Default: false.
      
         Description: If set, an attempt to perform any request except
         'auth', 'ping', 'id', or 'vote' over iproto without authentication
         or authenticated as guest will raise ER_AUTH_REQUIRED error.
      
         Implementation: Error will be raised by security_check_session()
         called by tx_check_msg(), which in turn is called before starting
         to process any message received over iproto.
      
       - box.cfg.password_lifetime_days. Type: number. Default: 0. Unit: days.
      
         Description: If > 0, an attempt to authenticate as a user that hasn't
         reset the password for more than box.cfg.password_lifetime_days will
         fail with ER_PASSWORD_EXPIRED error.
      
         Implementation: Error will be raised by security_check_auth_post()
         called by authenticate() right after successfully authenticating
         the user. Note, we can't raise the error in security_check_auth_pre()
         because that would enable user enumeration.
      
      All the new options are dynamic. The option values will be stored and
      used in C code so we'll have to define a configuration callback for them
      in EE: box.internal.cfg_set_security. Also note that since the new
      options configure authentication behavior, they should be set before
      the box port is opened: we'll do that in security_cfg() called by
      box_storage_init().
      
      Needed for https://github.com/tarantool/tarantool-ee/issues/299
      Needed for https://github.com/tarantool/tarantool-ee/issues/300
      Needed for https://github.com/tarantool/tarantool-ee/issues/301
      
      NO_DOC=ee
      NO_TEST=ee
      NO_CHANGELOG=ee
      9422ca70
    • Sergey Bronnikov's avatar
      http: escape encoded params before sending request · 438a78a9
      Sergey Bronnikov authored
      Closes #7931
      
      @TarantoolBot document
      Title: Document a percent-encoding of params passed to http client
      
      The HTTP client has `params` option, where a user may provide a table of
      query parameters (added in #6832). Those parameters are encoded into a
      `?foo=bar&tweedledoo=tweedledee` string verbatim. If a name or a value of
      a query parameter contains `&`, `=` (or any another symbol with specific
      meaning in the URI query component), the query may be interpreted
      incorrectly by a server. Now key and values passed in a table as
      `params` option are percent-encoded and then encoded to a query string.
      This will be made automatically. Percent-encoding depends on used HTTP
      method: with `GET`, `HEAD` and `DELETE` parameters `uri.QUERY_PART`
      are used and with other HTTP method `uri.FORM_URLENCODED` is used.
      438a78a9
    • Sergey Bronnikov's avatar
      uri: escape params · a3b2056b
      Sergey Bronnikov authored
      Commit "uri: encode table with http params to a string" (b31aec89)
      introduced two functions `params()` and `encode_kv()` that encodes a
      table with http key-value parameters to a http query string. However,
      keys and values could be interpreted wrong by http server if they
      contains reserved symbols that have special meaning for http server.
      
      Patch adds escaping for key-values before encoding params to query
      string.
      
      Needed for #7931
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      a3b2056b
    • Sergey Bronnikov's avatar
      uri: speedup encode and decode functions · dcd46244
      Sergey Bronnikov authored
      
      Patch replaces encoding and decoding functions written in Lua with
      functions implemented in C.
      
      Performance of Lua implementation (before the patch):
      
      ```
      uri.escape   152.37  runs/sec
      uri.unescape 263.44  runs/sec
      ```
      
      Performance of C implementation (after the patch):
      
      ```
      uri.escape   4983.03  runs/sec
      uri.unescape 4197.19  runs/sec
      ```
      
      Follows up #3682
      
      NO_CHANGELOG=see previous commit
      NO_DOC=see previous commit
      
      Co-authored-by: default avatarAlexander Turenko <alexander.turenko@tarantool.org>
      dcd46244
    • Sergey Bronnikov's avatar
      uri: add escape and unescape functions · 5ebe1865
      Sergey Bronnikov authored
      Closes #3682
      
      @TarantoolBot document
      Title: Document a new functions to encode and decode parts of URI
      
      New functions `uri.escape()` and `uri.unescape()` have been introduced.
      First one allows to escape symbols to a string and second one to
      unescape symbols to a string according to RFC 3986 [1].
      
      Examples:
      
      ```
      tarantool> uri.escape("тарантул")
      ---
      - '%D1%82%D0%B0%D1%80%D0%B0%D0%BD%D1%82%D1%83%D0%BB'
      ...
      
      tarantool> uri.unescape("%D1%82%D0%B0%D1%80%D0%B0%D0%BD%D1%82%D1%83%D0%BB")
      ---
      - тарантул
      ...
      
      ```
      
      `uri.escape()` accepts a string that will be encoded and optionally a
      table with encoding options: string with unreserved symbols that will
      *not* be encoded and boolean option that enables/disables encoding of a
      space characters as '+'. By default `uri.escape()` uses a set of
      unreserved symbols defined in RFC 3986 ("2.3. Unreserved Characters")
      and encoding of space characters as '+' is disabled. Table with default
      encoding options is defined as `uri.RFC3986`.
      
      `uri.unescape()` accepts a string that will be decoded and optionally a
      table with decoding options: string with unreserved symbols (these
      symbols are actually unused by decoding function) and boolean option
      that enables/disables decoding of '+' as a space character. Table with
      default decoding options is defined as `uri.RFC3986`.
      
      See detailed description in RFC "http: add percent-encoding/decoding of
      query string in request" [2].
      
      NO_WRAP
      1. Uniform Resource Identifier (URI): Generic Syntax
         https://datatracker.ietf.org/doc/html/rfc3986
      2. https://www.notion.so/tarantool/http-add-percent-encoding-decoding-of-query-string-in-request-76a2425a4c4744a1a72643527a4fe7f7
      NO_WRAP
      5ebe1865
    • Georgiy Lebedev's avatar
      box: sending arbitrary IPROTO packets · b9892247
      Georgiy Lebedev authored
      Add translation table for `box.iproto.key` constants encoding to simplify
      packet assembly.
      
      Add new `box.iproto.send` method to Lua and `box_iproto_send` function to C
      API, which allow sending arbitrary IPROTO packets, using active IPROTO
      sessions. Packets are sent asynchronously using Kharon.
      
      Add `xregion_join` to the `xalloc` API.
      
      Change gh-7894 test: instead of simply comparing `box.iproto` table to the
      reference table, iterate over `box.iproto` and check that corresponding
      non-{function, thread, userdata} type values exist in the reference table.
      
      Closes #7897
      
      @TarantoolBot document
      Title: Document sending arbitrary IPROTO packets feature
      For the API description and usage examples, see:
      * [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#a2cc04da89d34fad8f8564c150cd9977);
      * tarantool/tarantool#7897.
      b9892247
    • Georgiy Lebedev's avatar
      msgpack: use `lua_hash{string}` for encoding with translation table · 92fb07b2
      Georgiy Lebedev authored
      The translation table (and the future msgpack object dictionary) uses
      string keys coming from Lua and hence using `lua_hash{string}` as the hash
      function is beneficial.
      
      Needed for #7897
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      92fb07b2
    • Georgiy Lebedev's avatar
      test: fix leak of fiber region in `test_key_def_dup` of module API test · 1398cf6b
      Georgiy Lebedev authored
      `test_key_def_dup` uses `box_key_def_dump_parts` which allocates memory on
      the fiber region, but the test does not clean it up: fix that by creating
      a region savepoint and truncating the fiber region in the end of the test.
      
      Needed for #7897
      
      NO_CHANGELOG=<test fix>
      NO_DOC=<test>
      1398cf6b
    • Georgiy Lebedev's avatar
      trivia: add string conversion to lower case · e56ee46c
      Georgiy Lebedev authored
      In some cases we need to convert a string to lower case (e.g., when
      normalizing an upper-case constant): add helper functions that do this
      in-place or by creating a copy of the original string.
      
      Needed for #7897
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      e56ee46c
  9. Dec 26, 2022
    • Nikita Zheleztsov's avatar
      replication: replicaset state machine assert fail · 7ec82674
      Nikita Zheleztsov authored
      Currently replicaset state machine tracking the number of connected,
      loading and synced appliers may perform unnecessary decrementing of
      their count. On debug version this may lead to assertion failure.
      Here's the way it may happen:
        1. Any kind of exception occurs in applier thread and leads to
           invoking its destructor (applier_thread_data_destroy), which
           is set with scoped guard;
        2. Cbus call is made in order to remove the corresponding applier
           from the thread. According to the fact that cbus_call is
           synchronous, we yield, waiting for the result from the applier
           thread.
        3. During yielding user calls reconfiguration, which invokes
           replicaset_update. Old appliers are pruned: for every replica
           trigger on changing state machine counter is deleted after which
           we stop fiber and wait its join.
        4. If the first replica in replicaset_foreach is not the errored
           one and the errored fiber wakes up during yielding with
           fiber_join, then zero decrementing happens.
      
      Let's clear the above mentioned triggers for all replicas at the
      first place and only after that stop and join their applier fibers.
      
      Closes #7590
      
      NO_DOC=bugfix
      7ec82674
    • Gleb Kashkin's avatar
      fiber: add channel close mode option to compat · de9b9308
      Gleb Kashkin authored
      Before the change there was an unexpected behavior when using
      channel:close(), as it closed the channel entirely and discarded all
      unread events.
      
      This commit introduces graceful channel close option in tarantool.compat
      (gh-7000) that allows to select new or old behavior.
      
      With the new behavior `close()` marks channel as closed for writing.
      Only when all events are extracted, the channel is closed entirely. If
      there are no events in the channel, it is closed as usual.
      
      Document that describes new API can be found on notion (private):
      https://www.notion.so/fiber-channel-graceful-close-53b2788ed1f144598c4c0e1229c2eb69
      
      Requires #7060
      Requires #8007
      Closes #7746
      See also #7000
      
      @TarantoolBot document
      Title: new compat option fiber_channel_close_mode
      
      New behavior is gracefully closing fiber channel by marking it
      read-only, instead of destroying. Full API description can be found
      on notion (private):
      https://www.notion.so/tarantool/Fiber-channel-graceful-close-53b2788ed1f144598c4c0e1229c2eb69
      de9b9308
  10. Dec 25, 2022
    • Alexander Turenko's avatar
      test: stabilize jit.dump() smoke test · 92794ddc
      Alexander Turenko authored
      
      A jiggle in tarantool's initialization code can lead to trace numbers
      above 9. We can either:
      
      * accept trace numbers above 9 in the test
      * or drop traces from the initialization code in the test and assume
        that the new trace will have number 1.
      
      Igor Munkin suggested to stick with the second approach to avoid
      dependency on the initialization code.
      
      NO_DOC=test fixup, no user visible changes
      NO_CHANGELOG=see NO_DOC
      
      Co-authored-by: default avatarIgor Munkin <imun@tarantool.org>
      92794ddc
  11. Dec 23, 2022
    • Ilya Verbin's avatar
      core: unblock fatal (crashing) signals in all threads · 706bfea4
      Ilya Verbin authored
      Currently all non-main threads have all the signals blocked, however
      according to `man pthread_sigmask':
      
      > If SIGBUS, SIGFPE, SIGILL, or SIGSEGV are generated while they
      > are blocked, the result is undefined, unless the signal was
      > generated by kill(2), sigqueue(3), or raise(3).
      
      On macOS they are actually blocked, causing the faulting instruction
      to loop indefinitely. While on Linux they are not blocked, however the
      signal handler registered by sigaction is not executed. Don't block them.
      
      Closes #8023
      Closes #8083
      
      NO_DOC=bugfix
      706bfea4
    • Ilya Verbin's avatar
      crash: handle all unblockable signals · 6c3ce7a6
      Ilya Verbin authored
      There are 4 fatal signals that cannot be blocked by sigmask if they
      are caused by the CPU exception (rather than kill, sigqueue or raise):
      SIGILL, SIGBUS, SIGFPE, SIGSEGV. Currently the crash module handles
      only SIGSEGV and SIGFPE. This patch adds handlers for SIGBUS and SIGILL.
      SIGBUS is usually raised by macOS on access to the unmapped memory, and
      SIGILL is possible, for example, while running AVX version of memcpy on
      a CPU without AVX support.
      
      Faulting address siginfo->si_addr is valid for all these signals, so
      print it unconditionally.
      
      Part of #8023
      Part of #8083
      
      NO_DOC=See next commit
      NO_CHANGELOG=See next commit
      6c3ce7a6
    • Andrey Saranchin's avatar
      core: drop constraints in two phases · f72efbc2
      Andrey Saranchin authored
      Currently, core constraints are dropped on commit. That is why
      it is impossible to drop constraint and drop objects it references to
      at the same transaction. Let's drop constraints in two steps - detach
      them when DDL occurs, then reattach on rollback or delete on commit.
      
      Closes #7339
      
      NO_DOC=bugfix
      f72efbc2
  12. Dec 22, 2022
    • Mergen Imeev's avatar
      sql: drop code of SQL check constraint · ea31df2d
      Mergen Imeev authored
      This patch removes code that was used to implement the SQL check
      constraint as they are now replaced by BOX constraint. Also, the syntax
      for enabling/disabling check constraints has been removed as BOX
      constraints do not support this feature.
      
      Follow-up #6986
      
      NO_DOC=Already introduced.
      NO_CHANGELOG=Already introduced.
      ea31df2d
    • Vladimir Davydov's avatar
      box: add password history check stub · 5424e3e8
      Vladimir Davydov authored
      - Add box.cfg.password_history_length configuration option.
        It will specify the max number of entries to keep in the
        auth_history field of the _user system space.
      - Add new error code ER_OLD_PASSWORD, which will be raised on
        an attempt to reuse an old password.
      - Set auth_history if box.internal.prepare_auth_history is defined.
        The function takes a user id.
      - Add auth_history argument to box.internal.check_password so that
        it can check if the new password matches an old one.
      - Add box_lua_security_init, because auth checks will be done in C.
      
      Needed for https://github.com/tarantool/tarantool-ee/issues/298
      
      NO_DOC=ee
      NO_TEST=ee
      NO_CHANGELOG=ee
      5424e3e8
    • Vladimir Davydov's avatar
      box: add auth_history and last_modified fields to _user space · 1c33484d
      Vladimir Davydov authored
      See the doc bot request for the description of the new fields.
      
      Note that we only store the value of the 'last_modified' field
      in struct user_def, because 'auth_history' will be used only in
      Lua code.
      
      Needed for https://github.com/tarantool/tarantool-ee/issues/298
      Needed for https://github.com/tarantool/tarantool-ee/issues/299
      
      NO_CHANGELOG=no user-visible effects in CE; will be added to EE
      
      @TarantoolBot document
      Title: Document auth_history and last_modified _user space fields
      
      Field name: auth_history. Field no: 6. Type: array.
      Description: The field stores an array of previous authentication data:
      when a user password is changed, the last value of the 'auth' field is
      appended to 'auth_history'.  The length of the history is configured by
      the `box.cfg.password_history_length` option, which is available only
      in Tarantool EE, where it's used to prevent users from reusing old
      passwords. In Tarantool CE, the array is always empty.
      
      Field name: last_modified. Field no: 7. Type: unsigned.
      Description: The field stores the timestamp (seconds since Unix epoch)
      of the last user password update. It's never used in Tarantool CE.
      In Tarantool EE, it's used to disable users that haven't changed the
      password for more than `box.cfg.password_lifetime_days`.
      
      `box.schema.upgrade()` sets the new field values to an empty array
      and 0 for users that haven't updated them yet.
      1c33484d
Loading