Skip to content
Snippets Groups Projects
  1. Jun 04, 2024
    • Vladislav Shpilevoy's avatar
      box: introduce box_localize_vclock · b8463960
      Vladislav Shpilevoy authored
      The function takes the burden of explaining why this hack about
      setting local component in a remote vclock is needed. It also
      creates a new vclock, not alters an existing one. This is to
      signify that the vclock is no longer what was received from a
      remote host.
      
      Otherwise it is too easy to actually mistreat this mutant vlock as
      a remote vclock. That btw did happen and is fixed in following
      commits.
      
      In scope of #10047
      
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      b8463960
  2. May 31, 2024
    • Alexander Turenko's avatar
      connpool: move source code file · e0c9b652
      Alexander Turenko authored
      The source files for built-in Lua modules are generally placed on the
      same level: in `src/lua` or in `src/box/lua`, disregarding whether
      they're public or internal.
      
      The recently introduced `experimental.connpool` built-in module is
      placed in the `experimental` subdirectory.
      
      This commit moves `src/box/lua/experimental/connpool.lua` to
      `src/box/lua/connpool.lua` to follow the existing file structure.
      Public, internal and experimental modules are all on the same level now.
      
      The `connpool` module is still experimental and
      `require('experimental.connpool')` is needed to use it.
      
      This commit doesn't change the code of the module.
      
      NO_DOC=no code changes
      NO_CHANGELOG=see NO_DOC
      NO_TEST=see NO_DOC
      e0c9b652
  3. May 29, 2024
    • Georgiy Lebedev's avatar
      box: prevent demoted leader from being a candidate in the next elections · 05d03a1c
      Georgiy Lebedev authored
      
      Currently, the demoted leader sees that nobody has requested a vote in the
      newly persisted term (because it has just written it without voting, and
      nobody had time to see the new term yet), and hence votes for itself,
      becoming the most probable winner of the next elections.
      
      To prevent this from happening, let's forbid the demoted leader to be a
      candidate in the next elections using `box_raft_leader_step_off`.
      
      Closes #9855
      
      NO_DOC=<bugfix>
      
      Co-authored-by: default avatarSerge Petrenko <sergepetrenko@tarantool.org>
      05d03a1c
    • Georgiy Lebedev's avatar
      box: refactor `box_demote` to make it more comprehensible · ff010fe9
      Georgiy Lebedev authored
      
      Suggested by Nikita Zheleztsov in the scope of #9855.
      
      Needed for #9855
      
      NO_CHANGELOG=<refactoring>
      NO_DOC=<refactoring>
      NO_TEST=<refactoring>
      
      Co-authored-by: default avatarNikita Zheleztsov <n.zheleztsov@proton.me>
      ff010fe9
    • Georgiy Lebedev's avatar
      txn: run statement `on_rollback` triggers before rolling back statement · d529082f
      Georgiy Lebedev authored
      Logically, we call triggers after running statements. These triggers can
      make significant changes (for instance, DDL triggers), so, for consistency,
      we should call the statement's `on_rollback` triggers before rolling back
      the statement. This also adheres to the logic that transaction
      `on_rollback` triggers are called before rolling back individual
      transaction statements.
      
      One particular bug that this patch fixes is rolling back of DDL on the
      `_space` space. DDL is essentially a replace operation on the `_space`
      space, which also invokes the `on_replace_dd_space` trigger. In this
      trigger, among other things, we swap the indexes of the original space,
      `alter->old_space`, which is equal to the corresponding transaction
      `stmt->space`, with the indexes of the newly created space,
      `alter->new_space`:
      https://github.com/tarantool/tarantool/blob/de80e0264f7deb58ea86ef85b37b92653a803430/src/box/alter.cc#L1036-L1047
      
      If then a rollback happens, we first rollback the replace operation, using
      `stmt->space`, and only after that do we swap back the indexes in
      `alter_space_rollback`:
      https://github.com/tarantool/tarantool/blob/de80e0264f7deb58ea86ef85b37b92653a803430/src/box/memtx_engine.cc#L659-L669
      https://github.com/tarantool/tarantool/blob/de80e0264f7deb58ea86ef85b37b92653a803430/src/box/alter.cc#L916-L925
      
      For DDL on the _space space, the replace operation and DDL occur on the
      same space. This means that during rollback of the replace, we will try to
      do a replace in the empty indexes that were created for `alter->new_space`.
      Not only does this break the replace operation, but also the newly inserted
      tuple, which remains in the index, gets deleted, and access to it causes
      undefined behavior (heap-use-after-free).
      
      As part of the work on this patch, tests of rollback of DDL on system
      spaces which use `on_rollback` triggers were enumerated:
      * `_sequence` — box/sequence.test.lua;
      * `_sequence_data` — box/sequence.test.lua;
      * `_space_sequence` — box/sequence.test.lua;
      * `_trigger` — sql/ddl.test.lua, sql/errinj.test.lua;
      * `_collation` — engine-luatest/gh_4544_collation_drop_test.lua,
                       box/ddl_collation.test.lua;
      * `_space` — box/transaction.test.lua, sql/ddl.test.lua;
      * `_index` — box/transaction.test.lua, sql/ddl.test.lua;
      * `_cluster` — box/transaction.test.lua;
      * `_func` — box/transaction.test.lua, box/function1.test.lua;
      * `_priv` — box/errinj.test.lua,
                  box-luatest/rollback_ddl_on__priv_space_test.lua;
      * `_user` — box/transaction.test.lua,
                  box-luatest/gh_4348_transactional_ddl_test.lua.
      
      Closes #9893
      
      NO_DOC=<bugfix>
      d529082f
    • Georgiy Lebedev's avatar
      box: pass statement being rolled back (if any) to `priv_grant` · 797c04ff
      Georgiy Lebedev authored
      In scope of #9893 we are going to run statement `on_rollback` triggers
      before rolling back the corresponding statement. During rollback of DDL in
      the `_priv` space, the database is accessed from `user_reload_privs` to
      reload user privileges, so we need it to account for the current statement
      being rolled back: i.e., the new tuple that was introduced (if any) must
      not be used, while the old tuple (if any) must be used.
      
      Needed for #9893
      
      NO_CHANGELOG=<refactoring>
      NO_DOC=<refactoring>
      797c04ff
    • Magomed Kostoev's avatar
      Add the extra include directory for salad library · 4ccb6dbc
      Magomed Kostoev authored
      It's used to introduce new data structures in the Tarantool EE.
      
      NO_DOC=no functional changes
      NO_TEST=no functional changes
      NO_CHANGELOG=no functional changes
      4ccb6dbc
    • Vladislav Shpilevoy's avatar
      test: fix flaky downstream lag test · d4ea121b
      Vladislav Shpilevoy authored
      It could fail in ASAN build. Can't tell why just there.
      
      The main reason was that in a topology server1 + server2->server3
      one of the cases
      - did a txn on server1,
      - then enabled server2->server3 replication,
      - then waited for server2->server3 sync,
      - and instantly assumed the txn reached server3.
      
      Surely it not always did. At the server2->server3 sync the txn
      might not had reached server2 itself yet.
      
      The fix is as simple as explicitly ensure the txn is on server2
      before waiting server2->server3 sync.
      
      Another potential for flakiness was that the default timeout in
      luatest.helpers.retrying is super low, just 5 seconds. The patch
      manually bumps it to 60 seconds to be sure any future failures
      wouldn't be related to too small timeout.
      
      Closes #10031
      
      NO_DOC=test
      NO_CHANGELOG=test
      d4ea121b
  4. May 28, 2024
    • Nikolay Shirokovskiy's avatar
      error: use typed errors with diag instead of their ClientError mappings · 01915e6d
      Nikolay Shirokovskiy authored
      We have 2 options to set for example `IllegalParams` error in C code:
      
      `diag_set(IllegalParams, ...)`
          and
      `diag_set(ClientError, ER_ILLEGAL_PARAMS, ...)`
      
      We will have different error messages and error types in these 2 cases.
      Let's leave only `IllegalParams` and add an assertion for the second case.
      
      Also let's instantiate `IllegalParams` error from Lua in expressions
      like `box.error(box.error.ILLEGAL_PARAMS)`.
      
      Part of #9914
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      01915e6d
    • Nikolay Shirokovskiy's avatar
      error: drop OOM hanldling · 32c49236
      Nikolay Shirokovskiy authored
      Since the commit 5a031fb6 ("exception: drop OOM handling") we
      use xmalloc for allocating exceptions so we don't need to check
      for NULL.
      
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      32c49236
  5. May 24, 2024
    • Georgiy Lebedev's avatar
      static-build: bump the OpenSSL library version to 3.2.1 · 8de22969
      Georgiy Lebedev authored
      
      Bump the OpenSSL library version to 3.2.1 and remove OpenSSL patches which
      are already present in the updated library version.
      
      Disable modules in OpenSSL configuration to make sure the OpenSSL 3.0
      legacy provider is compiled into the library.
      
      Closes #7502
      
      NO_DOC=<dependency bump>
      NO_TEST=<dependency bump>
      
      Co-authored-by: default avatarSergey Bronnikov <sergeyb@tarantool.org>
      8de22969
    • Alexander Turenko's avatar
      config: give better error if a role is not a table · e1c2b237
      Alexander Turenko authored
      An attempt to use a non-table module as a role now reports a more
      descriptive error:
      
      > Unable to use module <...> as a role: expected table, got <...>
      
      Fixes #10049
      
      NO_DOC=The error is reported as before this commit as well as after it,
             but the error message is changed. The error message is not part
             of the API.
      e1c2b237
    • Alexander Turenko's avatar
      config: expose configuration status from box.info · a1544d3b
      Alexander Turenko authored
      Fixes #10044
      
      @TarantoolBot document
      Title: Configuration status is shown in `box.info.config` now
      
      There is the `config:info([version])` method, but in order to access it
      over iproto an application developer should add something like the
      following into the application code:
      
      ```lua
      _G.config = require('config')
      ```
      
      It is not convenient, at least because it requires an attention from the
      application developer and it can't be solved solely by an administrator.
      
      Now, the `config:info('v2')` result is reported in the `config` field of
      the `box.info` table. It is accessible over iproto if appropriate
      privileges are granted for a calling user.
      a1544d3b
    • Aleksandr Lyapunov's avatar
      net.replicaset: fix a flaky test · f1efc1bd
      Aleksandr Lyapunov authored
      Fix several problems that led to unexpected results:
      * Handle error thrown by box.clt.promote. Not very often but
        it could fail ruining the test. Fixed by retry.
      * It's not so obvious but RO server after successful call of
        box.ctl.promote for some time remains RO. On the other hand
        both net.replicaset and the test rely box.info.ro to define
        the leader, for in their view the actual switch to the new
        leader happens some (perhaps different) time after call to
        box.ctl.promote. That asynchrony sometimes led to unexpected
        results. Fixed by waiting of RW status.
      * The fix above (wait for RW of the new leader) introduced new
        problem: in case with sequential leader change and broadcast
        of new values, the new leader has good chances (but not 100%)
        to deliver old broadcasted value to the watcher. Fix it by
        splitting the sequence to the leader change with verification
        and broadcast with verification.
      * The similar problem happens when a test starts. It may happen
        that all instances are RO, so the initial broadcast on actual
        leader is run with wrong value (that a replica must use).
        Fix it by waiting for a leader to become RW.
      
      Follow-up #9823
      
      NO_DOC=fix flaky test
      NO_CHANGELOG=fix flaky test
      f1efc1bd
    • Aleksandr Lyapunov's avatar
      test: add much more tests for update splice · a9846f49
      Aleksandr Lyapunov authored
      This patch tests update splice operation more thoroughly:
      * All variants of splice: delete/insert/replace/noop.
      * All variants of position: negative/positive/zero/OOB.
      * One operation and several sequential operations.
      * Tuple update, space update, space upsert.
      * String of varbinary field/arg.
      
      Follow up #9997
      Closes #10032
      
      NO_DOC=tests
      NO_CHANGELOG=tests
      a9846f49
    • Aleksandr Lyapunov's avatar
      box: support varbinary in splice update operation · aaa0ad0c
      Aleksandr Lyapunov authored
      @TarantoolBot document
      Title: splice update operation now supports varbinary
      
      Just in case splice is an update operation that can modify string
      field, deleting a part of it and inserting something instead.
      ```
      tarantool> box.tuple.new{'1234567'}:update{{':', 1, 2, 3, '!'}}
      ---
      - ['1!567']
      ...
      ```
      In the this example splice operation ':' takes field 1 of tuple,
      takes position 2 in string, removes 3 symbols and inserts '!' to
      that position.
      
      Both deletion and insertion can naturally degrade to no-op.
      
      Before this patch a splice operation required both field and
      inserting argument to be strings.
      
      This patch allows the field and the argument to be varbinary.
      ```
      tarantool> varb = require('varbinary')
      tarantool> t = box.tuple.new{varb.new('1234567')}
      tarantool> t:update{{':', 1, 2, 3, varb.new('!')}}
      ---
      - [!!binary MSE1Njc=]
      ...
      
      ```
      
      That also allows to insert strings into varbinary fields and
      insert varbinary data into string. The actual field type after
      update such operation remains the same, so updated string will
      be string, while updated varbinary will be varbinary.
      
      Closes #9997
      aaa0ad0c
    • Nikolay Shirokovskiy's avatar
      lua: add box.error.is · 8ed61049
      Nikolay Shirokovskiy authored
      Part of #9914
      
      @TarantoolBot document
      Title: Add box.error.is
      Product: Tarantool
      Since: 3.2
      
      The function checks whether argument passed is box.error.
      
      tarantool> box.error.is(box.error.new(box.error.UNKNOWN))
      ---
      - true
      ...
      
      tarantool> box.error.is('foo')
      ---
      - false
      ...
      8ed61049
    • Nikolay Shirokovskiy's avatar
      error: support level with errcode · 09ccd29a
      Nikolay Shirokovskiy authored
      The level argument has same meaning as when error created or raised
      with table/error argument. Examples:
      
      ```lua
      e = box.error.new(box.error.SOME_ERROR, <error arguments>, level)
      box.error(box.error.SOME_ERROR, <error arguments>, level)
      ```
      
      Follows up #9792
      Part of #9914
      
      NO_DOC=minor
      09ccd29a
    • Yaroslav Lobankov's avatar
      ci: temporary disable aarch64 packaging job · ef3152cd
      Yaroslav Lobankov authored
      We are experiencing some issues with aarch64 runners at this moment, so
      we have to disable aarch64 packaging job until the issue is resolved.
      Other aarch64 jobs can be disabled via GitHub UI, so let's do it there.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      ef3152cd
  6. May 23, 2024
    • Nikolay Shirokovskiy's avatar
      core: build fix for recent gcc · fb6b6c60
      Nikolay Shirokovskiy authored
      ```
      /home/shiny/dev/tarantool/src/lib/core/coio_task.c:114:58:
      	error: ‘calloc’ sizes specified with ‘sizeof’ in the earlier argument
      	and not in the later argument [-Werror=calloc-transposed-args]
        114 |         struct cord *cord = (struct cord *)calloc(sizeof(struct cord), 1);
      ```
      
      NO_TEST=build fix
      NO_CHANGELOG=build fix
      NO_DOC=build fix
      fb6b6c60
  7. May 22, 2024
    • Andrey Saranchin's avatar
      space_upgrade: respect min_field_count of both old and new formats · c449ada4
      Andrey Saranchin authored
      When upgrading a space, attribute `has_optional_parts` of indexes can be
      changed. So in order to correctly index both old and new tuples we should
      set new min_field_count value to the minimal min_field_count of old and
      new formats. Actual value will be set when space upgrade completes.
      
      Part of tarantool/tarantool-ee#698
      Part of tarantool/tarantool-ee#750
      
      NO_TEST=in ee
      NO_CHANGELOG=in ee
      NO_DOC=bugfix
      c449ada4
    • Aleksandr Lyapunov's avatar
      box: introduce next and previous prefix iterators · 96df090f
      Aleksandr Lyapunov authored
      Implement 'np' (next prefix) and 'pp' (previous prefix) iterators.
      They work only in memtx tree and in a nutshell searches for
      strings with greater ('np') or less ('pp') prefix of size as in
      given key, comparing with given key.
      
      Closes #9994
      
      @TarantoolBot document
      Title: 'np' and 'pp' (next/previous prefix) iterators
      
      Now there are two more iterators available: 'np' (next prefix)
      and 'pp' (previous prefix). They work only in memtx tree. Also,
      if the last part of key is not a string, they degrade to 'gt'
      and 'lt' iterators.
      
      These iterators introduce special comparison of the last part of
      key (if it is a string). In terms of lua, if s is the search part,
      and t is the corresponding tuple part, 'np' iterator searches for
      the first tuple with string.sub(t, 1, #s) > s, while 'pp' searches
      for the last tuple with string.sub(t, 1, #s) < s.
      
      Comparison of all other parts of the key remains normal.
      
      As usual, these iterators are available both in select and pairs,
      in index and space methods.
      
      Similar to all other tree iterators, they change only initial
      search of selection. Once the first tuple found, the rest are
      selected sequentially in direct (for 'np') or reverse (for 'pp')
      order of the index.
      
      For example:
      ```
      tarantool> s:select{}
      ---
      - - ['a']
        - ['aa']
        - ['ab']
        - ['b']
        - ['ba']
        - ['bb']
        - ['c']
        - ['ca']
        - ['cb']
      ...
      
      tarantool> s:select({'b'}, {iterator = 'np'})
      ---
      - - ['c']
        - ['ca']
        - ['cb']
      ...
      
      tarantool> s:select({'b'}, {iterator = 'pp'})
      ---
      - - ['ab']
        - ['aa']
        - ['a']
      ...
      ```
      96df090f
  8. May 21, 2024
    • Serge Petrenko's avatar
      wal: fix wal_queue_max_size assignment during initial box.cfg · ab0f7913
      Serge Petrenko authored
      wal_queue_max_size took effect only after the initial box.cfg call,
      meaning that users with non-zero `replication_sync_timeout` still synced
      using the default 16 Mb queue size. In some cases the default was too
      big and the same issues described in #5536 arose.
      
      Fix this.
      
      Closes #10013
      
      NO_DOC=bugfix
      ab0f7913
  9. May 20, 2024
    • Vladimir Davydov's avatar
      vinyl: fix index name in duplicate key error message · 2cfba5eb
      Vladimir Davydov authored
      The code setting ER_TUPLE_FOUND uses index_name_by_id() to find
      the index name, but it passes an index in the dense index map to
      it while the function expects an index in the sparse index map.
      Apparently, this doesn't work as expected after an index is removed
      from the middle of the index map. This bug was introduced by
      commit fc3834c0 ("vinyl: check key uniqueness before modifying
      tx write set").
      
      Instead of just fixing the index passed to index_name_by_id(), we do
      a bit of refactoring. We stop passing index_name and space_name to
      vy_check_is_unique_*() functions and instead get them right before
      raising ER_TUPLE_FOUND. Note, to get the space name, we need to call
      space_by_id() but it should be fine because (a) the space is very likely
      to be cached as the last accessed one and (b) this is an error path so
      it isn't performance critical. We also drop index_name_by_id() and
      extract the index name from the LSM tree object.
      
      Closes #5975
      
      NO_DOC=bug fix
      2cfba5eb
    • Vladimir Davydov's avatar
      vinyl: fix index build crash on invalid UPSERT · 5ac0d26a
      Vladimir Davydov authored
      Like UPDATE, UPSERT must not modify primary key parts. Unlike UPDATE,
      such an invalid UPSERT statement doesn't fail (raise an error) - we
      just log the error and ignore the statement. The problem is, we don't
      clear txn_stmt. As a result, if we're currently building a new index,
      the on_replace trigger installed by the build procedure will try to
      process this statement, triggering the assertion in the transaction
      manager that doesn't expect any statements in a secondary index without
      the corresponding statement in the primary index:
      
        ./src/box/vy_tx.c:728: vy_tx_prepare:
          Assertion `lsm->space_id == current_space_id' failed.
      
      Let's fix this by clearing the txn_stmt corresponding to a skipped
      UPSERT.
      
      Note, this also means that on_replace triggers installed by the user
      won't run on invalid UPSERT (hence test/vinyl/on_replace.result update),
      but this is consistent with the memtx engine, which doesn't run them
      in this case, either.
      
      Closes #10026
      
      NO_DOC=bug fix
      5ac0d26a
  10. May 17, 2024
    • Vladislav Shpilevoy's avatar
      relay: update lag on any acked txn · 39af9fbe
      Vladislav Shpilevoy authored
      Not only for own txns, but also on the txns authored by other
      instances.
      
      Note that the lag isn't updated when the replica got new txns from
      another master. The lag still only reflects the replication
      between this relay and its specific applier.
      
      The motivation is that otherwise the lag sometimes shows
      irrelevant things, like that the replica is very outdated, while
      it keeps replicating just fine. Only not txns of this specific
      master, who might even turned into a replica itself already.
      
      Closes #9748
      
      NO_DOC=bugfix
      39af9fbe
    • Vladislav Shpilevoy's avatar
      relay: enforce prev and new ack vclocks relation · 71dbb47c
      Vladislav Shpilevoy authored
      From the code it isn't obvious, but relay->status_msg.vclock and
      relay->last_recv_ack.vclock are both coming from the applier.
      Status_msg is the previous ack, last_recv_ack is the latest ack.
      
      They can never go down. And are not affected anyhow by the master
      committing its own transactions. I.e. master can commit something,
      relay->r->vclock (recovery cursor) will go up, and recovery vclock
      might become incomparable with the last ACK vclock. But the prev
      and last ACK vclocks are always comparable and always go up.
      
      This invariant was broken though, because relay on restart didn't
      nullify the current applier status (status_msg). It could break
      if the replica would loose its xlog files or its ID would be
      taken by another instance - then its vclock would go down, making
      last_recv_ack.vclock < status_msg.vclock. But that is not right
      and is fixed in this patch.
      
      In scope of #9748
      
      NO_DOC=bugfix
      NO_TEST=test 5158 already covers it
      NO_CHANGELOG=bugfix
      71dbb47c
    • Vladislav Shpilevoy's avatar
      relay: move ack handling into new function · d6f15a10
      Vladislav Shpilevoy authored
      To reduce the insane indentation level. And to isolate the further
      changes in next commits more.
      
      Part of #9748
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      d6f15a10
    • Vladislav Shpilevoy's avatar
      applier: drop lag to zero on reconnect · dda42035
      Vladislav Shpilevoy authored
      Before the patch if the applier was reconnected, the master would
      see downstream lag equal to the time since it replicated the last
      txn to this applier.
      
      This happened because applier between reconnects kept the txn
      timestamp used for acks. On the master's side the relay was
      recreated, received the ack, thought the applier just applied this
      txn, and displayed this as a lag.
      
      The test makes a master restart because this is the easiest way to
      reproduce it. Most importantly, the applier shouldn't be
      re-created, and relay should restart.
      
      Part of #9748
      
      NO_DOC=bugfix
      NO_CHANGELOG=later
      dda42035
    • Vladislav Shpilevoy's avatar
      applier: move applier_txn_last_tm into applier · 8e5d9f2a
      Vladislav Shpilevoy authored
      It was stored in struct replica, now is in struct applier. The
      motivation is that applier-specific data must be inside the
      applier.
      
      Also it makes the next commits look more logical. They are going
      to change this timestamp when applier progresses through its state
      machine. It looks strange when the applier is changing the replica
      object. Replica is on an upper level in the hierarchy. It owns the
      applier and the applier ideally mustn't know about struct replica
      (hardly possible to achieve), or at least not change it (this is
      feasible).
      
      In scope of #9748
      
      NO_DOC=internal
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      8e5d9f2a
    • Vladimir Davydov's avatar
      vinyl: fix bug when tuple not committed to unique nullable index · 2e689063
      Vladimir Davydov authored
      A unique nullable key definition extended with primary key parts
      (cmp_def) assumes that two tuples are equal *without* comparing
      primary key fields if all secondary key fields are equal and not
      nulls, see tuple_compare_slowpath(). This is a hack required to
      ignore the uniqueness constraint for nulls in memtx. The memtx
      engine can't use the secondary key definition as is (key_def) for
      comparing tuples in the index tree, as it does for a non-nullable
      unique index, because this wouldn't allow insertion of any
      duplicates, including nulls. It couldn't use cmp_def without this
      hack, either, because then conflicting tuples with the same
      secondary key fields would always compare as not equal due to
      different primary key parts.
      
      For Vinyl, this hack isn't required because it explicitly skips
      the uniqueness check if any of the indexed fields are nulls, see
      vy_check_is_unique_secondary(). Furthermore, this hack is harmful
      because Vinyl relies on the fact that two tuples compare as equal by
      cmp_def if and only if *all* key fields (both secondary and primary)
      are equal. For example, this is used in the transaction manager,
      which overwrites statements equal by cmp_def, see vy_tx_set_entry().
      
      Let's disable this hack by resetting unique_part_count in cmp_def.
      
      Closes #9769
      
      NO_DOC=bug fix
      2e689063
    • Alexander Turenko's avatar
      console: autorequire frequently used modules · 8c9965a1
      Alexander Turenko authored
      Fixes #9986
      
      @TarantoolBot document
      Title: Interactive console now autorequires a couple of built-in modules
      
      There are built-in modules that are frequently used for administration
      or debugging purposes. It is convenient to have them accessible in the
      interactive console without extra actions.
      
      They're accessible now without a manual `require` call if the
      `console_session_scope_vars` compat option is set to `new` (see also
      tarantool/doc#4191).
      
      The list of the autorequired modules is below.
      
      * clock
      * compat
      * config
      * datetime
      * decimal
      * ffi
      * fiber
      * fio
      * fun
      * json
      * log
      * msgpack
      * popen
      * uuid
      * varbinary
      * yaml
      
      See tarantool/tarantool#9986 for motivation behind this feature.
      
      This list forms so called initial environment for an interactive console
      session. The default initial environment may be adjusted by an
      application, for example, to include application specific administrator
      functions.
      
      Two public functions are added for this purpose: `console.initial_env()`
      and `console.set_initial_env(env)`.
      
      Example 1 (keep autorequired modules, but add one more variable):
      
      ```lua
      local console = require('console')
      
      -- Add myapp_info function.
      local initial_env = console.initial_env()
      initial_env.myapp_info = function()
          <...>
      end
      ```
      
      Example 2 (replace the whole initial environment):
      
      ```lua
      local console = require('console')
      
      -- Add myapp_info function, discard the autorequired modules.
      console.set_initial_env({
          myapp_info = function()
              <...>
          end,
      })
      ```
      
      The `console.set_initial_env()` call without an argument or with a `nil`
      argument drops the initial environment to its default.
      
      A modification of the initial environment doesn't affect existing
      console sessions. It affects console sessions that are created
      after the modification.
      
      Please, adjust the `console_session_scope_vars` compat option
      description and extend the built-in `console` module reference with the
      new functions.
      8c9965a1
    • Alexander Turenko's avatar
      console: define module table earlier · b107b763
      Alexander Turenko authored
      I need to capture the module table inside a module function in a next
      commit.
      
      NO_DOC=refactoring, no behavior changes
      NO_CHANGELOG=see NO_DOC
      NO_TEST=see NO_DOC
      b107b763
    • Ilya Verbin's avatar
      perf: speed up column scan test initialization · ef30ca9b
      Ilya Verbin authored
      It was intended to do 1000 inserts per transaction, but by mistake
      box.commit() was called after each insertion.
      
      NO_DOC=perf test
      NO_TEST=perf test
      NO_CHANGELOG=perf test
      ef30ca9b
  11. May 16, 2024
    • Vladimir Davydov's avatar
      vinyl: fix use-after-free of LSM tree in scheduler · 1c4605bb
      Vladimir Davydov authored
      Between picking an LSM tree from a heap and taking a reference to it in
      vy_task_new() there are a few places where the scheduler may yield:
       - in vy_worker_pool_get() to start a worker pool;
       - in vy_task_dump_new() to wait for a memory tree to be unpinned;
       - in vy_task_compaction_new() to commit an entry to the metadata log
         after splitting or coalescing a range.
      
      If a concurrent fiber drops and deletes the LSM tree in the meanwhile,
      the scheduler will crash. To avoid that, let's take a reference to
      the LSM tree.
      
      It's quite difficult to write a functional test for it without a bunch
      of ugly error injections so we rely on fuzzing tests.
      
      Closes #9995
      
      NO_DOC=bug fix
      NO_TEST=fuzzing
      1c4605bb
  12. May 14, 2024
    • Alexander Turenko's avatar
      console: create per-session variables scope · d36493c0
      Alexander Turenko authored
      Fixes #9985
      
      @TarantoolBot document
      Title: Interactive console now has its own per-session variables scope
      
      It is counter-intuitive that all the non-local assignments in the
      console affect globals and may interfere with application's logic.
      
      It is also counter-intuitive that the non-local assignments are shared
      between different console sessions.
      
      Now, each console session has its own variables scope and the non-local
      assignments use it instead of globals.
      
      Let's consider examples of the new behavior.
      
      Example 1. A console session has a variable scope that is separate from
      globals.
      
      ```lua
      console_1> _G.x = 1
      console_1> x = 2
      
      console_1> _G.x
      ---
      - 1
      ...
      console_1> x
      ---
      - 2
      ...
      ```
      
      Note: A global variable is still accessible using `_G` even if the same
      named session scope variable exists.
      
      Example 2. A global variable is read if there is no session local
      variable.
      
      ```lua
      console_1> _G.x = 1
      console_1> x
      ---
      - 1
      ...
      ```
      
      Example 3. Different console sessions have separate variable scopes.
      
      ```lua
      console_1> x = 1
      console_2> x = 2
      
      console_1> x
      ---
      - 1
      ...
      console_2> x
      ---
      - 2
      ...
      ```
      
      The new behavior is enabled using the `console_session_scope_vars`
      compat option. The option is `old` by default in Tarantool 3.X, `new` by
      default in 4.X. The `old` behavior is to be removed in 5.X.
      
      Please, create the following page:
      https://tarantool.io/compat/console_session_scope_vars
      
      Please, add the new compat option into the configuration reference.
      d36493c0
    • Alexander Turenko's avatar
      test/interactive: add connect() function · bb430c55
      Alexander Turenko authored
      It encapsulates all the needed actions to connect to a remote console
      using a Unix socket.
      
      Part of #9985
      
      NO_DOC=testing helper change
      NO_CHANGELOG=see NO_DOC
      bb430c55
    • Alexander Turenko's avatar
      test/interactive: disable hide/show prompt feature · 23094b6f
      Alexander Turenko authored
      See #7169 for details about the hide/show prompt feature. In short, it
      hides readline's prompt before `print()` or `log.<level>()` calls and
      restores the prompt afterwards.
      
      This feature sometimes badly interferes with
      `test.interactive_tarantool` heuristics about readline's command
      echoing.
      
      This commit disables the feature in `test.interactive_tarantool` by
      default and enables it explicitly where needed.
      
      Part of #9985
      
      NO_DOC=testing helper change
      NO_CHANGELOG=see NO_DOC
      23094b6f
    • Alexander Turenko's avatar
      test/interactive: allow to assert for nil response · 7d9e8569
      Alexander Turenko authored
      Before this patch the `:roundtrip()` method in the
      `test.interative_tarantool` instance considered the following calls as
      equivalent:
      
      ```lua
      g.it = it.new()
      
      -- Doesn't check the response.
      g.it:roundtrip('x')
      
      -- Before the patch it was the same as above.
      --
      -- Now it checks that the response is nil.
      local expected = nil
      g.it:roundtrip('x', expected)
      
      -- It is the same as previous.
      g.it:roundtrip('x', nil)
      ```
      
      Now the response is checked against the provided expected value if the
      value is passed to arguments, even if it is `nil`.
      
      Also, a command's response is now returned from the method. It may be
      useful if the response returns some dynamic information (such as a TCP
      port number or a file descriptor) that is used later in the test or if
      the response should be verified in some non-trivial way, not just a deep
      compare.
      
      The `:roundtrip()` method is just `:execute_command()` plus
      `:read_response()` plus `luatest.assert_equals()`. However, I like using
      `:roundtrip()` even when the assertion is not needed, because it is
      shorter and because using the same method brings less context to a
      reader.
      
      For example,
      
      ```lua
      g.it = it.new()
      g.it:roundtrip('x = 2')
      g.it:roundtrip('y = 3')
      g.it:roundtrip('x + y', 6)
      ```
      
      Part of #9985
      
      NO_DOC=testing helper change
      NO_CHANGELOG=see NO_DOC
      NO_TEST=see NO_DOC
      7d9e8569
  13. May 07, 2024
    • Georgiy Lebedev's avatar
      box: pass source tuple to `default_func` field option of `space:format` · b32b20c6
      Georgiy Lebedev authored
      Let's pass the source tuple received as the argument to DML to the
      `default_func` field option of `space:format` to give users more
      versatility and the opportunity to compute the field value using other
      fields from the source tuple.
      
      For the tuple argument, we create a tuple rather than pass a MsgPack
      object for consistency with our other box APIs, even though it is
      suboptimal in terms of performance.
      
      We create the tuple argument with the empty default format, however, in the
      future it is possible to create it with a `names_only=true` format so that
      the source tuple can have the space format's data dictionary.
      
      We create the tuple argument from the source tuple data, which implies the
      following: (i) fields may not adhere to the space format; (ii) nil fields
      are always nil (i.e., the `default` value and the `default_func` are
      not used). (i) is because we can only validate the tuple after we finish
      building it. (ii) is because trying to use the `default` value and the
      `default_func` would have field build ordering ambiguity and would hurt
      performance of field building.
      
      Closes #9825
      
      @TarantoolBot document
      Title: Tuple argument of `default_func` field option of `space:format`
      Product: Tarantool
      Since: 3.2
      Root documents: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_space/format/
      
      The source tuple (i.e., the argument of DML) is now passed as a second
      argument to the `default_func` field option of `space:format`.
      
      See also tarantool/tarantool#9825 and [PRD](https://www.notion.so/tarantool/Pass-tuple-as-argument-to-field-s-default_func-8785637fb79f43e4b8ca729e75fc4582).
      
      Please note that the tuple argument is created from the source tuple data,
      which implies the following: (i) fields may not adhere to the space format;
      (ii) nil fields are always nil (i.e., the `default` value and the
      `default_func` are not used).
      b32b20c6
Loading