Skip to content
Snippets Groups Projects
  1. Jul 05, 2023
    • Magomed Kostoev's avatar
      box: compare and hash msgpack value of double key field as double · 51af059c
      Magomed Kostoev authored
      1. Make double-formatted fields accept integer and float values.
      2. Make indexes compare the values as double if the field key type
         is FIELD_TYPE_DOUBLE.
      3. Make hashers cast double key field to double before hashing, so
         we are able to insert and select any int, uint, float or double
         if their value casted to double is equal (for double keys).
      
      Notes about tuple_compare.cc:
      
      Since now `mp_compare_double` casts any value placed in field to
      double it was renamed to `mp_compare_as_double` to not semantically
      conflict with existing `mp_compare_double_*` functions.
      
      Notes about tuple_hash.cc:
      
      The hashee cast result is encoded in MP_DOUBLE and hashed for
      backward compatibility reasons.
      
      Since now the field hashing function (tuple_hash_field) requires
      field type to hash the field correctly, a new parameter has been
      introduced.
      
      By the way added assertions to the generic `field_hash` to prevent
      invalid hashing for new precompiled hashers and made
      `key_hash_slowpath` static cause it's only used in this file.
      
      Closes #7483
      Closes #5933
      Unblocks tarantool/crud#298
      
      @TarantoolBot document
      Title: It's not required to ffi-cast integral floating point to
      double anymore.
      
      The page describing tarantool data model states that:
      
      > In Lua, fields of the double type can only contain non-integer
      > numeric values...
      
      If the patch is merged this isn't the case anymore, so this
      statement and the code snippet below it should be updated.
      
      Link to the document: [Data storage](https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#field-type-details).
      
      Affected segments:
      
      > double. The double field type exists mainly to be equivalent
      > to Tarantool/SQL’s DOUBLE data type. In msgpuck.h (Tarantool’s
      > interface to MsgPack), the storage type is MP_DOUBLE and the
      > size of the encoded value is always 9 bytes. In Lua, fields of
      > the double type can only contain non-integer numeric values and
      > cdata values with double floating-point numbers. Examples: 1.234,
      > -44, 1.447e+44.
      >
      > To avoid using the wrong kind of values inadvertently, use
      > ffi.cast() when searching or changing double fields. For example,
      > instead of space_object:insert{value} use ffi = require('ffi')
      > ... space_object:insert({ffi.cast('double',value)}). Example:
      >
      > ```
      > s = box.schema.space.create('s', {format = {{'d', 'double'}}})
      > s:create_index('ii')
      > s:insert({1.1})
      > ffi = require('ffi')
      > s:insert({ffi.cast('double', 1)})
      > s:insert({ffi.cast('double', tonumber('123'))})
      > s:select(1.1)
      > s:select({ffi.cast('double', 1)})
      > ```
      51af059c
    • Magomed Kostoev's avatar
      Bump msgpuck submodule · be47f8fb
      Magomed Kostoev authored
      This update pulls the following commits:
      * Add mp_read_double_lossy without direct convertibility checks
      * Fix mp_read_double_lossy tests freebsd build
      
      These commits introduce a function required to compare and hash
      msgpack values of double key fields as double.
      
      Need for #7483, #5933
      
      NO_DOC=see the next commit
      NO_CHANGELOG=see the next commit
      be47f8fb
    • Kirill Yukhin's avatar
      Generate changelog for 3.0.0-alpha1 · ef6335e8
      Kirill Yukhin authored
      NO_DOC=no code changes
      NO_TEST=no code changes
      NO_CHANGELOG=no code changes
    • Alexander Turenko's avatar
      config: add changelog entry re config · 4fd61fff
      Alexander Turenko authored
      Follows up #8724.
      
      NO_DOC=just add a changelog entry
      NO_TEST=see NO_DOC
      4fd61fff
    • Yaroslav Lobankov's avatar
      ci/cd: drop distro-specific packaging workflows · 7e06e771
      Yaroslav Lobankov authored
      Tarantool 3.0.0 is not going to be distributed as a set of packages for
      specific distros. Instead, it is planned to have a few `deb` and `rpm`
      packages with a statically compiled Tarantool binary inside, which will
      work on almost any distro. Also, we drop building packages with GC32.
      Now only GC64 packages will be shipped.
      
      Static build packaging was added at #8771.
      
      NO_DOC=ci
      NO_TEST=ci
      7e06e771
  2. Jul 04, 2023
    • Igor Munkin's avatar
      luajit: bump new version · 852664b4
      Igor Munkin authored
      * test: fix flaky <unit-jit-parse.test.lua>
      * Fix use-def analysis for vararg functions.
      * Fix use-def analysis for BC_VARG.
      * Fix TNEW load forwarding with instable types.
      * Fix memory probing allocator to check for valid end address, too.
      * Another fix for lua_yield() from C hook.
      * Fix lua_yield() from C hook.
      * Fix saved bytecode encapsulated in ELF objects.
      * x64: Fix 64 bit shift code generation.
      * Fix canonicalization of +-0.0 keys for IR_NEWREF.
      * test: add utility for parsing `jit.dump`
      * test: split utils.lua into several modules
      * test: rewrite lj-49-bad-lightuserdata test in C
      * test: rewrite misclib-sysprof-capi test in C
      * test: rewrite misclib-getmetrics-capi test in C
      * test: introduce utils.h helper for C tests
      * test: introduce module for C tests
      * test: fix setting of {DY}LD_LIBRARY_PATH variables
      * build: fix build with LUAJIT_USE_GDBJIT enabled
      
      Closes #8718
      Part of #7900
      Part of #8516
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      852664b4
    • Alexander Turenko's avatar
      config: restrict iproto.advertise.client validation · ce9a6109
      Alexander Turenko authored
      The following syntax variants are fobidden now:
      
      * user@
      * user:pass@
      * user@host:port
      * user:pass@host:port
      
      Only host:port is allowed (inet URI or unix socket URI).
      
      The idea is that we shouldn't distribute login/password information
      using the configuration, so it is better to explicitly forbid such
      usage of the option.
      
      Closes #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      ce9a6109
    • Alexander Turenko's avatar
      config: relax replication.peers autobuild constraints · aa51fbec
      Alexander Turenko authored
      Before this commit all the peers of the given replicaset were obligated
      to have suitable URI to connect -- either iproto.advertise.peer or one
      of iproto.listen URIs.
      
      Now replication.peers is successfully constructed if at least one
      suitable URI is constructed (except URI of the given instance itself).
      
      However, if an unknown user is used in the iproto.advertise.peer option
      of any peer, it is considered as an error (startup failure), just like
      before.
      
      The testing code for unsuccessful cases of replication.peers
      autobuilding is rewritten to reflect that just one bad peer doesn't
      cause a startup failure anymore. Now it also needs stripping of log
      messages, because the peers autobuilding tells about unsuitable URIs.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      aa51fbec
    • Alexander Turenko's avatar
      config: add replicaset.failover = "election" · 6b3eb60c
      Alexander Turenko authored
      This failover mode enables automatic leader election on a replicaset.
      Assigning a leader manually (`leader` option) and assigning RO/RW mode
      (`database.mode` option) are forbidden in this failover mode.
      
      Configuration example:
      
      ```yaml
      replication:
        failover: election         # !!
      
      groups:
        group-001:
          replicasets:
            replicaset-001:
              instances:
                instance-001: {}
                instance-002: {}
                instance-003: {}
      ```
      
      All the replicaset instances are so called candidates by default: they
      can vote for a leader, they can be elected as a leader. However, it is
      possible to set `replicaset.election_mode` to 'voter', 'off' or
      'manual'. See more detailed description in the box_cfg applier comments
      in the code and in the box.cfg() options documentation [1].
      
      Note: Unlike box.cfg()'s option 'election_mode', the
      replication.election_mode = "off" forces the instance to be read-only
      (if failover = "election" is enabled). box.cfg() call has no cluster
      configuration and it doesn't know, whether the given replicaset is
      managed by the built-in election algorithm or an external coordinator.
      
      The election failover may be used to improve cluster's availability.
      
      [1]: https://www.tarantool.io/en/doc/latest/reference/configuration/#cfg-replication-election-mode
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      6b3eb60c
    • Alexander Turenko's avatar
      config: add replicaset.failover = "manual" · 4fab39bb
      Alexander Turenko authored
      This failover mode allows to set a leader for a replicaset by its
      instance name instead of per-instance database.mode option. For example:
      
      ```yaml
      replication:
        failover: manual           # !!
      
      groups:
        group-001:
          replicasets:
            replicaset-001:
              leader: instance-001 # !!
              instances:
                instance-001: {}
                instance-002: {}
                instance-003: {}
      ```
      
      The "manual" failover mode doesn't allow several leaders in a
      replicaset. If it is desired, use the "off" failover mode.
      
      The "manual" mode doesn't perform a proper leader switching at the
      moment. An administrator should resign the old leader (by unsetting the
      leader option or setting it to `null`), wait till all the data arrives
      to the upcoming leader and then configure it as the new leader.
      
      The proper leader switching is subject of a future work. Also, the next
      commit will add replicaset.failover = "election", which handles such
      problems.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      4fab39bb
    • Alexander Turenko's avatar
      config: add replication.failover option · cf55d71b
      Alexander Turenko authored
      It has the only one value for now: 'off'. Two other ones are planned:
      'manual' and 'election'.
      
      The 'off' value means exactly what is implemented now: the database.mode
      option controls whether a particular instance should serve write
      requests.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      cf55d71b
    • Alexander Turenko's avatar
      config: forbid attempt to join read-only replicaset · 18dad9d3
      Alexander Turenko authored
      If the given instance has no existing snapshot, it means that it is not
      registered in the replicaset and will attempt to join as a new replica.
      It has no chance to succeed if all the instances in the replicaset are
      in the read-only mode: neither of them can register the new instance.
      
      Report an error and fail the startup in the case to make the situation
      clear.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      18dad9d3
    • Alexander Turenko's avatar
      config: add safe startup mode · c80b1215
      Alexander Turenko authored
      If the startup may take a long time (reads of an existing snapshot or
      fetches it from a remote master), then
      
      * If there are other instances in the replicaset, force the read-only
        mode on the database startup[^1].
      * After the database startup, re-read the configuration and apply the
        fresh one (now without any RO/RW mangling).
      
      If the startup shouldn't take a long time[^2], the configuration is
      read and applied once.
      
      [^1]: The read-only mode is not forced if there is no existing snapshot.
            It may lead to missing bootstrap leader in the replicaset. See
            details in the comment in the box_cfg applier code.
      [^2]: There is the only case that we consider as such: startup of an
            instance without an existing snapshot without other instances in
            the same replicaset.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      NO_TEST=it is not trivial to verify automatically
      c80b1215
    • Alexander Turenko's avatar
      config: start singleton instance in RW by default · 8ee2b0d8
      Alexander Turenko authored
      Enabling read-write by default is unsafe for an instance in a replicaset
      with more than one instance. The only meaningful default here is
      read-only.
      
      On the other hand, a signleton instance (the only one in its replicaset)
      almost always started in the read-write mode in practice.
      
      Let's use these values as defaults for these situations.
      
      The name of the option is changed from `rw` (boolean) to `mode` (enum
      with allowed values `'ro'` and `'rw'`). We agreed on the enum after a
      long controversy whether it should be `ro` or `rw`.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      8ee2b0d8
    • Alexander Turenko's avatar
      config: split iproto.advertise to several options · 345d52b8
      Alexander Turenko authored
      In brief:
      
      * client -- for external clients
      * peer -- for connections within the cluster, in particular for replicas
      * sharding -- for routers and a rebalancer
      
      See the instance_config.lua file for the details.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      345d52b8
    • Alexander Turenko's avatar
      config: accept user[:pass]@ in iproto.advertise · b3beba61
      Alexander Turenko authored
      The following new syntax variants are introduced for `iproto.advertise`.
      
      * `user@` -- use the given user, a password from the `credentials`
        section and `host:port` from `iproto.listen`
      * `user:pass@` -- use the given user and password, use `host:port` from
        `iproto.listen`
      * `user@host:port` -- use the given user, host and port, use a password
        from the `credentials` section
      
      It allows to don't repeat the same information in different places of
      the config.
      
      The `test_no_advertise_*` test cases are generalized: now the same
      boilerplate code is used for these and new cases.
      
      The `test.luatest_helpers.server` util gains ability to parse all the
      new `iproto.advertise` variants.
      
      The replicaset and etcd configuration examples are updated to use the
      `user@` syntax.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      b3beba61
    • Alexander Turenko's avatar
      config: filter iproto.listen at replicaset building · 55c9a82a
      Alexander Turenko authored
      There are a few nuances in using the `iproto.listen` option as a peer
      URI at replicaset building (constructing a list of peer URIs).
      
      * `iproto.listen` is a comma separated list of URIs: the single URI is
        just a special case.
      * `iproto.listen` may contain `0.0.0.0` or `::` host and/or zero port.
        It is perfectly valid for listening, but doesn't give enough
        information to connect.
      
      Let's handle `iproto.listen` appropriately: split it the URI list, walk
      over the URIs and check whether the URI is suitable to connect. The
      first suitable URI is used as the peer URI.
      
      If no suitable URI is found, raise an error (it means a startup failure
      if occurs at a startup).
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      55c9a82a
    • Alexander Turenko's avatar
      config: forbid incorrect iproto.listen values · 4b90516c
      Alexander Turenko authored
      Aside of a fast feedback for a user that feeds an incorrect
      configuration, this validation simplifies further box_cfg applier code:
      we can assume that the URI list is valid.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      4b90516c
    • Alexander Turenko's avatar
      config: forbid incorrect/senseless advertise URIs · 64902055
      Alexander Turenko authored
      The following values shouldn't be accepted as an advertise URI:
      
      * Not an URI.
      * Comma separated list of URIs.
      * An URI with IPv4/IPv6 INADDR_ANY host (`0.0.0.0` or `::`).
      * An URI with zero TCP port.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      64902055
    • Alexander Turenko's avatar
      config: raise error if replicaset can't be built · cd90b91c
      Alexander Turenko authored
      We can't just pass it over and start the instance:
      
      1. If it is an initial bootstrap (there is no data for the given
         instance yet), the instance will form its own replicaset instead of
         joining into the existing one.
      2. If it is a startup of an existing instance, the local data may be
         outdated and serving requests is undesirable.
      
      Part of #8810
      
      NO_DOC=the old behavior was not released, the documentation request will
             be registered manually
      NO_CHANGELOG=see NO_DOC
      cd90b91c
    • Georgiy Lebedev's avatar
      box: use format data instead of field def array in space read view · d8abfe34
      Georgiy Lebedev authored
      In scope of #4693, we now create runtime tuple formats from format data, so
      we need to adapt space read views accordingly: they use a field definition
      array for creating tuple formats and for accessing field names and types —
      for the latter case we will also allocate a field definition array
      separately.
      
      Follows-up #4693
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      d8abfe34
    • Sergey Bronnikov's avatar
      test: fix flakiness in gh_6128_background_mode_test · 1c8e7124
      Sergey Bronnikov authored
      Previous attempt to fix flakiness in commit 6a2c73f8 ("test: fix
      flakiness in gh_6128_background_mode_test") used a constant buffer size
      in check_err_msg function. Tarantool 2.10 has a bit bigger log before a
      desired message that other versions of Tarantool and it leads to a this
      resulted in a truncated message ("entering the even" instead of
      "entering the event loop"). Patch replaces check_err_msg()
      implementation to grep_log used in luatest, it reads the whole log.
      
      Also patch renames check_err_msg to check_msg, because "entering the
      event loop" is not an error message.
      
      Follows up #6128
      
      NO_CHANGELOG=fix test
      NO_DOC=fix test
      1c8e7124
  3. Jul 03, 2023
    • Mergen Imeev's avatar
      config: rework the way the meta is populated · a98ff3f8
      Mergen Imeev authored
      This patch reworks the way the meta is populated. This is done to
      incrementally populate the metadata, instead of set the metadata at the
      end of reading data from the source. This allows to get the correct meta
      in cases where getting data from the source failed.
      
      Follow-up #8789
      
      NO_DOC=feature not yet released
      NO_TEST=tested in EE
      NO_CHANGELOG=feature not yet released
      a98ff3f8
    • Vladimir Davydov's avatar
      yaml: fix decoding single-byte char codes >= 0x80 · fa6b08a8
      Vladimir Davydov authored
      The bug was fixed in the libyaml repository. This commit just updates
      the submodule and adds a test.
      
      Closes #8782
      
      NO_DOC=bug fix
      fa6b08a8
    • Magomed Kostoev's avatar
      box: check permissions on constraint functions on creation · 6b8f2c5f
      Magomed Kostoev authored
      Function execution permissions should only be checked on constraint
      creation.
      
      So when the function is used to check a tuple access rights don't
      have to be checked on each call for the current user.
      
      Closes #7873
      
      NO_DOC=bugfix
      6b8f2c5f
    • Magomed Kostoev's avatar
      box: check permissions on functional index functions on creation · ddbdb77a
      Magomed Kostoev authored
      Function execution permissions should only be checked on functional
      index creation and on functional index function set.
      
      So when the function is used by key_list_iterator its rights don't
      have to be checked on each call for the current user.
      
      Part of #7873
      
      NO_DOC=bugfix
      NO_CHANGELOG=see the next commit
      ddbdb77a
    • Georgiy Lebedev's avatar
      memtx: fix heap-use-after-free of tuple stories caused by space alter · e1ed31bb
      Georgiy Lebedev authored
      When a space is altered, we abort all in-progress transactions and delete
      all stories related to that space: the problem is we don't delete the
      stories' read gaps, which are also linked to the stories' transactions,
      which get cleaned up on transaction destruction — this, in turn, results in
      heap-use-after-free. To fix this, clean up stories' read gap in
      `memtx_on_space_delete` — we don't do this in `memtx_tx_story_delete` since
      it expects the story to not have any read gaps (see
      `memtx_tx_story_gc_step`).
      
      Tested this patch manually against Nick Shirokovskiy's experimental
      small-ASAN integration branch.
      
      Closes #8781
      
      NO_DOC=bugfix
      NO_TEST=<already covered by existing tests, but was not detectable by ASAN>
      e1ed31bb
    • Sergey Vorontsov's avatar
      ci: add workflow for static build packaging · 9041d7ed
      Sergey Vorontsov authored
      
      The created packages can be found in the job artifacts.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      
      Co-authored-by: default avatarYaroslav Lobankov <y.lobankov@tarantool.org>
      9041d7ed
    • Yaroslav Lobankov's avatar
      make: add new target `package-static` to .pack.mk · ed406003
      Yaroslav Lobankov authored
      This target is a wrapper on the command to build DEB and RPM packages
      with a statically compiled Tarantool binary inside. It just runs the
      `./static-build/make_packages.sh` script with the properly defined
      VERSION env variable.
      
      Example of usage:
      
          $ make -f .pack.mk package-static
      
      NO_DOC=make
      NO_TEST=make
      NO_CHANGELOG=make
      ed406003
    • Sergey Vorontsov's avatar
      build: make packages with static binary inside · 390cddbb
      Sergey Vorontsov authored
      
      This patch adds facility to make DEB and RPM packages with a statically
      compiled Tarantool binary inside. The build is performed in a Docker
      container, using PackPack docker image (centos-7) and CPack.
      
      The packpack/packpack:centos-7 image has all the necessary dependencies
      for building Tarantool and quite old glibc 2.17 which theoretically
      provides compatibility of the created packages with any distro where
      glibc >= 2.17.
      
      The build can be run with the command below:
      
          $ VERSION=3.0.0 ./static-build/make_packages.sh
      
      In the `static_build` directory, there will be the following packages:
      
          tarantool_3.0.0-1_amd64.deb
          tarantool-dev_3.0.0-1_amd64.deb
          tarantool-3.0.0-1.x86_64.rpm
          tarantool-devel-3.0.0-1.x86_64.rpm
      
      `tarantool_3.0.0-1_amd64.deb`, `tarantool-3.0.0-1.x86_64.rpm` are
      packages with the Tarantool server binary inside.
      
      `tarantool-dev_3.0.0-1_amd64.deb`, `tarantool-devel-3.0.0-1.x86_64.rpm`
      are packages with the Tarantool server development files inside.
      
      NO_DOC=build
      NO_TEST=build
      
      Co-authored-by: default avatarYaroslav Lobankov <y.lobankov@tarantool.org>
      390cddbb
    • Georgiy Lebedev's avatar
      box: introduce formats for standalone tuples · dc26e47e
      Georgiy Lebedev authored
      Introduce `box.tuple.format` object, a Lua wrapper around tuple format:
      these objects own a tuple format, which is almost equivalent to
      `space:format`, except for check constraints and foreign key constraints
      being disabled (though they appear to be present for compatibility with
      `space:format`).
      
      Add an option table argument to `box.tuple.new` with 'format' option,
      allowing to create formatted spaceless tuples.
      
      Closes #4693
      
      @TarantoolBot document
      Title: Formats for standalone tuples and `box_tuple_new_vararg` compat opt
      
      A new box.tuple.format library was added, with a tuple format constructor
      (`new`) and a tuple format validator (`is`).
      
      New tuple format objects (userdata) were added, which can be used with the
      same format clause as for the `space:format` method (except that check
      constraints and foreign keys are disabled for them):
      NO_WRAP
      ```lua
      f = box.tuple.format.new(box.space._space:format())
      f = box.tuple.format.new{{name = 'field1', type = 'string', is_nullable = true,
                                nullable_action = 'none', collation = 'unicode_uk_s2',
                                default = 'UPPER("string")',
                                constraint = {ck = 'box.schema.user.info'},
                                foreign_key = {fk = {space = '_space', field = 'name'}}},
                                {name = 'field2', nullable_action = 'ignore',
                                foreign_key = {fk = {space = '_space', field = 1}}}}
      ```
      NO_WRAP
      
      Format objects have several introspection methods: `:pairs`, `:ipairs`,
      `totable`, and also have a `__serialize` metamethod — these methods return
      the original (i.e., user-provided) format clause. `:pairs` is an alias to
      `ipairs` (since the format clause is an array by nature), and the `totable`
      method is an alias to the `__serialize` metamethod, which returns an array
      of field definitions.
      
      Format objects also have a `:tostring` method, which simply returns a
      "box.tuple.format" literal.
      
      The standalone tuple constructor, `box.tuple.new` was extended with an
      options parameter which currently has one available option, `format`
      (default value is `nil`, i.e., no format). The format option is either a
      tuple format object previously created using `box.tuple.format.new` or a
      format clause.
      
      Examples of standalone tuple creation with formats:
      NO_WRAP
      ```lua
      box.tuple.new({1}, {format = {{name = 'field', type = 'number'}}})
      box.tuple.new({1}, {format = {{'field', type = 'number'}}})
      box.tuple.new({1}, {format = {{'field', 'number'}}})
      
      f = box.tuple.format.new({{name = 'field', type = 'number'}})
      box.tuple.new({}, {format = f})
      box.tuple.new({1}, {format = f})
      box.tuple.new({'str'}, {format = f})
      -- error: Tuple field 1 (field) type does not match one required by operation: expected number, got string
      box.tuple.new({'str'}, {format = f})
      ```
      NO_WRAP
      
      See also the design document https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#a33e2d7418d249679969e5f21ef2832c
      
      A new `box_tuple_new_vararg` compatibility option was introduced: a new
      page needs to be created for it (https://tarantool.io/compat/box_tuple_new_vararg)
      
      This option controls whether `box.tuple.new` should interpret an argument
      list as an array of tuple fields (i.e., vararg, old behaviour), or as a
      value plus a tuple format (new default behaviour). The value can be either
      a scalar, an array or a box tuple. The old behaviour does not allow
      creating formatted standalone tuples.
      
      Old behaviour examples:
      ```lua
      box.tuple.new(1)
      box.tuple.new{1}
      box.tuple.new(1, 2, 3)
      box.tuple.new{1, 2, 3}
      -- This won't create a formatted tuple: the format option will become the
      -- second tuple field.
      box.tuple.new({1, 2, 3}, {format = box.tuple.format.new{{'field'}}})
      ```
      
      New behaviour examples:
      ```lua
      box.tuple.new(1)
      box.tuple.new(1, {format = box.tuple.format.new{{'field'}}})
      box.tuple.new{1}
      box.tuple.new({1}, {format = box.tuple.format.new{{'field'}}})
      box.tuple.new(1, 2, 3) -- error
      box.tuple.new(1, 2, 3, {format = box.tuple.format.new{{'field'}}}) -- error
      box.tuple.new{1, 2, 3}
      box.tuple.new({1, 2, 3}, {format = box.tuple.format.new{{'field'}}})
      ```
      
      See also the design document https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#6f74f0c70005463b8438830edd1a0117.
      dc26e47e
    • Georgiy Lebedev's avatar
      box: account for all tuple format field definitions in `cmp` and `hash` · 05e8b1de
      Georgiy Lebedev authored
      Previously, reusable tuple formats were only used for ephemeral spaces and
      by `net.box` (which only used 'name' definitions), so non-space tuple
      format definitions were ignored in tuple format comparison and hash
      functions, but in scope of #4693 reusable tuple formats will be
      interchangeable with space formats, so now we need to account for these
      too.
      
      Needed for #4693
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      05e8b1de
    • Georgiy Lebedev's avatar
      box: save MsgPack encoding of original (user-provided) format clause · 602060ae
      Georgiy Lebedev authored
      In scope of #4693 we need to save the MsgPack encoding of the original
      (i.e, user-provided) format clause in the tuple format structure for
      serialization to Lua and IPROTO (#8147, #8633): since tuple formats for
      spaces are created from space definitions, we need to also save the MsgPack
      encoding in the latter.
      
      Since we need to pass the format clause MsgPack encoding to the runtime
      tuple format constructor, refactor it to perform field definition decoding
      in-place. We cannot use default field definition array decoding for
      external formats (67578d1f), so introduce a new field name decoder for this
      case.
      
      Needed for #4693
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      602060ae
    • Georgiy Lebedev's avatar
      box: make Lua tuple formats objects userdata instead of cdata · 3a73bad3
      Georgiy Lebedev authored
      userdata is the preferred type for Lua objects wrapping C structures, but
      tuple formats were made cdata for no good reason: change them to userdata.
      
      Needed for #4693
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      3a73bad3
    • Georgiy Lebedev's avatar
      box: move Lua tuple format wrapper to new `tuple_format` submodule · f6db0464
      Georgiy Lebedev authored
      Since we are going to make public Lua tuple format objects, introduce a new
      `tuple_format` submodule and move the code related to these objects there.
      
      Needed for #4693
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      f6db0464
  4. Jun 30, 2023
    • Sergey Bronnikov's avatar
      test: fix flakiness in gh_6128_background_mode_test · 47380bb7
      Sergey Bronnikov authored
      Test runs an external process with tarantool that writes to a log file.
      Then test reads that log file and searches a string with required
      message in it (see function check_err_msg). Test was flaky on macOS and
      I suspect it was happening due to a high log level - timeout was not
      enough to wait message in the log file.
      
      Patch decreases a log level to a default value and replaces io
      functions with the similar alternatives in a fio module. Using
      fio functions allows to not block fibers.
      
      NO_CHANGELOG=test fix
      NO_DOC=test fix
      47380bb7
    • Vladimir Davydov's avatar
      lua/xlog: don't ignore unknown header fields · 8a25d170
      Vladimir Davydov authored
      The xlog reader Lua module uses the xlog_cursor_next_row, which decodes
      the row header with xrow_header_decode. The latter silently ignores any
      unknown fields, which complicates catching bugs when garbage is written
      to a row header by mistake, for example, see #8783.
      
      Let's parse a row header without using xrow_header_decode in the xlog
      reader module, like we parse a row body, and output all unknown/invalid
      keys as is.
      
      To do that, we have to extend the xlog cursor API with the new method
      xlog_cursor_next_row_raw that returns a pointer to the position in the
      tx buffer where the next xrow is stored without advancing it. To avoid
      a memory leak in case the caller fails to parse an xrow returned by this
      function, we also have to move the call to xlog_tx_cursor_destroy from
      xlog_tx_cursor_next_row to xlog_cursor_next_tx.
      
      While we are at it,
       - Don't raise an error if a key type encountered in a row body is
         invalid (not an integer). Instead, silently ignore such keys.
       - Remove the useless body MsgPack validness check because we already
         check it after decoding the header.
       - Add error injection based tests to check all the corner cases.
      
      NO_DOC=bug fix
      8a25d170
    • Vladimir Davydov's avatar
      txn: reset stream_id row header field · f058cee7
      Vladimir Davydov authored
      To avoid garbage written to xlog.
      
      Closes #8783
      
      NO_DOC=bug fix
      NO_TEST=next commit
      f058cee7
  5. Jun 29, 2023
    • Nikolay Shirokovskiy's avatar
      update: add tests related to multiple update to the same field · 3e1c2772
      Nikolay Shirokovskiy authored
      These are misc tests that can be related to the issue. Not sure all of
      them do not work before the patch set for the issue. It is nice to have
      them and be sure everything works fine.
      
      Closes #8658
      
      NO_DOC=minor
      3e1c2772
    • Nikolay Shirokovskiy's avatar
      update: fix xrow_update_err_double message usage · 7a8177bb
      Nikolay Shirokovskiy authored
      We can replace the last `xrow_update_err_double` occurrence with
      `xrow_update_err_no_such_field`. The last is correct one for this place
      as `xrow_update_op_do_field_##op_type` is called only for non terminal
      paths. Thus if XUPDATE_SCALAR is encountered then the field referenced
      does not exist.
      
      Part of #8658
      
      NO_DOC=minor
      NO_CHANGELOG=minor
      7a8177bb
Loading