Skip to content
Snippets Groups Projects
  1. Dec 24, 2021
  2. Dec 23, 2021
  3. Dec 21, 2021
    • AnaNek's avatar
      build: link bundled libcurl with nghttp2 · fa8d70ca
      AnaNek authored
      Before this patch Tarantool http client did not support HTTP/2.
      The reasons to enable HTTP/2 support are efficiency offered by
      the protocol and potential ability to interact with a GRPC server.
      The CMake version requirement is updated from 3.2 to 3.3, because
      we need generator expressions in cmake for setting multiple paths
      in CMAKE_FIND_ROOT_PATH for nghttp2 support.
      
      Closes #5771
      
      @TarantoolBot document
      Title: Now we require CMake 3.3 to build tarantool
      
      In tarantool/doc#2065 we requested to update
      the CMake minimum version to 3.2. Now it is time for 3.3.
      See details in the linked commit.
      fa8d70ca
    • Andrey Saranchin's avatar
      build: fix build with glibc-2.34 · 9c01b325
      Andrey Saranchin authored
      Macros SIGSTKSZ used to be an integral constant but
      in glibc-2.34 it turned into a runtime function so it
      cannot be used as constant known size for arrays anymore.
      
      Beyond this, SIGSTKSZ is not enough for alt. signal stack size
      when you use ASAN, so the size was increased.
      
      Closes #6686
      9c01b325
  4. Dec 20, 2021
    • Vladimir Davydov's avatar
      box: add takes_raw_args lua function option · d2a01245
      Vladimir Davydov authored
      Closes #3349
      
      @TarantoolBot document
      Title: Document that takes_raw_args function option
      
      A new function option was added - `takes_raw_args`:
      
      ```lua
      box.schema.func.create('my_func', {takes_raw_args = true})
      ```
      
      If set for a Lua function, the functions arguments will be passed
      wrapped in a msgpack object (see `msgpack.object()`) when the function
      is called over IPROTO or by `box.func.NAME:call()`:
      
      ```lua
      local msgpack = require('msgpack')
      local my_func = function(mp)
          assert(msgpack.is_object(mp))
          local args = mp:decode() -- array of arguments
      end
      ```
      
      Using this option might improve performance in case a function forwards
      most of its arguments to another instance or writes them to a database,
      because it eliminates msgpack decoding in Lua.
      d2a01245
    • Vladimir Davydov's avatar
      schema: allow to pass is_multikey func opt without sub-table · 66f5368b
      Vladimir Davydov authored
      Currently, `is_multikey` function option must be wrapped in `opts`
      sub-table, which looks ugly:
      
      ```lua
      box.schema.func.create('my_func', {
          is_deterministic = true,
          opts = {is_multikey = true}
      })
      ```
      
      Let's allow to pass the option directly.
      
      Note, the `is_multikey` option is reported by `box.func` at the same
      level as `is_deterministic` so this would only be consistent:
      
      ```lua
      box.func.my_func.is_deterministic
      box.func.my_func.is_multikey
      ```
      
      @TarantoolBot document
      Title: Document that is_multikey func opt may be passed without sub-table
      
      It's not necessary anymore to wrap `is_multikey` in `opts` table when
      creating a function:
      
      ```lua
      box.schema.func.create('my_func', {is_multikey = true})
      ```
      
      is equivalent to
      
      ```lua
      box.schema.func.create('my_func', {opts = {is_multikey = true}})
      ```
      
      which still works, but deprecated.
      
      Pease update [this][func-index] documentation entry.
      
      [func-index]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_space/create_index/#creating-a-functional-index
      66f5368b
    • Vladimir Davydov's avatar
      net.box: add return_raw request option · 44234562
      Vladimir Davydov authored
      Closes #4861
      
      @TarantoolBot document
      Title: Document return_raw net.box option
      
      If the `return_raw` flag is set, net.box will return response data
      wrapped in a msgpack object (see `msgpack.object()`) instead of decoding
      it to Lua. This can be useful when a response is supposed to be passed
      through without decoding or with partial decoding - creating a msgpack
      object can reduce pressure on the Lua garbage collector in this case.
      
      Example:
      
      ```lua
      local c = require('net.box').connect(uri)
      local mp = c.eval('eval ...', {1, 2, 3}, {return_raw = true})
      mp:decode() -- {1, 2, 3}
      ```
      
      The option is ignored for methods that return nil (`begin`, `commit`,
      `rollback`, `upsert`, `prepare` will return nil even if `return_raw` is
      set) and for `index.count` (it returns a number). For `execute`, the
      option is applied only to data (`rows`) while metadata is decoded
      irrespective of the value of `return_raw`.
      44234562
    • Vladimir Davydov's avatar
      net.box: allow to pass msgpack object to dml · 42acde67
      Vladimir Davydov authored
      Internally, net.box can do some transformations of a key passed to
      select/update/delete, see luamp_covert_key. We need to skip those if
      a msgpack object is passed.
      
      Part of #3349
      42acde67
    • Vladimir Davydov's avatar
      net.box: allow to pass msgpack object to call/eval · dfe71aa9
      Vladimir Davydov authored
      Msgpack object is already handled by the msgpack encoder.
      All we need to do is to remove the args check from call/eval.
      
      Part of #3349
      dfe71aa9
    • Vladimir Davydov's avatar
      msgpack: introduce msgpack.object · 36da3962
      Vladimir Davydov authored
      Closes #1629
      Closes #3909
      Closes #5316
      Needed for #3349
      Needed for #4861
      
      @TarantoolBot document
      Title: Document msgpack.object
      
      The following new methods were added to the Lua `msgpack` module:
       - `msgpack.object()`: Encodes an arbitrary Lua object given in its only
         argument in MsgPack and returns the encoded MsgPack data encapsulated
         in a MsgPack object.
       - `msgpack.object_from_raw()`: Creates a MsgPack object from raw data
         given either as a string or as a pointer and size.
       - `msgpack.is_object()`: Checks is the given argument is a MsgPack
         object.
      
      Example:
      
      ```lua
      local buffer = require('buffer')
      local msgpack = require('msgpack')
      -- Create an object from a Lua object.
      local mp = msgpack.object(123)
      local mp = msgpack.object("foobar")
      local mp = msgpack.object({1, 2, 3})
      local mp = msgpack.object({foo = 1, bar = 2})
      local mp = msgpack.object(box.tuple.new(1, 2, 3))
      -- Create an object from raw data given as a string.
      local data = msgpack.encode({1, 2, 3})
      local mp = msgpack.object_from_raw(data)
      -- Create an object from raw data given as ptr and size.
      local buf = buffer.ibuf()
      msgpack.encode({1, 2, 3}, buf)
      local mp = msgpack.object_from_raw(buf.buf, buf:size())
      -- Check object.
      msgpack.is_object(mp) -- true
      msgpack.is_object({}) -- false
      ```
      
      A MsgPack object has the following methods:
      
       - `mp:decode()`: Decodes MsgPack and returns a Lua object.
       - `mp:iterator()`: Returns an iterator over MsgPack data.
      
      A MsgPack iterator has the following methods:
      
       - `it:decode_array_header()`: Decodes a MsgPack array header under the
         cursor and returns the number of elements in the array. After calling
         this function the iterator points to the first element of the array
         or to the value following the array if the array is empty. Raises an
         error if the type of the value under the iterator cursor is not
         `MP_ARRAY`.
       - `it:decode_map_header()`: Decodes a MsgPack map header under the
         cursor and returns the number of key value pairs in the map. After
         calling this function the iterator points to the first key stored in
         the map or to the value following the map if the map is empty. Raises
         an error if the type of the value under the iterator cursor is not
         `MP_MAP`.
       - `it:decode()`: Decodes a MsgPack value under the iterator cursor and
         advances the cursor. Returns a Lua object corresponding to the
         MsgPack value. Raises a Lua error if there's no data to decode.
       - `it:take()`: Returns a MsgPack value under the iterator cursor as a
         MsgPack object (without decoding). Raises a Lua error if there's no
         data to decode. This function doesn't copy MsgPack data - instead it
         takes a reference to the original object.
       - `it:skip()`: Advances the iterator cursor by skipping one MsgPack
         value under the cursor. Returns nothing. Raises a Lua error if
         there's not data to skip.
      
      Usage example:
      
      ```lua
      local msgpack = require('msgpack')
      local mp = msgpack.object({foo = 123, bar = {1, 2, 3}})
      local it = mp:iterator()
      it:decode_map_header()  -- returns 2
      it:decode()             -- returns 'foo'
      it:decode()             -- returns 123
      it:skip()               -- returns none, skips 'bar'
      local mp2 = it:take()
      mp2:decode()            -- returns {1, 2, 3}
      ```
      
      For more examples, see `test/app-luatest/msgpack_test.lua`.
      
      A MsgPack object can be passed to MsgPack encoder with the effect being
      the same as passing the original Lua object:
      
      ```lua
      local msgpack = require('msgpack')
      local mp = msgpack.object(123)
      msgpack.object({mp, mp}):decode()         -- returns {123, 123}
      msgpack.decode(msgpack.encode({mp, mp}))  -- returns {123, 123}
      ```
      
      In particular, this means that if a MsgPack object stores an array,
      it can be inserted into a database space:
      
      ```lua
      box.space.my_space:insert(msgpack.object({1, 2, 3}))
      ```
      36da3962
    • Timur Safin's avatar
      box, datetime: datetime comparison for indices · 3bd87026
      Timur Safin authored
      * storage hints implemented for `struct datetime` values;
      * proper comparison for indices of those datetime values;
      
      * introduced `engine-tap/` for engine tests, and added `datetime.test.lua`
        there for testing of storing and sorting of datetime values.
      
      Follow-up to #5941
      Part of #5946
      
      @TarantoolBot document
      
      Title: Storage support for datetime values
      
      It's now possible to store datetime values to spaces and to create
      indexed datetime fields.
      3bd87026
    • Timur Safin's avatar
      box, datetime: messagepack support for datetime · 768f5016
      Timur Safin authored
      Serialize `struct datetime` as newly introduced MP_EXT type.
      It saves 1 required integer field and 3 optional fields:
      - epoch is required field;
      - but nsec, tzoffset and tzindex are optional;
      
      * supported json, yaml serialization formats, lua output mode;
      
      Please refer to the
      https://hackmd.io/@Mons/S1Vfc_axK#%D0%A3%D0%BF%D0%B0%D0%BA%D0%BE%D0%B2%D0%BA%D0%B0-%D0%B2-msgpack
      for a description of a messagepack serialization schema for datetime
      values.
      
      Follow-up to #5941
      Part of #5946
      768f5016
    • Timur Safin's avatar
      datetime, lua: interval arithmetics · b0463da8
      Timur Safin authored
      Add interval arithmetics as date:add{} and date:sub{}, or
      overloaded '+' and '-' operators (__add/_sub metamethods).
      There is no special processing for timezone specific
      daylight saving time changes, which impacts result of arithmetics.
      This will be completed after Olsen database support added.
      
      If one needs to control a way arithmetic proceeded for days
      of months then there is optional argument to `:add()` or `:sub()`,
      which defines how we round day in month after arithmetic
      operation:
      
      - 'last' - mode when day snaps to the end of month, if happens:
      
        -- 29.02.2004 -> 31.03.2004
        dt:add{month = 1, adjust = "last" }
      
      - 'none' only truncation toward the end of month performed
        (**default mode**):
      
        -- 29.02.* -> 29.03.*
        dt:add{month = 1, adjust = "none" }
      
      - 'excess' - overflow mode, without any snap or truncation to
        the end of month, straight addition of days in month, stopping
        over month boundaries if there is less number of days.
      
        dt = new{year = 2004, month = 1, day = 31}
        dt:add{month = 1, adjust = 'excess'} -> 02.03.2004
      
        -- i.e. there were 29 days in February 2004, 2 excessive days
           from original date 31 make result 2nd March 2004
      
      Intervals objects, created via `datetime.interval.new()` call
      and objects of the same attributes are equivalent for interval
      arithmetics and exchangeable. i.e.
      
         dt + date.interval.new{year = 1, month = 2}
      
      is the same as
         dt + {year = 1, month = 2}
      
      Follow-up to #5941
      b0463da8
    • Timur Safin's avatar
      datetime: fix negative years formatting · 8394e530
      Timur Safin authored
      Negative Julian dates used to be incorrectly formatted, if there were
      non-zero hour/minutes/second.
      
      ```
      tarantool> T = date.new{year = -1, month = 6, day = 10, hour = 12, min = 10, sec = 10}
      
      tarantool> T
      - -001-06-11T-11:-49:-50Z
      ```
      
      The problem is due to operator % in C/C++ which is returning negative
      modulo toward 0. Use Euclidian (floored) modulo instead.
      
      Correct both `tostring(T)`/`T:format()` and `strftime`-like
      'T:format(fmt)'
      
      ```
      tarantool> T:format()
      - -001-06-10T12:10:10Z
      
      tarantool> T:format('%FT%T.%f%z')
      - -001-06-10T12:10:10.000+0000
      ```
      
      Follow-up to #5941
      8394e530
  5. Dec 17, 2021
    • Vladimir Davydov's avatar
      uri: don't create params table in parse if there's no params · 7483bcab
      Vladimir Davydov authored
      uri.parse always creates params table even if there's no query
      parameters in the given uri:
      
      tarantool> require('uri').parse('localhost:3301')
      ---
      - host: localhost
        service: '3301'
        params: []
      ...
      
      This is confusing and inconsistent with other uri properties, which are
      created only if they actually exist. Let's create the params table only
      if the given uri actually has any params.
      7483bcab
    • Vladimir Davydov's avatar
      cmake: add option to embed luazip · 4b24cf55
      Vladimir Davydov authored
      Needed for Tarantool EE. Off by default.
      4b24cf55
    • mechanik20051988's avatar
      uri: fix break of backward compatibility · c2b5704e
      mechanik20051988 authored
      After updating the uri library, the "params" table was added
      to the URI structure. It breaks backward compatibility in
      case when URI passed to "uri.format" function in old way,
      without params table.
      c2b5704e
    • Vladimir Davydov's avatar
      cmake: allow to add extra export symbols · 16f8fd10
      Vladimir Davydov authored
      This commit defines two cmake variables:
      
       - EXTRA_REEXPORT_LIBRARIES: list of libraries appended to
         reexport_libraries.
       - EXTRA_EXPORTS_FILE_SOURCES: list of exports file sources appended to
         export_file_sources.
      
      Needed to export extra symbols in Tarantool EE build.
      16f8fd10
    • Vladimir Davydov's avatar
      Introduce luarocks hardcoded config for static build · fa0cee99
      Vladimir Davydov authored
      Currently, we use the same luarocks hardcoded config for static build as
      for regular build. As a result, LUA_BINDIR, LUA_INCDIR, SYSCONFDIR, and
      PREFIX point to locations inside the build directory, which doesn't make
      any sense.
      
      Let's introduce a special hardcoded config which will set those
      directories to locations relative to the binary location:
      
       - LUA_BINDIR: directory where the binary is located (bindir)
       - PREFIX: <dir> if bindir is <dir>/bin, otherwise bindir.
       - LUA_INCDIR: PREFIX/include/tarantool
       - SYSCONFDIR: PREFIX/etc/tarantool/rocks
      
      Also, let's add the PREFIX/rocks to the rocks path if present.
      
      This is needed for Tarantool SDK bundle to work as expected.
      In particular the check for <dir>/bin is needed, because SDK
      installs binaries in bundle root, not in /bin.
      fa0cee99
    • Vladimir Davydov's avatar
      Remove unused options from luarocks hardcoded config · 1c05c631
      Vladimir Davydov authored
      The options are not used anymore since update to 3.1.1,
      see third_party/luarocks/src/luarocks/core/cfg.lua.
      
      They should have been removed by commit
      4222c1f6
      ("rocks: update luarocks to 3.1.1")
      1c05c631
    • Vladimir Davydov's avatar
      Format luarocks hardcoded config · 7ac60487
      Vladimir Davydov authored
      It contains Lua code. Format it appropriately. No functional changes.
      7ac60487
  6. Dec 16, 2021
    • Alxander V. Tikhonov's avatar
      test: flaky replication/long_row_timeout.test.lua · 15a26877
      Alxander V. Tikhonov authored
      
      On heavy loaded hosts either slow machines like VMware found the
      following issues:
      
        [010] --- replication/long_row_timeout.result   Fri May  8 08:56:08 2020
        [010] +++ var/rejects/replication/long_row_timeout.reject       Mon Jun 21 04:39:08 2021
        [010] @@ -23,7 +23,7 @@
        [010]  ...
        [010]  box.info.replication[2].downstream.status
        [010]  ---
        [010] -- follow
        [010] +- stopped
        [010]  ...
        [010]  -- make applier incapable of reading rows in one go, so that it
        [010]  -- yields a couple of times.
        [010]
      
      It happened because replication downstream status check occurred too
      early, when it was only in 'stopped' state. This situation happens
      when replica done its initial join, but not reached subscription yet.
      To give the replication status check routine ability to reach the
      needed 'follow' state, it need for it using test_run:wait_downstream()
      routine.
      
      We don't see this issue anymore, but let's fix it still in case we ever
      encounter it again.
      
      Also remove the test from fragile list.
      
      Closes #4351
      
      Co-developed-by: default avatarSerge Petrenko <sergepetrenko@tarantool.org>
      15a26877
    • Mergen Imeev's avatar
      sql: introduce binding for MAP · b6a08c5c
      Mergen Imeev authored
      After this patch, MAP values can be used as bind variables. However, due
      to the current syntax for binding in Lua, the only possible way is to
      use MAP values as the named bind variable.
      
      Part of #4763
      b6a08c5c
    • Mergen Imeev's avatar
      sql: introduce binding for ARRAY · befcbd21
      Mergen Imeev authored
      After this patch, ARRAY values can be used as bind variables. However,
      due to the current syntax for binding in Lua, the only possible way is
      to use ARRAY values as the named bind variable.
      
      Part of #4762
      befcbd21
  7. Dec 14, 2021
    • Alexander Turenko's avatar
      ci: add linter job for changelog entries · 8b1ce351
      Alexander Turenko authored
      What is bad: a considerable amount of boilerplate code should be added
      to just run a simple script. I hope we'll do something with this
      in #6604.
      8b1ce351
    • Alexander Turenko's avatar
      doc: fix several typos in changelog entries · 5b6a2b95
      Alexander Turenko authored
      After this change the release notes are able to be built.
      
      Changed '---' to '----', so comments for a release manager are parsed
      correctly.
      5b6a2b95
    • Alexander Turenko's avatar
      ci: rename luacheck.yml to lint.yml · f0980af8
      Alexander Turenko authored
      The idea is to allow to add more checks here: I'm going to add
      changelogs check in a next commit.
      f0980af8
    • Vladimir Davydov's avatar
      cmake: allow to set arbitrary version · 8509f163
      Vladimir Davydov authored
      This commit allows to override Tarantool version detection by setting
      CMake TARANTOOL_VERSION variable. This is needed to build Tarantool as a
      part of tarantool/sdk, which sets its own, extended version.
      
      Also, this commit updates box-py/args.test.py to allow arbitrary tags
      appended to the version (tarantool/sdk appends -rNNN).
      8509f163
    • Vladimir Davydov's avatar
      cmake: drop auto-update of submodules · 79f531c9
      Vladimir Davydov authored
      Submodules are auto-updated in two cases:
      1. When cmake is called for the first time (and there's no
         src/lib/small/CMakeLists.txt).
      2. If git-describe version doesn't match the version stored in the
         VERSION file.
      
      The first case looks okay, but the second one is confusing. It means
      that the following sequence of commands does NOT update submodules:
      
      ```sh
      cmake . # initial configuration, fetches small
      cd src/lib/small && git checkout HEAD^ && cd - # checkout small
      cmake . # does NOT update small
      ```
      
      while the following sequence of commands does:
      
      ```sh
      git commit --allow-empty -m dummy
      cmake . # updates small!
      ```
      
      This looks confusing, inconsistent, and dangerous. Let's drop this code
      and let the developers update submodules manually when they need to.
      
      This effectively reverts commit c83eea60
      ("CMake: automatically update submodules").
      79f531c9
    • Vladimir Davydov's avatar
      cmake: take git version from main project · c7546efe
      Vladimir Davydov authored
      Use git from CMAKE_SOURCE_DIR instead of PROJECT_SOURCE_DIR so that if
      Tarantool is built as a sub-project, we will use the version of the main
      project.
      c7546efe
    • Vladimir Davydov's avatar
      Add audit log implementation stub · 3f3bb245
      Vladimir Davydov authored
      To build audit log as a part of the box library, set the following cmake
      variables:
       - ENABLE_AUDIT_LOG: ON
       - AUDIT_LOG_SOURCES: audit log source files
       - EXTRA_BOX_INCLUDE_DIRS: header files needed for compilation
      
      and add "audit_impl.h" to EXTRA_BOX_INCLUDE_DIRS - then it will be
      included by "audit.h".
      
      If ENABLE_AUDIT_LOG is unset, then a stub implementation is built, which
      spits a warning to the log on an attempt to configure audit log.
      3f3bb245
    • Vladimir Davydov's avatar
      test: remove {grep,wait}_log limit in some tests · 4ebdd8c0
      Vladimir Davydov authored
      Needed for the tests to pass with enabled audit log.
      4ebdd8c0
  8. Dec 13, 2021
    • Aleksandr Lyapunov's avatar
      lib: use memcpy for unaligned load/store · 7b40dcce
      Aleksandr Lyapunov authored
      Now there's a lot of load_*/store_* functions that are designed
      for unaligned access to values in data stream. Unfortunately
      they are written in a way that makes new compilers to warn about
      outside bounds.
      
      Rewrite it in the most safe, cross-platform and efficient way -
      using memcpy. Note that memcpy with compile-time-defined size is
      recognized by the magority of compilers and they make the most
      efficient operation.
      
      Closes #6703
      7b40dcce
    • Vladimir Davydov's avatar
      Make lookup of statically built binary more robust · 88332b35
      Vladimir Davydov authored
      If the tarantool repository is used as a submodule named <foobar> in
      another repository, then the statically built binary will be placed in
      
        <binary_dir>/<foobar>/src/tarantool
      
      not in
      
        <binary_dir>/src/tarantool
      
      where static-build/CMakeLists.txt currently tries to look it up in order
      to run `ctest` and so we can't use static-build/CMakeLists.txt as is.
      
      Let's instead use
      
        <install_dir>/bin/tarantool
      
      This way `ctest` will work for static-build in both open-source and EE
      repository without requiring any modifications.
      88332b35
    • Alexander Turenko's avatar
      ci: run the whole test suite in debug_aarch64 job · c7fd1d95
      Alexander Turenko authored
      We anyway run all tests on graviton machines in centos_{7,8}_aarch64 and
      fedora_{33,34}_aarch64 jobs.
      c7fd1d95
    • Alexander Turenko's avatar
      ci: bump actions/checkout from v1 to v2.3.4 · 0ae7e209
      Alexander Turenko authored
      Regarding debug_aarch64: we had old git on odroid machines (see
      PR #6180), but now we use graviton ones and, it seems, it becomes
      irrelevant.
      
      Regarding memtx_allocator_based_on_malloc: it seems, checkout@v1 here is
      due to usage of Debian Stretch (with git 2.11) in docker, as in older
      release workflow. Updated it in the spirit of PR #5949.
      
      Left actions/checkout@v1 in containers jobs (coverity and jepsen*): it
      should be revisited separately.
      
      Left actions/checkout@v1 in perf_* jobs: they run on special machines
      and I don't want to dig inside this part of the infrastructure ATM.
      0ae7e209
Loading