Skip to content
Snippets Groups Projects
  1. Aug 02, 2022
    • Alexander Turenko's avatar
      static build: fix icu build on Mac OS 11 and newer · 03e8c562
      Alexander Turenko authored
      Without `-isysroot <SDK_PATH>` the build fails:
      
      ```
      /Library/Developer/CommandLineTools/usr/bin/make[4]: Making `all' in   \
          `makeconv'
      /Library/Developer/CommandLineTools/usr/bin/c++ -O2 -W -Wall -pedantic \
          -Wpointer-arith -Wwrite-strings -Wno-long-long -std=c++11          \
          -Wno-ambiguous-reversed-operator     -o ../../bin/makeconv         \
          gencnvex.o genmbcs.o makeconv.o ucnvstat.o -L../../lib -licutu     \
          -L../../lib -licui18n -L../../lib -licuuc -L../../stubdata         \
          -licudata -lpthread -lm
      ld: library not found for -lpthread
      clang: error: linker command failed with exit code 1 (use -v to see    \
          invocation)
      ```
      
      ICU is written on C++, so we should pass `CXXFLAGS`, not only `CFLAGS`.
      
      Note: `CPPFLAGS` stands for C preprocessor flags, not C++ flags.
      
      Fixes #7459
      
      NO_DOC=it fixes a build failure for one of build types, nothing to
             document
      NO_TEST=it will be tested in a next commit by enabling corresponding
              GitHub Action workflow
      03e8c562
    • Sergey Bronnikov's avatar
      build: bump libyaml · 6aa30d05
      Sergey Bronnikov authored
      Patch updates libyaml to the version that contains fixes of stack
      overflows [1]. PR in upstream with original patch: "Avoid recursion in
      the document loader" [2].
      
      1. https://github.com/yaml/libyaml/issues/107
      2. https://github.com/yaml/libyaml/pull/127
      
      NO_DOC=internal
      NO_TEST=internal
      6aa30d05
    • Alexander Turenko's avatar
      third_party: update nghttp2 from 1.47.0 to 1.48.0 · c78cc103
      Alexander Turenko authored
      This is just regular update to bring bugfixes and improvements in the
      library into tarantool.
      
      https://github.com/nghttp2/nghttp2/releases/tag/v1.48.0
      
      NO_DOC=there is no known behavior changes, consider the change as not
             visible for a user
      NO_TEST=there is no specific problem we want to solve and verify here
      NO_CHANGELOG=see NO_DOC
      c78cc103
    • Yaroslav Lobankov's avatar
      ci: hotfixes for `submodule_update.yml` workflow · 8d51a20c
      Yaroslav Lobankov authored
      - Run the workflow in tarantool/tarantool repo only
      - Fix the condition to compose PR title prefix
      - Add a small description for created PRs
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      8d51a20c
    • Sergey Bronnikov's avatar
      static-build: bump readline from 8.0 to 8.0p1 · 9c6744d6
      Sergey Bronnikov authored
      Just regular update to bring readline security fixes into tarantool.
      
      Added a patch to fix file descriptor leak with zero-length history file.
      
      Source of patch: https://ftp.gnu.org/gnu/readline/readline-8.0-patches/
      Announcement: https://lists.gnu.org/archive/html/bug-readline/2019-08/msg00004.html
      
      NO_TEST=security update of a dependency
      NO_DOC=security update of a dependency
      9c6744d6
    • Sergey Bronnikov's avatar
      static-build: bump ncurses from 6.2 to 6.3-20220716 · b78d10ed
      Sergey Bronnikov authored
      Just regular update to bring ncurses security fixes into tarantool.
      
      New version brings a number of functional and security fixes:
      - ncurses 6.3 before patch 20220416 has an out-of-bounds read and
      segmentation violation in convert_strings in tinfo/read_entry.c in the
      terminfo library (CVE-2022-29458).
      - ncurses through v6.2-1. _nc_captoinfo in captoinfo.c has a heap-based
      buffer overflow (CVE-2021-39537).
      
      CVE-2022-29458 Detail: https://nvd.nist.gov/vuln/detail/CVE-2022-29458
      CVE-2021-39537 Detail: https://nvd.nist.gov/vuln/detail/CVE-2021-39537
      Release announcement: https://lists.gnu.org/archive/html/bug-ncurses/2022-07/msg00008.html
      
      NOTE: Build system uses a link to tarball on a local storage instead of
      official mirror [1].
      
      1. https://invisible-mirror.net/archives/ncurses/current/
      
      NO_TEST=security update of a dependency
      NO_DOC=security update of a dependency
      b78d10ed
    • Vladimir Davydov's avatar
      Fix a low-probability crash in our qsort implementation · adc5c05e
      Vladimir Davydov authored
      This is a backport of commit
      https://github.com/postgres/postgres/commit/9d6077abf9d6efd992a59f05ef5aba981ea32096
      
      It's standard for quicksort implementations, after having partitioned the
      input into two subgroups, to recurse to process the smaller partition and
      then handle the larger partition by iterating.  This method guarantees
      that no more than log2(N) levels of recursion can be needed.  However,
      Bentley and McIlroy argued that checking to see which partition is smaller
      isn't worth the cycles, and so their code doesn't do that but just always
      recurses on the left partition.  In most cases that's fine; but with
      worst-case input we might need O(N) levels of recursion, and that means
      that qsort could be driven to stack overflow.  Such an overflow seems to
      be the only explanation for today's report from Yiqing Jin of a SIGSEGV
      in med3_tuple while creating an index of a couple billion entries with a
      very large maintenance_work_mem setting.  Therefore, let's spend the few
      additional cycles and lines of code needed to choose the smaller partition
      for recursion.
      
      NO_DOC=bug fix
      NO_TEST=backport
      adc5c05e
    • Pavel Balaev's avatar
      rocks: bump 3.1.3 version · d253b690
      Pavel Balaev authored
      The current version of Luarocks used in tarantool has a
      security vulnerability:
      https://github.com/luarocks/luarocks/pull/1019
      
      This issue is fixed in release 3.1.3, commit:
      6ffa8cbe2bcdea2c1ee0f4b32291d179210707b6
      
      Part of https://github.com/tarantool/security/issues/12
      
      NO_DOC=bugfix
      NO_TEST=bugfix
      NO_CHANGELOG=bugfix
      d253b690
    • Sergey Vorontsov's avatar
      ci: update tarantool submodule in tarantool-ee · 7601fb43
      Sergey Vorontsov authored
      Bump tarantool submodule in tarantool-ee via pull requests:
      
      * tarantool@master -> tarantool-ee@TarantoolBot/update-tarantool-master
      * tarantool@2.10 -> tarantool-ee@TarantoolBot/update-tarantool-2.10
      
      If there is an open PR already, it will be updated every time when
      a new commit is pushed to tarantool@master or tarantool@2.10.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      
      Closes tarantool/tarantool-ee#175
      7601fb43
    • Mergen Imeev's avatar
      sql: always treat NaN as NULL · 7c5651af
      Mergen Imeev authored
      In most cases, NaN was treated as NULL. But in case NaN was returned as
      a result of a Lua or C user defined function, it was considered a
      double. After this patch, NaN will also be considered NULL in the
      specified cases.
      
      Closes #6374
      Closes #6572
      
      NO_DOC=bugfix
      7c5651af
    • Mergen Imeev's avatar
      sql: fix wrong flag is_res_neg in sql_rem_int() · 1f4bb194
      Mergen Imeev authored
      This patch makes the is_res_neg flag false in the sql_rem_int() function
      if the left value is negative and the result is 0. Prior to this patch,
      the value of the flag was true, which resulted in an assertion during
      encoding 0 as MP_INT.
      
      Closes #6575
      
      NO_DOC=bugfix
      1f4bb194
    • Mergen Imeev's avatar
      sql: do nothing in ROUND() if precision is too big · 4c216c4c
      Mergen Imeev authored
      The smallest positive double value is 2.225E-307, and the value before
      the exponent has a maximum of 15 digits after the decimal point. This
      means that double values cannot have more than 307 + 15 digits after
      the decimal point.
      
      After this patch, ROUND() will return its first argument unchanged if
      the first argument is DOUBLE and the second argument is INTEGER greater
      than 322.
      
      Closes #6650
      
      NO_DOC=bugfix
      4c216c4c
  2. Aug 01, 2022
    • Vladimir Davydov's avatar
      memtx: decompress tuples in snapshot iterator · f167c1af
      Vladimir Davydov authored
      Compressed tuples aren't supposed to be seen outside memtx internals:
      we always decompresses tuples before returning them to the user; tuples
      are written decompressed to xlog. We should also decompress tuples
      before writing them to a snapshot or sending them to a remote replica,
      but currently we don't, which results in a crash trying to recover from
      a snapshot that contains compressed spaces.
      
      This patch fixes this by decompressing all tuples in snapshot iterator.
      Since the decompressed tuples are allocated on the region, we should
      also call fiber_gc() after each snapshot iterator iteration. Note that
      the decompression is unconditional, i.e. we try to decompress all
      tuples stored in an index. In general we can't figure out if a tuple is
      compressed, because its format may be unavailable (when a tuple is
      freed, its format_id is overwritten).
      
      Needed for https://github.com/tarantool/tarantool-ee/issues/171
      
      NO_DOC=bug fix
      NO_TEST=ee
      NO_CHANGELOG=ee
      f167c1af
    • Vladimir Davydov's avatar
      memtx: get rid of memtx_tuple_maybe_decompress · 8d8e6e60
      Vladimir Davydov authored
      memtx_tuple_maybe_decompress calls memtx_tuple_decompress if the tuple
      is compressed, otherwise it returns the tuple as is. Actually, the only
      place we call memtx_tuple_decompress directly is memtx_tuple_validate,
      but there we check if the tuple is compressed first so we could just as
      well call memtx_tuple_maybe_decompress.
      
      Let's make memtx_tuple_decompress work like memtx_tuple_maybe_decompress
      and kill the latter.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      8d8e6e60
    • Andrey Saranchin's avatar
      box: add changelog and doc request · b24791a6
      Andrey Saranchin authored
      @TarantoolBot document
      Title: Fiber slice
      
      Introduce fiber slices. Fiber slice is over if fiber is being
      executed without yield longer than the value of the slice.
      One can check if the slice is over. Also it is checked during
      space iteration when one uses select/pairs/get methods and
      before DML queries. Fiber slice consists of two parts: warning
      and error slices. When warning slice is over, warning
      "fiber has not yielded for more than <warn_slice> seconds"
      will be written in log, and when error slice is over, an error
      "FiberSliceIsExceeded: fiber slice is exceeded" will be
      thrown. Default fiber slice is 0.5 sec for warning and 1.0 sec
      for error.
      
      Lua API:
      slice = {warn = time_in_seconds, err = time_in_seconds}
      - fiber.check_slice() - write in warn log if warning slice is over,
      throw an error if error slice is over.
      - fiber.set_max_slice([fib_id], slice) - set max slice to passed fiber,
      otherwise set default max slice that will used by all fibers without
      max slice. One can pass number instead of table to set error slice
      and disable warning.
      - fiber.set_slice(slice) - set slice for current fiber execution.
      One can pass number instead of table to set error slice
      and disable warning.
      - fiber.extend_slice(slice) - extend slice of current fiber execution.
      One can pass number instead of table to extend error slice only.
      
      One can send SIGURG to an instance to set current slice to 0.
      It will stop iteration over space.
      
      Examples:
      
      Using fiber slice to stop doing stuff.
      ```lua
      fiber = require('fiber')
      function fib_f()
          -- Run until condition is false or error slice is over.
          while condition do
              fiber.check_slice()
              do_stuff()
          end
      end
      
      fib = fiber.new(fib_f)
      -- Set error slice 0.5 sec and no warning slice for our fiber.
      fib:set_max_slice(0.5)
      fib:set_joinable(true)
      fib:join()
      ```
      
      Using fiber slice to limit iteration in space.
      ```lua
      fiber = require('fiber')
      function fib_f()
          -- Full scan on a huge space will be stopped.
          box.space.huge_space:select{}
      end
      
      fib = fiber.new(fib_f)
      -- Set error timeout 3 sec and warning timeout 0.2 sec for our fiber.
      fib:set_max_slice({warn = 0.2, err = 3})
      fib:set_joinable(true)
      fib:join()
      ```
      
      NO_TEST=no changes
      b24791a6
    • Andrey Saranchin's avatar
      fiber: allow to reset fiber slice with SIGURG · 1a3b710d
      Andrey Saranchin authored
      The patch introduces opportunity for user to reset
      slice of current fiber execution. It allows to limit
      iteration in space with SIGURG.
      
      NO_CHANGELOG=see later commits
      NO_DOC=see later commits
      1a3b710d
    • Andrey Saranchin's avatar
      box: allow to limit space iteration with timeout · bc053c55
      Andrey Saranchin authored
      Currently, there is no way to interrupt a long execution of a
      request (such as s:select(nil)). This patch introduces this
      opportunity.
      
      Box will use fiber deadline timeout as a timeout for DML usage.
      Thus, when deadline of current fiber is up, all DML requests will
      end with a particular error.
      
      Closes #6085
      
      NO_CHANGELOG=see later commits
      NO_DOC=see later commits
      bc053c55
    • Andrey Saranchin's avatar
      test: adapt tests to iteration limit · b33ea6ea
      Andrey Saranchin authored
      Part of #6085
      
      NO_TEST=no behavior changes
      NO_CHANGELOG=no behavior changes
      NO_DOC=no behavior changes
      b33ea6ea
    • Andrey Saranchin's avatar
      fiber: introduce fiber slice · e9bd2250
      Andrey Saranchin authored
      This patch introduces execution time slice for fiber. Later, we will use
      this mechanism to limit iteration in space.
      
      Part of #6085
      
      NO_CHANGELOG=see later commits
      NO_DOC=see later commits
      e9bd2250
    • Alexander Turenko's avatar
      fiber_channel: add accessor to internal functions · 395c30e8
      Alexander Turenko authored
      The Rust module [1] leans on several internal symbols. They were open in
      Tarantool 2.8 (see #2971 and #5932), but never were in the public API.
      Tarantool 2.10.0 hides the symbols and we need a way to get them back to
      use in the module.
      
      We have the following options:
      
      1. Design and expose a module API for fiber channels.
      2. Export the symbols with a prefix like `tnt_internal_` (to don't spoil
         the global namespace).
      3. Provide a `dlsym()` alike function to get an address of an internal
         symbol for users who knows what they're doing.
      
      I think that the third way offers the best compromise between amount of
      effort, quality of the result and opportunities to extend. In this
      commit I hardcoded the list of functions to make the change as safe as
      possible. Later I'll return here to autogenerate the list.
      
      Exported the following function from the tarantool executable:
      
      ```c
      void *
      tnt_internal_symbol(const char *name);
      ```
      
      I don't add it into the module API headers, because the function is to
      perform a dark magic and we don't suggest it for users.
      
      While I'm here, added `static` to a couple of fiber channel functions,
      which are only used within the compilation unit.
      
      [1]: https://github.com/picodata/tarantool-module
      
      Part of #7228
      Related to #6372
      
      NO_DOC=don't advertize the dangerous API
      NO_CHANGELOG=don't advertize the dangerous API
      395c30e8
  3. Jul 29, 2022
  4. Jul 27, 2022
    • Ilya Verbin's avatar
      box: fix thread_id check in box.stat.net.thread[] · 969b76ac
      Ilya Verbin authored
      The valid range for thread_id is [0, iproto_threads_count - 1].
      
      Closes #7196
      
      NO_DOC=bugfix
      969b76ac
    • Igor Munkin's avatar
      ci: make LuaJIT integration workflow rerunnable · 7a41bc9b
      Igor Munkin authored
      
      If Tarantool tests fail in LuaJIT integration workflow used in
      tarantool/luajit repository and LuaJIT submodule is already bumped in
      tarantool/tarantool repository, one can't rerun the failed job since
      "LuaJIT bump" step brings down the whole pipeline with "nothing to
      commit" reason.
      
      There are still many flaky tests, so to make this integration workflow
      rerunnable from tarantool/luajit repository --allow-empty option is
      added to the git commit command.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      
      Reviewed-by: default avatarYaroslav Lobankov <y.lobankov@tarantool.org>
      Reviewed-by: default avatarSergey Kaplun <skaplun@tarantool.org>
      Signed-off-by: default avatarIgor Munkin <imun@tarantool.org>
      7a41bc9b
    • Ilya Verbin's avatar
      box: check for foreign keys on space:truncate() · 7ec71b4f
      Ilya Verbin authored
      Add a missed check to on_replace_dd_truncate, similar to
      on_replace_dd_space and on_replace_dd_index.
      
      Closes #7309
      
      NO_DOC=bugfix
      7ec71b4f
    • Igor Munkin's avatar
      luajit: bump new version · 3214fa04
      Igor Munkin authored
      * test: make sysprof tests more deterministic
      * test: increase sampling interval in sysprof tests
      * LJ_GC64: Fix IR_VARG offset for fixed number of results.
      * sysprof: implement stack sandwich support
      * symtab: fix .symtab section dump of the executable
      * sysprof: disable proto and trace dumps in default
      * ci: introduce GitHub action for environment setup
      * build: configure parallel jobs
      * ci: replace hw.ncpu with hw.logicalcpu for macOS
      * ci: add Tarantool integration testing
      * ci: remove GC64 matrix entries for ARM64 workflows
      * ci: fix --parallel argument for MacOS runners
      
      Closes #7172
      Closes #7244
      Closes #7264
      Part of #7230
      Needed for #7472
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      3214fa04
    • Andrey Saranchin's avatar
      core: introduce helper tt_sigaction · 9839812c
      Andrey Saranchin authored
      The problem is that even if we block all signals on all
      threads except the main thread, the signals still can be
      delivered to other threads (#7206). And another problem
      is that user can spawn his own thread and not block
      signals.
      
      That is why the patch introduces tt_sigaction function that
      guarantees that all signals will be handled only by the main
      thread. We use this helper in clock_lowres module.
      This is supposed to solve the problem, described in #7408.
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      9839812c
    • Andrey Saranchin's avatar
      crash: do not use fiber() macro in crash_collect() · 277b70b5
      Andrey Saranchin authored
      Currently, tarantool uses fiber() macro in crash_collect()
      to collect backtraces, which is redundant and leads to
      NULL dereferencing if crash signal callback is executed
      on thread with no initialized cord.
      
      The patch makes it possible not to use fiber module for
      collecting backtraces and gets rid of fiber() macro
      in crash_collect().
      
      NO_CHANGELOG=internal
      NO_TEST=internal
      NO_DOC=internal
      277b70b5
  5. Jul 26, 2022
    • Alexander Turenko's avatar
      tuple: add JSON path field accessor to module API · bcca0b2b
      Alexander Turenko authored
      Added a function (see the API in the documentation request below), which
      reflects the `tuple[json_path]` Lua API (see #1285).
      
      Part of #7228
      
      @TarantoolBot document
      Title: tuple: access a field using JSON path via module API
      
      The following function is added into the module API:
      
      ```c
      /**
       * Return a raw tuple field in the MsgPack format pointed by
       * a JSON path.
       *
       * The JSON path includes the outmost field. For example, "c" in
       * ["a", ["b", "c"], "d"] can be accessed using "[2][2]" path (if
       * index_base is 1, as in Lua). If index_base is set to 0, the
       * same field will be pointed by the "[1][1]" path.
       *
       * The first JSON path token may be a field name if the tuple
       * has associated format with named fields. A field of a nested
       * map can be accessed in the same way: "foo.bar" or ".foo.bar".
       *
       * The return value is valid until the tuple is destroyed, see
       * box_tuple_ref().
       *
       * Return NULL if the field does not exist or if the JSON path is
       * malformed or invalid. Multikey JSON path token [*] is treated
       * as invalid in this context.
       *
       * \param tuple a tuple
       * \param path a JSON path
       * \param path_len a length of @a path
       * \param index_base 0 if array element indexes in @a path are
       *        zero-based (like in C) or 1 if they're one-based (like
       *        in Lua)
       * \retval a pointer to a field data if the field exists or NULL
       */
      API_EXPORT const char *
      box_tuple_field_by_path(box_tuple_t *tuple, const char *path,
      			uint32_t path_len, int index_base);
      ```
      bcca0b2b
    • Alexander Turenko's avatar
      tuple: add index base to JSON path accessors · 75f470cf
      Alexander Turenko authored
      It is preparatory change to expose the `box_tuple_field_by_path()`
      function into the module API. A user should have ability to choose
      zero-based (as in C) or one-based (as in Lua) array element selectors.
      This commit adds the `index_base` parameter into several internal
      functions, which access a field using a JSON path.
      
      Part of #7228
      
      NO_DOC=refactoring, no user-visible changes
      NO_TEST=will be tested in a next commit implicitly by
              box_tuple_field_by_path() test cases
      NO_CHANGELOG=refactoring, no user-visible changes
      75f470cf
    • Nick Volynkin's avatar
      ci: remove tests for Ubuntu Impish (21.10) · a3423f3d
      Nick Volynkin authored
      Ubuntu Impish Indri (21.10) has reached end of life
      (https://wiki.ubuntu.com/Releases). There will be
      no further releases of Tarantool for Ubuntu 21.10,
      hence there's no need to run tests and check
      packaging.
      
      NO_TEST=ci
      NO_DOC=ci
      
      Resolves tarantool/tarantool-qa#257
      a3423f3d
  6. Jul 25, 2022
    • Ilya Verbin's avatar
      box: return 1-based fkey field numbers to Lua · 014f5aa1
      Ilya Verbin authored
      In Lua field's numbers are counted from base 1, however currently
      space:format() and space.foreign_key return zero-based foreign key
      fields, which leads to an error on space:format(space:format()).
      
      Closes #7350
      
      NO_DOC=bugfix
      014f5aa1
    • Ilya Verbin's avatar
      box: do not modify format arg by normalize_format · a8b6fd0c
      Ilya Verbin authored
      Currently a foreign_key field in the `format` argument, passed to
      normalize_format, can be changed inside normalize_foreign_key_one.
      Fix this by using a local copy of def.field.
      
      NO_DOC=bugfix
      NO_CHANGELOG=minor bug
      a8b6fd0c
  7. Jul 22, 2022
  8. Jul 21, 2022
    • Ilya Verbin's avatar
      box: drop space_id for FK referring to same space · f21f8e90
      Ilya Verbin authored
      It is inconvenient to create self-referencing FK constraints, as the
      space ID will only be generated during space creation. This is
      especially useful for SQL, since the format for the space is created
      at compile time, and the space ID is only obtained at run time.
      
      Closes #7200
      
      @TarantoolBot document
      Title: Describe foreign keys referring to the same space
      Since: 2.11
      Root document: https://www.tarantool.io/en/doc/latest/book/box/data_model/#foreign-keys
      
      It is possible to create a foreign key that refers to the same space (a
      child space equals to the parent space).
      To do that, omit `space` in the `foreign_key` parameter, or set it to
      the id or to the name of the current space.
      f21f8e90
    • Boris Stepanenko's avatar
      test: fix execpl call in unit/coio test · 7c434166
      Boris Stepanenko authored
      According to man 3 exec: "The first argument, by convention, should point
      to the filename associated with the file being executed.". Using empty
      string as the first argument while calling `true` program, provided by
      coreutils led to error message being printed to stderr, which failed the
      test.
      
      This patch passes 'true' as the first argument.
      
      Closes #7452.
      
      NO_DOC=test
      NO_CHANGELOG=test
      7c434166
    • Nikita Zheleztsov's avatar
      net.box: add checking of options · d7da59b4
      Nikita Zheleztsov authored
      Currently net.box's methods doesn't check types of passed options.
      This can lead to Lua's internal errors, which are not self-explaining.
      
      Let's add this functionality and raise errors with meaningful messages
      in case of incorrect options.
      
      Closes #6063
      Closes #6530
      
      NO_DOC=Not a visible change
      d7da59b4
Loading