Skip to content
Snippets Groups Projects
  1. Oct 18, 2024
    • Andrey Saranchin's avatar
      memtx: always read prepared tuples of system spaces · 7f0b2bee
      Andrey Saranchin authored
      Since we often search spaces, users, funcs and so on in internal caches
      that have `read-committed` isolation level (prepared tuples are seen),
      let's always allow to read prepared tuples of system spaces.
      
      Another advantage of such approach is that we never handle MVCC when
      working with system spaces, so after the commit they will behave in the
      same way - prepared tuples will be seen. The only difference is that
      readers of prepared rows will be aborted if the row will be rolled back.
      
      By the way, the inconsistency between internal caches and system spaces
      could lead to crash in some sophisticated scenarios - the commit fixes
      this problem as well because now system spaces and internal caches are
      synchronized.
      
      Closes #10262
      Closes tarantool/security#131
      
      NO_DOC=bugfix
      
      (cherry picked from commit b33f17b25de6bcbe3ebc236250976e4a0250e75e)
      7f0b2bee
    • Andrey Saranchin's avatar
      alter: wait for previous alters to commit on DDL · ea1c829f
      Andrey Saranchin authored
      Yielding DDL operations acquire DDL lock so that the space cannot be
      modified under its feet. However, there is a case when it actually can:
      if a yielding DDL has started when there is another DDL is being
      committed and it gets rolled back due to WAL error, `struct space`
      created by rolled back DDL is deleted - and it's the space being altered
      by the yielding DDL. In order to fix this problem, let's simply wait for
      all previous alters to be committed.
      
      We could use `wal_sync` to wait for all previous transactions to be
      committed, but it is more complicated - we need to use `wal_sync` for
      single instance and `txn_limbo_wait_last_txn` when the limbo queue has
      an owner. Such approach has more pitfalls and requires more tests to
      cover all cases. When relying on `struct alter_space` directly, all
      situations are handled with the same logic.
      
      Alternative solutions that we have tried:
      1. Throw an error in the case when user tries to alter space when
         there is another non-committed alter. Such approach breaks applier
         since it applies rows asynchronously. Trying applier to execute
         operations synchronously breaks it even harder.
      2. Do not use space in `build_index` and `check_format` methods. In this
         case, there is another problem: rollback order. We have to rollback
         previous alters firstly, and the in-progress one can be rolled back
         only after it's over. It breaks fundamental memtx invariant: rollback
         order must be reverse of replace order. We could try to use
         `before_replace` triggers for alter, but the patch would be bulky.
      
      Closes #10235
      
      NO_DOC=bugfix
      
      (cherry picked from commit fee8c5dd6b16471739ed8512ba4137ff2e7274aa)
      ea1c829f
  2. Oct 16, 2024
    • Ilya Verbin's avatar
      box: fix SIGSEGV on unaligned access to `struct applier` · 8a1f72b6
      Ilya Verbin authored
      All structures with a non-default alignment (set by `alignas()`) must be
      allocated by `aligned_alloc()`, otherwise an access to such a structure
      member fill crash, e.g. if compiled with AVX-512 support.
      
      See also commit a60ec82d4f07 ("box: fix SIGSEGV on unaligned access to a
      struct with extended alignment").
      
      Closes #10699
      
      NO_DOC=bugfix
      NO_CHANGELOG=minor
      NO_TEST=tested by debug_asan_clang workflow
      
      (cherry picked from commit bf091358806ed17bf44efd2cf382a43c0ba49fe0)
      8a1f72b6
    • Sergey Bronnikov's avatar
      cmake: fix build with GCC's AddressSanitizer · 89815cd3
      Sergey Bronnikov authored
      GNU GCC compiler has AddressSanitizer support since 4.8.0 [1],
      but it was unsupported in tarantool's build. The patch
      fixes a build by GNU GCC with enabled AddressSanitizer.
      
      1. https://gcc.gnu.org/gcc-4.8/changes.html
      
      NO_CHANGELOG=build
      NO_DOC=build
      NO_TEST=build
      
      (cherry picked from commit ef91f92a22c6d7910ecdd00ab14da359343a2ec2)
      89815cd3
  3. Oct 15, 2024
    • Nikolay Shirokovskiy's avatar
      say: fix NULL pointer dereference in log_syslog_init · f67e047a
      Nikolay Shirokovskiy authored
      If opts.identity is NULL and strdup is failed we do NULL pointer
      dereference when reporting the error. Let's just panic if strdup() failed.
      While at it replace another strdup() with xstrdup() in this function. Our
      current approach is to panic on runtime OOM.
      
      Closes tarantool/security#128
      
      NO_TEST=issue is not possible after the fix
      NO_CHANGELOG=not reproducible
      NO_DOC=bugfix
      
      (cherry picked from commit 47b72f44986797466b95b9431a381dbef7dd64fd)
      f67e047a
  4. Oct 14, 2024
    • Alexander Turenko's avatar
      build: update libcurl to curl-8_10_1-241-g461ce6c61 · 89b2760e
      Alexander Turenko authored
      The reason is that the previous libcurl submodule update in commit
      0919f390802f146852b462215327ef03e2730cfc ("third_party: update libcurl from
      8.8.0 to 8.10.1") reveals the following regression:
      
      NOWRAP
      ```c
      $ tarantool -e "require('http.client').new():get('https://google.com') collectgarbage()"
      tarantool: ./third_party/curl/lib/multi.c:3691: curl_multi_assign: Assertion `!(multi)' failed.
      Aborted (core dumped)
      ```
      NOWRAP
      
      The stacktrace is the following:
      
      NOWRAP
      ```c
      <...>
      #4  __assert_fail
      #5  curl_multi_assign            // <- called by us
      #6  curl_multi_sock_cb           // <- this is our callback
      #7  Curl_multi_pollset_ev
      #8  cpool_update_shutdown_ev
      #9  cpool_discard_conn
      #10 cpool_close_and_destroy_all
      #11 Curl_cpool_destroy
      #12 curl_multi_cleanup
      #13 curl_env_finish              // <- destroy the multi handle
      #14 httpc_env_finish
      #15 luaT_httpc_cleanup
      #16 lj_BC_FUNCC
      #17 gc_call_finalizer
      #18 gc_finalize
      #19 gc_onestep
      #20 lj_gc_fullgc
      #21 lua_gc
      #22 lj_cf_collectgarbage
      #23 lj_BC_FUNCC
      #24 lua_pcall
      #25 luaT_call
      #26 lua_main
      #27 run_script_f
      #28 fiber_cxx_invoke
      #29 fiber_loop
      #30 coro_init
      ```
      NOWRAP
      
      The multi handle is during the destroy, but our
      `CURLMOPT_SOCKETFUNCTION` callback is invoked and the
      `curl_multi_assign()` call (invoked to associate a libev watcher to the
      given file descriptor) fails on the assertion. Everything is as
      described in https://github.com/curl/curl/issues/15201.
      
      The first bad libcurl's commit is [curl-8_10_0-4-g48f61e781][1], but
      later it was fixed in [curl-8_10_1-241-g461ce6c61][2]. This commit
      updates libcurl to this revision to fix the regression.
      
      Adjusted build options in our build script:
      
      * Added `CURL_DISABLE_IPFS=ON`: [curl-8_10_1-57-gce7d0d413][3]
      * Added `CURL_TEST_BUNDLES=OFF`: [curl-8_10_1-67-g71cf0d1fc][4]
      * Changed `ENABLE_WEBSOCKETS=OFF` to `CURL_DISABLE_WEBSOCKETS=ON`:
        [curl-8_10_1-130-gd78e129d5][5]
      
      [1]: https://github.com/curl/curl/commit/48f61e781a01e6a8dbc4a347e280644b1c68ab6a
      [2]: https://github.com/curl/curl/commit/461ce6c6160b86439ddd74c59541231ec9e8558e
      [3]: https://github.com/curl/curl/commit/ce7d0d41378007eda676c83ad6b86c59870cc9f1
      [4]: https://github.com/curl/curl/commit/71cf0d1fca9e1f53524e1545ef0c08d174458d80
      [5]: https://github.com/curl/curl/commit/d78e129d50b2d190f1c1bde2ad1f62f02f152db0
      
      NO_DOC=bugfix
      NO_CHANGELOG=fixes an unreleased commit
      NO_TEST=can't reproduce without https to add a test case, verified
              locally
      
      (cherry picked from commit fbe6d0a0a40945c42609f5119a007b5c3980c232)
      89b2760e
    • Ilya Verbin's avatar
      box: fix UBSan error regarding misaligned store in field_map.c · d835c495
      Ilya Verbin authored
      The type cast is unnecessary and causes false-positive errors:
      
      NO_WRAP
      ```
      ./src/box/field_map.c:110:10: runtime error: store to misaligned address 0x507000071082 for type 'uint32_t *' (aka 'unsigned int *'), which requires 4 byte alignment
      0x507000071082: note: pointer points here
       01 00  00 00 be be be be f0 ff  ff ff 02 00 00 00 be be  be be be be be be 00 00  00 00 00 00 00 00
                    ^
      SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ./src/box/field_map.c:110:10
      ```
      NO_WRAP
      
      Closes #10631
      
      NO_DOC=bugfix
      NO_CHANGELOG=minor
      NO_TEST=tested by debug_asan_clang workflow
      
      (cherry picked from commit 5ddbd85cc377a29dc27d01ad06acdc6acc24cc5b)
      d835c495
    • Ilya Verbin's avatar
      small: bump version · ada3dddf
      Ilya Verbin authored
      New commits:
      * mempool: fix UBSan errors regarding misaligned stores
      
      NO_DOC=submodule bump
      NO_TEST=submodule bump
      NO_CHANGELOG=submodule bump
      
      (cherry picked from commit 9dd56f49be85dc8a1fe874629711a828835f740c)
      ada3dddf
    • Ilya Verbin's avatar
      box: fix SIGSEGV on unaligned access to a struct with extended alignment · 9c36990e
      Ilya Verbin authored
      All structures with a non-default alignment (set by `alignas()`) must be
      allocated by `aligned_alloc()`, otherwise an access to such a structure
      member fill crash, e.g. if compiled with AVX-512 support.
      
      Closes #10215
      Part of #10631
      
      NO_DOC=bugfix
      NO_TEST=tested by debug_asan_clang workflow
      NO_CHANGELOG=fix is actually not user-visible, because tarantool still
                   doesn't work with enabled AVX-512 (#10671)
      
      (cherry picked from commit a60ec82d4f07720148b0724e5feff31f76291b56)
      9c36990e
    • Ilya Verbin's avatar
      Revert "hotfix: change aligned_alloc to posix_memalign" · 10aecd64
      Ilya Verbin authored
      This reverts commit 3c25c667.
      
      `aligned_alloc()` is supported by macOS since 10.15.
      I believe that we do not support older versions now.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      
      (cherry picked from commit 2f4594f748cff99d15f8f6d603797a308793de86)
      10aecd64
  5. Oct 11, 2024
    • Alexander Turenko's avatar
      ci: fix Coverity workflow · fc613299
      Alexander Turenko authored
      It doesn't work since 2023-11-18. The uploading succeeds, but the
      website says:
      
      > The Coverity Build tool version is no longer supported. Please
      > download the latest version for your platform from
      > https://scan.coverity.com/download...
      
      It seems, some specific toolset is installed in the
      `tarantool/testing:debian-buster` image and it was deprecated 11 months
      ago.
      
      Recently the CI workflow starts to fail due to use of the old image with
      an old CMake in it:
      
      > [  2%] Performing configure step for 'bundled-nanoarrow-project'
      > -- Building using CMake version: 3.13.4
      > -- Configuring incomplete, errors occurred!
      > CMake Error at CMakeLists.txt:19 (cmake_minimum_required):
      >   CMake 3.14 or higher is required.  You are running version 3.13.4
      
      It is likely due to commit 49c160c28c97 ("third_party: initial import of
      nanoarrow").
      
      Here I refine the workflow file:
      
      * Get rid of the custom docker image with preinstalled Coverity toolset.
      * Use a nice unofficial-coverity-scan GitHub Action ([1]).
      * Add the `libreadline-dev` dependency installation, because it is
        needed to build tarantool on Ubuntu 24.04.
      * Drop related `.test.mk` rules, because it looks more readable to
        invoke a few commands from the workflow file directly.
      * Drop testing artifacts uploading that seems a copy-paste from some
        workflow that runs the tests and the given directory unlikely has any
        file in our case.
      * Drop unused step that adds a comment to the pull request.
      
      And things seems to start working. At least, after a testing run of the
      workflow now I see the following status on the website:
      
      > Last Build Status: Running. Your build is currently being analyzed
      
      [1]: https://github.com/marketplace/actions/unofficial-coverity-scan
      
      See also #10651.
      
      NO_DOC=developer tools
      NO_CHANGELOG=see NO_DOC
      NO_TEST=see NO_DOC
      
      (cherry picked from commit f5daacfac84fbea3bb67991fa71ea4e789184ec8)
      fc613299
    • Alexander Turenko's avatar
      build: don't depend on libidn2 · 88220b45
      Alexander Turenko authored
      The recent libcurl update in commit 0919f390802f ("third_party: update
      libcurl from 8.8.0 to 8.10.1") reveals an old problem in our libcurl's
      build script: it attempts to disable the libidn2 dependency using the
      `-DHAVE_LIBIDN2=OFF` option instead of `-DUSE_LIBIDN2=OFF`.
      
      It doesn't work after a recent curl building machinery change, see
      https://github.com/curl/curl/pull/14555
      
      This commit fixes the problem by using `-DUSE_LIBIDN2=OFF`.
      
      Reported in #10621
      
      NO_DOC=build mechanics fixup
      NO_CHANGELOG=fixes a non-released commit
      NO_TEST=verified locally, seems to need an extra work on a separate CI
              workflow with a particular building environment; no time for
              that tonight, sorry
      
      (cherry picked from commit a4b4158f2196442308aba7df8f8e21b98b5319fc)
      88220b45
    • Sergey Bronnikov's avatar
      test: add additional tests for a strptime() · 5f1e308b
      Sergey Bronnikov authored
      The testsuite "Datetime string parsing by format (detailed)"
      tests parsing of a string with various conversion specifications.
      However, `tostring(dt)` is used as a test oracle, and all these
      testcases do not test some conversion specifications at all
      because the metamethod `__tostring` for the datetime object uses
      `:format()` with the default format string.
      
      Due to missed tests for conversion specifications, the behavior
      described in #10470 was missed:
      
      ```
      tarantool> dt = date.parse('Mon', {format = '%a'})
      tarantool> dt
      ---
      - 1970-01-01T00:00:00Z
      ...
      
      tarantool> dt:format('%a')
      ---
      - Thu
      ...
      ```
      
      The patch adds tests for `strptime` with all possible conversion
      specifications described in strftime(3) [1][2].
      
      1. https://man.freebsd.org/cgi/man.cgi?strftime(3)
      2. https://pubs.opengroup.org/onlinepubs/009695399/functions/strptime.html
      
      Follows up #8588
      Follows up #6731
      Relates to #10470
      
      NO_CHANGELOG=testing
      NO_DOC=testing
      
      (cherry picked from commit 109b18df4da0b34f4408f67902e0db04540cb32b)
      5f1e308b
  6. Oct 10, 2024
    • Col-Waltz's avatar
      third_party: update libcurl from 8.8.0 to 8.10.1 · a94c42df
      Col-Waltz authored
      Updates curl module to the version 8.10.1. The new version brings
      several new options, such as:
      
      CURL_USE_RUSTLS - Enables Rustls for SSL/TLS. Added in commit
      curl/curl@ed76a23fccc1 ("cmake: add rustls")
      
      CURL_USE_WOLFSSH - Option to use wolfSSH. Added in commit
      curl/curl@0d8fdd1c7421 ("cmake: add wolfSSH support")
      
      CURL_USE_GSASL - Option to use libgsasl.
      Added in commit curl/curl@66bf995d1cfc
      ("cmake: add CURL_USE_GSASL option with detection + CI test")
      
      CURL_DISABLE_SHA512_256 - Disables SHA-512/256 hash algorithm.
      Added in commit curl/curl@33629949488c
      ("build: add options to disable SHA-512/256 hash algo")
      
      CURL_USE_LIBUV - Use libuv for event-based tests. Added in commit
      curl/curl@f7d5f47059c3 ("cmake: add support for `CURL_USE_LIBUV` option")
      
      Corrected http_client test according to curl commit curl/curl@b7e769dc872d
      ("vtls: stop offering alpn http/1.1 for http2-prior-knowledge")
      
      Build file missed several options and paths not used in the current build.
      List of these options was added into curl-excluded-options.txt for the
      convenience of the following bumps.
      
      Closes #10576
      
      https://curl.se/changes.html#8_10_1
      https://github.com/curl/curl/releases/tag/curl-8_10_1
      
      NO_DOC=libcurl submodule bump
      NO_TEST=libcurl submodule bump
      
      (cherry picked from commit 0919f390802f146852b462215327ef03e2730cfc)
      
      @Totktonada: the http_client test mentioned in the original commit
      message doesn't present in the `release/2.11` branch.
      a94c42df
    • Col-Waltz's avatar
      build: added curl options missed by previous bumps · 3b9952fb
      Col-Waltz authored
      Some curl options appear to missed by several previous curl bumps.
      Here is a list of missed options with curl commits in which
      they first appeared:
      
      BUILD_EXAMPLES - Build libcurl examples.
      Added in curl 8.8.0 release in commit curl/curl@dfdd978f7c60
      ("cmake: add `BUILD_EXAMPLES` option to build examples")
      
      USE_ECH - Enables ECH support.
      Added in curl 8.8.0 release in commit curl/curl@a362962b7289
      ("TLS: add support for ECH (Encrypted Client Hello)")
      
      USE_HTTPSRR - Enables HTTPS RR support for ECH.
      Added in curl 8.8.0 release in commit curl/curl@a362962b7289
      ("TLS: add support for ECH (Encrypted Client Hello)")
      
      BUILD_STATIC_CURL - Builds curl executable with static libcurl.
      Added in curl 8.3.0 release in commit curl/curl@1199308dbc90
      ("cmake: support building static and shared libcurl in one go")
      
      CURL_DISABLE_NEGOTIATE_AUTH - Disables negotiate authentication.
      Added in curl 8.3.0 release in commit curl/curl@e92edfbef644
      ("lib: add ability to disable auths individually")
      
      CURL_DISABLE_SRP - Disables TLS-SRP support.
      Added in curl 8.4.0 release in commit curl/curl@781242ffa44a
      ("cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS")
      
      NO_TEST=does not change tarantool behavior
      NO_DOC=does not change tarantool behavior
      
      (cherry picked from commit 97e3136ddc691f42cf0a5dff27881978b3c25d52)
      3b9952fb
  7. Oct 08, 2024
    • Sergey Bronnikov's avatar
      test: replace testcase by luatest's testcases · 24016012
      Sergey Bronnikov authored
      The tests `array.test.lua` and `map.test.lua` require the remote
      Tarantool instance for running testcases. When tests are executed
      by test-run.py, it runs this Tarantool instance. The testcases
      that require the remote instance were ported to luatest to make
      tests able to run with ctest. The testcases `builtins-13.1` and
      `builtins-14.1` are companion testcases and were moved as well.
      
      Required by patches for CTest support, because allows to
      execute aforementioned tests without test-run.py.
      
      NO_CHANGELOG=codehealth
      NO_DOC=codehealth
      NO_TEST=codehealth
      
      (cherry picked from commit 47295fdebc1910b990c391c26fe92918a9c0c001)
      24016012
    • Sergey Bronnikov's avatar
      test: fix minimal.test.lua · 4db968c2
      Sergey Bronnikov authored
      The patch set a path to `tarantool` executable binary explicitly
      in the aforementioned test.
      
      NO_CHANGELOG=codehealth
      NO_DOC=codehealth
      
      (cherry picked from commit 7844e6034bd67f3e5a57710443373da5200e441a)
      4db968c2
    • Sergey Bronnikov's avatar
      test: use built tarantool instead a system one · 62894f29
      Sergey Bronnikov authored
      The test gh_5747_crash_multiple_args_test.lua uses a `tarantool`
      binary that is available in a current PATH environment variable.
      In some cases, PATH may point to a system `tarantool` executable
      and this could be unexpected for those who run the test. The patch
      set a path to `tarantool` executable derived from a command-line
      used to run the test.
      
      NO_CHANGELOG=codehealth
      NO_DOC=codehealth
      
      (cherry picked from commit 1e907ddbd8db2587418d50dde3e06e985c47bded)
      62894f29
    • Sergey Bronnikov's avatar
      test: fix test filename · 41bbe328
      Sergey Bronnikov authored
      According to CMake policy CMP0110 whitespaces and other special
      characters are forbidden before CMake 3.19. This policy is
      triggered with the patch for CTest support and breaks test
      generation:
      
      NO_WRAP
      The following name given to add_test() is invalid if CMP0110 is not set or
      set to OLD:
      
        `test/box-luatest/gh_7217_repeatable_{in, up}sert_memtx_tx_conflict_test.lua´
      NO_WRAP
      
      The patch renames file without using whitespaces and special
      characters.
      
      Required for CTest support. Follows up commit 654cf498
      ("memtx: fix story delete statement list").
      
      1. https://cmake.org/cmake/help/latest/policy/CMP0110.html
      
      NO_CHANGELOG=codehealth
      NO_DOC=codehealth
      NO_TEST=codehealth
      
      (cherry picked from commit ec810b9d6b618c4ec5116dde2f02b5be01a13f6d)
      41bbe328
    • Sergey Bronnikov's avatar
      test: remove feedback_daemon.skipcond · 18fee9d3
      Sergey Bronnikov authored
      The aforementioned skipcond file is specific for Travis CI
      because it depends on the environment variable `TRAVIS_JOB_ID`.
      We have moved to GHA for a long time, it seems this skipcond is
      not useful anymore.
      
      NO_CHANGELOG=codehealth
      NO_DOC=codehealth
      NO_TEST=codehealth
      
      (cherry picked from commit 6d9620cd6d485f826acc2b54fb4ac7f971b56cde)
      18fee9d3
    • Sergey Bronnikov's avatar
      test: remove test in suite.ini · b85bd2d0
      Sergey Bronnikov authored
      The test has been renamed in commit 16d6e9d2
      ("console: remove ERRINJ_STDIN_ISATTY injection") and it is not
      used the error injection anymore, so it was removed in suite.ini.
      
      NO_CHANGELOG=codehealth
      NO_DOC=codehealth
      NO_TEST=codehealth
      
      (cherry picked from commit b220f050261d4c4ba0ec679c8d77a5e817367a3f)
      b85bd2d0
    • Vladimir Davydov's avatar
      box: log error that caused initial checkpoint failure · 607ff4cd
      Vladimir Davydov authored
      Currently, we just panic without providing any additional information
      if we failed to create the initial checkpoint on bootstrap. This
      complicates trouble shooting. Let's replace `panic()` with `say_error()`
      and raise the exception that caused the failure. The exception will be
      caught by `box_cfg()`, which will log it and then panic.
      
      NO_DOC=error logging
      NO_TEST=error logging
      NO_CHANGELOG=error logging
      
      (cherry picked from commit e1b5114d99ed2f224e9e9a17bf29882e50be3653)
      607ff4cd
  8. Oct 07, 2024
    • Nikita Zheleztsov's avatar
      upgrade: introduce 2.11.5 schema version · 46cac24c
      Nikita Zheleztsov authored
      We decided to introduce new schema version, which does nothing in order
      to distinguish, which 2.11 schema we can safely use to allow persistent
      names on it.
      
      Follow up #10549
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      NO_TEST=nothing to test
      46cac24c
    • Vladislav Shpilevoy's avatar
      schema: allow _cluster update after join · 2be2e75c
      Vladislav Shpilevoy authored
      The function replica_check_id() is called on any change in
      _cluster: insert, delete, update. It was supposed to check if the
      replica ID is valid - not nil, not out of range (VCLOCK_MAX).
      
      But it was also raising an error when the ID matched this
      instance's ID unless the instance was joining. That happened even
      if a _cluster tuple was updated without changing the ID at all.
      For example, if one would just do
      _cluster:replace(_cluster:get(box.info.id)).
      
      Better do the check in the only place where the mutation can
      happen - on deletion. Since replica ID is a primary key in
      _cluster, it can't be updated there. Only inserted or deleted.
      
      This commit is backported to 2.11, since we want to allow using
      persistent names as early as we can in order to simplify the upgrade
      process. We also bump the schema version in the following commit in
      order to distinguish this version from overs 2.11.X, where persistent
      names doesn't work.
      
      Closes #10549
      
      NO_DOC=bugfix and refactoring
      NO_CHANGELOG=cannot happen without touching system spaces
      NO_TEST=too insignificant for an own test
      
      (cherry picked from commit cb8f4715)
      2be2e75c
    • Sergey Bronnikov's avatar
      httpc: replace ibuf_alloc with xibuf_alloc · 5bdda673
      Sergey Bronnikov authored
      There is no check for NULL for a value returned by `ibuf_alloc`,
      the NULL will be passed to `memcpy()` if the aforementioned
      function will return a NULL. The patch fixes that by replacing
      `ibuf_alloc` with macros `xibuf_alloc` that never return NULL.
      
      Found by Svace.
      
      NO_CHANGELOG=codehealth
      NO_DOC=codehealth
      NO_TEST=codehealth
      
      (cherry picked from commit b4ee146fde6e418aed590ac6054cff75c2a59626)
      5bdda673
    • Astronomax's avatar
      limbo: speed up synchronous transaction queue processing · d615f3f7
      Astronomax authored
      This patch optimizes the process of collecting ACKs from replicas for
      synchronous transactions. Before this patch, collecting confirmations
      was slow in some cases. There was a possible situation where it was
      necessary to go through the entire limbo again every time the next ACK
      was received from the replica. This was especially noticeable in the
      case of a large number of parallel synchronous requests.
      For example, in the 1mops_write bench with parameters --fibers=6000
      --ops=1000000 --transaction=1, performance increases by 13-18 times on
      small clusters of 2-4 nodes and 2 times on large clusters of 31 nodes.
      
      Closes #9917
      
      NO_DOC=performance improvement
      NO_TEST=performance improvement
      
      (cherry picked from commit 4a866f64d64c610a3c8441835fee3d8dda5eca71)
      d615f3f7
    • Astronomax's avatar
      vclock: introduce `vclock_nth_element` and `vclock_count_ge` · c2c87816
      Astronomax authored
      Two new vclock methods have been added: `vclock_nth_element` and
      `vclock_count_ge`.
      * `vclock_nth_element` takes n and returns whatever element would occur in
      nth position if vclock were sorted. This method is very useful for
      synchronous replication because it can be used to find out the lsn of the
      last confirmed transaction - it's simply the result of calling this
      method with argument {vclock_size - replication_synchro_quorum} (provided
      that vclock_size >= replication synchro quorum, otherwise it is obvious
      that no transaction has yet been confirmed).
      * `vclock_count_ge` takes lsn and returns the number of components whose
      value is greater than or equal to lsn. This can be useful to understand
      how many replicas have already received a transaction with a given lsn.
      
      Part of #9917
      
      NO_CHANGELOG=Will be added in another commit
      NO_DOC=internal
      
      (cherry picked from commit 58f3c93b660499e85f08a4f63373040bcae28732)
      c2c87816
  9. Oct 04, 2024
    • Andrey Saranchin's avatar
      memtx: do not pass NULL to memcpy when creating gap item in MVCC · e92f7806
      Andrey Saranchin authored
      According to the C standard, passing `NULL` to `memcpy` is UB, even if
      it copies nothing (number of bytes to copy is 0). The commit fixes such
      situation in memtx MVCC.
      
      Closes tarantool/security#129
      
      NO_TEST=fix UB
      NO_CHANGELOG=fix UB
      NO_DOC=fix UB
      
      (cherry picked from commit 24d38cef5adff900bea2484235762678ac1c5234)
      e92f7806
  10. Oct 02, 2024
    • Vladimir Davydov's avatar
      test: disable fiber slice check in replica_apply_order test · a800ccc6
      Vladimir Davydov authored
      Since commit e19bca5a74e8 ("box: check fiber slice in generic
      implementation of index count"), Vinyl's version of `index.count()`
      checks the fiber slice. As a result, the test may fail if it runs
      under a heavy load:
      
      ```
      | @@ -94,6 +94,7 @@
      |      end
      |  end;
      |   | ---
      | + | - error: fiber slice is exceeded
      |   | ...
      |  -- Verify that at any moment max index is corresponding to amount of tuples,
      |  -- which means that changes apply order is correct
      ```
      
      Let's set the max fiber slice to a big value to avoid that.
      
      NO_DOC=test fix
      NO_CHANGELOG=test fix
      
      (cherry picked from commit b5fb66437a22ea65ed27c2ed14636e8036b079d3)
      a800ccc6
  11. Sep 30, 2024
    • Alexander Turenko's avatar
      ci: fix module API publish job (again) · ca1e0dee
      Alexander Turenko authored
      The new version of the publishing action leads to the following error:
      
      NOWRAP
      ```
      Error: File not found: '/home/runner/work/_actions/JamesIves/github-pages-deploy-action/v4.6.6/lib/main.js'
      ```
      NOWRAP
      
      Let's revert it to v4.6.4 (because v4.6.5 change was reverted in v4.6.6).
      
      See also https://github.com/JamesIves/github-pages-deploy-action/issues/1697
      
      NO_DOC=no code changes
      NO_CHANGELOG=see NO_DOC
      NO_TEST=see NO_DOC
      
      (cherry picked from commit 7cf41a1bb9b3f0a2eac82c296a764688f5d8b35c)
      ca1e0dee
    • Alexander Turenko's avatar
      ci: fix module API build/publish job · 57816db8
      Alexander Turenko authored
      The `ubuntu-latest` image is now `ubuntu-24.04`, see [1]. The job fails
      on this image with the following error:
      
      NOWRAP
      ```
      CMake Error at /usr/local/share/cmake-3.30/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
        Could NOT find Readline (missing: READLINE_INCLUDE_DIR READLINE_LIBRARY)
      ```
      NOWRAP
      
      It seems, the libreadline-dev package is missing. Let's install it.
      
      Also, update a version of the publishing action to the latest at the
      moment. I didn't perform any check, but I guess that a new version of
      NodeJS is needed and the latest action version has better support of it.
      
      [1]: https://github.blog/changelog/2024-09-25-actions-new-images-and-ubuntu-latest-changes/
      
      NO_DOC=no code changes
      NO_CHANGELOG=see NO_DOC
      NO_TEST=see NO_DOC
      
      (cherry picked from commit 3e09e9f293925195ddebb13a4fcfd64762e52b89)
      57816db8
  12. Sep 25, 2024
    • Vladimir Davydov's avatar
      vinyl: fix crash when empty PK DDL races with DML · b4304df7
      Vladimir Davydov authored
      Vinyl doesn't support altering the primary index of a non-empty space,
      but the check forbidding this isn't entirely reliable - the DDL function
      may yield to wait for pending WAL writes to finish after ensuring that
      the space doesn't contain any tuples. If a new tuples is inserted into
      the space in the meantime, the DDL operation will proceed rebuilding
      the primary index and trigger a crash because the code is written on
      the assumption that it's rebuilding a secondary index:
      
      ```
      ./src/box/vinyl.c:1572: vy_check_is_unique_secondary_one: Assertion `lsm->index_id > 0' failed.
      ```
      
      Let's fix this by moving the check after syncing on WAL.
      
      Closes #10603
      
      NO_DOC=bug fix
      
      (cherry picked from commit 955537b57c2aade58b7ca42501a9bbe50dd91f26)
      b4304df7
  13. Sep 24, 2024
    • Col-Waltz's avatar
      build: curl option BUILD_MISC_DOCS set OFF · 41bcff0e
      Col-Waltz authored
      Curl option BUILD_MISC_DOCS builds misc man pages and set ON by default.
      Other documentation building options such as ENABLE_CURL_MANUAL and
      BUILD_LIBCURL_DOCS was set OFF in BuildLibCurl.cmake.
      I suppose this option has to be added in commit 7192bf66
      ("third_party: update libcurl from 8.7.0 to 8.8.0+patches") and set OFF.
      
      Follows up #9885
      
      NO_TEST=does not change tarantool behavior
      NO_DOC=does not change tarantool behavior
      
      (cherry picked from commit 1080995fa4083c4de6aa19e964b98f0ffb7e34c5)
      41bcff0e
  14. Sep 23, 2024
    • Vladimir Davydov's avatar
      box: check fiber slice in generic implementation of index count · 55fffaed
      Vladimir Davydov authored
      `index.count()` may hang for too long in Vinyl if a substantial
      consecutive hunk of the space is stored in memory. Let's add
      a fiber slice check to it to prevent it from blocking the TX thread
      for too long.
      
      Closes #10553
      
      NO_DOC=bug fix
      
      (cherry picked from commit e19bca5a74e83d2521fe770f2a93c3e3d3ad4801)
      55fffaed
    • Vladimir Davydov's avatar
      vinyl: fix cache corruption on skipping unconfirmed tuple · eef3d7d2
      Vladimir Davydov authored
      The tuple cache doesn't store historical data. It stores only the newest
      tuple versions, including prepared but not yet confirmed (committed but
      not written to WAL) tuples. This means that transactions sent to a read
      view shouldn't add any new chains to the cache because such a chain may
      bypass a tuple invisible from the read view.
      
      A transaction may be sent to a read view in two cases:
      
       1. If some other transactions updates data read by it.
       2. If the transaction is operating in the 'read-confirmed' isolation
          mode and skips an unconfirmed tuple while scanning the memory level.
          This was added in commit 588170a7 ("vinyl: implement transaction
          isolation levels").
      
      The second point should be checked by the read iterator itself, and it
      is indeed for the standard case when we scan the memory level before
      reading the disk. However, there's the second case: if some other tuples
      are inserted into the memory level while the read iterator was waiting
      for a disk read to complete, it rescans the memory level and may skip
      a new unconfirmed tuple that wasn't there the first time we scanned
      the memory level. Currently, if this happens, it won't send itself to
      a read view and may corrupt the cache by inserting a chain that skips
      over the unconfirmed tuple. Fix this by adding the missing check.
      
      While we are at it, let's simplify the code a bit by moving the check
      inside `vy_read_iterator_scan_mem()`. It's okay because sending to
      a read view a transaction that's already in the read view is handled
      correctly by `vy_tx_send_to_read_view()`.
      
      Closes #10558
      
      NO_DOC=bug fix
      
      (cherry picked from commit a3feee322e76a1e10ab874e63f17f97b6457b59d)
      eef3d7d2
  15. Sep 20, 2024
    • Vladimir Davydov's avatar
      vinyl: fix compaction crash on disk read error · 45138738
      Vladimir Davydov authored
      `vy_slice_stream_next()` clears the return value on failure. This isn't
      expected by `vy_write_iterator_merge_step()`, which doesn't update
      the source position in the `vy_wirte_iterator::src_heap` in this case.
      As a result, an attempt to remove `end_of_key_src` from the heap in
      `vy_write_iterator_build_history()` may crash as follows:
      
      ```
       # 1  0x572a2ecc21a6 in crash_collect+256
       # 2  0x572a2ecc2be2 in crash_signal_cb+100
       # 3  0x7cfef6645320 in __sigaction+80
       # 4  0x572a2eab16de in tuple_format+16
       # 5  0x572a2eab1a25 in vy_stmt_is_key+24
       # 6  0x572a2eab1be8 in vy_stmt_compare+89
       # 7  0x572a2eab1e37 in vy_entry_compare+74
       # 8  0x572a2eab2913 in heap_less+88
       # 9  0x572a2eab21e3 in vy_source_heap_sift_up+255
       # 10 0x572a2eab20b9 in vy_source_heap_update_node+54
       # 11 0x572a2eab25c1 in vy_source_heap_delete+249
       # 12 0x572a2eab4134 in vy_write_iterator_build_history+1497
       # 13 0x572a2eab4995 in vy_write_iterator_build_read_views+193
       # 14 0x572a2eab4ce6 in vy_write_iterator_next+380
       # 15 0x572a2eadd20b in vy_task_write_run+1132
       # 16 0x572a2eade6cf in vy_task_compaction_execute+124
       # 17 0x572a2eadfa8d in vy_task_f+445
       # 18 0x572a2e9ea143 in fiber_cxx_invoke(int (*)(__va_list_tag*), __va_list_tag*)+34
       # 19 0x572a2eccee7c in fiber_loop+219
       # 20 0x572a2f0aef18 in coro_init+120
      ```
      
      Normally, a function shouldn't update the return value on failure so
      let's fix `vy_slice_stream_next()`.
      
      Closes #10555
      
      NO_DOC=bug fix
      
      (cherry picked from commit f1144c533b6c52c324ffe1cc4fcaeab1f2f6cd9f)
      45138738
    • Vladimir Davydov's avatar
      vinyl: use ERROR_INJECT_COUNTDOWN where appropriate · 69ea9ba1
      Vladimir Davydov authored
      ERRINJ_VY_RUN_OPEN and ERRINJ_VY_STMT_ALLOC are countdown injections.
      Let's name them appropriately and use the helper macro. Also, let's
      raise the ER_INJECTION error code for them to make it clear that they
      aren't real errors.
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      
      (cherry picked from commit 21fe14582c948f560720fa285ed3e21483d11dc2)
      69ea9ba1
    • Vladimir Davydov's avatar
      errinj: fix ERROR_INJECT_COUNTDOWN · 70a3a976
      Vladimir Davydov authored
      We shouldn't decrement the counter if it's negative - otherwise it may
      wrap around and mistakenly trigger the error injection.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      
      (cherry picked from commit d11d4576b0d0cbfc03dc1a3570573b7bbf1126b5)
      70a3a976
  16. Sep 18, 2024
    • Sergey Bronnikov's avatar
      datetime: introduce tz in datetime.parse() · c42b850d
      Sergey Bronnikov authored
      There is an option tz in `datetime.parse()`, it was added in
      commit 3c403661 ("datetime, lua: date parsing functions").
      The option is not documented, and the commit message says that
      option `tz` is "Not yet implemented in this commit.".
      
      The patch added tests and a doc request for this option.
      The behaviour of the option `tz` is the same as with option
      `tzoffset`:
      - if timezone was not set in a parsed string then it is set to
        a value specified by `tz`
      - if timezone was set in a parsed string then option `tz` is
        ignored
      
      ```
      tarantool> date.parse("1970-01-01T01:00:00 MSK", { tz = 'Europe/Paris' })
      ---
      - 1970-01-01T01:00:00 MSK
      - 23
      ...
      
      tarantool> date.parse("1970-01-01T01:00:00", { tz = 'Europe/Paris' })
      ---
      - 1970-01-01T01:00:00 Europe/Paris
      - 19
      ...
      ```
      
      Follows up #6731
      Fixes #10420
      
      @TarantoolBot document
      Title: Introduce option `tz` in `datetime.parse()`
      
      The option `tz` is added in a function `datetime.parse()`.
      The option set timezone to a passed value if it was not set in
      a parsed string.
      
      (cherry picked from commit c6bab23a6dc4f819167cbc78eb93859847a389ea)
      c42b850d
    • Sergey Bronnikov's avatar
      datetime: use tzoffset in a parse() with custom format · 45d40d13
      Sergey Bronnikov authored
      The patch fixes a behaviour, when `datetime.parse()` ignores
      `tzoffset` option if custom format is used.
      
      Fixes #8333
      Relates to #10420
      
      NO_DOC=bugfix
      
      (cherry picked from commit 04811e032f29afe0fa6206ef2c7a0f8434861830)
      45d40d13
Loading