Skip to content
Snippets Groups Projects
  1. Aug 08, 2023
    • Sergey Vorontsov's avatar
      build: change BACKUP_STORAGE URL for static build · 86bc6b2f
      Sergey Vorontsov authored
      NO_DOC=build
      NO_TEST=build
      NO_CHANGELOG=build
      
      (cherry picked from commit bb74d6c9)
      86bc6b2f
    • Kirill Yukhin's avatar
      Add owners for /.test.mk and /.github · 31a74698
      Kirill Yukhin authored
      Add code owners for CI-related script and
      for github automation directory.
      
      NO_CHANGELOG=no code changes
      NO_TEST=no code changes
      NO_DOC=no code changes
      
      (cherry picked from commit 9234763a)
      31a74698
    • Kirill Yukhin's avatar
      Add owners for test/ and test-run/ · 32ff5a73
      Kirill Yukhin authored
      In order to improve tests of Tarantool core
      assign dedicated team to perform review of each
      and every change.
      
      NO_CHANGELOG=no code changes
      NO_TEST=no code changes
      NO_DOC=no code changes
      
      (cherry picked from commit 532bada7)
      32ff5a73
    • Kirill Yukhin's avatar
      Add CODEOWNERS file · 0beee753
      Kirill Yukhin authored
      To make changelog preparation less stressful let's pass each and every
      change to changelogs/ directly through the doc team.
      
      NO_CHANGELOG=no code changes
      NO_TEST=no code changes
      NO_DOC=no code changes
      
      (cherry picked from commit 2970bd57)
      0beee753
    • Aleksandr Lyapunov's avatar
      box: forbid foreign keys for incompatible temp/local spaces · 23cea1df
      Aleksandr Lyapunov authored
      There must be a couple of rules:
      * foreign key from non-temporary space to temporary space must be
        forbidden since after restart all existing links will be broken.
      * foreign key from non-local space to local space must be forbidden
        on any replica all existing can be broken.
      
      This patch implements the rules.
      
      Closes #8936
      
      NO_DOC=bugfix
      
      (cherry picked from commit 7d23b339)
      23cea1df
  2. Aug 07, 2023
    • Aleksandr Lyapunov's avatar
      box: loose truncate check in case of foreign key · fec86b65
      Aleksandr Lyapunov authored
      In #7309 a truncation of a space that was referenced by foreign
      key from some other space was prohibited.
      
      It appeared that this solution is too bothering since a user can't
      truncate a space even if he truncated referring space before that.
      
      Fix it by allowing space truncate if referring spaces are empty.
      Also allow drop of the primary index in the same case with the
      same reason: logically the index along with all space data is
      not needed for consistency if there's no referring data.
      
      Note that by design space truncate is implemented quite similar
      to space drop. Both delete all indexes, from secondary to primary.
      Since this patch allows deletion of the primary index (which is
      the action that actually deletes all data from the space), this
      patch changes the result of space drop too: the space remains
      alive with no indexes, while before this patch it remained alive
      with no secondary indexes but with present primary. In both cases
      the behaviour is quite strange and must be fixed in #4348. To
      make tests pass I had to perform drop in box.atomic manually.
      
      Closes #8946
      
      NO_DOC=bugfix
      
      (cherry picked from commit 983a7ec2)
      fec86b65
    • Vladimir Davydov's avatar
      test: increase max fiber slice for box/before_replace · a7dbc83a
      Vladimir Davydov authored
      The test fails on osx_debug quite frequently with the following error:
      
      NO_WRAP
      > box/before_replace.test.lua                                     [ fail ]
      >
      > Test failed! Result content mismatch:
      > --- box/before_replace.result	Mon Aug  7 10:36:06 2023
      > +++ /tmp/t/rejects/box/before_replace.reject	Mon Aug  7 10:42:03 2023
      > @@ -899,6 +899,7 @@
      >  ...
      >  for i = 1,17 do gen_inserts() end
      >  ---
      > +- error: fiber slice is exceeded
      >  ...
      >  test_run:cmd('restart server test')
      >  s = box.space.test_on_schema_init
      > @@ -906,7 +907,7 @@
      >  ...
      >  s:count()
      >  ---
      > -- 1
      > +- 0
      >  ...
      >  -- For this test the number of invocations of the before_replace
      >  -- trigger during recovery multiplied by the amount of return values
      > @@ -921,7 +922,8 @@
      >  -- the value is only increased by 1, and this change is visible only in memory.
      >  s:get{1}[2] == 1 + 16999 + 17000 + 1 or s:get{1}[2]
      >  ---
      > -- true
      > +- error: '[string "return s:get{1}[2] == 1 + 16999 + 17000 + 1 o..."]:1: attempt to
      > +    index a nil value'
      >  ...
      >  test_run:cmd('switch default')
      >  ---
      NO_WRAP
      
      Let's try to increase the max fiber slice up to 15 seconds to avoid
      that, like we do in other potentially long tests. (The default value
      is 1 second.)
      
      NO_DOC=test fix
      NO_CHANGELOG=test fix
      
      (cherry picked from commit 371d4110)
      a7dbc83a
    • Vladimir Davydov's avatar
      memtx: assert that space is not NULL in index_read_view_create · d4213e06
      Vladimir Davydov authored
      This should suppress the following coverity issues:
      
      https://scan7.scan.coverity.com/reports.htm#v39198/p13437/fileInstanceId=146712118&defectInstanceId=18978766&mergedDefectId=1563095
      https://scan7.scan.coverity.com/reports.htm#v39198/p13437/fileInstanceId=146712113&defectInstanceId=18978750&mergedDefectId=1563094
      
      While we are at it, let's use space_by_id instead of space_cache_find
      because read view creation is a rare operation affecting all spaces so
      caching the last space by id doesn't make any sense.
      
      NO_DOC=code health
      NO_TEST=code health
      NO_CHANGELOG=code health
      
      (cherry picked from commit f2886dd0)
      d4213e06
  3. Aug 04, 2023
    • Igor Munkin's avatar
      asan: enable ASan and LSan support for LuaJIT back · e987aedb
      Igor Munkin authored
      All LuaJIT related LSan warnings were suppressed in the scope of the
      commit 985548e4 ("asan: suppress all
      LSAN warnings related to LuaJIT"), since all compiler flags tweaks were
      enclosed in LuaJIT CMake machinery. As a result of the commit in LuaJIT
      submodule tarantool/luajit@a86e376 ("build: introduce LUAJIT_USE_ASAN
      option") ASan and LSan support has been finally added to LuaJIT runtime,
      so it was decided to remove LSan suppressions for LuaJIT functions.
      Unfortunately, it was not so easy as it looked like.
      
      At first, Lua global state is not closed properly at Tarantool instance
      exit (see <tarantool_free> in src/main.cc and <tarantool_lua_free> in
      src/lua/init.c for more info), so LSan false-positive leaks are detected
      (for more info, see #3071). Hence, the original LSan suppression for
      lj_BC_FUNCC is returned back (temporarily) until the aforementioned
      issue is not resolved.
      
      Furthermore, the internal LuaJIT memory allocator is not instrumented
      yet, so to find any memory faults it's worth building LuaJIT with system
      provided memory allocator (i.e. enable LUAJIT_USE_SYSMALLOC option).
      However, again, since Tarantool doesn't finalize Lua universe the right
      way, so running Tarantool testing routine with LUAJIT_USE_SYSMALLOC
      enabled generates false-positive LSan leaks. Return back here to enable
      LUAJIT_USE_SYSMALLOC, when #3071 is resolved.
      
      Last but not least, the default value of fiber stack size is 512Kb, but
      several tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT submodule
      (e.g. some cases with deep recursion in errors.lua or pm.lua) have
      already been tweaked according to the limitations mentioned in #5782,
      but the crashes still occur while running LuaJIT tests with ASan support
      enabled. Experiments once again confirm the notorious quote that "640 Kb
      ought to be enough for anybody".
      
      Anyway, LuaJIT tests are added to <test-release-asan> target in .test.mk
      and LUAJIT_TEST_ENV is extended with required ASan and LSan options.
      
      Follows up #5878
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      
      (cherry picked from commit bacf4e56)
      Unverified
      e987aedb
  4. Aug 02, 2023
    • Igor Munkin's avatar
      luajit: bump new version · cddf86ba
      Igor Munkin authored
      * ci: introduce testing workflow with sanitizers
      * build: introduce LUAJIT_USE_ASAN option
      * test: introduce test:done TAP helper
      * memprof: remove invalid assertions
      * ci: clean up workflow for exotic builds
      
      Closes #5878
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      NO_CHANGELOG=LuaJIT submodule bump
      cddf86ba
    • Vladimir Davydov's avatar
      compat: move is_new and is_old option methods to metatable · 14e4dfc0
      Vladimir Davydov authored
      The is_new and is_old methods are the same for all compat options so
      they should be defined in a metatable. A good thing about this change
      is that it removes is_new and is_old from serialization:
      
      * Before:
      
      NO_WRAP
        tarantool> require('compat').yaml_pretty_multiline
        ---
        - is_new: 'function: 0x4175d6e8'
          is_old: 'function: 0x4175d790'
          brief: |
            Whether to encode in block scalar style all multiline strings or ones
            containing "\n\n" substring. The new behavior makes all multiline string output
            as single text block which is handier for the reader, but may be incompatible
            with some existing applications that rely on the old style.
      
            https://tarantool.io/compat/yaml_pretty_multiline
          current: default
          default: new
        ...
      NO_WRAP
      
      * After:
      
      NO_WRAP
        tarantool> require('compat').yaml_pretty_multiline
        ---
        - current: default
          brief: |
            Whether to encode in block scalar style all multiline strings or ones
            containing "\n\n" substring. The new behavior makes all multiline string output
            as single text block which is handier for the reader, but may be incompatible
            with some existing applications that rely on the old style.
      
            https://tarantool.io/compat/yaml_pretty_multiline
          default: new
        ...
      NO_WRAP
      
      To achieve that, we have to remove the option name from the usage error
      message but it seems to be okay because such errors shouldn't happen in
      practice and the error message is clear enough to figure out what went
      wrong.
      
      Follow-up #8807
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      
      (cherry picked from commit 75b5fd05)
      14e4dfc0
  5. Jul 27, 2023
    • Serge Petrenko's avatar
      applier: fix use after free · 0c4293d7
      Serge Petrenko authored
      Applier thread uses lsregion to allocate the messages for tx thread. The
      messages are freed upon return to the applier thread using a
      corresponding lsr_id.
      
      Due to a typo, one of the lsregion allocations was made with a postfix
      increment of lsr_id instead of the prefix one. Essentially, part of a
      new message was allocated with an old lsr_id, and might be freed early
      by a return of a previous message.
      
      Fix this.
      
      Closes #8848
      
      NO_DOC=bugfix
      NO_TEST=covered by asan in #8901
      NO_CHANGELOG=bugfix
      
      (cherry picked from commit 0d5bd6b7)
      0c4293d7
  6. Jul 26, 2023
    • Nikita Zheleztsov's avatar
      relay: fix unterminated final_join thread · c061893a
      Nikita Zheleztsov authored
      
      Currently if tarantool exits during relay's final join stage,
      corresponding thread isn't terminated. This causes the flakiness
      of the replicaset_ro_mostly.test.lua.
      
      Let's reuse the same relay, in which subscribe cord is running, for
      the final join stage. This way the cord will be cancelled during
      replication_free().
      
      Closes #8082
      
      NO_DOC=not user-visible
      NO_TEST=fix flaky test
      NO_CHANGELOG=not user-visible
      
      Co-authored-by: default avatarSergey Petrenko <sergepetrenko@tarantool.org>
      
      (cherry picked from commit 70a68836)
      c061893a
  7. Jul 25, 2023
    • Vladimir Davydov's avatar
      git: add diff=cpp attribute for C/C++ source files · e65d532d
      Vladimir Davydov authored
      This improves diff hunk name detection. Needed for checkpatch to
      correctly detect if patched code belongs to a function.
      
      NO_DOC=git
      NO_TEST=git
      NO_CHANGELOG=git
      
      (cherry picked from commit 642584fd)
      e65d532d
    • Mergen Imeev's avatar
      sql: fix wrong region allocation · ce06ab74
      Mergen Imeev authored
      This patch fixes an issue in generate_column_metadata(). Prior to this
      patch, the number of variable-only expressions was counted incorrectly
      when temporary memory was allocated on region to store their positions.
      However, although this allocation was incorrect, this did not lead to
      any problems due to the specifics of the region allocations.
      
      This patch fixes this by removing the temporary memory allocation.
      
      Closes #8763
      
      NO_DOC=no user-visible changes
      NO_TEST=no user-visible changes
      NO_CHANGELOG=no user-visible changes
      
      (cherry picked from commit d4f143ad)
      ce06ab74
  8. Jul 24, 2023
    • Georgy Moiseev's avatar
      msgpack: fix decoding intervals with int64 · f98b9771
      Georgy Moiseev authored
      It is possible for interval to have days, hours, minutes and seconds
      larger than INT_MAX (or less than INT_MIN). Before this patch, msgpack
      decoding had failed to parse intervals with msgpack int64 and uint64.
      int64_t should be enough to store any value allowed for datetime
      intervals.
      
      Closes #8887
      
      NO_DOC=small bug fix
      
      (cherry picked from commit 01c7ae11)
      Unverified
      f98b9771
  9. Jul 21, 2023
    • Sergey Kaplun's avatar
      lua: replace `api_check()` with `assert()` · cb131f6c
      Sergey Kaplun authored
      `api_check()` is the LuaJIT internal assertion. To prevent inconsistency
      during internal assertion changes (for example, during backporting), use
      glibc's `assert()` instead.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      
      (cherry picked from commit e78afb3b)
      Unverified
      cb131f6c
  10. Jul 20, 2023
    • Igor Munkin's avatar
      luajit: bump new version · 18b08eb8
      Igor Munkin authored
      * FFI: Fix recording of union initialization.
      * Fix maxslots when recording BC_VARG, part 2.
      * Fix maxslots when recording BC_VARG.
      * Fix BC_UCLO insertion for returns.
      * ci: update job concurrency group definition
      
      Part of #8825
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      18b08eb8
  11. Jul 19, 2023
    • Georgy Moiseev's avatar
      datetime: allow boundary values for interval.new · 230aba54
      Georgy Moiseev authored
      Before this patch, one couldn't create new datetime interval with
      boundary value from Lua. At the same time, it was possible to create
      such interval from Lua through addition and subtraction. C range
      verification allow to create boundary value intervals, error message
      also implies that they should be allowed. (See #8878 for more info.)
      
      Closes #8878
      
      NO_DOC=small bug fix
      
      (cherry picked from commit b2a001cc)
      Unverified
      230aba54
  12. Jul 18, 2023
    • Rimma Tolkacheva's avatar
      test/fuzz: refactor LuaJIT fuzzer · 8ebe4851
      Rimma Tolkacheva authored
      This refactoring will:
      
      1. Move macros from a header to the source file.
      Macros should be used in header only with undef to avoid redefinitions.
      Undef directive is not useful since we want to use these macros in the
      source file.
      
      2. Remove `using namespace lua_grammar` from header.
      https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-using-directive
      
      3. Moving serializer entry point and constant parameters into
      luajit_fuzzer namespace.
      It's a common practice in C++ to avoid name collisions.
      
      4. Move serializer functions into anonymous namespace.
      These functions are not a part of the interface so should have
      static linkage.
      https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-unnamed2
      
      5. Fix ConvertToStringDefault function.
      It was logically wrong so it would generate an identifier `123` from
      `*123`.
      
      NO_CHANGELOG=internal
      NO_DOC=fuzzer fix
      
      (cherry picked from commit 56488e15)
      8ebe4851
    • klauwier's avatar
      test/fuzz: fix luaJIT fuzzer timeout · 9c59bbc8
      klauwier authored
      LuaJIT fuzzer used to stop due to timeout caused by infinite cycles and
      recursions. Counters were introduced for every cycle and function to
      address LuaJIT fuzzer timeouts.
      
      The idea is to add unique counters for every cycle and function to
      ensure finite code execution, if it wasn't already. For while, repeat,
      for cycles, local and global named, anonymous functions, counters will
      be initialized before the code generated from protobuf, and checked
      in the first body statement. An entry point for the serializer was
      created to count cycles and functions for counter initialization.
      
      The idea was taken from a paper "Program Reconditioning: Avoiding
      Undefined Behaviour When Finding and Reducing Compiler Bugs" [1].
      
      Here is an example of a change in serialized code made by this commit.
      
      Before:
      ```lua
      while (true) do
          foo = 'bar'
      end
      function bar()
          bar()
      end
      ```
      
      After:
      ```lua
      counter_0 = 0
      counter_1 = 0
      while (true) do
          if counter_0 > 5 then
              break
          end
          counter_0 = counter_0 + 1
          foo = 'bar'
      end
      function bar()
          if counter_1 > 5 then
              return
          end
          counter_1 = counter_1 + 1
          bar()
      end
      ```
      Protobuf structures that reproduce the timeout problem were added to
      the LuaJIT fuzzer corpus.
      
      [1] https://www.doc.ic.ac.uk/~afd/homepages/papers/pdfs/2023/PLDI.pdf
      
      NO_CHANGELOG=internal
      NO_DOC=fuzzer fix
      
      (cherry picked from commit 4d004bbe)
      9c59bbc8
    • klauwier's avatar
      test/fuzz: add breaks to switch-case · 61f7892e
      klauwier authored
      Cases in two switches had no breaks, so they were falling
      through. Breaks were added to solve the problem. Code
      generated by the LuaJIT fuzzer became more various.
      
      NO_CHANGELOG=internal
      NO_DOC=fuzzer fix
      
      (cherry picked from commit 4430cac9)
      61f7892e
    • Sergey Bronnikov's avatar
      cmake: propagate CMAKE_BUILD_TYPE for ProtobufMutator · aa728c89
      Sergey Bronnikov authored
      Follows-up #4823
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      NO_TEST=internal
      
      (cherry picked from commit 95d62cfc)
      aa728c89
    • Sergey Bronnikov's avatar
      test/fuzz: fix datetime_strptime fuzzing test · 504a0f88
      Sergey Bronnikov authored
      Function `datetime_strptime` decodes string with datetime according to
      specified format, it accepts a datetime struct, buffer with datetime and
      string with format in arguments. Fuzzing test used static string
      "iso8601" as a format and it blocked fuzzing test to cover functions
      used by datetime_strptime under the hood. Fuzz introspector shows that
      code coveraged by a test is quite low.
      
      Patch updates the test to make it more effective: buffer with datetime
      and format string are generated using FDP (Fuzzing Data Provider).
      
      Test file extension was changed to .cc, because FuzzingDataProvider is
      used and we need building it by C++ compiler.
      
      Function `tnt_strptime` uses assert, that triggered by fuzzing tests.
      Therefore it was replaced with to if..then.
      
      1. https://storage.googleapis.com/oss-fuzz-introspector/tarantool/
      
      Fixes #8490
      
      NO_CHANGELOG=fuzzing test
      NO_DOC=fuzzing test
      NO_TEST=fuzzing test
      
      (cherry picked from commit a1bd6e0b)
      504a0f88
    • Timur Safin's avatar
      datetime: fix buffer overflow in tnt_strptime · 70b0fc1f
      Timur Safin authored
      Fixes #8502
      Needed for #8490
      
      NO_DOC=bugfix
      NO_TEST=covered by fuzzing test
      
      (cherry picked from commit 783a7040)
      70b0fc1f
    • Dmitriy Nesterov's avatar
      test/fuzz: add grammar-based LuaJIT fuzzer · 87f4c930
      Dmitriy Nesterov authored
      Patch adds a LuaJIT fuzzer based on libprotobuf-mutator and LibFuzzer.
      Grammar is described via messages in protobuf format, serializer is
      applied to convert .proto format to string.
      
      For displaying generated code on the screen during fuzzing set
      the environment variable 'LPM_DUMP_NATIVE_INPUT'.
      
      For displaying error messages from lua functions set
      the environment variable 'LUA_FUZZER_VERBOSE'.
      
      Note: UndefinedBehaviourSanitizer is unsupported by LuaJIT (see #8473),
      so fuzzing test is disabled when CMake option ENABLE_UB_SANITIZER is
      passed.
      
      Closes #4823
      
      NO_DOC=<fuzzing testing of LuaJIT>
      NO_TEST=<fuzzing testing of LuaJIT>
      
      (cherry picked from commit a287c853)
      87f4c930
    • Dmitriy Nesterov's avatar
      cmake: add dependencies for LuaJIT and SQL fuzzers · 70469594
      Dmitriy Nesterov authored
      Added Google's 'libprotobuf-mutator' and 'protobuf' libraries
      for developing grammar-based LuaJIT and SQL fuzzers based on
      LibFuzzer.
      
      It is needed to build protobuf module from source because
      by default, the system-installed version of protobuf is used
      by libprotobuf-mutator, and this version can be too old.
      
      Part of #4823
      
      NO_CHANGELOG=<dependencies>
      NO_DOC=<dependencies>
      NO_TEST=<dependencies>
      
      (cherry picked from commit b11072a6)
      70469594
    • Dmitriy Nesterov's avatar
      test/fuzz: add options for better fuzzing · 85496d4a
      Dmitriy Nesterov authored
      Added options for fuzzing and for getting more information
      on debugging.
      
      NO_CHANGELOG=<fuzzing options>
      NO_DOC=<fuzzing options>
      NO_TEST=<fuzzing options>
      
      (cherry picked from commit 69f21e25)
      85496d4a
  13. Jul 14, 2023
    • Vladimir Davydov's avatar
      box: allow to truncate temp and local spaces in ro mode · 2fc54ba1
      Vladimir Davydov authored
      To achieve that, we bypass the read-only check for the _truncate system
      space in box_process1() and perform it in the on_replace system trigger
      instead, when we know which space is truncated.
      
      Note, we have to move the check for insertion of a new record into the
      _truncate system space before the read-only check in the on_replace
      trigger callback; this is needed for initial recovery with a non-empty
      _truncate space to work. While we are at it, let's use recovery_state to
      make the check explicit.
      
      Closes #5616
      
      @TarantoolBot document
      Title: Mention that temp and local spaces can be truncated in ro mode
      
      DML operations on temporary and local spaces can be performed even if
      the instance is in the read-only mode, but DDL operations (such as
      `alter`) are forbidden in this case[^1]. Technically, `truncate` is
      a DDL operation so initially it was forbidden as well. However, it
      should be safe to perform this operation on a temporary or local space
      because logically it only modifies the data stored in the space (like
      DML) and it isn't replicated (see tarantool/tarantool#4263). So starting
      from Tarantool 2.11.1 we allow users to truncate temporary spaces in the
      read-only mode.
      
      [^1]: https://www.tarantool.io/en/doc/latest/concepts/replication/repl_architecture/#replication-local
      
      (cherry picked from commit 054526ac)
      2fc54ba1
    • Vladimir Davydov's avatar
      vinyl: fix use-after-free in vy_read_iterator_next · 2a6a9b75
      Vladimir Davydov authored
      A read source iterator stores statements in a vy_history object using
      vy_history_append_stmt(). If a statement can be referenced, it's
      reference counter is incremented. If it can't, i.e. it belongs to a
      memory source, it's stored in a vy_history object without referencing.
      
      This works fine because memory sources are append-only. A problem arises
      only when we get to scanning disk sources. Since we yield while reading
      disk, a dump task may complete concurrently dropping the memory sources
      and possibly invalidating statements stored in the iterator history.
      Although we drop the history accumulated so far and restart the
      iteration from scratch in this case, there's still an issue that can
      result in a use-after-free bug in vy_read_iterator_next().
      
      The problem is that we access the current candidate for the next
      statement while evaluating a disk source after a disk read. If 'next'
      refers to a referenced statement, it's fine, but if it refers to a
      statement from a memory source, it may cause use-after-free because
      the memory source may be dropped during a disk read.
      
      To fix this issue, let's make vy_history_append_stmt() copy statements
      coming from memory sources. This should be fine performance-wise because
      we copied memory statements eventually in vy_history_apply() anyway,
      before returning them to the user.
      
      Note that we also have to update vy_read_iterator_restore_mem() because
      it implicitly relied on the fact that 'next' coming from a memory source
      can't be freed by vy_mem_iterator_restore(), which cleans up the memory
      source history. Now, it isn't true anymore so we have to temporarily
      take a reference to 'next' explicitly.
      
      Closes #8852
      
      NO_DOC=bug fix
      NO_TEST=tested by ASAN
      
      (cherry picked from commit 0e5a3cc2)
      2a6a9b75
  14. Jul 13, 2023
    • Igor Munkin's avatar
      cmake: introduce FIBER_STACK_SIZE option · aae1daab
      Igor Munkin authored
      In scope of the commit 82f4b4a3 ("lib/core/fiber: Increase default
      stack size") the default value of fiber stack size is increased up to
      512 Kb (you can find the reasons in the aforementioned commit message
      and in https://github.com/tarantool/tarantool/issues/3418 description).
      
      Some of the tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT repo
      (e.g. some cases with deep recursion in errors.lua or pm.lua) have
      already been tweaked according to the limitations mentioned in
      https://github.com/tarantool/tarantool/issues/5782, but the crashes
      still occurs while running LuaJIT tests with ASan support enabled.
      
      To make the testing routine more convenient, FIBER_STACK_SIZE option is
      introduced to Tarantool CMake machinery. One can provide the size either
      by raw digits (i.e. in bytes) or using Kb/Mb suffixes for convenience.
      
      A couple of important nits:
      * If the given value is not a multiple of 4Kb, CMake machinery adjusts
        it up to the nearest one greater than this value.
      * If the adjusted value is less than 512Kb, configuration fails with the
        corresponding CMake fatal error.
      
      Follows up #3418
      Relates to #5782
      
      @TarantoolBot document
      Title: introduce FIBER_STACK_SIZE configuration option
      
      To make managing of the default fiber stack size more convenient, the
      corresponding CMake option is added.
      
      **NB**: The stack size can't be less than 512Kb and if the given value
      is not a multiple of 4Kb, CMake machinery adjusts it up to the nearest
      one greater than this value.
      
      (cherry picked from commit ff57f990)
      Unverified
      aae1daab
    • Gleb Kashkin's avatar
      compat: fix err msg in compat.<option>:is_new() · d33e0635
      Gleb Kashkin authored
      In the original commit 5f6d367c ("compat: add is_new and
      is_old to options") `compat.<option_name>:is_new()` and `:is_old()`
      were introduced, but by mistake they contained different usage
      messages. This patch updates `:is_new()` usage msg to more
      informative one from `:is_old()`.
      
      Follows up #8807
      
      NO_CHANGELOG=changelog from 5f6d367c is valid
      NO_DOC=doc from 5f6d367c is valid
      
      (cherry picked from commit f590ec22)
      Unverified
      d33e0635
  15. Jul 12, 2023
    • Gleb Kashkin's avatar
      compat: add is_new and is_old to options · 380956f4
      Gleb Kashkin authored
      It used to be somewhat complicated to check the effective value
      of a compat option, because `<option_name>.current` could contain
      'default' state.
      This patch introduces helper functions that take care of that.
      
      The following alternatives were considered:
      * `compat.<option_name>.effective` - it is excessive in the presence
        if `current` and `default`, and is visible in serialization
      * `compat.<option_name>.get()` - while it is a function, it does only
        half of the work required, user still has to compare result to 'new'
      
      Closes #8807
      
      @TarantoolBot document
      Title: Add `:is_new/old()` helpers to tarantool.compat options
      
      `compat.<option_name>.current` can be 'new', 'old' or 'default',
      thus when it is default there must be an additional check if
      `compat.<option_name>.default` is 'new'. It is handier to have a
      helper to deal with that instead of complicated `if`:
      * check if effective value is 'new' before the patch:
        ```lua
        if compat.<option_name>.current == 'new' or
                (compat.<option_name>.current == 'default' and
                 compat.<option_name>.default == 'new') then
            ...
        end
        ```
      * after the patch:
        ```lua
        if compat.<option_name>:is_new() then
            ...
        end
        ```
      
      Please update [tutorial on using compat], maybe add an example to
      [Listing options details].
      
      [tutorial on using compat]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/compat/compat_tutorial/
      [Listing options details]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/compat/compat_tutorial/#listing-options-details
      
      (cherry picked from commit 5f6d367c)
      Unverified
      380956f4
  16. Jul 06, 2023
    • Magomed Kostoev's avatar
      box: compare and hash msgpack value of double key field as double · 45044d20
      Magomed Kostoev authored
      1. Make double-formatted fields accept integer and float values.
      2. Make indexes compare the values as double if the field key type
         is FIELD_TYPE_DOUBLE.
      3. Make hashers cast double key field to double before hashing, so
         we are able to insert and select any int, uint, float or double
         if their value casted to double is equal (for double keys).
      
      Notes about tuple_compare.cc:
      
      Since now `mp_compare_double` casts any value placed in field to
      double it was renamed to `mp_compare_as_double` to not semantically
      conflict with existing `mp_compare_double_*` functions.
      
      Notes about tuple_hash.cc:
      
      The hashee cast result is encoded in MP_DOUBLE and hashed for
      backward compatibility reasons.
      
      Since now the field hashing function (tuple_hash_field) requires
      field type to hash the field correctly, a new parameter has been
      introduced.
      
      By the way added assertions to the generic `field_hash` to prevent
      invalid hashing for new precompiled hashers and made
      `key_hash_slowpath` static cause it's only used in this file.
      
      Closes #7483
      Closes #5933
      Unblocks tarantool/crud#298
      
      @TarantoolBot document
      Title: It's not required to ffi-cast integral floating point to
      double anymore.
      
      The page describing tarantool data model states that:
      
      > In Lua, fields of the double type can only contain non-integer
      > numeric values...
      
      If the patch is merged this isn't the case anymore, so this
      statement and the code snippet below it should be updated.
      
      Link to the document: [Data storage](https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#field-type-details).
      
      Affected segments:
      
      > double. The double field type exists mainly to be equivalent
      > to Tarantool/SQL’s DOUBLE data type. In msgpuck.h (Tarantool’s
      > interface to MsgPack), the storage type is MP_DOUBLE and the
      > size of the encoded value is always 9 bytes. In Lua, fields of
      > the double type can only contain non-integer numeric values and
      > cdata values with double floating-point numbers. Examples: 1.234,
      > -44, 1.447e+44.
      >
      > To avoid using the wrong kind of values inadvertently, use
      > ffi.cast() when searching or changing double fields. For example,
      > instead of space_object:insert{value} use ffi = require('ffi')
      > ... space_object:insert({ffi.cast('double',value)}). Example:
      >
      > ```
      > s = box.schema.space.create('s', {format = {{'d', 'double'}}})
      > s:create_index('ii')
      > s:insert({1.1})
      > ffi = require('ffi')
      > s:insert({ffi.cast('double', 1)})
      > s:insert({ffi.cast('double', tonumber('123'))})
      > s:select(1.1)
      > s:select({ffi.cast('double', 1)})
      > ```
      
      (cherry picked from commit 51af059c)
      45044d20
    • Magomed Kostoev's avatar
      Bump msgpuck submodule · 206ed899
      Magomed Kostoev authored
      This update pulls the following commits:
      * Add mp_read_double_lossy without direct convertibility checks
      * Fix mp_read_double_lossy tests freebsd build
      
      These commits introduce a function required to compare and hash
      msgpack values of double key fields as double.
      
      Need for #7483, #5933
      
      NO_DOC=see the next commit
      NO_CHANGELOG=see the next commit
      
      (cherry picked from commit be47f8fb)
      206ed899
    • Sergey Bronnikov's avatar
      test: fix tarantool process teardown · 56024743
      Sergey Bronnikov authored
      Test uses a popen module that starts tarantool process in background
      mode. Tarantool process started in background mode forks a new process
      and closes a parent, after that popen loses a PID of the started process
      and `ph:kill()` and `ph:terminate()` doesn't work anymore. It leads to
      non-terminated tarantool processes after running the test.
      
      Patch fixes that by running `kill` using os.execute with a PID of
      tarantool process written to a pid file.
      
      Follows up #6128
      
      NO_CHANGELOG=fix test
      NO_DOC=fix test
      
      (cherry picked from commit 88686227)
      Unverified
      56024743
    • Ilya Grishnov's avatar
      box: fix shared lang between connected clients · 93812b3b
      Ilya Grishnov authored
      Fixed the implementation of the box console.
      Before this fix, result of `\set language` is shared between clients
      via `console.connect`, despite the fact that clients have different
      `box.session.id`. Now the parameter of the selected language is stored
      by each client in his own `box.session.storage`.
      
      Fixes #8817
      
      NO_DOC=bugfix
      
      (cherry picked from commit e4fda4b7)
      Unverified
      93812b3b
  17. Jul 04, 2023
    • Igor Munkin's avatar
      luajit: bump new version · ae11d314
      Igor Munkin authored
      * test: fix flaky <unit-jit-parse.test.lua>
      * Fix use-def analysis for vararg functions.
      * Fix use-def analysis for BC_VARG.
      * Fix TNEW load forwarding with instable types.
      * Fix memory probing allocator to check for valid end address, too.
      * Another fix for lua_yield() from C hook.
      * Fix lua_yield() from C hook.
      * Fix saved bytecode encapsulated in ELF objects.
      * x64: Fix 64 bit shift code generation.
      * Fix canonicalization of +-0.0 keys for IR_NEWREF.
      * test: add utility for parsing `jit.dump`
      * test: split utils.lua into several modules
      * test: rewrite lj-49-bad-lightuserdata test in C
      * test: rewrite misclib-sysprof-capi test in C
      * test: rewrite misclib-getmetrics-capi test in C
      * test: introduce utils.h helper for C tests
      * test: introduce module for C tests
      * test: fix setting of {DY}LD_LIBRARY_PATH variables
      * build: fix build with LUAJIT_USE_GDBJIT enabled
      * ci: update the branch name for Tarantool 2.11
      
      Closes #8718
      Part of #7900
      Part of #8516
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      ae11d314
    • Sergey Bronnikov's avatar
      test: fix flakiness in gh_6128_background_mode_test · 9d5cd29d
      Sergey Bronnikov authored
      Previous attempt to fix flakiness in commit 6a2c73f8 ("test: fix
      flakiness in gh_6128_background_mode_test") used a constant buffer size
      in check_err_msg function. Tarantool 2.10 has a bit bigger log before a
      desired message that other versions of Tarantool and it leads to a this
      resulted in a truncated message ("entering the even" instead of
      "entering the event loop"). Patch replaces check_err_msg()
      implementation to grep_log used in luatest, it reads the whole log.
      
      Also patch renames check_err_msg to check_msg, because "entering the
      event loop" is not an error message.
      
      Follows up #6128
      
      NO_CHANGELOG=fix test
      NO_DOC=fix test
      
      (cherry picked from commit 1c8e7124)
      Unverified
      9d5cd29d
    • Magomed Kostoev's avatar
      box: check permissions on constraint functions on creation · 2cde4c66
      Magomed Kostoev authored
      Function execution permissions should only be checked on constraint
      creation.
      
      So when the function is used to check a tuple access rights don't
      have to be checked on each call for the current user.
      
      Closes #7873
      
      NO_DOC=bugfix
      
      (cherry picked from commit 6b8f2c5f)
      2cde4c66
Loading