Skip to content
Snippets Groups Projects
  1. Jan 19, 2024
    • Nikita Zheleztsov's avatar
      relay: fix unterminated final_join thread · 040cd9e3
      Nikita Zheleztsov authored
      
      Currently if tarantool exits during relay's final join stage,
      corresponding thread isn't terminated. This causes the flakiness
      of the replicaset_ro_mostly.test.lua.
      
      Let's reuse the same relay, in which subscribe cord is running, for
      the final join stage. This way the cord will be cancelled during
      replication_free().
      
      Closes #8082
      
      NO_DOC=not user-visible
      NO_TEST=fix flaky test
      NO_CHANGELOG=not user-visible
      
      Co-authored-by: default avatarSergey Petrenko <sergepetrenko@tarantool.org>
      
      (cherry picked from commit 70a68836)
      040cd9e3
    • Vladimir Davydov's avatar
      git: add diff=cpp attribute for C/C++ source files · 6148e590
      Vladimir Davydov authored
      This improves diff hunk name detection. Needed for checkpatch to
      correctly detect if patched code belongs to a function.
      
      NO_DOC=git
      NO_TEST=git
      NO_CHANGELOG=git
      
      (cherry picked from commit 642584fd)
      6148e590
    • Mergen Imeev's avatar
      sql: fix wrong region allocation · 9fef276e
      Mergen Imeev authored
      This patch fixes an issue in generate_column_metadata(). Prior to this
      patch, the number of variable-only expressions was counted incorrectly
      when temporary memory was allocated on region to store their positions.
      However, although this allocation was incorrect, this did not lead to
      any problems due to the specifics of the region allocations.
      
      This patch fixes this by removing the temporary memory allocation.
      
      Closes #8763
      
      NO_DOC=no user-visible changes
      NO_TEST=no user-visible changes
      NO_CHANGELOG=no user-visible changes
      
      (cherry picked from commit d4f143ad)
      9fef276e
    • Georgy Moiseev's avatar
      msgpack: fix decoding intervals with int64 · aa48086c
      Georgy Moiseev authored
      It is possible for interval to have days, hours, minutes and seconds
      larger than INT_MAX (or less than INT_MIN). Before this patch, msgpack
      decoding had failed to parse intervals with msgpack int64 and uint64.
      int64_t should be enough to store any value allowed for datetime
      intervals.
      
      Closes #8887
      
      NO_DOC=small bug fix
      
      (cherry picked from commit 01c7ae11)
      aa48086c
    • Sergey Kaplun's avatar
      lua: replace `api_check()` with `assert()` · e5a18019
      Sergey Kaplun authored
      `api_check()` is the LuaJIT internal assertion. To prevent inconsistency
      during internal assertion changes (for example, during backporting), use
      glibc's `assert()` instead.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      
      (cherry picked from commit e78afb3b)
      e5a18019
    • Igor Munkin's avatar
      luajit: bump new version · f259e74a
      Igor Munkin authored
      * FFI: Fix recording of union initialization.
      * Fix maxslots when recording BC_VARG, part 2.
      * Fix maxslots when recording BC_VARG.
      * Fix BC_UCLO insertion for returns.
      * ci: update job concurrency group definition
      
      Part of #8825
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      f259e74a
    • Georgy Moiseev's avatar
      datetime: allow boundary values for interval.new · c369dd8f
      Georgy Moiseev authored
      Before this patch, one couldn't create new datetime interval with
      boundary value from Lua. At the same time, it was possible to create
      such interval from Lua through addition and subtraction. C range
      verification allow to create boundary value intervals, error message
      also implies that they should be allowed. (See #8878 for more info.)
      
      Closes #8878
      
      NO_DOC=small bug fix
      
      (cherry picked from commit b2a001cc)
      c369dd8f
    • Rimma Tolkacheva's avatar
      test/fuzz: refactor LuaJIT fuzzer · e08a0c7c
      Rimma Tolkacheva authored
      This refactoring will:
      
      1. Move macros from a header to the source file.
      Macros should be used in header only with undef to avoid redefinitions.
      Undef directive is not useful since we want to use these macros in the
      source file.
      
      2. Remove `using namespace lua_grammar` from header.
      https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-using-directive
      
      3. Moving serializer entry point and constant parameters into
      luajit_fuzzer namespace.
      It's a common practice in C++ to avoid name collisions.
      
      4. Move serializer functions into anonymous namespace.
      These functions are not a part of the interface so should have
      static linkage.
      https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-unnamed2
      
      5. Fix ConvertToStringDefault function.
      It was logically wrong so it would generate an identifier `123` from
      `*123`.
      
      NO_CHANGELOG=internal
      NO_DOC=fuzzer fix
      
      (cherry picked from commit 56488e15)
      e08a0c7c
    • klauwier's avatar
      test/fuzz: fix luaJIT fuzzer timeout · 89cbb737
      klauwier authored
      LuaJIT fuzzer used to stop due to timeout caused by infinite cycles and
      recursions. Counters were introduced for every cycle and function to
      address LuaJIT fuzzer timeouts.
      
      The idea is to add unique counters for every cycle and function to
      ensure finite code execution, if it wasn't already. For while, repeat,
      for cycles, local and global named, anonymous functions, counters will
      be initialized before the code generated from protobuf, and checked
      in the first body statement. An entry point for the serializer was
      created to count cycles and functions for counter initialization.
      
      The idea was taken from a paper "Program Reconditioning: Avoiding
      Undefined Behaviour When Finding and Reducing Compiler Bugs" [1].
      
      Here is an example of a change in serialized code made by this commit.
      
      Before:
      ```lua
      while (true) do
          foo = 'bar'
      end
      function bar()
          bar()
      end
      ```
      
      After:
      ```lua
      counter_0 = 0
      counter_1 = 0
      while (true) do
          if counter_0 > 5 then
              break
          end
          counter_0 = counter_0 + 1
          foo = 'bar'
      end
      function bar()
          if counter_1 > 5 then
              return
          end
          counter_1 = counter_1 + 1
          bar()
      end
      ```
      Protobuf structures that reproduce the timeout problem were added to
      the LuaJIT fuzzer corpus.
      
      [1] https://www.doc.ic.ac.uk/~afd/homepages/papers/pdfs/2023/PLDI.pdf
      
      NO_CHANGELOG=internal
      NO_DOC=fuzzer fix
      
      (cherry picked from commit 4d004bbe)
      89cbb737
    • klauwier's avatar
      test/fuzz: add breaks to switch-case · 53bed159
      klauwier authored
      Cases in two switches had no breaks, so they were falling
      through. Breaks were added to solve the problem. Code
      generated by the LuaJIT fuzzer became more various.
      
      NO_CHANGELOG=internal
      NO_DOC=fuzzer fix
      
      (cherry picked from commit 4430cac9)
      53bed159
    • Sergey Bronnikov's avatar
      cmake: propagate CMAKE_BUILD_TYPE for ProtobufMutator · 8cb37b89
      Sergey Bronnikov authored
      Follows-up #4823
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      NO_TEST=internal
      
      (cherry picked from commit 95d62cfc)
      8cb37b89
    • Sergey Bronnikov's avatar
      test/fuzz: fix datetime_strptime fuzzing test · 9304de91
      Sergey Bronnikov authored
      Function `datetime_strptime` decodes string with datetime according to
      specified format, it accepts a datetime struct, buffer with datetime and
      string with format in arguments. Fuzzing test used static string
      "iso8601" as a format and it blocked fuzzing test to cover functions
      used by datetime_strptime under the hood. Fuzz introspector shows that
      code coveraged by a test is quite low.
      
      Patch updates the test to make it more effective: buffer with datetime
      and format string are generated using FDP (Fuzzing Data Provider).
      
      Test file extension was changed to .cc, because FuzzingDataProvider is
      used and we need building it by C++ compiler.
      
      Function `tnt_strptime` uses assert, that triggered by fuzzing tests.
      Therefore it was replaced with to if..then.
      
      1. https://storage.googleapis.com/oss-fuzz-introspector/tarantool/
      
      Fixes #8490
      
      NO_CHANGELOG=fuzzing test
      NO_DOC=fuzzing test
      NO_TEST=fuzzing test
      
      (cherry picked from commit a1bd6e0b)
      9304de91
    • Timur Safin's avatar
      datetime: fix buffer overflow in tnt_strptime · cbc92fe8
      Timur Safin authored
      Fixes #8502
      Needed for #8490
      
      NO_DOC=bugfix
      NO_TEST=covered by fuzzing test
      
      (cherry picked from commit 783a7040)
      cbc92fe8
    • Dmitriy Nesterov's avatar
      test/fuzz: add grammar-based LuaJIT fuzzer · 23d56ddf
      Dmitriy Nesterov authored
      Patch adds a LuaJIT fuzzer based on libprotobuf-mutator and LibFuzzer.
      Grammar is described via messages in protobuf format, serializer is
      applied to convert .proto format to string.
      
      For displaying generated code on the screen during fuzzing set
      the environment variable 'LPM_DUMP_NATIVE_INPUT'.
      
      For displaying error messages from lua functions set
      the environment variable 'LUA_FUZZER_VERBOSE'.
      
      Note: UndefinedBehaviourSanitizer is unsupported by LuaJIT (see #8473),
      so fuzzing test is disabled when CMake option ENABLE_UB_SANITIZER is
      passed.
      
      Closes #4823
      
      NO_DOC=<fuzzing testing of LuaJIT>
      NO_TEST=<fuzzing testing of LuaJIT>
      
      (cherry picked from commit a287c853)
      23d56ddf
    • Dmitriy Nesterov's avatar
      cmake: add dependencies for LuaJIT and SQL fuzzers · 07ee9a4f
      Dmitriy Nesterov authored
      Added Google's 'libprotobuf-mutator' and 'protobuf' libraries
      for developing grammar-based LuaJIT and SQL fuzzers based on
      LibFuzzer.
      
      It is needed to build protobuf module from source because
      by default, the system-installed version of protobuf is used
      by libprotobuf-mutator, and this version can be too old.
      
      Part of #4823
      
      NO_CHANGELOG=<dependencies>
      NO_DOC=<dependencies>
      NO_TEST=<dependencies>
      
      (cherry picked from commit b11072a6)
      07ee9a4f
    • Dmitriy Nesterov's avatar
      test/fuzz: add options for better fuzzing · 481d7c7a
      Dmitriy Nesterov authored
      Added options for fuzzing and for getting more information
      on debugging.
      
      NO_CHANGELOG=<fuzzing options>
      NO_DOC=<fuzzing options>
      NO_TEST=<fuzzing options>
      
      (cherry picked from commit 69f21e25)
      481d7c7a
    • Vladimir Davydov's avatar
      vinyl: fix use-after-free in vy_read_iterator_next · 0a7d6e7f
      Vladimir Davydov authored
      A read source iterator stores statements in a vy_history object using
      vy_history_append_stmt(). If a statement can be referenced, it's
      reference counter is incremented. If it can't, i.e. it belongs to a
      memory source, it's stored in a vy_history object without referencing.
      
      This works fine because memory sources are append-only. A problem arises
      only when we get to scanning disk sources. Since we yield while reading
      disk, a dump task may complete concurrently dropping the memory sources
      and possibly invalidating statements stored in the iterator history.
      Although we drop the history accumulated so far and restart the
      iteration from scratch in this case, there's still an issue that can
      result in a use-after-free bug in vy_read_iterator_next().
      
      The problem is that we access the current candidate for the next
      statement while evaluating a disk source after a disk read. If 'next'
      refers to a referenced statement, it's fine, but if it refers to a
      statement from a memory source, it may cause use-after-free because
      the memory source may be dropped during a disk read.
      
      To fix this issue, let's make vy_history_append_stmt() copy statements
      coming from memory sources. This should be fine performance-wise because
      we copied memory statements eventually in vy_history_apply() anyway,
      before returning them to the user.
      
      Note that we also have to update vy_read_iterator_restore_mem() because
      it implicitly relied on the fact that 'next' coming from a memory source
      can't be freed by vy_mem_iterator_restore(), which cleans up the memory
      source history. Now, it isn't true anymore so we have to temporarily
      take a reference to 'next' explicitly.
      
      Closes #8852
      
      NO_DOC=bug fix
      NO_TEST=tested by ASAN
      
      (cherry picked from commit 0e5a3cc2)
      0a7d6e7f
    • Igor Munkin's avatar
      cmake: introduce FIBER_STACK_SIZE option · a6a52d1b
      Igor Munkin authored
      In scope of the commit 82f4b4a3 ("lib/core/fiber: Increase default
      stack size") the default value of fiber stack size is increased up to
      512 Kb (you can find the reasons in the aforementioned commit message
      and in https://github.com/tarantool/tarantool/issues/3418 description).
      
      Some of the tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT repo
      (e.g. some cases with deep recursion in errors.lua or pm.lua) have
      already been tweaked according to the limitations mentioned in
      https://github.com/tarantool/tarantool/issues/5782, but the crashes
      still occurs while running LuaJIT tests with ASan support enabled.
      
      To make the testing routine more convenient, FIBER_STACK_SIZE option is
      introduced to Tarantool CMake machinery. One can provide the size either
      by raw digits (i.e. in bytes) or using Kb/Mb suffixes for convenience.
      
      A couple of important nits:
      * If the given value is not a multiple of 4Kb, CMake machinery adjusts
        it up to the nearest one greater than this value.
      * If the adjusted value is less than 512Kb, configuration fails with the
        corresponding CMake fatal error.
      
      Follows up #3418
      Relates to #5782
      
      @TarantoolBot document
      Title: introduce FIBER_STACK_SIZE configuration option
      
      To make managing of the default fiber stack size more convenient, the
      corresponding CMake option is added.
      
      **NB**: The stack size can't be less than 512Kb and if the given value
      is not a multiple of 4Kb, CMake machinery adjusts it up to the nearest
      one greater than this value.
      
      (cherry picked from commit ff57f990)
      a6a52d1b
    • Gleb Kashkin's avatar
      compat: fix err msg in compat.<option>:is_new() · 546e63d3
      Gleb Kashkin authored
      In the original commit 5f6d367c ("compat: add is_new and
      is_old to options") `compat.<option_name>:is_new()` and `:is_old()`
      were introduced, but by mistake they contained different usage
      messages. This patch updates `:is_new()` usage msg to more
      informative one from `:is_old()`.
      
      Follows up #8807
      
      NO_CHANGELOG=changelog from 5f6d367c is valid
      NO_DOC=doc from 5f6d367c is valid
      
      (cherry picked from commit f590ec22)
      546e63d3
    • Gleb Kashkin's avatar
      compat: add is_new and is_old to options · 05eb81e2
      Gleb Kashkin authored
      It used to be somewhat complicated to check the effective value
      of a compat option, because `<option_name>.current` could contain
      'default' state.
      This patch introduces helper functions that take care of that.
      
      The following alternatives were considered:
      * `compat.<option_name>.effective` - it is excessive in the presence
        if `current` and `default`, and is visible in serialization
      * `compat.<option_name>.get()` - while it is a function, it does only
        half of the work required, user still has to compare result to 'new'
      
      Closes #8807
      
      @TarantoolBot document
      Title: Add `:is_new/old()` helpers to tarantool.compat options
      
      `compat.<option_name>.current` can be 'new', 'old' or 'default',
      thus when it is default there must be an additional check if
      `compat.<option_name>.default` is 'new'. It is handier to have a
      helper to deal with that instead of complicated `if`:
      * check if effective value is 'new' before the patch:
        ```lua
        if compat.<option_name>.current == 'new' or
                (compat.<option_name>.current == 'default' and
                 compat.<option_name>.default == 'new') then
            ...
        end
        ```
      * after the patch:
        ```lua
        if compat.<option_name>:is_new() then
            ...
        end
        ```
      
      Please update [tutorial on using compat], maybe add an example to
      [Listing options details].
      
      [tutorial on using compat]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/compat/compat_tutorial/
      [Listing options details]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/compat/compat_tutorial/#listing-options-details
      
      (cherry picked from commit 5f6d367c)
      05eb81e2
    • Magomed Kostoev's avatar
      box: compare and hash msgpack value of double key field as double · 71f5992f
      Magomed Kostoev authored
      1. Make double-formatted fields accept integer and float values.
      2. Make indexes compare the values as double if the field key type
         is FIELD_TYPE_DOUBLE.
      3. Make hashers cast double key field to double before hashing, so
         we are able to insert and select any int, uint, float or double
         if their value casted to double is equal (for double keys).
      
      Notes about tuple_compare.cc:
      
      Since now `mp_compare_double` casts any value placed in field to
      double it was renamed to `mp_compare_as_double` to not semantically
      conflict with existing `mp_compare_double_*` functions.
      
      Notes about tuple_hash.cc:
      
      The hashee cast result is encoded in MP_DOUBLE and hashed for
      backward compatibility reasons.
      
      Since now the field hashing function (tuple_hash_field) requires
      field type to hash the field correctly, a new parameter has been
      introduced.
      
      By the way added assertions to the generic `field_hash` to prevent
      invalid hashing for new precompiled hashers and made
      `key_hash_slowpath` static cause it's only used in this file.
      
      Closes #7483
      Closes #5933
      Unblocks tarantool/crud#298
      
      @TarantoolBot document
      Title: It's not required to ffi-cast integral floating point to
      double anymore.
      
      The page describing tarantool data model states that:
      
      > In Lua, fields of the double type can only contain non-integer
      > numeric values...
      
      If the patch is merged this isn't the case anymore, so this
      statement and the code snippet below it should be updated.
      
      Link to the document: [Data storage](https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#field-type-details).
      
      Affected segments:
      
      > double. The double field type exists mainly to be equivalent
      > to Tarantool/SQL’s DOUBLE data type. In msgpuck.h (Tarantool’s
      > interface to MsgPack), the storage type is MP_DOUBLE and the
      > size of the encoded value is always 9 bytes. In Lua, fields of
      > the double type can only contain non-integer numeric values and
      > cdata values with double floating-point numbers. Examples: 1.234,
      > -44, 1.447e+44.
      >
      > To avoid using the wrong kind of values inadvertently, use
      > ffi.cast() when searching or changing double fields. For example,
      > instead of space_object:insert{value} use ffi = require('ffi')
      > ... space_object:insert({ffi.cast('double',value)}). Example:
      >
      > ```
      > s = box.schema.space.create('s', {format = {{'d', 'double'}}})
      > s:create_index('ii')
      > s:insert({1.1})
      > ffi = require('ffi')
      > s:insert({ffi.cast('double', 1)})
      > s:insert({ffi.cast('double', tonumber('123'))})
      > s:select(1.1)
      > s:select({ffi.cast('double', 1)})
      > ```
      
      (cherry picked from commit 51af059c)
      71f5992f
    • Magomed Kostoev's avatar
      Bump msgpuck submodule · e8bbe9db
      Magomed Kostoev authored
      This update pulls the following commits:
      * Add mp_read_double_lossy without direct convertibility checks
      * Fix mp_read_double_lossy tests freebsd build
      
      These commits introduce a function required to compare and hash
      msgpack values of double key fields as double.
      
      Need for #7483, #5933
      
      NO_DOC=see the next commit
      NO_CHANGELOG=see the next commit
      
      (cherry picked from commit be47f8fb)
      e8bbe9db
    • Sergey Bronnikov's avatar
      test: fix tarantool process teardown · d3f7e18f
      Sergey Bronnikov authored
      Test uses a popen module that starts tarantool process in background
      mode. Tarantool process started in background mode forks a new process
      and closes a parent, after that popen loses a PID of the started process
      and `ph:kill()` and `ph:terminate()` doesn't work anymore. It leads to
      non-terminated tarantool processes after running the test.
      
      Patch fixes that by running `kill` using os.execute with a PID of
      tarantool process written to a pid file.
      
      Follows up #6128
      
      NO_CHANGELOG=fix test
      NO_DOC=fix test
      
      (cherry picked from commit 88686227)
      d3f7e18f
    • Ilya Grishnov's avatar
      box: fix shared lang between connected clients · ab3f317d
      Ilya Grishnov authored
      Fixed the implementation of the box console.
      Before this fix, result of `\set language` is shared between clients
      via `console.connect`, despite the fact that clients have different
      `box.session.id`. Now the parameter of the selected language is stored
      by each client in his own `box.session.storage`.
      
      Fixes #8817
      
      NO_DOC=bugfix
      
      (cherry picked from commit e4fda4b7)
      ab3f317d
    • Igor Munkin's avatar
      luajit: bump new version · c27cdcb2
      Igor Munkin authored
      * test: fix flaky <unit-jit-parse.test.lua>
      * Fix use-def analysis for vararg functions.
      * Fix use-def analysis for BC_VARG.
      * Fix TNEW load forwarding with instable types.
      * Fix memory probing allocator to check for valid end address, too.
      * Another fix for lua_yield() from C hook.
      * Fix lua_yield() from C hook.
      * Fix saved bytecode encapsulated in ELF objects.
      * x64: Fix 64 bit shift code generation.
      * Fix canonicalization of +-0.0 keys for IR_NEWREF.
      * test: add utility for parsing `jit.dump`
      * test: split utils.lua into several modules
      * test: rewrite lj-49-bad-lightuserdata test in C
      * test: rewrite misclib-sysprof-capi test in C
      * test: rewrite misclib-getmetrics-capi test in C
      * test: introduce utils.h helper for C tests
      * test: introduce module for C tests
      * test: fix setting of {DY}LD_LIBRARY_PATH variables
      * build: fix build with LUAJIT_USE_GDBJIT enabled
      * ci: update the branch name for Tarantool 2.11
      
      Closes #8718
      Part of #7900
      Part of #8516
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      c27cdcb2
    • Sergey Bronnikov's avatar
      test: fix flakiness in gh_6128_background_mode_test · 2ce71928
      Sergey Bronnikov authored
      Previous attempt to fix flakiness in commit 6a2c73f8 ("test: fix
      flakiness in gh_6128_background_mode_test") used a constant buffer size
      in check_err_msg function. Tarantool 2.10 has a bit bigger log before a
      desired message that other versions of Tarantool and it leads to a this
      resulted in a truncated message ("entering the even" instead of
      "entering the event loop"). Patch replaces check_err_msg()
      implementation to grep_log used in luatest, it reads the whole log.
      
      Also patch renames check_err_msg to check_msg, because "entering the
      event loop" is not an error message.
      
      Follows up #6128
      
      NO_CHANGELOG=fix test
      NO_DOC=fix test
      
      (cherry picked from commit 1c8e7124)
      2ce71928
    • Magomed Kostoev's avatar
      box: check permissions on constraint functions on creation · 357629d2
      Magomed Kostoev authored
      Function execution permissions should only be checked on constraint
      creation.
      
      So when the function is used to check a tuple access rights don't
      have to be checked on each call for the current user.
      
      Closes #7873
      
      NO_DOC=bugfix
      
      (cherry picked from commit 6b8f2c5f)
      357629d2
    • Magomed Kostoev's avatar
      box: check permissions on functional index functions on creation · ba9698af
      Magomed Kostoev authored
      Function execution permissions should only be checked on functional
      index creation and on functional index function set.
      
      So when the function is used by key_list_iterator its rights don't
      have to be checked on each call for the current user.
      
      Part of #7873
      
      NO_DOC=bugfix
      NO_CHANGELOG=see the next commit
      
      (cherry picked from commit ddbdb77a)
      ba9698af
    • Georgiy Lebedev's avatar
      memtx: fix heap-use-after-free of tuple stories caused by space alter · 5070f252
      Georgiy Lebedev authored
      When a space is altered, we abort all in-progress transactions and delete
      all stories related to that space: the problem is we don't delete the
      stories' read gaps, which are also linked to the stories' transactions,
      which get cleaned up on transaction destruction — this, in turn, results in
      heap-use-after-free. To fix this, clean up stories' read gap in
      `memtx_on_space_delete` — we don't do this in `memtx_tx_story_delete` since
      it expects the story to not have any read gaps (see
      `memtx_tx_story_gc_step`).
      
      Tested this patch manually against Nick Shirokovskiy's experimental
      small-ASAN integration branch.
      
      Closes #8781
      
      NO_DOC=bugfix
      NO_TEST=<already covered by existing tests, but was not detectable by ASAN>
      
      (cherry picked from commit e1ed31bb)
      5070f252
    • Sergey Bronnikov's avatar
      test: fix flakiness in gh_6128_background_mode_test · 791ff79a
      Sergey Bronnikov authored
      Test runs an external process with tarantool that writes to a log file.
      Then test reads that log file and searches a string with required
      message in it (see function check_err_msg). Test was flaky on macOS and
      I suspect it was happening due to a high log level - timeout was not
      enough to wait message in the log file.
      
      Patch decreases a log level to a default value and replaces io
      functions with the similar alternatives in a fio module. Using
      fio functions allows to not block fibers.
      
      NO_CHANGELOG=test fix
      NO_DOC=test fix
      
      (cherry picked from commit 47380bb7)
      791ff79a
    • Vladimir Davydov's avatar
      lua/xlog: don't ignore unknown header fields · d1064aca
      Vladimir Davydov authored
      The xlog reader Lua module uses the xlog_cursor_next_row, which decodes
      the row header with xrow_header_decode. The latter silently ignores any
      unknown fields, which complicates catching bugs when garbage is written
      to a row header by mistake, for example, see #8783.
      
      Let's parse a row header without using xrow_header_decode in the xlog
      reader module, like we parse a row body, and output all unknown/invalid
      keys as is.
      
      To do that, we have to extend the xlog cursor API with the new method
      xlog_cursor_next_row_raw that returns a pointer to the position in the
      tx buffer where the next xrow is stored without advancing it. To avoid
      a memory leak in case the caller fails to parse an xrow returned by this
      function, we also have to move the call to xlog_tx_cursor_destroy from
      xlog_tx_cursor_next_row to xlog_cursor_next_tx.
      
      While we are at it,
       - Don't raise an error if a key type encountered in a row body is
         invalid (not an integer). Instead, silently ignore such keys.
       - Remove the useless body MsgPack validness check because we already
         check it after decoding the header.
       - Add error injection based tests to check all the corner cases.
      
      NO_DOC=bug fix
      
      (cherry picked from commit 8a25d170)
      d1064aca
    • Vladimir Davydov's avatar
      txn: reset stream_id row header field · e29782da
      Vladimir Davydov authored
      To avoid garbage written to xlog.
      
      Closes #8783
      
      NO_DOC=bug fix
      NO_TEST=next commit
      
      (cherry picked from commit f058cee7)
      e29782da
    • Ilya Verbin's avatar
      core: strip the PAC out of IP during backtrace on AArch64 macOS · bda1e6c3
      Ilya Verbin authored
      Apple's libunwind for AArch64 returns the Instruction Pointer with the
      Pointer Authentication Codes (bits 47-63) even though Tarantool is compiled
      for arm64 (not arm64e) architecture, so we have to strip them out [1].
      Although there is the ptrauth_strip macro for this purpose, it works only
      if compilation target is arm64e (not arm64) [2].
      
      1. https://developer.apple.com/documentation/security/preparing_your_app_to_work_with_pointer_authentication#3042105
      2. https://github.com/dotnet/runtime/issues/42955#issuecomment-886910180
      
      Closes #8074
      Closes tarantool/tarantool-qa#308
      Closes tarantool/tarantool-qa#309
      
      NO_DOC=bugfix
      
      (cherry picked from commit 88990e2f)
      bda1e6c3
    • Igor Munkin's avatar
      box: reduce luaT_pushtuple Lua GC memory usage · 42563c8d
      Igor Munkin authored
      
      <luaT_pushtuple> function created a new GCfunc object for a tuple __gc
      handler (i.e. <lbox_tuple_gc>), having no upvalues, on each call when
      tuple object for Lua world is created.
      
      The change introduces the reference to the single GCfunc object created
      on Tarantool startup and stored to Lua registry. Using this approach in
      scope of <luaT_pushtuple> is aimed to reduce Lua GC memory usage.
      
      Part of #5201
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      
      Reviewed-by: default avatarVladimir Davydov <vdavydov@tarantool.org>
      Reviewed-by: default avatarSergey Kaplun <skaplun@tarantool.org>
      Signed-off-by: default avatarIgor Munkin <imun@tarantool.org>
      (cherry picked from commit 8d7d1460)
      42563c8d
    • Georgiy Lebedev's avatar
      box: fix memory leaks on `ER_MULTISTATEMENT_TRANSACTION` in DDL · 513a0ea7
      Georgiy Lebedev authored
      Space index build and space format checking operations don't destroy space
      iterator on `txn_check_singlestatement` failure — fix this.
      
      Closes #8773
      
      NO_DOC=bugfix
      NO_TEST=<leak happens in small, cannot be detected by sanitizer>
      
      (cherry picked from commit 6689f511)
      513a0ea7
    • Georgiy Lebedev's avatar
      core: use `malloc` instead of `region` allocator in procedure name cache · 0c378ba0
      Georgiy Lebedev authored
      The procedure name cache uses a region for hash table entry allocation, but
      the cache is a thread local variable, while the `region` uses a `slab`
      allocator the lifetime of which is bounded by `main`, while thread locals
      are destroyed after `main`: use the `malloc` allocator instead.
      
      Benchmark from #7207 shows that this change does not effect performance.
      
      Closes #8777
      
      NO_CHANGELOG=<leak does not affect users>
      NO_DOC=bugfix
      NO_TEST=<detected by ASAN>
      
      (cherry picked from commit 7135910e)
      0c378ba0
    • Sergey Bronnikov's avatar
      test: testing tarantool in background mode · 62df7880
      Sergey Bronnikov authored
      Before a commit ec1af129 ("box: do not close xlog file descriptors in
      the atfork handler") there was a bug when Tarantool with enabled
      background mode via environment variable could lead a crash:
      
      NO_WRAP
      ```
      $ TT_PID_FILE=tarantool.pid TT_LOG=tarantool.log TT_BACKGROUND=true TT_LISTEN=3301 tarantool -e 'box.cfg{}'
      $ tail -3 tarantool.log
      2021-11-02 16:05:43.672 [2341202] main init.c:696 E> LuajitError: cannot read stdin: Resource temporarily unavailable
      2021-11-02 16:05:43.672 [2341202] main F> fatal error, exiting the event loop
      2021-11-02 16:05:43.672 [2341202] main F> fatal error, exiting the event loop
      ```
      NO_WRAP
      
      With commit ec1af129 ("box: do not close xlog file descriptors in
      the atfork handler") described bug could not be reproduced.
      
      Proposed patch adds a test that starts Tarantool in background mode
      enabled via box.cfg option and via environment variable TT_BACKGROUND to
      make sure this behaviour will not be broken in a future.
      
      Closes #6128
      
      NO_DOC=test
      
      (cherry picked from commit f676fb7c)
      62df7880
    • Ilya Verbin's avatar
      build: disable backtrace feature on AArch64 Linux · 4b0d4d98
      Ilya Verbin authored
      There are sporadic segfaults in libunwind during backtrace_collect().
      Reproducible with the latest version, and there are open issues with
      similar stacks:
      
      https://github.com/libunwind/libunwind/issues/150
      https://github.com/libunwind/libunwind/issues/260
      https://github.com/libunwind/libunwind/issues/473
      
      Let's disable ENABLE_BACKTRACE for AArch64 Linux until these issues are
      resolved in libunwind.
      
      Closes #8572
      Part of #8791
      
      NO_DOC=bugfix
      
      (cherry picked from commit 418e749c)
      4b0d4d98
    • Ilya Verbin's avatar
      build: remove backtrace feature compiler dependency from rpm spec · 14e8b3d3
      Ilya Verbin authored
      The ability to support backtraces is checked in cmake/compiler.cmake,
      it makes no sense to duplicate the check in rpm/tarantool.spec. Also do
      not enable backtraces unconditionally in apk/APKBUILD and static-build.
      
      Part of #6998
      
      NO_DOC=build
      NO_TEST=build
      NO_CHANGELOG=build
      
      (cherry picked from commit f7c4a34a)
      14e8b3d3
    • Aleksandr Lyapunov's avatar
      memtx: abort readers of rollbacked prepared · 3e33cfa5
      Aleksandr Lyapunov authored
      There's case when a transaction is rolled back from prepared
      state. This happens when WAL fails, synchronized replication
      confirmation failure or perhaps in other similar cases. By design
      other RW transactions and transactions with READ_COMMITTED
      isolation level are allowed to read prepared state. All these
      transactions must be aborted in case of rollback of prepared
      transaction since they definitely have read no more possible
      database state.
      
      This patch implements this abortion.
      
      Closed #8654
      
      NO_DOC=bugfix
      
      (cherry picked from commit 54986902)
      3e33cfa5
Loading