Skip to content
Snippets Groups Projects
  1. Sep 04, 2023
    • Georgy Moshkin's avatar
      box: fully temporary spaces · 70e423e9
      Georgy Moshkin authored
      Introduce fully temporary spaces: same as data-temporary space but with
      temporary metadata. Basically temporary spaces now do not exist on
      restart and do not exist on replicas. They can also be created, altered
      and deleted when box.cfg.read_only = true.
      
      To avoid conflicts with spaces created on replicas, the temporary
      space ids by default start in a special range starting at
      BOX_SPACE_ID_TEMPORARY_MIN.
      
      Temporary spaces currently do not support several features e.g.
      foreign key references (to and from), functional indexes, sql sequences,
      sql triggers, etc. This may change in the future.
      
      Implementing temporary spaces requires temporary tuples to be
      inserted into system spaces: tuples which are neither replicated or
      persisted. This mostly done in on_replace_dd_* triggers by dropping the
      txn->stmt->row.
      
      Closes #8323
      
      @TarantoolBot document
      Title: Introduce fully temporary spaces with temporary metadata
      
      Temporary spaces are now data-temporary spaces with temporary metadata.
      Created by specifying { type = "temporary" } in the options.
      Temporary spaces will not exist upon server restart and will not
      exist on replicas. They can also be created in read-only mode.
      70e423e9
    • Georgy Moshkin's avatar
      box: factor out new_tuple def construction · 96a5224a
      Georgy Moshkin authored
      A tiny preparatory commit for meta-temporary spaces
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      96a5224a
    • Georgy Moshkin's avatar
      box: extract txn_update_row_counts function · 3619b9b4
      Georgy Moshkin authored
      Move code that handles txn row counters into a separate function in
      preparation of meta-temporary spaces introduction.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      3619b9b4
    • Georgy Moshkin's avatar
      core: rename temporary spaces to data-temporary · 2375b3a3
      Georgy Moshkin authored
      Everywhere where we refer to temporary spaces we now say data-temporary.
      This is because temporary spaces were never truly temporary because
      their definitions would still be persisted and replicated and they
      couldn't be created on read-only replicas. In a following commit we will
      introduce a new fully temporary type of spaces, which will be just
      called 'temporary', so this commit signifies this terminology change.
      
      NO_DOC=renaming
      NO_CHANGELOG=renaming
      NO_TEST=renaming
      2375b3a3
    • Georgy Moshkin's avatar
      box: introduce space type · ec9953ea
      Georgy Moshkin authored
      Introduces a new field `type` to the space definition. Currently it can
      only be "normal" or "data-temporary". It is backwards compatible with
      temporary=true.
      
      @TarantoolBot document
      Title: Introduce space field type
      
      A new space definition field "type" can now be used to specify the type
      of the space. Usage: box.schema.create_space("s", { type = "normal" }).
      Currently only 2 types are supported: "normal" & "data-temporary", which
      is equivalent to { temporary = true }. Old-style { temporary = true } is
      still supported, but only one option either 'temporary' or 'type' may be
      specified at the same time.
      
      Space type "temporary" will be introduced in a later commit.
      In the future options "local", "synchronous", etc. may also be
      supported.
      
      NO_TEST=will be tested in the following commit
      ec9953ea
    • Ilya Verbin's avatar
      test: increase maximum fiber slice for wal_off tests · 06be32fb
      Ilya Verbin authored
      If a lot of tests are running in parallel, 50 sec limit may not be enough.
      Let's increase it to 120 sec.
      
      Closes tarantool/tarantool-qa#273
      
      NO_DOC=test
      NO_CHANGELOG=test
      06be32fb
  2. Sep 01, 2023
    • Nikolay Shirokovskiy's avatar
      lua: provide tarantool build info before loading lua modules · f58cc96f
      Nikolay Shirokovskiy authored
      This way we will have access to build info in those modules. In
      particularly build.asan flag is going to be used in buffer.lua in scope
      of #7327.
      
      Part of #7327
      
      NO_TEST=internal
      NO_DOC=internal
      NO_CHANGELOG=internal
      f58cc96f
    • Nikolay Shirokovskiy's avatar
      lua: provide whether ASAN build in tarantool.build.asan · 23012356
      Nikolay Shirokovskiy authored
      We already use this info in one of the test and going to use it more.
      
      Part of #7327
      
      @TarantoolBot document
      Title: new tarantool.build.asan flag
      
      It is `true` if `ENABLE_ASAN` build option is set and `false` otherwise.
      23012356
    • Vladimir Davydov's avatar
      vinyl: add statistic for total size of memory occupied by tuples · 6606d587
      Vladimir Davydov authored
      Vinyl tuples returned to the user are allocated with malloc. They may be
      pinned by Lua indefinitely. Currently, there's no way to figure out how
      much memory is occupied by these tuples. This commit adds a statistic to
      box.stat.vinyl() that accounts them.
      
      Closes #8485
      
      @TarantoolBot document
      Title: Document `memory.tuple` statistic of `box.stat.vinyl()`
      
      The new statistic shows the total size of memory in bytes occupied by
      Vinyl tuples. It includes cached tuples and tuples pinned by the Lua
      world.
      6606d587
  3. Aug 30, 2023
    • Vladimir Davydov's avatar
      box: fix schema upgrade replication · a5d7f342
      Vladimir Davydov authored
      Commit 97c2c9a4 ("box: disable DDL with old schema") added a check
      to the on-replace trigger installed on all system spaces that fails the
      operation if the schema version is outdated unless it's recovery time or
      the operation was issued by the fiber performing a schema upgrade.
      
      This new check breaks the replication use case:
      
       1. Tarantool binary is updated on all instances to a version that
          requires a newer schema - OK.
       2. box.schema.upgrade() is called on the master instance - OK.
       3. Operations performed by the master to upgrade the schema are
          replicated to the replicas - FAIL.
      
      To fix this issue, let's bypass the schema version check for applier
      fibers.
      
      Follow-up #7149
      Closes #9048
      
      NO_DOC=bug fix
      NO_CHANGELOG=unreleased
      a5d7f342
    • Yaroslav Lobankov's avatar
      Add testing for static build packaging · 292164ed
      Yaroslav Lobankov authored
      This patch finally brings desired testing for static build packages.
      
      How it works:
      
      In a few words, we have two workflow files: calling and callable. The
      callable workflow (static_build_pack_test_deploy.yml) is parametrized
      and contains all the logic with the building, testing, and deploying
      packages. It takes just two inputs: package platform and JSON matrix
      for testing. The calling workflow (packaging.yml) just runs callable
      one with specific parameters and contains all the logic related to
      triggering by events and concurrency.
      
      The static_build_pack_test_deploy.yml workflow consists of three jobs:
      `build`, `test`, and `deploy`. Artifacts between jobs are passed via
      the `upload-artifact` and `download-artifact` actions. The `test` job
      is a matrix one and verifies packages on provided Linux distros passed
      through input. After the testing is done, the `deploy` job is intended
      to upload packages to repositories on a tag push, which means release
      or pre-release.
      
      Note, for starting Docker containers to test packages we use PackPack
      images because they have almost all requirements to run tests.
      
      Follows up #8771
      Follows up #8840
      Follows up #8866
      
      Closes tarantool/tarantool-qa#322
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      292164ed
    • Igor Munkin's avatar
      luajit: bump new version · 88333d13
      Igor Munkin authored
      * Fix maxslots when recording BC_TSETM.
      * Fix TDUP load forwarding after table rehash.
      * Fix binary number literal parsing.
      * Fix maxslots when recording BC_VARG, part 3.
      * test: fix flaky <unit-jit-parse.test.lua> again
      * Fix predict_next() in parser.
      * Revert to trivial pow() optimizations to prevent inaccuracies.
      * Fix pow() optimization inconsistencies.
      * Improve assertions.
      * Remove pow() splitting and cleanup backends.
      * test: introduce `samevalues()` TAP checker
      * MIPS: Add MIPS64 R6 port.
      * DynASM/MIPS: Fix shadowed variable.
      * MIPS64: Fix register allocation in assembly of HREF.
      * Prevent integer overflow while parsing long strings.
      * Fix LJ_MAX_JSLOTS assertion in rec_check_slots().
      * Fix debug.getinfo() argument check.
      * ARM: Fix GCC 7 -Wimplicit-fallthrough warnings.
      * DynASM: Fix warning.
      * Fix GCC 7 -Wimplicit-fallthrough warnings.
      * Cleanup math function compilation and fix inconsistencies.
      * FFI: Eliminate hardcoded string hashes.
      * Windows: Add UWP support, part 1.
      * build: fix non-Linux/macOS builds
      * PPC: Add soft-float support to JIT compiler backend.
      * PPC: Add soft-float support to interpreter.
      * MIPS64: Add soft-float support to JIT compiler backend.
      * MIPS: Fix handling of spare long-range jump slots.
      * test: introduce mcode generator for tests
      * MIPS: Use precise search for exit jump patching.
      * sysprof: improve parser's memory footprint
      * tools: add execution permission to sysprof parser
      * sysprof: remove `split by vmstate` option
      
      Part of #8825
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      88333d13
  4. Aug 29, 2023
    • Yaroslav Lobankov's avatar
      ci: fix artifact names in Clang related workflows · 44bc3675
      Yaroslav Lobankov authored
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      44bc3675
    • Yaroslav Lobankov's avatar
      ci: bump Clang version to 16 in release build ASAN testing · e3ff3d5c
      Yaroslav Lobankov authored
      Run release build ASAN testing inside a Docker container created from
      the `tarantool/testing:ubuntu-jammy-clang16` Docker image with Clang 16
      installed.
      
      Closes tarantool/tarantool-qa#319
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      e3ff3d5c
    • Alexander Turenko's avatar
      test: fix flaky box.cfg() env vars test · dfa67ae6
      Alexander Turenko authored
      The test starts a child tarantool instance in the current working
      directory and run box.cfg(). The current working directory is a source
      directory. test-run.py runs many tests from it in parallel and in some
      circumstances it appears that the directory is locked by some other
      box.cfg() call from some other tarantool instance (maybe another test is
      doing the same).
      
      The test is rewritten to use a temporary directory for such runs.
      
      The test.interactive_tarantool helper is used for convenience instead of
      a popen wrapper that is written specifically for the test.
      
      NO_DOC=It is a fix of a test.
      NO_CHANGELOG=see NO_DOC
      dfa67ae6
    • Ilya Verbin's avatar
      test: disable ASAN for box/tx_man.test.lua · 8c7da931
      Ilya Verbin authored
      For some unknown reason ASAN crashes with SIGSEGV on this test during
      shutdown. See tarantool/tarantool-qa#324 for details.
      
      NO_DOC=test
      NO_CHANGELOG=test
      8c7da931
    • Oleg Chaplashkin's avatar
      test: bump test-run to new version · 6c825cb7
      Oleg Chaplashkin authored
      Bump test-run to new version with the following improvements:
      
      - Fix non-supported character in log file name [1]
      - luatest: bump luatest to 0.5.7-43-g251b35f [2]
      - luatest: bump luatest to 0.5.7-44-g2d51155 [3]
      
      [1] tarantool/test-run@c8465a5
      [2] tarantool/test-run@e84fd00
      [3] tarantool/test-run@22951e0
      
      NO_DOC=test
      NO_TEST=test
      NO_CHANGELOG=test
      6c825cb7
  5. Aug 28, 2023
    • Alexander Turenko's avatar
      Add @Totktonada as codeowner for config · 30fc5655
      Alexander Turenko authored
      I want to receive review request notifications about the changes and let
      people know who is responsible to accept them.
      
      NO_CHANGELOG=no code changes
      NO_TEST=no code changes
      NO_DOC=no code changes
      30fc5655
    • Gleb Kashkin's avatar
      test: add error rethrow to :read_response() in `it` · 793713e6
      Gleb Kashkin authored
      Before this patch interactive_tarantool `:read_response()` helper used
      to just deserialise Lua errors as tables like `[{error = 'msg'}]`.
      Now it checks if response is actually an error and rethrows it.
      This provides additional verification for many tests that use
      interactive_tarantool.
      
      This patch is a followup to commit 261a21bd ("test/config: adjust
      initial permissions in a test"). It provides additional output checks
      to prevent similar issues.
      
      Part of #8967
      
      NO_DOC=test helper update
      NO_CHANGELOG=test helper update
      NO_TEST=test helper update
      793713e6
    • Nikolay Shirokovskiy's avatar
      fiber: fix heap-buffer-overflow in fiber_stack_watermark_create · 750e4890
      Nikolay Shirokovskiy authored
      Fiber flags are initialized after fiber stack creation. As result
      currently check for custom stack in fiber_stack_watermark_create does
      not work. This leads to heap-buffer-overflow on putting watermark
      if custom stack size is less than FIBER_STACK_SIZE_WATERMARK.
      
      Close #9026
      
      NO_DOC=bugfix
      750e4890
    • Nikolay Shirokovskiy's avatar
      iproto: fix use-after-free in net_end_join · 4916389a
      Nikolay Shirokovskiy authored
      `msg` is used after it is freed in iproto_msg_delete.
      
      Close #9037
      
      NO_TEST=tested by ASAN
      NO_DOC=bugfix
      4916389a
  6. Aug 25, 2023
    • Mergen Imeev's avatar
      config: introduce audit options · 22e1532b
      Mergen Imeev authored
      This patch introduces all audit options.
      
      Closes #8861
      
      NO_DOC=Was already described before.
      22e1532b
    • Nikolay Shirokovskiy's avatar
      fiber: fix use-after-free on fiber destroy/recycle · c480a867
      Nikolay Shirokovskiy authored
      When fiber region is freed/destroyed and ENABLE_BACKTRACE is set then
      `fiber_on_gc_truncate` callback is called. At this time both `used`
      argument and `fiber->gc_initial_size` are equal to 0. Thus
      `fiber->first_alloc_bt` is accessed which is already freed.
      
      With a bad luck freeing fiber region can put slab back into slab arena.
      So writing after free can change memory used by another thread.
      
      Closes #9020
      
      NO_TEST=tested by ASAN
      NO_DOC=bugfix
      c480a867
    • Ilya Verbin's avatar
      lua: fix uninitialized var usage in luamp_encode_with_translation · bbfaef3b
      Ilya Verbin authored
      `*type_out` was set to uninitialized value for `field->type == MP_EXT`.
      This was introduced by commit 9f9142d6 ("box: cleanup on tuple encoding
      failure")
      
      Closes #9023
      
      NO_DOC=bugfix
      NO_CHANGELOG=not user-visible
      bbfaef3b
    • Sergey Bronnikov's avatar
      ci: fix an error in workflow · a6eeec3c
      Sergey Bronnikov authored
      Commit f14cb97d ("ci: update paths in a fuzzing workflow") breaks
      fuzzing workflow because one may only define one of `paths` and
      `paths-ignore` for a single event. The patch fixes that.
      
      NO_CHANGELOG=ci
      NO_DOC=ci
      NO_TEST=ci
      a6eeec3c
    • Mergen Imeev's avatar
      app: increase max configuration parameters length · f5511d45
      Mergen Imeev authored
      The maximum length of box.cfg{} string parameters is now 512 instead of
      256 before.
      
      NO_DOC=no need to document
      NO_TEST=will be added in EE
      f5511d45
    • Vladimir Davydov's avatar
      test: drop ok/is/isnt message from unit/xmalloc test · e13f2bf5
      Vladimir Davydov authored
      Just to demonstrate that the format argument of ok/is/isnt macros is now
      optional.
      
      NO_DOC=test
      NO_CHANGELOG=test
      e13f2bf5
    • Vladimir Davydov's avatar
      test: allow to omit message in ok/is/isnt unit test helpers · 62ae8bf3
      Vladimir Davydov authored
      It's really annoying to add a message after each check in a unit test.
      Let's make this optional. If the message is omitted, "line %d" will be
      used instead. Also, let's print the expression on failure because it
      may be useful if exact sources are unavailable.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      62ae8bf3
    • Vladimir Davydov's avatar
      test: add missing trailing semicolon after ok/is/isnt unit test helpers · 4871b697
      Vladimir Davydov authored
      The ok/is/isnt macros expand to {} so they may be used without
      a trailing semicolon. This is going to be fixed so let's add
      missing semicolons.
      
      NO_DOC=code cleanup
      NO_CHANGELOG=code cleanup
      4871b697
    • Vladimir Davydov's avatar
      test: eliminate code duplication in unit test helpers · a235080c
      Vladimir Davydov authored
      There's no need to duplicate all unit test helpers for TAP compatible
      tests. The only difference between them is that plan() prints the TAP
      version so let's do just that.
      
      NO_DOC=code cleanup
      NO_CHANGELOG=code cleanup
      a235080c
    • Nikita Zheleztsov's avatar
      test: fix flaky gh_7515_sync_node_sees_leader_hang · 44a6821f
      Nikita Zheleztsov authored
      The test checks, whether the instance notices leader hang during
      sync. For that it stops server2 and generates some data on the
      master (server1). The problem is, when connection to server3
      flakes, server1 resigns and becomes read only due to fencing
      enabled.
      
      Let's disable fencing for this test, as we don't want automatic
      leader resigning here.
      
      Closes tarantool/tarantool-qa#325
      
      NO_CHANGELOG=test
      NO_DOC=test
      44a6821f
  7. Aug 24, 2023
    • Ilya Verbin's avatar
      box: fix memory leak on error_unpack_unsafe() failure · b367fb98
      Ilya Verbin authored
      Memory is leaked in the following scenario:
      - MP_ERROR_STACK with 2 errors is passed to error_unpack_unsafe():
        1. A correct MP_MAP with MP_ERROR_* fields;
        2. Something unexpected, e.g. MP_INT;
      - This first call to mp_decode_error_one() allocates memory for the first
        error in error_build_xc() -> `new ClientError()`;
      - The second call to mp_decode_error_one() returns NULL, and
        error_unpack_unsafe() returns NULL too. Memory from the previous step
        is leaked.
      
      Closes #8921
      
      NO_DOC=bugfix
      b367fb98
    • Andrey Saranchin's avatar
      lua: introduce module trigger · 7f392d75
      Andrey Saranchin authored
      The patch introduces Lua module trigger, which allows to set, delete and
      call triggers from event registry.
      
      Closes #8656
      
      NO_DOC=later
      7f392d75
    • Andrey Saranchin's avatar
      lua: introduce func_adapter_lua_get_func method · dba0ec0d
      Andrey Saranchin authored
      Future module trigger will allow user to call triggers from Lua. We have
      function adapter to call an abstract function from any language, but
      it's convenient to call Lua functions directly when they are called from
      Lua, so let's add a method that allows to get underlying Lua function (or
      another callable object).
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      dba0ec0d
    • Andrey Saranchin's avatar
      core: introduce event subsystem · 5e890b6a
      Andrey Saranchin authored
      The patch introduces new event subsystem. This subsystem is designed to
      store user-defined triggers and has nothing in common with core triggers.
      
      Each trigger has its own name and is represented by func_adapter.
      Triggers are stored in events - named wrappers over rlist. Event objects
      are opaque, hence rlist field should not be used directly - event
      provides event_find_trigger, event_reset_triggers methods and
      event_trigger_iterator. Iterator provides stable iteration and all the
      non-deleted triggers will surely be traversed.
      
      On way to the goal this patch also fixes include list in func_adapter.h.
      
      Part of #8656
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      5e890b6a
    • Ilya Verbin's avatar
      box: validate key_def->part_count prior to memory allocation · ef9e3320
      Ilya Verbin authored
      part_count was checked in index_def_check(), which was called too late.
      Before that check:
      1. `malloc(sizeof(*part_def) * part_count)` can fail for huge part_count;
      2. key_def_new() can crash for zero part_count because of out of bound
         access in:
      
      NO_WRAP
         - #1 key_def_contains_sequential_parts (def=0x5555561a2ef0) at src/box/tuple_extract_key.cc:26
         - #2 key_def_set_extract_func (key_def=0x5555561a2ef0) at src/box/tuple_extract_key.cc:442
         - #3 key_def_set_func (def=0x5555561a2ef0) at src/box/key_def.c:162
         - #4 key_def_new (parts=0x7fffc4001350, part_count=0, for_func_index=false) at src/box/key_def.c:320
      NO_WRAP
      
      Closes #8688
      
      NO_DOC=bugfix
      ef9e3320
    • Vladislav Shpilevoy's avatar
      election: fix box.ctl.demote() nop in off-mode · 1afe2274
      Vladislav Shpilevoy authored
      box.ctl.demote() used not to do anything with election_mode='off'
      if the synchro queue didn't belong to the caller in the same term
      as the election state.
      
      The reason could be that if the synchro queue term is "outdated",
      there is no guarantee that some other instance doesn't own it in
      the latest term right now.
      
      The "problem" is that this could be workarounded easily by just
      calling promote + demote together.
      
      There isn't much sense in fixing it for the off-mode because the
      only reasons off-mode exists are 1) for people who don't use
      synchro at all, 2) who did use it and want to stop. Hence they
      need demote just to disown the queue.
      
      The patch "legalizes" the mentioned workaround by allowing to
      perform demote in off-mode even if the synchro queue term is old.
      
      Closes #6860
      
      NO_DOC=bugfix
      1afe2274
  8. Aug 23, 2023
    • Sergey Bronnikov's avatar
      ci: update paths in a publish module api doc workflow · 9af36ae1
      Sergey Bronnikov authored
      Module API documentation is described in Doxygen comments in C/C++ files
      in src/ directory. No sense to run this job for other changes. The patch
      changes paths in workflow accordingly.
      
      NO_CHANGELOG=ci
      NO_DOC=ci
      NO_TEST=ci
      9af36ae1
    • Sergey Bronnikov's avatar
      ci: update paths in a fuzzing workflow · f14cb97d
      Sergey Bronnikov authored
      Fuzzing is a heavyweight job, we can reduce a set of paths used for
      triggering this job and therefore reduce an overall time of testing in
      some cases.
      
      The patch replaces `.github/workflows/**` to
      `.github/workflows/fuzzing.yml` because workflows are independent and
      changes in other workflows does not affect fuzzing at all and patch adds
      Lua files (`**.lua`) to ignores because fuzzing focused on C/C++ code.
      
      NO_CHANGELOG=ci
      NO_DOC=ci
      NO_TEST=ci
      f14cb97d
    • Aleksandr Lyapunov's avatar
      sql: refactor update_view_references a bit · 392fe93a
      Aleksandr Lyapunov authored
      The function update_view_references is called when an SQL view
      is created or dropped. The goal of this function is to modify
      (increment or decrement) view_ref_count member of spaces that
      the view references.
      
      There were a several issues that deserves to be refactored:
      * By design in case of error it left the job partially done, so
        some space references were modified while some other - not.
        Although there was no bug since special steps were made in case
        of error, this pattern is inconvenient and should be avoided.
      * In case of error the failing space name was returned via special
        argument which is not flexible and even requires allocation.
      * Another argument - suppress_error - has actually never
        suppressed any error because the only case when an error could
        occur is creation of a view, which used suppress_error = false.
      * Fail of that function was not actually covered with tests.
      
      So this commit:
      * Makes the function to do all or nothing.
      * Forces the function to set diag by itself in case of error.
      * Removes suppress_error argument while adding several asserts.\
      * Adds a small test that fulfills coverage.
      
      NO_DOC=refactoring
      NO_CHANGELOG=reafactoring
      392fe93a
Loading