Skip to content
Snippets Groups Projects
  1. Jul 21, 2022
    • Nikita Zheleztsov's avatar
      net.box: add checking of options · d7da59b4
      Nikita Zheleztsov authored
      Currently net.box's methods doesn't check types of passed options.
      This can lead to Lua's internal errors, which are not self-explaining.
      
      Let's add this functionality and raise errors with meaningful messages
      in case of incorrect options.
      
      Closes #6063
      Closes #6530
      
      NO_DOC=Not a visible change
      d7da59b4
    • Nikita Zheleztsov's avatar
      test: refactor gh_6305_net_box_autocomplete · a5179846
      Nikita Zheleztsov authored
      Test is needed to be updated every time net.box's internals are patched
      (e.g. in case of adding new functions of deleting old ones), as the order
      of suggested autocomplete options changes.
      
      Let's replace `assert_equals` with `assert_items_equals`, which is not
      order dependent.
      
      NO_DOC=test
      NO_CHANGELOG=test
      a5179846
  2. Jul 19, 2022
    • Yaroslav Lobankov's avatar
      ci: support parallel cmake build in `test-jepsen` · 58ea8b5e
      Yaroslav Lobankov authored
      In order to support parallel cmake build in the `test-jepsen` target
      we need to run Jepsen testing in a `debian-buster` container instead
      of `debian-stretch` due to the old `cmake` version in it. The `cmake`
      utility started to support `--parallel` option since 3.12 version [1].
      
      In Debian Stretch `cmake` has version 3.7.
      In Debian Buster `cmake` has version 3.13.
      
      [1] https://cmake.org/cmake/help/latest/release/3.12.html#command-line
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      58ea8b5e
    • Yaroslav Lobankov's avatar
      ci: rm old mentions about .travis.mk in comments · 78a04083
      Yaroslav Lobankov authored
      A note for the `.gitlab.mk` makefile:
      
      Mentions about `.travis.mk` in `.gitlab.mk` file will be removed in
      the scope of refactoring it later.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      78a04083
    • Yaroslav Lobankov's avatar
      ci: drop .travis.mk makefile in favour of .test.mk · 316fc1db
      Yaroslav Lobankov authored
      Now it's time to drop the `.travis.mk` makefile because `.test.mk` has
      fully replaced it.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      316fc1db
    • Yaroslav Lobankov's avatar
      ci: use targets from .test.mk in workflows · 95b4dd07
      Yaroslav Lobankov authored
      A note for arm64 OSX workflows:
      
      The `arch -arm64` command prior to calling the makefile was omitted
      because now the GitHub Actions agent has a build for arm64 machines.
      Hence the agent is able to run without Rosetta 2 binary translator.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      95b4dd07
    • Yaroslav Lobankov's avatar
      ci: add actions with installing build/test deps · 0dbdd420
      Yaroslav Lobankov authored
      This patch adds actions with installing deps for Debian, FreeBSD, and
      OSX operation systems. Also, it adds an action with installing deps for
      Jepsen testing. Debian, FreeBSD, and OSX actions should be used before
      running tests on the corresponding operation systems. Jepsen action
      should be used before running Jepsen tests.
      
      Reasoning:
      
      1. The makefile w/o targets with deps installation is OS-independent.
         It should work on all systems.
      
      2. Deps installation is more about env preparation rather than building
         and testing. Moreover, it is OS-dependent. So it's logical to take it
         outside the makefile into a GitHub action.
      
      3. If we need testing on other operation systems, for example, CentOS,
         we will just create a special action with deps installation for that.
         No changes for the makefile will be needed. All should work out of
         the box.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      0dbdd420
    • Yaroslav Lobankov's avatar
      ci: introduce .test.mk instead of .travis.mk · cbaf71ed
      Yaroslav Lobankov authored
      This patch introduces the new test makefile `.test.mk` instead of the
      ancient `.travis.mk`. The `.test.mk` file is a fully reworked version of
      `.travis.mk` with the huge refactoring and contains only those targets
      that are indeed used.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      cbaf71ed
    • Timur Safin's avatar
      datetime: datetime_unpack test · 07eaf61a
      Timur Safin authored
      Created simplistic unit test for checking boundary conditions
      for `datetime_unpack` function.
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      07eaf61a
    • Timur Safin's avatar
      datetime: fuzzer for messagepuck decoder · 9bc2a20f
      Timur Safin authored
      Introduced fuzzing testing for datetime decoder.
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      9bc2a20f
    • Timur Safin's avatar
      datetime: validate unpacked data · fda1f6f5
      Timur Safin authored
      Introduced few checks of validity of incoming data
      at the moment messagepack is converted to datetime.
      
      - we check that size is either big enough for single
        epoch or whole record;
      - and we check range of data in decoded fields.
      
      Fixes #6723
      
      NO_DOC=bugfix
      NO_TEST=see the other commit for fuzzer
      fda1f6f5
    • Timur Safin's avatar
      datetime: refactor NANOS_PER_SEC · 9efc75b8
      Timur Safin authored
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      9efc75b8
    • Rianov Viacheslav's avatar
      lua: fix sigint behavior in multiline input mode · e8abbcbc
      Rianov Viacheslav authored
      This patch fixes the ctrl+c behavior in multiline mode.
      The old behavior was ignoring sigint in this mode.
      The new behavior is quitting this mode and printing the new
      default prompt.
      Added a new test for covering the mentioned case.
      
      Follows up #2717
      Closes #7109
      
      NO_DOC=BUGFIX
      e8abbcbc
  3. Jul 15, 2022
    • Vladimir Davydov's avatar
      memtx: refactor tuple read view API · b87ef00b
      Vladimir Davydov authored
      This commit replaces enter/leave_delayed_free_mode() MemtxAllocator
      methods with open/close_read_view(). The open_read_view() method
      returns a pointer to the new read view, which is supposed to be pssed
      to close_read_view() when the read view is no longer needed. The new
      API will allow us to optimize garbage collection, see #7185.
      
      Currently, open_read_view() always returns nullptr and both ReadView
      and memtx_read_view_opts are empty structs. We will extend them in
      the future.
      
      Closes #7364
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      b87ef00b
    • Vladimir Davydov's avatar
      trivia: add std::get by type · ebf71a35
      Vladimir Davydov authored
      std::get by type is unavailable in C++11. Implement our own version.
      Let's name it util::get and put it in trivia/tuple.h.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      ebf71a35
    • Vladimir Davydov's avatar
      memtx: use stailq instead of lifo for tuple garbage collection list · 3ddb5920
      Vladimir Davydov authored
      To immediately delete tuples that were allocated before a read view was
      created, but are not visible from the read view (see #7185), we will
      need to store them in per-read view garbage collection lists. We will
      also need to merge those lists. With lifo, merging is impossible. Let's
      switch to stailq, which allows this operation (see stailq_concat).
      
      Note, since the min value of box.cfg.slab_alloc_granularity is 4, we
      align memtx_tuple by 4 bytes.
      
      Needed for #7185
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      3ddb5920
    • Vladimir Davydov's avatar
      memtx: ignore slab_alloc_granularity < machine word size · ccfef5fd
      Vladimir Davydov authored
      box.cfg.slab_alloc_granularity sets the min alignment used for struct
      memtx_tuple. By default, it's set to the machine word size (8 bytes
      typically), but it can be set to 4 bytes to reduce memory usage.
      
      The problem is that we add memtx tuples to a garbage collection list
      using the intrusive list pattern (basically, store a pointer to the next
      entry in the first 8 bytes of the `memtx_tuple` struct). If
      `box.cfg.slab_alloc_factor` is set to 4, the pointer will be unaligned
      while the compiler will generate the machine code assuming it is
      naturally aligned. For some architectures (x86), this works fine, but
      for others (ARM), this may result in a runtime failure.
      
      To fix this issue, we need to instruct the compiler that the list
      pointer stored in `memtx_tuple` may be unaligned.
      
      For now, let's silently ignore granularity < the machine word size,
      because we're planning to switch from lifo to stailq for memtx garbage
      collection lists, which would result in explicit compiler warnings.
      
      See #7422
      Needed for #7185
      
      NO_DOC=invisible to the user
      NO_CHANGELOG=invisible to the user
      NO_TEST=will be added later when the bug is fixed
      ccfef5fd
    • Vladimir Davydov's avatar
      memtx: drop memtx_engine_free_mode · 0aab49f0
      Vladimir Davydov authored
      It isn't needed - we can check if delay_free_mode > 0 to figure our if
      we are in the delayed free mode.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      0aab49f0
    • Vladimir Davydov's avatar
      test: add unit test for MemtxAllocator · 0381159c
      Vladimir Davydov authored
      Checking #7185 with functional tests is complicated. Let's add a unit
      test for this.
      
      Needed for #7185
      
      NO_DOC=test
      NO_CHANGELOG=test
      0381159c
    • Alexander Turenko's avatar
      static-build: bump openssl from 1.1.1n to 1.1.1q · fd35f8c7
      Alexander Turenko authored
      Just regular update to bring openssl security fixes into tarantool.
      
      Added a patch for Mac OS to fix a build failure, see
      https://github.com/openssl/openssl/issues/18720.
      
      Changelog: https://www.openssl.org/news/cl111.txt
      Vulnerabilities: https://www.openssl.org/news/vulnerabilities.html
      
      NO_TEST=security update of a dependency
      NO_DOC=security update of a dependency
      fd35f8c7
  4. Jul 14, 2022
    • Nikolay Shirokovskiy's avatar
      console: use streams in binary console · 29542070
      Nikolay Shirokovskiy authored
      Now we can make interactive transactions like in case of
      local or txt remote consoles if peer supports streams.
      
      Closes #7413
      
      NO_DOC=minor change
      29542070
    • Nikolay Shirokovskiy's avatar
      console: rollback txn on eval error if begin was in expr · b013233f
      Nikolay Shirokovskiy authored
      The issue is if in binary remote console a error is thrown in expression
      like "box.begin() error('something') box.commit()" then it is
      overwritten by iproto check for active transactions at the end of eval.
      The solution is to rollback active transaction in this case in console
      code before iproto check.
      
      Let's also assert in tests a behaviour that after successful transaction
      begin it stays active in next evaluations (successful or not) until
      explicit rollback.
      
      Closes #7288
      
      NO_DOC=minor change
      b013233f
    • Nikolay Shirokovskiy's avatar
      test: fix luatest server to work after chdir · b5e59bd9
      Nikolay Shirokovskiy authored
      Starting server is failed after chdir(3) for example if we call
      box.cfg{} with work_dir set.
      
      NO_DOC=fix testing harness
      NO_TEST=fix testing harness
      NO_CHANGELOG=fix testing harness
      b5e59bd9
    • Nikolay Shirokovskiy's avatar
      test: rename socketdir to vardir in luatest server · 42727489
      Nikolay Shirokovskiy authored
      We keep per server directories there too, not only socket files.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      42727489
    • Nikolay Shirokovskiy's avatar
      console: put console EOF check in single place · 0a0d6c47
      Nikolay Shirokovskiy authored
      For the client_print there are also other checks and actions but they
      are unnecesessary.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      0a0d6c47
    • Nikolay Shirokovskiy's avatar
      test: fix box/transaction.test.lua · 6a508fc3
      Nikolay Shirokovskiy authored
      The ; separator was missed after select (check the first hunk) and as
      a result the rollback was not executed. So the next test was failed merely
      due to the unfinished transaction of the previous test.
      
      We also need to fix the third test to use a different tuple.
      Also fix a couple of comments.
      
      NO_DOC=test fix
      NO_TEST=test fix
      NO_CHANGELOG=test fix
      6a508fc3
    • Ilya Verbin's avatar
      lua-yaml: encode malformed error messages in base64 · 8f799cd8
      Ilya Verbin authored
      If MP_ERROR contains an error message with invalid UTF-8 sequences,
      encode it in Base64 to obtain printable string.
      
      Closes #6781
      Closes #6934
      
      NO_DOC=bugfix
      8f799cd8
    • Ilya Verbin's avatar
      console: handle internal.format_lua exceptions · e17515d3
      Ilya Verbin authored
      Currently the call to internal.format_lua in format_lua_value is covered
      by pcall only for `status == true`, however the opposite case also can
      raise an exception. This patch adds the missed exception handling.
      
      Part of #6781
      Part of #6934
      
      NO_DOC=bugfix
      NO_CHANGELOG=minor bug
      NO_TEST=not possible to trigger format_lua failure for !status
      e17515d3
    • Ilya Verbin's avatar
      console: raise error if lua_encode failed · 69852e1f
      Ilya Verbin authored
      lua_encode can raise an exception (e.g. from luaL_checkfield) or return
      `nil, err` result. Handle it in lbox_console_format_lua, similar to
      lbox_console_format_yaml.
      
      When fixed, it will expose another issue - nil value can not be
      serialized:
      
      ```
      tarantool> \set output lua
      true;
      tarantool> 1, nil, 2
      1, {error = "console: exception while formatting the output:
          \"serializer: unexpected data (nd.field.size 0 nd.field.type 5)\""}, 2;
      tarantool>
      ```
      
      Fix this too.
      
      Part of #6781
      Part of #6934
      
      NO_DOC=bugfix
      NO_TEST=not a visible change
      NO_CHANGELOG=not a visible change
      69852e1f
    • Ilya Verbin's avatar
      console: handle internal.format_yaml exceptions · 405bfe40
      Ilya Verbin authored
      Currently the call to internal.format_yaml in output_handlers["yaml"]
      is covered by pcall only for `status == true`, however the opposite
      case also can raise an exception. This patch adds the missed exception
      handling. Two distinct calls are required, because it is not possible to
      assing variadic arguments (...) to a variable if some of them is nil.
      If the first call fails, internal.format_yaml will be called for the
      second time. Hopefully, it will not fail during formatting of the error
      message received from libyaml.
      
      Before:
      ```
      tarantool> require('net.box').self:call('\x80')
      LuajitError: builtin/box/console.lua:710: expected SCALAR, SEQUENCE-START,
          MAPPING-START, or ALIAS fatal error, exiting the event loop
      ~/test$ echo $?
      0
      ~/test$
      ```
      
      After:
      ```
      tarantool> require('net.box').self:call('\x80')
      ---
      - error: 'console: an exception occurred when formatting the output:
          expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS'
      ...
      
      tarantool>
      ```
      
      Part of #6781
      Part of #6934
      
      NO_DOC=bugfix
      NO_TEST=see later commits
      NO_CHANGELOG=see later commits
      405bfe40
    • Ilya Verbin's avatar
      console: check lua_yaml_encode error in lbox_console_format_yaml · c84cdc73
      Ilya Verbin authored
      This patch adds a missed check for `lua_yaml_encode` return value,
      similar to the check in `console_dump_plain`. Now Lua error is raised
      if YAML encoding failed for any reason (e.g. OOM or formatting error).
      
      Before:
      ```
      tarantool> box.error.new(box.error.ILLEGAL_PARAMS, '\x80')
      ~/test$ echo $?
      0
      ~/test$
      ```
      
      After:
      ```
      tarantool> box.error.new(box.error.ILLEGAL_PARAMS, '\x80')
      ---
      - error: 'console: an exception occurred when formatting the output:
          expected SCALAR, SEQUENCE-START, MAPPING-START, or ALIAS'
      ...
      
      tarantool>
      ```
      
      Part of #6781
      Part of #6934
      
      NO_DOC=bugfix
      NO_TEST=see later commits
      NO_CHANGELOG=see later commits
      c84cdc73
    • Serge Petrenko's avatar
      box: fix finding snapshot in upgrade script · 30db3c23
      Serge Petrenko authored
      The upgrade script first tries to determine if the node is booted from
      old snaps not recoverable on current Tarantool versions. If this is the
      case, it sets up special triggers so that snaps are automatically
      converted to a suitable format.
      
      This happens before box.cfg{}, so the workdir is not set at this point
      in time, and the upgrade script should take configured work_dir into
      account explicitly. Fix this.
      
      Closes #7232
      
      NO_DOC=bugfix
      30db3c23
  5. Jul 12, 2022
    • Mergen Imeev's avatar
      sql: introduce OpenSpace opcode · 49109769
      Mergen Imeev authored
      Prior to this patch, some opcodes could use a pointer to struct space
      that was set during parsing. However, the pointer to struct space is not
      really something that defines spaces. A space can be identified by its
      ID or name. In most cases, specifying space by pointer works fine, but
      truncate() changes the pointer to space, resulting in a sigfault for
      prepared statements using the above opcodes. To avoid this problem, a
      new opcode has been introduced. This opcode uses the space ID to
      determine the pointer to the struct space at runtime and stores it in
      the MEM, which is later used in the mentioned opcodes.
      
      Closes #7358
      
      NO_DOC=bugfix
      49109769
    • Mergen Imeev's avatar
      sql: drop unused opcode IteratorReopen · 1c9599be
      Mergen Imeev authored
      Opcode IteratorReopen is not used and should be dropped.
      
      Part of #7358
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      1c9599be
    • Vladimir Davydov's avatar
      memtx: drop infrastructure for allocating memory chunks · 36eeca7c
      Vladimir Davydov authored
      We used to use it for allocating functional keys, but now we allocate
      those as tuples. Let's drop the legacy infrastructure and make alloc and
      free MemtxAllocator methods private.
      
      Follow-up #7376
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      36eeca7c
    • Vladimir Davydov's avatar
      memtx: allocate functional index key parts as tuples · 6cd463e1
      Vladimir Davydov authored
      Functional index keys are allocated and freed with MemtxAllocator's
      alloc and free methods. In contrast to tuples, which are allocated and
      freed with alloc_tuple and free_tuple, freeing a functional index key
      happens immediately, irrespective of whether there's a snapshot in
      progress or not. It's acceptable, because snapshot only uses primary
      indexes, which can't be functional. However, to reuse the snapshot
      infrastructure for creating general purpose user read views, we will
      need to guarantee that functional index keys stay alive until all read
      views using them are closed.
      
      To achieve that, this commit turns functional index keys into tuples,
      which automatically makes them linger if there's an open read view.
      We use the same global tuple format for allocating functional keys,
      because the key format is checked in key_list_iterator_next.
      
      Closes #7376
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      6cd463e1
    • Alexander Turenko's avatar
      http client: return back CA certificates search · 9d620bd0
      Alexander Turenko authored
      Without CA certificates the HTTP client will unable to verify server's
      certificate, so the only way to perform an HTTPS request would be use
      the `verify_peer = false` option -- disable certificate validation at
      all.
      
      The runtime search of system CA bundle/certificates was unintentionally
      disabled in 2.10.0 (PR #7119). The patch enabled is back.
      
      The main motivation behind the runtime search is difference in paths on
      different systems. Since we ship Tarantool Enterprise Edition as
      executable with ability to run on different Linux distributions, we
      can't choose one particular path at build time. See details in #5746.
      
      The `CURL_CA_BUNDLE_SET` and `CURL_CA_PATH_SET` options were removed,
      because they are not 'real' curl configuration options, but rather
      cached values to don't repeat file/directory search at re-configuration.
      It looks as internal logic of Curl's CMake script.
      
      NO_DOC=Lack of proper HTTPS support is definitely broken behavior, there
             is no sense to document it or the opposite.
      NO_TEST=A simple test would require to send a request to external host.
              It would not work without internet connection or in a sandbox.
              Such test also would be unstable and would fail from time to
              time due to network conditions. I verified the patch manually. I
              have an idea to add more thorough http client testing later.
      
      Fixes #7372
      9d620bd0
  6. Jul 11, 2022
    • Ilya Verbin's avatar
      core: get rid of fiber_set_cancellable in latch.h · 5404abaa
      Ilya Verbin authored
      fiber_wakeup has been adapted to spurious wakeups, so this protection
      is no longer needed.
      
      Part of #7166
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      5404abaa
    • Ilya Verbin's avatar
      core: fix potential misorder of latch waiters · 294e8379
      Ilya Verbin authored
      Currently the latch doesn't guarantee the order in which it is acquired
      by fibers, which requested it. E.g. it is possible to wake up spuriously
      a fiber which is yielding in the latch_lock, it will be removed from
      l->queue by fiber_make_ready, then it will be inserted to l->queue
      again, but for this time, to the head of the list instead of its
      original place in the queue.
      
      Fix this by using latch_waiter structure, which is linked into l->queue.
      
      Part of #7166
      
      @TarantoolBot document
      Title: Update box_latch_lock description
      Since: 2.11
      
      Add "Locks are acquired in the strict order as they were requested." to
      the box_latch_lock description in C API reference - Module latch.
      294e8379
  7. Jul 08, 2022
    • Yaroslav Lobankov's avatar
      build: fix error on FreeBSD 13.1 · 15b823f0
      Yaroslav Lobankov authored
      This patch fixes the following error while tarantool build on
      FreeBSD 13.1:
      
          [100%] Linking CXX executable watcher.test
          ld: error: undefined symbol: set_sigint_cb
          >>> referenced by console.c:261 (lua/console.c:261)
          >>>               console.c.o:(lbox_console_readline) in
                              archive ../../src/box/libbox.a
          >>> referenced by console.c:342 (lua/console.c:342)
          >>>               console.c.o:(lbox_console_readline) in
                              archive ../../src/box/libbox.a
          >>> referenced by console.c:352 (lua/console.c:352)
          >>>               console.c.o:(lbox_console_readline) in
                              archive ../../src/box/libbox.a
          >>> referenced 1 more times
          c++: error: linker command failed with exit code 1
              (use -v to see invocation)
          gmake[2]: *** [test/unit/CMakeFiles/watcher.test.dir/build.make:152:
              test/unit/watcher.test] Error 1
          gmake[1]: *** [CMakeFiles/Makefile2:10528:
              test/unit/CMakeFiles/watcher.test.dir/all] Error 2
          gmake: *** [Makefile:156: all] Error 2
      
      NO_DOC=minor changes
      NO_TEST=minor changes
      NO_CHANGELOG=minor changes
      15b823f0
Loading