Skip to content
Snippets Groups Projects
  1. May 11, 2022
    • Yaroslav Lobankov's avatar
      ci: add performance test reporting for sysbench · e2803cb1
      Yaroslav Lobankov authored
      Since analyzing the performance test results via the workflow artifacts
      is quite inconvenient (we need to download the artifact, unpack it, look
      through results), we need a more human approach to see the test report.
      Moreover, it would be cool to see the performance difference between the
      commits.
      
      So this patch adds reporting the performance test results to MyTeam chat
      and mailing list. Furthermore, the comparison of test results is added
      to the report. Seeing the test report in the chat or mail is very simple
      and convenient, plus forever results history as a bonus.
      
      The test report looks something like this:
      
          # Curr:
          #   branch:  add-feature-y
          #   build:   1.2.3-42-g6c71c3901
          #   summary: Add feature Y
          #   machine: x86_64
          #   distrib: ce
          #   gc64:    false
          # Prev:
          #   branch:  add-feature-x
          #   build:   1.2.3-41-g5271240ea
          #   summary: Add feature X
          #   machine: x86_64
          #   distrib: ce
          #   gc64:    false
          +----------------------+----------+----------+----------+
          |Sysbench              | Curr(rps)| Prev(rps)|     Ratio|
          +----------------------+----------+----------+----------+
          |oltp_delete           | 11408.000| 11532.000|     0.989|
          |oltp_insert           | 31455.000| 31322.000|     1.004|
          |oltp_point_select     | 65031.000| 66486.000|     0.978|
          |oltp_read_only        |  1732.000|  1735.000|     0.998|
          |oltp_read_write       |  1211.000|  1196.000|     1.013|
          |oltp_update_index     | 15580.000| 15570.000|     1.001|
          |oltp_update_non_index | 16129.000| 16093.000|     1.002|
          |oltp_write_only       |  4482.000|  4434.000|     1.011|
          |select_random_points  | 18525.000| 18517.000|     1.000|
          |select_random_ranges  |  4364.000|  4331.000|     1.008|
          +----------------------+----------+----------+----------+
          |gmean                 |  9223.593|  9220.182|     1.000|
          +----------------------+----------+----------+----------+
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      e2803cb1
    • Timur Safin's avatar
      datetime: handle timezone names in tz · 7036b55a
      Timur Safin authored
      Since recently we partially support timezone names (i.e. as
      abbreviations) so we may modify tz attribute support for
      datetime constructors or :set() operations.
      
      Closes #7076
      Relates to #7007
      
      @TarantoolBot document
      Title: datetime tz attribute
      
      Now `tz` attribute is properly handled in datetime value
      constructors or `:set{}` method modifiers.
      
      ```
      tarantool> T = date.new{year = 1980, tz = 'MSK'}
      ---
      ...
      
      tarantool> T.tzoffset
      ---
      - 180
      ...
      
      tarantool> T.tz
      ---
      - MSK
      ...
      tarantool> T = date.new{year = 1980, tzoffset = 180}
      ---
      ...
      
      tarantool> T.tzindex
      ---
      - 0
      ...
      
      tarantool> T.tz
      ---
      -
      ...
      
      tarantool> T.tzoffset
      ---
      - 180
      ...
      
      tarantool> T:set{tz = 'MSK'}
      ---
      ...
      
      tarantool> T.tz
      ---
      - MSK
      ...
      
      ```
      7036b55a
  2. May 06, 2022
    • Yaroslav Lobankov's avatar
      cleanup: delete old unused files · eaf96984
      Yaroslav Lobankov authored
      It looks like we have some files in the source tree that are not used
      anymore. So there is no sense to keep them.
      Files to be deleted: .appveyor.yml, Jenkinsfile, snapcraft.yaml.
      
      NO_DOC=files removal
      NO_TEST=files removal
      NO_CHANGELOG=files removal
      eaf96984
    • Vladimir Davydov's avatar
      test: fix luatest server cleanup · 00d6dfcd
      Vladimir Davydov authored
      It doesn't work, because fio.rmtree doesn't accept wildcards.
      Let's delete the whole server working tree on cleanup.
      
      NO_DOC=test
      NO_CHANGELOG=test
      00d6dfcd
    • Vladimir Davydov's avatar
      fiber_cond: add static initializer · 0016e80a
      Vladimir Davydov authored
      Similar to RLIST_HEAD_INITIALIZER and RLIST_HEAD. Needed to initialize
      a global fiber_cond variable without calling fiber_cond_create.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      0016e80a
    • Vladimir Davydov's avatar
      schema: space upgrade fixes · 44fadfc4
      Vladimir Davydov authored
       - Add func_id_by_name to box.internal. It will be used in EE.
       - Don't call box.internal.space.upgrade from box.schema.space.upgrade -
         we will redefine box.schema.space.upgrade in EE instead.
      
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      44fadfc4
    • Yaroslav Lobankov's avatar
      ci: follow-up improvements for gc64 builds · 63bedc76
      Yaroslav Lobankov authored
      This change contains the following improvements:
      
      1. Prettify view in the job list.
      
         Before:
           - centos_7 / centos_7 (GC64=OFF)
           - centos_7 / centos_7 (GC64=ON)
      
         After:
           - centos_7 / centos_7
           - centos_7 / centos_7 (gc64)
      
      2. Change possible values for the ${GC64} env variable to
         `true/false` instead of `ON/OFF` which is more traditional.
      
      3. Use `-DLUAJIT_ENABLE_GC64=ON` flag in the `debian/rules` and
         `rpm/tarantool.spec` files directly instead of providing it
         in the workflow files.
      
      Follows-up tarantool/tarantool-qa#159
      Follows-up tarantool/tarantool-qa#161
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      63bedc76
    • Vladimir Davydov's avatar
      changelogs: delete released changelogs · c99bbdd6
      Vladimir Davydov authored
       - Cleanup the unreleased directory: leave only fix-string-dequoting.md
         there, which was added after last branching (2.10).
       - Remove 2.10.0-beta1.md, 2.10.0-beta2.md, 2.8.1.md, because they are
         available at the corresponding branches.
      
      NO_DOC=changelogs
      NO_TEST=changelogs
      c99bbdd6
  3. May 05, 2022
  4. May 04, 2022
    • Nikita Pettik's avatar
      build: include extra directories for main.cc build · 3a0bb34d
      Nikita Pettik authored
      main.cc includes some headers from Tarantool Enterpise (to be more precise -
      flightrec.h). To build it we should include proper extra directories.
      
      NO_DOC=<Build fix>
      NO_CHANGELOG=<Build fix>
      NO_TEST=<Build fix>
      3a0bb34d
    • Nikita Pettik's avatar
      prbuf: fix "wrong argument type" error · f715e36a
      Nikita Pettik authored
      ffi.typeof() returns cdata, which in turn can't be concatenated with string
      without explicit call of `tostring`. Moreover, passing data of native Lua
      type would also generate wrong error. So let's simply raise general error
      without mentioning passed type.
      
      NO_DOC=<Internal fix>
      NO_CHANGELOG=<Internal fix>
      f715e36a
    • Nikita Pettik's avatar
      box/iproto: introduce reqstart to iproto_msg · da1892a9
      Nikita Pettik authored
      Previously we decided to pass `msg->p_inbuf->rpos` to flight recorder in
      order to dump request. However, this is wrong way to do it since `rpos`
      is moved only request has been processed, i.e. at the same moment several
      requests may have the same `rpos` value. To fix this let's save the start
      of unparsed request to `iproto_msg`. There's one detail that should be
      clarified to understand that in this case we'll always get valid
      pointer to ibuf. Imagine following state of ibuf:
      ```
      +_____________________
      |   |  |          |
      +---------------------
          ^  ^          ^
        RPOS R2       WPOS
         R1
      ```
      R1 is the first request in the buffer (i.e. rpos points to it);
      R2 is the second. One can argue that if R2 is processed faster than R1
      than during the dump of R1 its `reqstart` may point to the garbage.
      However, dump of request takes place at the bery beginning of request
      execution in TX thread. As far as messages are started to be processed
      exactly in the same way as they were received, then R1 will be always
      dumped before R2 and its further processing.
      
      Follow-up 247515e9
      
      NO_DOC=<No user visible changes>
      NO_TEST=<No functional changes>
      NO_CHANGELOG=<No functional changes>
      da1892a9
    • Yaroslav Lobankov's avatar
      ci: enable gc64 builds for opensuse systems · e73ca8a5
      Yaroslav Lobankov authored
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      e73ca8a5
    • Yaroslav Lobankov's avatar
      ci: enable gc64 builds for debian-like systems · 0d27174d
      Yaroslav Lobankov authored
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      0d27174d
    • artembo's avatar
      ci: use separate repo for gc64 packages · 16f47bdf
      artembo authored
      
      It looks like the simplest solution to deliver gc64 tarantool builds
      is to have a separate repo for it. So this patch removes old stuff
      related to the 'tarantool-gc64' package and adds the new logic to
      store gc64 packages in a separate repo.
      
      Closes: tarantool/tarantool-qa#161
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      
      Co-authored-by: default avatarYaroslav Lobankov <y.lobankov@tarantool.org>
      16f47bdf
    • Vladimir Davydov's avatar
      alter: fix space upgrade check · 34da4ce9
      Vladimir Davydov authored
      space_upgrade_check_alter() must be called after the new space
      definition is updated.
      
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      34da4ce9
  5. Apr 29, 2022
  6. Apr 28, 2022
    • Denis Smirnov's avatar
      sql: fix -Wnull-pointer-subtraction warning · 0289433a
      Denis Smirnov authored
      
      clang 13 includes a check for subtraction from NULL pointer which
      is considered UB: historically, we had an alignment checking macro
      which is affected by this. It seems like the intention of the
      macro's author was to implicitly cast the pointer being checked to
      uintptr_t without including stddef.h — replace this subtraction
      with an explicit cast.
      
      There is no way to set SQL_4_BYTE_ALIGNED_MALLOC, so the corresponding
      part of the code was removed. Now there are only 8 byte alignment
      assertions.
      
      NO_CHANGELOG=UB fix
      NO_DOC=UB fix
      NO_TEST=UB fix
      
      Co-authored-by: default avatarGeorgiy Lebedev <curiousgeorgiy@gmail.com>
      0289433a
    • Denis Smirnov's avatar
      sql: fix string dequoting · 76e09548
      Denis Smirnov authored
      
      Previously,
      
      select "t1"."a" from (select "a" from "t") as "t1";
      
      returned a result column name `t1` instead of `t1.a` because of
      incorrect work of a dequoting function. The reason was that
      previously sqlDequote() function finished its work when found the
      first closing quote.
      
      Old logic worked for simple selects where the column name doesn't
      contain an explicit scan name ("a" -> a).
      But for the sub-queries results sqlDequote() finished its work right
      after the scan name ("t1"."a" -> t1). Now the function continues its
      deqouting till it gets the null terminator at the end of the string.
      
      Closes #7063
      
      NO_DOC=don't change any public API, only a bug fix
      
      Co-authored-by: default avatarMergen Imeev <imeevma@gmail.com>
      76e09548
    • Vladimir Davydov's avatar
      errinj: add SPACE_UPGRADE_DELAY error injection · d772aef9
      Vladimir Davydov authored
      It will be used to stall background space upgrade in tests.
      
      While we are at it, move ERRINJ_TX_DELAY_PRIO_ENDPOINT to restore
      the sorted order of the error injection list.
      
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      d772aef9
    • Vladimir Davydov's avatar
      space_upgrade: pass format to space_upgrade_new · 5269f2ea
      Vladimir Davydov authored
      We need the new format to apply the upgrade function to a tuple.
      
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      5269f2ea
    • Vladimir Davydov's avatar
      test: check space upgrade stubs · 764d18ab
      Vladimir Davydov authored
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=test
      NO_CHANGELOG=test
      764d18ab
    • Vladimir Davydov's avatar
      test: add skip_if_enterprise luatest helper · d2b216e8
      Vladimir Davydov authored
      The helper skips a running test with luatest.skip_if in case executed by
      Tarantool Enterprise. It's better than checking the package directly in
      the test, because luatest.skip_if prints the reason why the test is
      skipped.
      
      NO_DOC=test
      NO_CHANGELOG=test
      d2b216e8
    • Vladimir Davydov's avatar
      func_cache: add space upgrade func holder type · 98bb3bd4
      Vladimir Davydov authored
      Online space upgrade applies a function to all tuples in a space.
      The function should be pinned so we need a new holder type.
      
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      98bb3bd4
    • Vladimir Davydov's avatar
      opt_def: make opt_def::enum_strs const char *const * · 624076ba
      Vladimir Davydov authored
      Enum keys are usually stored in a const array of const strings.
      Fix opt_def::enum_strs type accordingly.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      624076ba
    • Vladimir Davydov's avatar
      field_def: pass char ** instead of char * to field_def_array_decode · 78b0d83a
      Vladimir Davydov authored
      So that the caller can figure out the end of fields array without
      calling mp_next(). Needed for decoding space upgrade format.
      
      While we are at it, let's also
       - Replace MP_ARRAY assertion with diag_set. This makes the function
         easier to use when the MsgPack format isn't guaranteed to be
         MP_ARRAY.
       - Rearrange arguments so that [out] arguments are grouped together.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      78b0d83a
    • Yaroslav Lobankov's avatar
      ci: fix jepsen testing · 45bc9281
      Yaroslav Lobankov authored
      Fix the following error:
      
          {"badRequest": {"message": "The requested availability zone is not
          available", "code": 400}}
      
      NO_DOC=testing stuff
      NO_TEST=testing stuff
      NO_CHANGELOG=testing stuff
      45bc9281
    • Yaroslav Lobankov's avatar
      cmake: drop --vardir option when running tests · 96d7642e
      Yaroslav Lobankov authored
      This option very often makes tests fail due to the issue with the max
      length of socket paths (107 chars). Let's rely on the VARDIR environment
      variable or another default value (/tmp/t) when VARDIR is not defined.
      
      NO_DOC=testing stuff
      NO_TEST=testing stuff
      NO_CHANGELOG=testing stuff
      96d7642e
    • Yaroslav Lobankov's avatar
      luatest: use /tmp/t as default socket dir · c90a227e
      Yaroslav Lobankov authored
      This patch changes the length of default socket paths for luatest tests
      (when the VARDIR environment variable is not defined). It should reduce
      the chance for the issue with the max length of socket paths (107 chars)
      to happen.
      
      NO_DOC=testing stuff
      NO_TEST=testing stuff
      NO_CHANGELOG=testing stuff
      c90a227e
    • Yaroslav Lobankov's avatar
      ci: add VARDIR to PRESERVE_ENVVARS env variable · 7ae22101
      Yaroslav Lobankov authored
      This change adds VARDIR to the PRESERVE_ENVVARS environment variable
      to deliver it to 'packpack' docker containers while running packaging
      workflows.
      
      NO_DOC=ci
      NO_CHANGELOG=ci
      NO_TEST=ci
      7ae22101
    • Yaroslav Lobankov's avatar
      ci: change value for VARDIR env variable to /tmp/t · f483eac0
      Yaroslav Lobankov authored
      This patch changes the length of socket paths for tests by 2 chars.
      It should reduce the chance for the issue with the max length of socket
      paths (107 chars) to happen.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      f483eac0
    • Yaroslav Lobankov's avatar
      test-run: bump to new version · 60abb06d
      Yaroslav Lobankov authored
      Bump test-run to new version with the following changes:
      
      - Change default value for `vardir` option [1]
      - Read default value for `vardir` option from env [1]
      
      [1] tarantool/test-run#338
      
      NO_DOC=testing stuff
      NO_TEST=testing stuff
      NO_CHANGELOG=testing stuff
      60abb06d
  7. Apr 26, 2022
  8. Apr 25, 2022
    • Mons Anderson's avatar
      clock: return signed ints from time64 functions · fea8c8e4
      Mons Anderson authored
      Changed return value type in Lua for the following functions from
      `uint64_t` to `int64_t`:
      
      * clock.time64
      * clock.realtime64
      * clock.monotonic64
      * clock.process64
      * clock.thread64
      * fiber.time64
      * fiber.clock64
      
      Changed return value type in Module API for the following functions from
      `uint64_t` to `int64_t`:
      
      * fiber_time64
      * fiber_clock64
      
      Also enhanced clock.test.lua tap test with more testcases
      
      Due to unsigned values returned from time64 family function time
      calculations may lead to undesired value overflows. Having two 64-bit
      timestamps I cannot just subtract them without prior comparison. It is
      used often, for example when calculating time to a deadline (in queues
      or for expiration).
      
      Let's measure time, that pass between two sequential clock.time calls:
      
      ```lua
      print(clock.time() - clock.time()) -- -9.5367431640625e-07
      ```
      
      It's ok, with double type time return, but what about 64-bit integers?
      
      ```
      print(clock.time64() - clock.time64()) -- 18446744073709550616
      ```
      
      It returns weird integer overflow instead of expected `-1000`
      
      This is counterintuitive and gets conflicted with every time functions
      from system headers (which are always signed).
      
      If we consider some hypothetical code, that waits some time for a
      deadline, then instead of exiting loop, this code will go to "infinite"
      (actually 585 years) wait in the case when the deadline was passed.
      
      ```lua
      do
      local ts = clock.time64() + 1e9 -- some_timestamp_in_future, ex: now+1s
      -- ...
      local remaining = clock.time64() - ts
      -- ...
      if remaining > 0 then
          fiber.sleep( tonumber(remaining/1e9) )
      end
      print "1 second passed"
      end
      ```
      
      This is hard to notice (in real tests deadlines are always in the
      future), but in a real environment and under high load such cases occur.
      
      1. Since Lua functions will return ffi.typeof `ctype<int64_t>` instead
         of old `ctype<uint64_t>` any code, that uses _strict_ ffi type check
         will fail. But I hardly believe such code ever exists.
      2. Low-level Module API also changed, and code, which compiles with
         `-Werror` may also fail. This could be an issue and in such a case we
         may remove the update of module API change. Still, I consider this
         change reasonable, since all C-level API uses signed time values.
      
      @TarantoolBot document
      Title: Return type of time64 functions is changed
      
      These C functions in the public API used to return `uint64_t`, now they
      return `int64_t`: `clock_realtime64`, `clock_monotonic64`,
      `clock_process64`, `clock_thread64`, `fiber_time64()`,
      `fiber_clock64()`.
      
      These Lua functions used to return cdata of type `uint64_t`, now they
      return cdata of type `int64_t`: `fiber.time64()`, `fiber.clock64()`.
      fea8c8e4
    • Valeria Khokha's avatar
      doc: proofread changelogs/unreleased · 55cd16b5
      Valeria Khokha authored
      
      Fix wording, punctuation, and formatting.
      Fix dashes, trailing whitespaces, indentation, and file names.
      
      Needed for: #6430
      
      NO_DOC=changelog
      NO_TEST=changelog
      
      Co-authored-by: default avatarValeria Khokha <valeria.khokha@gmail.com>
      Co-authored-by: default avatarKseniia Antonova <xuniq.is.here@gmail.com>
      55cd16b5
    • Maxim Kokryashkin's avatar
      luajit: bump new version · c6b038b0
      Maxim Kokryashkin authored
      LuaJIT submodule is bumped to introduce the following changes:
      * GC64: disable sysprof support
      * build: make -DLUAJIT_DISABLE_SYSPROF work
      * test: disable sysprof C API tests with backtrace
      * tools: introduce parsers for sysprof
      * sysprof: introduce Lua API
      * memprof: add profile common section
      * core: introduce lua and platform profiler
      * memprof: move symtab to a separate module
      * core: separate the profiling timer from lj_profile
      * vm: save topframe info into global_State
      
      Within this changeset a parser for binary data dumped via the sampling
      profiler to Tarantool binary. It is a set of the following Lua modules:
      * sysprof/parse.lua: decode the sampling profiler event stream
      * sysprof/collapse.lua: collapse stacks obtained while profiling
      * sysprof.lua: Lua script and module to display data
      
      Besides, all new public Lua C API introduced within this changeset is
      added to extra/exports.
      
      Closes #781
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      c6b038b0
    • artembo's avatar
      ci: build Tarantool with enabled gc64 · 4af8e8d6
      artembo authored
      To make tarantool-gc64 available in repository we need to provide
      a product name in a package itself and in the repository.
      
      To build Tarantool with gc64 enabled, the flag -DLUAJIT_ENABLE_GC64
      needs to be passed to cmake flags. For this purposes GC64 env var
      was added to all workflows for Debian, Ubuntu, Centos and Fedora.
      
      This variable is processed to rename PRODUCT_NAME. If GC64 is
      eanbled, PRODUCT_NAME is `tarantool-gc64`, otherwise it stays
      `tarantool`, which goes to repository name and the package itself.
      Thus, Tarantool with gc64 enabled can be installed by
      `<packaen_manager> install -y tarantool-gc64`
      in the one single repository.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      
      Closes: tarantool/tarantool-qa#161
      4af8e8d6
    • Sergey Bronnikov's avatar
      third_party: update libcurl from 7.76.0 to 7.80.0 · a353ec91
      Sergey Bronnikov authored
      Changelog: https://curl.se/changes.html#7_80_0
      
      NO_DOC=libcurl submodule bump
      NO_CHANGELOG=libcurl submodule bump
      NO_TEST=libcurl submodule bump
      
      Closes: #6029
      Closes: https://github.com/tarantool/security/issues/10
      a353ec91
    • Nikita Pettik's avatar
      box: add OS stubs for flight recorder requests · 247515e9
      Nikita Pettik authored
      This patch introduces two stubs: one to log incoming requests and
      another one to log outgoing responses. Log itself takes place in TX
      thread, in function family `tx_process_*`. At the start of TX processing
      we are able to use raw msgpack from input buffer to log it as it was
      received. In the same way we can log raw msgpack for responses (before
      sending message to IProto thread).
      
      NO_DOC=<No user visible changes>
      NO_TEST=<No functional changes>
      NO_CHANGELOG=<No functional changes>
      247515e9
Loading