Skip to content
Snippets Groups Projects
  1. Oct 28, 2021
    • Mergen Imeev's avatar
      sql: fix possible undefined behavior during cast · 3c60e3fa
      Mergen Imeev authored
      This patch fixes possible undefined behavior during the implicit cast of
      INTEGER to DOUBLE. The problem is, if the INTEGER is close enough to
      2^64, it will be cast to 2^64 when it is cast to DOUBLE. Since we have a
      check for loss of precision, this will cause this DOUBLE to be cast to
      an INTEGER, which will result in undefined behavior since this DOUBLE is
      outside the range of INTEGER.
      3c60e3fa
    • Mergen Imeev's avatar
      sql: remove MEM_Zero flag from struct MEM · 401f72de
      Mergen Imeev authored
      This patch removes zeroblob optimization from SQL code. This
      optimization complicates the code, and there is almost no profit from
      it.
      
      Closes #6113
      Needed for #4145
      401f72de
    • Yaroslav Lobankov's avatar
      ci: add integration check for avro-schema module · 480afc0b
      Yaroslav Lobankov authored
      This patch extends the 'integration.yml' workflow and adds a new
      workflow call for running tests to verify integration between tarantool
      and the avro-schema module.
      
      Part of #5265
      Part of #6056
      Closes #6558
      480afc0b
  2. Oct 27, 2021
    • Yaroslav Lobankov's avatar
      ci: add integration check for metrics module · 7056061e
      Yaroslav Lobankov authored
      This patch extends the 'integration.yml' workflow and adds a new
      workflow call for running tests to verify integration between tarantool
      and the metrics module.
      
      Part of tarantool/tarantool#5265
      Part of tarantool/tarantool#6056
      Closes tarantool/tarantool#6526
      7056061e
    • Vladimir Davydov's avatar
      box: don't delete inprogress files during normal GC · f9bbfff9
      Vladimir Davydov authored
      *.inprogress files shouldn't be deleted during normal GC, because they
      may be in use. E.g. GC may happen to run while WAL is rotating xlog;
      if GC removes the transient xlog.inprogress file created by WAL (see
      xdir_create_xlog), WAL will fail to rename it and abort the current
      transaction.
      
      Initially, inprogress files were removed only after recovery. For this
      reason, xdir_collect_inprogress, which is used for deleting inprogress
      files, accesses FS directly, without COIO. This was changed by commit
      5aa243de ("recovery: build secondary
      index in hot standby mode") which moved xdir_collect_inprogress from
      memtx_engine_end_recovery to memtx_engine_collect_garbage so that a hot
      standby instance doesn't delete inprogress files of the master instance
      by mistake.
      
      To fix this issue, let's move xdir_collect_inprogress back where it
      belongs, to engine_end_recovery, and introduce a new callback for memtx
      to build its secondary keys before entering the hot standby mode -
      engine_begin_hot_standby.
      
      Let's also remove engine_collect_garbage from gc_init, which was added
      there by the aforementioned commit, probably to make tests pass.
      
      The bug was reported by the vinyl/deferred_delete test (#5089).
      
      Closes #6554
      f9bbfff9
  3. Oct 26, 2021
    • VitaliyaIoffe's avatar
      Migrate quorum.test.lua · c3e231d5
      VitaliyaIoffe authored
      Quorum.test.lua was migrated to luatest.
      It was divided into several tests.
      
      Delete quorum.test.lua, fix suite.ini in replication tests.
      
      Part of tarantool/test-run#304
      c3e231d5
    • VitaliyaIoffe's avatar
      Migrate msgpack.test.lua to luatest · 6e163c17
      VitaliyaIoffe authored
      Use luatest for checking correctness behavioral for encode/decode
      functions
      
      Delete msgpack.test.lua.
      
      Part of tarantool/test-run#304
      6e163c17
    • VitaliyaIoffe's avatar
      Provide helpers for luatest · 61bf2985
      VitaliyaIoffe authored
      The commit includes a few helpers files for luatest infrastructure.
      'Server' based on luatest Server with additional functionality like
      waiting Server for start, create Server's own env including box.cfg
      provided by test.
      
      Servers could be built and added in a cluster (cluster.lua file).
      
      Asserts.lua is additional checks, which are useful for many tests.
      
      Helpers are supportive function, could be also used for many tests.
      
      Part of tarantool/test-run#304
      61bf2985
  4. Oct 25, 2021
    • mechanik20051988's avatar
      iproto: implement timeout for iproto transactions · b9f7204b
      mechanik20051988 authored
      Same as for local transactions, timeout for iproto transactions
      was implemented. If timeout is not specified in client request
      it's sets to box.cfg.txn_timeout, which specified on server side.
      
      Closes #6177
      
      @TarantoolBot document
      Title: ability to set timeout for iproto transactions was implemented
      A new `IPROTO_TIMEOUT 0x56` key has been added. Currently it is used to
      set a timeout for transactions over iproto streams. It is stored in the
      body of 'IPROTO_BEGIN' request. If user want's to specify timeout using
      netbox (3s for example), he should use 'stream:begin({timeout = 3}).
      b9f7204b
    • mechanik20051988's avatar
      txn: implement timeout for transactions · a76eb6ef
      mechanik20051988 authored
      Client code errors or manual mistakes can create transactions that are
      never closed. Such transaction will work as a memory leak. Implement
      timeout for transactions after which they are rolled back.
      
      Part of #6177
      
      @TarantoolBot document
      Title: ability to set timeout for transactions was implemented
      Previously transactions are never closed until commit or rollback.
      Timeout for transactions was implemented after which they are rolled
      back. For these purpose, in `box.begin` the optional table parameter
      was added. For example if user want to start transaction with timeout
      3s, he should use `box.begin({timeout = 3})`. Also was implement new
      configuration option `box.cfg.txn_timeout` which determines timeout for
      transactions, for which the timeout was not explicitly set. By default
      this option is set to infinity (TIMEOUT_INFINITY = 365 * 100 * 86400).
      Also in C API was added new function to set timeout for transaction -
      'box_txn_set_timeout'.
      a76eb6ef
    • mechanik20051988's avatar
      txn: ignore changes after rollback on yield · 9c5d3ed7
      mechanik20051988 authored
      Previously, if a yield occurs for a transaction that does not
      support it, we roll back all its statements, but still process
      its new statements (they will roll back with each yield). Also,
      the transaction will be rolled back when a commit is attempted.
      Now we stop processing any new statements right after first yield,
      if transaction doesn't support it. This is done so that when we
      implement timeout for transactions, the rollback of the transaction
      at its expiration and at yield has the same behavior.
      
      Part of #6177
      9c5d3ed7
    • mechanik20051988's avatar
      txn: rework txn_on_yield trigger · 243c207a
      mechanik20051988 authored
      We are going to implement timeout for transaction, after which it
      will be rolled back. The timeout starts counting from the moment of
      the first yield. Previously `txn_on_yield` trigger installed only
      in case when engine does not support yields for transactions. Now it
      is installed when transaction is created, but if transaction supports
      yields it doesn't do anything. In the following patches, we will use
      this trigger to start the transaction timeout countdown.
      
      Part of #6177
      243c207a
  5. Oct 22, 2021
  6. Oct 21, 2021
    • Yaroslav Lobankov's avatar
      ci: switch to 'ubuntu-20.04-self-hosted' run label · b2108023
      Yaroslav Lobankov authored
      The idea of these changes is to run testing workflows on self-hosted
      ubuntu-20.04 machines instead of GitHub ubuntu-20.04 runners. It is
      planned to use GitHub ubuntu-20.04 runners only for integration testing
      of tarantool with a module/connector where ephemeral environments will
      allow us avoiding various cleanup pains on self-hosted runners like
      
          - if: always()
            run: sudo apt-get -y purge 'tarantool*'
      
      or
      
          - if: always()
            run: rm -rf .rocks
      
      or
      
          - if: always()
            run: rm -rf pytest-venv
      
      Also, switch workflows running on GitHub ubuntu-18.04 runners to
      self-hosted ubuntu-20.04 machines.
      b2108023
    • Georgy Moiseev's avatar
      debian: bump debian/compat to 10 · d3c3031b
      Georgy Moiseev authored
      Bump debian/compat to 10 since 9 is deprecated. Bump minimal required
      debhelper to 10 (except for Ubuntu Trusty and Xenial) since it is the
      recommended practice for compatibility level setup.
      
      Closes #6393
      d3c3031b
    • mechanik20051988's avatar
      test: decrease instance file name length · b1d91509
      mechanik20051988 authored
      The maximal unix socket file length is 108 symbols on Linux, so
      we should decrease file name to fit in this size.
      b1d91509
    • Vladimir Davydov's avatar
      iproto: fix kharon use after free · 2ad64078
      Vladimir Davydov authored
      The session can be closed while kharon is travelling between tx and
      iproto. If this happens, kharon shouldn't go back to iproto when it
      returns to tx, even if there are pending pushes, because the connection
      is going to be freed soon.
      
      Thanks to @Gerold103 for the simple fix.
      
      Closes #6520
      2ad64078
  7. Oct 20, 2021
    • Mergen Imeev's avatar
      sql: do not truncate DECIMAL in LIMIT and OFFSET · c72edb0d
      Mergen Imeev authored
      This patch removes the DECIMAL truncation in LIMIT and OFFSET, because
      according to the implicit casting rules, DECIMAL with digits after the
      decimal point cannot be implicitly cast to INTEGER.
      
      Closes #6485
      c72edb0d
    • Mergen Imeev's avatar
      sql: fix cast of small negative DECIMAL to INTEGER · 7bce8428
      Mergen Imeev authored
      This patch fixes an assertion when casting DECIMAL value less than 0 and
      greater than -1 to INTEGER.
      
      Part of #6485
      7bce8428
    • Mergen Imeev's avatar
      sql: fix truncation of DECIMAL in implicit cast · 18a5ee8e
      Mergen Imeev authored
      In case the DECIMAL value is implicitly cast to INTEGER during a search
      using an index, it was possible that DECIMAL would be truncated, which
      is not correct according to the implicit cast rules. This patch removes
      this truncation.
      
      Part of #6485
      18a5ee8e
    • mechanik20051988's avatar
      iproto: fix tarantool blindness in case of invalid listen uri. · 97be44af
      mechanik20051988 authored
      In case user enters invalid listen address, tarantool closes previous listen
      socket, but bind on invalid address fails also, so tarantool becames blind -
      no listening socket at all. This patch fixed this behaviour, now tarantool
      still listen old listen address.
      
      Closes #6092
      97be44af
    • mechanik20051988's avatar
      iproto: fix crash if box.cfg listen is woken up. · e7a9fd0b
      mechanik20051988 authored
      There was access to previously freed memory in case when `cbus_call`
      is interrupted: `cbus_call_msg` in iproto allocates on stack, and if
      `cbus_call` failed due to fiber cancelation or wake up, `cbus_call_msg`
      memory is released. But function called through cbus is still work in
      iproto thread and there will be an attempt to access this memory when
      this function in iproto thread finished it's work. This patch rework
      this behaviour, now before `cbus_call` we reset FIBER_IS_CANCELLABLE
      flag, to prevent fiber cancellation or it's wake up.
      
      Closes #6480
      e7a9fd0b
  8. Oct 18, 2021
    • Yaroslav Lobankov's avatar
      ci: change permissions in WS for reusable build · 16a484a3
      Yaroslav Lobankov authored
      For self-host runners, where the workspace with sources is saving
      between different workflow runs, it is needed to be sure that all file
      permissions are correct for running user to be sure that later call to
      sources checkout action by the same user won't fail on sources cleanup.
      
      Follows up tarantool/tarantool-qa#102
      16a484a3
    • Yaroslav Lobankov's avatar
      ci: add integration check for cartridge module · 41b3051f
      Yaroslav Lobankov authored
      This patch extends the 'integration.yml' workflow and adds a new
      workflow call for running tests to verify integration between tarantool
      and the cartridge module.
      
      Part of #5265
      Part of #6056
      Closes #6512
      41b3051f
  9. Oct 15, 2021
    • Yaroslav Lobankov's avatar
      ci: pass SHA to reusable_build.yml instead of ref · 2f2e2ff4
      Yaroslav Lobankov authored
      Imagine the following situation. One pushed a commit to the master
      branch, the 'integration.yml' workflow is triggered and starts to work.
      The first step of this workflow is to build tarantool from the pushed
      commit. Sometimes we have to wait for a free runner for some period of
      time because our resources are limited. One pushed a commit to the
      master branch one more time, but tarantool build for the previous
      commit hasn't started yet. When tarantool build from the previous
      commit starts, actually tarantool will be built from the last commit
      because ${{ github.ref }} is passed to the 'reusable_build.yml'
      workflow as an input parameter. So we need to pass ${{ github.sha }}.
      2f2e2ff4
    • Alexander Turenko's avatar
      test: update test-run (--tags) · 9ac85ace
      Alexander Turenko authored
      Now it is possible to mark test files with tags and run some group of
      tests using the `--tags` option.
      
      Usage:
      
       | ./test-run.py --tags foo
       | ./test-run.py --tags foo,bar app/ app-tap/
      
      Show the list of tags:
      
       | ./test-run.py --tags
       | ./test-run.py app-tap/ --tags
      
      See syntax and more details in
      https://github.com/tarantool/test-run/pull/314
      
      See the task description and possible usages in
      https://github.com/tarantool/test-run/issues/22
      Unverified
      9ac85ace
  10. Oct 14, 2021
    • Georgy Moiseev's avatar
      debian: promote a new maintainer for deb packages · 4bca4861
      Georgy Moiseev authored
      Promote @Totktonada to be a new maintainer for debian packages.
      
      Closes #6392
      4bca4861
    • Georgy Moiseev's avatar
      debian: bump tarantool-common dependency · 575203f1
      Georgy Moiseev authored
      Bump tarantool-common dependency to use luarocks 3.
      Prior to this patch, it was permitted to have
      new tarantool package (version >= 2.2.1) installed with
      pre-luarocks 3 tarantool-common package (version << 2.2.1).
      It caused rocks install to fail.
      
      Closes #5429
      575203f1
    • Timur Safin's avatar
      datetime: changelog for datetime module · c088b001
      Timur Safin authored
      Introduced module for timestamp and interval types support.
      
      Closes #5941
      c088b001
    • Timur Safin's avatar
      datetime - %f flag support in date:format() · 5c7dfa9f
      Timur Safin authored
      For the purposes of format support in datetime we need to modify
      standard strftime() implementation so it will be accepting %f flag
      we want to use for displaying of fractional part of seconds.
      
      Used Olson' strftime() implementation, simplified their code and
      header file, and adapted to work with our `struct datetime` data
      structure.
      We store timezone information there, seconds since epoch, and
      nanoseconds, thus we modified a way how those have being retrieved
      in the original implementation.
      
      We have also added missing `%f` and width modifiers support.
      
      ```
      tarantool> T:format('%d')
      ---
      - '14'
      ...
      
      tarantool> T:format('%3d')
      ---
      - 3d
      ...
      tarantool> T:format('%3f')
      ---
      - 371
      ...
      tarantool> T:format()
      ---
      - 2021-09-14T12:10:30.371895+0300
      ...
      tarantool> T:format('%FT%T.%f%z')
      ---
      - 2021-09-14T12:10:30.371895+0300
      ...
      ```
      
      Created detailed strftime formats test to cover all of known
      format flags.
      
      Part of #5941
      5c7dfa9f
    • Timur Safin's avatar
      build, lua: built-in module datetime · 43e10ed3
      Timur Safin authored
      Introduce a new builtin Tarantool module `datetime.lua` for timestamp
      and interval types support.
      
      New third_party module - c-dt
      -----------------------------
      
      * Integrated chansen/c-dt parser as 3rd party module to the
        Tarantool cmake build process;
      * We use tarantool/c-dt instead of original chansen/c-dt to
        have an easier cmake build integration, as we have added some
        changes, which provide cmake support, and allow to rename symbols
        if necessary (this symbol renaming is similar to that we see
        with xxhash or icu).
      
      New built-in module `datetime`
      ------------------------------
      
      * created a new Tarantool built-in module `datetime`, which uses
        `struct datetime` data structure for keeping timestamp values;
      * Lua module uses a number of `dt_*` functions from `c-dt` library,
        but they were renamed to `tnt_dt_*` at the moment of exporting
        from executable - to avoid possible name clashes with external
        libraries.
      
      * At the moment we libc `strftime` for formatting of datetime
        values according to flags passed, i.e. `date:format('%FT%T%z')`
        will return something like '1970-01-01T00:00:00+0000', but
        `date:format('%A %d, %B %Y')` will return 'Thursday 01, January 1970'
      
      * if there is no format provided then we use default
        `tnt_datetime_to_string()` function, which converts datetime
        to their default ISO-8601 output format, i.e.
        `tostring(date)` will return string like "1970-01-01T00:00:00Z"
      
      * There are a number of simplified interfaces
        - totable() for exporting table with attributes names as provided
          by `os.date('*t')`
        - set() method provides unified interface to set values using
          the set of attributes as defined above in totable()
      
      Example,
      
      ```
      local dt = datetime.new {
      	nsec      = 123456789,
      
      	sec       = 19,
      	min       = 29,
      	hour      = 18,
      
      	day       = 20,
      	month     = 8,
      	year      = 2021,
      
      	tzoffset  = 180
      }
      
      local t = dt:totable()
      --[[
      {
      	sec = 19,
      	min = 29,
      	wday = 6,
      	day = 20,
      	nsec = 123456789,
      	isdst = false,
      	yday = 232,
      	tzoffset = 180,
      	month = 8,
      	year = 2021,
      	hour = 18
      }
      --]]
      
      dt:format()   -- 2021-08-21T14:53:34.032Z
      dt:format('%Y-%m-%dT%H:%M:%S')   -- 2021-08-21T14:53:34
      
      dt:set {
      	usec      = 123456,
      
      	sec       = 19,
      	min       = 29,
      	hour      = 18,
      
      	day       = 20,
      	month     = 8,
      	year      = 2021,
      
      	tzoffset  = 180,
      
      }
      dt:set {
      	timestamp = 1629476485.124,
      
      	tzoffset  = 180,
      }
      
      ```
      
      Coverage is
      
      File                 Hits Missed Coverage
      -----------------------------------------
      builtin/datetime.lua 299  23     92.86%
      -----------------------------------------
      Total                299  23     92.86%
      
      Part of #5941
      
      @TarantoolBot document
      Title: Introduced a new `datetime` module for timestamp and interval support
      
      Create `datetime` module for timestamp and interval types support.
      It allows to create date and timestamp values using either object interface,
      or via parsing of string values conforming to iso-8601 standard.
      One may manipulate (modify, subtract or add) timestamp and interval values.
      
      Please refer to https://hackmd.io/@Mons/S1Vfc_axK#Datetime-in-Tarantool
      for a more detailed description of module API.
      43e10ed3
  11. Oct 12, 2021
    • Yaroslav Lobankov's avatar
      ci: add workflow for checking module integration · 8ac76fd6
      Yaroslav Lobankov authored
      For now, there is no testing for the tarantool project verifying its
      integration with different modules and connectors from the ecosystem.
      This is a quite huge gap in our CI system that is going to be covered
      by these changes.
      
      This patch introduces a couple of new workflow files that are named
      'integration.yml' and 'reusable_build.yml'. The main workflow is
      'integration.yml' that will run automatically per push to master and
      release branches. Also, this workflow can be run manually against a
      development branch. The 'reusable_build.yml' workflow is called by the
      main workflow and builds needed tarantool packages (at this moment for
      Ubuntu Focal Fossa only), then stores them as a build artifact. After
      that, the main workflow calls the 'reusable_testing.yml' workflow from
      a module project that tarantool is going to verify integration with.
      The testing workflow should download the tarantool build artifact and
      run tests against it.
      
      The basic scheme describing the verification process is represented
      below:
      
            integration.yml
                   |
                   |
                   v
           reusable_build.yml --> <artifact>
                   |                  |
                   |                  |
                   v                  |
          reusable_testing.yml <------+
                   |
                   |
                   v
                <result>
      
      For now, we start only with the vshard module. In the future, we are
      going to extend the module list incrementally.
      
      Part of #5265
      Part of #6056
      Closes #4972
      8ac76fd6
    • AnastasMIPT's avatar
      lua: fix bug with discarding arguments in box.func:call() · 298eebf7
      AnastasMIPT authored
      Fixes incorrect handling of variable number of arguments in box.func:call().
      
      Closes #6405
      298eebf7
    • Alexander Turenko's avatar
      github-ci: drop Ubuntu Trusty (14.04) support · cd675879
      Alexander Turenko authored
      Reasons, in short:
      
      * It reaches end of the standard support.
      * It looks barely usable with old ca-certificates due to Let's Encrypt
        certificate expire problem.
      * It has problems that block updating compat level for our *.deb
        packages.
      
      All details and links can be found in the linked issue.
      
      Fixes #6502
      cd675879
  12. Oct 11, 2021
  13. Oct 07, 2021
    • Nikita Pettik's avatar
      txm: rollback all statements related to space on alter · ce5752ce
      Nikita Pettik authored
      There was a bug that led to dirty read after space alter. For the
      simplicity sake imagine following setup:
      
      -- space 's' is empty
      tx1:begin()
      tx1('s:replace{2}')
      s:alter({format = format})
      s:select{}
      
      Last select returns tuple {2}, however transaction tx1 hasn't been
      committed. This happens due to the fact that during alter operation we
      create new space, swap all unchanged parts of old space and then delete
      old space. During removal of old space we also clean-up all stories
      related to it. In turn story destruction may make dirty tuple clean in
      case it remains in the index. In the previous implementation there was
      no removal of uncommitted tuples from corresponding indexes. So let's
      rollback all changes happened to the space right in time of alter. It is
      legal since DDL operation anyway aborts ALL other transactions.
      
      Closes #6318
      Closes #6263
      ce5752ce
    • Vladimir Davydov's avatar
      xrow: pass xrow_header to xrow_on_decode_error · 68ee11a7
      Vladimir Davydov authored
      Since all decoding functions except xrow_decode_header() decode a packet
      body, let's pass an xrow_header object to xrow_on_decode_error() instead
      of raw pointers to msgpack data. This will simplify packet body decoding
      functions a bit while in xrow_decode_header() we can use dump_row_hex()
      directly without hurting readability.
      
      While we are at it, make dump_row_hex() static, because it's not used
      outside its compilation unit.
      68ee11a7
Loading