Skip to content
Snippets Groups Projects
  1. Nov 08, 2021
  2. Nov 03, 2021
    • Yaroslav Lobankov's avatar
      ci: add integration check for mysql module · 71a789f0
      Yaroslav Lobankov authored
      This patch extends the 'integration.yml' workflow and adds a new
      workflow call for running tests to verify integration between tarantool
      and the mysql module.
      
      Part of #5265
      Part of #6056
      Closes #6577
      71a789f0
    • Mergen Imeev's avatar
      sql: introduce literals for DECIMAL · ea637984
      Mergen Imeev authored
      Part of #6356
      
      @TarantoolBot document
      Title: Literals for INTEGER, DECIMAL and DOUBLE
      
      The rules for parsing numeric values have changed:
      1) a value consisting of digits without decimal point and exponent will
      be parsed as INTEGER;
      2) a value consisting of digits and a decimal point will be parsed as
      DECIMAL;
      3) a value consisting of digits, containing an exponent and possibly
      containing a decimal point, will be parsed as DOUBLE.
      ea637984
  3. Nov 02, 2021
    • Alexander Turenko's avatar
      jepsen: fix race condition with apt's sources.list · 20e72426
      Alexander Turenko authored
      Jepsen testing starts one or several virtual machines and runs tarantool
      instances on them. The first (first important for us here) command on
      the virtual machine is `apt-get <...> update`: we should download
      packages list to allow Jepsen to install necessary dependencies.
      
      However we can access the virtual machine (using ssh) before it is fully
      initialized by the cloud-init script. In particular, the cloud-init
      script replaces apt's mirror list file (`/etc/apt/sources.list`).
      
      Normally we should call `apt-get <...> update` after the package list
      update, but here cloud-init races with the update command.
      
      In the bad case the commands are executed in the opposite order:
      
      * Terraform calls `apt-get <...> update`.
      * cloud-init replaces `/etc/apt/sources.list`.
      
      Now an attempt to install a package using apt-get will give the 'unable
      to locate package' error, because we have no packages list for the 'new'
      mirrors.
      
      The problem is nicely described in [1]. See also the linked issue for
      details.
      
      [1]: https://github.com/hashicorp/packer/issues/41#issuecomment-21288589
      
      Fixes https://github.com/tarantool/jepsen.tarantool/issues/87
      20e72426
    • Vladimir Davydov's avatar
      test: cleanup vinyl suite · a737f7e4
      Vladimir Davydov authored
       - Remove tests linked to fixed issues from the flaky list.
       - Remove filters needed to produce stable output when a test failed
         because of #5436.
      a737f7e4
    • Vladimir Davydov's avatar
      test: fix flaky app/fiber_channel test · 7c3f0b06
      Vladimir Davydov authored
      The test has three known failures:
      
       1. Happens, because the test doesn't wait for a cancelled fiber
          to exit. As a result, it keeps reading from the test channel.
          Fix this by waiting for the test fiber to be dead for sure.
          ```
          --- app/fiber_channel.result    Sun Apr 26 17:45:32 2020
          +++ var/050_app/fiber_channel.result    Tue May 12 05:10:14 2020
          @@ -200,381 +200,14 @@
           ...
           ch:count()
           ---
          -- 1
          +- 0
           ...
           ch:is_full()
           ---
          -- true
          +- false
           ...
           ch:is_empty()
           ---
          -- false
          +- true
           ...
           ch:get(box.info.pid) == box.info.pid
          ```
       2. The test writes 16 values to a table, but waits for only 15 values.
          Fix this and also cancel test fibers before proceeding to the next
          test case.
          ```
          --- app/fiber_channel.result	Fri Feb 21 11:21:32 2020
          +++ var/049_app/fiber_channel.result	Tue May 12 01:05:37 2020
          @@ -346,7 +346,6 @@
             - 42
             - 43
             - 44
          -  - 45
           ...
           ch = fiber.channel(1)
           ---
          ```
       3. The test uses a 100 iteration loop to wait for a condition to
          become true. If the host is busy, 100 may be not enough. Let's
          use test_run:wait_cond() instead - it's more reliable.
          ```
          --- app/fiber_channel.result	Fri Feb 21 12:08:07 2020
          +++ app/fiber_channel.reject	Fri May  8 08:20:52 2020
          @@ -487,7 +487,7 @@
           ...
           count > 2000, #test_res, test_res;
           ---
          -- true
          +- false
           - 10
          unit/fiber_channel.test                                         [ pass ]
           - - true
             - true
          ```
      
      Closes #4961
      7c3f0b06
    • Vladimir Davydov's avatar
      test: fix flaky box/gh-5998-one-tx-for-ddl-errinj test · 1e56b157
      Vladimir Davydov authored
      The test tries to build secondary indexes in two spaces in parallel and
      checks that only one of them succeeded. For some reason, the test
      assumes that only the first space can succeed, but sometimes the second
      space may get lucky. Let's fix the check to remove this assumption.
      1e56b157
    • Vladimir Davydov's avatar
      test: fix flaky box-tap/gh-5602-environment-vars-cfg test · 3bc17662
      Vladimir Davydov authored
      The test is flaky, because it tries to listen on a fixed TCP port, which
      may be busy. Let's use Unix sockets to fix the test.
      
      While we are at it, let's also remove useless creation of a temporary
      directory - the test copies a script there, but never uses it.
      3bc17662
    • Vladimir Davydov's avatar
      test: fix flaky xlog/panic_on_broken_lsn test · 4336265e
      Vladimir Davydov authored
      The test injects an error for a replication of a particular LSN. The
      problem there's a pending record about the new replica written to the
      _cluster space on join, which may not have been flushed to WAL by the
      time the test injects an error. Due to the race, the test may inject an
      error for the _cluster row instead of the test row, which breaks its
      expectations.
      
      ```
      [009] xlog/panic_on_broken_lsn.test.lua                               [ fail ]
      [009]
      [009] Test failed! Result content mismatch:
      [009] --- xlog/panic_on_broken_lsn.result       Tue Nov  2 13:29:53 2021
      [009] +++ var/rejects/xlog/panic_on_broken_lsn.reject   Tue Nov  2 13:29:59 2021
      [009] @@ -155,8 +155,8 @@
      [009]  ...
      [009]  (found:gsub('^.*, req: ', ''):gsub('lsn: %d+', 'lsn: <lsn>'))
      [009]  ---
      [009] -- '{type: ''INSERT'', replica_id: 1, lsn: <lsn>, space_id: 9000, index_id: 0, tuple:
      [009] -  [2, "v1"]}'
      [009] +- '{type: ''INSERT'', replica_id: 1, lsn: <lsn>, space_id: 320, index_id: 0, tuple:
      [009] +  [2, "34898d18-eed4-4f8a-97ff-4ffba7b42892"]}'
      [009]  ...
      [009]  test_run:cmd('cleanup server replica')
      [009]  ---
      [009]
      ```
      
      Fix this by flushing WAL before writing a broken row.
      
      Closes #4508
      4336265e
    • Yaroslav Lobankov's avatar
      ci: add integration check for 'checks' module · 7083b20b
      Yaroslav Lobankov authored
      This patch extends the 'integration.yml' workflow and adds a new
      workflow call for running tests to verify integration between tarantool
      and the 'checks' module.
      
      Part of #5265
      Part of #6056
      Closes #6562
      7083b20b
    • Vladimir Davydov's avatar
      net.box: add watchers support · eb7712e3
      Vladimir Davydov authored
      Part of #6257
      
      @TarantoolBot document
      Title: Document net.box watchers
      
      Using the new `watch` method of a net.box connection, one can subscribe
      to events broadcasted by a remote host. The method has the same syntax
      as the `box.watch()` function, which is used for subscribing to events
      locally. It takes a key name (string) to subscribe to and a callback to
      invoke when the key value is updated. It returns a watcher handle that
      can be used to unregister the watcher. Note, garbage collection of a
      watcher handle doesnt result in unregistering the watcher so it's okay
      to discard the result of `box.watch` if the watcher is never going to be
      unregistered.
      
      A watcher callback is first invoked unconditionally after the watcher
      registration. Subsequent invocations are triggered by `box.broadcast()`
      called on the remote host. A watcher callback is passed the name of the
      key the watcher was subscribed to and the current key value. A watcher
      callback is always executed in a new fiber so it's okay to yield inside
      it. A watcher callback never runs in parallel with itself: if the key
      to which a watcher is subscribed is updated while the watcher callback
      is running, the callback will be invoked again with the new value as
      soon as it returns.
      
      Watchers survive reconnect (see `reconnect_after` connection option):
      all registered watchers are automatically resubscribed as soon as the
      connection is reestablished.
      
      If a remote host supports watchers, the 'watchers' key will be set in
      connection's `peer_protocol_features`.
      
      Example usage:
      
       * Server:
         ```lua
         -- Broadcast value 123 for key 'foo'.
         box.broadcast('foo', 123)
         ```
       * Client:
         ```lua
         conn = net.box.connect(URI)
         -- Subscribe to updates of key 'foo'.
         w = conn:watch('foo', function(key, value)
             assert(key == 'foo')
             -- do something with value
         end)
         -- Unregister the watcher when it's no longer needed.
         w:unregister()
         ```
      eb7712e3
    • Vladimir Davydov's avatar
      iproto: add watchers support · 4be5de4c
      Vladimir Davydov authored
      Part of #6257
      
      @TarantoolBot document
      Title: Document IPROTO watchers
      
      There are three new commands to support asynchronous server->client
      notifications signaled with `box.broadcast()`:
      
       - `IPROTO_WATCH` (code 74). Registers a new watcher for the given
         notification key or acknowledges a notification if a watcher is
         already registered. The key name is passed in `IPROTO_EVENT_KEY`
         (code 0x56). The watcher will be notified unconditionally after
         registration and then every time the key is updated with
         `box.broadcast()` provided the last notification was acknowledged.
         The server doesn't reply to the request unless it fails to parse
         the packet.
      
       - `IPROTO_UNWATCH` (code 75). Unregisters a watcher registered for the
         given notification key. The key name is passed in `IPROTO_EVENT_KEY`
         (code 0x56). A server doesn't reply to the request unless it fails to
         parse the packet.
      
       - `IPROTO_EVENT` (code 76). Sent by the server to notify a client
         about a key update. The key name is passed in `IPROTO_EVENT_KEY`
         (code 0x56). The key data (optional) is passed in `IPROTO_EVENT_DATA`
         (code 0x57).
      
      When a connection is closed, all watchers registered for it are
      unregistered.
      
      Servers that support the new feature set the `IPROTO_FEATURE_WATCHERS`
      feature bit (bit 3) in reply to the `IPROTO_ID` command.
      4be5de4c
    • Vladimir Davydov's avatar
      box/lua: introduce box.watch and box.broadcast · 11f2d999
      Vladimir Davydov authored
      Part of #6257
      
      @TarantoolBot document
      Title: Document box.watch and box.broadcast
      
      `box.watch(key, func)` registers a watcher for the given key and returns
      a watcher handle, which can be used to unregister the watcher (by
      calling the `unregister` method). A key is an arbitrary string. It's
      possible to register more than one watcher for the same key. Note,
      garbage collection of a watcher handle doesnt result in unregistering
      the watcher so it's okay to discard the result of `box.watch` if the
      watcher is never going to be unregistered.
      
      `box.broadcast(key, value)` updates the value of the given key and
      signals all watchers registered for it.
      
      A watcher callback is first invoked unconditionally after the watcher
      registration. Subsequent invocations are triggered by `box.broadcast()`
      called on the local host. A watcher callback is passed the name of the
      key the watcher was subscribed to and the current key value. A watcher
      callback is always executed in a new fiber so it's okay to yield inside
      it. A watcher callback never runs in parallel with itself: if the key
      to which a watcher is subscribed is updated while the watcher callback
      is running, the callback will be invoked again with the new value as
      soon as it returns.
      
      `box.watch` and `box.broadcast` may be used before `box.cfg`.
      
      Example usage:
      
      ```lua
      -- Broadcast value 123 for key 'foo'.
      box.broadcast('foo', 123)
      -- Subscribe to updates of key 'foo'.
      w = box.watch('foo', function(key, value)
          assert(key == 'foo')
          -- do something with value
      end)
      -- Unregister the watcher when it's no longer needed.
      w:unregister()
      ```
      11f2d999
    • Vladimir Davydov's avatar
      box: implement watcher infrastructure · e8b9dffc
      Vladimir Davydov authored
      Part of #6257
      
      This commit introduces a C module that will later be used for
      implementation of the watchers API in both Lua and IPROTO.
      
      Key points:
       - A watcher callback can be registered for an arbitrary string key with
         box_register_watcher().
       - Watcher callbacks are invoked in the background by the worker fiber.
         If a watcher is asynchronous (WATCHER_RUN_ASYNC flag set), the worker
         fiber will invoke the callback in a new fiber.
       - A newly registered watcher callback is scheduled for execution
         unconditionally after registration, and then whenever box_broadcast()
         is called for the specified key.
       - The caller may pass arbitrary data to box_broadcast() which will be
         stored internally and passed to registered callbacks.
       - A callback is not executed again until it acknowledges the last
         notification (explicitly by calling watcher_ack() or implicitly by
         returning if WATCHER_EXPLICIT_ACK is unset).
      e8b9dffc
    • Alexander Turenko's avatar
      ci: don't fail job on failed personal Telegram notification · 2a678329
      Alexander Turenko authored
      To be honest, I have not proceeded with necessary actions to receive
      personal Telegram notifications for my branches, because GitHub already
      sends them to my email.
      
      I kept this feature (I don't know, whether someone uses it), but let the
      action ignore failures in the case.
      
      A failed attempt to send the notification from `master`, a release
      branch (`1.10`, `2.8`, ...) or a tag will fail the job as before. The
      change affects only personal notifications for developer branches.
      
      Fixes https://github.com/tarantool/tarantool-qa/issues/130
      2a678329
  4. Nov 01, 2021
    • Nikita Pettik's avatar
      access: fix diag raised when privileges are granted not from admin · 644ba590
      Nikita Pettik authored
      There are two kind of schema objects: objects themselves (featuring
      specific ids) and entities (representing all kinds of objects of the
      concrete type). Granting privileges for entities from user different from
      admin is not allowed. In this case diag is raised. Accidentally, there
      was a bug in diag: instead of printing entity name attempt to print
      object name took place. That led to assertion fault on debug version.
      Let's fix this and print entity name as it supposed to be.
      
      Closes #5389
      644ba590
    • Nikita Pettik's avatar
      Introduce enum schema_object_type converters · 72a273b8
      Nikita Pettik authored
      enum schema_object_type fits values of two different ranges, so let's
      add converters to transform from one to another.
      72a273b8
    • Alexander Turenko's avatar
      ci: reduce number of arm64 job runs · afd6da30
      Alexander Turenko authored
      We have limited amount of arm64 runners and arm64 jobs are often queued
      for hours. So I disabled running of arm64 jobs on developer branches and
      external PRs. Now we run them on master, release branches, tags and for
      branches named as 'something-full-ci'.
      
      While I'm here, disabled those jobs explicitly on forks: I guess
      otherwise it'll be it a 'waiting a runner' state infinitely.
      
      Part of https://github.com/tarantool/tarantool-qa/issues/142
      afd6da30
  5. Oct 31, 2021
  6. Oct 29, 2021
    • Yaroslav Lobankov's avatar
      ci: add integration check for queue module · 50635084
      Yaroslav Lobankov authored
      This patch extends the 'integration.yml' workflow and adds a new
      workflow call for running tests to verify integration between tarantool
      and the queue module.
      
      Part of #5265
      Part of #6056
      Closes #6559
      50635084
  7. Oct 28, 2021
    • Serge Petrenko's avatar
      replication: fix replica disconnect upon reconfiguration · 5994892c
      Serge Petrenko authored
      Replication reconfiguration used to work as follows: upon receiving a
      new config disconnect from all the existing masters and try to connect
      to all the masters from the new config.
      
      This lead to instances doing extra work when old config and new config
      had the same nodes in them: instead of doing nothing, tarantool
      reinitialized the connection.
      
      There was another problem: when an existing connection is broken, master
      takes some time to notice that. So, when replica resets the connection,
      it may try to reconnect faster than the master is able to notice its
      absence. In this case replica wouldn't be able to reconnect due to
      `duplicate connection with the same replica UUID`. So replication would
      stop for a replication_timeout, which may be quite large (seconds or
      tens of seconds).
      
      Let's prevent tarantool from reconnecting to the same instance, if there
      already is a working connection.
      
      Closes #4669
      5994892c
    • Serge Petrenko's avatar
      replicaiton: make anon replica connect to quorum upon reconfiguration · 66a0852e
      Serge Petrenko authored
      Once an anonymous replica tries to register, it reconnects to every
      other instance in its replication list in order to receive updated
      ballots and choose someone to register on.
      
      Make the instance wait until it connects to quorum before letting it
      choose the node to register on.
      66a0852e
    • Mergen Imeev's avatar
      sql: remove field argv from struct sql_context · bb9958ea
      Mergen Imeev authored
      Since the function arguments are always sequential in the array of all
      VDBE MEMs, we don't need to store the position of each argument. The
      position of the first MEM and the number of arguments are sufficient to
      describe all the arguments.
      
      Part of #4145
      bb9958ea
    • Mergen Imeev's avatar
      sql: remove MEM_TYPE_AGG · db7f7f58
      Mergen Imeev authored
      This patch removed MEM_TYPE_AGG, which is no longer used due to recent
      changes.
      
      Part of #4145
      db7f7f58
    • Mergen Imeev's avatar
      sql: remove copying of result in finalizers · 76c3e03e
      Mergen Imeev authored
      This patch removes copying of the result in the finalizers of the SQL
      built-in aggregate functions.
      
      Part of #4145
      76c3e03e
    • Mergen Imeev's avatar
      sql: refactor GROUP_CONCAT() function · f0b6a705
      Mergen Imeev authored
      Part of #4145
      f0b6a705
    • Mergen Imeev's avatar
      sql: refactor MIN() and MAX() functions · bc4f34fd
      Mergen Imeev authored
      Part of #4145
      bc4f34fd
    • Mergen Imeev's avatar
      sql: refactor COUNT() function · 536ed0bb
      Mergen Imeev authored
      Part of #4145
      536ed0bb
    • Mergen Imeev's avatar
      sql: refactor AVG() function · 9cc33308
      Mergen Imeev authored
      Part of #4145
      9cc33308
    • Mergen Imeev's avatar
      sql: refactor TOTAL() function · 4d79d922
      Mergen Imeev authored
      Part of #4145
      4d79d922
    • Mergen Imeev's avatar
      sql: refactor SUM() function · 8c887292
      Mergen Imeev authored
      Part of #4145
      8c887292
    • Mergen Imeev's avatar
      sql: remove sql_vdbemem_finalize() · b42c4d9e
      Mergen Imeev authored
      We don't need this function, since it is easier to call finalizers
      directly. This patch also allows us to make further simplifications.
      
      Needed for #4145
      b42c4d9e
    • Mergen Imeev's avatar
      sql: introduce mem_append() · ca18b298
      Mergen Imeev authored
      This patch introduces the mem_append() function. This function appends
      the specified string to the end of the STRING or VARBINARY contained in
      MEM. In case MEM needs to increase the size of allocated memory,
      extra memory is allocated in an attempt to reduce the total number of
      allocations.
      
      Needed for #4145
      ca18b298
    • Mergen Imeev's avatar
      sql: move collation to struct sql_context · 83d78b89
      Mergen Imeev authored
      This patch makes it easier to get a collation by a function.
      
      Needed for #4145
      83d78b89
    • Mergen Imeev's avatar
      sql: remove AggStep0 and OP_BuiltinFunction0 · 923e0185
      Mergen Imeev authored
      This patch moves the initialization of sql_context out of the VDBE. This
      allows us to remove the opcodes OP_BuiltinFunction0 and OP_AggStep0,
      which work in a rather strange way. Moreover, due to the changes these
      opcodes make to the VDBEs, it is possible that the estimated size of the
      VDBE may change during execution of VDBE, which could lead to various
      problems.
      
      Needed for #4145
      923e0185
    • Mergen Imeev's avatar
      sql: use register P1 for number of arguments · af4e9a63
      Mergen Imeev authored
      This patch makes OP_FunctionByName, OP_AggStep and OP_BuiltinFunction to
      use register P1 for the number of arguments instead of register P5. This
      makes it easier to use these opcodes.
      
      Needed for #4145
      af4e9a63
    • Mergen Imeev's avatar
      sql: fix possible undefined behavior during cast · 3c60e3fa
      Mergen Imeev authored
      This patch fixes possible undefined behavior during the implicit cast of
      INTEGER to DOUBLE. The problem is, if the INTEGER is close enough to
      2^64, it will be cast to 2^64 when it is cast to DOUBLE. Since we have a
      check for loss of precision, this will cause this DOUBLE to be cast to
      an INTEGER, which will result in undefined behavior since this DOUBLE is
      outside the range of INTEGER.
      3c60e3fa
Loading