Skip to content
Snippets Groups Projects
  1. Nov 10, 2021
    • Georgy Moiseev's avatar
      debian: actualize licenses · 5a8fea52
      Georgy Moiseev authored
      Add missing third party licenses to debian/copyright. Update copyright
      dates for modules. Remove license entries for unused modules.
      
      Closes #6391
      5a8fea52
  2. Nov 08, 2021
    • Georgy Moiseev's avatar
      debian: compile executable with PIE · 5e003c5a
      Georgy Moiseev authored
      This patch fixes hardening-no-pie lintian warning. Ubuntu Xenial uses
      older versions of gcc which not build ELF binaries with PIE by default,
      but since Xenial builds did not trigger this warning, we do not change
      the behavior with additional flags.
      
      Part of #5372, closes #6390
      5e003c5a
    • Georgy Moiseev's avatar
      debian: bump Standards-Version · 4947a8eb
      Georgy Moiseev authored
      This patch fixes ancient-standards-version lintian warning for modern
      systems. Standards-Version is 4.5.1 recommended for Debian Bullseye,
      Ubuntu Groovy and Hirsute. Older versions do not support 4.5.1 and thus
      yield newer-standards-version warning. This patch overrides it.
      
      Most notable changes:
      
      - Packages must not call /etc/init.d scripts directly even as a fallback,
        and instead must always use invoke-rc.d (which is essential and
        shouldn’t require any conditional).
      
      - Packages may not install files in both /path and /usr/path, and must
        manage any backward-compatibility symlinks so that they don’t break
        if /path and /usr/path are the same directory.
      
      - Packages are recommended to build reproducibly even when build paths
        and most environment variables are allowed to vary.
      
      - Clarify that programs may invoke either /usr/bin/editor and
        /usr/bin/pager directly, or use editor and pager and rely on PATH.
      
      - If /etc/staff-group-for-usr-local does not exist, /usr/local and all
        subdirectories created by packages should have permissions 0755 and
        be owned by root:root. If the file exists, the old permissions of 2775
        and ownership of root:staff should remain.
      
      - Packages should not contain a non-default series file. That is, dpkg’s
        vendor-specific patch series feature should not be used for packages
        in the Debian archive.
      
      - Binaries should be stripped using strip --strip-unneeded
        --remove-section=.comment --remove-section=.note
        (as dh_strip already does).
      
      - Packages that include system services should include systemd service
        units to start or stop those services.
      
      - Use of update-rc.d is required if the package includes an init script
        (previously, Policy said in one place that it was required, and
        in another said that it was recommended).
      
      - Shared libraries must now invoke ldconfig by means of triggers,
        instead of maintscripts.
      
      - Required targets must not write outside of the unpacked source package
        tree, except for TMPDIR, /tmp and /var/tmp.
      
      You can read full changelog here:
      https://www.debian.org/doc/debian-policy/upgrading-checklist.html
      
      No changes was introduced in package building pipeline after upgrade
      since there aren't any affecting behavior changes.
      
      Part of #6390
      4947a8eb
    • Georgy Moiseev's avatar
      debian: remove Windows binaries from source tars · 4785bb8c
      Georgy Moiseev authored
      This patch fixes source-contains-prebuilt-windows-binary lintian
      warnings.
      
      Part of #6390
      4785bb8c
    • Georgy Moiseev's avatar
      debian: add PackPack to uploaders · 6df61f8c
      Georgy Moiseev authored
      Before this patch all PackPack-builded packages was treated as
      non-maintainer updates. It fixes no-nmu-in-changelog and
      source-nmu-has-incorrect-version-number lintian warnings.
      
      Part of #6390
      6df61f8c
    • Georgy Moiseev's avatar
      debian: ignore debhelper lintian warning · 1e5590f0
      Georgy Moiseev authored
      This patch overrides package-needs-versioned-debhelper-build-depends
      lintian warning. This warning triggers when debhelper minimal required
      version is less than compat level. Ubuntu Xenial repos do not contain
      debhelper 10. This warning also triggers on Ubuntu Bionic and Debian
      Stretch and Buster despite using debhelper 10 or newer, which highly
      likely is a lintian bug. Debian Bullseye and Ubuntu newer that Bionic
      pipelines do not yield this warning.
      
      Part of #6390
      1e5590f0
    • Georgy Moiseev's avatar
      debian: ignore embedded library lintian errors · f5cf717c
      Georgy Moiseev authored
      This patch overrides embedded-library lintian errors
      for curl and libyaml.
      
      Part of #6390
      f5cf717c
    • Georgy Moiseev's avatar
      debian: remove outdated tags override · 4bb13253
      Georgy Moiseev authored
      This patch fixes malformed-override lintian warning.
      
      Part of #6390
      4bb13253
    • Georgy Moiseev's avatar
      debian: remove duplicate pattern from copyright · ff93b7ad
      Georgy Moiseev authored
      This patch fixes duplicate-globbing-patterns lintian warning.
      
      Part of #6390
      ff93b7ad
  3. Nov 03, 2021
    • Yaroslav Lobankov's avatar
      ci: add integration check for mysql module · 71a789f0
      Yaroslav Lobankov authored
      This patch extends the 'integration.yml' workflow and adds a new
      workflow call for running tests to verify integration between tarantool
      and the mysql module.
      
      Part of #5265
      Part of #6056
      Closes #6577
      71a789f0
    • Mergen Imeev's avatar
      sql: introduce literals for DECIMAL · ea637984
      Mergen Imeev authored
      Part of #6356
      
      @TarantoolBot document
      Title: Literals for INTEGER, DECIMAL and DOUBLE
      
      The rules for parsing numeric values have changed:
      1) a value consisting of digits without decimal point and exponent will
      be parsed as INTEGER;
      2) a value consisting of digits and a decimal point will be parsed as
      DECIMAL;
      3) a value consisting of digits, containing an exponent and possibly
      containing a decimal point, will be parsed as DOUBLE.
      ea637984
  4. Nov 02, 2021
    • Alexander Turenko's avatar
      jepsen: fix race condition with apt's sources.list · 20e72426
      Alexander Turenko authored
      Jepsen testing starts one or several virtual machines and runs tarantool
      instances on them. The first (first important for us here) command on
      the virtual machine is `apt-get <...> update`: we should download
      packages list to allow Jepsen to install necessary dependencies.
      
      However we can access the virtual machine (using ssh) before it is fully
      initialized by the cloud-init script. In particular, the cloud-init
      script replaces apt's mirror list file (`/etc/apt/sources.list`).
      
      Normally we should call `apt-get <...> update` after the package list
      update, but here cloud-init races with the update command.
      
      In the bad case the commands are executed in the opposite order:
      
      * Terraform calls `apt-get <...> update`.
      * cloud-init replaces `/etc/apt/sources.list`.
      
      Now an attempt to install a package using apt-get will give the 'unable
      to locate package' error, because we have no packages list for the 'new'
      mirrors.
      
      The problem is nicely described in [1]. See also the linked issue for
      details.
      
      [1]: https://github.com/hashicorp/packer/issues/41#issuecomment-21288589
      
      Fixes https://github.com/tarantool/jepsen.tarantool/issues/87
      20e72426
    • Vladimir Davydov's avatar
      test: cleanup vinyl suite · a737f7e4
      Vladimir Davydov authored
       - Remove tests linked to fixed issues from the flaky list.
       - Remove filters needed to produce stable output when a test failed
         because of #5436.
      a737f7e4
    • Vladimir Davydov's avatar
      test: fix flaky app/fiber_channel test · 7c3f0b06
      Vladimir Davydov authored
      The test has three known failures:
      
       1. Happens, because the test doesn't wait for a cancelled fiber
          to exit. As a result, it keeps reading from the test channel.
          Fix this by waiting for the test fiber to be dead for sure.
          ```
          --- app/fiber_channel.result    Sun Apr 26 17:45:32 2020
          +++ var/050_app/fiber_channel.result    Tue May 12 05:10:14 2020
          @@ -200,381 +200,14 @@
           ...
           ch:count()
           ---
          -- 1
          +- 0
           ...
           ch:is_full()
           ---
          -- true
          +- false
           ...
           ch:is_empty()
           ---
          -- false
          +- true
           ...
           ch:get(box.info.pid) == box.info.pid
          ```
       2. The test writes 16 values to a table, but waits for only 15 values.
          Fix this and also cancel test fibers before proceeding to the next
          test case.
          ```
          --- app/fiber_channel.result	Fri Feb 21 11:21:32 2020
          +++ var/049_app/fiber_channel.result	Tue May 12 01:05:37 2020
          @@ -346,7 +346,6 @@
             - 42
             - 43
             - 44
          -  - 45
           ...
           ch = fiber.channel(1)
           ---
          ```
       3. The test uses a 100 iteration loop to wait for a condition to
          become true. If the host is busy, 100 may be not enough. Let's
          use test_run:wait_cond() instead - it's more reliable.
          ```
          --- app/fiber_channel.result	Fri Feb 21 12:08:07 2020
          +++ app/fiber_channel.reject	Fri May  8 08:20:52 2020
          @@ -487,7 +487,7 @@
           ...
           count > 2000, #test_res, test_res;
           ---
          -- true
          +- false
           - 10
          unit/fiber_channel.test                                         [ pass ]
           - - true
             - true
          ```
      
      Closes #4961
      7c3f0b06
    • Vladimir Davydov's avatar
      test: fix flaky box/gh-5998-one-tx-for-ddl-errinj test · 1e56b157
      Vladimir Davydov authored
      The test tries to build secondary indexes in two spaces in parallel and
      checks that only one of them succeeded. For some reason, the test
      assumes that only the first space can succeed, but sometimes the second
      space may get lucky. Let's fix the check to remove this assumption.
      1e56b157
    • Vladimir Davydov's avatar
      test: fix flaky box-tap/gh-5602-environment-vars-cfg test · 3bc17662
      Vladimir Davydov authored
      The test is flaky, because it tries to listen on a fixed TCP port, which
      may be busy. Let's use Unix sockets to fix the test.
      
      While we are at it, let's also remove useless creation of a temporary
      directory - the test copies a script there, but never uses it.
      3bc17662
    • Vladimir Davydov's avatar
      test: fix flaky xlog/panic_on_broken_lsn test · 4336265e
      Vladimir Davydov authored
      The test injects an error for a replication of a particular LSN. The
      problem there's a pending record about the new replica written to the
      _cluster space on join, which may not have been flushed to WAL by the
      time the test injects an error. Due to the race, the test may inject an
      error for the _cluster row instead of the test row, which breaks its
      expectations.
      
      ```
      [009] xlog/panic_on_broken_lsn.test.lua                               [ fail ]
      [009]
      [009] Test failed! Result content mismatch:
      [009] --- xlog/panic_on_broken_lsn.result       Tue Nov  2 13:29:53 2021
      [009] +++ var/rejects/xlog/panic_on_broken_lsn.reject   Tue Nov  2 13:29:59 2021
      [009] @@ -155,8 +155,8 @@
      [009]  ...
      [009]  (found:gsub('^.*, req: ', ''):gsub('lsn: %d+', 'lsn: <lsn>'))
      [009]  ---
      [009] -- '{type: ''INSERT'', replica_id: 1, lsn: <lsn>, space_id: 9000, index_id: 0, tuple:
      [009] -  [2, "v1"]}'
      [009] +- '{type: ''INSERT'', replica_id: 1, lsn: <lsn>, space_id: 320, index_id: 0, tuple:
      [009] +  [2, "34898d18-eed4-4f8a-97ff-4ffba7b42892"]}'
      [009]  ...
      [009]  test_run:cmd('cleanup server replica')
      [009]  ---
      [009]
      ```
      
      Fix this by flushing WAL before writing a broken row.
      
      Closes #4508
      4336265e
    • Yaroslav Lobankov's avatar
      ci: add integration check for 'checks' module · 7083b20b
      Yaroslav Lobankov authored
      This patch extends the 'integration.yml' workflow and adds a new
      workflow call for running tests to verify integration between tarantool
      and the 'checks' module.
      
      Part of #5265
      Part of #6056
      Closes #6562
      7083b20b
    • Vladimir Davydov's avatar
      net.box: add watchers support · eb7712e3
      Vladimir Davydov authored
      Part of #6257
      
      @TarantoolBot document
      Title: Document net.box watchers
      
      Using the new `watch` method of a net.box connection, one can subscribe
      to events broadcasted by a remote host. The method has the same syntax
      as the `box.watch()` function, which is used for subscribing to events
      locally. It takes a key name (string) to subscribe to and a callback to
      invoke when the key value is updated. It returns a watcher handle that
      can be used to unregister the watcher. Note, garbage collection of a
      watcher handle doesnt result in unregistering the watcher so it's okay
      to discard the result of `box.watch` if the watcher is never going to be
      unregistered.
      
      A watcher callback is first invoked unconditionally after the watcher
      registration. Subsequent invocations are triggered by `box.broadcast()`
      called on the remote host. A watcher callback is passed the name of the
      key the watcher was subscribed to and the current key value. A watcher
      callback is always executed in a new fiber so it's okay to yield inside
      it. A watcher callback never runs in parallel with itself: if the key
      to which a watcher is subscribed is updated while the watcher callback
      is running, the callback will be invoked again with the new value as
      soon as it returns.
      
      Watchers survive reconnect (see `reconnect_after` connection option):
      all registered watchers are automatically resubscribed as soon as the
      connection is reestablished.
      
      If a remote host supports watchers, the 'watchers' key will be set in
      connection's `peer_protocol_features`.
      
      Example usage:
      
       * Server:
         ```lua
         -- Broadcast value 123 for key 'foo'.
         box.broadcast('foo', 123)
         ```
       * Client:
         ```lua
         conn = net.box.connect(URI)
         -- Subscribe to updates of key 'foo'.
         w = conn:watch('foo', function(key, value)
             assert(key == 'foo')
             -- do something with value
         end)
         -- Unregister the watcher when it's no longer needed.
         w:unregister()
         ```
      eb7712e3
    • Vladimir Davydov's avatar
      iproto: add watchers support · 4be5de4c
      Vladimir Davydov authored
      Part of #6257
      
      @TarantoolBot document
      Title: Document IPROTO watchers
      
      There are three new commands to support asynchronous server->client
      notifications signaled with `box.broadcast()`:
      
       - `IPROTO_WATCH` (code 74). Registers a new watcher for the given
         notification key or acknowledges a notification if a watcher is
         already registered. The key name is passed in `IPROTO_EVENT_KEY`
         (code 0x56). The watcher will be notified unconditionally after
         registration and then every time the key is updated with
         `box.broadcast()` provided the last notification was acknowledged.
         The server doesn't reply to the request unless it fails to parse
         the packet.
      
       - `IPROTO_UNWATCH` (code 75). Unregisters a watcher registered for the
         given notification key. The key name is passed in `IPROTO_EVENT_KEY`
         (code 0x56). A server doesn't reply to the request unless it fails to
         parse the packet.
      
       - `IPROTO_EVENT` (code 76). Sent by the server to notify a client
         about a key update. The key name is passed in `IPROTO_EVENT_KEY`
         (code 0x56). The key data (optional) is passed in `IPROTO_EVENT_DATA`
         (code 0x57).
      
      When a connection is closed, all watchers registered for it are
      unregistered.
      
      Servers that support the new feature set the `IPROTO_FEATURE_WATCHERS`
      feature bit (bit 3) in reply to the `IPROTO_ID` command.
      4be5de4c
    • Vladimir Davydov's avatar
      box/lua: introduce box.watch and box.broadcast · 11f2d999
      Vladimir Davydov authored
      Part of #6257
      
      @TarantoolBot document
      Title: Document box.watch and box.broadcast
      
      `box.watch(key, func)` registers a watcher for the given key and returns
      a watcher handle, which can be used to unregister the watcher (by
      calling the `unregister` method). A key is an arbitrary string. It's
      possible to register more than one watcher for the same key. Note,
      garbage collection of a watcher handle doesnt result in unregistering
      the watcher so it's okay to discard the result of `box.watch` if the
      watcher is never going to be unregistered.
      
      `box.broadcast(key, value)` updates the value of the given key and
      signals all watchers registered for it.
      
      A watcher callback is first invoked unconditionally after the watcher
      registration. Subsequent invocations are triggered by `box.broadcast()`
      called on the local host. A watcher callback is passed the name of the
      key the watcher was subscribed to and the current key value. A watcher
      callback is always executed in a new fiber so it's okay to yield inside
      it. A watcher callback never runs in parallel with itself: if the key
      to which a watcher is subscribed is updated while the watcher callback
      is running, the callback will be invoked again with the new value as
      soon as it returns.
      
      `box.watch` and `box.broadcast` may be used before `box.cfg`.
      
      Example usage:
      
      ```lua
      -- Broadcast value 123 for key 'foo'.
      box.broadcast('foo', 123)
      -- Subscribe to updates of key 'foo'.
      w = box.watch('foo', function(key, value)
          assert(key == 'foo')
          -- do something with value
      end)
      -- Unregister the watcher when it's no longer needed.
      w:unregister()
      ```
      11f2d999
    • Vladimir Davydov's avatar
      box: implement watcher infrastructure · e8b9dffc
      Vladimir Davydov authored
      Part of #6257
      
      This commit introduces a C module that will later be used for
      implementation of the watchers API in both Lua and IPROTO.
      
      Key points:
       - A watcher callback can be registered for an arbitrary string key with
         box_register_watcher().
       - Watcher callbacks are invoked in the background by the worker fiber.
         If a watcher is asynchronous (WATCHER_RUN_ASYNC flag set), the worker
         fiber will invoke the callback in a new fiber.
       - A newly registered watcher callback is scheduled for execution
         unconditionally after registration, and then whenever box_broadcast()
         is called for the specified key.
       - The caller may pass arbitrary data to box_broadcast() which will be
         stored internally and passed to registered callbacks.
       - A callback is not executed again until it acknowledges the last
         notification (explicitly by calling watcher_ack() or implicitly by
         returning if WATCHER_EXPLICIT_ACK is unset).
      e8b9dffc
    • Alexander Turenko's avatar
      ci: don't fail job on failed personal Telegram notification · 2a678329
      Alexander Turenko authored
      To be honest, I have not proceeded with necessary actions to receive
      personal Telegram notifications for my branches, because GitHub already
      sends them to my email.
      
      I kept this feature (I don't know, whether someone uses it), but let the
      action ignore failures in the case.
      
      A failed attempt to send the notification from `master`, a release
      branch (`1.10`, `2.8`, ...) or a tag will fail the job as before. The
      change affects only personal notifications for developer branches.
      
      Fixes https://github.com/tarantool/tarantool-qa/issues/130
      2a678329
  5. Nov 01, 2021
    • Nikita Pettik's avatar
      access: fix diag raised when privileges are granted not from admin · 644ba590
      Nikita Pettik authored
      There are two kind of schema objects: objects themselves (featuring
      specific ids) and entities (representing all kinds of objects of the
      concrete type). Granting privileges for entities from user different from
      admin is not allowed. In this case diag is raised. Accidentally, there
      was a bug in diag: instead of printing entity name attempt to print
      object name took place. That led to assertion fault on debug version.
      Let's fix this and print entity name as it supposed to be.
      
      Closes #5389
      644ba590
    • Nikita Pettik's avatar
      Introduce enum schema_object_type converters · 72a273b8
      Nikita Pettik authored
      enum schema_object_type fits values of two different ranges, so let's
      add converters to transform from one to another.
      72a273b8
    • Alexander Turenko's avatar
      ci: reduce number of arm64 job runs · afd6da30
      Alexander Turenko authored
      We have limited amount of arm64 runners and arm64 jobs are often queued
      for hours. So I disabled running of arm64 jobs on developer branches and
      external PRs. Now we run them on master, release branches, tags and for
      branches named as 'something-full-ci'.
      
      While I'm here, disabled those jobs explicitly on forks: I guess
      otherwise it'll be it a 'waiting a runner' state infinitely.
      
      Part of https://github.com/tarantool/tarantool-qa/issues/142
      afd6da30
  6. Oct 31, 2021
  7. Oct 29, 2021
    • Yaroslav Lobankov's avatar
      ci: add integration check for queue module · 50635084
      Yaroslav Lobankov authored
      This patch extends the 'integration.yml' workflow and adds a new
      workflow call for running tests to verify integration between tarantool
      and the queue module.
      
      Part of #5265
      Part of #6056
      Closes #6559
      50635084
  8. Oct 28, 2021
    • Serge Petrenko's avatar
      replication: fix replica disconnect upon reconfiguration · 5994892c
      Serge Petrenko authored
      Replication reconfiguration used to work as follows: upon receiving a
      new config disconnect from all the existing masters and try to connect
      to all the masters from the new config.
      
      This lead to instances doing extra work when old config and new config
      had the same nodes in them: instead of doing nothing, tarantool
      reinitialized the connection.
      
      There was another problem: when an existing connection is broken, master
      takes some time to notice that. So, when replica resets the connection,
      it may try to reconnect faster than the master is able to notice its
      absence. In this case replica wouldn't be able to reconnect due to
      `duplicate connection with the same replica UUID`. So replication would
      stop for a replication_timeout, which may be quite large (seconds or
      tens of seconds).
      
      Let's prevent tarantool from reconnecting to the same instance, if there
      already is a working connection.
      
      Closes #4669
      5994892c
    • Serge Petrenko's avatar
      replicaiton: make anon replica connect to quorum upon reconfiguration · 66a0852e
      Serge Petrenko authored
      Once an anonymous replica tries to register, it reconnects to every
      other instance in its replication list in order to receive updated
      ballots and choose someone to register on.
      
      Make the instance wait until it connects to quorum before letting it
      choose the node to register on.
      66a0852e
    • Mergen Imeev's avatar
      sql: remove field argv from struct sql_context · bb9958ea
      Mergen Imeev authored
      Since the function arguments are always sequential in the array of all
      VDBE MEMs, we don't need to store the position of each argument. The
      position of the first MEM and the number of arguments are sufficient to
      describe all the arguments.
      
      Part of #4145
      bb9958ea
    • Mergen Imeev's avatar
      sql: remove MEM_TYPE_AGG · db7f7f58
      Mergen Imeev authored
      This patch removed MEM_TYPE_AGG, which is no longer used due to recent
      changes.
      
      Part of #4145
      db7f7f58
    • Mergen Imeev's avatar
      sql: remove copying of result in finalizers · 76c3e03e
      Mergen Imeev authored
      This patch removes copying of the result in the finalizers of the SQL
      built-in aggregate functions.
      
      Part of #4145
      76c3e03e
    • Mergen Imeev's avatar
      sql: refactor GROUP_CONCAT() function · f0b6a705
      Mergen Imeev authored
      Part of #4145
      f0b6a705
    • Mergen Imeev's avatar
      sql: refactor MIN() and MAX() functions · bc4f34fd
      Mergen Imeev authored
      Part of #4145
      bc4f34fd
    • Mergen Imeev's avatar
      sql: refactor COUNT() function · 536ed0bb
      Mergen Imeev authored
      Part of #4145
      536ed0bb
    • Mergen Imeev's avatar
      sql: refactor AVG() function · 9cc33308
      Mergen Imeev authored
      Part of #4145
      9cc33308
    • Mergen Imeev's avatar
      sql: refactor TOTAL() function · 4d79d922
      Mergen Imeev authored
      Part of #4145
      4d79d922
Loading