Instead of writing to the socket fd directly using sio, we wrap it in
iostream. This will allow us to use complex communication protocols in
iproto.

One thing that should be noted about this patch is how we handle
ev_io_start when we need to wait for the socket to become readable or
writable. Since iostream_write can block because it wants to read from
the socket and iostream_read can block because it wants to write to the
socket, we might need to update input/output events before ev_io_start.
Since ev_io events can't be updated while ev_io is active, we need to
stop ev_io for this.

It's better than using ev_feed_event and ev_is_active directly.

Also, let's use EV_CUSTOM instead EV_READ/EV_WRITE for signaling,
to emphasize that this is an artificial event, which has nothing
to do with fd read/write readiness. It's okay, because input/output
callbacks don't use events at all.

It's better than using sio_socketname directly.

iproto_connection->input.fd != -1 iff state != IPROTO_CONNECTION_ALIVE.
Let's check the state instead of input.fd or output.fd, because it's
easier for understanding.

While we are at it, drop the stale comment to iproto_connection_is_idle:
the function doesn't use evio_has_fd at all.

It's not needed - we can use the iostream created in iproto
(just like a raw fd).

It's not needed - we can use the applier/relay io directly
(just like a raw fd).

Use coio_write_timeout_noxc instead.

Reading/writing fd directly doesn't let us add any data processing
transparently to the users. To overcome this limitation, let's wrap fd
in struct iostream. The new struct exposes virtual read/write methods,
which should be defined by a concrete implementation. It also allows to
access the associated fd, which is needed to poll the stream via libev.
For now, there's the only iostream implementation - plain iostream
without any processing - but we may add other implementations in future.

Apart from introducing the iostream struct, this patch also makes coio
helpers use it. From now on, coio read/write methods take iostream
instead of ev_io and create ev_io internally (in coio_wait). This is
fine, because creation of a new ev_io on stack is cheap.

Basically, this patch updates all coio users so that they call
iostream_create and iostream_close instead of coio_create and
coio_close_io. Plus, it adds a call to iostream_destroy, because in
contrast to ev_io, iostream must be destroyed explicitly.

We are going to wrap fd in iostream struct and pass it to all coio
methods. When we do that, the name 'coio' won't make any sense. Let's
switch to 'io'. We do renaming in a separate patch to reduce the blast
radius of the main patch.

Currently, we create and pass a new ev_io object wrapping the accepted
socket fd. Let's pass fd directly instead.

This is a step towards hiding all ev_io manipulations in coio internals
so that coio users don't have to deal with ev_io directly.

coio_accept uses fd from ev_io passed to it. Let's pass fd explicitly
and use coio_wait for waiting for the socket to become ready.

This is a step towards hiding all ev_io manipulations in coio internals
so that coio users don't have to deal with ev_io directly.

Drop unused coio_bind declaration while we are at it (there's no
function definition).

coio_connect sets fd in ev_io passed to it. Let's return fd explicitly
and remove the ev_io argument (we can use coio_wait, which creates a
temporary ev_io object, inside coio_connect - it's cheap).

This is a step towards hiding all ev_io manipulations in coio internals
so that coio users don't have to deal with ev_io directly.

The functions are not used anywhere. Let's drop them so that we don't
need to patch them when we switch all coio socket rw methods from ev_io
to iostream.

All it does is includes coio_buf.h, which has nothing in it but a few
inline functions.

This patch extends the 'integration.yml' workflow and adds a new
workflow call for running tests to verify integration between tarantool
and tarantool-c connector.

Part of #5265
Part of #6056
Closes #6582

tuple_bless() uses a tail call to ffi.gc() with return to the caller.
This tail call uses the current (tuple_bless) frame instead of creating
the frame for the callee (ffi.gc). When JIT tries to compile return from
`ffi.gc()` to the frame below it aborts the trace recording with the
error "NYI: return to lower frame".

This patch replaces the tail call with using additional local variable
returned to the caller right after.

Reviewed-by: Nikita Pettik <korablev@tarantool.org>
Reviewed-by: Igor Munkin <imun@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

This patch checks out to up-to-date tarantool/checks master branch
with checking out to up-to-date tarantool/test-run master branch.
Two commits have been added to tarantool/checks. The first one is
related to submodule CI: tarantool/checks#27, the second one contains
license authors fix: tarantool/checks#28.

Add missing third party licenses to debian/copyright. Update copyright
dates for modules. Remove license entries for unused modules.

Closes #6391

This patch fixes hardening-no-pie lintian warning. Ubuntu Xenial uses
older versions of gcc which not build ELF binaries with PIE by default,
but since Xenial builds did not trigger this warning, we do not change
the behavior with additional flags.

Part of #5372, closes #6390

This patch fixes ancient-standards-version lintian warning for modern
systems. Standards-Version is 4.5.1 recommended for Debian Bullseye,
Ubuntu Groovy and Hirsute. Older versions do not support 4.5.1 and thus
yield newer-standards-version warning. This patch overrides it.

Most notable changes:

- Packages must not call /etc/init.d scripts directly even as a fallback,
  and instead must always use invoke-rc.d (which is essential and
  shouldn’t require any conditional).

- Packages may not install files in both /path and /usr/path, and must
  manage any backward-compatibility symlinks so that they don’t break
  if /path and /usr/path are the same directory.

- Packages are recommended to build reproducibly even when build paths
  and most environment variables are allowed to vary.

- Clarify that programs may invoke either /usr/bin/editor and
  /usr/bin/pager directly, or use editor and pager and rely on PATH.

- If /etc/staff-group-for-usr-local does not exist, /usr/local and all
  subdirectories created by packages should have permissions 0755 and
  be owned by root:root. If the file exists, the old permissions of 2775
  and ownership of root:staff should remain.

- Packages should not contain a non-default series file. That is, dpkg’s
  vendor-specific patch series feature should not be used for packages
  in the Debian archive.

- Binaries should be stripped using strip --strip-unneeded
  --remove-section=.comment --remove-section=.note
  (as dh_strip already does).

- Packages that include system services should include systemd service
  units to start or stop those services.

- Use of update-rc.d is required if the package includes an init script
  (previously, Policy said in one place that it was required, and
  in another said that it was recommended).

- Shared libraries must now invoke ldconfig by means of triggers,
  instead of maintscripts.

- Required targets must not write outside of the unpacked source package
  tree, except for TMPDIR, /tmp and /var/tmp.

You can read full changelog here:
https://www.debian.org/doc/debian-policy/upgrading-checklist.html

No changes was introduced in package building pipeline after upgrade
since there aren't any affecting behavior changes.

Part of #6390

This patch fixes source-contains-prebuilt-windows-binary lintian
warnings.

Part of #6390

Before this patch all PackPack-builded packages was treated as
non-maintainer updates. It fixes no-nmu-in-changelog and
source-nmu-has-incorrect-version-number lintian warnings.

Part of #6390

This patch overrides package-needs-versioned-debhelper-build-depends
lintian warning. This warning triggers when debhelper minimal required
version is less than compat level. Ubuntu Xenial repos do not contain
debhelper 10. This warning also triggers on Ubuntu Bionic and Debian
Stretch and Buster despite using debhelper 10 or newer, which highly
likely is a lintian bug. Debian Bullseye and Ubuntu newer that Bionic
pipelines do not yield this warning.

Part of #6390

This patch overrides embedded-library lintian errors
for curl and libyaml.

Part of #6390

This patch fixes malformed-override lintian warning.

Part of #6390

This patch fixes duplicate-globbing-patterns lintian warning.

Part of #6390

This patch extends the 'integration.yml' workflow and adds a new
workflow call for running tests to verify integration between tarantool
and the mysql module.

Part of #5265
Part of #6056
Closes #6577

Part of #6356

@TarantoolBot document
Title: Literals for INTEGER, DECIMAL and DOUBLE

The rules for parsing numeric values have changed:
1) a value consisting of digits without decimal point and exponent will
be parsed as INTEGER;
2) a value consisting of digits and a decimal point will be parsed as
DECIMAL;
3) a value consisting of digits, containing an exponent and possibly
containing a decimal point, will be parsed as DOUBLE.

Jepsen testing starts one or several virtual machines and runs tarantool
instances on them. The first (first important for us here) command on
the virtual machine is `apt-get <...> update`: we should download
packages list to allow Jepsen to install necessary dependencies.

However we can access the virtual machine (using ssh) before it is fully
initialized by the cloud-init script. In particular, the cloud-init
script replaces apt's mirror list file (`/etc/apt/sources.list`).

Normally we should call `apt-get <...> update` after the package list
update, but here cloud-init races with the update command.

In the bad case the commands are executed in the opposite order:

* Terraform calls `apt-get <...> update`.
* cloud-init replaces `/etc/apt/sources.list`.

Now an attempt to install a package using apt-get will give the 'unable
to locate package' error, because we have no packages list for the 'new'
mirrors.

The problem is nicely described in [1]. See also the linked issue for
details.

[1]: https://github.com/hashicorp/packer/issues/41#issuecomment-21288589

Fixes https://github.com/tarantool/jepsen.tarantool/issues/87

 - Remove tests linked to fixed issues from the flaky list.
 - Remove filters needed to produce stable output when a test failed
   because of #5436.

The test has three known failures:

 1. Happens, because the test doesn't wait for a cancelled fiber
    to exit. As a result, it keeps reading from the test channel.
    Fix this by waiting for the test fiber to be dead for sure.
    ```
    --- app/fiber_channel.result    Sun Apr 26 17:45:32 2020
    +++ var/050_app/fiber_channel.result    Tue May 12 05:10:14 2020
    @@ -200,381 +200,14 @@
     ...
     ch:count()
     ---
    -- 1
    +- 0
     ...
     ch:is_full()
     ---
    -- true
    +- false
     ...
     ch:is_empty()
     ---
    -- false
    +- true
     ...
     ch:get(box.info.pid) == box.info.pid
    ```
 2. The test writes 16 values to a table, but waits for only 15 values.
    Fix this and also cancel test fibers before proceeding to the next
    test case.
    ```
    --- app/fiber_channel.result	Fri Feb 21 11:21:32 2020
    +++ var/049_app/fiber_channel.result	Tue May 12 01:05:37 2020
    @@ -346,7 +346,6 @@
       - 42
       - 43
       - 44
    -  - 45
     ...
     ch = fiber.channel(1)
     ---
    ```
 3. The test uses a 100 iteration loop to wait for a condition to
    become true. If the host is busy, 100 may be not enough. Let's
    use test_run:wait_cond() instead - it's more reliable.
    ```
    --- app/fiber_channel.result	Fri Feb 21 12:08:07 2020
    +++ app/fiber_channel.reject	Fri May  8 08:20:52 2020
    @@ -487,7 +487,7 @@
     ...
     count > 2000, #test_res, test_res;
     ---
    -- true
    +- false
     - 10
    unit/fiber_channel.test                                         [ pass ]
     - - true
       - true
    ```

Closes #4961

The test tries to build secondary indexes in two spaces in parallel and
checks that only one of them succeeded. For some reason, the test
assumes that only the first space can succeed, but sometimes the second
space may get lucky. Let's fix the check to remove this assumption.

The test is flaky, because it tries to listen on a fixed TCP port, which
may be busy. Let's use Unix sockets to fix the test.

While we are at it, let's also remove useless creation of a temporary
directory - the test copies a script there, but never uses it.

The test injects an error for a replication of a particular LSN. The
problem there's a pending record about the new replica written to the
_cluster space on join, which may not have been flushed to WAL by the
time the test injects an error. Due to the race, the test may inject an
error for the _cluster row instead of the test row, which breaks its
expectations.

```
[009] xlog/panic_on_broken_lsn.test.lua                               [ fail ]
[009]
[009] Test failed! Result content mismatch:
[009] --- xlog/panic_on_broken_lsn.result       Tue Nov  2 13:29:53 2021
[009] +++ var/rejects/xlog/panic_on_broken_lsn.reject   Tue Nov  2 13:29:59 2021
[009] @@ -155,8 +155,8 @@
[009]  ...
[009]  (found:gsub('^.*, req: ', ''):gsub('lsn: %d+', 'lsn: <lsn>'))
[009]  ---
[009] -- '{type: ''INSERT'', replica_id: 1, lsn: <lsn>, space_id: 9000, index_id: 0, tuple:
[009] -  [2, "v1"]}'
[009] +- '{type: ''INSERT'', replica_id: 1, lsn: <lsn>, space_id: 320, index_id: 0, tuple:
[009] +  [2, "34898d18-eed4-4f8a-97ff-4ffba7b42892"]}'
[009]  ...
[009]  test_run:cmd('cleanup server replica')
[009]  ---
[009]
```

Fix this by flushing WAL before writing a broken row.

Closes #4508

This patch extends the 'integration.yml' workflow and adds a new
workflow call for running tests to verify integration between tarantool
and the 'checks' module.

Part of #5265
Part of #6056
Closes #6562

Part of #6257

@TarantoolBot document
Title: Document net.box watchers

Using the new `watch` method of a net.box connection, one can subscribe
to events broadcasted by a remote host. The method has the same syntax
as the `box.watch()` function, which is used for subscribing to events
locally. It takes a key name (string) to subscribe to and a callback to
invoke when the key value is updated. It returns a watcher handle that
can be used to unregister the watcher. Note, garbage collection of a
watcher handle doesnt result in unregistering the watcher so it's okay
to discard the result of `box.watch` if the watcher is never going to be
unregistered.

A watcher callback is first invoked unconditionally after the watcher
registration. Subsequent invocations are triggered by `box.broadcast()`
called on the remote host. A watcher callback is passed the name of the
key the watcher was subscribed to and the current key value. A watcher
callback is always executed in a new fiber so it's okay to yield inside
it. A watcher callback never runs in parallel with itself: if the key
to which a watcher is subscribed is updated while the watcher callback
is running, the callback will be invoked again with the new value as
soon as it returns.

Watchers survive reconnect (see `reconnect_after` connection option):
all registered watchers are automatically resubscribed as soon as the
connection is reestablished.

If a remote host supports watchers, the 'watchers' key will be set in
connection's `peer_protocol_features`.

Example usage:

 * Server:
   ```lua
   -- Broadcast value 123 for key 'foo'.
   box.broadcast('foo', 123)
   ```
 * Client:
   ```lua
   conn = net.box.connect(URI)
   -- Subscribe to updates of key 'foo'.
   w = conn:watch('foo', function(key, value)
       assert(key == 'foo')
       -- do something with value
   end)
   -- Unregister the watcher when it's no longer needed.
   w:unregister()
   ```

Part of #6257

@TarantoolBot document
Title: Document IPROTO watchers

There are three new commands to support asynchronous server->client
notifications signaled with `box.broadcast()`:

 - `IPROTO_WATCH` (code 74). Registers a new watcher for the given
   notification key or acknowledges a notification if a watcher is
   already registered. The key name is passed in `IPROTO_EVENT_KEY`
   (code 0x56). The watcher will be notified unconditionally after
   registration and then every time the key is updated with
   `box.broadcast()` provided the last notification was acknowledged.
   The server doesn't reply to the request unless it fails to parse
   the packet.

 - `IPROTO_UNWATCH` (code 75). Unregisters a watcher registered for the
   given notification key. The key name is passed in `IPROTO_EVENT_KEY`
   (code 0x56). A server doesn't reply to the request unless it fails to
   parse the packet.

 - `IPROTO_EVENT` (code 76). Sent by the server to notify a client
   about a key update. The key name is passed in `IPROTO_EVENT_KEY`
   (code 0x56). The key data (optional) is passed in `IPROTO_EVENT_DATA`
   (code 0x57).

When a connection is closed, all watchers registered for it are
unregistered.

Servers that support the new feature set the `IPROTO_FEATURE_WATCHERS`
feature bit (bit 3) in reply to the `IPROTO_ID` command.

Part of #6257

@TarantoolBot document
Title: Document box.watch and box.broadcast

`box.watch(key, func)` registers a watcher for the given key and returns
a watcher handle, which can be used to unregister the watcher (by
calling the `unregister` method). A key is an arbitrary string. It's
possible to register more than one watcher for the same key. Note,
garbage collection of a watcher handle doesnt result in unregistering
the watcher so it's okay to discard the result of `box.watch` if the
watcher is never going to be unregistered.

`box.broadcast(key, value)` updates the value of the given key and
signals all watchers registered for it.

A watcher callback is first invoked unconditionally after the watcher
registration. Subsequent invocations are triggered by `box.broadcast()`
called on the local host. A watcher callback is passed the name of the
key the watcher was subscribed to and the current key value. A watcher
callback is always executed in a new fiber so it's okay to yield inside
it. A watcher callback never runs in parallel with itself: if the key
to which a watcher is subscribed is updated while the watcher callback
is running, the callback will be invoked again with the new value as
soon as it returns.

`box.watch` and `box.broadcast` may be used before `box.cfg`.

Example usage:

```lua
-- Broadcast value 123 for key 'foo'.
box.broadcast('foo', 123)
-- Subscribe to updates of key 'foo'.
w = box.watch('foo', function(key, value)
    assert(key == 'foo')
    -- do something with value
end)
-- Unregister the watcher when it's no longer needed.
w:unregister()
```

Part of #6257

This commit introduces a C module that will later be used for
implementation of the watchers API in both Lua and IPROTO.

Key points:
 - A watcher callback can be registered for an arbitrary string key with
   box_register_watcher().
 - Watcher callbacks are invoked in the background by the worker fiber.
   If a watcher is asynchronous (WATCHER_RUN_ASYNC flag set), the worker
   fiber will invoke the callback in a new fiber.
 - A newly registered watcher callback is scheduled for execution
   unconditionally after registration, and then whenever box_broadcast()
   is called for the specified key.
 - The caller may pass arbitrary data to box_broadcast() which will be
   stored internally and passed to registered callbacks.
 - A callback is not executed again until it acknowledges the last
   notification (explicitly by calling watcher_ack() or implicitly by
   returning if WATCHER_EXPLICIT_ACK is unset).