libiconv must be installed before tarantool build. Otherwise iconv test
will fails.
More info: https://github.com/tarantool/tarantool/issues/3791

python-daemon is not used anymore:
https://github.com/tarantool/test-run/commit/8797cb15bf34f8c70519221b3b9c78f08696aba6
https://github.com/tarantool/tarantool/commit/130cf4659bf938d1fa1bf0e936b0a43a53fa8dcb



python-msgpack is enabled in test-run as a submodule, while
python-gevent and python-yaml are actually used in test-run.

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

Python3 symlink is missing after installing python3X from pkg on FreeBSD
12.2 and 13. This breaks tests.

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

Running tests inside FreeBSD github runner sandbox directory will fail with:
af_unix socket length exceed.

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

FreeBSD images for MCS was created. Now we can run tests directly from
MCS instances.

There are no official github runner for FreeBSD, but we can use
unofficial runner that supports FreeBSD:
https://github.com/ChristopherHX/github-act-runner.

It has limitation:
You won't be able to run steps after a failure without using continue-on-error: true
So steps: artifacts and Telegram message on failure was removed.

Closes https://github.com/tarantool/tarantool-qa/issues/143



Signed-off-by: Pavel Balaev <balaev@tarantool.org>

In FreeBSD sed is not support \s as whitespace characters definition.
However, since FreeBSD-13 sed behaivour changed from ignore to error:
sed complains about trailing backslash (\). We should use whitespace
definition from POSIX standart.

More info: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253893



CLoses #6637

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

Closes tarantool/tarantool-qa#95

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

This patch introduces a new syntax that allows to create ARRAY values in
an SQL query.

Part of #4762

@TarantoolBot document
Title: Syntax for ARRAY in SQL

The syntax for creating ARRAY values is available in SQL. You can use
`[` and `]` to create an ARRAY value - all values in those brackets will
be part of ARRAY. The position of the values will be translated to the
same positions in ARRAY.

Examples:
```
tarantool> box.execute("SELECT [1, 'a', 1.5];")
---
- metadata:
  - name: COLUMN_1
    type: array
  rows:
  - [[1, 'a', 1.5]]
...
```

```
tarantool> box.execute("SELECT [1, 'a', ['abc', 321], 1.5];")
---
- metadata:
  - name: COLUMN_1
    type: array
  rows:
  - [[1, 'a', ['abc', 321], 1.5]]
...
```

This function is used to create ARRAY value from array of MEMs. Since
ARRAY was added to SQL, this function needs to be refactored.

Part of #4762

This patch changes the signature of mpstream_encode_vdbe_mem(), so it
now follows the general rule that applies to most functions in mem.c.

1. The net.box state machine is supposed to set E_NO_CONNECTION on
   error, but netbox_transport_connect returns an error returned by
   coio_connect and coio_read as is. Fix this.

2. netbox_transport_communicate sets the error message to
   strerror(errno) on coio_read or coio_write error. This works now,
   because iostream can only be backed by plain sockets, but once we
   implement encryption, errno won't make any sense. We should use
   error::errmsg instead.

3. While we are at it, fix luaT_netbox_transport_make_request: it's
   wrong to pass errmsg directly to box_error_raise - we should use a
   format specifier (%s).

Fixes commit c13b3a31 ("net.box: rewrite
state machine (transport) in C").

Currently, we append strerror(errno) when logging those errors while
error::errmsg only contains the message passed to the error constructor,
which usually points at the system call that failed, e.g. sio_connect()
sets it to

  connect, called on fd 50, aka 127.0.0.1:33728

Errno can shed light at the reason of failure so we'd better append it
to errmsg too:

  connect, called on fd 50, aka 127.0.0.1:33728: Connection refused

In particular, this improves error messages returned by the net.box
state machine, which was recently switched to diag, see commit
c13b3a31 ("net.box: rewrite state
machine (transport) in C").

After this patch, we don't need to append strerror(errno) to SystemError
message manually anymore so this patch also walks over the whole code
base and removes strerror from diag_set calls - there are just two
modules that do that, fio and say.

Also, this patch slightly changes logged error messages:
 - A colon (:) is printed after SystemError, because SystemError::log
   doesn't override Exception::log anymore. This makes SystemError
   logging consistent with other error logging.
 - For the same reason we now print the actual error type of errors
   derived from SystemError (SocketError, OutOfMemory and TimedOut)
   instead of SystemError to the log (this should be fine), and don't
   print strerror(errno) for OutOfMemory and TimedOut (it's useless
   anyway).

tt_sprintf, tt_snprintf, and tt_vsnprintf now have compile-time
checks for their arguments fitting the format string.

Caught by coverity.

Error messages tried to printf uint64_t term as %llu, but
apparently it does not always work. Needs explicit cast to
unsigned long long.

Caught by coverity.

Transactions should be committed after they reach quorum of "real"
cluster members. Therefore, anonymous replicas shouldn't
participate in the quorum.

Closes #5418

The truncate method could be called from within a transaction. The flag
of GROUP_LOCAL was set in truncate method after statement row had been
being checked on the GROUP_LOCAL. Accordingly, after a local transaction
NOP row was not appended.

Closes #6123

Fixes commit c13b3a31 ("net.box: rewrite
state machine (transport) in C").

Found by Coverity.

While we are at it, fix malformed indentation.

Fixes commit c13b3a31 ("net.box: rewrite
state machine (transport) in C").

Found by Coverity.

Problem:

For now, the integration testing runs only on a push of changes/tags
to the 'master' and release branches. This approach is not good enough
because we become aware of integration issues after the changes were
merged already to the target branch.

So this patch adds the facility to run the integration testing on a dev
branch to catch integration issues before the changes are merged to the
target branch. Now it can be done via naming the branch as `*-full-ci`
and pushing it to the main repository or setting the 'full-ci' label on
the pull request.

This commit actually reverts 2.10.0-beta1-136-gaa2a6bc83 ('ci:
temporarily disable deploy for CentOS 7 for aarch64').

I built packages necessary for testing for CentOS 7 on aarch64 and
placed them into packpack backports [1].

The source packages are from CentOS 8 [2] and are the following:

* Cython-0.29.14-4.module_el8.3.0+441+3b561464.src.rpm
* python-gevent-1.2.2-4.el8.src.rpm
* PyYAML-5.3.1-1.module_el8.3.0+441+3b561464.src.rpm
* python-six-1.12.0-9.module_el8.3.0+441+3b561464.src.rpm

[1]: https://packagecloud.io/packpack/backports
[2]: https://vault.centos.org/8.3.2011/AppStream/Source/SPackages/

If you're curious about steps to re-build the packages for CentOS 7,
read below.

----

Build steps are the following:

```sh
$ ssh <aarch64 machine>
$ docker run -it -v <...>:/src -v <...>:/res centos:7
```

Install tools:

```sh
$ yum install yum-utils rpm-build
```

(Here I also installed epel-release, but I doubt that it is necessary.)

Build and install Cython:

```sh
$ rpmbuild --rebuild /src/Cpython*.src.rpm
$ vim /root/rpmbuild/SPECS/Cython.spec
<..fix changelog dates..>
$ rpmbuild -bb /root/rpmbuild/SPECS/Cython.spec
<...>
error: Failed build dependencies:
  gcc is needed by Cython-0.29.14-4.el7.aarch64
  python3-devel is needed by Cython-0.29.14-4.el7.aarch64
  python3-setuptools is needed by Cython-0.29.14-4.el7.aarch64
  python3-rpm-macros is needed by Cython-0.29.14-4.el7.aarch64
$ yum install gcc python3-devel python3-rpm-macros python3-setuptools
$ rpmbuild -bb /root/rpmbuild/SPECS/Cython.spec # repeat, yep
$ ls -1 /root/rpmbuild/RPMS/aarch64/
Cython-debuginfo-0.29.14-4.el7.aarch64.rpm
python3-Cython-0.29.14-4.el7.aarch64.rpm
$ cp /root/rpmbuild/RPMS/aarch64/* /res/
$ rpm -i /root/rpmbuild/RPMS/aarch64/*
```

Build python-gevent:

```sh
$ rpmbuild --rebuild /src/python-gevent*.src.rpm
<...>
error: Failed build dependencies:
  c-ares-devel is needed by python-gevent-1.2.2-4.el7.aarch64
  libev-devel is needed by python-gevent-1.2.2-4.el7.aarch64
  python2-devel is needed by python-gevent-1.2.2-4.el7.aarch64
$ yum install c-ares-devel libev-devel # but not python2-devel
$ vim /root/rpmbuild/SPECS/python-gevent.spec
<..leave '%bcond_with python2'..>
$ rpmbuild -bb /root/rpmbuild/SPECS/python-gevent.spec
$ ls -1 /root/rpmbuild/RPMS/aarch64/*gevent*
/root/rpmbuild/RPMS/aarch64/python-gevent-debuginfo-1.2.2-4.el7.aarch64.rpm
/root/rpmbuild/RPMS/aarch64/python3-gevent-1.2.2-4.el7.aarch64.rpm
$ cp /root/rpmbuild/RPMS/aarch64/*gevent* /res/
```

Build PyYAML:

```sh
$ rpmbuild --rebuild /src/PyYAML*.src.rpm
<...>
error: Failed build dependencies:
    libyaml-devel is needed by PyYAML-5.3.1-1.el7.aarch64
$ yum install libyaml-devel
$ rpmbuild --rebuild /src/PyYAML*.src.rpm # again, yep
$ ls -1 /root/rpmbuild/RPMS/aarch64/*{pyyaml,PyYAML}*
/root/rpmbuild/RPMS/aarch64/PyYAML-debuginfo-5.3.1-1.el7.aarch64.rpm
/root/rpmbuild/RPMS/aarch64/python3-pyyaml-5.3.1-1.el7.aarch64.rpm
$ cp /root/rpmbuild/RPMS/aarch64/*{pyyaml,PyYAML}* /res/
```

Build python-six:

```sh
$ rpmbuild --rebuild /src/python-six*.src.rpm
error: Failed build dependencies:
    python3-wheel is needed by python-six-1.12.0-9.el7.noarch
    python3-pytest is needed by python-six-1.12.0-9.el7.noarch
    python3-tkinter is needed by python-six-1.12.0-9.el7.noarch
$ yum install python3-wheel python3-pytest python3-tkinter
$ rpmbuild --rebuild /src/python-six*.src.rpm # again, yep
error: Failed build dependencies:
    python3-pytest is needed by python-six-1.12.0-9.el7.noarch
$ vim /root/rpmbuild/SPECS/python-six.spec
<..change '%bcond_without tests' to '%bcond_with tests'..>
$ rpmbuild -bb /root/rpmbuild/SPECS/python-six.spec
$ ls -1 /root/rpmbuild/RPMS/noarch/
python3-six-1.12.0-9.el7.noarch.rpm
$ cp /root/rpmbuild/RPMS/noarch/* /res/
```

If 'box.cfg' fails with error, 'log.cfg' still gets updated: instead,
update 'log.cfg' after verifying 'box.cfg', and rollback if update of
'log.cfg' fails.

Closes #6086

box_api_cfg_set_log_{level, format} are part of dynamic_cfg (see
box/lua/load_cfg.lua) and are expected to throw exceptions (see
reload_cfg ib.).

Needed for #6086

On big instances core produces too much logs related to shapshotting,
which feels redudant: reduce snapshot verbosity.

Closes #6220

Make say_ratelimit_check emit warnings about suppressed messages only if
the correspoding log level is greater or equal then WARN.

Add say_crit_ratelimited and say_info_ratelimited macros for
convenience.

Needed for #6220

* memprof: add info about trace start to symtab
* memprof: group allocations on traces by traceno
* memprof: refactor location parsing
* test: separate memprof Lua API tests into subtests

Closes #5814

This patch introduces MAP to SQL. After this patch, all SQL operations
and built-in functions should work correctly with MAP values. However,
there is currently no way to create MAP values using only SQL tools.

Part of #4763

@TarantoolBot document
Title: Field type MAP in SQL

Properties of type MAP in SQL:
1) a value of type MAP can be implicitly and explicitly cast only to
ANY;
2) only a value of type ANY with primitive type MAP can be explicitly
cast to MAP;
3) a value of any other type cannot be implicitly cast to MAP;
4) a value of type MAP cannot participate in arithmetic, bitwise,
comparison, and concatination operations.

This patch introduces ARRAY to SQL. After this patch, all SQL operations
and built-in functions should work correctly with ARRAY values. However,
there is currently no way to create ARRAY values using only SQL tools.

Part of #4762

@TarantoolBot document
Title: Field type ARRAY in SQL

Properties of type ARRAY in SQL:
1) a value of type ARRAY can be implicitly and explicitly cast only
   to ANY;
2) only a value of type ANY with primitive type ARRAY can be explicitly
   cast to ARRAY;
3) a value of any other type cannot be implicitly cast to ARRAY;
4) a value of type ARRAY cannot participate in arithmetic, bitwise,
   comparison, and concatination operations.

This patch introduces the mem_snprintf() function, which writes the
string representation of a MEM to buf.

This patch removes quotes from the representation of UUID values in the
error description. This is because UUID values are printed without
quotes elsewhere.

After this commit only three workflow are run on pull request or push to
a developer branch:
 - luacheck
 - release
 - debug_coverage

To run all other tests, one should either name the branch `*-full-ci`
and push it to the main repository or set the 'full-ci' label on the
pull request.

It is also possible to disable all tests on push by naming branch as
`*-notest' or setting the 'notest' label on the pull request.

**Caveats**:
 - Unfortunately, currently it doesn't seem to be possible to run
   workflows automatically when a particular label is set - the best we
   can do is run workflows when *any* label is set. So labeling a PR
   that has the 'full-ci' label set will trigger all workflows!
 - For the same reason, removing the 'notest' label doesn't trigger ci.
   One has to synchronize the PR afterwards. We could trigger ci on the
   'unlabel' event, but this would trigger tests when any label is
   removed, not necessarily 'notest'. Since 'notest' is supposed to be
   used only by developers, who can sync the branch, this should be
   acceptable.

While we are at it:
 - Remove the check disabling certain workflow runs on forks - it's
   pointless, because forks don't have ci. Anyway, we don't bother
   disabling most of our workflows on forks, even those that we run on
   self-hosted machines, so that would only be consistent.
 - Remove the condition from the coverity workflow - coverity doesn't
   run on push or PR so it doesn't make any sense.
 - Remove the condition from the 'source' workflow. Instead trigger it
   only when a tag is pushed. This is needed to avoid showing it as a
   skipped workflow in PRs and commits.

Closes #6605

Line-by-line (mostly) migration of the net.box state machine
implementation from Lua to C. The C implementation uses the iostream
abstration, which will allow us to to easily support complex network
protocols under IPROTO in future.

Closes #6291

Currently,

 - socket.tcp_server sets AI_PASSIVE.
 - socket.tcp_connect sets no flags, but passes 'tcp' for protocol.
 - coio_connect sets AI_ADDRCONFIG|AI_PASSIVE|AI_NUMERICSERV.
 - evio_service sets AI_ADDRCONFIG|AI_PASSIVE.

This leads to different getaddrinfo results on some platforms (freebsd)
and, as a result, test breakages (net.box_connect_timeout_gh-2054) after
we switch net.box from socket.tcp_connect to coio_connect.

Let's use compatible flags:
 - AI_ADDRCONFIG for clients (coio_connect and tcp_connect).
   (AI_PASSIVE doesn't make sense for connect.)
   (AI_NUMERICSERV isn't really needed in our case.)
 - AI_ADDRCONFIG|AI_PASSIVE for servers (evio_service, tcp_server).

Let's also set ai_protocol to 'tcp' in both tcp_connect and tcp_server
for consistency, because both these functions are only supposed to work
with the tcp protocol, judging by their names.

It's too low-level. Use conn:_request instead.

While we are at it, let's explicitly check the error in
net.box_long-poll_input_gh-3400.test.lua to make it more
robust.

We will use iostream in net.box when we convert the state machine to C.
Rework netbox_communicate() so that it's easy to migrate to iostream.

xrow_decode_auth() expects the password to be set so omitting it while
preparing a request is incorrect. Net.box state machine (in Lua) sets
the password to an empty string if it's nil so the code handling NULL
password in netbox_encode_auth() is actually dead. Let's remove it.

The callback passed to the worker fiber (in create_transport())
references the connection object. Since a fiber is a GC root, it means
that the worker fiber pins the connection object. To avoid this, we use
a weak callback reference. To prevent the callback from being GC-ed
prematurely, we need to store a strong reference to it somewhere.
Currently, we have some GC magic in create_transport(): we store a
strong reference to the callback in the bound parameters of
transport.stop(). This works fine, but when we migrate the transport
implementation to C, we won't be able to override transport.stop() so
let's store a strong reference in the connection object instead.

While we are at it, let's drop create_transport method from net.box
module exports, because it's not used anywhere in Tarantool code or
tests, neither is it documented.

A GC callback must not yield.

Fixes commit 72adeda6 ("net.box:
implement connection closing without dropping requests");

Closes #6617

The callback isn't supposed to normally raise exceptions (it runs client
code in pcall). If something abnormal happens, like a memory allocation
error, better break the loop and stop the state machine.

Also, raise an exception if we failed to decode an IPROTO_EVENT packet -
this means that something is wrong with the other end and we better stop
the connection (this is what we do if we fail to decode other packets).

Currently, some errors raise exception (e.g. memory allocation, fiber is
cancelled) while others are handled by the state machine code. Let's
pass all errors in exceptions and drop error_sm. This simplifies the
state machine code a little bit.

We can implement it in Lua, because the transport invokes the callback
whenever the state changes. This will simplify conversion of the state
machine to C.