Tuple field count overflow is not handled: before inserting, explicitly
compare the current tuple size to box.schema.FIELD_MAX and set a newly
introduced diagnostic message if overflow occurs.

Closes #6198

Tests inserting fields into tuples, causing it to append lots of
nil's, work unacceptably slow: optimize mp_next.

Needed for #6198

On completion of compaction tasks we remove compacted run files created
after the last checkpoint immediately to save disk space. In order to
perform this optimization we compare the unused runs' dump LSN with the
last checkpoint's one.

But during replica's initial JOIN stage we set the LSN of all rows
received from remote master to 0 (see
box/box.cc/boostrap_journal_write). Considering that the LSN of an
initial checkpoint is also 0, our optimization stops working, and we
receive a huge disk space usage spike (as the unused run files will
only get removed when garbage collection occurs).

We should check the vinyl space engine's status and perform
our optimization unconditionally if we are in replica's initial JOIN
stage.

Closes #6568

In the 'pack_and_deploy' action we have such a line:

    echo PACKPACK_EXTRA_DOCKER_RUN_PARAMS='--init' | tee -a $GITHUB_ENV

Usually, the `echo FOO=bar | tee -a $GITHUB_ENV` construction is used
to propagate an env variable outside the action/step so it could be used
in other actions/steps via ${{ env.FOO }} or $FOO.

The 'PACKPACK_EXTRA_DOCKER_RUN_PARAMS' variable is used only inside the
action and we can change the mentioned line to as follows:

    export PACKPACK_EXTRA_DOCKER_RUN_PARAMS='--init'

This patch introduces the use of RWS (repository web service) instead
of old update_repo.sh script for uploading packages to S3 repositories.
RWS fully replaces facilities of update_repo.sh + provides new features
like support of uploading ARM packages. Also, it is very simple to use,
just do `curl` to the RWS endpoint with some params and that's it.

Closes #6482

This commit fixes the following failure:

```
--- box/func_reload.result        Fri Dec  3 15:15:05 2021
+++ var/rejects/box/func_reload.reject    Fri Dec  3 15:15:12 2021
@@ -202,11 +202,11 @@
 ...
 ch:get()
 ---
+- [[2]]
+...
+ch:get()
+---
 - [[1]]
-...
-ch:get()
----
-- [[2]]
 ...
 s:drop()
 ---
```

The function reloaded in this test yields, so there's no guarantee that
the first reload completes before the second one. Fix this by discarding
`ch:get()` result.

Closes https://github.com/tarantool/tarantool-qa/issues/15

This reverts commit 5546cde0.

And remove the test from the flaky list. The test seems to have been
fixed by commit c13b3a31 ("net.box:
rewrite state machine (transport) in C").

Closes #5081

Sometimes to fix an issue we need to patch our submodule, update it
in tarantool and then patch tarantool using changes in submodule.

During development and review stages we have to create a branch (S)
in submodule and a branch (T) in tarantool that references head of
(S) branch.

When the fix is merged to mater it's very simple to make a mistake:
merge-and-push submodule's branch S and then tarantool's branch T.
It sounds obvious, but that's wrong: tarantool's master should
reference a commit from submodule's master, not a commit from
temporary developer's branch.

It's not hard to fix it but still a maintainer must always remember
that problem. In order to simplify their life it was decided to
create a script that is designed to check such thing before pushing
tarantool to origin/master.

Here is it, with simple usage:
./tools/check_push_master.sh && git push origin master

This patch changes the way values are displayed in debug information.

libiconv must be installed before tarantool build. Otherwise iconv test
will fails.
More info: https://github.com/tarantool/tarantool/issues/3791

python-daemon is not used anymore:
https://github.com/tarantool/test-run/commit/8797cb15bf34f8c70519221b3b9c78f08696aba6
https://github.com/tarantool/tarantool/commit/130cf4659bf938d1fa1bf0e936b0a43a53fa8dcb



python-msgpack is enabled in test-run as a submodule, while
python-gevent and python-yaml are actually used in test-run.

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

Python3 symlink is missing after installing python3X from pkg on FreeBSD
12.2 and 13. This breaks tests.

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

Running tests inside FreeBSD github runner sandbox directory will fail with:
af_unix socket length exceed.

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

FreeBSD images for MCS was created. Now we can run tests directly from
MCS instances.

There are no official github runner for FreeBSD, but we can use
unofficial runner that supports FreeBSD:
https://github.com/ChristopherHX/github-act-runner.

It has limitation:
You won't be able to run steps after a failure without using continue-on-error: true
So steps: artifacts and Telegram message on failure was removed.

Closes https://github.com/tarantool/tarantool-qa/issues/143



Signed-off-by: Pavel Balaev <balaev@tarantool.org>

In FreeBSD sed is not support \s as whitespace characters definition.
However, since FreeBSD-13 sed behaivour changed from ignore to error:
sed complains about trailing backslash (\). We should use whitespace
definition from POSIX standart.

More info: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253893



CLoses #6637

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

Closes tarantool/tarantool-qa#95

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

This patch introduces a new syntax that allows to create ARRAY values in
an SQL query.

Part of #4762

@TarantoolBot document
Title: Syntax for ARRAY in SQL

The syntax for creating ARRAY values is available in SQL. You can use
`[` and `]` to create an ARRAY value - all values in those brackets will
be part of ARRAY. The position of the values will be translated to the
same positions in ARRAY.

Examples:
```
tarantool> box.execute("SELECT [1, 'a', 1.5];")
---
- metadata:
  - name: COLUMN_1
    type: array
  rows:
  - [[1, 'a', 1.5]]
...
```

```
tarantool> box.execute("SELECT [1, 'a', ['abc', 321], 1.5];")
---
- metadata:
  - name: COLUMN_1
    type: array
  rows:
  - [[1, 'a', ['abc', 321], 1.5]]
...
```

This function is used to create ARRAY value from array of MEMs. Since
ARRAY was added to SQL, this function needs to be refactored.

Part of #4762

This patch changes the signature of mpstream_encode_vdbe_mem(), so it
now follows the general rule that applies to most functions in mem.c.

1. The net.box state machine is supposed to set E_NO_CONNECTION on
   error, but netbox_transport_connect returns an error returned by
   coio_connect and coio_read as is. Fix this.

2. netbox_transport_communicate sets the error message to
   strerror(errno) on coio_read or coio_write error. This works now,
   because iostream can only be backed by plain sockets, but once we
   implement encryption, errno won't make any sense. We should use
   error::errmsg instead.

3. While we are at it, fix luaT_netbox_transport_make_request: it's
   wrong to pass errmsg directly to box_error_raise - we should use a
   format specifier (%s).

Fixes commit c13b3a31 ("net.box: rewrite
state machine (transport) in C").

Currently, we append strerror(errno) when logging those errors while
error::errmsg only contains the message passed to the error constructor,
which usually points at the system call that failed, e.g. sio_connect()
sets it to

  connect, called on fd 50, aka 127.0.0.1:33728

Errno can shed light at the reason of failure so we'd better append it
to errmsg too:

  connect, called on fd 50, aka 127.0.0.1:33728: Connection refused

In particular, this improves error messages returned by the net.box
state machine, which was recently switched to diag, see commit
c13b3a31 ("net.box: rewrite state
machine (transport) in C").

After this patch, we don't need to append strerror(errno) to SystemError
message manually anymore so this patch also walks over the whole code
base and removes strerror from diag_set calls - there are just two
modules that do that, fio and say.

Also, this patch slightly changes logged error messages:
 - A colon (:) is printed after SystemError, because SystemError::log
   doesn't override Exception::log anymore. This makes SystemError
   logging consistent with other error logging.
 - For the same reason we now print the actual error type of errors
   derived from SystemError (SocketError, OutOfMemory and TimedOut)
   instead of SystemError to the log (this should be fine), and don't
   print strerror(errno) for OutOfMemory and TimedOut (it's useless
   anyway).

tt_sprintf, tt_snprintf, and tt_vsnprintf now have compile-time
checks for their arguments fitting the format string.

Caught by coverity.

Error messages tried to printf uint64_t term as %llu, but
apparently it does not always work. Needs explicit cast to
unsigned long long.

Caught by coverity.

Transactions should be committed after they reach quorum of "real"
cluster members. Therefore, anonymous replicas shouldn't
participate in the quorum.

Closes #5418

The truncate method could be called from within a transaction. The flag
of GROUP_LOCAL was set in truncate method after statement row had been
being checked on the GROUP_LOCAL. Accordingly, after a local transaction
NOP row was not appended.

Closes #6123

Fixes commit c13b3a31 ("net.box: rewrite
state machine (transport) in C").

Found by Coverity.

While we are at it, fix malformed indentation.

Fixes commit c13b3a31 ("net.box: rewrite
state machine (transport) in C").

Found by Coverity.

Problem:

For now, the integration testing runs only on a push of changes/tags
to the 'master' and release branches. This approach is not good enough
because we become aware of integration issues after the changes were
merged already to the target branch.

So this patch adds the facility to run the integration testing on a dev
branch to catch integration issues before the changes are merged to the
target branch. Now it can be done via naming the branch as `*-full-ci`
and pushing it to the main repository or setting the 'full-ci' label on
the pull request.

This commit actually reverts 2.10.0-beta1-136-gaa2a6bc83 ('ci:
temporarily disable deploy for CentOS 7 for aarch64').

I built packages necessary for testing for CentOS 7 on aarch64 and
placed them into packpack backports [1].

The source packages are from CentOS 8 [2] and are the following:

* Cython-0.29.14-4.module_el8.3.0+441+3b561464.src.rpm
* python-gevent-1.2.2-4.el8.src.rpm
* PyYAML-5.3.1-1.module_el8.3.0+441+3b561464.src.rpm
* python-six-1.12.0-9.module_el8.3.0+441+3b561464.src.rpm

[1]: https://packagecloud.io/packpack/backports
[2]: https://vault.centos.org/8.3.2011/AppStream/Source/SPackages/

If you're curious about steps to re-build the packages for CentOS 7,
read below.

----

Build steps are the following:

```sh
$ ssh <aarch64 machine>
$ docker run -it -v <...>:/src -v <...>:/res centos:7
```

Install tools:

```sh
$ yum install yum-utils rpm-build
```

(Here I also installed epel-release, but I doubt that it is necessary.)

Build and install Cython:

```sh
$ rpmbuild --rebuild /src/Cpython*.src.rpm
$ vim /root/rpmbuild/SPECS/Cython.spec
<..fix changelog dates..>
$ rpmbuild -bb /root/rpmbuild/SPECS/Cython.spec
<...>
error: Failed build dependencies:
  gcc is needed by Cython-0.29.14-4.el7.aarch64
  python3-devel is needed by Cython-0.29.14-4.el7.aarch64
  python3-setuptools is needed by Cython-0.29.14-4.el7.aarch64
  python3-rpm-macros is needed by Cython-0.29.14-4.el7.aarch64
$ yum install gcc python3-devel python3-rpm-macros python3-setuptools
$ rpmbuild -bb /root/rpmbuild/SPECS/Cython.spec # repeat, yep
$ ls -1 /root/rpmbuild/RPMS/aarch64/
Cython-debuginfo-0.29.14-4.el7.aarch64.rpm
python3-Cython-0.29.14-4.el7.aarch64.rpm
$ cp /root/rpmbuild/RPMS/aarch64/* /res/
$ rpm -i /root/rpmbuild/RPMS/aarch64/*
```

Build python-gevent:

```sh
$ rpmbuild --rebuild /src/python-gevent*.src.rpm
<...>
error: Failed build dependencies:
  c-ares-devel is needed by python-gevent-1.2.2-4.el7.aarch64
  libev-devel is needed by python-gevent-1.2.2-4.el7.aarch64
  python2-devel is needed by python-gevent-1.2.2-4.el7.aarch64
$ yum install c-ares-devel libev-devel # but not python2-devel
$ vim /root/rpmbuild/SPECS/python-gevent.spec
<..leave '%bcond_with python2'..>
$ rpmbuild -bb /root/rpmbuild/SPECS/python-gevent.spec
$ ls -1 /root/rpmbuild/RPMS/aarch64/*gevent*
/root/rpmbuild/RPMS/aarch64/python-gevent-debuginfo-1.2.2-4.el7.aarch64.rpm
/root/rpmbuild/RPMS/aarch64/python3-gevent-1.2.2-4.el7.aarch64.rpm
$ cp /root/rpmbuild/RPMS/aarch64/*gevent* /res/
```

Build PyYAML:

```sh
$ rpmbuild --rebuild /src/PyYAML*.src.rpm
<...>
error: Failed build dependencies:
    libyaml-devel is needed by PyYAML-5.3.1-1.el7.aarch64
$ yum install libyaml-devel
$ rpmbuild --rebuild /src/PyYAML*.src.rpm # again, yep
$ ls -1 /root/rpmbuild/RPMS/aarch64/*{pyyaml,PyYAML}*
/root/rpmbuild/RPMS/aarch64/PyYAML-debuginfo-5.3.1-1.el7.aarch64.rpm
/root/rpmbuild/RPMS/aarch64/python3-pyyaml-5.3.1-1.el7.aarch64.rpm
$ cp /root/rpmbuild/RPMS/aarch64/*{pyyaml,PyYAML}* /res/
```

Build python-six:

```sh
$ rpmbuild --rebuild /src/python-six*.src.rpm
error: Failed build dependencies:
    python3-wheel is needed by python-six-1.12.0-9.el7.noarch
    python3-pytest is needed by python-six-1.12.0-9.el7.noarch
    python3-tkinter is needed by python-six-1.12.0-9.el7.noarch
$ yum install python3-wheel python3-pytest python3-tkinter
$ rpmbuild --rebuild /src/python-six*.src.rpm # again, yep
error: Failed build dependencies:
    python3-pytest is needed by python-six-1.12.0-9.el7.noarch
$ vim /root/rpmbuild/SPECS/python-six.spec
<..change '%bcond_without tests' to '%bcond_with tests'..>
$ rpmbuild -bb /root/rpmbuild/SPECS/python-six.spec
$ ls -1 /root/rpmbuild/RPMS/noarch/
python3-six-1.12.0-9.el7.noarch.rpm
$ cp /root/rpmbuild/RPMS/noarch/* /res/
```

If 'box.cfg' fails with error, 'log.cfg' still gets updated: instead,
update 'log.cfg' after verifying 'box.cfg', and rollback if update of
'log.cfg' fails.

Closes #6086

box_api_cfg_set_log_{level, format} are part of dynamic_cfg (see
box/lua/load_cfg.lua) and are expected to throw exceptions (see
reload_cfg ib.).

Needed for #6086

On big instances core produces too much logs related to shapshotting,
which feels redudant: reduce snapshot verbosity.

Closes #6220

Make say_ratelimit_check emit warnings about suppressed messages only if
the correspoding log level is greater or equal then WARN.

Add say_crit_ratelimited and say_info_ratelimited macros for
convenience.

Needed for #6220

* memprof: add info about trace start to symtab
* memprof: group allocations on traces by traceno
* memprof: refactor location parsing
* test: separate memprof Lua API tests into subtests

Closes #5814

This patch introduces MAP to SQL. After this patch, all SQL operations
and built-in functions should work correctly with MAP values. However,
there is currently no way to create MAP values using only SQL tools.

Part of #4763

@TarantoolBot document
Title: Field type MAP in SQL

Properties of type MAP in SQL:
1) a value of type MAP can be implicitly and explicitly cast only to
ANY;
2) only a value of type ANY with primitive type MAP can be explicitly
cast to MAP;
3) a value of any other type cannot be implicitly cast to MAP;
4) a value of type MAP cannot participate in arithmetic, bitwise,
comparison, and concatination operations.

This patch introduces ARRAY to SQL. After this patch, all SQL operations
and built-in functions should work correctly with ARRAY values. However,
there is currently no way to create ARRAY values using only SQL tools.

Part of #4762

@TarantoolBot document
Title: Field type ARRAY in SQL

Properties of type ARRAY in SQL:
1) a value of type ARRAY can be implicitly and explicitly cast only
   to ANY;
2) only a value of type ANY with primitive type ARRAY can be explicitly
   cast to ARRAY;
3) a value of any other type cannot be implicitly cast to ARRAY;
4) a value of type ARRAY cannot participate in arithmetic, bitwise,
   comparison, and concatination operations.

This patch introduces the mem_snprintf() function, which writes the
string representation of a MEM to buf.

This patch removes quotes from the representation of UUID values in the
error description. This is because UUID values are printed without
quotes elsewhere.

After this commit only three workflow are run on pull request or push to
a developer branch:
 - luacheck
 - release
 - debug_coverage

To run all other tests, one should either name the branch `*-full-ci`
and push it to the main repository or set the 'full-ci' label on the
pull request.

It is also possible to disable all tests on push by naming branch as
`*-notest' or setting the 'notest' label on the pull request.

**Caveats**:
 - Unfortunately, currently it doesn't seem to be possible to run
   workflows automatically when a particular label is set - the best we
   can do is run workflows when *any* label is set. So labeling a PR
   that has the 'full-ci' label set will trigger all workflows!
 - For the same reason, removing the 'notest' label doesn't trigger ci.
   One has to synchronize the PR afterwards. We could trigger ci on the
   'unlabel' event, but this would trigger tests when any label is
   removed, not necessarily 'notest'. Since 'notest' is supposed to be
   used only by developers, who can sync the branch, this should be
   acceptable.

While we are at it:
 - Remove the check disabling certain workflow runs on forks - it's
   pointless, because forks don't have ci. Anyway, we don't bother
   disabling most of our workflows on forks, even those that we run on
   self-hosted machines, so that would only be consistent.
 - Remove the condition from the coverity workflow - coverity doesn't
   run on push or PR so it doesn't make any sense.
 - Remove the condition from the 'source' workflow. Instead trigger it
   only when a tag is pushed. This is needed to avoid showing it as a
   skipped workflow in PRs and commits.

Closes #6605

Line-by-line (mostly) migration of the net.box state machine
implementation from Lua to C. The C implementation uses the iostream
abstration, which will allow us to to easily support complex network
protocols under IPROTO in future.

Closes #6291