Replication of data in the test should happened before the data presence
test on the replica. Wait for lsn helper is used.

Closes #6663

Test for #6198 handles an edge case (box.schema.FIELD_MAX tuple
field count), hence consuming a huge amount of memory (see #6684): mark
it as long run.

Closes #6684

This is a second attempt to stabilize #5568 test since
5105c2d7 ("test: fix flaky
read_only_reason test").

It had several failures with various frequency.

* test_read_only_reason_synchro() could see ro_reason as nil even
  after box.error.READONLY was raised (and some yields passed);

* test_read_only_reason_orphan() could do the same;

* `box.ctl.demote()` could raise error "box.ctl.demote does not
  support simultaneous invocations".

The orphan failure couldn't be reproduced. It was caught only
locally, so maybe it was just some unnoticed diff breaking the
test.

Failure of test_read_only_reason_synchro() could happen when
demote() was called in a previous test case, then the current test
case called promote(), then got box.error.READONLY on the replica,
then that old demote() was delivered to replica, and the attempt
to get ro_reason returned nil. It is attempted to be fixed with
replication_synchro_quorum = 2, so master promote()/demote() will
implicitly push the previous operation to the replica. Via term
bump and quorum wait.

Additionally, huge replication_synchro_timeout is added for manual
promotions. Automatic promotion is retried so here the timeout is
not so important.

`box.ctl.demote` failure due to `simultaneous invocations` seems
to be happening because the original auto-election win didn't
finish limbo transition yet. Hence the instance calling demote()
now would think it is called 'simultaneously' with another
promote()/demote().

It is attempted to be fixed from 2 sides:

- Add waiting for `not box.info.ro` on the leader node after
  auto-promotion. To ensure the limbo is taken by the leader;

- The first option didn't help much, so `box.ctl.demote()` is
  simply called in a loop until it succeeds.

Closes #6670

assert(cur < end) after mp_load_u8 in mp_check_binl was failing

 | [100%] Linking C static library libcurl-d.a
 | <...>
 | make[3]: *** No rule to make target `build/curl/dest/lib/libcurl.a',
 |     needed by `test/unit/luaL_iterator.test'. Stop.

The problem appears on Mac OS. It is a bit strange that we don't see it
on Linux. However I didn't dig into this, just fixed the observed
problem.

Related: https://github.com/curl/curl/issues/2121

Fixes #6656

The test fails quite often with one of the following results.

Either

[001] @@ -60,7 +60,7 @@
[001]  ...
[001]  test_timeout()
[001]  ---
[001] -- true
[001] +- false
[001]  ...
[001]  test_run:cmd("switch default")
[001]  ---
[001]

Or

[034]  test_timeout()
[034]  ---
[034] -- true
[034] +- error: 'replicas are not in the follow status'
[034]  ...

Both errors are caused by wait_cond checking saved
`box.info.replication` values instead of actual `box.info.replication`
output. Fix this.

Also, the test's quite long for no reason.

The wait_cond waiting for replication to break lasts a whole
`replication_timeout`, which is excess.

The test's idea, as stated in the commit that has introduced it
(195d4462: Send relay heartbeat if wal changes won't be send) is to
constantly relay some data to the remote peers and make
it so their relays never send heartbeats.

If we wait for a whole replication timeout between inserts, there's a high
chance of peer relays waking up naturally (after replication_timeout passes).

In this case the test tests nothing. Fix the issue by waiting for ~ 1/5
of replication_timeout in between the despatches. This also reduces test
run time from ~4.5 to ~1.5 seconds on my machine.

Remove the test from fragile list, since it shouldn't be flaky anymore.

Closes #4940

vfork got deprecated on macOS: for now, change the vfork deprecation
error to a warning until we find a suitable solution. We plan to review
popen's design and get rid of vfork completely or change vfork to
posix_spawn only on macOS in #6674.

Closes #6576

vfork got deprecated on macOS: change vfork to more portable
posix_spawn.

Needed for #6576

If slab order of first several pools was equal to zero,
all these pools added to same group with pools with slab
order is equal to one. It's leds to incorrect behavior
of `smfree` function, because this function relies on
same slab order size of all pools in mempool group.
Fix this and add assertion and test to check it.

Closes #6659

Tuple field count overflow is not handled: before inserting, explicitly
compare the current tuple size to box.schema.FIELD_MAX and set a newly
introduced diagnostic message if overflow occurs.

Closes #6198

Tests inserting fields into tuples, causing it to append lots of
nil's, work unacceptably slow: optimize mp_next.

Needed for #6198

On completion of compaction tasks we remove compacted run files created
after the last checkpoint immediately to save disk space. In order to
perform this optimization we compare the unused runs' dump LSN with the
last checkpoint's one.

But during replica's initial JOIN stage we set the LSN of all rows
received from remote master to 0 (see
box/box.cc/boostrap_journal_write). Considering that the LSN of an
initial checkpoint is also 0, our optimization stops working, and we
receive a huge disk space usage spike (as the unused run files will
only get removed when garbage collection occurs).

We should check the vinyl space engine's status and perform
our optimization unconditionally if we are in replica's initial JOIN
stage.

Closes #6568

In the 'pack_and_deploy' action we have such a line:

    echo PACKPACK_EXTRA_DOCKER_RUN_PARAMS='--init' | tee -a $GITHUB_ENV

Usually, the `echo FOO=bar | tee -a $GITHUB_ENV` construction is used
to propagate an env variable outside the action/step so it could be used
in other actions/steps via ${{ env.FOO }} or $FOO.

The 'PACKPACK_EXTRA_DOCKER_RUN_PARAMS' variable is used only inside the
action and we can change the mentioned line to as follows:

    export PACKPACK_EXTRA_DOCKER_RUN_PARAMS='--init'

This patch introduces the use of RWS (repository web service) instead
of old update_repo.sh script for uploading packages to S3 repositories.
RWS fully replaces facilities of update_repo.sh + provides new features
like support of uploading ARM packages. Also, it is very simple to use,
just do `curl` to the RWS endpoint with some params and that's it.

Closes #6482

This commit fixes the following failure:

```
--- box/func_reload.result        Fri Dec  3 15:15:05 2021
+++ var/rejects/box/func_reload.reject    Fri Dec  3 15:15:12 2021
@@ -202,11 +202,11 @@
 ...
 ch:get()
 ---
+- [[2]]
+...
+ch:get()
+---
 - [[1]]
-...
-ch:get()
----
-- [[2]]
 ...
 s:drop()
 ---
```

The function reloaded in this test yields, so there's no guarantee that
the first reload completes before the second one. Fix this by discarding
`ch:get()` result.

Closes https://github.com/tarantool/tarantool-qa/issues/15

This reverts commit 5546cde0.

And remove the test from the flaky list. The test seems to have been
fixed by commit c13b3a31 ("net.box:
rewrite state machine (transport) in C").

Closes #5081

Sometimes to fix an issue we need to patch our submodule, update it
in tarantool and then patch tarantool using changes in submodule.

During development and review stages we have to create a branch (S)
in submodule and a branch (T) in tarantool that references head of
(S) branch.

When the fix is merged to mater it's very simple to make a mistake:
merge-and-push submodule's branch S and then tarantool's branch T.
It sounds obvious, but that's wrong: tarantool's master should
reference a commit from submodule's master, not a commit from
temporary developer's branch.

It's not hard to fix it but still a maintainer must always remember
that problem. In order to simplify their life it was decided to
create a script that is designed to check such thing before pushing
tarantool to origin/master.

Here is it, with simple usage:
./tools/check_push_master.sh && git push origin master

This patch changes the way values are displayed in debug information.

libiconv must be installed before tarantool build. Otherwise iconv test
will fails.
More info: https://github.com/tarantool/tarantool/issues/3791

python-daemon is not used anymore:
https://github.com/tarantool/test-run/commit/8797cb15bf34f8c70519221b3b9c78f08696aba6
https://github.com/tarantool/tarantool/commit/130cf4659bf938d1fa1bf0e936b0a43a53fa8dcb



python-msgpack is enabled in test-run as a submodule, while
python-gevent and python-yaml are actually used in test-run.

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

Python3 symlink is missing after installing python3X from pkg on FreeBSD
12.2 and 13. This breaks tests.

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

Running tests inside FreeBSD github runner sandbox directory will fail with:
af_unix socket length exceed.

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

FreeBSD images for MCS was created. Now we can run tests directly from
MCS instances.

There are no official github runner for FreeBSD, but we can use
unofficial runner that supports FreeBSD:
https://github.com/ChristopherHX/github-act-runner.

It has limitation:
You won't be able to run steps after a failure without using continue-on-error: true
So steps: artifacts and Telegram message on failure was removed.

Closes https://github.com/tarantool/tarantool-qa/issues/143



Signed-off-by: Pavel Balaev <balaev@tarantool.org>

In FreeBSD sed is not support \s as whitespace characters definition.
However, since FreeBSD-13 sed behaivour changed from ignore to error:
sed complains about trailing backslash (\). We should use whitespace
definition from POSIX standart.

More info: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253893



CLoses #6637

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

Closes tarantool/tarantool-qa#95

Signed-off-by: Pavel Balaev <balaev@tarantool.org>

This patch introduces a new syntax that allows to create ARRAY values in
an SQL query.

Part of #4762

@TarantoolBot document
Title: Syntax for ARRAY in SQL

The syntax for creating ARRAY values is available in SQL. You can use
`[` and `]` to create an ARRAY value - all values in those brackets will
be part of ARRAY. The position of the values will be translated to the
same positions in ARRAY.

Examples:
```
tarantool> box.execute("SELECT [1, 'a', 1.5];")
---
- metadata:
  - name: COLUMN_1
    type: array
  rows:
  - [[1, 'a', 1.5]]
...
```

```
tarantool> box.execute("SELECT [1, 'a', ['abc', 321], 1.5];")
---
- metadata:
  - name: COLUMN_1
    type: array
  rows:
  - [[1, 'a', ['abc', 321], 1.5]]
...
```

This function is used to create ARRAY value from array of MEMs. Since
ARRAY was added to SQL, this function needs to be refactored.

Part of #4762

This patch changes the signature of mpstream_encode_vdbe_mem(), so it
now follows the general rule that applies to most functions in mem.c.

1. The net.box state machine is supposed to set E_NO_CONNECTION on
   error, but netbox_transport_connect returns an error returned by
   coio_connect and coio_read as is. Fix this.

2. netbox_transport_communicate sets the error message to
   strerror(errno) on coio_read or coio_write error. This works now,
   because iostream can only be backed by plain sockets, but once we
   implement encryption, errno won't make any sense. We should use
   error::errmsg instead.

3. While we are at it, fix luaT_netbox_transport_make_request: it's
   wrong to pass errmsg directly to box_error_raise - we should use a
   format specifier (%s).

Fixes commit c13b3a31 ("net.box: rewrite
state machine (transport) in C").

Currently, we append strerror(errno) when logging those errors while
error::errmsg only contains the message passed to the error constructor,
which usually points at the system call that failed, e.g. sio_connect()
sets it to

  connect, called on fd 50, aka 127.0.0.1:33728

Errno can shed light at the reason of failure so we'd better append it
to errmsg too:

  connect, called on fd 50, aka 127.0.0.1:33728: Connection refused

In particular, this improves error messages returned by the net.box
state machine, which was recently switched to diag, see commit
c13b3a31 ("net.box: rewrite state
machine (transport) in C").

After this patch, we don't need to append strerror(errno) to SystemError
message manually anymore so this patch also walks over the whole code
base and removes strerror from diag_set calls - there are just two
modules that do that, fio and say.

Also, this patch slightly changes logged error messages:
 - A colon (:) is printed after SystemError, because SystemError::log
   doesn't override Exception::log anymore. This makes SystemError
   logging consistent with other error logging.
 - For the same reason we now print the actual error type of errors
   derived from SystemError (SocketError, OutOfMemory and TimedOut)
   instead of SystemError to the log (this should be fine), and don't
   print strerror(errno) for OutOfMemory and TimedOut (it's useless
   anyway).

tt_sprintf, tt_snprintf, and tt_vsnprintf now have compile-time
checks for their arguments fitting the format string.

Caught by coverity.

Error messages tried to printf uint64_t term as %llu, but
apparently it does not always work. Needs explicit cast to
unsigned long long.

Caught by coverity.

Transactions should be committed after they reach quorum of "real"
cluster members. Therefore, anonymous replicas shouldn't
participate in the quorum.

Closes #5418

The truncate method could be called from within a transaction. The flag
of GROUP_LOCAL was set in truncate method after statement row had been
being checked on the GROUP_LOCAL. Accordingly, after a local transaction
NOP row was not appended.

Closes #6123

Fixes commit c13b3a31 ("net.box: rewrite
state machine (transport) in C").

Found by Coverity.

While we are at it, fix malformed indentation.

Fixes commit c13b3a31 ("net.box: rewrite
state machine (transport) in C").

Found by Coverity.

Problem:

For now, the integration testing runs only on a push of changes/tags
to the 'master' and release branches. This approach is not good enough
because we become aware of integration issues after the changes were
merged already to the target branch.

So this patch adds the facility to run the integration testing on a dev
branch to catch integration issues before the changes are merged to the
target branch. Now it can be done via naming the branch as `*-full-ci`
and pushing it to the main repository or setting the 'full-ci' label on
the pull request.

This commit actually reverts 2.10.0-beta1-136-gaa2a6bc83 ('ci:
temporarily disable deploy for CentOS 7 for aarch64').

I built packages necessary for testing for CentOS 7 on aarch64 and
placed them into packpack backports [1].

The source packages are from CentOS 8 [2] and are the following:

* Cython-0.29.14-4.module_el8.3.0+441+3b561464.src.rpm
* python-gevent-1.2.2-4.el8.src.rpm
* PyYAML-5.3.1-1.module_el8.3.0+441+3b561464.src.rpm
* python-six-1.12.0-9.module_el8.3.0+441+3b561464.src.rpm

[1]: https://packagecloud.io/packpack/backports
[2]: https://vault.centos.org/8.3.2011/AppStream/Source/SPackages/

If you're curious about steps to re-build the packages for CentOS 7,
read below.

----

Build steps are the following:

```sh
$ ssh <aarch64 machine>
$ docker run -it -v <...>:/src -v <...>:/res centos:7
```

Install tools:

```sh
$ yum install yum-utils rpm-build
```

(Here I also installed epel-release, but I doubt that it is necessary.)

Build and install Cython:

```sh
$ rpmbuild --rebuild /src/Cpython*.src.rpm
$ vim /root/rpmbuild/SPECS/Cython.spec
<..fix changelog dates..>
$ rpmbuild -bb /root/rpmbuild/SPECS/Cython.spec
<...>
error: Failed build dependencies:
  gcc is needed by Cython-0.29.14-4.el7.aarch64
  python3-devel is needed by Cython-0.29.14-4.el7.aarch64
  python3-setuptools is needed by Cython-0.29.14-4.el7.aarch64
  python3-rpm-macros is needed by Cython-0.29.14-4.el7.aarch64
$ yum install gcc python3-devel python3-rpm-macros python3-setuptools
$ rpmbuild -bb /root/rpmbuild/SPECS/Cython.spec # repeat, yep
$ ls -1 /root/rpmbuild/RPMS/aarch64/
Cython-debuginfo-0.29.14-4.el7.aarch64.rpm
python3-Cython-0.29.14-4.el7.aarch64.rpm
$ cp /root/rpmbuild/RPMS/aarch64/* /res/
$ rpm -i /root/rpmbuild/RPMS/aarch64/*
```

Build python-gevent:

```sh
$ rpmbuild --rebuild /src/python-gevent*.src.rpm
<...>
error: Failed build dependencies:
  c-ares-devel is needed by python-gevent-1.2.2-4.el7.aarch64
  libev-devel is needed by python-gevent-1.2.2-4.el7.aarch64
  python2-devel is needed by python-gevent-1.2.2-4.el7.aarch64
$ yum install c-ares-devel libev-devel # but not python2-devel
$ vim /root/rpmbuild/SPECS/python-gevent.spec
<..leave '%bcond_with python2'..>
$ rpmbuild -bb /root/rpmbuild/SPECS/python-gevent.spec
$ ls -1 /root/rpmbuild/RPMS/aarch64/*gevent*
/root/rpmbuild/RPMS/aarch64/python-gevent-debuginfo-1.2.2-4.el7.aarch64.rpm
/root/rpmbuild/RPMS/aarch64/python3-gevent-1.2.2-4.el7.aarch64.rpm
$ cp /root/rpmbuild/RPMS/aarch64/*gevent* /res/
```

Build PyYAML:

```sh
$ rpmbuild --rebuild /src/PyYAML*.src.rpm
<...>
error: Failed build dependencies:
    libyaml-devel is needed by PyYAML-5.3.1-1.el7.aarch64
$ yum install libyaml-devel
$ rpmbuild --rebuild /src/PyYAML*.src.rpm # again, yep
$ ls -1 /root/rpmbuild/RPMS/aarch64/*{pyyaml,PyYAML}*
/root/rpmbuild/RPMS/aarch64/PyYAML-debuginfo-5.3.1-1.el7.aarch64.rpm
/root/rpmbuild/RPMS/aarch64/python3-pyyaml-5.3.1-1.el7.aarch64.rpm
$ cp /root/rpmbuild/RPMS/aarch64/*{pyyaml,PyYAML}* /res/
```

Build python-six:

```sh
$ rpmbuild --rebuild /src/python-six*.src.rpm
error: Failed build dependencies:
    python3-wheel is needed by python-six-1.12.0-9.el7.noarch
    python3-pytest is needed by python-six-1.12.0-9.el7.noarch
    python3-tkinter is needed by python-six-1.12.0-9.el7.noarch
$ yum install python3-wheel python3-pytest python3-tkinter
$ rpmbuild --rebuild /src/python-six*.src.rpm # again, yep
error: Failed build dependencies:
    python3-pytest is needed by python-six-1.12.0-9.el7.noarch
$ vim /root/rpmbuild/SPECS/python-six.spec
<..change '%bcond_without tests' to '%bcond_with tests'..>
$ rpmbuild -bb /root/rpmbuild/SPECS/python-six.spec
$ ls -1 /root/rpmbuild/RPMS/noarch/
python3-six-1.12.0-9.el7.noarch.rpm
$ cp /root/rpmbuild/RPMS/noarch/* /res/
```

If 'box.cfg' fails with error, 'log.cfg' still gets updated: instead,
update 'log.cfg' after verifying 'box.cfg', and rollback if update of
'log.cfg' fails.

Closes #6086

box_api_cfg_set_log_{level, format} are part of dynamic_cfg (see
box/lua/load_cfg.lua) and are expected to throw exceptions (see
reload_cfg ib.).

Needed for #6086