master1.lua did not reach all needed remote branches with
adb0a01b.
It is needed for join_vclock test.

socket.test had a number of flaky problems:
- socket readiness expectation & read timeouts
- race conditions on socket shutdown in emulation test cases
- UDP datagrams losses on mac os
- excessive random port searches
Now they are solved. 127.0.0.1 is now used instead of 0.0.0.0 or
localhost to prevent wrong connections where appropriate. Socket test
is not fragile anymore.

Closes #4426
Closes #4451
Closes #4469

(cherry picked from commit 4137134c)

join_vclock test is assumed to verify that changes are not being lost
on the replica. Due to this the test is changed to explicitly check
that all changes on master are applied on replica.
Previously this test was also indirectly verifying that changes are
being applied in the correct order. Now there is separate test for
this, called replica_apply_order.
As far as changed join_vclock test might fail due to #4669, we are now
creating cluster out of fresh instances instead of using default
instance. Considering mentioned fixes it is not fragile anymore.

Closes #4160

(cherry picked from commit adb0a01b)

* Support to set default SQL engine using _session_settings space in
  addition to pragma sql_default_engine. This feature is only for
  *.test.sql tests. Needed for #4511.

* Use exact IPv4/IPv6 address in test_run:cmd() in order to avoid rare
  failures due to using wrong address (PR #197).

(cherry picked from commit a6535ab3b8d3e395d2f81699cf35329118412307)

In scope of 8fac6972 it was fixed misbehavior while passing floating
point values to integer iterator. Unfortunately, that patch introduced
off-by-one error. Number of constraints (equalities and inequalities)
can not be greater than number of key parts (obviously, each constraint
can be tested against at most one index part). Inequality constraint can
involve only field representing last key part. To get it number of
constraints was used as index. However, since array is 0-based, the last
key part should be calculated as parts[eq_numb - 1]. Before this patch
it was parts[eq_numb].

Closes #4558

(cherry picked from commit bb905d1d)

The test re encoding of -2^63 Lua number value did use update by a field
name, which does not supported in 1.10 and 2.2 branches. Field name
updates are orthogonal to Lua number serialization and we don't intend
to test them here. So it is safe and logical to get rid of them in the
test.

This change allow the test to pass on 1.10 and 2.2 branches.

Follows up #4672.

Reviewed-by: Alexander Tikhonov <avtikhon@tarantool.org>
Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
(cherry picked from commit 408b9ad4)

Accidentally, number of indexes to be considered during query planning
in presence of INDEXED BY is calculated wrong. Instead of one (INDEXED
BY is not a hint but requirement) index to be used (which is indicated
in INDEXED BY clause), all space indexes take part in query planning.
There are not so many tests checking this feature, so unfortunately this
bug was hidden. Let's fix it and force only one index to be used in QP
in case of INDEXED BY clause.

(cherry picked from commit 49fedfe3)

libcurl-7.66.0 and older returns CURLE_WRITE_ERROR when a server
responds with unknown or unsupported Content-Encoding (see [1] and [2]).
This was fixed in future libcurl-7.67.0 and proper
CURLE_BAD_CONTENT_ENCODING code will be returned in this case.

We should process the code in the same way as we do for
CURLE_WRITE_ERROR.

[1]: curl/curl#4310
[2]: curl/curl#4449

Closes #4579

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
(cherry picked from commit e3432636)

After executing curl request we need to process curl_request code. It
might be CURLE_WRITE_ERROR. We had special case for it, which assumed
diagnostic message being set and contained corresponding assert, though
it is incorrect. Better way is to handle it as any other non-standard
event.

It was discovered while adding accept_encoding option. In case of
unknown encoding curl_request code is currently set to CURLE_WRITE_ERROR
and therefore we come to an assert assuming we have some diagnostics
set. However, it is not being set and it is totally fine. This means we
are failing on assert and it is not correct behavior.

Prerequisites: #4232

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
(cherry picked from commit 728d08a6)

This patch fixes a bug that appeared after commit
3a13be1d ('lua: fix lua_tofield()
for 2**64 value') .

Due to this bug, -2^63 was serialized as double, although it
should be serialized as integer.

Closes #4672

(cherry picked from commit 06aeb768)

Before this patch if user passed {verbose = false}
to http client it was considered as "true"

This patch fixes such behaviour and takes into account
user's value

(cherry picked from commit 77936721c02a89e17701c64f0f442abdf78ed7fc)

JetBrains IDEs use .idea folder to store configuration files.
To provide more comfort for it's users .idea folder should be ignored.

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
(cherry picked from commit afa692e5)

As long as we are sure, that strlen(sd_unix_path) < sizeof(sa.sun_path)
we can assume that there is always enough space and the path will be
null-terminated. Thus, copy 1 byte less to get rid of the warning.

Closes #4515

(cherry picked from commit 49fa68a7)

The test started failing after my commit:
ca07088c
(func: fix not unloading of unused modules), because I forgot to
update the result file.

Follow up #4648

(cherry picked from commit eddc8cc0)

C functions are loaded from .so/.dylib dynamic libraries. A
library is loaded when any function from there is called first
time. And was supposed to be unloaded, when all its functions are
dropped from the schema (box.schema.func.drop()), and none of them
is still in a call. But the unloading part was broken.

In fact, box.schema.func.drop() never unloaded anything. Moreover,
when functions from the module were added again without a restart,
it led to a second mmap of the same module. And so on, the same
library could be loaded any number of times.

The problem was in a useless flag in struct module preventing its
unloading even when it is totally unused. It is dropped.

Closes #4648

Error injections are used to simulate an error. They are
represented as a flag, or a number, and are used in Lua tests. But
they don't have any feedback. That makes impossible to use the
injections to check that something has happened. Something very
needed to be checked, and impossible to check in a different way.

More certainly, the patch is motivated by a necessity to count
loaded dynamic libraries to ensure, that they are loaded and
unloaded when expected. This is impossible to do in a platform
independent way. But an error injection as a debug-only counter
would solve the problem.

Needed for #4648

snrpintf always null-terminates the passed string, and it also returns
the number of bytes that "would have been written if there was enough
space", so not only we don't have to null-terminate the string, but even
more so we shouldn't do it erroneously. The only case when a string
should be null-terminated manually is when the print cycle doesn't run
at all, so move the termination before the cycle.

Closes #4636

(cherry picked from commit 2066f297)

It was requested to be raised from 32 to 128

Closes #4670

(cherry picked from commit a4e514e46ca87242fe9d72e63e0079a1fec24990)

Closes #4519

@TarantoolBot document
Title: key_def.new() accept both 'field' and 'fieldno'

Before the patch key_def.new() took an index part
array as it is returned in <index_object>.parts: each
part should include 'type', 'fieldno', and what else
.parts element contains.

But it was not possible to create a key_def from an
index definition - the array passed to
<space_object>.create_index() 'parts' argument. Because
key_def.new() didn't recognize 'field' option. That
might be useful, when a key_def is needed on a remote
client, where a space object and its indexes do not
exist. And it would be strange to force a user to
create them just so as he would be able to access

    <net_box connection>.space.<space_name>.
        index.<index_name>.parts

As well as it would be crutchy to make a user manually
replace 'field' with 'fieldno' in its index definition
just to create a key_def.

Additionally, an ability to pass an index definition
to a key_def constructor makes the API more symmetric.

Note, that it still is not 100% symmetric, because a
user can't pass field names to the key_def
constructor. A space is needed for that anyway.

(cherry picked from commit 39918baf)

The problem was in that the test uses the global trigger
box.session.on_disconnect() to set a global variable by one
connection. But test-run can do multiple connects/reconnects to
the same instance. That led to multiple invocations of
box.session.on_disconnect(), which could override the global
variable in unexpected ways and moments.

The patch makes only one session execute that trigger.

Probably related to https://github.com/tarantool/test-run/issues/46


Follow up #4627

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
(cherry picked from commit 9643fdc7)

List of changes
     - fold: keep type of emitted CONV in sync with its mode
     - test: adjust the test name related to PAIRSMM flag

(cherry picked from commit e12930d0)

Turns on LUAJIT_ENABLE_PAIRSMM flag for tarantool build.
Now __pairs/__ipairs metamethods are available.

Closes #4650

(cherry picked from commit b504ca1a096a839f3a4fddc72a33457a3f0dc700)

Added build + test jobs in GitLab-CI and build + test + deploy jobs on
Travis-CI for Fedora 31.

Updated testing dependencies in the RPM spec to follow the new Python 2
package naming scheme that was introduced in Fedora 31: it uses
python2-' prefix rather then 'python-'.

Fedora 31 does not provide python2-gevent and python2-greenlet packages,
so they were pushed to https://packagecloud.io/packpack/backports
repository. This repository is enabled in our build image
(packpack/packpack:fedora-31) by default. Those dependencies are build-time,
so nothing was changed for a user. The source RPM packages were gathered
from https://rpms.remirepo.net/rpmphp/

.

Closes #4612

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
(cherry picked from commit 9e09b07c)

Strengthen test_run:cmd() against temporary connection failures (#193).

We recently added 'replication/box_set_replication_stress' test that may
exceed file descriptor limit. When test_run:cmd() function executes a
command ('switch master' in the case), it tries to create a new socket
and connect it to test-run's inspector, but it may fail to do so in the
case, because of the file descriptor limit.

The sockets that the test produces are closed in background, so if we'll
keep trying to create and connect a socket we'll succeed once. This is
exactly that the test-run's patch doing: it fails test_run:cmd()
function only if a socket cannot be connected during 100 seconds.

I guess that the reason why sockets are not closed immediately is that
relays wait until replicas will close its side of a socket and only then
closes its side. Didn't investigate it deeper, to be honest.

(cherry picked from commit 5fccf003)

Inside json_decode() struct luaL_serializer is allocated on stack, but
json context stores pointer to it:

   998	static int json_decode(lua_State *l)
   999	{
  ...
  1007	    if (lua_gettop(l) == 2) {
  1008	        struct luaL_serializer user_cfg = *luaL_checkserializer(l);
  1009	        luaL_serializer_parse_options(l, &user_cfg);
  1010	        lua_pop(l, 1);
  1011	        json.cfg = &user_cfg;
  1012      }

Later (for instance in json_decode_descend()), it can be dereferenced
which in turn results in stack-use-after-scope (object turns into
garbage right after scope is ended). To fix it let's simply avoid
allocating and copying luaL_serializer on stack and instead use pointer
to it.

Bug is found by ASAN: test app-tap/json.test.lua fails with enabled
ASAN. Current fix allows to pass all tests.

Thanks to @Korablev77 for the initial investigation.

Closes #4637

(cherry picked from commit 6508ddb7)

This reverts commit a0b196dd.

This commit was pushed occasionally and points to a draft commit in
test-run repository. See also
https://github.com/tarantool/test-run/issues/195

(cherry picked from commit 4acdeeda)

There were some pass conditions in quorum test which could take some
time to be satisfied. Now they are wrapped using test_run:wait_cond to
make the test stable.

Closes #4586

(cherry picked from commit f6775e86)

In replicaset_follow we iterate anon replicas list: list of replicas
that haven't received an UUID. In case of successful connect replica
link is being removed from anon list. If it happens immediately,
without yield in applier, iteration breaks. Now it is fixed by
rlist_foreach_entry_safe instead of common rlist_foreach_entry.
Relevant test case is added.

Part of #4586
Closes #4576
Closes #4440

(cherry picked from commit 6f038f4b)

During pruning of appliers some anon replicas might connect
from replicaset_follow called in another fiber. Therefore we need to
prune appliers of anon replicas first and, moreover, prune them one by
one instead of iterating them, as far as any of them might connect
while we are stopping the other one and it will break iteration.

Part of #4586
Closes #4643

(cherry picked from commit 36ff3c89)

Stabilize tcp_connect in test_run:cmd() (tarantool/test-run#193)

(cherry picked from commit a0b196dd)

Binary session disconnect trigger yield could lead to use after
free of the session object. That happened because iproto thread
sent two requests to TX thread at disconnect:

    - Close the session and run its on disconnect triggers;

    - If all requests are handled, destroy the session.

When a connection is idle, all requests are handled, so both these
requests are sent. If the first one yielded in TX thread, the
second one arrived and destroyed the session right under the feet
of the first one.

This can be solved in two ways - in TX thread, and in iproto
thread.

Iproto thread solution (which is chosen in the patch): just don't
send destroy request until disconnect returns back to iproto
thread.

TX thread solution (alternative): add a flag which says whether
disconnect is processed by TX. When destroy request arrives, it
checks the flag. If disconnect is not done, the destroy request
waits on a condition variable until it is.

The iproto is a bit tricker to implement, but it looks more
correct.

Closes #4627

(cherry picked from commit 6da9d395)

Bootstrap.snap is created from a normal snapshot file, but with
erased VClock option in the header:

    SNAP
    0.13
    Version: 2.2.1-122-g1146bb78d
    Instance: 03d3836a-e608-421c-9f8d-ad9beefe7440
    VClock: {}

In a normal snapshot it is 'VClock: {1: ...}'. To erase the option
usually developers use 'vim'. But when a binary file is opened in
vim without any arguments, like this:

    vim bootstrap.snap

on close it will edit some parts of the file in unexpected ways,
depending on local vim settings.

To forbid any implicit changes binary mode should be used:

    vim -b bootstrap.snap

The patch regenerates bootstrap.snap and drops VClock using binary
mode vim.

Closes #4510

Add LUAJIT_ENABLE_PAIRSMM flag as a build option for luajit.
If the flag is set, pairs/ipairs metamethods are available in
Lua 5.1.
For Tarantool this option is enabled by default.

(cherry picked from commit 93e710d5de0d723086bda6fedc9cb383a8e5e477)

Replication's applier encoded an auth request with exactly the
same parameters as extracted by the URI parser. I.e. when no
password was specified, the parser returned it as NULL, and it was
not encoded. The relay, received such an auth request, complained
that IPROTO_TUPLE field is not specified (this is password).

Such an error confuses - a user didn't do anything illegal, he
just used URI like 'login@host:port', without a password after the
login.

The patch makes the applier use an empty string as a default
password.

An alternative was to force a user always set a password even if
it is an empty string, like that: 'login:@host:port'. And if a
password was not found in an auth request, then reject it with a
password mismatch error. But in that case a URI of kind
'login@host:port' becomes useless - it can never pass. In
addition, netbox already uses an empty string as a default
password. So the only way to make it consistent, and don't break
anything - repeat netbox logic for replication URIs.

Closes #4605

Conflicts:
	test/replication/suite.cfg

(cherry picked from commit 6c01ca48)

Conflicts:
	test/replication/suite.cfg

Functions are stored in lists inside module objects. Module
objects are stored in a hash table, where key is a package name.
But the key was a pointer at one of module's function definition
object. Therefore, when that function was deleted, its freed
package name memory was still in the hash key, and could be
accessed, when another function was deleted.

Now module does not use memory of its functions, and keep a copy
of the package name.

(cherry picked from commit fa2893ea)

The problem appears after 6c627af3
('test: tarantoolctl: verify delayed box.cfg()'), where the test case
was changed and it doesn't more assume an error at the instance start.
So we need to stop it to prevent a situation when instances are stay
after `make test`.

Fixes #4600.
Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>

(cherry picked from commit 8d363c43)

Before commit 03f85d4c ('app: fix
boolean handling in argparse module') the module does not expect a value
after a 'boolean' argument. However there was the problem: a 'boolean'
argument can be passed only at end of an argument list, otherwise it
wrongly consumes a next argument and gives a confusing error message.

The mentioned commit fixes this behaviour in the following way: it still
allows to pass a 'boolean' argument at end of the list w/o a value, but
requires a value ('true', 'false', '1', '0') if a 'boolean' argument is
not at the end to be provided using {'--foo=true'} or {'--foo', 'true'}
syntax.

Here this behaviour is changed: a 'boolean' argument does not assume an
explicitly passed value despite its position in an argument list. If a
'boolean' argument appears in the list, then argparse.parse() returns
`true` for its value (a list of `true` values in case of 'boolean+'
argument), otherwise it will not be added to the result.

This change also makes the behaviour of long (--foo) and short (-f)
'boolean' options consistent.

The motivation of the change is simple: it is easier and more natural to
type, say, `tarantoolctl cat --show-system 00000000000000000000.snap`
then `tarantoolctl cat --show-system true 00000000000000000000.snap`.

This commit adds several new test cases, but it does not mean that we
guarantee that the module behaviour will not be changed around some
corner cases, say, handling of 'boolean+' arguments. This is internal
module.

Follows up #4076.
Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>

(cherry picked from commit e47f2c91)

The admin user has universal privileges before bootstrap or
recovery are done. That allows to, for example, bootstrap from a
remote master, because to do that the admin should be able to
insert into system spaces, such as _priv.

But after the patch on online credentials update was implemented
(#2763, 48d00b0e) the admin could
loose its universal access if, for example, a role was granted to
him before universal access was recovered.

That happened by two reasons:

    - Any change in access rights, even in granted roles, led to
      rebuild of universal access;

    - Any change in access rights updated the universal access in
      all existing sessions, thanks to #2763.

What happened: two tarantools were started. One of them master,
granted 'replication' role to admin. Second node, slave, tried to
bootstrap from the master. The slave created an admin session and
started loading data. After it loaded 'grant replication role to
admin' command, this nullified admin universal access everywhere,
including this session. Next rows could not be applied.

Closes #4606

(cherry picked from commit 95237ac8)

After the issue #4537 fixed for the data segment size limit,
the temporary blocked tests because of it unblocked.

Part of #4271

(cherry picked from commit e6866550)