It started to hang after the last merge.

box.cfg() updates only those options that have actually changed.
However, for replication it is not always true: box.cfg{replication = x}
and box.cfg{replication = {x}} are treated differently, and as
the result - replication is restarted. The patch fixes such behaviour.

Closes #3711

Currently, run_count_per_level index option is applied to each LSM tree
level. As a result, we may end up storing each key run_count_per_level
times at the last level alone, which would result in prohibitive space
amplification. To avoid that, let's ignore run_count_per_level for the
last level.

Note, we have to tweak quite a few vinyl tests, because they implicitly
relied on the fact that producing run_count_per_level dumps would never
trigger compaction.

Closes #3657

I. Fixes tarantoolctl rocks install hanging in resctricted networks
corner-case.

A customer configured two rocks servers:
1. offline (file:///path/to/rocks)
2. and default online (rocks.tarantool.org)

He tries to do `rocks install http 1.0.5-1`.

Online server is unavailable due to his local network policy, but
the rock is available offline. Despite anything, luarocks still
tries to fetch manifest online, which results in 30 sec hang since
network access is restricted.

This change aborts scanning when exact match is found

II. Remove cyclic dependencies

This is required to embed luarocks into tarantool, as
current tarantool preloader can't preload cyclic dependencies.
There should be a unidirectional dependency graph and
predictable order.

Note: as a consequence of this patch, operating systems other that
unix-compatible ones are no longer supported. This is because I
had to manually resolve dependency graph for predictable require()
order.

III. Use digest.md5_hex to compute md5 digests instead of openssl

luarocks has support for calculating md5 with 'md5' rock if it's
present, but we don't have it in tarantool, and instead have the
'digest' module. That's why luarocks falls back to 'openssl' binary
to calculate md5 digests.

This patch will allow luarocks to use our internal digests module.

In box_cfg() we have a call to gc_set_wal_watcher(), which creates pipes
between 'wal' and 'tx' under the hood using cbus_pair().
While pipes are being created, the fiber calling gc_set_wal_watcher()
will process all the messages coming to 'tx' thread from iproto. This is
wrong, since we have a separate fiber pool to handle iproto messages,
and background fibers shouldn't participate in these messages
processing. For example, this causes occasional credential corruption in
the fiber executing box_cfg().

Since tx fiber pool is already created at the time gc_set_wal_watcher()
is called, we may forbid message processing for the fiber which calls
the function, one of the tx fiber pool fibers will wake us up when the
pipes are created.

Closes #3779

Closes: #3761

Closes #3790

FreeBSD 10.4 has no libdl.so.

Fixes #3750.

FreeBSD does not include headers recursively, so we need to include it
explicitly at least for using IPPROTO_UDP macro.

Thanks Po-Chuan Hsieh (@sunpoet) for the fix proposal (PR #3739).

Fixes #3677.

If field isn't defined by space format, than in case of multiple
indexes field option is_nullable was the same as it was for last
index that defines it. This is wrong as it should be 'true' only
if it is 'true' for all indexes that defines it.

Closes #3744.

Trace corresponding Lua state as well as normal C stack frames while
fiber backtracing. This might be useful for debugging purposes.

Fixes: #3538

Lua trigger invocation reuses a fiber lua state if exists instead of
creating of new one for each new invocation. This is needed for a lua
stack reconstruction during backtracing.

Relates: #3538

Each yielded fiber has a preserved coro state stored in a corresponding
variable however an executing fiber has a volatile state placed in CPU
registers (stack pointer, instruction pointer and non-volatile registers)
and corresponding context-storing variable value is invalid.
For already yielded fiber we have to use a special asm-written handler to make
a temporary switch to the preserved state and capture executing context what
is not needed for executing fiber.
After the patch for the executing fiber NULL is passed to the backtrace
function as coro context and then backtrace function could decide should
it use special context-switching handler or might just use unw_getcontext
from the unwind library.

Do not inline coro_unwcontext because the unwind handler expects
for a separate stack frame.

Starting from libcurl 7.62.0, CURL_SSL_CACERT is defined as a macro
alias to CURLE_PEER_FAILED_VERIFICATION, see

  https://github.com/curl/curl/commit/3f3b26d6feb0667714902e836af608094235fca2

This breaks compilation:

  httpc.c:337:7: error: duplicate case value 'CURLE_PEER_FAILED_VERIFICATION'
          case CURLE_PEER_FAILED_VERIFICATION:
               ^
  httpc.c:336:7: note: previous case defined here
          case CURLE_SSL_CACERT:
               ^
  curl.h:589:26: note: expanded from macro 'CURLE_SSL_CACERT'
  #define CURLE_SSL_CACERT CURLE_PEER_FAILED_VERIFICATION
                           ^

Fix this by using CURLE_SSL_CACERT only if libcurl version is less
than 7.62.0.

Note, we can't use CURL_AT_LEAST_VERSION to check libcurl version,
because it isn't available in libcurl shipped with CentOS 6.

Obviously, the version check as it is now won't work once libcurl 8.0.0
is released. Use LIBCURL_VERSION_NUM to correctly check libcurl version.

Note, we can't use CURL_AT_LEAST_VERSION to check libcurl version,
because it isn't available in libcurl shipped with CentOS 6.

Fixex commit 7e62ac79 ("Add HTTP client based on libcurl").

In Vinyl DDL aborts all affected writers before modifying a space so we
must use pcall() to avoid hang.

Closes #3786

When space.before_replace trigger modifies the result of a remote
operation, we clear the request header so that it gets rebuilt on
commit. This is incorrect, because as a result we don't bump the
master's component of the replica's vclock, which leads to the request
being applied again when the replica reconnects. The issue manifests
itself in sporadic replication/before_replace test failures.

Fix it by updating the request header rather than clearing it so that
replica id and lsn get preserved.

Closes #3722

To be able to switch to hot_standby instance with minimal downtime, we
need to know how far is it behind the primary instance, i.e. up to what
vclock we have recovered. Previously this was impossible because
box.info.vclock always referenced replicaset.vclock, which isn't updated
during hot_standby.

Introduce a pointer to relevant vclock: either recovery vclock (during
local recovery) or replicaset.vclock (at all other times) and use it in
box.info.vclock, box.info.lsn and box.info.signature.

@locker: renamed last_row_vclock to box_vclock and constified it.

Closes #3002

Increased socket.readable / socket.wait timeouts.

Rewritten port choosing: repeat bind+listen until succeeds, exclude
incorrect port 65536 from the range.

All these changes are needed to run the test in parallel on several
test-run workers to investigate flaky failures of the test / of the
suite. Some of these changes can also eliminate possible flaky failures.

In some cases luajit does not collect cdata objects which were
transformed with ffi.cast as tuple_bless does. In consequence, internal
table with gc callback overflows and then lua crashes. There might be an
internal luajit issue because it fires only for jitted code. But assigning
a gc callback before transformation fixes the problem.

Closes #3751

Back when bloom filters were introduced, neither box.info.memory() nor
box.stat.vinyl().memory didn't exist so bloom filters were accounted to
box.runtime.info().used for lack of a better place. Now, there's no
point to account them there. In fact, it's confusing, because bloom
filters are allocated with malloc(), not from the runtime arena, so
let's drop it.

If a tuple read from a run by a slice stream happens to be out of the
slice bounds, it will never be freed. Fix it.

The leak was introduced by commit c174c985 ("vinyl: implement new
simple write iterator").

Upload tarballs of alpha and beta tarantool versions (*.0 and *.1
branches) into 2x (3x, 4x...) buckets. See more details about the
release process in the documentation: [1].

[1]: https://tarantool.io/en/doc/2.0/dev_guide/release_management/

This patch adds logging amount of rows received by applier during the
join stage, the same way that recovery has it.

Closes #3165

Remove function which deletes from cache, making replace more general:
it might be used for both insertions, deletions and replaces. Also, put
assert on equality of space pointer found in cache to old one into
replace routine.

Now if the WAL thread fails to preallocate disk space needed to commit
a transaction, it will delete old WAL files until it succeeds or it
deletes all files that are not needed for local recovery from the oldest
checkpoint. After it deletes a file, it notifies the garbage collector
via the WAL watcher interface. The latter then deactivates consumers
that would need deleted files.

The user doesn't see a ENOSPC error if the WAL thread successfully
allocates disk space after deleting old files. Here's what's printed
to the log when this happens:

  wal/101/main C> ran out of disk space, try to delete old WAL files
  wal/101/main I> removed /home/vlad/src/tarantool/test/var/001_replication/master/00000000000000000005.xlog
  wal/101/main I> removed /home/vlad/src/tarantool/test/var/001_replication/master/00000000000000000006.xlog
  wal/101/main I> removed /home/vlad/src/tarantool/test/var/001_replication/master/00000000000000000007.xlog
  main/105/main C> deactivated WAL consumer replica 82d0fa3f-6881-4bc5-a2c0-a0f5dcf80120 at {1: 5}
  main/105/main C> deactivated WAL consumer replica 98dce0a8-1213-4824-b31e-c7e3c4eaf437 at {1: 7}

Closes #3397

In order to implement WAL auto-deletion, we need a notification channel
through which the WAL thread could notify TX that a WAL file was deleted
so that the latter can shoot off stale replicas. We will reuse existing
wal_watcher API for this. Currently, wal_watcher invokes the registered
callback on each WAL write so using it as is would be inefficient. To
avoid that, let's allow the caller to specify events of interest when
registering a wal_watcher.

Needed for #3397

We will add another event bitmap to wal_watcher. To avoid confusion
between them, let's rename wal_watcher->events.

This should make it easier to pass some extra information along with the
event mask. For example, we will use it to pass the vclock of the oldest
stored WAL, which is needed for WAL auto-deletion.

Needed for #3397

Updated the test case for #2780 to check a last snapshot file
modification time instead of search log messages.

The test was flaky, because of small timeouts on Linux, but now we
spinning on a condition check to achieve both stable results and fast
execution.

Follows up #2780.
Fixes #3684.