After the previous commit ('popen: require popen handle to be non-NULL')
it turns out that popen_state() function always succeeds. There is no
reason to return a success / failure value from it.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

Further commits will add proper entries into the diagnostics area for
failures inside popen functions. We should either report handle == NULL
case via the diagnostics area or ensure that the NULL handle case is not
possible.

The latter approach is implemented in this commit. There are two
reasons for this:

* This way simplifies function contracts (one less kind of failure).
* The popen Lua module (that will be implemented in the further commits)
  will not construct any logic using NULL as a handle. When 'NULL
  handle' error is not possible in the C API, it will be easier to
  verify that this failure is not possible the Lua API.

A user of the C API should take care to don't call those functions with
NULL handle.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

Test that diag_raise doesn't happen if async transaction
fails inside replication procedure.

Side note: I don't like merging tests with patches in
general and I hate doing so for big tests with a passion
because it hides the patch code itself. So here is a
separate patch on top of the fix.

Test-of #4730

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Currently when transaction rollback happens we just drop an existing
error setting ClientError to the replicaset.applier.diag. This action
leaves current fiber with diag=nil, which in turn leads to sigsegv once
diag_raise() called right after applier_apply_tx():

 | applier_f
 |   try {
 |   applier_subscribe
 |     applier_apply_tx
 |       // error happens
 |       txn_rollback
 |         diag_set(ClientError, ER_WAL_IO)
 |         diag_move(&fiber()->diag, &replicaset.applier.diag)
 |         // fiber->diag = nil
 |       applier_on_rollback
 |         diag_add_error(&applier->diag, diag_last_error(&replicaset.applier.diag)
 |         fiber_cancel(applier->reader);
 |     diag_raise() -> NULL dereference
 |   } catch { ... }

Thus:
 - use diag_set_error() instead of diag_move() to not drop error
   from a current fiber() preventing a nil dereference;
 - put fixme mark into the code: we need to rework it in a
   more sense way.

Fixes #4730

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

To make it a bit more readable.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

For some reason the replica_by_id member (which is an
array of pointers) is allocated dynamically. Moreover
VCLOCK_MAX = 32 by now and extending it to some new
limit will require a way more efforts than just increase
the number.

Thus reserve memory for replica_by_id inside replicaset
statically. This allows to simplify code a bit and
drop calloc/free calls.

The former code comes from edd76a2a without any
explanation why the dynamic member is needed.

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

In case if we're hitting memory limit allocating triggers
we should setup diag error to prevent nil dereference
in diag_raise call (for example from applier_apply_tx).

Note that there are region_alloc_xc helpers which are
throwing errors but as far as I understand we need the
rollback action to process first instead of immediate
throw/catch thus we use diag_set.

Acked-by: Sergey Ostanevich <sergos@tarantool.org>
Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

In request_create_from_tuple and request_handle_sequence
we may be unable to request memory for tuples, don't
forget to setup diag error otherwise diag_raise will
lead to nil dereference.

Acked-by: Sergey Ostanevich <sergos@tarantool.org>
Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Instead of calling memset which is useless here
just use trigger_create helper.

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

We're not starting new master node but
a new instance instead. The comment simply
leftover from older modifications.

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Before this patch prepared statements didn't reset bound values after
its execution. As a result, if during next execution cycle not all
parameters were provided, cached values would appear. For instance:

prep = box.prepare('select :a, :b, :c')
prep:execute({{[':a'] = 1}, {[':b'] = 2}, {[':c'] = 3}}
-- [1, 2, 3]
prep:execute({{[':a'] = 1}, {[':b'] = 2}})
-- [1, 2, 3]

However, expected result for the last query should be [1, 2, NULL].
Let's fix it and always reset all binding values before next execution.

Closes #4825

This patch introduces support of stacked errors in IProto protocol and
in net.box module.

Closes #1148

@TarantoolBot document
Title: Stacked error diagnostic area

Starting from now errors can be organized into lists. To achieve this
Lua table representing error object is extended with .prev field and
e:set_prev(err) method. .prev field returns previous error if any exist.
e:set_prev(err) method expects err to be error object or nil and sets
err as previous error of e. For instance:
```
e1 = box.error.new({code = 111, reason = "cause"})
e2 = box.error.new({code = 111, reason = "cause of cause"})

e1:set_prev(e2)
assert(e1.prev == e2) -- true
```
Cycles are not allowed for error lists:
```
e2:set_prev(e1)
- error: 'builtin/error.lua: Cycles are not allowed'
```
Nil is valid input to :set_prev() method:
```
e1:set_prev(nil)
assert(e1.prev == nil) -- true
```
Note that error can be 'previous' only to the one error at once:
```
e1:set_prev(e2)
e3:set_prev(e2)
assert(e1.prev == nil) -- true
assert(e3.prev == e2) -- true
```
Setting previous error does not erase its own previous members:
```
-- e1 -> e2 -> e3 -> e4
e1:set_prev(e2)
e2:set_prev(e3)
e3:set_prev(e4)
e2:set_prev(e5)
-- Now there are two lists: e1->e2->e5 and e3->e4
assert(e1.prev == e2) -- true
assert(e2.prev == e5) -- true
assert(e3.prev == e4) -- true
```
Alternatively:
```
e1:set_prev(e2)
e2:set_prev(e3)
e3:set_prev(e4)
e5:set_prev(e3)
-- Now there are two lists: e1->e2 and e5->e3->e4
assert(e1.prev == e2) -- true
assert(e2.prev == nil) -- true
assert(e5.prev == e3) -- true
assert(e3.prev == e4) -- true
``
Stacked diagnostics is also supported by IProto protocol. Now responses
containing errors always (even if there's only one error to be returned)
include new IProto key: IPROTO_ERROR_STACK (0x51). So, body corresponding to
error response now looks like:
```
MAP{IPROTO_ERROR : string, IPROTO_ERROR_STACK : ARRAY[MAP{ERROR_CODE : uint, ERROR_MESSAGE : string}, MAP{...}, ...]}
```
where IPROTO_ERROR is 0x31 key, IPROTO_ERROR_STACK is 0x52, ERROR_CODE
is 0x01 and ERROR_MESSAGE is 0x02.
Instances of older versions (without support of stacked errors in
protocol) simply ignore unknown keys and still rely only on IPROTO_ERROR
key.

This patch makes box.error() always promote error to the diagnostic
area despite of passed arguments.

Closes #4829

@TarantoolBot document
Title: always promote error created via box.error() to diag

box.error() is able to accept two types of argument: either pair of code
and reason (box.error{code = 555, reason = 'Arbitrary message'}) or error
object (box.error(err)). In the first case error is promoted to
diagnostic area, meanwhile in the latter - it is not:
```
e1 = box.error.new({code = 111, reason = "cause"})
box.error({code = 111, reason = "err"})
- error: err
box.error.last()
- err
box.error(e1)
- error: cause
box.error.last()
- err
```
From now box.error(e1) sets error to diagnostic area as well:
```
box.error(e1)
- error: cause
box.error.last()
- cause
```

Refactor iproto_reply_error and iproto_write_error with a new
mpstream-based helper mpstream_iproto_encode_error that encodes
error object for iproto protocol on a given stream object.
Previously each routine implemented an own error encoding, but
with the increasing complexity of encode operation with following
patches we need a uniform way to do it.

The iproto_write_error routine starts using region location
to use region-based mpstream. It is not a problem itself, because
errors reporting is not really performance-critical path.

Needed for #1148

Since we've introduced stacked diagnostic in previous commit, let's use
it in the code implementing functional indexes.

Part of #1148

In terms of implementation, now struct error objects can be organized
into double-linked lists. To achieve this pointers to the next and
previous elements (cause and effect correspondingly) have been added to
struct error. It is worth mentioning that already existing rlist and
stailq list implementations are not suitable: rlist is cycled list, as a
result it is impossible to start iteration over the list from random
list entry and finish it at the logical end of the list; stailq is
single-linked list leaving no possibility to remove elements from the
middle of the list.

As a part of C interface, box_error_add() has been introduced. In
contrast to box_error_set() it does not replace last raised error, but
instead it adds error to the list of diagnostic errors having already
been set. If error is to be deleted (its reference counter hits 0 value)
it is unlinked from the list it belongs to and destroyed. Meanwhile,
error destruction leads to decrement of reference counter of its
previous error and so on.

To organize errors into lists in Lua, table representing error object in
Lua now has .prev field (corresponding to 'previous' error) and method
:set_prev(e). The latter accepts error object (i.e. created via
box.error.new() or box.error.last()) and nil value. Both field .prev and
:set_prev() method are implemented as ffi functions. Also note that
cycles are not allowed while organizing errors into lists:
e1 -> e2 -> e3; e3:set_prev(e1) -- would lead to error.

Part of #1148

Found that 'openssl-1.1.0h' is EOL and its external link was removed.
Fixed static build based on Dockerfile with the new link to the external
openssl version 'openssl-1.1.1f'.

Close #4830

@Changelog
Fixed the tarantoolctl rocks remove flag --force

Forwarding of the --force flag to tarantoolctl rocks module was added.
(Command: tarantoolctl rocks remove --force)

Fixes: #3632

@Changelog
Fixed the tarantoolctl rocks search flag --all

Forwarding of the --all flag to tarantoolctl rocks module was added.
(Command: tarantoolctl rocks search --all)

Fixes: #4529

Fixed static build with '-DBUILD_STATIC=ON' option:

 - installed liblzma-dev library for libunwind static, due to found that
   static libunwind library uses undefined lzma functions:
     nm -a /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a | grep lzma
                 U lzma_index_buffer_decode
                 U lzma_index_end
                 U lzma_index_size
                 U lzma_index_uncompressed_size
                 U lzma_stream_buffer_decode
                 U lzma_stream_footer_decode
   while dynamic libunwind correctly sees liblzma installed:
     ldd /usr/lib/x86_64-linux-gnu/libunwind-x86_64.so | grep lzma
       liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8fd1c23000)
   so to fix it the static library of lzma was needed.

 - added lzma library to unwind library for Tarantool build at file:
     cmake/compiler.cmake
   due to fail:
     /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o):
       In function `xz_uncompressed_size':
     ./src/elfxx.c:194: undefined reference to `lzma_stream_footer_decode'
     ./src/elfxx.c:201: undefined reference to `lzma_index_buffer_decode'
     ./src/elfxx.c:205: undefined reference to `lzma_index_size'
     ./src/elfxx.c:210: undefined reference to `lzma_index_end'
     ./src/elfxx.c:207: undefined reference to `lzma_index_uncompressed_size'
     ./src/elfxx.c:210: undefined reference to `lzma_index_end'
     /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o):
       In function `_Uelf64_extract_minidebuginfo':
     ./src/elfxx.c:278: undefined reference to `lzma_stream_buffer_decode'
     collect2: error: ld returned 1 exit status
     test/unit/CMakeFiles/luaL_iterator.test.dir/build.make:134:
       recipe for target 'test/unit/luaL_iterator.test' failed
     make[2]: *** [test/unit/luaL_iterator.test] Error 1

 - added dl library to gomp library for test/unit tests
   binaries builds at file:
     cmake/BuildMisc.cmake
   due to fail:
     /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o):(.text+0x34d):
       more undefined references to `dlsym' follow
     /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o): In function
       `gomp_target_init':
     (.text+0x9cc): undefined reference to `dlerror'
     collect2: error: ld returned 1 exit status

  - added dl library to icu library for test/unit tests
   binaries builds at file:
     cmake/FindICU.cmake
   due to fail:
     /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
       In function `uprv_dl_open_60':
     (.text+0x1ce2): undefined reference to `dlopen'
     /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
       In function `uprv_dlsym_func_60':
     (.text+0x1d3d): undefined reference to `dlsym'
     /usr/x86_64-linux-gnu/libicuuc.a(putil.ao):
       In function `uprv_dl_close_60':
     (.text+0x1d21): undefined reference to `dlclose'
     collect2: error: ld returned 1 exit status

Added static build to gitlab-ci in release check criteria named
as static_build job. Previously named static_build job renamed to
static_docker_build, due to it checks the build at Dockerfile.

Also moved static build make targets from .gitlab.mk to .travis.mk
to store it in common place with the other test/build make targets.
Moved environement from .gitlab-ci.yml file into make targets to
make this targets true building in static w/o additional setup.

Close #4551

Found that wget may fail on downloading the file from flaky
available servers with 500 HTTP error, like it was seen on
icu4c sources downloading, please check the issue:
https://sourceforge.net/p/forge/site-support/20071/

Found that curl successfully downloads needed files even in
such situations. Decided to use curl instead of wget tool
to avoid of such errors feather.

Also found that sourceforge site too often has issues and
responds with 500 HTTP error. Decided to use the link from
github instead of sourceforge to download the icu4c sources,
as suggested on icu4c web site.

Added the command for 'build' directory cleanup. It purges all
artefacts produced for curl build, including the old configuration
in build/curl.

Set Dockerfile WORKDIR from the very start of Tarantool sources
builds to make the Dockerfile code more readable and removed all
duplicating calls to Tarantool sources directory changes. Check:
  https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#workdir

Removed LD_LIBRARY_PATH environment from curl build, due to the
path is empty in the docker image and no other commands from the
Dockerfile filled it, seemed that it was not needed.

Added '-j' option to make for tools buildings, it decreased build time in 4 times.

The change enables memory leaks detection to existing ASAN testing
routine and introduces suppression files with the corresponding
exception list:
 * address sanitizer for compile-time: asan/asan.supp
 * memory leak sanitizer for run-time: asan/lsan.supp

Furthermore, added engine and replication suites for ASAN testing
routine.

Additionally to the tests blacklisted within #4359,
'box/on_shutdown.test.lua' is also disabled since it fails the
introduced leak check. All blacklisted tests have to be enabled
within #4360.

Close #2058

All local connection timeout settings not related to the testing
scenario are removed within this change. Instead of removed values
the default one from src/box/lua/load_cfg.lua will be used.
The approach with a single default value helps to avoid flaky test
results regarding different timeout values and makes the future
maintainence easier.

The change is required for LSAN and ASAN testing machinery since it
introduces a little overhead and cause failures for tests with
excessively strict time limits.

Needed for #2058

Switched OSX 10.14 test jobs to MAC mini hosts.
Renamed Gitlab make rule target 'test_%_no_deps' to 'test_%', which is
used to pass all appropriate target calls to Travis make rule target.

Added popen build-in module to exclusions of pretest_clean script. It'll
allow to test the module from a test of 'core = tarantool' type.

Part of #4031

To achieve this let's refactor luaT_error_create() to return error
object instead of setting it via box_error_set().
luaT_error_create() is used both to handle box.error() and
box.error.new() invocations, and box.error() is still expected to set
error to diagnostic area. So, luaT_error_call() which implements
box.error() processing at the end calls diag_set_error().
It is worth mentioning that net.box module relied on the fact that
box.error.new() set error to diagnostic area: otherwise request errors
don't get to diagnostic area on client side.

Needed for #1148
Closes #4778

@TarantoolBot document
Title: Don't promote error created via box.error.new to diagnostic area

Now box.error.new() only creates error object, but doesn't set it to
Tarantool's diagnostic area:
```
box.error.clear()
e = box.error.new({code = 111, reason = "cause"})
assert(box.error.last() == nil)
---
- true
...
```
To set error in diagnostic area explicitly box.error.set() has been
introduced. It accepts error object which is set as last system error
(i.e. becomes available via box.error.last()).
Finally, box.error.new() does not longer accept error object as an
argument (this was undocumented feature).
Note that patch does not affect box.error(), which still pushes error to
diagnostic area. This fact is reflected in docs:
'''
Emulate a request error, with text based on one of the pre-defined
Tarantool errors...
'''

File descriptor was set to -1 regardless of whether
the object was closed properly. As a result, in case
of an error the descriptor would leak.

GC finalizer of a descriptor is left intact not to
overcomplicate it.

swim object created a new fiber in its GC function, because C
function swim_delete() yields, and can't be called from an ffi.gc
hook.

It is not needed since the fiber module has a single worker
exactly for such cases. The patch uses it.

Follow up #4727

Fio.open() returned a file descriptor, which was not closed
automatically after all its links were nullified. In other words,
GC didn't close the descriptor.

This was not really useful, because after fio.open() an exception
may appear, and user needed to workaround this to manually call
fio_object:close(). Also this was not consistent with io.open().

Now fio.open() object closes the descriptor automatically when
GCed.

Closes #4727

@TarantoolBot document
Title: fio descriptor is closed automatically by GC

fio.open() returns a descriptor which can be closed manually by
calling :close() method, or it will be closed automatically, when
it has no references, and GC deletes it.

:close() method existed always, auto GC was added just now.

Keep in mind, that the number of file descriptors is limited, and
they can end earlier than GC will be triggered to collect not
used descriptors. It is always better to close them manually as
soon as possible.

fiber._internal.schedule_task() is an API for a singleton fiber
worker object. It serves for not urgent delayed execution of
functions. Main purpose - schedule execution of a function, which
is going to yield, from a context, where a yield is not allowed.
Such as an FFI object's GC callback.

It will be used by SWIM and by fio, whose destruction yields, but
they need to use GC finalizer, where a yield is not allowed.

Part of #4727

Fixed OSX host setup for Tarantool build:
- set brew installation from Homebrew repository instructions;
- set in use Python 2 latest commit from tapped local formula,
  since Python 2 is EOL, also removed extra pip installation with
  get-pip script, because tapped formula installs pip itself.
  python@2 was deleted from homebrew/core in commit 028f11f9e:
    python@2: delete (https://github.com/Homebrew/homebrew-core/issues/49796)
    EOL 1 January 2020.
  Tapped formula created from the latest formula before its removal:
    git -C "$(brew --repo homebrew/core)" show 028f11f9e^:Formula/python@2.rb
- added upgrade packages call to avoid of fails on already
  installed packages, but with previous version;
- fixed the gitlab-ci configuration for sudo on testing hosts and removed pip
  option '--user' to avoid of use the users paths with special setup for it.

Fixed OSX host setup for Tarantool test:
- set maximum processes limit value to 2500 for testing process;
- new Mac machines are going to be added into CI and usernames on them
  are long according to internal policies. It makes a home directory to
  be long too and so a path to a unix socket created during testing can
  be longer then UNIX_PATH_MAX=108 constant which is described as issue
    https://github.com/tarantool/tarantool/issues/4634
  To avoid of it the short working directory for testing set by option:
    --vardir /tmp/tnt

box.error.set(err) sets err to instance's diagnostics area. Argument err
is supposed to be instance of error object. This method is required
since we are going to avoid adding created via box.error.new() errors to
Tarantool's diagnostic area.

Needed for #1148
Part of #4778

We are going to introduce more tests related to error module, so let's
move all error-related tests from box/misc.test.lua to a separate test
file (box/error.test.lua).

Needed for #1148

Let's rename diag_add_error() to diag_set_error() because it actually
replaces an error object in diagnostic area with a new one and this name
is not representative. Moreover, we are going to introduce a new
diag_add_error() which will place error at the top of stack diagnostic
area.

Needed for #1148