The test for multikey index prefix compatibility was insufficient
because JSON path is relative for some fieldno. Those root field
identifiers also must coincide.

Follow up #1257

One another problem discovered with UDP broadcast test is that
it can affect other tests, even after termination. Doing
swim:broadcast() on one test a programmer can't be sure, who will
listen it, answer, and break the test scenario.

This commit reduces probability of such a problem by

    * allowance to set a codec before swim:cfg(). It allows to
      protect SWIM nodes of different tests from each other -
      they will not understand messages from other tests. By the
      way, the same problem can appear in real applications too;

    * do not binding again a URI passed by test-run into the
      test and closed here. If a test closes a URI given to it,
      it can't be sure, that next bind() will be successful -
      test-run could already reuse it.

Follow up #3234

First of all, the problem in a nutshell was that ev_timer with
non-zero 'repeat' field in fact is a ev_periodic. It is restarted
*automatically*, even if a user does not write ev_timer_again()
nor ev_timer_start().

This led to a situation, that a round message send is scheduled,
and next round step timer alarm happens before the message is
actually sent. It, in turn, led to an assertion on attempt to
schedule a task twice.

This patch fixes the swim test harness to behave like ev_timer
with 'repeat' > 0, and on first idle round step stops the timer -
it will be restarted once the currently hanging task will be
finally sent.

Follow up #3234

They are caused by

    * too slow network, when SWIM tests are run under high load;

    * UDP packets late arrival or drop.

Follow up #3234

Follow up #3234

Follow up #3234

It's too early to assert msgpack type as an array when a multikey field
is encountered - we haven't checked the field type yet so the type might
as well be a map, in which case we will raise an error just a few lines
below. Remove the assertion and add a test case.

If an indexed field expects array/map, it shouldn't be allowed to insert
null instead, because this might break expectations of field accessors.
For unikey indexes inserting null instead of array/map works fine though
somewhat confusing: for a non-nullable field you get a wrong error
message ("field is missing" instead of "array/map expected, got nil");
for a nullable field, this silently works, just looks weird as there's a
clear type mismatch here. However, for a multikey field you get a crash
as tuple_multikey_count() doesn't expect to see null where an array
should be according to the format:

  tuple_raw_multikey_count: Assertion `mp_typeof(*array_raw) == MP_ARRAY' failed.

This issue exists, because we assume all fields are nullable by default
for some reason. Fix that and add some tests.

Note, you can still omit nullable fields, e.g. if field "[2].a[1]" is
nullable you may insert tuple [1, {a = {}}] or [1, {b = 1}] or even [1],
you just can't pass box.NULL instead of an array/map.

Tarantool used to assume that offset_slot has an extension
iff field_map_get_offset is called with multikey_idx >= 0.
In fact, when some part of the index contains a multikey index
placeholder, tuple_compare_* routines pass a tuple_hint in
meaning of multikey index for each tuple_field_raw_by_part call,
even for regular key_part that doesn't have array index
placeholder (and, correspondingly, field_map extension).
Thus this assumption is invalid.

This patch uses the fact that field_map slots that have extension
store negative offset to distinguish multikey and normal usage
of the field_map_get_offset routine.

Closes #4234

Empty string as a no-payload-flag was not a good idea, because
then a user can't write something like:

    if not member:payload() then
        ...

Follow up #3234

Encryption with an arbitrary algorithm and any mode with a
configurable private key.

Closes #3234

SWIM is going to be used in and between datacenters, which means,
that its packets will go through public networks. Therefore raw
SWIM packets are vulnerable to attacks.

An attacker can do any and all of the following things:

  1) Extract secret information from member payloads, like
     credentials to Tarantool binary ports;

  2) Change UUIDs and addresses in the packets and break a
     topology;

  3) Catch the packets and pretend being a Tarantool instance,
     which could lead to undefined behaviour depending on an
     application logic.

SWIM packets need a protection layer. This commit introduces it.
SWIM transport level allows to choose an encryption algorithm
with a private key to encrypt each packet with that key.

Besides, each packet is encrypted using a random public key
prepended to the packet.

SWIM now provides a public API to choose an encryption algorithm
and a private key.

Part of #3234

At this moment swim_scheduler_on_output() is a relatively simple
function. It takes a task, builds its meta and flushes a result
into the network. But soon SWIM will be able to encrypt messages.

It means, that in addition to regular preprocessing like building
meta headers a new phase will appear - encryption. What is more -
conditional encryption, because a user may want to do not encrypt
messages.

All the same is about swim_scheduler_on_input() - if a SWIM
instance uses encryption, it should decrypt incoming messages
before forwarding them into the SWIM core logic.

The chosen strategy - lets reuse on_output/on_input virtuality
and create two version of on_input/on_output functions:

    swim_on_plain_input()  | swim_on_encrypted_input()
    swim_on_plain_output() | swim_on_encrypted_output()

One of these pairs is chosen depending on if the instance uses
encryption.

To make these 4 functions as simple and short as possible this
commit creates two sets of functions, doing all the logic except
encryption:

    swim_begin_send()
    swim_do_send()
    swim_complete_send()

    swim_begin_recv()
    swim_do_recv()
    swim_complete_recv()

These functions will be used by on_input/on_output functions with
different arguments.

Part of #3234

Each time a member was returned from a SWIM instance object, it
was wrapped by a table with a special metatable, cached payload.

But next the same lookup returned a new table. It

  - created garbage as a new member wrapper;
  - lost cached decoded payload.

This commit caches in a private table all wrapped members and
returns an existing wrapper on a next lookup. A microbenchmark
showed, that cached result retrieval is 10 times faster, than
each time create a new table.

Cache table keeps week references - it means, that when a member
object looses all its references in a user's application, it is
automatically dropped from the table.

Part of #3234

Users of Lua SWIM module likely will use Lua objects as a
payload. Lua objects are serialized into MessagePack
automatically, and deserialized back on other instances. But
deserialization of 1.2Kb payload on each member:payload()
invocation is quite heavy operation. This commit caches decoded
payloads to return them again until change.

A microbenchmark showed, that cached payload is returned ~100
times faster, than it is decoded each time. Even though a tested
payload was quite small and simple:

    s:set_payload({a = 100, b = 200})

Even this payload is returned 100 times faster, and does not
affect GC.

Part of #3234

Sometimes, especially in tests, it is useful to make something
like this:

    s:add_member({uuid = member:uuid(), uri = member:uri()})

But member:uuid() is cdata struct tt_uuid. This commit allows
that.

Part of #3234

Expose iterators API to be able to iterate over a member table in
a 'for' loop like it would just be a Lua table.

Part of #3234

Expose API to search members by UUID, to read their attributes,
to set payload.

Part of #3234

Expose methods to add, remove, probe members by uri, uuid. Expose
broadcast method to probe multiple members by port.

Part of #3234

SWIM as a library can be useful not only for server internals,
but for users as well. This commit exposes Lua bindings to SWIM
C API. Here only basic bindings are introduced to create, delete,
quit, check a SWIM instance. With sanity tests.

Part of #3234

Similar methods validate their arguments: add_member,
remove_member. Validate here as well for consistency.

Part of #3234

Firstly, I thought that there is an error - swim_begin_step()
does not reschedules round timer, when new_round() fails. But
then new_round() appeared never failing. This commit makes it
void to eliminate confusion.

Probably it is a legacy since the shuffled members array was
allocated and freed in new_round().

Part of #3234

Appeared, that libev does not allow to change ev_timer values in
flight. A timer, reset via ev_timer_set(), should be restarted,
because the function changes 'ev_timer.at', which in turn is used
internally by timer routines.

Part of #3234

Lua, which suffers from lack of ability to pass values by
pointers into FFI functions, nor has an address operator '&' to
take an address of integer or char or anything. Because of that
a user need to either use ffi.new(type[1]) or use static buffer,
but for such small allocations they are both too expensive and
aggravate GC problem.

Now buffer module provides preallocated basic types to use in FFI
functions. The commit is motivated by one another place where
ffi.new('int[1]') appeared, in SWIM module, to obtain payload
size as an out parameter of a C function.

Static allocator gives memory blocks from cyclic BSS memory
block of 3 pages 4096 bytes each. It is much faster than
malloc, when a temporary buffer is needed. Moreover, it does not
complicate GC job. Despite being faster than malloc, it is still
slower, than ffi.new() of size <= 128 known in advance (according
to microbenchmarks).

ffi.new(size<=128) works either much faster or with the same
speed as static_alloc, because internally FFI allocator library
caches small blocks and can return them without malloc().

A simple micro benchmark showed, that ffi.new() vs
buffer.static_alloc() is ~100 times slower on allocations of
> 128 size,  on <= 128 when size is not inlined.

To better understand what is meant as 'inlined': this

    ffi.new('char[?]', < <=128 >)

works ~100 times faster than this:

    local size = <= 128
    ffi.new('char[?]', size)

ffi.new() with inlined size <= 128 works faster than light, and
even static allocator can't beat it.

Certain kinds of DML requests don't update secondary indexes, e.g.
UPDATE that doesn't touch secondary index parts or DELETE for which
generation of secondary index statements is deferred. For such a request
vy_is_committed(env, space) may return false on recovery even if it has
actually been dumped: since such a statement is not dumped for secondary
indexes, secondary index's vy_lsm::dump_lsn may be less than such
statement's signature, which makes vy_is_committed() assume that the
statement hasn't been dumped. Further in the code we have checks that
ensure that if we execute a request on recovery, it must not be dumped
for the primary index (as the primary index is always dumped after
secondary indexes for the sake of recovery), which fires in this case.

To fix that, let's refactor the code basing on the following two facts:
 - Primary index is always updated by a DML request.
 - Primary index may only be dumped after secondary indexes.

Closes #4222

Yet another fix for building of small library as part of tarantool.
Before this commit slab_arena test fails:

 | [019] Test failed! Result content mismatch:
 | [019] --- small/slab_arena.result	Mon May 20 21:37:46 2019
 | [019] +++ small/slab_arena.reject	Mon May 20 21:47:01 2019
 | [019] @@ -23,3 +23,4 @@
 | [019]  arena->maxalloc = 2000896
 | [019]  arena->used = 0
 | [019]  arena->slab_size = 65536
 | [019] +ERROR: Expected dd flag on VMA address 0x7f3ec2080000

See the corresponding commit in the small submdoule for more info.

The autoincrement code was written when there were no nested field.
Now, it isn't enough to just skip to the autoincrement field - we also
need to descend deeper if key_part->path is set.

Note, the code expects the nested field to be present and set to NULL.
That is, if field path is [1].a.b, the tuple must have all intermediate
fields set: {{a = {b = box.NULL}}} (usage of box.NULL is mandatory to
create a tuple like that in Lua).

Closes #4210

Closes #4009

@TarantoolBot document
Title: Sequence can now be set for an index part other than the first

Initially one could attach a sequence (aka autoincrement) only to the
first index part. Now it's possible to attach a sequence to any primary
index part. The part still must be integer though.

Syntax:

```
box.schema.space.create('test')
box.space.test:create_index('primary', {
    parts = {{1, 'string'}, {2, 'unsigned'}, {3, 'unsigned'}},
    sequence = true, sequence_part = 2
})
box.space.test:insert{'a', box.null, 1} -- inserts {'a', 1, 1}
```

Note, `sequence_part` option is 1-base.

If `sequence_part` is omitted, 1 is used, which assures backward
compatibility with the original behavior.

One can also attach a sequence to another index part using
`index.alter` (the code below continues the example above):

```
box.space.test.index.primary:alter{sequence_part = 3}
box.space.test:insert{'a', 1, box.null, 'x'} -- inserts {'a', 1, 2, 'x'}
```

A check was missing in index.alter. This resulted in an attempt to drop
the sequence attached to the altered index even if the sequence was not
modified.

Closes #4214

When schema.lua was introduced, there was no such thing as space format
and we had to access tuple fields by no. Now we can use human readable
names. Let's do it - this should improve code readability.

A note about box/alter.test.lua: for some reason it clears format of
_space and _index system spaces, which apparently breaks our assumption
about field names. Let's zap those pointless test cases.

Updated small submodule with the corresponding fix.

Background. Coio provides a way to schedule arbitrary tasks
execution in worker threads. A task consists of a function to
execute, and a custom destructor.

To push a task the function coio_task_post(task, timeout) was
used. When the function returns 0, a caller can obtain a result
and should free the task manually. But the trick is that if
timeout was 0, the task was posted in a detached state. A
detached task frees its memory automatically despite
coio_task_post() result, and does not even yield. Such a task
object can't be accessed and so much the more freed manually.

coio_getaddrinfo() used coio_task_post() and freed the task when
the latter function returned 0. It led to double free when
timeout was set 0. The bug was introduced here
800cec73 in an attempt to do not
yield in say_logrotate, because it is not fiber-safe.

Now there are two functions: coio_task_execute(task, timeout),
which never detaches a task completed successfully, and
coio_task_post(task), which posts a task in a detached state.

Closes #4209

According to the standard by Open Group, getaddrinfo() hints
argument is optional - it can be NULL. When it is NULL, hints
is assumed to have 0 in ai_flags, ai_socktype, and ai_protocol;
AF_UNSPEC in ai_family.

See The Open Group Base Specifications.

The result file of the test app-tap/init_script.test.lua was not updated
in 549140b3 ('box/memtx: Allow to skip
tuple memory from coredump').

Follow up #3509.

Made fixes:

- Added CMAKE_EXTRA_PARAMS environment to docker's container
  runs to enable -DENABLE_LTO=ON/OFF cmake option.

- Added CC/CXX environment to docker's container runs to set
  clang for cmake. Also the additional environment variables
  {CC,CXX}_FOR_BUILD were postponed, because we didn't
  run cross-compilation at the moment, for more info check:

    https://docs.travis-ci.com/user/languages/cpp/#choosing-compilers-to-test-against

- Changed LTO docker's image to 'debian-buster' due to LTO needed
  higher versions of packages, check for more information commit:

    f9e28ce4 ('Add LTO support')

- Fixed sources to avoid of failures on builds by GCC with LTO:

1)  src/box/memtx_rtree.c: In function ‘mp_decode_rect’:
    src/box/memtx_rtree.c:86:24: error: ‘c’ may be used uninitialized
      in this function [-Werror=maybe-uninitialized]
        rect->coords[i * 2] = c;
                            ^
    src/box/memtx_rtree.c:74:10: note: ‘c’ was declared here
      coord_t c;
              ^

2)  src/box/sql/func.c: In function ‘quoteFunc’:
    src/box/sql/func.c:1103:3: error: ‘b’ may be used uninitialized
      in this function [-Werror=maybe-uninitialized]
       sql_result_text(context, sql_value_boolean(argv[0]) ?
       ^
    src/box/sql/vdbeapi.c:217:7: note: ‘b’ was declared here
      bool b;
           ^

3)  src/box/tuple_update.c: In function ‘update_read_ops’:
    src/box/tuple_update.c:1022:4: error: ‘field_no’ may be used
      uninitialized in this function [-Werror=maybe-uninitialized]
        diag_set(ClientError, ER_NO_SUCH_FIELD_NO, field_no);
        ^
    src/box/tuple_update.c:1014:11: note: ‘field_no’ was declared here
       int32_t field_no;
               ^

4)  src/httpc.c: In function ‘httpc_set_verbose’:
    src/httpc.c:267:2: error: call to ‘_curl_easy_setopt_err_long’
      declared with attribute warning: curl_easy_setopt expects a long
      argument for this option [-Werror]
      curl_easy_setopt(req->curl_request.easy, CURLOPT_VERBOSE, curl_verbose);
      ^

5)  src/lua/httpc.c: In function ‘luaT_httpc_request’:
    src/lua/httpc.c:128:64: error: ‘MEM[(int *)&parser + 20B]’ may be used
      uninitialized in this function [-Werror=maybe-uninitialized]
      lua_pushinteger(L, (parser.http_minor > 0) ? parser.http_minor: 0);
                                                                    ^
    src/lua/httpc.c:67:21: note: ‘MEM[(int *)&parser + 20B]’ was declared here
      struct http_parser parser;
                         ^
    src/lua/httpc.c:124:64: error: ‘MEM[(int *)&parser + 16B]’ may be used
      uninitialized in this function [-Werror=maybe-uninitialized]
      lua_pushinteger(L, (parser.http_major > 0) ? parser.http_major: 0);
                                                                    ^
    src/lua/httpc.c:67:21: note: ‘MEM[(int *)&parser + 16B]’ was declared here
      struct http_parser parser;
                         ^

Close #4215

In case if there are huge amount of tuples the whole
memory goes to coredump file even if we don't need it
for problem investigation. In result coredump may
blow up to gigabytes in size.

Lets allow to exclude this memory from dumping via
box.cfg::strip_core boolean parameter.

Note that the tuple's arena is used not only for tuples
themselves but for memtx->index_extent_pool and
memtx->iterator_pool as well, so they are affected
too.

Fixes #3509

@TarantoolBot document
Title: Document box.cfg.strip_core

When Tarantool runs under a heavy load the memory allocated
for tuples may be very huge in size and to eliminate this
memory from being present in `coredump` file the `box.cfg.strip_core`
parameter should be set to `true`.

The default value is `false`.

Negative size led to an assertion. The commit adds a check if
size is negative.

Closes #4224