When the read iterator stops reading a chain of statements from the
cache it advances all other sources by calling next_key() until the
last_stmt is reached. This effectively cancels the benefit of using
the cache, because all statements skipped due to the cache are fetched
from in-memory trees or, even worse, on-disk runs.

To fix this, let's introduce and use skip() method which makes the
source iterator jump to the first statement following a particular key.
Its implementation is similar to and reuses the code from start and
restore procedures.

With this new method, we don't need to mangle iterator_type/key when
reopening source iterators during restoration so that they start
iteration from last_stmt: instead we can advance them with skip() on
the first iteration. Let's do this too, because the iterator can
benefit from knowing the real iterator type (e.g. cache can stop
ITER_EQ iteration even if there's no chain in the cache, by looking at
vy_cache_entry::left_boundary_level,right_boundary_level).

There is no such thing as vy_stmt_iterator anymore so split the header
in vy_stmt_stream.h and vy_read_view.h.

vy_read_iterator was the only user of this interface. As now it handles
sources of different types differently, the interface is not needed any
more.

The generic approach trying to build the merge procedure around the
vy_stmt_iterator interface didn't pan out, because sources are way too
different: in contrast to other sources, the cache stores intervals;
run iterators may yield; txw does not preserve statement history.

Let's rewrite vy_read_iterator_next_{key,lsn} in such a way that they
do not use this generic interface. This results in a quite bit of code
being duplicated, because loops over sources are unrolled, but this is
intentional - hopefully it makes the code easier to follow. The patch
isn't supposed to change the merge algorithm or remove any optimization
implemented in it.

This reverts commit be8ee29a.

Taking a reference to the search key in source iterators is pointless -
it can't go away while we are using them.

The only part of this patch that makes sense is removing the const
specifier from vy_point_iterator->key.

Closes #2558

If read of a single statement from vinyl takes more than the value of
box.cfg.too_long_threshold, the request will be logged:

  512/1: select([1], EQ) => REPLACE([100001, 1], lsn=200006) took too long: 0.626 sec

This is useful for debugging.

While we are at it, let's also remove 'timeout' from the vinyl engine
constructor arguments and set it with box_set_vinyl_timeout() on box
initialization instead, similarly to vinyl_max_tuple_size.

Closes #2871

Use generic tuple_snprint() and tuple_str() instead.

The vinyl_iterator struct was introduced as a C++ wrapper around
vy_cursor. Since there's no C++ code left in the engine, and both
structures are defined in the same file, we can merge them now.

The engine infrastructure was initially implemented in C++ so we
needed the wrappers to provide C++ API to Vinyl. Now everything is
in C so we don't need them any more. Let's fold them in vinyl.c.

Note, this patch does not touch vinyl_engine, vinyl_index, and
vinyl_iterator structures, they are still there, it just gets rid
of the intermediate layer of wrapper functions, which is not needed
any more.

Accessing configuration from inside an engine implementation
violates encapsulation.

Remove iobuf_is_idle(). It uses obuf.wpos, which itself
needs to be removed from obuf.

Spare box from iproto I/O to simplify transition of control
of output buffer from iproto to tx thread.
In scope of gh-946.

Bloom filter depends on hash function, which depends on ICU
version, which may vary.

+ Don't use id=0 for collations

Follow up #2649

Fix tuple_hash_field() to handle the following cases properly:

 - Nullable string field (crash in vinyl on dump).
 - Scalar field with collation enabled (crash in memtx hash index).

Add corresponding test cases.

First, unique but nullable indexes are not rebuilt when the primary key
is altered although they should be, because they can contain multiple
NULLs. Second, when rebuilding such indexes we use a wrong key def
(index_def->key_def instead of cmp_def), which results in lost stable
order after recovery. Fix both these issues and add a test case.

Needed to check if the key definition loaded from vylog to send initial
data to a replica has the collation properly recovered.

It isn't stored currently, but this doesn't break anything, because the
primary key, which is the only key whose definition is used after having
been loaded from vylog, can't be nullable. Let's store it there just in
case. Update the vinyl/layout test to check that.

Collations were disabled in vinyl by commmit 2097908f ("Fix
collation test on some platforms and disable collation in vinyl"),
because a key_def referencing a collation could not be loaded from
vylog on recovery (collation objects are created after vylog is
recovered). Now, it isn't a problem anymore, because the decoding
procedure, key_def_decode_parts(), deals with struct key_part_def,
which references a collation by id and hence doesn't need a collation
object to be created. So we can enable collations in vinyl.

This patch partially reverts the aforementioned commit (it can't
do full revert, because that commit also fixed some tests along
the way).

Closes #2822

We can't use key_def_decode_parts() when recovering vylog if key_def has
a collation, because vylog is recovered before the snapshot, i.e. when
collation objects haven't been created yet, while key_def_decode_parts()
tries to look up the collation by id. As a result, we can't enable
collations for vinyl indexes.

To fix this, let's rework the decoding procedure so that it works with
struct key_part_def instead of key_part.  The only difference between
the two structures is that the former references the collation by id
while the latter by pointer.

Needed for #2822

Writer fiber should be stopped before re-connect to avoid
sending unwanted IPROTO_OK replication acknowledges.

Fixes #2726

SUBSCRIBE command is not multiplexed in the binary protocol.
When relay exits with an error during subscribe, remote replica
still continue to send IPROTO_OK replication acknowledges to
the master. These packets are unwanted by IPROTO decoder.

Close socket on errors during SUBSCRIBE.

Fixes #2726

The corresponding comparator is missing, which leads to a crash.
Fix it and add a test case checking that nullable indexes work fine
with all available types.

Symbol resolving can be expensive. Introduce an option for fiber.info():

    fiber.info({ backtrace = true })
    fiber.info({ bt = true })

Fixes #2878

- Change signature of function access_check_func.
  Now it returns status instead of function.

Close #2816

> src/tarantool/src/box/vinyl.c:2111:33: error: initializer element is not a compile-time constant
>         static const double weight = 1 - exp(-VY_QUOTA_UPDATE_INTERVAL /
>                                      ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Remove the "static" qualifier. It is not really needed as any sane
compiler will pre-calculate the value of 'weight' at compile time
(checked on gcc 6.3.0 with -O0).

 - Remove vy_stat::rmean statistics, which were left from Sophia, as now
   we have per index statistics which are much more verbose than those.

 - Move vy_stat::dump_bw to vy_env and remove struct vy_stat as there's
   nothing left in it.

 - Move quota statistics from box.info.vinyl().performance.memory to
   box.info.vinyl().quota. Remove 'ratio', which equals used / limit, as
   this kind of calculations should be done by a script aggregating
   statistics. Report 'use_rate' and 'dump_bandwidth' there.

 - Report 'limit' in cache statistics to make them consistent with
   'quota' statistics, where 'limit' is reported. Rename 'cache.count'
   to 'cache.tuples'. Remove vy_cache_env::cache_count, use mempool
   stats instead.

 - Move 'tx_allocated', 'txv_allocated', 'read_interval', 'read_view'
   from box.info.vinyl().performance to box.info.vinyl().tx and name
   them 'transactions', 'statements', 'gap_locks', and 'read_views',
   respectively. Remove vy_tx_stat::active and 'tx.active' as the same
   value is shown by 'tx.transactions', extracted from the mempool.

 - Zap box.info.vinyl().performance - there's nothing left there.

Now global statistics look like:

  tarantool> box.info.vinyl()
  ---
  - cache:
      limit: 134217728
      tuples: 32344
      used: 34898794
    tx:
      conflict: 1
      commit: 324
      rollback: 13
      statements: 10
      transactions: 3
      gap_locks: 4
      read_views: 1
    quota:
      dump_bandwidth: 10000000
      watermark: 119488351
      use_rate: 1232703
      limit: 134217728
      used: 34014634
  ...

Closes #2861

We have a timer for updating watermark every second. Let's reuse it for
quota use rate calculation. This will allow us to get rid of legacy
vinyl statistics.

Also, let's use EWMA for calculating the average. It is a more efficient
and common method, which allows to easily tune the period over which the
value is averaged.

Follow up #1557

Collation was simply ignored for non-string parts, that could
confuse potential user.

Generate a readable error in this case.

Fix #2862 part 2

Now collation is silently ignored for type='scalar' parts.

Use collation for string scalar fields.

Fix #2862 part 1

Show collation name (if present) in space.index.name.parts[no].

Fix #2862 part 4

test:create_index('unicode_s1', {parts = {{1, 'STR', collation =
'UNICODE'}}}) will work now.

Fix #2862 part 3