Vinyl assigns a unique id to each index so that it can be identified in
vylog, see vy_index->id, but outside Vinyl index id means something
completely different - it's the ordinal number of the index in a space.
This creates a lot of confusion. To resolve this, let's rename vy_index
to vy_lsm and refer to Vinyl indexes as LSM trees in comments so that
Vinyl's index_id turns into lsm_id.

Also, rename vy_log_record's index_def_id and space_def_id back to
conventional index_id and space_id as they doesn't conflict with Vinyl
index id anymore (which is now lsm_id).

Thanks to @Gerold103 for suggesting the new name.

Currently, DDL isn't properly rolled back if WAL write fails, leaving
the space in an inconsistent state - see alter_space_do() that calls
space_commit_alter() right in on_replace trigger. It isn't clear how to
fix this properly right now. Let's disable the corresponding test case
until we figure out how to resolve this problem, so as not to stall
further development.

See #3289

Fixes 25fa5e21 vinyl: do not use index lsn to identify indexes in vylog

Truncation of a space is equivalent to recreation of all space indexes
with the same definition. The reason why we use a special system space
to trigger space truncation (_truncate) is that we don't have
transactional DDL while space truncation has to be done atomically.
However, apart from the new system space, implementation of truncation
entailed a new vylog record (VY_LOG_TRUNCATE_INDEX) and quite a few
lines of code to handle it. So why couldn't we just invoke ALTER that
would recreate all indexes?

To answer this question, one needs to recall that back then vinyl used
LSN to identify indexes in vylog. As a result, we couldn't recreate more
than one index in one operation - if we did that, they would all have
the same LSN and hence wouldn't be distinguishable in vylog. So we had
to introduce a special vylog operation (VY_LOG_TRUNCATE_INDEX) that
bump the truncation counter of an index instead of just dropping and
recreating it. We also had to introduce a pair of new virtual space
methods, prepare_truncate and commit_truncate so that we could write
this new command to vylog in vinyl. Putting it all together, it becomes
obvious why we couldn't reuse ALTER code for space truncation.

Fortunately, things have changed since then. Now, vylog identifies
indexes by space_id/index_id. That means that now we can simplify
space truncation implementation a great deal by

 - reusing alter_space_do() for space truncation,
 - dropping space_vtab::prepare_truncate and commit_truncate,
 - removing truncate_count from space, index, and vylog.

vy_log_record::index_lsn serves two purposes. First, it is used as a
unique object identifier in vylog (it is similar to range_id or slice_id
in this regard). Second, it is the LSN of the WAL row that committed the
index, and we use it to lookup the appropriate index incarnation during
WAL recovery. Mixing these two functions is a bad design choice because
as a result we can't create two vinyl indexes in one WAL row, which may
happen on ALTER of a primary key. Besides, we can't create an index
object before WAL write, which is also needed for ALTER, because at that
time there's no LSN assigned to the index yet.

That said, we need to split this variable in two: index_id and
commit_lsn. To be backward compatible, we rename index_lsn to
index_id everywhere in vylog and add a new record field commit_lsn;
if commit_lsn is missing for a create_index record, then this must
be a record left from an old vylog and so we initialize it with
index_id (former index_lsn) - see vy_log_record_decode().

I'm planning to assign a unique identifier to each vinyl index so that
it could be used instead of lsn for identifying indexes in vylog. In
order not to confuse it with the index ordinal number, let's rename
vy_log_record::index_id to index_def_id and, for consistency, space_id
to space_def_id.

Throughout Vinyl we use the term 'id' for calling members representing
unique object identifiers: vy_slice::id, vy_run::id, vy_range::id.
There's one exception though: vy_index::id is the ordinal number of the
index in a space. This is confusing. Besides, I'm planning to assign a
unique id to each vinyl index so that I could look them up in vylog.
I'd like to call the new member 'id' for consistency. So let's rename
vy_index::id to index_id.

The vy_recovery structure was initially designed as opaque to the
outside world - to iterate over objects stored in it, one is supposed to
use vy_recovery_iterate(), which invokes the given callback for each
recovered object encoded as vy_log_record that was used to create it.

Such a design gets extremely difficult to use when we need to preserve
some context between callback invocations - e.g. see how ugly backup and
garbage collection procedures look. And it is going to become even more
obfuscated once we introduce the notion of incomplete indexes (indexes
that are currently being built by ALTER).

So let's refactor vylog recovery procedure: let's make the vy_recovery
structure transparent and allow to iterate over internal representations
of recovered objects directly, without callbacks.

* feedback daemon sends information about instance to the
  specified host.
* Add new options to box.cfg:
    - feedback_enabled - switch on/off daemon, default=true.
    - feedback_host - host to which feedbacks are sent,
      default="https://feedback.tarantool.io".
    - feedback_interval - time interval in seconds of feedbacks
    sending, default=3600.
* Add possibility to generate feedback file in json format with
function box.feedback.save

Closes #2762

Introduce two functions
* fiber.new() - create fiber, schedules it into the
ready queue but doesn't call it and doesn't yield.
Signature of the method is the same as for fiber.create
* fiber.join() - waits until the specified fiber finishes
its execution and returns result or error. Applicable only
to joinable fibers.
* fiber.set_joinable() - sets the fiber joinable flag

Closes #1397

Empty strings should be ignored, rather than throw an error.

Passing only empty strings (or nothing) to `pathjoin` will return '.' which means the current directory

Every path part passed to `pathjoin` is now converted to a string

The `gsub('/+', '/')` call already does what the removed code does, so avoid the unnecessary work
Simply check if the result equals '/' before removing a trailing '/'.

The previous check did extra work for no gain.

It is possible to discard non-sent responses using a special
sequence of requests and yields. In details: if DML requests
yield on commit too long, there are fast read requests, and a
network is saturated, then some non-sent DML responses are
discarded.

Closes #3255

A vinyl space can be altered in such a way, that key definitions
of indexes are not changed, but comparators do. It is because
space format reset can make some indexed fields optional.

To be able update key definitions in place, they must not be used
in a worker thread. So lets copy key_defs for a worker, and
update index key definitions in place.

An alternative is key_defs reference counting, but there is open
questions what to do in key_defs in mems, ranges, iterators,
runs, slices. Now lets do a hotfix of a crash, and then
refactoring.

Closes #3229

Use key_def_delete instead. It is more safe to use one
destructor everywhere for a case, if in a future key_def will not
be deleted by simple free().

Under FreeBSD getline prototype is not provided by
default due to compatibility problems.
Get rid of getline (use fgets instead).
Based on @locker proposal.

Closes #3217.

log_rotate writes informational message only in plain format,
which is inappropriate when logger is configured to log in json format.

Fix it with replacing write with say_info which is safe
because it is ev signal callback not signal handler.

Fix bug say_format in json formatting,
Log message was invalid in several cases.

Closes #2987

Current log rotation is not async signal safe.
In order to make it so refactor signal handling
with ev_signal.

Log rotation for each logger performs in separate
coio_task to provide async and thread-safe execution.

Relates #3015

Remove yielding and waiting task complete in coio_task_post
in case if timeout is zero.
This patch is inspired by the need in log_rotate posting coio
task. This post should not yield there because the implementation
of multiple loggers works with linked list structure of loggers
which is not fiber-safe.

When a space format is updated, a new min field count must be
calculated before a new format construction to check that some
of fields became optional.

Part of #3229

This patch adds a new Lua function, box.stat.reset(), which resets
all incremental box statistic counters, namely box.stat, box.stat.net,
box.info.vinyl, and index.info. This is useful for performance analysis
and debugging.

Closes #3198

When scanning a secondary index, we actually track each tuple in the
transaction manager twice - as a part of the interval read from the
secondary index and as a point in the primary index when retrieving
the full tuple. This bloats the read set - instead of storing just one
interval for a range request, we also store each tuple returned by it,
which may count to thousands. There's no point in this extra tracking,
because whenever we change a tuple in the primary index, we also update
it in all secondary indexes. So let's remove it to save us some memory
and cpu cycles.

This is an alternative fix for #2534
It should also mitigate #3197

We never use vy_point_lookup directly, instead we open vy_read_iterator,
which automatically falls back on vy_point_lookup if looking for exact
match (EQ + full key). Due to this we can't add a new point lookup
specific argument (we would have to propagate it through the read
iterator, which is ugly). Let's call vy_point_lookup directly when we
know that vy_read_iterator will fall back on it anyway.

This reverts commit a31c2c10.

The commit reverted by this patch forces all autocommit SELECTs to open
a read view immediately, as a result they can't update tuple cache.
Turned out that one of our customers intensively uses such SELECTs, and
disabling cache for them results in performance degradation.

The reason why that commit was introduced in the first place was to
avoid read set bloating for big SELECTs (e.g. space.count()): currently
we track not only read interval boundaries, but also each tuple fetched
from the primary index if it is a secondary index that is being scanned.
However, it doesn't seem that we really need to do that - tracking an
interval read from a secondary index guarantees that if a tuple returned
by the iterator is modified the transaction will be aborted and so
there's no need to track individual tuples read from the primary index.

That said, let's revert this commit and instead remove point lookup
tracking in case it is a secondary index that is being scanned (done
later in the series).