Needed for #910

Get rid of unprotected usage of Lua API.

Needed for #910

Needed for #910

Optimize error_log(diag_last_error(diag_get())) pattern.

Create memtx format format in memtx engine, and vinyl format in vinyl
engine.
This is a pre-requisite patch for gh-2476 (predictable memtx tree
iteration).

The space may have a state, we should commit alter of the new space object,
not old one.

We can always use disk_format, in case of a single key space
it is identical to mem_format.

Add xlog_cursor crc, meta parsnig and reading error injections.
Add vy run data reading injection, forward injection from vy run data
reading to xlog.
See #1610

Fixes gh-2660.

Perserve triggers during truncate and rollback.

A test case to follow in a separate commit.

Fix inaccurate patching by ed412cd9 commit in LuaJIT.

Add an option to enable LuaJIT's GC64 mode and enable this option
by default on OS X.

Closes #2643

LuaJIT has some configuration flags which can be switched via
-DLUAJIT_USE_OPTION defines, e.g. USE_ASAN or USE_VALGRIND.
Before this patch these definitions were added by CMake to XCFLAGS
variable and passed to LuaJIT's Makefile. However, some of these
flags also affect the content of lj_xxx.h internal header files,
which were included by Tarantool (see utils.c) and compiled WITHOUT
proper flags. This situation might lead to inconsistent ABI between
compiled libluajit.a and Tarantool.

This patch adds all LUAJIT_USE_XXX defines to CMake's COMPILE_FLAGS
in order to enable these flags globally, then parses COMPILE_FLAGS
and adds them to LuaJIT's XCFLAGS.

Needed for #2643

Make tuple_format referencing API similar to other refable objects.

Rename vy_index membes, add comments.

Move a comment. Now that the index def list is ordered, we don't have to
treat the primary key in a special way.

Preserve the primary key position (first) in the index def list
when creating a list of index definitions during alter.

This simplifies the code in space_new() and down the call stack
and makes it easier to remove space->format.

We plan to remove space->format altogether and store it in the engine.
This is a preparatory patch.

Get rid of struct space dependency in vy_index.

Needed for unit tests.

In context of #2090

Needed for unit tests.

In context of #2090

If the iterator type is ITER_LE and a run_iterator can not find
a start page (all page min_keys are >= select key), it would
iterate over all pages starting from the last.

Fix it by setting search_ended flag at the page search phase, before
calling next_key.

Closes #2614

And fix a bug in vy_cache_iterator_restore, which could ignore
restore to a previous statement.

Needed for #2614

Implement box.session.type() which returns one of the:
applier, background, repl, console, binary, depending on the
source of box queries.

Add tests.

cfg_XXX() methods must not be used outside vy_conf.

Follow up #2253

Re-create .index files in case of vy_run_recover error().
Old file will be deleted and a new index file will be created.

Closes #2253
Closes #2395

recover_remaining_wals() doesn't proceed to the next xlog until the
current one is finalized (EOF marker is written). As a result, if an
xlog wasn't finalized for some reason (IO error occurred or tarantool
was killed in cold blood with SIGKILL), replication will stall on the
unfinished xlog forever.

To preclude that, let's differentiate WAL write and rotation in WAL
watcher and force recover_remaining_wals() rescan the WAL directory on
any WAL rotation so that it will continue to the next WAL even if the
current one wasn't properly finalized. In case of hot standby, we can't
reliably detect WAL rotation (e.g. on Mac OS inotify may not work) so
also rescan the WAL directory every wal_dir_rescan_delay seconds.

Closes #2294

 - Handle cord_costart() errors properly.
 - Don't use scoped_guard as we don't actually need it there.
 - Drop relay_destroy() and rename relay_create() to relay_init().

Instead of notifying relay about new WALs with ev_async, establish a
cbus connection between WAL and each relay and send cbus messages after
wal_write. This is more flexible, because unlike ev_async we can pass
extra information along with cbus messages, e.g. LSN or event mask.

Closes #2275

This patch introduces cbus_pair() and cbus_unpair() helpers to
pair/unpair two existing cbus endpoints in one function call.

cbus_pair() creates pipes from the caller's cord to the given
destination and back. It blocks the caller until both pipes are
created. The caller may specify a callback to be called at the
destination right after the pipe to the source has been created,
which may be useful for preparing the destination to receiving
messages from the source.

cbus_unpair() initiates destruction of pipes created with cbus_pair()
and blocks the caller until destruction is complete. Since there may
be messages queued at the destination endpoint when this function is
called, before proceeding to pipe destruction this function flushes
all messages at the destination by sending an empty message from the
source to the destination and back. The caller may specify a callback
to be invoked by this message at the destination endpoint. This can
be used to notify the destination that the channel is about to be
destroyed and it must stop generating new messages.

Use local unw_context variable instead of passing a pointer to a backtrace
function.

We're going to need a key def of secondary key augmented
with key parts from the primary key in memtx engine as well.
Initialize it when creating struct index_def.

This patch reverts f42e1c15,
i.e. the fix for gh-2386 (box_index_key_def() as part of module
API). Exporting this function to C API was premature. Luckily
it is not used anywhere except 1.8. For 1.8 this patch introduces
a faster function, space_index_key_def(), which takes a pointer
to space rather than space id.

In the next patch I plan to add one more parameters to index_def
constructor, namely the primary key def.
Refactor the constructor to expect a fully constructed key_def as an
argument.

The patch happens to simplify the code in schema.cc, cluttered by
index_def vs. key_def split.

Since 2ab97c01 "introduce vy_stmt_env and vinyl_max_tuple_size option"
Vinyl's memory is limited both by `struct quota` and `struct vy_quota`.
This causes sporadic allocation errors because `vy_quota` may
exceed in some cases.

Closes #2629

sysconf(_SC_IOV_MAX) can return -1 on glibc 2.25.90.

Closes #2637

We do not store id = 0 in xlog, yet when parsing the log
file expect space id to be present in a request (a mandatory key).

So a space with id 0 is not possible to recovery from a
snapshot.

Simply reserve id 0 to avoid the problem.

region_truncate() is not required there as box_index_get() cleans up
fiber()->gc anyway - see txn_commit_ro_stmt(). Actually, it's even
dangerous: if fiber_gc() called by box_index_get() happens to free
memory, region_truncate() will crash on the assertion:

  region_truncate: Assertion `region_used(region) >= used' failed.