To make functions in execute.h more universal we should reduce
their dependence on IPROTO. This patch removes IPROTO functions
from execute.c.

Needed for #3505

New method dump_lua dumps saved in port tuples to Lua stack. It
will allow us to call this method without any other interaction
with port.

Needed for #3505

Refactored sql_request structure to store pointer to sql string
data and it's length instead of pointer to msgpack
representation.
This is required to use this structure in sql.c where the query
has a different semantics and can be obtained from stack as a C
string.

Needed for #3505.

tonumber64() doesn't understand strings with "ULL" like "123ULL". The
expected output for tonumber64("123ULL") is 123, since 123ULL is a
correct number notation in lua. However, our function returns null.
This happens because suffix isn't trimmed in tonumber64.

Trim ULL/LLU, LL suffixes, but only when no base is specified or
base is equal to either 2, 10 or 16.

Closes #3431

Part of #2436, #3232

- Box configuration parameter `memtx_memory` is increased, because the
  test `lua` after `tuple` failed with the error:
  `Failed to allocate 368569 bytes in slab allocator for memtx_tuple`
  despite `collectgarbage('collect')` calls after cases with huge/many
  tuples.
  The statistics before the allocation fail gives the following values:
  ```
  box.slab.info()
  ---
  - items_size: 72786472
    items_used_ratio: 4.43%
    quota_size: 107374592
    quota_used_ratio: 93.75%
    arena_used_ratio: 6.1%
    items_used: 3222376
    quota_used: 100663296
    arena_size: 100663296
    arena_used: 6105960
  ```
  The reason of the fail seems to be a slab memory fragmentation. It is
  not clear for now whether we should consider this as a tarantool
  issue.

- Test `snapshot_stress` counts snapshot files present in the
  working directory and can reach the default 'checkpoint_count' value
  `2` if a previous test write its snapshots before.

- Restarting the default server w/o cleaning a working directory
  can leave a snapshot that holds a state saved at the middle of a test,
  before dropping of the space 'tweedledum' (because WAL is disabled),
  that can cause the error `Space 'tweedledum' already exists` for a
  following test.

- Use unix sockets because of errors `Address already in use`.

Part of #2436

Too many autogenerated ids leads to SEGFAULT. This problem
appeared because region was cleaned twice: once in
sqlite3VdbeHalt() and once in sqlite3VdbeDelete() which was
executed during sqlite3_finalize(). Autogenerated ids that were
saved there, were fetched after sqlite3VdbeHalt() and before
sqlite3_finalize(). In this patch region cleaning in
sqlite3VdbeHalt() has been removed.

Follow up #2618
Follow up #3199

Before this patch MSGPACK received using SELECT statement through
net.box was unpacked. Fixed in this patch.

Previously if an error occured in box_index_len() called from
sql_analysis_load(), the return code (-1 on error) was cast to uint32_t
and used later as size of memory to be allocated. This lead to assertion
failures in slab_order() since allocation size was too big. This was
discovered during investigation of #3779.
Fix error handling and add some error logging.

Follow-up #3779

Again, this is something that was introduced by commit f2bccc18
("Use WAL vclock instead of TX vclock in most places") without any
justification.

TX has its own copy of the current vclock - there's absolutely no need
to inquire it from the WAL thread. Actually, we already use TX local
vclock in box_process_vote(). No reason to treat join/subscribe any
different. Moreover, it's even harmful - there may be a gap at the end
of a WAL file, in which case WAL vclock will be slightly ahead of TX
vclock so that should a replica try to subscribe it would never finish
syncing, see #3830.

Closes #3830

Because this is pointless and confusing. This "feature" was silently
introduced by commit f2bccc18 ("Use WAL vclock instead of TX vclock
in most places"). Let's revert this change. This will allow us to
clearly separate WAL checkpointing from WAL flushing, which will in turn
facilitate implementation of the checkpoint-on-WAL-threshold feature.

There are two problems here, however. First, not rotating the log breaks
expectations of replication/gc test: an xlog file doesn't get deleted in
time as a consequence. This happens, because we don't delete xlogs
relayed to a replica after join stage is complete - we only do it during
subscribe stage - and if we don't rotate WAL on subscribe the garbage
collector won't be invoked. This is actually a bug - we should advance
the WAL consumer associated with a replica once join stage is complete.
This patch fixes it, but it unveils another problem - this time in the
WAL garbage collection procedure.

Turns out, when passed a vclock, the WAL garbage collection procedure
removes all WAL files that were created before the vclock. Apparently,
this isn't quite correct - if a consumer is in the middle of a WAL file,
we must not delete the WAL file, but we do. This works as long as
consumers never track vlcocks inside WAL files - currently they are
advanced only when a WAL file is closed and naturally they are advanced
to the beginning of the next WAL file. However, if we want to advance
the consumer associated with a replica when join stage ends (this is
what the previous paragraph is about), it might occur that we will
advance it to the middle of a WAL file. If that happens the WAL garbage
collector might remove a file which is actually in use by a replica.
Fix this as well.

First, this is consistent with other engine callbacks, such as
checkpoint or backup.

Second, a vclock can be used as a search key in a vclock set,
which in turn can make code more straightforward, e.g. look how
this patch simplifies vy_log_prev_checkpoint().

In the updated version rb_proto/rb_gen use const qualifier for the key
argument, which allows to pass pointers to const objects to search
methods.

Removed INDEX_XINFO pragma and updated INDEX_INFO with its columns with
the following corrections: replaced 6th result column (whether an index
column is key/aux one) with type name of the index column.

Cleaned up pragma column names array (pragma.h::pragCName).

Fixes #3194.

Allow constraints to appear along with columns definitions. Disallow typing
a constraint name without specifying the constraint and after it.

Closes #3504

 - Before checking that old WAL files have been removed, wait for the
   garabe collection to remove them to avoid a spurious test failure.
   Currently, the test waits until old checkpoints are removed, but the
   garbage collector can now remove checkpoints, but keep WAL files.
   This is a follow-up for commit 9c5d851d ("replication: remove old
   snapshot files not needed by replicas").
 - Remove a few pointless box.info.gc().checkpoints checks.
 - Use test_run.wait_cond for waiting instead of while-do-sleep loops.

The test is very intricate and relies on timings too much so that should
we change anything in the SUBSCRIBE procedure, we could easily get a
spurious test failure. Let's rewrite it a little bit to make it more
robust. The details are in the comment in the code.

While we are at it, let's also fix comments and use test_run.wait_cond
helper for waiting for a condition to come true instead of repeat-sleep
loops.

When SQL DELETE statement comes in most primitive from without WHERE
clause and foreign key constraints, it is optimized and processed with
one VDBE instruction (instead of several OP_Delete). However, it was
forgotten to account affected tuples by row counter. Current patch fixes
this obvious defect.

Closes #3816

To fix the warning on bootstrap:

  xlog.c:1920 E> can't open tx: bootstrap: has some data after eof marker at 1906

It started to hang after the last merge.

Introduced a new tuple_format_use_key_part that makes format
initialization for specified key_part and configuration.
This decrease tuple_format_create routine complexity and would
be used to initialize structures in format for JSON path.

Need for #1012

The new tuple_field_go_to_path routine is used in function
tuple_field_raw_by_path to retrieve data by JSON path from field.
We need this routine exported in future to access data by JSON
path specified in key_part.

Need for #1012

Introduced a new key_def_parts_are_sequential routine that test,
does specified key_def have sequential fields. This would be
useful with introducing JSON path as there would be another
complex criteria as fields with JSONs can't be 'sequential' in
this meaning.

Need for #1012

Refactored key_def_find routine to use key_part as a second
argument. Introduced key_def_find_by_fieldno helper to use in
scenarios where no key_part exists.
New API is more convenient for complex key_part that will appear
with JSON paths introduction.

Need for #1012

After patch that introduced "none" collation (a953051e),
box.internal.bootstrap() started to fail due to inability to drop
mentioned collation. Lets turn off system triggers for _collation space
in order to process its complete purification during bootstrap.

box.cfg() updates only those options that have actually changed.
However, for replication it is not always true: box.cfg{replication = x}
and box.cfg{replication = {x}} are treated differently, and as
the result - replication is restarted. The patch fixes such behaviour.

Closes #3711

Currently, run_count_per_level index option is applied to each LSM tree
level. As a result, we may end up storing each key run_count_per_level
times at the last level alone, which would result in prohibitive space
amplification. To avoid that, let's ignore run_count_per_level for the
last level.

Note, we have to tweak quite a few vinyl tests, because they implicitly
relied on the fact that producing run_count_per_level dumps would never
trigger compaction.

Closes #3657

I. Fixes tarantoolctl rocks install hanging in resctricted networks
corner-case.

A customer configured two rocks servers:
1. offline (file:///path/to/rocks)
2. and default online (rocks.tarantool.org)

He tries to do `rocks install http 1.0.5-1`.

Online server is unavailable due to his local network policy, but
the rock is available offline. Despite anything, luarocks still
tries to fetch manifest online, which results in 30 sec hang since
network access is restricted.

This change aborts scanning when exact match is found

II. Remove cyclic dependencies

This is required to embed luarocks into tarantool, as
current tarantool preloader can't preload cyclic dependencies.
There should be a unidirectional dependency graph and
predictable order.

Note: as a consequence of this patch, operating systems other that
unix-compatible ones are no longer supported. This is because I
had to manually resolve dependency graph for predictable require()
order.

III. Use digest.md5_hex to compute md5 digests instead of openssl

luarocks has support for calculating md5 with 'md5' rock if it's
present, but we don't have it in tarantool, and instead have the
'digest' module. That's why luarocks falls back to 'openssl' binary
to calculate md5 digests.

This patch will allow luarocks to use our internal digests module.

Before this patch region wasn't truncated after box.snapshot() so some
tests failed due to "memory leak". To fix that, let's truncate region in
vinyl and use malloc() in memtx (we can't cleanup region there as
objects allocated when checkpoint is started are used until checkpoint
is completed). The latter isn't a big deal, because box.snapshot() isn't
a hot path and we do other allocations with malloc() anyway.

Closes #3732

Before this patch our SQL implementation relied on strange rules
when comparing strings with different collations:

- if either operand has an explicit collating function assignment using
  the postfix COLLATE operator, then the explicit collating function is
  used for comparison, with precedence to the collating function of the
  left operand;

- if either operand is a column, then the collating function of that
  column is used with precedence to the left operand.

The main concern in this implementation is that collation of the left
operand is forced over right one (even if both operands come with
explicit COLLATE clause). This contradicts ANSI SQL and seems to be
quite misleading, since if user simple swap LHS and RHS - result of
query may change.

Lets introduce restrictions concerning collations compatibilities.
Collations of LHS and RHS are compatible (i.e. no "Illegal mix of
collations" is thrown) if:

- one of collations is mentioned alongside with explicit COLLATE clause,
  which forces this collation over another one. It is allowed to have
  the same forced collations;

- both collations are derived from table columns and they are the same;

- one collation is derived from table column and another one is not
  specified (i.e. COLL_NONE).

The compound SELECT operators UNION, INTERSECT and EXCEPT perform implicit
comparisons between values. Hence, all new rules stated above are
applied to parts of compound SELECT. Otherwise, an error is raised.
In other words, before current patch queries like below were allowed:

SELECT 'abc' COLLATE binary UNION SELECT 'ABC' COLLATE "unicode_ci";
---
- - ['ABC']
  - ['abc']
...

If we swap collations, we will get another result:

SELECT 'ABC' COLLATE "unicode_ci" UNION SELECT 'abc' COLLATE BINARY
---
- - ['abc']
...

Now such queries are illegal.

Closes #3185

This patch introduces two new collation sequences: "none" and "binary".
Despite the fact that they use the same comparing algorithm (simple
byte-by-byte comparison), they don't mean the same. "binary" collation
get to the format if user explicitly points it: either specifies this
collation in space format manually or adds <COLLATE BINARY> clause to
column definition within CREATE TABLE statement. "none" collation is
used when user doesn't specify any collation at all. "none" collation always
comes with id == 0 and it can't be changed (since its id vastly used
under the hood as an indicator of absence of collation).
Difference between these collations is vital for ANSI SQL: mixing
"binary" with other collations is prohibited, meanwhile "none" collation
can be used alongside with others.
In this respect current patch extends list of available collations: now
not only ICU collations are allowed, but also BINARY.

Note, that in SQL some queries have changed their query plan. That
occurred due to the fact that our parser allows using <COLLATE> clause
with numeric fields:

CREATE TABLE (id INT PRIMARY KEY);
SELECT id COLLATE "binary" ...

In the example collation of LHS (id column) is NULL, but collation of
RHS is "binary". Before this patch both collations were NULL. Hence,
usage of certain indexes may not be allowed by query planner.
On the other hand, this feature is obviously broken, so that doesn't
seem to be big deal.

Needed for #3185

Since now we don't have real binary collation, it was allowed to use its
name in different cases. For instance: BiNaRY, "bInArY", "BINARY" etc.
All these names were valid and accounted for binary collation.
However, we are going to introduce real entry in _collation space to
represent binary collation. Thus, for now we allow using only standard
"binary" name.

Needed for #3185

We don't need to add explicit COLLATE "BINARY" clause since binary
collation is anyway default. On the other hand, it may confuse
due to ambiugty when comparing two terms.

Needed for #3185

changes() is legacy function which came from SQLite. However, its name
is not perfect since it's "number of rows affected", not"number of rows
changed". Moreover, to make it real row count, we are nullifying counter
representing it in case operation is not accounted for DML or DDL.
For example, before this patch behavior was like:

INSERT INTO t VALUES (1), (2), (3);
START TRANSACTION;
SELECT changes();
---
- - [3]
...

As one can see, row counter remained unchanged from last INSERT
operation. However, now START TRANSACTION set it to 0 and as a
consequence row_count() also returns 0.

Closes #2181