Lets evaluate an expression type during processing of expression's AST
and code generation. It allows to calculate resulting columns data types
and export them as IProto meta alongside with columns' names.
Also, correct types are also returned for binding parameters as well.

Note that NULL literal has type "BOOLEAN". It was made on purpose -
different DBs interpret NULL's type in different ways: some of them
use INT; others - VARCHAR; still others - UNKNOWN. We've decided that
NULL is rather of type "BOOLEAN", since NULL is kind if subset of
"BOOLEAN" values: any comparison with NULL results in neither TRUE nor
FALSE, but in NULL.

Part of #2620

Most DBs (at least PostgreSQL, Oracle and DB2) allow to process
following queries:

CREATE TABLE t1 (id INT PRIMARY KEY);
INSERT INTO t1 VALUES (1.123), ('2');

In this particular case, 1.123 should be simply truncated to 1,
and '2' - converted to literal number 2.

After passing real type to Tarantool (instead of <SCALAR>), example
above would fail without conversions. Thus, lets add implicit
conversions inside VDBE to make this example be legal.
However, still some types conversions must be prohibited. For instance,
<BLOB> can't be converted to integer or floating point numerical,
and vice versa.

In SQLite unary plus behaves as implicit conversion to numeric type.
Consider following example:

CREATE TABLE t1 (id INT PRIMARY KEY, a TEXT, b BLOB);
INSERT INTO t1 VALUES (1, '99', '99');

SELECT * FROM t1 WHERE a = b; (*)
SELECT * FROM t1 WHERE +a = +b; (**)

Since BLOB and TEXT are incompatible, result of (*) would be empty set.
However, comparison in second query (**) would be of <NUMERIC> types,
and result set would consist of one tuple [1, '99', '99'].

Lets discard this conversion produced by unary plus, since it implicitly
affects result set of query and no one other DB support such behaviour.
Instead, simply use type of operand it is related to.
Note, that unary minus doesn't affect types in any way.

As a main part of introducing strict typing in SQL it is required to
prohibit typeless columns in parser's grammar. Originally, SQLite simply
assigns typeless columns to BLOB affinity. Moreover, due to historical
reasons, all columns were stored with <SCALAR> type in Tarantool core
(except for <INTEGER> when it comes to primary key).  Column type should
be defined on table creation. Allowed data types are: <TEXT>, <VARCHAR>,
<CHAR>, <BLOB>, <INT[EGER]>, <REAL>, <FLOAT>, <NUMERIC>, <DECIMAL>,
<DOUBLE> <DATE> and <DATETIME>. However, still any declared data type is
converted to one of <BLOB>, <TEXT>, <REAL> or <INTEGER> affinities.
While affinity reaches space format, it is (again) converted to
Tarantool's field type. To be more precise, table of conversions:

+----------+----------+------------+
| SQL TYPE | AFFINITY | FIELD TYPE |
+----------+----------+------------+
| FLOAT    | REAL     | NUMBER     |
| REAL     | REAL     | NUMBER     |
| DOUBLE   | REAL     | NUMBER     |
| NUMERIC  | REAL     | NUMBER     |
| DECIMAL  | REAL     | NUMBER     |
| INTEGER  | INTEGER  | INTEGER    |
| TEXT     | TEXT     | STRING     |
| VARCHAR  | TEXT     | STRING     |
| CHAR     | TEXT     | STRING     |
| BLOB     | BLOB     | SCALAR     |
| DATETIME | REAL     | NUMBER     |
| DATE     | REAL     | NUMBER     |
| TIME     | REAL     | NUMBER     |
+----------+----------+------------+

<VARCHAR> and <CHAR> types should be specified with length
(e.g. name VARCHAR(10)), but this length currently is not used when
types are processed. Only purpose is to support ANSI syntax.
The same for <NUMERIC> and <DECIMAL> - it is allowed to specify scale
and precision, but they don't affect the way they are stored in memory.

Note that patch is not self-sufficient: a lot of tests still fail due to
wrong types conversions. Fix for that comes as next two patches.

Closes #3018
Closes #3104
Closes #2494
Closes #3459

Any SQL function should have declared type of returning value.
Lets temporary add affinity to function definition as a type of
returning value.

Lets introduce separate field in struct Expr to store conflict action of
RAISE() function, instead of messing it with affinity.

Before this patch, _sequence_data system space wasn't taken into
consideration when space was dropped. However, entries in this space
may appear after recovery. For example:

CREATE TABLE t (id INT PRIMARY KEY AUTOINCREMENT);
INSERT INTO t VALUES (NULL);
box.snapshot()
os.exit()
...
box.cfg{}
DROP TABLE t;

Last DROP statement didn't generate VDBE code to delete entry from
_sequence_data, but it had to do so. This patch simply modifies code
generation to remove entry from _sequence_data space before space is
dropped.

Closes #3712

nKey member (indicating number of parts in key) now is used only for
COUNT routine. On the other hand, it is always equal to 0 (as well as
key is really NULL), since functions which are relied on this argument
(tarantoolSqlite3Count and tarantoolSqlite3EphemeralCount) are called to
perform OP_Count opcode. In its turn, OP_Count is emitted only for
queries like <SELECT COUNT(*) FROM tab>, i.e. simple queries without
filter conditions. Hence, nothing prevents us from removing this field
at all.

If query is like "INSERT INTO table SELECT ..." then it is
completely fine that the result of SELECT isn't integer. These
values wouldn’t be checked properly if it was inserted in
column with autoincrement. This patch fixes mentioned check.

Closes #3670

In case statement "CREATE TABLE ..." fails it can left some
records in system spaces that shouldn't be there. These records
won't be left behind after this patch.

@TarantoolBot document
Title: Clean up after failure of CREATE TABLE
Usually CREATE TABLE creates no less than two objects which are
space and index. If creation of index (or any object after space)
failed, created space (and other created objects) won't be deleted
though operation failed. Now these objects will be deleted
properly.

Closes #3592

Introduce a new function, scramble_reencode(), which allows one cluster
instance to reencode a scramble received from  a client using the salt
from another cluster instance. This is needed for proxy to mimic client
connections when connecting to remote instances.

Needed for #2625

This patch adds an upgrade script to grant CREATE, ALTER, DROP
privileges to users which have READ+WRITE on respective objects.
This is needed after removing 1.7 compatibility mode for privileges.

Closes #3539

Before version 1.7.7 there were no CREATE or ALTER privileges.
READ+WRITE permitted object creation and altering.
In 1.10 CREATE and ALTER privileges were introduced together with a
compatibility mode in access_check_ddl() which assumed user had CREATE
and ALTER if it had READ and WRITE on a respective object.
Now its time for us to remove this compatibility mode.

Part of #3539

Since SQL front-end is not using box API,
no checkes for read access are performed by VDBE engine.
Add check to IteratorOpen op-code to make sure that read
privilege exists for given space.
Note, that there's is no need to perform DML/DDL checkes as
they're performed by Tarantool's core.

@TarantoolBot document
Title: Document behaviour of SQL in presence of
read access restrictions. Need to clarify, that
if there's no read access to the space, then not
only SELECT statements will fail, but also those DML
which implies reading from spaces indirectly, e.g.:
  UPDATE t1 SET a=2 WHERE b=3;

Closes #2362

Since hash, which maps space name to space pointer was introduced in
previous patch, use it in SQL front-end as it is heavily needed.

@locker: removed a couple of useless assertions and variable renames

Since SQL is heavily using name -> space mapping, introduce (instead of
scanning _space space) dedicated cache, which maps space name to space
pointer.

@locker: code cleanup

sql_compile_view() may fail, in which case the space will never be
deleted from (in case of space creation) or inserted back into (in case
of space drop) the space cache, because commit/rollback triggers, which
are supposed to do the job, are only installed after preparing a view.
Fix this by installing triggers before sql_compile_view().

No need to write a test as without this commit sql/view test will
crash after applying the next commit (the one that introduces space
name cache).

Fixes commit dc358cb0 ("sql: rework VIEW internals").

In some cases luajit does not collect cdata objects which were
transformed with ffi.cast as tuple_bless does. In consequence, internal
table with gc callback overflows and then lua crashes. There might be an
internal luajit issue because it fires only for jitted code. But assigning
a gc callback before transformation fixes the problem.

Closes #3751

Back when bloom filters were introduced, neither box.info.memory() nor
box.stat.vinyl().memory didn't exist so bloom filters were accounted to
box.runtime.info().used for lack of a better place. Now, there's no
point to account them there. In fact, it's confusing, because bloom
filters are allocated with malloc(), not from the runtime arena, so
let's drop it.

If a tuple read from a run by a slice stream happens to be out of the
slice bounds, it will never be freed. Fix it.

The leak was introduced by commit c174c985 ("vinyl: implement new
simple write iterator").

After changing behavior of the `IS` operator (#b3a3ddb5),
`SET NULL` was rewritten to use `EQ` instead. Which doesn't respect
NULLs.

This commit fixes the null related behavior by emitting logical
constructions equivalent for this case to old `IS`.
The new expression works differently than old `IS` for nulls, however
the difference doesn't change anything, because matched rows are then
searched in a child table with `EQ` expression which do not match nulls.
Before:
`oldval` old_is `newval`
Now:
`oldval` is_null or (`newval` is_not_null and `oldval` eq `newval`)

Closes #3645

Upload tarballs of alpha and beta tarantool versions (*.0 and *.1
branches) into 2x (3x, 4x...) buckets. See more details about the
release process in the documentation: [1].

[1]: https://tarantool.io/en/doc/2.0/dev_guide/release_management/

This patch adds logging amount of rows received by applier during the
join stage, the same way that recovery has it.

Closes #3165

Remove function which deletes from cache, making replace more general:
it might be used for both insertions, deletions and replaces. Also, put
assert on equality of space pointer found in cache to old one into
replace routine.

The idea behind this change is to have 2x (and maybe later 3x, 4x, ...)
bucket for alpha and beta releases. See more details about the release
process in the documentation: [1].

[1]: https://tarantool.io/en/doc/2.0/dev_guide/release_management/

Now if the WAL thread fails to preallocate disk space needed to commit
a transaction, it will delete old WAL files until it succeeds or it
deletes all files that are not needed for local recovery from the oldest
checkpoint. After it deletes a file, it notifies the garbage collector
via the WAL watcher interface. The latter then deactivates consumers
that would need deleted files.

The user doesn't see a ENOSPC error if the WAL thread successfully
allocates disk space after deleting old files. Here's what's printed
to the log when this happens:

  wal/101/main C> ran out of disk space, try to delete old WAL files
  wal/101/main I> removed /home/vlad/src/tarantool/test/var/001_replication/master/00000000000000000005.xlog
  wal/101/main I> removed /home/vlad/src/tarantool/test/var/001_replication/master/00000000000000000006.xlog
  wal/101/main I> removed /home/vlad/src/tarantool/test/var/001_replication/master/00000000000000000007.xlog
  main/105/main C> deactivated WAL consumer replica 82d0fa3f-6881-4bc5-a2c0-a0f5dcf80120 at {1: 5}
  main/105/main C> deactivated WAL consumer replica 98dce0a8-1213-4824-b31e-c7e3c4eaf437 at {1: 7}

Closes #3397

In order to implement WAL auto-deletion, we need a notification channel
through which the WAL thread could notify TX that a WAL file was deleted
so that the latter can shoot off stale replicas. We will reuse existing
wal_watcher API for this. Currently, wal_watcher invokes the registered
callback on each WAL write so using it as is would be inefficient. To
avoid that, let's allow the caller to specify events of interest when
registering a wal_watcher.

Needed for #3397

We will add another event bitmap to wal_watcher. To avoid confusion
between them, let's rename wal_watcher->events.

This should make it easier to pass some extra information along with the
event mask. For example, we will use it to pass the vclock of the oldest
stored WAL, which is needed for WAL auto-deletion.

Needed for #3397

Added -DENABLE_LTO=ON/OFF cmake option, OFF by default.

LTO speeds up cpu-intensive workloads by up to 20% (see [1] and [2]).

Requirements to enable LTO:

- cmake >= 3.9;
- Linux: ld.bfd / ld.gold from binutils >= 2.31 (or later 2.30)
  (gold >= 1.15);
- Mac OS: xcode >= 8 (earlier versions are not tested).

The requirement of the recent ld version is due to bug with exporting
symbols from dynamic list when LTO is enabled, see [3].

Note: -Wlto-type-mismatch on GCC (enabled by default with -flto) gives
namy warnings. Filed [4] to investigate it.

Note: LuaJIT will be compiled w/o LTO despite the option set, filed [5].

[1]: https://github.com/tarantool/tarantool/wiki/performance-research
[2]: https://gist.github.com/Khatskevich/31a2da6ab46ce903120e7a03d65966db
[3]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84901
[4]: https://github.com/tarantool/tarantool/issues/3742
[5]: https://github.com/tarantool/tarantool/issues/3743

Closes #3117

f

In case of very aggressive optimizations the compiler can
optimize guard-breaker function away and the `unit/guard`
test would fail.

2.0 was renamed to 2.1.

Updated the test case for #2780 to check a last snapshot file
modification time instead of search log messages.

The test was flaky, because of small timeouts on Linux, but now we
spinning on a condition check to achieve both stable results and fast
execution.

Follows up #2780.
Fixes #3684.