JSON paths has no a strict standard, but definitely there is no
an implementation, allowing to omit '.' after [], if a next token
is a key. For example:

    [1]key

is invalid. It should be written like that:

    [1].key

Strangely, but we even had tests on the invalid case.

Closes #4419

Disable a check whether yaml responses are well-formed for 'core =
tarantool' tests in test-run. The check was unable to handle complex
(dictionary / list) keys in a dictionary, because pyyaml does not
support them.

See also https://github.com/yaml/pyyaml/issues/88

Fixes #4421.

In one place that test sends a packet and expects that it has
arrived two lines below. Under high load it may take more time.
The patch makes the test explicitly wait for the packet arrival.

Closes #4392

There is compile time option PACKAGE in cmake to define
current build distribution info. By default it's
"Tarantool" for the community version and "Tarantool Enterprise"
for the enterprise version.

It's displayed in console greeting and in `box.info().package`,
but, unfortunately, it can't be accessed from Lua before `box.cfg`.

This patch exposes `require('tarantool').package`.

Close #4408

@TarantoolBot document
Title: Extend module "tarantool" with the field "package"

Beside from build info and version, module "tarantool" now provides
"package" field. By default it equals string "Tarantool", but
can differ for other distributions like "Tarantool Enterprise".

Example:

```console
tarantool> require('tarantool')
---
- version: 2.3.0-3-g302bb3241
  build:
    target: Linux-x86_64-RelWithDebInfo
    options: cmake . -DCMAKE_INSTALL_PREFIX=/opt/tarantool-install
-DENABLE_BACKTRACE=ON
    mod_format: so
    flags: ' -fexceptions -funwind-tables -fno-omit-frame-pointer
-fno-stack-protector
      -fno-common -fopenmp -msse2 -std=c11 -Wall -Wextra
-Wno-strict-aliasing -Wno-char-subscripts
      -Wno-format-truncation -fno-gnu89-inline -Wno-cast-function-type'
    compiler: /usr/bin/cc /usr/bin/c++
  pid: 'function: 0x40016cd0'
  package: Tarantool
  uptime: 'function: 0x40016cb0'
...

```

The _func_index space trigger used to reject an insertion of a
tuple that defines an invalid functional index.
As insertion in _index space had been completed before, a garbage
is kept in _index space in such case.

We need to do something with the yelding _func_index trigger(that
rebuilds an index) to wrap all index creation operation in DDL
transaction in further patches (because only the first DDL
operation may yeld now).

This problem could be trivially solved with preparatory
initialization of index_def function ponter: the memtx_tree
index construction would perform all required job in such case.
Therefore the following index rebuild in _func_index trigger
becomes redundant and should be omitted.

In other words, a trivial prefetch operation makes possible
a transactional index creation (with space:create_index operation).

As for index construction during recovery (a lack of function
cache during recovery was the main motivation to introduce
_func_index space), it's workflow is kept unchanged.

Follow up #1250
Needed for #4348
Closes #4401

As per request by Kostja, commit an RFC document with a brief history of
the vinyl metadata log infrastructure, issues it was intended to solve,
problems we are facing now, and possible ways to solve them.

In case an error occurs between relay_start() and cord_costart() in
relay_subscribe(), the relay status won't be reset to STOPPED. As a
result, any further attempt to re-subscribe will fail with ER_CFG:
duplicate connection with the same replica UUID. This may happen, for
example, if the WAL directory happens to be temporarily inaccessible on
the master.

Closes #4399

The function

 | require('console').get_default_output()

requires no arguments. Make it explcicit and print
an error otherwise.

Part-of #3834

Instead of allocating a variable for optional args
testing we should use dot notation instead. Otherwise
it won't work for trivial test case as

```
require('console').set_default_output('lua,block')
require('decimal').new('1234.5678')
```

 | builtin/box/console.lua:47: expected decimal, number or string as 2 argument

and program exits.

Part-of #3834

In output_verify_opts and output_parse we return status variable
to point if function processed without error. This is redundant
we can simply return either error or nil, which is enough.

Part-of #3834

It was decided that null value in SQL by default should be of type
boolean. Justification of such change is that according to ANSI boolean
type in fact has three different values: true, false and unknown. The
latter is basically an alias to null value.

If during creation of constraint its name is not specified, then it is
generated automatically. Occasionally, default names for each type of
constraint turn out to be different. This patch makes them follow the
same pattern: "shortcut"_unnamed_"table_name"_"ordinal_numb". For
instance: fk_unnamed_T1_1 or ck_unnamed_T1_3 etc

If UNIQUE constraint is specified in CREATE TABLE statement and it has
given name, one is mangled with pattern "unique_%s_%d", where %s is
original name of constraint and %d - current iid. For instance:

CREATE TABLE t (id INT PRIMARY KEY, a INT CONSTRAINT i1 UNIQUE);

This statement results in secondary index creation with name
"unique_I1_1". Justification for mangling is that constraint namespace
should be independent from index namespace. However, ALTER TABLE ADD
CONSTRAINT UNIQUE which is alias to CREATE INDEX doesn't mangle name.
What is more, users may wonder why name of index is different from
name of created constraint. Hence, it has been decided to remove this
mangling and create index exactly with specified constraint's name.

Its purpose is to copy integer value from one memory cell to another.
In other words, it is particular case of OP_SCopy instruction. Since it
is used only during creation of AUTOINCREMENT property, it seems to be
reasonable to replace it with a bit general OP_SCopy and erase
OP_IntCopy at all reducing size of SQL codebase.

It's not used anymore and can be removed to reduce size of codebase.

After the removal of the LOOKASIDE system, it was found that the
performance dropped. To restore performance, this patch partially
returns the LOOKASIDE system in almost the same form as it was
before the patch 4326ca7d.

Closes #4319

In accordance with the Lua coding style in Tarantool, all errors
returned in Lua should be returned using 'return nil, error'.
However, box.execute() throws an exception in case of an error.
This patch causes box.execute() to return an error, as described
in the coding style.

Closes #4390

Currently, if we have to return errors using the format
'return nil, error', we must do it manually. Since this error
return method is described in our Lua coding style, it makes sense
to create a function that will return an error in this format.
This patch creates mentioned function.

Needed for #4390

Follow-up #3589

According to ANSI, LIKE should match characters taking into
account passed collation.

ISO/IEC JTC 1/SC 32 2011, Part 2: Foundation, 8.5

Closes #3589

@TarantoolBot document
Title: LIKE depends on collations

Now <LIKE> operator depends on arguments' collations. Collation to be
used is determined by common rules: if explicit collation or collations
(and in case they are compatible) is specified, then it is used to
process pattern matching; otherwise implicit collation of arguments are
checked to be compatible and (in case they are) resulting collation is
used. Moreover, it means that PRAGMA "case_sensitive_like" has been
removed.

According to ANSI, LIKE should match characters taking into account
collations of arguments, and this is done in the next patch. In turn,
this patch makes LIKE be always case sensitive and erases
case_sensitive_like pragma. Alongside with it, code related to no-case
LIKE optimization is removed as well.

Part of #3589

Current patch introduces new type available in SQL:
 - VARBINARY now is reserved keyword;
 - Allow to specify VARBINARY column and CAST type;
 - All literals which start from 'x' are assumed to be of this type;
 - There's no available implicit or explicit conversions between
   VARBINARY and other types;
 - Under the hood all values of VARBINARY type are stored as MP_BIN
   msgpack format type.

Closes #4206

According to ANSI specification, if TRIM function accepts binary string
and trim octet is not specified, then it is implicitly set to X'00'.
Before this patch trim octet was set to ' ' both for string and binary
string arguments. In turn, ' ' is equal to X'20' in hex representation.
Hence, TRIM function cut wrong characters:

TRIM(X'004420') -> X‘0044'

This patch sets default trim octet to X'00' for binary string arguments.

Part of #4206

Since values of type 'varbinary' can't be cast to any other type, let's
patch built-in functions which are not assumed to accept arguments of
this type to raise an error in case argument turn out to be of type
varbinary.

Part of #4206

We are going to introduce new column type 'VARBINARY', which allows to
store values with MP_BIN msgpack format. On the other hand, now it is also
possible to meet this type: all literals in form of x'...' are
supposed to be inserted into SCALAR column type exactly with MP_BIN
encoding. Prior to this moment type of such values (encoded as MP_BIN)
was called 'blob'. Thus, let's fix all visible to user messages using
'varbinary' name of type instead of 'blob'.

Before this patch, resulting type for CASE-WHEN statement was assumed to
be the same as type of argument of first THEN clause. Obviously, it is
wrong and could lead to sad consequence (e.g. creating ephemeral table
with inconsistent format). To deal with this, we check all THEN
arguments: if all of them have the same type, then such type will be
resulting of the whole statement; if at least two types are different,
we can't determine actual resulting type during compilation stage and
assign SCALAR as a most general type in SQL now.

Need for #4206

Function which converts values to string representation
(sqlVdbeMemStringify()) erase MEM_Int/MEM_Real/MEM_Bool flags only when
it is specified by 'force' parameter. Hence, when 'force' argument is
false, memory cell after conversion will contain string value, but flag
indicating its type will be equal to combination of MEM_Str and one of
mentioned flags. It seems to be remains of affinity routines, since in
current state memory cell must have only one type.  What is more, it can
lead to unpredicted consequences, for instance assertion fault
(sql_value_type() assumes that value has one specific type). Let's fix
it removing 'force' argument from sqlVdbeMemStringify() and always clean
up type flag.

Box recently added support of transactional DDL allowing to do
any number of non-yielding DDL operations atomically. This is
really a big relief of one of the biggest pains of SQL. Before
this patch each multirow SQL DDL statement needed to prepare its
own rollback procedure for a case if something would go wrong.

Now with box support SQL wraps each DDL statement into a
transaction, and doesn't need own escape-routes in a form of
'struct save_record' and others.

Closes #4086

@TarantoolBot document
Title: SQL DDL is transactional

SQL DDL operations are atomic now. For example, if a CREATE TABLE
request fails somewhere in the middle, it won't leave any
garbage. Like a space without indexes, or unused sequences. Even
if the instance is powered off during the request.

Also, SQL DDL can be manually included into transactions, with
certain limitations - such a transaction can't yield.

For example, this is legal:

    START TRANSACTION;
    CREATE TABLE test(a INTEGER PRIMARY KEY, b INTEGER);
    CREATE INDEX test_a ON test(a);
    COMMIT;

If you want to test it in the console, then wrap it into a
function to do not get a rollback by yield, because the console
yields after each command:

    function create()
        box.execute('START TRANSACTION;')
        box.execute('CREATE TABLE test(a INTEGER PRIMARY KEY, b INTEGER);')
        box.execute('CREATE INDEX test_a ON test(a);')
        box.execute('COMMIT;')
    end

    create()

But the following example is illegal and you will get an error:

    box.execute('CREATE TABLE test(a INTEGER PRIMARY KEY, b INTEGER, c INTEGER);')
    box.execute('INSERT INTO test VALUES (1, 1, 1), (2, 2, 2), (3, 3, 3);')

    function index()
        box.execute('START TRANSACTION;')
        box.execute('CREATE INDEX test_b ON test(b);')
        box.execute('CREATE INDEX test_c ON test(c);')
        box.execute('COMMIT;')
    end

    tarantool> index()
    ---
    - error: Can not perform index build in a multi-statement transaction
    ...

The error is because an attempt to build an index on a non-empty
space leads to immediate yield.

When transactional DDL is introduced, 'DROP TABLE' will remove a
space and its cks in one transaction. At the moment of that
transaction commit the space is already removed from _space and
from space cache, and can't be accessed from other triggers.

This patch makes all space-related actions in on_replace.

Part of #4086

Vinyl writes a signature of each statement that created or dropped or
modified an LSM tree to vylog. This is needed to match xlog rows to
vylog records on recovery. So alter_space_commit() passes txn->signature
to index_commit_create(), index_commit_drop(), and index_commit_modify()
methods. If a transaction has multiple DDL statements, all of them will
have the same signature. Per se this is fine. The problem is that on
recovery we apply statements one-by-one, not transactionally, and so we
pass signatures assigned to individual statements, not to transactions.
As a result, while recovering a multi-statement DDL transaction vinyl
gets different signatures for them instead of those that were used when
those statements were committed, which apparently breaks its assumptions
and leads to a crash or even a data loss.

To fix this issue, let's, rather than using txn->signature, make
alter_space_commit() pass the signature of the actual statement that
committed a DDL operation, which is pretty easy to compute given the
number of statements in a transaction at the time when the statement was
committed.

Fixes commit f266559b ("ddl: allow to execute non-yielding DDL
statements in transactions").

Closes #4350
Follow-up #4083

There are two reasons to disable support of tables with __tostring
metamethods as opts.headers values:

* It seems this feature is not much needed: a type convertion can be
  easily done on a user's end with explicit tostring() call if needed.
* It never did work since introduction in 2.1.1-311-g85e1d78bc ('httpc:
  add checking of headers in httpc:request'): __tostring presence was
  verified, but a function in the field was not invoked.

Now http client accepts only a Lua string as a header key or a value.

Closes #3679 (again).

A txn_stmt keeps a reference to the space it modifies. Memtx uses this
space reference to revert the statement on error or voluntary rollback
so the space must stay valid throughout the whole transaction.

The problem is a DML statement may be followed by a DDL statement that
modifies the target space in the same transaction. If we try to roll
it back before running the rollback triggers installed by the DDL
statement, it will access an invalid space object (e.g. missing an
index), which will result in a crash.

To fix this problem, let's run triggers installed by a statement right
after rolling back the statement.

Closes #4368

We must un-account index.stat.txw.rows not only when a whole transaction
is rolled back, but also when we undo statements using a savepoint.

After 2.2.0-390-ga7c855e5b ("net.box: fetch '_vcollation' sysview into
the module") net.box fetches _vcollation view unconditionally, while the
view was added in 2.2.0-389-g3e3ef182f and, say, tarantool-1.10 and
tarantool-2.1 do not have it. This leads to a runtime error "Space '277'
does not exist" on a newer client that connects to an older server.

Now the view is fetched conditionally depending of a version of a
server: if it is above 2.2.1, then net.box will fetch it. Note: at the
time there are no release with a number above 2.2.1.

When _vcollation view is available, a collation in an index part will be
shown by its name (with 'collation' field), otherwise it will be shown
by its ID (in 'collation_id' field). For example:

Connect to tarantool 1.10:

 | tarantool> connection = require('net.box').connect('localhost:3301')
 | ---
 | ...
 |
 | tarantool> connection.space.s.index.sk.parts
 | ---
 | - - type: string
 |     is_nullable: false
 |     collation_id: 2
 |     fieldno: 2
 | ...

Connect to tarantool 2.2.1 (when it will be released):

 | tarantool> connection = require('net.box').connect('localhost:3301')
 | ---
 | ...
 |
 | tarantool> connection.space.s.index.sk.parts
 | ---
 | - - type: string
 |     is_nullable: false
 |     collation: unicode_ci
 |     fieldno: 2
 | ...

Fixes #4307.

Before this patch it was allowed to specify REAL/FLOAT/DOUBLE types
which matched with NUMBER NoSQL type in space format. However, NUMBER is
different from standard floating point types, since it is able to hold
integers in range [-2^63; 2^64-1] alongside with double precision
floating point values. Hence, to not confuse users it has been decided
to remove support of REAL/FLOAT/DOUBLE types from SQL grammar and use
instead original NUMBER type naming.

TEXT type is called "string" in the original Tarantool NoSQL, so it would
be rational to allow using the same type name in SQL.

When reverting to a savepoint inside a DDL transaction, apart from
undoing changes done by the DDL statements to the system spaces, we also
have to

 - Run rollback triggers installed after the savepoint was set, because
   otherwise changes done to the schema by DDL won't be undone.
 - Remove commit triggers installed after the savepoint, because they
   are not relevant anymore, apparently.

To achieve that let's append DDL triggers right to txn statements.
This allows us to easily discard commit triggers and run rollback
triggers when a statement is rolled back.

Note, txn commit/rollback triggers are not removed, because they are
still used by applier and Lua box.on_commit/on_rollback functions.

Closes #4364
Closes #4365

It's not used anywhere anymore.

txn_stmt_new is supposed to simply allocate an initialize a new txn_stmt
struct. Adding the new statement to the txn's statement list and setting
up a savepoint for rollback looks confusing. Move it to txn_begin_stmt.