Speed up txn_is_distributed() check by keeping track of the number of
local rows of the transaction.

We need to keep count of different kind of rows in a transaction:
- rows which already exist in some WAL and are replayed locally.
  These used to be called n_remote_rows, and renamed to n_applier_rows
- rows which were created locally, on this server (previously called
  n_local_rows, renamed to n_new_rows).

Of the latter, we need to distinguish between GROUP_ID=LOCAL rows,
i.e. rows which need not be replicated, and GROUP_ID=REPLICA rows,
which need to be replicated. For example, a remote transaction can
fire local triggers which generate local rows. If these triggers
generate rows which need to be replicated, the transaction has to be
aborted.

In a subsequent patch I plan to add n_local_rows, which tracks the
number of new rows with GROUP_ID=local and use it in
txn_is_distributed() check.

Disallow changes for non-local spaces during replication stream
applying. As we do not support distributed transaction yet we could not
provide a transactional replication for such side effects if there are
not NOPed.

Needed for: #2798
Follow up for: 27283deb

The primary reason of this change is to keep compatibility of 1.10
release series with tarantool/stat-0.3.1, which expects that each
box.stat.net.<...> and box.stat.net().<...> item is a table.

This commit changes CONNECTIONS metric to be a table with 'current'
field, which in turn contains current number of connections.

Fixes #4039.

@TarantoolBot document
Title: box.stat.net.CONNECTIONS becomes a table

The format of box.stat.net.CONNECTIONS and box.stat.net().CONNECTIONS is
changed in order to keep all items being tables, because
tarantool/stat-0.3.1 expects them to be tables (see [1] for more
information).

Example of box.stat.net() **before** this commit:

```
tarantool> box.stat.net()
---
- SENT:
    total: 0
    rps: 0
  CONNECTIONS: 0
  RECEIVED:
    total: 0
    rps: 0
...
```

And after:

```
tarantool> box.stat.net()
---
- SENT:
    total: 0
    rps: 0
  CONNECTIONS:
    current: 0
  RECEIVED:
    total: 0
    rps: 0
...
```

Look at the comment to lbox_stat_net_call() (see the linked commit) for
meaning of total/rps/current fields.

[1]: https://github.com/tarantool/tarantool/issues/4039

We must use long long unsigned numbers on ffi types build to
avoid precision lost.

Follow-up
        f5721edc test: check hints corner cases

Closes #4080

Added necessary cleanup, because the test fails when both memtx and
vinyl configurations are run on the same test-run worker.

Aside of that disabled the test on vinyl, because it is engine-agnostic
and only checks iproto.

This is the fix for the commit 7676b2b1 ('sql: make SQL_BIND optional in
an iproto request').

Follows up #4077.

Applier fetch incoming rows to form a transaction and then apply it.
Rows are fetched and stored on fiber gc region until last transaction row
with is_commit was fetched. After fetch a multi row transaction is going to be
applied into txn_begin/txn_commit/txn_rolback boundaries. At this time
we could not apply single row transaction in such boundaries because of
ddl which does not support non auto commit transactions.

Closes: #2798
Needed for: #980

Introduced a new sql_normalize_name routine performing SQL name
conversion to case-normal form via unicode character folding.
For example, ß is converted to SS. The result is similar to SQL
UPPER function.

Closes #3931

Refactored sqlExpr routine as sql_expr_new and reworked it to set
diag message in case of memory allocation error. Also performed some
additional name refactoring in adjacent places.
This change is necessary because the sqlExpr body has a
sqlNormalizeName call that will be changed in subsequent patches.

After that patch there are basically 2 ways of errors
processing and forwarding:
     - Use diag only. It works for all the places out of
       src/box/sql, and for some functions inside it. For
       example, sql_expr_new();
     - Use global flags Parse.is_aborted, sql.mallocFailed.

It is easy to see, that some places use both of them
implicitly. For example, sql_expr_new() and every other
place which uses SQLite memory allocators + diag. But it
is ok until the former is removed. What is more important,
is that at least one of these two methods should be
consistent and finished in every functions. And match a
declared behaviour.

For example, it is incorrect to declare a function as setting
flags on an error, but in fact set diag only. Or vice versa,
that it throws a diag, but factually sets flags only.

Part of #3931

Refactored triggerSterAllocate routine as sql_trigger_step_new
and reworked it to set diag_set in case of memory allocation
error. Also performed some additional name refactoring in
adjacent places.
This change is necessary because the sql_trigger_step_allocate
body has a sqlNormalizeName call that will be changed in
subsequent patches.

Part of #3931

Refactored sqlNameFromToken routine as sql_name_from_token and
reworked it to use diag_set in case of memory allocation error.
This change is necessary because the sql_name_from_token body has
a sqlNormalizeName call that will be changed in subsequent
patches.

Part of #3931

Refactored sqlIdListAppend routine as sql_id_list_append and
reworked it to use diag_set in case of memory allocation error.
This change is necessary because the sql_id_list_append body has
a sqlNameFromToken call that will be changed in subsequent
patches.

This patch refers to a series of preparatory patches that provide
the use of Tarantool errors in the call tree that includes
sqlNormalizeName, since this call can later return errors.

This patch is not self-sufficient, its sqlNameFromToken call
remained to be non-Tarantool (for now). It means, that if
sqlNameFromToken fails in sql_id_list_append there is no
diag message created.

Part of #3931

Refactored sqlSrcListAppend routine as sql_src_list_append and
reworked it to use diag_set in case of memory allocation error.
This change is necessary because the sql_src_list_append body has
a sqlNameFromToken call that will be changed in subsequent
patches.

This patch refers to a series of preparatory patches that provide
the use of Tarantool errors in the call tree that includes
sqlNormalizeName, since this call can later return errors.

This patch is not self-sufficient, its sqlNameFromToken call
remained to be non-Tarantool (for now). It means, that if
sqlNameFromToken fails in sql_src_list_append there is no
diag message created.

Part of #3931

Refactored sqlSrcListEnlarge routine as sql_src_list_enlarge and
reworked to use diag_set in case of memory allocation error. This
will ensure that the sqlSrcListAppend function throws an error
using diag in subsequent patches.

This patch refers to a series of preparatory patches that provide
the use of Tarantool errors in the call tree that includes
sqlNormalizeName, since this call can later return errors.

This patch is not self-sufficient, its usage in sqlSrcListAppend
remained to be non-Tarantool (for now). It means, that if
sql_src_list_enlarge fails in sqlSrcListAppend the diag will
never be thrown.

Part of #3931

Refactored sqlAllocSrcList routine as sql_src_list_new and
reworked it to use diag_set in case of memory allocation error.
This will ensure that the sqlSrcListAppend function throws an
error using diag in subsequent patches.

This patch refers to a series of preparatory patches that provide
the use of Tarantool errors in the call tree that includes
sqlNormalizeName, since this call can later return errors.

This patch is not self-sufficient, its usage in sqlSrcListAppend
remained to be non-Tarantool (for now). It means, that if
sql_src_list_new fails in sqlSrcListAppend and sets a diag, it is
never thrown to a user (now).

Part of #3931

We call it after each invocation of space_cache_replace() so better
fold it. Panic on error running triggers as space_cache_replace()
isn't supposed to fail. Remove empty on_create_space_commit() while
we are at it.

Follow-up commit 082cffca ("Synchronize lua schema update with
space cache").

In some cases(like foreign keys) the SQL code used
32-bit bit mask, while 64-bit bit masks will perform better
column optimizations. There was refactored code to work with 64b
bitmasks where required.
The 32b bitmasks are still used to specify constant OP_Function
arguments because this change would require changing the P1 type
of the VDBE p1 argument, which is not desirable. Moreover, the
64 function's arguments is an explicit overkill.

The ticket was created in connection with the introduction of
foreign keys and their use of 32-bit bit masks. In the rest of
the scripts in SQL already use 64 bit masks, and the "smart" bit
of the mask is not applicable.

Closes #3571

Allow single statement transactions within begin/commit in case of an
ddl operation instead of auto commit requirements. This is essential
for a transactional applier.

Needed for: #2798

The documentation [1] says this field is optional. I don't know which
commit lead to the regression, only that 2.1.1-7-gd381a45b6 is good.

[1]: https://tarantool.io/en/doc/2.1/dev_guide/internals/sql_protocol/

Fixes #4077.

This patch completely replaces sqlErrorMsg() with diag_set() and
removes sqlErrorMsg().

Closes #3965
Closes #3036

This patch reworks some of SQL semantic errors.

Part of #3965

Stop a file processing loop only when it is guaranteed that we will not
find a record that match user-provided filters later in this file. If
--replica R is provided one time and we're meet a record from R with a
LSN equal or above of a --to value, we'll stop the loop. Otherwise (no
--replica, several --replica arguments) a file will be read until an end
even if --to is provided.

Fixes #3827.

Exposed this unified code (filter_xlog() function) and wrote a unit
test.

Allow to run app-tap/tarantoolctl.test.lua w/o test-run.

Needed for #3827.

Update lua schema as soon as space cache replace was done instead of
doing this while on_commit trigger executes. In opposite then case
schema changes would not be visible until commit was finished.

Needed for: #2798

It's OK to use json format with the boot logger. As for syslog, let's
add a check to Lua's log_format() so that it fails gracefully rather
than raising an assertion.

Closes #3946

Abort a new index creation if truncate couldn't be finished because of
rollback or an error. Without this vinyl fails because of internal
scheduler assertion.

Needed for: 2798

We must to enshure that hints don't broke tuple comparison.
As different field_types use different routines to calculate hint,
we must to ensure that the calculated values are compatible with
each other.

Also tested the corner cases for large double numbers, due to the
fact that there may be additional difficulties in hint construction.

Follow-up
    9fba29ab memtx: introduce tuple compare hint

Allow to call box.session.exists() and box.session.fd() without
any arguments. In such case, current session is used.
The box.session.peer() already support such feature, so we need
it to be consistent.

Closes #4021

The test used to write big rows (20 mb in size), so when run in parallel
mode, it put high load on the disk and processor, which made appliers
time out multiple times during read, and caused the test to fail
occasionally.
So, instead of writing huge rows in test, introduce a new error
injection restricting sio from reading more than a couple of bytes per
request. This ensures that the test is still relevant and makes it a lot
more lightweight.

Closes #4062

Tarantool could not start from the snapshot created by version
2.1.0 because the new version 2.1.1 does not support the
index.opts.sql index opt and stops the execution.
Introduced a special state OPT_DEF_LEGACY macro to ignore legacy
options and introduced migration code in upgrade.lua.

At this point, an ANALYZE statement can lead to many problems. It
was decided to temporarily ban this statement.

Closes #4069

Close #4050

Debian Wheezy is EOL since 31st May 2018, mirrors are disabled now.

Development branch is now named master, so
update conditions for extended extended checks on
Travis.
Also, set new repo name.

There's a bug in the code building index hash on recovery: we replace
a dropped index with any newer index, even incomplete one. Apparently,
this is wrong, because a dropped index may have been dropped during
final recovery and hence is still needed for initial recovery. If we
replace it with an incomplete index in the index hash, initial recovery
will fail with

  ER_INVALID_VYLOG_FILE: Invalid VYLOG file: LSM tree 512/1 not found

(see vy_lsm_recover()).

Fix this problem by checking create_lsn of the index that is going to
replace a dropped one - if it's negative, we must link it to the dropped
index via vy_lsm_recovery_info->prepared instead of inserting it into
the hash directly.

Closes #4066

Perform a log rotation instead of a termination according to
the documentation. Fix regression caused by 9f99bc62

Follow up for: 9f99bc62
Closes: #4063

Previously, all tuples created with frommap() used default format
table_format_runtime, and therefore the created data tuples were
not checked for the ability to be inserted into the target space.

Moreover frommap(...):tomap(...) procedures sequence also did not
work because tomap(..) routine assumes that the tuple has format
with field names.

Closes #4045

Implement functions for retrieving tuple hints for a particular
key_def. Hint is an integer that can be used for tuple comparison
optimization: if a hint of one tuple is less than a hint of another
then the first tuple is definitely less than the second; only if
hints are equal tuple_compare must be called for getting comparison
result. Hints are calculated using only the first part of key_def.

@locker:
 - Rework key_def_set_hint_func.
 - Refactor functions calculating hints.
 - Drop has_collation template argument (we don't use templates
   for collations anywhere else).
 - Add part_count argument to key_hint (it's conventional to pass
   part_count along with decoded key).
 - Improve comments, rename a few functions, and cleanup code.

Close #3961