This patch removes zeroblob optimization from SQL code. This
optimization complicates the code, and there is almost no profit from
it.

Closes #6113
Needed for #4145

*.inprogress files shouldn't be deleted during normal GC, because they
may be in use. E.g. GC may happen to run while WAL is rotating xlog;
if GC removes the transient xlog.inprogress file created by WAL (see
xdir_create_xlog), WAL will fail to rename it and abort the current
transaction.

Initially, inprogress files were removed only after recovery. For this
reason, xdir_collect_inprogress, which is used for deleting inprogress
files, accesses FS directly, without COIO. This was changed by commit
5aa243de ("recovery: build secondary
index in hot standby mode") which moved xdir_collect_inprogress from
memtx_engine_end_recovery to memtx_engine_collect_garbage so that a hot
standby instance doesn't delete inprogress files of the master instance
by mistake.

To fix this issue, let's move xdir_collect_inprogress back where it
belongs, to engine_end_recovery, and introduce a new callback for memtx
to build its secondary keys before entering the hot standby mode -
engine_begin_hot_standby.

Let's also remove engine_collect_garbage from gc_init, which was added
there by the aforementioned commit, probably to make tests pass.

The bug was reported by the vinyl/deferred_delete test (#5089).

Closes #6554

Quorum.test.lua was migrated to luatest.
It was divided into several tests.

Delete quorum.test.lua, fix suite.ini in replication tests.

Part of tarantool/test-run#304

Use luatest for checking correctness behavioral for encode/decode
functions

Delete msgpack.test.lua.

Part of tarantool/test-run#304

The commit includes a few helpers files for luatest infrastructure.
'Server' based on luatest Server with additional functionality like
waiting Server for start, create Server's own env including box.cfg
provided by test.

Servers could be built and added in a cluster (cluster.lua file).

Asserts.lua is additional checks, which are useful for many tests.

Helpers are supportive function, could be also used for many tests.

Part of tarantool/test-run#304

Same as for local transactions, timeout for iproto transactions
was implemented. If timeout is not specified in client request
it's sets to box.cfg.txn_timeout, which specified on server side.

Closes #6177

@TarantoolBot document
Title: ability to set timeout for iproto transactions was implemented
A new `IPROTO_TIMEOUT 0x56` key has been added. Currently it is used to
set a timeout for transactions over iproto streams. It is stored in the
body of 'IPROTO_BEGIN' request. If user want's to specify timeout using
netbox (3s for example), he should use 'stream:begin({timeout = 3}).

Client code errors or manual mistakes can create transactions that are
never closed. Such transaction will work as a memory leak. Implement
timeout for transactions after which they are rolled back.

Part of #6177

@TarantoolBot document
Title: ability to set timeout for transactions was implemented
Previously transactions are never closed until commit or rollback.
Timeout for transactions was implemented after which they are rolled
back. For these purpose, in `box.begin` the optional table parameter
was added. For example if user want to start transaction with timeout
3s, he should use `box.begin({timeout = 3})`. Also was implement new
configuration option `box.cfg.txn_timeout` which determines timeout for
transactions, for which the timeout was not explicitly set. By default
this option is set to infinity (TIMEOUT_INFINITY = 365 * 100 * 86400).
Also in C API was added new function to set timeout for transaction -
'box_txn_set_timeout'.

New function box.txn_id(), which returns the id of the current transaction if
called within a transaction, nil otherwise.

Closes #6396

@TarantoolBot document
Title: new function box.txn_id()
New function in module box:  box.txn_id(), which returns the id of the current
transaction if called within a transaction, nil otherwise.

The maximal unix socket file length is 108 symbols on Linux, so
we should decrease file name to fit in this size.

The session can be closed while kharon is travelling between tx and
iproto. If this happens, kharon shouldn't go back to iproto when it
returns to tx, even if there are pending pushes, because the connection
is going to be freed soon.

Thanks to @Gerold103 for the simple fix.

Closes #6520

This patch removes the DECIMAL truncation in LIMIT and OFFSET, because
according to the implicit casting rules, DECIMAL with digits after the
decimal point cannot be implicitly cast to INTEGER.

Closes #6485

This patch fixes an assertion when casting DECIMAL value less than 0 and
greater than -1 to INTEGER.

Part of #6485

In case the DECIMAL value is implicitly cast to INTEGER during a search
using an index, it was possible that DECIMAL would be truncated, which
is not correct according to the implicit cast rules. This patch removes
this truncation.

Part of #6485

In case user enters invalid listen address, tarantool closes previous listen
socket, but bind on invalid address fails also, so tarantool becames blind -
no listening socket at all. This patch fixed this behaviour, now tarantool
still listen old listen address.

Closes #6092

There was access to previously freed memory in case when `cbus_call`
is interrupted: `cbus_call_msg` in iproto allocates on stack, and if
`cbus_call` failed due to fiber cancelation or wake up, `cbus_call_msg`
memory is released. But function called through cbus is still work in
iproto thread and there will be an attempt to access this memory when
this function in iproto thread finished it's work. This patch rework
this behaviour, now before `cbus_call` we reset FIBER_IS_CANCELLABLE
flag, to prevent fiber cancellation or it's wake up.

Closes #6480

For the purposes of format support in datetime we need to modify
standard strftime() implementation so it will be accepting %f flag
we want to use for displaying of fractional part of seconds.

Used Olson' strftime() implementation, simplified their code and
header file, and adapted to work with our `struct datetime` data
structure.
We store timezone information there, seconds since epoch, and
nanoseconds, thus we modified a way how those have being retrieved
in the original implementation.

We have also added missing `%f` and width modifiers support.

```
tarantool> T:format('%d')
---
- '14'
...

tarantool> T:format('%3d')
---
- 3d
...
tarantool> T:format('%3f')
---
- 371
...
tarantool> T:format()
---
- 2021-09-14T12:10:30.371895+0300
...
tarantool> T:format('%FT%T.%f%z')
---
- 2021-09-14T12:10:30.371895+0300
...
```

Created detailed strftime formats test to cover all of known
format flags.

Part of #5941

Introduce a new builtin Tarantool module `datetime.lua` for timestamp
and interval types support.

New third_party module - c-dt
-----------------------------

* Integrated chansen/c-dt parser as 3rd party module to the
  Tarantool cmake build process;
* We use tarantool/c-dt instead of original chansen/c-dt to
  have an easier cmake build integration, as we have added some
  changes, which provide cmake support, and allow to rename symbols
  if necessary (this symbol renaming is similar to that we see
  with xxhash or icu).

New built-in module `datetime`
------------------------------

* created a new Tarantool built-in module `datetime`, which uses
  `struct datetime` data structure for keeping timestamp values;
* Lua module uses a number of `dt_*` functions from `c-dt` library,
  but they were renamed to `tnt_dt_*` at the moment of exporting
  from executable - to avoid possible name clashes with external
  libraries.

* At the moment we libc `strftime` for formatting of datetime
  values according to flags passed, i.e. `date:format('%FT%T%z')`
  will return something like '1970-01-01T00:00:00+0000', but
  `date:format('%A %d, %B %Y')` will return 'Thursday 01, January 1970'

* if there is no format provided then we use default
  `tnt_datetime_to_string()` function, which converts datetime
  to their default ISO-8601 output format, i.e.
  `tostring(date)` will return string like "1970-01-01T00:00:00Z"

* There are a number of simplified interfaces
  - totable() for exporting table with attributes names as provided
    by `os.date('*t')`
  - set() method provides unified interface to set values using
    the set of attributes as defined above in totable()

Example,

```
local dt = datetime.new {
	nsec      = 123456789,

	sec       = 19,
	min       = 29,
	hour      = 18,

	day       = 20,
	month     = 8,
	year      = 2021,

	tzoffset  = 180
}

local t = dt:totable()
--[[
{
	sec = 19,
	min = 29,
	wday = 6,
	day = 20,
	nsec = 123456789,
	isdst = false,
	yday = 232,
	tzoffset = 180,
	month = 8,
	year = 2021,
	hour = 18
}
--]]

dt:format()   -- 2021-08-21T14:53:34.032Z
dt:format('%Y-%m-%dT%H:%M:%S')   -- 2021-08-21T14:53:34

dt:set {
	usec      = 123456,

	sec       = 19,
	min       = 29,
	hour      = 18,

	day       = 20,
	month     = 8,
	year      = 2021,

	tzoffset  = 180,

}
dt:set {
	timestamp = 1629476485.124,

	tzoffset  = 180,
}

```

Coverage is

File                 Hits Missed Coverage
-----------------------------------------
builtin/datetime.lua 299  23     92.86%
-----------------------------------------
Total                299  23     92.86%

Part of #5941

@TarantoolBot document
Title: Introduced a new `datetime` module for timestamp and interval support

Create `datetime` module for timestamp and interval types support.
It allows to create date and timestamp values using either object interface,
or via parsing of string values conforming to iso-8601 standard.
One may manipulate (modify, subtract or add) timestamp and interval values.

Please refer to https://hackmd.io/@Mons/S1Vfc_axK#Datetime-in-Tarantool
for a more detailed description of module API.

Fixes incorrect handling of variable number of arguments in box.func:call().

Closes #6405

There was a bug that led to dirty read after space alter. For the
simplicity sake imagine following setup:

-- space 's' is empty
tx1:begin()
tx1('s:replace{2}')
s:alter({format = format})
s:select{}

Last select returns tuple {2}, however transaction tx1 hasn't been
committed. This happens due to the fact that during alter operation we
create new space, swap all unchanged parts of old space and then delete
old space. During removal of old space we also clean-up all stories
related to it. In turn story destruction may make dirty tuple clean in
case it remains in the index. In the previous implementation there was
no removal of uncommitted tuples from corresponding indexes. So let's
rollback all changes happened to the space right in time of alter. It is
legal since DDL operation anyway aborts ALL other transactions.

Closes #6318
Closes #6263

Prior to this patch, an assertion was thrown if a tuple with an invalid
id was inserted into the _priv system space. This bug appeared only in
the debug build.

Closes #6295

Implement ability to pass timeout to 'fiber:join' function.
If timeout expired, join fails with 'timed out' error.

Closes #6203

@TarantoolBot document
Title: ability to set timeout for 'fiber:join' function was implemented
Implement ability to pass timeout to 'fiber:join' function.
If timeout expired, join fails with 'timed out' error.

Add new  metrics `REQUESTS_IN_PROGRESS` and `REQUESTS_IN_STREAM_QUEUE`
to `box.stat.net`, which contain detailed statistics for iproto requests.
These metrics contains same counters as other metrics in `box.stat.net`:
current, rps and total.

Part of #6293

@TarantoolBot document
Title: detailed iproto requests statistics was implemented
Add new  metrics `REQUESTS_IN_PROGRESS` and `REQUESTS_IN_STREAM_QUEUE`
to `box.stat.net`, which contain detailed statistics for iproto requests.
These metrics contains same counters as other metrics in `box.stat.net`:
current, rps and total.
```
-- statistics for requests currently being processed in tx thread.
REQUESTS_IN_PROGRESS:
current: -- count of requests currently being processed in the tx thread
rps: -- count of requests processed by the tx thread per second
total: -- total count of requests processed by tx thread

-- statistics for requests placed in queues of streams.
REQUESTS_IN_STREAM_QUEUE:
current: -- count of requests currently waiting in queues of streams
rps: -- count of requests placed in streams queues per second
total: -- total count of requests, which was placed in queues of streams
          for all time
```

It never returns NULL anymore, because it uses xmalloc for memory
allocations.

An mhash is used for allocating system objects. Failing to grow it is
likely to render the Tarantool instance unusuable so better fail early.

Checks of mh(new) and mh(put) return value will be removed in follow-up
patches.

We want to use the xmalloc helper throughout the code, not only in
the core lib. Move its definition to trivia/util.h and use fprintf+exit
instead of say/panic in order not to create circular dependencies.

Exporting symbols of a third party library is not a best practice,
as we know from [1]. Let's wrap the msgpack symbols that need to
be exported with the "tnt_" prefix.

While working on the patch, it was decided to export the msgpack
symbols that are used in "msgpuckffi.lua".
In test shared libraries where the symbols "mp_***_{decimal,uuid}"
are used, they are replaced to exported "tnt_mp_***_{decimal,uuid}",
because in the case of linking with "libcore.a" the "libcore.a"
needs to be rebuild with the "-fPIC" flag, that seems as overkill
for tests.

1. https://github.com/tarantool/memcached/issues/59

Closes #5932

If we insert a tuple in space with an index that is being built in background,
new tuple will or will not be inserted into new index depending on the result of
lexicographical comparison with tuple which was inserted into new index last.
The problem is hash index is unordered, so background build will not work properly
if primary key is HASH index.

To avoid this, disable building index in background if primary index is hash.

Closes #5977

The test uses error injection.

Follow-up 0428bbce ("vinyl: fix use of
dropped space in deferred DELETE handler").

For deferred DELETE statements to be recovered after restart, we write
them to a special 'blackhole' system space, _vinyl_deferred_delete,
which doesn't store any data, but is logged in the WAL, as a normal
space. In the on_replace trigger installed for this space, we insert
deferred DELETE statements into the memory (L0) level of the LSM tree
corresponding to the space for which the statement was generated. We
also wait for L0 quota in the trigger. The problem is a space can be
dropped while we are waiting for quota, in which case the trigger
function will crash once it resumes execution.

To fix this, let's wait for quota before we write the information about
the deferred DELETE statement to the _vinyl_deferred_delete space and
check if the LSM tree was dropped after yield. This way, everything will
work as expected even if a new space is created with the same id,
because we don't yield after checking quota.

Closes #6448

Will come in handy for testing #6448.

@TarantoolBot document
Title: Document box.stat.vinyl().regulator.blocked_writers

The new stat counter shows the number of fibers that are blocked waiting
for Vinyl level0 memory quota.

Use a special luaL_serializer with encode_error_as_ext disabled.
Default options are propagated to it via an update trigger.

It's not needed now, because error marshaling is enabled automatically
if the connector supports it (IPROTO_FEATURE_ERROR_EXTENSION is set in
IPROTO_ID features).

Closes #6428

@TarantoolBot document
Title: Drop error_marshaling_enabled session setting

box.session.setting.error_marshaling_enabled was used to enable encoding
errors returned by CALL/EVAL in the extended format (as the MP_ERROR
MsgPack extension). Now, the feature is enabled automatically if the
connector supports it (sets IPROTO_FEATURE_ERROR_EXTENSION in IPROTO_ID
features).

The new IPROTO protocol feature IPROTO_FEATURE_ERROR_EXTENSION enables
encoding errors returned by CALL/EVAL commands as the MP_ERROR MsgPack
extension. Note, the MP_ERROR extension can still be disabled globally
by setting msgpack.cfg.encode_error_as_ext to false. If an IPROTO client
doesn't set the feature bit, errors will be encoded as generic cdata
objects (converted to strings by default).

Needed for #6428

@TarantoolBot document
Title: Document IPROTO_FEATURE_ERROR_EXTENSION

A new feature bit for the IPROTO_ID command was added:

```
IPROTO_FEATURE_ERROR_EXTENSION = 2
```

The protocol version was incremented - now it equals 2.

If a network client sets this bit, errors returned by CALL/EVAL will be
encoded as the MP_ERROR MsgPack extension (unless disabled globally by
msgpack.cfg.encode_error_as_ext). If the bit is unset, errors will be
encoded according to the serialization rules used for generic cdata
objects (converted to strings by default).

The built-in net.box connector sets this feature bit.

The server sets this feature bit if it supports the MP_ERROR MsgPack
extension so a net.box client can explicitly request the feature upon
connecting to a server:

```lua
net.box.connect(uri, {required_protocol_features = {'error_extension'}})
```

This patch adds a new msgpack.cfg: encode_error_as_ext. Setting it makes
msgpack and msgpackffi modules encode errors as the MP_ERROR msgpack
extension. If the flag is unset, msgpack.encode behavior depends on
encode_load_metatables, encode_use_tostring, and encode_invalid_as_nil
options, see luaL_convertfield(), while msgpackffi.encode() will always
encode errors as strings. The latter needs to be fixed, but it's out of
the scope of this work and tracked separately, see #4499.

The new option is enabled by default.

Interaction with box.session.settings.error_marshaling_enabled: errors
are encoded as the MP_ERROR msgpack extension when returned via IPROTO
iff both error_marshaling_enabled and encode_error_as_ext are set.

Closes #6433

@TarantoolBot document
Title: Document msgpack.cfg.encode_error_as_ext

The new option determines how error objects (see box.error.new) are
encoded in the msgpack format:
 - If it's set, errors are encoded as the MP_ERROR msgpack extension.
   This is the default behavior.
 - If it's unset, the encoded format depends on other msgpack
   configuration options (encode_load_metatables, encode_use_tostring,
   encode_invalid_as_nil). With the otherwise default configuration,
   they are encoded as strings (see error.message).

Functions affected by the default configuration (msgpack.cfg):
 - msgpack and msgpackffi modules
 - Storing errors in tuples and spaces (box.tuple.new)
 - Returning errors from IPROTO CALL/EVAL

There's no reason not to enable extended errors for CALL_16.
Enable the feature and add a test.

Needed for #6433

There's no way to encode an error yet so the test just hard-codes
msgpack data. It will be fixed in the future, once we allow to encode
errors with msgpack/msgpackffi.

Needed for #6433

It's not used anymore.

Whether errors are encoded as a msgpack extension or not is determined
by the serializer_opts::error_marshaling_enabled flag. Although an
instance of serizlier_opts is passed to luamp_encode(), it doesn't
propagate it to luamp_encode_extension_box(). The latter encodes an
error as a msgpack extension if the error_marshaling_enabled flag is set
in serializer_opts of the current session. This leads to a bug when
luamp_encode() is called with error_marshaling_enabled unset while the
current session has the flag set:

 1. luaL_tofield() sets field->type to MP_EXT and field->ext_type to
    MP_UNKNOWN_EXTENSION, because the error_marshaling_enabled flag is
    unset:

    https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/lua/serializer.c#L548

 2. Basing on the ext_type, luamp_encode_r() skips the MP_ERROR
    swtich-case branch for the default branch and calls the
    luamp_encode_extension callback:

    https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/lua/msgpack.c#L203

 3. The callback implementation (luamp_encode_extension_box()) encodes
    the error, because the error_marshaling_enabled flag is set in the
    current session settings, and returns MP_EXT:

    https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/box/lua/init.c#L420

 4. luamp_encode_r() assumes that the callback didn't encode the
    extension, because it returned MP_EXT, and encodes it again as
    a string:

    https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/lua/msgpack.c#L209

This results in a broken msgpack content.

To fix this bug, let's do the following:
 - luaL_tofield() now sets ext_type to MP_ERROR unconditionally,
   irrespective of serializer_opts::error_marshaling_enabled.
 - luamp_encode_r() invokes the luamp_encode_extension callback for
   a MP_ERROR field only if error_marshaling_enabled is set. If the flag
   is unset, it proceeds with converting the field to string.
 - luamp_encode_extension_box() doesn't check serializer_opts anymore.
   It doesn't need to, because it's called iff error_marshaling_enabled
   is set.
 - YAML and JSON encoders are patched to handle the MP_ERROR field type
   by appending error::errmsg to the output (they use luaL_tofield()
   internally to determine the field type so they have to handle
   MP_ERROR).

This basically disables error encoding as msgpack extension
everywhere except returning an error from a Lua CALL/EVAL, in
particular:
 - when creating a tuple with box.tuple.new(),
 - when inserting an error into a space,
 - when encoding an error with the msgpack module.

This is okay, because the functionality has always been broken anyway.
We will introduce a separate msgpack encoder option to enable encoding
errors as MP_ERROR msgpack extension.

Looking at the code links above, one is likely to wonder why error
encoding was implemented via the encode extension callback in the first
place. The lua/msgpack module knows about the MP_ERROR extension and
even partially handles it so it'd be only natural to call the error
encoder function directly, as we do with decimals and uuids.
Unfortunately, we can't do it, because the error encoder is (surprise!)
a part of the box library. I filed a ticket to move it to the core lib,
see #6432.

Closes #6431

Now net.box sends IPROTO_ID request on (re)connect to query features
supported by the server and report its own features. The version and
features reported by the server are checked against the new connection
options:

 - required_protocol_version - min version (unsigned)
 - required_protocol_features - required features (array of strings)

If the server version is older than specified or the server lacks
certain features, the connection will fail.

Features supported by the server are stored in the peer_protocol_version
and peer_protocol_features fields of a connection.

Closes #6253

@TarantoolBot document
Title: Document required_protocol_version/features net.box options

Two new options can now be passed to net.box.connect():

 - required_protocol_version: min IPROTO protocol version that must be
   supported by the server. Type: unsigned integer.
 - required_protocol_features: array of IPROTO protocol features that
   must be supported by the server. Type: array of strings.

If the server version is less than the specified or the server lacks
certain features, the connection will fail with the corresponding error.

Querying server features is implemented via the IPROTO_ID command.
Currently, there are two features defined: streams and transactions.

Irrespective of the options used, the actual version and features are
reported via peer_protocol_version and peer_protocol_features fields of
the connection. Example:

```
tarantool> require('net.box').connect(3301, {
         > required_protocol_version = 1,
         > required_protocol_features = {'transactions'},
         > })
---
- peer_protocol_version: 1
  peer_uuid: 7a8cfdbd-6bbc-4d10-99e5-cbbd06a2382f
  opts:
    required_protocol_version: 1
    required_protocol_features:
    - transactions
  peer_protocol_features:
    transactions: true
    streams: true
  schema_version: 80
  protocol: Binary
  state: active
  peer_version_id: 133632
  port: '3301'
...
```

The new request can be used by a client to let the server know about
supported IPROTO protocol features. The request body contains two
fields: IPROTO protocol version and supported IPROTO protocol features.
In reply to a IPROTO_ID request, the server sends its own protocol
version and supported features.

Currently, the actual protocol version is 1 and there are two features
defined which are always set - streams and transactions.

Part of #6253

@TarantoolBot document
Title: Document IPROTO_ID request

The new request code is 73. It takes a map in the body with the
following keys:
 - IPROTO_VERSION (0x54) - protocol version (unsigned).
 - IPROTO_FEATURES (0x55) - array of protocol feature ids (unsigned).

A client (connector) can send this request to let the server know about
the protocol version and features it supports. The server may enable or
disable certain functionality basing on the features supported by the
client. In reply to the request, the server sends an IPROTO_OK response,
in the body of which it reports its own protocol version and features
(the format of the response body is the same as the request body).
The request doesn't need authentication to pass.

Currently, the actual protocol version is 1 and there are two features
defined:
 - IPROTO_FEATURE_STREAMS - streams support (IPROTO_STREAM_ID header
   key), id = 0.
 - IPROTO_FEATURE_TRANSACTIONS - transactions support (IPROTO_BEGIN,
   IPROTO_COMMIT, IPROTO_ROLLBACK commands), id = 1.