Will come in handy for testing #6448.

@TarantoolBot document
Title: Document box.stat.vinyl().regulator.blocked_writers

The new stat counter shows the number of fibers that are blocked waiting
for Vinyl level0 memory quota.

The struct is empty.

Use a special luaL_serializer with encode_error_as_ext disabled.
Default options are propagated to it via an update trigger.

This is a preparation for merging serializer_opts into luaL_serializer.

It's not needed now, because error marshaling is enabled automatically
if the connector supports it (IPROTO_FEATURE_ERROR_EXTENSION is set in
IPROTO_ID features).

Closes #6428

@TarantoolBot document
Title: Drop error_marshaling_enabled session setting

box.session.setting.error_marshaling_enabled was used to enable encoding
errors returned by CALL/EVAL in the extended format (as the MP_ERROR
MsgPack extension). Now, the feature is enabled automatically if the
connector supports it (sets IPROTO_FEATURE_ERROR_EXTENSION in IPROTO_ID
features).

The new IPROTO protocol feature IPROTO_FEATURE_ERROR_EXTENSION enables
encoding errors returned by CALL/EVAL commands as the MP_ERROR MsgPack
extension. Note, the MP_ERROR extension can still be disabled globally
by setting msgpack.cfg.encode_error_as_ext to false. If an IPROTO client
doesn't set the feature bit, errors will be encoded as generic cdata
objects (converted to strings by default).

Needed for #6428

@TarantoolBot document
Title: Document IPROTO_FEATURE_ERROR_EXTENSION

A new feature bit for the IPROTO_ID command was added:

```
IPROTO_FEATURE_ERROR_EXTENSION = 2
```

The protocol version was incremented - now it equals 2.

If a network client sets this bit, errors returned by CALL/EVAL will be
encoded as the MP_ERROR MsgPack extension (unless disabled globally by
msgpack.cfg.encode_error_as_ext). If the bit is unset, errors will be
encoded according to the serialization rules used for generic cdata
objects (converted to strings by default).

The built-in net.box connector sets this feature bit.

The server sets this feature bit if it supports the MP_ERROR MsgPack
extension so a net.box client can explicitly request the feature upon
connecting to a server:

```lua
net.box.connect(uri, {required_protocol_features = {'error_extension'}})
```

This patch adds a new msgpack.cfg: encode_error_as_ext. Setting it makes
msgpack and msgpackffi modules encode errors as the MP_ERROR msgpack
extension. If the flag is unset, msgpack.encode behavior depends on
encode_load_metatables, encode_use_tostring, and encode_invalid_as_nil
options, see luaL_convertfield(), while msgpackffi.encode() will always
encode errors as strings. The latter needs to be fixed, but it's out of
the scope of this work and tracked separately, see #4499.

The new option is enabled by default.

Interaction with box.session.settings.error_marshaling_enabled: errors
are encoded as the MP_ERROR msgpack extension when returned via IPROTO
iff both error_marshaling_enabled and encode_error_as_ext are set.

Closes #6433

@TarantoolBot document
Title: Document msgpack.cfg.encode_error_as_ext

The new option determines how error objects (see box.error.new) are
encoded in the msgpack format:
 - If it's set, errors are encoded as the MP_ERROR msgpack extension.
   This is the default behavior.
 - If it's unset, the encoded format depends on other msgpack
   configuration options (encode_load_metatables, encode_use_tostring,
   encode_invalid_as_nil). With the otherwise default configuration,
   they are encoded as strings (see error.message).

Functions affected by the default configuration (msgpack.cfg):
 - msgpack and msgpackffi modules
 - Storing errors in tuples and spaces (box.tuple.new)
 - Returning errors from IPROTO CALL/EVAL

error_to_mpstream and error_to_mpstream_noext have a lot in common.
Factor it out into helper functions.

The function doesn't iterate over the stack of errors so it should have
suffix '_one', like mp_encode_error_one.

It's customary for mp_encode_* functions to take a pointer to the buffer
and return a pointer following the end of encoded data.
mp_encode_error_one takes (char **) and advances it. Fix it to be
consistent with other mp_encode_* functions.

There's no reason not to enable extended errors for CALL_16.
Enable the feature and add a test.

Needed for #6433

There's no way to encode an error yet so the test just hard-codes
msgpack data. It will be fixed in the future, once we allow to encode
errors with msgpack/msgpackffi.

Needed for #6433

It's not used anymore.

Whether errors are encoded as a msgpack extension or not is determined
by the serializer_opts::error_marshaling_enabled flag. Although an
instance of serizlier_opts is passed to luamp_encode(), it doesn't
propagate it to luamp_encode_extension_box(). The latter encodes an
error as a msgpack extension if the error_marshaling_enabled flag is set
in serializer_opts of the current session. This leads to a bug when
luamp_encode() is called with error_marshaling_enabled unset while the
current session has the flag set:

 1. luaL_tofield() sets field->type to MP_EXT and field->ext_type to
    MP_UNKNOWN_EXTENSION, because the error_marshaling_enabled flag is
    unset:

    https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/lua/serializer.c#L548

 2. Basing on the ext_type, luamp_encode_r() skips the MP_ERROR
    swtich-case branch for the default branch and calls the
    luamp_encode_extension callback:

    https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/lua/msgpack.c#L203

 3. The callback implementation (luamp_encode_extension_box()) encodes
    the error, because the error_marshaling_enabled flag is set in the
    current session settings, and returns MP_EXT:

    https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/box/lua/init.c#L420

 4. luamp_encode_r() assumes that the callback didn't encode the
    extension, because it returned MP_EXT, and encodes it again as
    a string:

    https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/lua/msgpack.c#L209

This results in a broken msgpack content.

To fix this bug, let's do the following:
 - luaL_tofield() now sets ext_type to MP_ERROR unconditionally,
   irrespective of serializer_opts::error_marshaling_enabled.
 - luamp_encode_r() invokes the luamp_encode_extension callback for
   a MP_ERROR field only if error_marshaling_enabled is set. If the flag
   is unset, it proceeds with converting the field to string.
 - luamp_encode_extension_box() doesn't check serializer_opts anymore.
   It doesn't need to, because it's called iff error_marshaling_enabled
   is set.
 - YAML and JSON encoders are patched to handle the MP_ERROR field type
   by appending error::errmsg to the output (they use luaL_tofield()
   internally to determine the field type so they have to handle
   MP_ERROR).

This basically disables error encoding as msgpack extension
everywhere except returning an error from a Lua CALL/EVAL, in
particular:
 - when creating a tuple with box.tuple.new(),
 - when inserting an error into a space,
 - when encoding an error with the msgpack module.

This is okay, because the functionality has always been broken anyway.
We will introduce a separate msgpack encoder option to enable encoding
errors as MP_ERROR msgpack extension.

Looking at the code links above, one is likely to wonder why error
encoding was implemented via the encode extension callback in the first
place. The lua/msgpack module knows about the MP_ERROR extension and
even partially handles it so it'd be only natural to call the error
encoder function directly, as we do with decimals and uuids.
Unfortunately, we can't do it, because the error encoder is (surprise!)
a part of the box library. I filed a ticket to move it to the core lib,
see #6432.

Closes #6431

Now net.box sends IPROTO_ID request on (re)connect to query features
supported by the server and report its own features. The version and
features reported by the server are checked against the new connection
options:

 - required_protocol_version - min version (unsigned)
 - required_protocol_features - required features (array of strings)

If the server version is older than specified or the server lacks
certain features, the connection will fail.

Features supported by the server are stored in the peer_protocol_version
and peer_protocol_features fields of a connection.

Closes #6253

@TarantoolBot document
Title: Document required_protocol_version/features net.box options

Two new options can now be passed to net.box.connect():

 - required_protocol_version: min IPROTO protocol version that must be
   supported by the server. Type: unsigned integer.
 - required_protocol_features: array of IPROTO protocol features that
   must be supported by the server. Type: array of strings.

If the server version is less than the specified or the server lacks
certain features, the connection will fail with the corresponding error.

Querying server features is implemented via the IPROTO_ID command.
Currently, there are two features defined: streams and transactions.

Irrespective of the options used, the actual version and features are
reported via peer_protocol_version and peer_protocol_features fields of
the connection. Example:

```
tarantool> require('net.box').connect(3301, {
         > required_protocol_version = 1,
         > required_protocol_features = {'transactions'},
         > })
---
- peer_protocol_version: 1
  peer_uuid: 7a8cfdbd-6bbc-4d10-99e5-cbbd06a2382f
  opts:
    required_protocol_version: 1
    required_protocol_features:
    - transactions
  peer_protocol_features:
    transactions: true
    streams: true
  schema_version: 80
  protocol: Binary
  state: active
  peer_version_id: 133632
  port: '3301'
...
```

No functional changes, just get rid of some code duplication.

The new request can be used by a client to let the server know about
supported IPROTO protocol features. The request body contains two
fields: IPROTO protocol version and supported IPROTO protocol features.
In reply to a IPROTO_ID request, the server sends its own protocol
version and supported features.

Currently, the actual protocol version is 1 and there are two features
defined which are always set - streams and transactions.

Part of #6253

@TarantoolBot document
Title: Document IPROTO_ID request

The new request code is 73. It takes a map in the body with the
following keys:
 - IPROTO_VERSION (0x54) - protocol version (unsigned).
 - IPROTO_FEATURES (0x55) - array of protocol feature ids (unsigned).

A client (connector) can send this request to let the server know about
the protocol version and features it supports. The server may enable or
disable certain functionality basing on the features supported by the
client. In reply to the request, the server sends an IPROTO_OK response,
in the body of which it reports its own protocol version and features
(the format of the response body is the same as the request body).
The request doesn't need authentication to pass.

Currently, the actual protocol version is 1 and there are two features
defined:
 - IPROTO_FEATURE_STREAMS - streams support (IPROTO_STREAM_ID header
   key), id = 0.
 - IPROTO_FEATURE_TRANSACTIONS - transactions support (IPROTO_BEGIN,
   IPROTO_COMMIT, IPROTO_ROLLBACK commands), id = 1.

 - Add names for IPROTO_REPLICA_ANON, IPROTO_ID_FILTER, IPROTO_ERROR,
   and IPROTO_TERM.
 - Use "error_24" for IPROTO_ERROR_24, "error" for IPROTO_ERROR.

So that it can be used in a struct definition.

Also, upper-case the name to conform to our coding conventions.

The test is flaky, supposedly because of a bug in Lua JIT.
Disabling Lua JIT makes the test pass.

Workaround for #5081

I think that our current process is better. But if someone will want to
understand why we stick with it, it is good to have explicit cons and
pros.

This patch reworks approach to fiber management in Lua. Before
this patch each action that should return fiber led to new
userdata creation that was quite slow and made GC suffer. This
patch introduces new field in struct fiber to store a reference to
userdata that was created once for a fiber. It allows speedup
operations as fiber.self() and fiber.id().
Simple benchmark shows that access to fiber storage is faster in
two times, fiber.find() - 2-3 times and fiber.new/create functions
don't have any changes.

Before this patch we used an assumption in lua fiber code that
all valid lua refs are positive numbers. However there are
special constants defined in lua header for such purpose LUA_NOREF
and LUA_REFNIL [1]. This patch introduces special value in fiber
module that is equal to LUA_REFNIL to be sure that this value will
be correct in future static assert is added.

Also this patch introduces several usages of this value - let's
make fiber code more clearer and start to initialize ref values
with LUA_NOREF value. Also it will be needed for future changes.

  [1] https://pgl.yoyo.org/luai/i/luaL_ref

For future changes we will need the fiber object, not only its id.

Ref is a lua reference to fiber storage. This patch just renames
storage.lua.ref to storage.lua.storage_ref to underline that
it's not a reference to fiber itself and needed only for fiber
storage.

The problem is if we add a tuple to index and then fiber yield occurs (build-in-background),
replace of this tuple will change it in building index, but it will not be changed back
in case of rollback.

Closes #5958

Currently, if we delete tuple while primary index is altering in background,
a reference to removed tuple will be kept by primary index.

Closes #6290

Part of #5958

This patch fixes the comparison between DECIMAL as a left value and
DOUBLE greater than or equal to 1e38 or less than or equal to -1e38 as a
right value. Any DOUBLE value greater than or equal to 1e38 is now
greater than any DECIMAL value, and a DOUBLE value less than or equal to
-1e38 is less than any DECIMAL value. This is because our decimal cannot
contain more than 38 digits.

Closes #6376

Nullify all the output parameters passed to xrow_decode_subscribe().

This should help us get rid of cases when some of the parameters were
left uninitialized when the SUBSCRIBE request didn't contain the
corresponding fields.

Also it's now possible to remove all the explicit parameter
initializations for xrow_decode_* functions using
xrow_decode_subscribe() internally. Do that as well.

Follow-up #6034

Fixed dropping incoming messages when connection is closed or SHUT_RDWR
received and net_msg_max or readahead limit is reached. Now we don't close
connection if an error occurred while writing response, but only set a
special flag to stop further writing. Connection will be closed when we
read 0 from socket and process all requests.

Closes #6292

Currently there are several statistic metrics for iproto, which can
be obtained by using `box.stat.net`: `CONNECTIONS`, `REQUESTS`, `SENT`
and `RECEIVED`. Add new metric `STREAMS` which contain statistics for
iproto streams.

Part of #6293

@TarantoolBot document
Title: getting statistics for iproto streams was implemented
Add new  metric `STREAMS` to `box.stat.net`, which contain statistics
for iproto streams. `STREAMS` contains same counters as `CONNECTIONS`
metric in `box.stat.net`: current, rps and total. For example:
STREAMS:
  current: 0
  rps: 17980
  totat: 112623

Currently getting iproto statistics per thread is possible only
in debug mode, using `ERRINJ_IPROTO_SINGLE_THREAD_STAT` (it can be
used only in tests). This way have several disadvantages: first of
all `errinj` is not a place to get statistics, secondly user may be
interested in getting iproto statistics per thread. In this patch
`box.stat.net.thread` was implemented, it can be used for getting
same statistics as from `box.stat.net` but per thread.

Part of #6293

@TarantoolBot document
Title: getting network statistic per thread was implemented
User has possibility to run several iproto threads, but at the moment
he can get only general statistics, which combines statistics for all
threads. Now `box.stat.net.thread` has been implemented to get the same
statistics as when using `box.stat.net`, but for a thread. User can call
`box.stat.net.thread()` as a function to get general network statistics
per threads. For example for two iproto threads:
---
- - SENT:
      total: 0
      rps: 0
    CONNECTIONS:
      current: 0
      rps: 0
      total: 0
    REQUESTS:
      current: 0
      rps: 0
      total: 0
    RECEIVED:
      total: 0
      rps: 0
  - SENT:
      total: 0
      rps: 0
    CONNECTIONS:
      current: 0
      rps: 0
      total: 0
    REQUESTS:
      current: 0
      rps: 0
      total: 0
    RECEIVED:
      total: 0
      rps: 0
...

Also user can indexed it as a table by thread number. For example
`box.stat.net.thread[1]` returns network statistics for first iproto
thread:
---
- SENT:
    total: 0
    rps: 0
  CONNECTIONS:
    current: 0
    rps: 0
    total: 0
  REQUESTS:
    current: 0
    rps: 0
    total: 0
  RECEIVED:
    total: 0
    rps: 0
...

Make lbox_tuple_format_new (exported as box.internal.new_tuple_format to
Lua) reuse tuple formats. It's safe, because formats exported to Lua are
immutable. This fixes a net.box error when one creates/closes a lot of
net.box connections: a net.box connection creates a few formats for a
local version of the schema so if the garbage collector isn't invoked to
clean up after it, Tarantool can run out of format ids.

Closes #6217

An ephemeral space format doesn't have a dictionary so we ignore
dictionaries in format hashing and comparison functions. However,
formats exported to Lua (for net.box schema), which could also be
reused, do define field names. Since we'd like to reuse those as well,
we need to take field names into account while doing format hashing and
comparison.

Needed for #6217

Apart from ephemeral space formats, we can also reuse formats exported
to Lua, e.g. for net.box schema. Let's rename the flag and update the
comments to be relevant for all cases.

Needed for #6217

This patch adds xmalloc, xcalloc, xrealloc, xstrdup, and xstrndup helper
functions. Each of them calls the corresponding memory allocation
function and panics if it fails. See the issue description for the full
justification.

Closes #3534

This patch fixes the problem with copying an empty string in mem_copy().
Previously, because the string length was 0, an error was thrown, but
the diag was not set, which could lead to an error due to an empty diag
or to a double free.

Closes #6157
Closes #6399

It contains test case verifying that during parallel index build the
first index to finish wins the race and aborts another TX(s). For the
synchronization sake channels were used. However, in fact containing the
same number of tuples in indexes we can't predict which index finishes
first, so sometimes channel:get() executed inside fiber building index
may hang. Hence, let's simply fill one of spaces with a bit more tuples
and remove channel:get() from functions building indexes.

Closes #6354