It can be used for instance to duplicate logs somewhere. In particular
case it is going to be used for flight recorder.

NO_DOC=<No user visible changes>
NO_TEST=<No functional changes>
NO_CHANGELOG=<No functional changes>

prbuf entry contains pointer to raw memory (char *) and its length.
Let's introduce :data() methods which would convert this memory chunk
to Lua string.

NO_DOC=<Feature for internal usage>
NO_CHANGELOG=<Feature for internal usage>

If a foreign key consists of multiple fields with `is_nullable=true`
attribute, it's OK to insert a tuple with all-null key, however a
combination of null and non-null fields is not allowed.

Closes #7046

NO_DOC=bugfix

Losing quorum didn't modify RAFT leader behaviour, this led to easy
split brain situation, when current leader gets disconnected from the
rest of the replicaset. Old leader remained RW while new leader could be
elected in the rest of the replicaset.

Now when current leader looses connection to replica (connection being
alive relay and applier for that replica) leader checks that he still
has at least quorum of alive connections to replicas. If less than
quorum of alive connection remain: leader resigns its role becoming
follower without starting new term. This makes old leader RO. Old leader
won't start new elections because of pre-vote, added in 421a0968.

Fenced leader will "freeze" its limbo, and won't CONFIRM nor ROLLBACK
any synchronous transactions until it regains leadership. If another
leader is elected while old one is fenced, once old leader is reconnected
to new one "frozen" transaction will be rolled back.

When quorum of connections is regained and new leader is not elected
old leader or another node will start elections and elect new leader.

Closes #6661

@TarantoolBot document
Title: RAFT leader fencing

RAFT documentation must include that RAFT leader will resign its
leadership if it has less than replication_synchro_quruom of alive
connections to replicas (alive connection is connected relay and applier)
if fencing is on. This applies to election_mode 'candidate' or 'manual'.
Resigning leadership is becoming follower in current RAFT term, this
leads to resigned leader becoming read-only.

Introduce a new `box.cfg` option - `election_fencing_enabled`. If set to
`true` fencing is on (default behaviour). If set to `false` fencing is off
and leader doesn't resign it's leadership when looses quorum. If enabled
on the current leader, when it doesn't have quorum of alive connections -
leader will immediately resign its leadership.

Extended datetime literal parser(s) with basic support for timezone
abbreviations. Not all names could be directly supported at the moment,
only non-ambiguous.

NO_DOC=will be documented with Olson patchset
NO_TEST=changelog only

Use different return values for indication that recognized
timezone suffix is ambiguous or not yet handled (before tzdata
integration to come in next patchset).

i.e. we could always parse '2020-02-02T01:03 MSK' as MSK is
static +03:00, but it's ambiguous if there is something like
'2020-02-02T01:03 AT' as there are multiple ATs (e.g.
'Alaska Time' and 'Atlantic Time').

NO_CHANGELOG=internal
NO_DOC=internal

Use tzindex field wherever we need to output datetime
value to string (with or without format in `:format()` or
`tostring()` correspondingly).

NO_DOC=yet
NO_CHANGELOG=yet

Use our own list of generated timezone names for symbol table
initialization, and then use this symbol table for datetime suffix
parsing without format.

Also, use newly integrated timezone names lookup functions in the
`tnt_strptime` if `%Z` flag was ordered. Return corresponding
`tm_gmtoff` (datetime->tzoffset) and `tm_tzindex`
(datetime->tzindex) for later consumption.

At the moment we could test only non-ambiguous abbreviations,
which might be converted to tzoffset in a deterministic way.

i.e. 'MSK' is always +0300, or 'MSD' is always +0400, but 'AT'
may be either 'Alaska Time' (-0900) or 'Atlantic Time' (-0400/-0300)
thus there is no (yet) any simple way to translate to their offset.

NO_DOC=yet
NO_CHANGELOG=yet

We use gen-zone-abbrevs.pl script to generate file describing timezones
and their fixed ids. Please see `perldoc gen-zone-abbrevs.pl` for more
detailed usage description.

Generator uses `zone-abbrevs.txt` as an input list of all known to
the date timezone abbreviations. We use this page as original source
https://www.timeanddate.com/time/zones/

Names encoding schema used by generator script supposed to be stable
enough for input set of properly sorted zone names, including:
- military zone names;
- zone abbreviations;
- human-readable IANA zone names.

It generates _future compatible_, strictly ordered, sequential
list.

NO_DOC=yet
NO_CHANGELOG=yet
NO_TEST=yet

NO_DOC=infra
NO_CHANGELOG=infra
NO_TEST=infra

Empty or nil `select` calls on user spaces tend to be
dangerous: we see Tarantool crashing with OOM, or Tarantool becoming
unacceptably slow and its transaction thread utilizing 100% CPU usage.
Also, debugging of such killer-requests is often too painful and
exhausting.

To mitigate this, create a critical log entry containing the current Lua
stack traceback upon such function calls — a user can explicitly
request a full scan though by passing `fullscan = true` to `select`'s
`options` table argument in which case the log entry will not be created.
These log entries are also rate limited.

Closes #6539

@TarantoolBot document
Title: new behaviour of empty or nil `select` calls on user spaces

A critical log entry containing the current stack
traceback is created upon empty or nil `select` calls on user spaces.

A user can explicitly request a full scan though by passing
`fullscan = true` to `select`'s `options` table argument in which a case a
log entry will not be created.

Introduce a `Ratelimit` class equivalent to `src/lib/core/ratelimit.h` to
`log` module for internal usage: it acts as a wrapper around the log API.

In scope of #6539 we need to make critical log entries: add S_CRIT alias
for this purpose and a `log_crit` method to `Ratelimit`.

Needed for #6539

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

When a node has Raft turned on, issuing `box.ctl.promote()` on it triggers
Raft elections. Such promotion bypasses the Pre-Vote check, because we
want manual promotion to succeed even if it disrupts the current leader.

However, in order to win elections, the node still has to collect votes
from a quorum of peers. When there's no quorum, promote() might enter an
infinite election loop, trying to promote the current node.

Let's not start elections when it's known for sure that the node won't
be able to win them, and make promote() throw an appropriate error.

Now the node still triggers at least one election round on
box.ctl.promote(), but it doesn't attempt re-running elections
infinitely when there is no quorum.

Follow-up #6654

NO_DOC=minor change

Raft cluster members only treat the leader as alive as long as they
preserve a direct connection to it. This means that as soon as one of
the followers loses leader connection, it will start new elections
bumping the term and forcing the existing leader to step off.

Moreover, when a server is partitioned from the rest of the cluster, it
will constantly rearrange elections by bumping the term and voting for
itself. It will never get any votes, since it is partitioned, but it
will increase its term to big numbers. Once such a server reconnects to
the rest of the cluster, it will make everyone read-only, because the
working majority will probably have a smaller term than this server.

Such elections, performed while there is a working leader, like in the
first example, or performed when there's no chance of winning, like in
the second example, are considered disruptive.

Diego Ongaro's thesis (https://github.com/ongardie/dissertation)
proposes the following solution to the problem of disruptive servers.

It's an additional election phase, called Pre-Vote.
The basic idea is as follows: only bump the term and start elections
once the quorum of peers reply OK to a Pre-Vote message.
The peers would reply OK when:
  - the candidate's log is sufficiently up to date
  - the peer doesn't see the leader

The downside of such approach is an additional request travelling to and
from the followers.

Tarantool replication architecture allows us to achieve the same goals
without issuing Pre-Vote requests. Here's what's done in this patch to
do so:
    1) Track number of live peer connections, and only start elections
       when there is a quorum of connected peers.
    2) Make every node broadcast whether it sees the leader of the
       current term or not. Every candidate collects info about who's
       seen as a leader in the current term. When at least one node sees
       a leader, the candidate doesn't start new elections.

Closes #6654

@TarantoolBot document
Title: elections: new `box.info` field and binary protocol changes

* `box.info.election` got a new field: `leader_idle`.
  When elections are enabled, it shows time in seconds since the last
  interaction with the known leader.

* `IPROTO_RAFT` request got 2 new fields:
  `IPROTO_RAFT_LEADER_ID = 0x04` - uint - the id of the current leader
  (as seen by the node which issued the request).
  `IPROTO_RAFT_IS_LEADER_SEEN = 0x05` - bool - whether the node has a
  direct connection to the leader.

Support Pre-Vote in Raft: when a node knows, that some of the peers see
the leader of the current term, it doesn't start elections. Even when it
doesn't have a connection to the leader.

Don't use the machinery yet: nodes never share info on whether they see
the leader.

Part-of #6654

NO_DOC=no user-visible changes
NO_CHANGELOG=no user-visible changes

Our checkpatch.pl script demands that every new function has a comment.
One new function will be added to raft_ev.h in the following commits,
and, according to the new rules, it has to have a comment.

Having one function in a file commented looks stupid, so let's add
comments to every such function in raft_ev.h

NO_DOC=making lint happy
NO_CHANGELOG=making lint happy
NO_TEST=making lint happy

raft_checkpoint_remote and raft_worker_handle_broadcast perform the same
job of filling the raft message with a complete raft state.

Let's fix that.

Part-of #6654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Start counting replicas, whose connections are "healthy", e.g. there are
both relay and applier in SUBSCRIBE state.

Introduce triggers for the events of "healthy quorum" loss and gain.
"healthy quorum" is the same as replication_synchro_quorum.

The triggers will be used in raft pre-vote and synchro queue owner
fencing. Both these features require tracking whether there is a quorum
of nodes reachable from the local node.

Pre-requisite #6654
Pre-requisite #6661

NO_DOC=no user-visible changes
NO_CHANGELOG=no user-visible changes
NO_TEST=preparatory patch

MVCC must track a read set of transaction, including cases when
the select returns an empty result. In this case MVCC has two
options: to store a point key if the select has a full key or
to store a gap otherwise.

By a mistake the second variant was not implemented. Fix it now.

Closes #7025

NO_DOC=bugfix

To implement online space upgrade, we need to add stub calls to the
following code paths and data structures:

 - struct space: add a pointer to space upgrade state.
 - struct space_opts: add a pointer to space upgrade definition.
 - CheckSpaceFormat::prepare: skip space check if the format is changed
   in the scope of space upgrade.
 - alter_space_do: check space upgrade state and fail alter if upgrade
   is in progress.
 - alter_space_commit: run background worker for space upgrade.
 - space_on_final_recovery_complete: restart upgrade after recovery.
 - result_processor: apply space upgrade transformations to tuples
   returned to the user by box functions.

We also need to:
 - Add a new error code ER_WRONG_SPACE_UPGRADE_OPTIONS, which we will
   use on error decoding upgrade options, stored in space options.
 - Load space upgrade Lua modules. The modules are supposed to define
   box.internal.space.upgrade method, which if available is used by
   box.schema.space.upgrade.
 - Add check_param, check_param_table and normalize_format helpers to
   box.internal, because we will use them from space.upgrade Lua code.

Note, the space upgrade state will be reference counted, because
background space upgrade may complete while some fiber is reading
from the upgraded space (there may be yields in Vinyl). For this fiber
to process the result correctly, it has to increment the reference
counter of the space upgrade state before reading from the space.

NO_DOC=ee
NO_TEST=ee
NO_CHANGELOG=ee

A convenient helper that is supposed to apply various transformations
to a tuple fetched from a space before returning it to the user.
Usage:

  struct result_processor res_proc;
  result_process_prepare(&res_proc, space);
  rc = index_get(index, key, part_count, result);
  result_process(&res_proc, &rc, result);

We need to split the procedure into two parts, because index_get and
similar methods may yield in case of Vinyl. The 'prepare' method is
supposed to pin all data structures needed to process the result in the
result_processor struct while the 'process' method is supposed to
release them.

Currently, the new methods do absolutely nothing. They will be used to
convert tuples during space upgrade.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

The new helper function returns the space the iterator was created for.
It looks up the space by id and caches it in iterator struct. We need it
for space upgrade: we will use it in box_iterator_next() to update
tuples returned by iterator_next().

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

It's a more appropriate name, because the function doesn't update
a space format - it just adjusts a format table to a normal form.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

After this patch, it will be possible to perform permitted arithmetic
operations on DATETIME and INTERVAL values.

Closes #6773

@TarantoolBot document
Title: Field type interval is available in SQL

The INTERVAL field type is now available in SQL. INTERVAL values cannot
be compared. INTERVAL values can be explicitly and implicitly cast to
ANY. There are currently no literals for the INTERVAL field type.

Prior to this patch, the right operand was checked before the left. This
can lead to some complications when arithmetic operations are added for
the DATETIME and INTERVAL values. To avoid this, let's check the left
operand before the right one. This will cause the description of the
error to change, but nothing else.

Needed for #6773

NO_DOC=Not documented feature.

Part of #6773

NO_DOC=Will be added later.
NO_CHANGELOG=Will be added later.

This patch introduces basic INTERVAL support in SQL. After this patch,
it will be allowed to select INTERVAL values from spaces, insert them
into spaces, and use them in functions. CAST() from INTERVAL to STRING
and ANY is also supported.

Part of #6773

NO_DOC=Will be added later.
NO_CHANGELOG=Will be added later.

This patch fixes the error description for comparison in case the left
operand is comparable but the right operand is not.

Part of #6668
Needed for #6668

NO_DOC=Will be added in another commit.
NO_CHANGELOG=Will be added in another commit.

After this patch, using MAP and ARRAY in GROUP BY will throw an error
instead of an assertion.

Part of #6668
Needed for #6773

NO_DOC=Will be added in another commit.
NO_CHANGELOG=Will be added in another commit.

After this patch, when DISTINCT is used with MAP or ARRAY, an error will
be thrown instead of a segmentation fault.

Part of #6668
Needed for #6773

NO_DOC=Will be added in another commit.
NO_CHANGELOG=Will be added in another commit.

LuaJIT submodule is bumped to introduce the following changes:
* memprof: enrich symtab with newly loaded symbols
* memprof: extend symtab with C symbols

Within this changeset the new Lua module providing minimalistic AVL tree
implementation required for processing C symbols is introduced:
* utils/avl.lua: minimalistic AVL tree implementation

Closes #5813

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

Signed-off-by: Igor Munkin <imun@tarantool.org>

New field type INTERVAL is now supported by BOX.

Part of #6773

NO_DOC=INTERVAL has already been introduced earlier.

Part of #6773

NO_DOC=INTERVAL has already been introduced earlier.
NO_CHANGELOG=It will be added in another commit.

Part of #6773

NO_DOC=INTERVAL has already been introduced earlier.
NO_CHANGELOG=It will be added in another commit.

Part of #6773

NO_CHANGELOG=It will be added in another commit.

@TarantoolBot document
Title: New MsgPack extension type MP_INTERVAL

MsgPack extension type MP_INTERVAL was created to serialize values of
struct interval.

Description of serialized value:

------------------------------------------------------------
|MP_EXT|Size of packed interval|MP_INTERVAL|packed interval|
------------------------------------------------------------

Packed interval - packed number of non-zero fields and packed non-null
fields. Each packed field consists of two packed integer values - the
field ID and its value. The number of non-null fields can be zero, in
which case `packed interval` will be just packed integer `0`.

List of IDs:
0 - `year`
1 - `month`
2 - `week`
3 - `day`
4 - `hour`
5 - `minute`
6 - `second`
7 - `nanosecond`
8 - `adjust`

Example of packed interval value:

|C7|09|06|03|00|01|01|CCC8|03|D0B3|

Where
C7 - MP_EXT;
09 - size of packed interval value, 9 bytes;
06 - MP_INTERVAL;
03 - number of non-zero fields;
00 - ID of field `year`;
01 - packed value `1`;
01 - ID of field `month`;
CCC8 - packed value `200`;
03 - ID of field `day`;
D0B3 - packed value `-77`.

From there we can see, that the packed interval is actually
`1 year, 200 months and -77 days`.

This method is necessary to compare two INTERVAL values in Luatest.

Part of #6773

NO_DOC=Internal function.
NO_CHANGELOG=INTERVAL has already been introduced earlier.

Half of allocation functions in space_def_dup and space_def_new already
use xmalloc. Use xmalloc for the rest of allocations.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

To implement space upgrade, we need a function that duplicates field_def
array, because we will store a new space format in space options.

To factor out field_def_array_dup from space_def_dup, let's allocate it
with a separate malloc rather than after space_def struct.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

NO_DOC=internal
NO_CHANGELOG=internal