Online space upgrade applies a function to all tuples in a space.
The function should be pinned so we need a new holder type.

Follow-up commit 38b25832 ("box: add space upgrade stubs").

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Enum keys are usually stored in a const array of const strings.
Fix opt_def::enum_strs type accordingly.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

So that the caller can figure out the end of fields array without
calling mp_next(). Needed for decoding space upgrade format.

While we are at it, let's also
 - Replace MP_ARRAY assertion with diag_set. This makes the function
   easier to use when the MsgPack format isn't guaranteed to be
   MP_ARRAY.
 - Rearrange arguments so that [out] arguments are grouped together.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Fix the following error:

    {"badRequest": {"message": "The requested availability zone is not
    available", "code": 400}}

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

This option very often makes tests fail due to the issue with the max
length of socket paths (107 chars). Let's rely on the VARDIR environment
variable or another default value (/tmp/t) when VARDIR is not defined.

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

This patch changes the length of default socket paths for luatest tests
(when the VARDIR environment variable is not defined). It should reduce
the chance for the issue with the max length of socket paths (107 chars)
to happen.

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

This change adds VARDIR to the PRESERVE_ENVVARS environment variable
to deliver it to 'packpack' docker containers while running packaging
workflows.

NO_DOC=ci
NO_CHANGELOG=ci
NO_TEST=ci

This patch changes the length of socket paths for tests by 2 chars.
It should reduce the chance for the issue with the max length of socket
paths (107 chars) to happen.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Bump test-run to new version with the following changes:

- Change default value for `vardir` option [1]
- Read default value for `vardir` option from env [1]

[1] tarantool/test-run#338

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

Changed return value type in Lua for the following functions from
`uint64_t` to `int64_t`:

* clock.time64
* clock.realtime64
* clock.monotonic64
* clock.process64
* clock.thread64
* fiber.time64
* fiber.clock64

Changed return value type in Module API for the following functions from
`uint64_t` to `int64_t`:

* fiber_time64
* fiber_clock64

Also enhanced clock.test.lua tap test with more testcases

Due to unsigned values returned from time64 family function time
calculations may lead to undesired value overflows. Having two 64-bit
timestamps I cannot just subtract them without prior comparison. It is
used often, for example when calculating time to a deadline (in queues
or for expiration).

Let's measure time, that pass between two sequential clock.time calls:

```lua
print(clock.time() - clock.time()) -- -9.5367431640625e-07
```

It's ok, with double type time return, but what about 64-bit integers?

```
print(clock.time64() - clock.time64()) -- 18446744073709550616
```

It returns weird integer overflow instead of expected `-1000`

This is counterintuitive and gets conflicted with every time functions
from system headers (which are always signed).

If we consider some hypothetical code, that waits some time for a
deadline, then instead of exiting loop, this code will go to "infinite"
(actually 585 years) wait in the case when the deadline was passed.

```lua
do
local ts = clock.time64() + 1e9 -- some_timestamp_in_future, ex: now+1s
-- ...
local remaining = clock.time64() - ts
-- ...
if remaining > 0 then
    fiber.sleep( tonumber(remaining/1e9) )
end
print "1 second passed"
end
```

This is hard to notice (in real tests deadlines are always in the
future), but in a real environment and under high load such cases occur.

1. Since Lua functions will return ffi.typeof `ctype<int64_t>` instead
   of old `ctype<uint64_t>` any code, that uses _strict_ ffi type check
   will fail. But I hardly believe such code ever exists.
2. Low-level Module API also changed, and code, which compiles with
   `-Werror` may also fail. This could be an issue and in such a case we
   may remove the update of module API change. Still, I consider this
   change reasonable, since all C-level API uses signed time values.

@TarantoolBot document
Title: Return type of time64 functions is changed

These C functions in the public API used to return `uint64_t`, now they
return `int64_t`: `clock_realtime64`, `clock_monotonic64`,
`clock_process64`, `clock_thread64`, `fiber_time64()`,
`fiber_clock64()`.

These Lua functions used to return cdata of type `uint64_t`, now they
return cdata of type `int64_t`: `fiber.time64()`, `fiber.clock64()`.

Fix wording, punctuation, and formatting.
Fix dashes, trailing whitespaces, indentation, and file names.

Needed for: #6430

NO_DOC=changelog
NO_TEST=changelog

Co-authored-by: Valeria Khokha <valeria.khokha@gmail.com>
Co-authored-by: Kseniia Antonova <xuniq.is.here@gmail.com>

LuaJIT submodule is bumped to introduce the following changes:
* GC64: disable sysprof support
* build: make -DLUAJIT_DISABLE_SYSPROF work
* test: disable sysprof C API tests with backtrace
* tools: introduce parsers for sysprof
* sysprof: introduce Lua API
* memprof: add profile common section
* core: introduce lua and platform profiler
* memprof: move symtab to a separate module
* core: separate the profiling timer from lj_profile
* vm: save topframe info into global_State

Within this changeset a parser for binary data dumped via the sampling
profiler to Tarantool binary. It is a set of the following Lua modules:
* sysprof/parse.lua: decode the sampling profiler event stream
* sysprof/collapse.lua: collapse stacks obtained while profiling
* sysprof.lua: Lua script and module to display data

Besides, all new public Lua C API introduced within this changeset is
added to extra/exports.

Closes #781

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

To make tarantool-gc64 available in repository we need to provide
a product name in a package itself and in the repository.

To build Tarantool with gc64 enabled, the flag -DLUAJIT_ENABLE_GC64
needs to be passed to cmake flags. For this purposes GC64 env var
was added to all workflows for Debian, Ubuntu, Centos and Fedora.

This variable is processed to rename PRODUCT_NAME. If GC64 is
eanbled, PRODUCT_NAME is `tarantool-gc64`, otherwise it stays
`tarantool`, which goes to repository name and the package itself.
Thus, Tarantool with gc64 enabled can be installed by
`<packaen_manager> install -y tarantool-gc64`
in the one single repository.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Closes: tarantool/tarantool-qa#161

Changelog: https://curl.se/changes.html#7_80_0

NO_DOC=libcurl submodule bump
NO_CHANGELOG=libcurl submodule bump
NO_TEST=libcurl submodule bump

Closes: #6029
Closes: https://github.com/tarantool/security/issues/10

This patch introduces two stubs: one to log incoming requests and
another one to log outgoing responses. Log itself takes place in TX
thread, in function family `tx_process_*`. At the start of TX processing
we are able to use raw msgpack from input buffer to log it as it was
received. In the same way we can log raw msgpack for responses (before
sending message to IProto thread).

NO_DOC=<No user visible changes>
NO_TEST=<No functional changes>
NO_CHANGELOG=<No functional changes>

It may turn out to be useful having output buffer savepoint pointing to
the start of response. Using it one can restore the whole encoded IProto
response. For instance, to dump it to auxiliary logs.

NO_DOC=<No user visible changes>
NO_TEST=<No functional changes>
NO_CHANGELOG=<No functional changes>

Let's add no-op interface functions and box cfg default parameters.

NO_DOC=<No user visible changes>
NO_TEST=<No functional changes>
NO_CHANGELOG=<No functional changes>

Let's add no-op interface functions and box cfg default parameters.

NO_DOC=<No user visible changes>
NO_TEST=<No functional changes>
NO_CHANGELOG=<No functional changes>

It can be used for instance to duplicate logs somewhere. In particular
case it is going to be used for flight recorder.

NO_DOC=<No user visible changes>
NO_TEST=<No functional changes>
NO_CHANGELOG=<No functional changes>

prbuf entry contains pointer to raw memory (char *) and its length.
Let's introduce :data() methods which would convert this memory chunk
to Lua string.

NO_DOC=<Feature for internal usage>
NO_CHANGELOG=<Feature for internal usage>

If a foreign key consists of multiple fields with `is_nullable=true`
attribute, it's OK to insert a tuple with all-null key, however a
combination of null and non-null fields is not allowed.

Closes #7046

NO_DOC=bugfix

Losing quorum didn't modify RAFT leader behaviour, this led to easy
split brain situation, when current leader gets disconnected from the
rest of the replicaset. Old leader remained RW while new leader could be
elected in the rest of the replicaset.

Now when current leader looses connection to replica (connection being
alive relay and applier for that replica) leader checks that he still
has at least quorum of alive connections to replicas. If less than
quorum of alive connection remain: leader resigns its role becoming
follower without starting new term. This makes old leader RO. Old leader
won't start new elections because of pre-vote, added in 421a0968.

Fenced leader will "freeze" its limbo, and won't CONFIRM nor ROLLBACK
any synchronous transactions until it regains leadership. If another
leader is elected while old one is fenced, once old leader is reconnected
to new one "frozen" transaction will be rolled back.

When quorum of connections is regained and new leader is not elected
old leader or another node will start elections and elect new leader.

Closes #6661

@TarantoolBot document
Title: RAFT leader fencing

RAFT documentation must include that RAFT leader will resign its
leadership if it has less than replication_synchro_quruom of alive
connections to replicas (alive connection is connected relay and applier)
if fencing is on. This applies to election_mode 'candidate' or 'manual'.
Resigning leadership is becoming follower in current RAFT term, this
leads to resigned leader becoming read-only.

Introduce a new `box.cfg` option - `election_fencing_enabled`. If set to
`true` fencing is on (default behaviour). If set to `false` fencing is off
and leader doesn't resign it's leadership when looses quorum. If enabled
on the current leader, when it doesn't have quorum of alive connections -
leader will immediately resign its leadership.

Extended datetime literal parser(s) with basic support for timezone
abbreviations. Not all names could be directly supported at the moment,
only non-ambiguous.

NO_DOC=will be documented with Olson patchset
NO_TEST=changelog only

Use different return values for indication that recognized
timezone suffix is ambiguous or not yet handled (before tzdata
integration to come in next patchset).

i.e. we could always parse '2020-02-02T01:03 MSK' as MSK is
static +03:00, but it's ambiguous if there is something like
'2020-02-02T01:03 AT' as there are multiple ATs (e.g.
'Alaska Time' and 'Atlantic Time').

NO_CHANGELOG=internal
NO_DOC=internal

Use tzindex field wherever we need to output datetime
value to string (with or without format in `:format()` or
`tostring()` correspondingly).

NO_DOC=yet
NO_CHANGELOG=yet

Use our own list of generated timezone names for symbol table
initialization, and then use this symbol table for datetime suffix
parsing without format.

Also, use newly integrated timezone names lookup functions in the
`tnt_strptime` if `%Z` flag was ordered. Return corresponding
`tm_gmtoff` (datetime->tzoffset) and `tm_tzindex`
(datetime->tzindex) for later consumption.

At the moment we could test only non-ambiguous abbreviations,
which might be converted to tzoffset in a deterministic way.

i.e. 'MSK' is always +0300, or 'MSD' is always +0400, but 'AT'
may be either 'Alaska Time' (-0900) or 'Atlantic Time' (-0400/-0300)
thus there is no (yet) any simple way to translate to their offset.

NO_DOC=yet
NO_CHANGELOG=yet

We use gen-zone-abbrevs.pl script to generate file describing timezones
and their fixed ids. Please see `perldoc gen-zone-abbrevs.pl` for more
detailed usage description.

Generator uses `zone-abbrevs.txt` as an input list of all known to
the date timezone abbreviations. We use this page as original source
https://www.timeanddate.com/time/zones/

Names encoding schema used by generator script supposed to be stable
enough for input set of properly sorted zone names, including:
- military zone names;
- zone abbreviations;
- human-readable IANA zone names.

It generates _future compatible_, strictly ordered, sequential
list.

NO_DOC=yet
NO_CHANGELOG=yet
NO_TEST=yet

NO_DOC=infra
NO_CHANGELOG=infra
NO_TEST=infra

Empty or nil `select` calls on user spaces tend to be
dangerous: we see Tarantool crashing with OOM, or Tarantool becoming
unacceptably slow and its transaction thread utilizing 100% CPU usage.
Also, debugging of such killer-requests is often too painful and
exhausting.

To mitigate this, create a critical log entry containing the current Lua
stack traceback upon such function calls — a user can explicitly
request a full scan though by passing `fullscan = true` to `select`'s
`options` table argument in which case the log entry will not be created.
These log entries are also rate limited.

Closes #6539

@TarantoolBot document
Title: new behaviour of empty or nil `select` calls on user spaces

A critical log entry containing the current stack
traceback is created upon empty or nil `select` calls on user spaces.

A user can explicitly request a full scan though by passing
`fullscan = true` to `select`'s `options` table argument in which a case a
log entry will not be created.

Introduce a `Ratelimit` class equivalent to `src/lib/core/ratelimit.h` to
`log` module for internal usage: it acts as a wrapper around the log API.

In scope of #6539 we need to make critical log entries: add S_CRIT alias
for this purpose and a `log_crit` method to `Ratelimit`.

Needed for #6539

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

When a node has Raft turned on, issuing `box.ctl.promote()` on it triggers
Raft elections. Such promotion bypasses the Pre-Vote check, because we
want manual promotion to succeed even if it disrupts the current leader.

However, in order to win elections, the node still has to collect votes
from a quorum of peers. When there's no quorum, promote() might enter an
infinite election loop, trying to promote the current node.

Let's not start elections when it's known for sure that the node won't
be able to win them, and make promote() throw an appropriate error.

Now the node still triggers at least one election round on
box.ctl.promote(), but it doesn't attempt re-running elections
infinitely when there is no quorum.

Follow-up #6654

NO_DOC=minor change

Raft cluster members only treat the leader as alive as long as they
preserve a direct connection to it. This means that as soon as one of
the followers loses leader connection, it will start new elections
bumping the term and forcing the existing leader to step off.

Moreover, when a server is partitioned from the rest of the cluster, it
will constantly rearrange elections by bumping the term and voting for
itself. It will never get any votes, since it is partitioned, but it
will increase its term to big numbers. Once such a server reconnects to
the rest of the cluster, it will make everyone read-only, because the
working majority will probably have a smaller term than this server.

Such elections, performed while there is a working leader, like in the
first example, or performed when there's no chance of winning, like in
the second example, are considered disruptive.

Diego Ongaro's thesis (https://github.com/ongardie/dissertation)
proposes the following solution to the problem of disruptive servers.

It's an additional election phase, called Pre-Vote.
The basic idea is as follows: only bump the term and start elections
once the quorum of peers reply OK to a Pre-Vote message.
The peers would reply OK when:
  - the candidate's log is sufficiently up to date
  - the peer doesn't see the leader

The downside of such approach is an additional request travelling to and
from the followers.

Tarantool replication architecture allows us to achieve the same goals
without issuing Pre-Vote requests. Here's what's done in this patch to
do so:
    1) Track number of live peer connections, and only start elections
       when there is a quorum of connected peers.
    2) Make every node broadcast whether it sees the leader of the
       current term or not. Every candidate collects info about who's
       seen as a leader in the current term. When at least one node sees
       a leader, the candidate doesn't start new elections.

Closes #6654

@TarantoolBot document
Title: elections: new `box.info` field and binary protocol changes

* `box.info.election` got a new field: `leader_idle`.
  When elections are enabled, it shows time in seconds since the last
  interaction with the known leader.

* `IPROTO_RAFT` request got 2 new fields:
  `IPROTO_RAFT_LEADER_ID = 0x04` - uint - the id of the current leader
  (as seen by the node which issued the request).
  `IPROTO_RAFT_IS_LEADER_SEEN = 0x05` - bool - whether the node has a
  direct connection to the leader.

Support Pre-Vote in Raft: when a node knows, that some of the peers see
the leader of the current term, it doesn't start elections. Even when it
doesn't have a connection to the leader.

Don't use the machinery yet: nodes never share info on whether they see
the leader.

Part-of #6654

NO_DOC=no user-visible changes
NO_CHANGELOG=no user-visible changes

Our checkpatch.pl script demands that every new function has a comment.
One new function will be added to raft_ev.h in the following commits,
and, according to the new rules, it has to have a comment.

Having one function in a file commented looks stupid, so let's add
comments to every such function in raft_ev.h

NO_DOC=making lint happy
NO_CHANGELOG=making lint happy
NO_TEST=making lint happy

raft_checkpoint_remote and raft_worker_handle_broadcast perform the same
job of filling the raft message with a complete raft state.

Let's fix that.

Part-of #6654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Start counting replicas, whose connections are "healthy", e.g. there are
both relay and applier in SUBSCRIBE state.

Introduce triggers for the events of "healthy quorum" loss and gain.
"healthy quorum" is the same as replication_synchro_quorum.

The triggers will be used in raft pre-vote and synchro queue owner
fencing. Both these features require tracking whether there is a quorum
of nodes reachable from the local node.

Pre-requisite #6654
Pre-requisite #6661

NO_DOC=no user-visible changes
NO_CHANGELOG=no user-visible changes
NO_TEST=preparatory patch

MVCC must track a read set of transaction, including cases when
the select returns an empty result. In this case MVCC has two
options: to store a point key if the select has a full key or
to store a gap otherwise.

By a mistake the second variant was not implemented. Fix it now.

Closes #7025

NO_DOC=bugfix

To implement online space upgrade, we need to add stub calls to the
following code paths and data structures:

 - struct space: add a pointer to space upgrade state.
 - struct space_opts: add a pointer to space upgrade definition.
 - CheckSpaceFormat::prepare: skip space check if the format is changed
   in the scope of space upgrade.
 - alter_space_do: check space upgrade state and fail alter if upgrade
   is in progress.
 - alter_space_commit: run background worker for space upgrade.
 - space_on_final_recovery_complete: restart upgrade after recovery.
 - result_processor: apply space upgrade transformations to tuples
   returned to the user by box functions.

We also need to:
 - Add a new error code ER_WRONG_SPACE_UPGRADE_OPTIONS, which we will
   use on error decoding upgrade options, stored in space options.
 - Load space upgrade Lua modules. The modules are supposed to define
   box.internal.space.upgrade method, which if available is used by
   box.schema.space.upgrade.
 - Add check_param, check_param_table and normalize_format helpers to
   box.internal, because we will use them from space.upgrade Lua code.

Note, the space upgrade state will be reference counted, because
background space upgrade may complete while some fiber is reading
from the upgraded space (there may be yields in Vinyl). For this fiber
to process the result correctly, it has to increment the reference
counter of the space upgrade state before reading from the space.

NO_DOC=ee
NO_TEST=ee
NO_CHANGELOG=ee

A convenient helper that is supposed to apply various transformations
to a tuple fetched from a space before returning it to the user.
Usage:

  struct result_processor res_proc;
  result_process_prepare(&res_proc, space);
  rc = index_get(index, key, part_count, result);
  result_process(&res_proc, &rc, result);

We need to split the procedure into two parts, because index_get and
similar methods may yield in case of Vinyl. The 'prepare' method is
supposed to pin all data structures needed to process the result in the
result_processor struct while the 'process' method is supposed to
release them.

Currently, the new methods do absolutely nothing. They will be used to
convert tuples during space upgrade.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal