Since commit d2537d9d ("relay: cleanup error handling")
recover_remaining_wals() doesn't log the error it throws -
now callers of this function should catch and log the error.
hot_standby_f() doesn't catch the error so the diagnostic
message is lost if we fail to apply a row in the hot standby
mode. Fix this.

NO_DOC=bug fix
NO_TEST=checked in next commit
NO_CHANGELOG=minor bug in logging

If a nested tuple field is indexed, it can be accessed by [*] aka
multikey or any token:

  s = box.schema.create_space('test')
  s:create_index('pk')
  s:create_index('sk', {parts = {{2, 'unsigned', path = '[1][1]'}}})
  t = s:replace{1, {{1}}}
  t['[2][1][*]'] -- returns 1!

If a nested field isn't indexed (remove creation of the secondary index
in the example above), then access by [*] returns nil.

Call graph:

  lbox_tuple_field_by_path:
    tuple_field_raw_by_full_path
      tuple_field_raw_by_path
        tuple_format_field_by_path
          json_tree_lookup_entry
            json_tree_lookup

And json_tree_lookup matches the first node if the key is [*].
We shouldn't match anything to [*].

Closes #5226

NO_DOC=bug fix

__gcov_flush was removed in gcc11.

Since gcc11 __gcov_dump calls __gcov_lock at the start and
__gcov_unlock before returning. Same is true for __gcov_reset.
Because of that using __gcov_reset right after __gcov_dump since gcc11
is the same as using __gcov_flush before gcc11.

Closes #7302

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

Covered most of box_promote and box_demote with tests:
1. Promote/demote unconfigured box
2. Promoting current leader with elections on and off
3. Demoting follower with elections on and off
4. Promoting current leader, but not limbo owner with elections on
5. Demoting current leader with elections on and off
6. Simultaneous promote/demote
7. Promoting voter
8. Interfering promote/demote while writing new term to wal
9. Interfering promote/demote while waiting for synchro queue
   to be emptied
10. Interfering promote while waiting for limbo to be acked
    (similar to replication/gh-5430-qsync-promote-crash.test.lua)

Closes #6033

NO_DOC=testing stuff
NO_CHANGELOG=testing stuff

The test failed with the following output:

 TAP version 13
 1..3
 # Started on Tue Jun 28 13:36:03 2022
 # Starting group: pre-vote
 not ok 1	pre-vote.test_no_direct_connection
 #   .../election_pre_vote_test.lua:46: expected: a value evaluating to
					true, actual: false
 #   stack traceback:
 #   .../election_pre_vote_test.lua:65: in function 'retrying'
 #   .../election_pre_vote_test.lua:64: in function
				    'pre-vote.test_no_direct_connection'
 #   ...
 #   [C]: in function 'xpcall'
 ok     2	pre-vote.test_no_quorum
 ok     3	pre-vote.test_promote_no_quorum
 # Ran 3 tests in 6.994 seconds, 2 succeeded, 1 failed

This is the moment when one of the followers disconnects from
the leader and expects its `box.info.election.leader_idle` to grow.

It wasn't taken into account that this disconnect might lead to leader
resign due to fencing, and then a new leader would emerge and
`leader_idle` would still be small.

IOW, the leader starts with fencing turned off, and only resumes
fencing, once it has connected to a quorum of nodes (one replica in this
test). If the replica that we just connected happens to be the one we
disconnect in the test, the leader might fence, if it hasn't yet
connected to the other replica, because it immediately loses a quorum of
healthy connections right after gaining it for the first time.

Fix this by waiting until everyone follows everyone before each test
case.

The test, of course, could be fixed by turning fencing off, but this
might hide any possible future problems with fencing.

Follow-up #6654
Follow-up #6661

NO_CHANGELOG=test fix
NO_DOC=test fix

After scanning disk, the Vinyl read iterator checks if it should restore
the iterator over the active memory tree, because new tuples could have
been inserted into it while we yielded reading disk. We assume that
existing tuples can't be deleted from the memory tree, but that's not
always true - a tuple may actually be deleted by rollback after a failed
WAL write. Let's reevaluate all scanned sources and reposition the read
iterator to the next statement if this happens.

Initially, the issue was fixed by commit 83462a5c ("vinyl: restart
read iterator in case L0 is changed"), but it introduced a performance
degradation and was reverted (see #5700).

NO_DOC=bug fix
NO_TEST=already there
NO_CHANGELOG=already there

The Vinyl read iterator, which is used for serving range select
requests, works as follows:

 1. Scan in-memory sources. If found an exact match or a chain in
    the cache, return it.
 2. If not found, scan disk sources. This operation may yield.
 3. If any new data was inserted into the active memory tree, go to
    step 1, effectively restarting the iteration from the same key.

Apparently, such an algorithm doesn't guarantee any progress of a read
operation at all - when we yield reading disk on step 2 after a restart,
even newer data may be inserted into the active memory tree, forcing us
to restart again. In other words, in presence of an intensive write
workload, read ops rate may drop down to literally 0.

It hasn't always been like so. Before commit 83462a5c ("vinyl:
restart read iterator in case L0 is changed"), we only restored the
memory tree iterator after a yield, without restarting the whole
procedure. This makes sense, because only memory tree may change after
a yield so there's no point in rescanning other sources, including
disk. By restarting iteration after a yield, the above-mentioned commit
fixed bug #3395: initially we assumed that statements may never be
deleted from a memory tree while actually they can be deleted by
rollback after a failed WAL write.

Let's revert this commit to fix the performance degradation. We will
re-fix bug #3395 in the next commit.

Closes #5700

NO_DOC=bug fix
NO_TEST=should be checked by performance tests

* Avoid conflict between 64 bit lightuserdata and ITERN key.
* Reorganize lightuserdata interning code.
* test: fix path storage for non-concatable objects
* ARM64: Fix assembly of HREFK.
* FFI/ARM64: Fix pass-by-value struct calling conventions.
* test: set DYLD_LIBRARY_PATH environment variable
* x64/LJ_GC64: Fix fallback case of asm_fuseloadk64().
* FFI: Handle zero-fill of struct-of-NYI.
* Fix interaction between profiler hooks and finalizers.
* Flush and close output file after profiling run.
* Fix debug.debug() for non-string errors.
* Fix write barrier for lua_setupvalue() and debug.setupvalue().
* Fix FOLD rule for strength reduction of widening.
* Fix bytecode dump unpatching.
* Fix tonumber("-0") in dual-number mode.
* Fix tonumber("-0").
* Give expected results for negative non-base-10 numbers in tonumber().
* Add missing LJ_MAX_JSLOTS check.
* Add stricter check for print() vs. tostring() shortcut.

Closes #6548
Fixes #4614
Fixes #4630
Fixes #5885
Fixes tarantool/tarantool-qa#234
Fixes tarantool/tarantool-qa#235
Follows up #2712

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

Normally, there shouldn't be any upserts on disk if the space has
secondary indexes, because we can't generate an upsert without a lookup
in the primary index hence we convert upserts to replace+delete in this
case. The deferred delete optimization only makes sense if the space has
secondary indexes. So we ignore upserts while generating deferred
deletes, see vy_write_iterator_deferred_delete.

There's an exception to this rule: a secondary index could be created
after some upserts were used on the space. In this case, because of the
deferred delete optimization, we may never generate deletes for some
tuples for the secondary index, as demonstrated in #3638.

We could fix this issue by properly handle upserts in the write iterator
while generating deferred delete, but this wouldn't be easy, because in
case of a minor compaction there may be no replace/insert to apply the
upsert to so we'd have to keep intermediate upserts even if there is a
newer delete statement. Since this situation is rare (happens only once
in a space life time), it doesn't look like we should complicate the
write iterator to fix it.

Another way to fix it is to force major compaction of the primary index
after a secondary index is created. This looks doable, but it could slow
down creation of secondary indexes. Let's instead simply disable the
deferred delete optimization if the primary index has upsert statements.
This way the optimization will be enabled sooner or later, when the
primary index major compaction occurs. After all, it's just an
optimization and it can be disabled for other reasons (e.g. if the space
has on_replace triggers).

Closes #3638

NO_DOC=bug fix

* doc: add allocators hierarchy diagram
* build: fix CMake warning
* small: fix compilation on macOS 12

NO_DOC=small submodule bump
NO_TEST=small submodule bump
NO_CHANGELOG=small submodule bump

Asserts are disabled in the Release build, that leads to:

$ cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS=-Werror
$ make

[ 43%] Building C object src/lib/bitset/CMakeFiles/bitset.dir/bitset.c.o
src/lib/bitset/bitset.c:169:9: error: variable 'cardinality_check' set but not used [-Werror,-Wunused-but-set-variable]
        size_t cardinality_check = 0;
               ^

NO_DOC=build fix
NO_TEST=build fix
NO_CHANGELOG=build fix

This will fix compilation on recent Archlinux. The issue is this distro
installs libbpf in base configuration and nghttp2 1.46.0 tries to
compile eBPF code if this library is present and failed if compiler is
gcc.

Closes #7292

NO_DOC=nghttp2 submodule bump
NO_TEST=nghttp2 submodule bump

Now fiber.top() does not use x86-specific instructions, so it can be
enabled for ARM.

Closes #4573

NO_TEST=<Tested in test/app/fiber.test.lua>
NO_DOC=<x86 or ARM are not mentioned in the fiber.top doc>

It doesn't make sense after switching from RDTSCP to
clock_gettime(CLOCK_MONOTONIC).

Part of #5869

@TarantoolBot document
Title: fiber: get rid of cpu_misses in fiber.top()
Since: 2.11

Remove any mentions of `cpu_misses` in `fiber.top()` description.

clock_gettime(CLOCK_MONOTONIC) is implemented via the RDTSCP instruction
on x86 an has the following advantages over the raw instruction:

* It checks for RDTSCP availability in CPUID.
  If RDTSCP is not supported, it switches to RDTSC.
* Linux guarantee that clock is monotonic, hence, the CPU miss
  detection is not needed.
* It works on ARM.

As for disadvantage, this function is about 2x slower compared to a
single RDTSCP instruction. Performance degradation measured by the
fiber switch benchmark [1] is about 3-7% for num_fibers == 10-1000.

Closes #5869

[1] https://github.com/tarantool/tarantool/issues/2694#issuecomment-546381304

NO_DOC=bugfix
NO_TEST=<Tested in test/app/fiber.test.lua>

Use this wrapper to simplify the code.

Part of #5869

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

clock_gettime() returns 0 for success, or -1 for failure.
Add missed checks for the return value.

Part of #5869

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

These 3 modules are low hanging fruits which right now can be
freed at return from main() without any effort.

There are still a lot of other modules whose freeing is not that
easy. A few hard to untangle knots, and there are more:

- Session, credentials, iproto, and fibers are tied together via
  the latter. Each fiber potentially has a session, its current
  credentials object. Each iproto connection has a session and a
  file descriptor which is stored in the session too.

  The possible solution would be to walk all the fibers and
  destroy them before proceeding to destroy everything else.

- Tuples depend on memtx, and Lua depends on tuples. Because there
  are tuples allocated on memtx->arena. Hence destruction of memtx
  and its arena makes the tuples still stored in Lua invalid.

  It seems Lua should be destroyed first, not last. It would free
  all the refs which might be kept at objects in C in the other
  modules.

- IProto connections leak when iproto is destroyed. They are not
  freed and their descriptors are not closed properly. That
  requires additional preparatory work to destroy them correctly
  on iproto module deconstruction.

Given amount of work, it should be done as a big separate ticket.

Follow up #7259

NO_CHANGELOG=Not a visible change
NO_DOC=Not a visible change
NO_TEST=Not a visible change

main() used to skip most of modules destruction in
tarantool_free(). That got ASAN complaining on clang-13 about a
leak of a fiber on_stop trigger which was allocated in Lua.

The patch makes fiber_free() called for the main cord. It destroys
and frees all the fibers together with their on_stop triggers.

Closes #7259

NO_CHANGELOG=Not a visible change
NO_DOC=Not a visible change
NO_TEST=Not a visible change

Before this patch struct tuple had two boolean bit fields: is_dirty and
has_uploaded_refs. It is worth mentioning that sizeof(boolean) is
implementation depended. However, in code it is assumed to be 1 byte
(there's static assertion restricting the whole struct tuple size by 10
bytes). So strictly speaking it may lead to the compilation error on
some non-conventional system. Secondly, bit fields anyway consume at
least one size of type (i.e. there's no space benefits in using two
uint8_t bit fields - they anyway occupy 1 byte in total). There are
several known pitfalls concerning bit fields:
 - Bit field's memory layout is implementation dependent;
 - sizeof() can't be applied to such members;
 - Complier may raise unexpected side effects
   (https://lwn.net/Articles/478657/).

Finally, in our code base as a rule we use explicit masks:
txn flags, vy stmt flags, sql flags, fiber flags.

So, let's replace bit fields in struct tuple with single member called
`flags` and several enum values corresponding to masks (to be more
precise - bit positions in tuple flags).

NO_DOC=<Refactoring>
NO_CHANGELOG=<Refactoring>
NO_TEST=<Refactoring>

We did not retain correctly `hour` attribute if modified
via `:set` method attributes `min`, `sec` or `nsec`.

```
tarantool> a = dt.parse '2022-05-05T00:00:00'

tarantool> a:set{min = 0, sec = 0, nsec = 0}
--
- 2022-05-05T12:00:00Z
...
```

Closes #7298

NO_DOC=bugfix

If vy_point_lookup called by vy_sauash_process yields (doing disk read),
a dump may be triggered bumping the L0 generation counter, in which case
we would insert a statement to a sealed vy_mem, as explained in #5080.
Let's check the generation counter and rotate the active vy_mem if
necessary after vy_point_lookup to avoid that.

Closes #5080

NO_DOC=bug fix
NO_TEST=complicated, need stress/perf test to catch bugs like this

We often call vy_lsm_rotate_mem_if_required if its generation or schema
version is older than the current one. Let's add a helper function for
that.

Needed for #5080

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The optimization is mostly useless, because it only works if there's no
data on disk. As explained in #5080, it contains a potential bug: if L0
dump is triggered between 'prepare' and 'commit', it will insert a
statement to a sealed vy_mem. Let's drop it.

Part of #5080

NO_DOC=bug fix
NO_CHANGELOG=later

gh_6634_different_log_on_tuple_new_and_free_test.lua verifies that
proper debug message gets into logs for tuple_new() and tuple_delete():
occasionally tuple_delete() printed wrong tuple address. However, still
there are two debug logs: one in tuple_delete() and another one in
memtx_tuple_delete(). So to avoid any possible confusions let's fix
regular expression to find proper log so that now it definitely finds
memtx_tuple_delete().

NO_CHANGELOG=<Test fix>
NO_DOC=<Test fix>

Net.box triggers (on_connect, on_schema_reload) are executed
by the net.box connection worker fiber so a request issued by
a trigger callback can't be processed until the trigger returns
execution to the net.box fiber. Currently, an attempt to issue
a synchronous request from a net.box trigger leads to a silent
hang of the connection, which is confusing. Let's instead raise
an error until #7291 is implemented.

We need to add the check to three places in the code:
 1. luaT_netbox_wait_result for future:wait_result()
 2. luaT_netbox_iterator_next for future:pairs()
 3. conn._request for all synchronous requests.
    (We can't add the check to luaT_netbox_transport_perform_request,
    because conn._request may also call conn.wait_state, which would
    hang if called from on_connect or on_schema_reload trigger.)

We also add an assertion to netbox_request_wait to ensure that we
never wait for a request completion in the net.box worker fiber.

Closes #5358

@TarantoolBot document
Title: Synchronous requests are not allowed in net.box triggers

An attempt to issue a synchronous request (e.g. `call`) from
a net.box trigger (`on_connect`, `on_schema_reload`) now raises
an error: "Synchronous requests are not allowed in net.box trigger"
(Before https://github.com/tarantool/tarantool/issues/5358 was
fixed, it silently hung.)

Invoking an asynchronous request (see `is_async` option) is allowed,
but the request will not be processed until the trigger returns and
an attempt to wait for the request completion with `future:pairs()`
or `future:wait_result()` will raise the same error.

`wal_write` has been adapted to spurious wakeups, so this protection is
no longer needed (see 4bf52367).

Part of #7166

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Currently it's possible to wakeup a fiber, which is waiting on
`limbo->wait_cond`, using Tarantool C API, so the protection
by `fiber_set_cancellable` doesn't make much sense. This patch
removes it. Spurious wakeups are already handled correctly.

Part of #7166

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Part of #7166

NO_DOC=fix comment
NO_TEST=fix comment
NO_CHANGELOG=fix comment

Procedure name cache was removed during refactoring in 8ce76364 based
off the interpretation that libunwind caches procedure names internally —
turned out it only caches unwinding info, which causes a severe performance
downgrade (#7207): return it to speed up frame resolving.

Closes #7207

NO_DOC=performance improvement
NO_TEST=performance improvement

For the sake of maximizing backtrace collection performance, build
the libunwind project submodule with "-O2" compiler flag.

Also, build it with the "-g" compiler flag just in case to simplify
debugging.

Last but not least, pass the same archive-maintaining program used by the
main CMake project to the libunwind project submodule to make the build
homogeneous.

Needed for #7207

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

exclude_null is a special index option, which makes the index ignore
tuples that contain null in any of the indexed fields. Currently, it
doesn't work for json and multikey indexes, because:
 1. index_filter_tuple ignores json path.
 2. index_filter_tuple ignores multikey index.

Issue no. 1 is easy to fix - we just need to use tuple_field_by_part
instead of tuple_field when checking if a key field is null.

Issue no. 2 is more complicated, because when we call index_filter_tuple
we don't know the multikey index. We address this issue by pushing the
index_filter_tuple call down to engine-specific index implementation.

For Vinyl, we make vy_stmt_foreach_entry, which iterates over multikey
tuple entries, skip entries that contain nulls.

For memtx, we move the check to index-specific index_replace function
implementation.  Fortunately, only tree indexes support nullable fields
so we just need to update the memtx tree implementation.

Ideally, we should handle multikey indexes in memtx at the top level,
because the implementation should essentially be the same for all kinds
of indexes, but this refactoring is complicated and will be done later.
For now, just fix the bug.

Closes #5861

NO_DOC=bug fix

For some reason, some test cases create memtx spaces irrespective of
the value of the engine parameter.

NO_DOC=test
NO_CHANGELOG=test

We may need the full (old) tuple to add some of its fields to WAL entry.

NO_CHANGELOG=<Later for EE>
NO_DOC=<Later for EE>
NO_TEST=<Later for EE>

NO_CHANGELOG=<No functional changes>
NO_DOC=<Later for EE>

These fields correspond to the tuple before DML request is executed
(old); and after - result (new). For example let index stores
tuple {1, 1}:
replace{1, 2} -- old == {1, 1}, new == {1, 2}

These fields rather make sense for update operation, which holds
a key and an array of update operations (not the old tuple).

`old_tuple`, `new_tuple` are going to be used as WAL extensions available
in enterprise version. Alongside with it let's reserve 0x2c and 0x2d
Iproto keys for these members.

NO_DOC=<No functional changes>
NO_TEST=<No functional changes>
NO_CHANGELOG=<No functional changes>

This marco is not set so the code depending on it is dropped.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This macro is not set, so unused code is dropped.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This macro does nothing, so it is dropped.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

These macros do nothing, so they are dropped.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring