Note that bootstrap leader uuid is not set when an anonymous replica
registers, because technically it's not performing a bootstrap.

Prerequisite #5272

NO_DOC=appended to next commit's doc request
NO_CHANGELOG=not user-visible

Previously replicas chose the remote master to boot from by comparing
master ballot, which are received in response to IPROTO_VOTE request
right on connection init.

Such information is not enough in some scenarios. For example, when
implementing anonymous replicas and retrying relica join, we had to
restart all connections in order to get the latest ballot information.

Let's change that: make replica subscribe to the built-in
"internal.ballot" event instead of relying on request-response scheme of
IPROTO_VOTE. Now replicas always have up-to-date ballot information and
there is no need to reinitialize connections to update the ballots.

Introduce a new fiber running in tx thread for this purpose: applier
ballot watcher. The fiber subscribes on "internal.ballot" event and
watches it all the time while the connection to master is alive.

In case the master isn't aware of IPROTO_WATCH request or of
"internal.ballot" event, old behaviour is also implemented: ballot
watcher simply waits for IPROTO_VOTE response and exits.

The ballot watcher is started whenever replica tries to connect or
reconnect to the remote master and is cancelled whenever its parent
connection to the master is closed.

We do not put much effort into restarting the fiber and retrying to
connect in case it fails. For now ballot info is only used during
bootstrap, and not trying to keep the fiber alive at all costs
simplifies the code quite a lot.

Later on ballot subscriptions will play a more significant role in
choosing the bootstrap leader: replicas will re-check remote ballots
every now and then during the bootstrap leader election.

Part-of #5272

NO_CHANGELOG=internal change
NO_TEST=tested by existing replication tests
NO_DOC=internal change

Extract common connection initialization code in a helper.

It'll be used in the next commit by auxiliary fibers connecting to the
same master.

Part-of #5272

NO_CHANGELOG=refactoring
NO_TEST=refactoring
NO_DOC=refactoring

Extract ballot body decoding logic from xrow_decode_ballot, it will be
reused to decode "internal.ballot" event in the next commit.

Prerequisite #5272

NO_CHANGELOG=refactoring
NO_TEST=refactoring
NO_DOC=refactoring

Add a new builtin event carrying instance's ballot information (that is,
what this instance would normally send in reply to IPROTO_VOTE request).

The event will be watched by connecting replicas to find the bootstrap
leader.

Prerequisite #5272

NO_DOC=technically user-visible, but not intended for users
NO_CHANGELOG=see NO_DOC

In-scope-of #5272

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

luatest_helpers/cluster module was recently added to luatest trunk and
renamed to replica_set.
Let's update its name everywhere in gh_6260_add_builtin_events_test,
since this test will be amended in the following commits and the new
module name will be used.

In-scope-of #5272

NO_DOC=refactoring
NO_CHANGELOG=refactoring

box.iproto table with iproto features and constants was exported to Lua
in commit fe89aabe ("box: export IPROTO constants and features to
Lua").

Add the table to the whitelist of what's available even before box.cfg.

Prerequisite #5272
Closes #8053

NO_DOC=intermediate state wasn't released, no changes necessary
NO_CHANGELOG=see NO_DOC
NO_TEST=used in next commit's tests

Extract mp_sizeof_ballot_max() and mp_encode_ballot() helpers from
iproto_reply_vote(), since they will be used by builtin
"internal.ballot" event soon.

While I'm at it, fix mp_sizeof_ballot() calculation: add forgotten map
element and replace mp_sizeof_uint(UINt*_MAX) with sizes of actual
values to be encoded.

Prerequisite #5272

NO_CHANGELOG=refactoring
NO_TEST=refactoring
NO_DOC=refactoring

Prompt bookkeeping introduced in 66ca6252 is not thread-safe, whilst the
logging environment is multithreaded: leave this feature only in main
(transaction) thread.

Closes #8124

NO_CHANGELOG=<gh-7169 was not release yet>
NO_DOC=bugfix
NO_TEST=<hard to make Tarantool flood log from multiple threads using
         current test harness>

Both of the callbacks in the `print` wrapper are expected to be called, but
`print` may throw errors, e.g.,
`print(setmetatable({}, {__tostring = error})`, so we need to call it in a
protected environment and execute the 'after' callback even if `print`
throws.

Closes #8136

NO_CHANGELOG=<gh-7186 was not released yet>
NO_DOC=bugfix

The EE CMakeConfig.txt adds CE as a sub-directory. For this to work,
we should use project dir instead of cmake dir in CE cmake files,
see commit d8097325 ("cmake: align folders dependencies").

Fixes commit 14f93aee ("libunwind: improve incremental
build/rebuild").

Follow-up #8019

NO_DOC=build fix
NO_TEST=build fix
NO_CHANGELOG=build fix

ICU symbol renaming was disabled in EE build by commit
https://github.com/tarantool/tarantool-ee/commit/f51346d682e3afd93592023d0dedfb1e45167c7a
("static-build: disable symbols renaming for libicu"), because EE build
exports ICU symbols so that they can be used by Lua modules. It isn't
necessary in CE build, but since we're planning to reuse the CE cmake
config in the EE repository, we should do that.

Needed for https://github.com/tarantool/tarantool-ee/issues/185

NO_DOC=no functional changes
NO_TEST=no functional changes
NO_CHANGELOG=no functional changes

Split it so that it can be reused in the EE repository:

 - static-build/cmake/AddDependencyProjects.cmake
   Adds the external projects that are required to build tarantool.
   The project names are stored in the TARANTOOL_DEPENDS variable.

 - static-build/cmake/AddTarantoolProject.cmake
   Should be called after AddDependencyProjects.cmake, because it
   uses the TARANTOOL_DEPENDS variable. Adds the Tarantool external
   project and sets the TARANTOOL_BINARY to the path to the built
   tarantool binary.

 - static-build/cmake/AddTests.cmake
   Should be called after AddTarantoolProject.cmake, because it uses
   the TARANTOOL_BINARY variable. Adds cmake tests for the static
   binary.

Now, static-build/CMakeLists.txt just includes the three helper files.
The helper files are designed in such a way that they can be included
from the EE repository's CMakeLists.txt. We split the original config
into the three helper files, because in the EE repository, we need to
add extra dependency projects and extra tests.

While we are at it, we also move the cmake tests from
static-build/test/static-build to static-build/test and
static-build/test/CheckDependencies.cmake to
static-build/cmake/CheckDependencies.cmake.

This commit introduces no functional changes - it just moves the code.

Needed for https://github.com/tarantool/tarantool-ee/issues/185

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Currently we use ExternalProject_Add to build libunwind subproject. The
usecases it supports are not quite aligned with our. Configuration and
build steps are rerun in case download step sees updated sources. But
we use git submodules to fetch third-party projects and thus subproject
is not rebuild when it's files are changed on disk. Either because of
we doing some sort of experiments locally or because the new sources
are brought by git pull.

Actually libunwind has a CMakeList.txt so that we could try to build it
just as project subdirectory. But it requires quite a fresh CMake
version, currently supports only build on Visual Studio and probably is
not up to date given it's package version variables. Keeping
CMakeLists.txt in sync with main autotools build is additional
maintenance burden. So I'd like to use main autotools build.

So ext_project_autotools function is added to provide nicer build
integration with third-party projects which are build using autotools.
Actually incremental rebuild works only if CMake is at least 3.12 but
I think this should be true for developer installations.

Hopefully we can reuse the function in case we need to bundle more
subprojects with autotools builds.

Follow-up #5665

NO_DOC=build improvement
NO_TEST=build improvement
NO_CHANGELOG=build improvement

Every libunwind error during backtrace collection is reported with
`say_error`. Since commit 19abfd2a ("misc: get rid of fiber_gc")
backtraces are collected on each fiber gc allocation, of which there are
plenty.

For some reason (https://github.com/tarantool/tarantool/issues/7980)
each unw_step fails on mac, and an error is spammed to instance logs,
even though the backtrace is actually collected.

Silence the errors, since there is no much use for them anyway. And
silence all of them just to be consistent.

This doesn't close #7980, because that issue still needs a proper fix.
Although its severity is ameliorated now.

In-scope-of #7980

NO_DOC=bugfix
NO_CHANGELOG=bugfix
NO_TEST=nothing to test

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

Fix grammar, wording, and formatting

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

Fix grammar, wording, and formatting

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

* ci: use strategy matrix for integration workflow
* ci: change runner dispatch for LuaJIT testing

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
NO_CHANGELOG=ci

lbox_push_event_f and lbox_push_event_f callback functions used for
passing the statement between txn and space on/before replace Lua
triggers don't assume that the transaction may be aborted by yield
after the current statement began (this may happen if a trigger callback
yields). In this case, all statements in txn would be rolled back and
txn_current_stmt would return NULL, leading to a crash.

Let's fix this by checking if the transaction is still active and
raising an error immediately if it isn't, thus skipping Lua triggers.

Notes:
 - We merged lbox_pop_txn_stmt_and_check_format into lbox_pop_txn_stmt,
   because the latter is only called by the former.
 - Since lbox_push_event_f callback may now fail, we have to update
   lbox_trigger_run to handle it.

Closes #8027

NO_DOC=bug fix

A remote space object presented by a net.box connection mimics the API
of a local space object presented by box.space. Currently, it misses
information about sequences. Let's add it.

Note, we have to handle the case when the recently introduced
_vspace_sequence system space view is missing on the remote host.
To check that this works correctly, we reuse the 2.10.4 test data
created by commit 1c33484d ("box: add auth_history and
last_modified fields to _user space"). We also add the 'gen.lua'
that can be used to regenerated the data.

Closes #7858

NO_DOC=bug fix

Note, this patch will be backported to 2.10 so we add upgrade function
for 2.10.5, not for 2.11.0.

Needed for #7858

@TarantoolBot document
Title: Document `_space_sequence` and `_vspace_sequence` system spaces

The `_space_sequence` system space was added long time ago (in 1.7.5)
along with the `_sequence` and `_sequence_data` system spaces, but it
was never documented. The space is used to attach sequences to spaces
and has the following fields:
 1. 'id', type 'unsigned'. Space id.
 2. 'sequence_id', type 'unsigned'. Id of the attached sequence.
 3. 'is_generated', type 'boolean'. True if the sequence was created
    automatically (`space:create_index('pk', {sequence = true})`)
 4. 'field', type 'unsigned'. Id of the space field that is set using
    the attached sequence.
 5. 'path', type 'string'. Path to the data within the field that is set
    using the attached sequence.

The `_vspace_sequence` is a system space view of the `_space_sequence`
space that, like any other system space view, shows only rows accessible
by the current user. It will be introduced in Tarantool 2.10.5.

Since commit 85ebbcc0 ("box: reset system space formats for
bootstrap"), it's illegal to use field names in the upgrade script.
The commit missed one place: upgrade of collation strength. Fix it.

NO_DOC=unreleased
NO_CHANGELOG=unreleased
NO_TEST=checked by upgrade script tests

If the first argument of box.atomic is a non-callable table then
consider it as options table for box.begin{}.

For test and debug purposes introduce internal getter of current
transaction isolation level as box.internal.txn_isolation().

Closes #7202

@TarantoolBot document
Title: Options in box.atomic

Now it's allowed to pass transaction options in the first argument
of box.atomic(..) call. The options must be a table, exactly as
in box.begin(..). If options are passed as the first arguments,
the second and the rest arguments are expected to be a functions
and its arguments, like in usual box.atomic.

When a transaction is in read-confirmed state it must ignore all
prepared changes, and if it actually ignores something - it must
fall to read-view state.

By a mistake the check relied not on actual skipping of a prepared
statement, but on the fact that there is a deleting statement. That
leads to excess conflicts for transactions with read-committed
isolation level.

Fix it by raising a conflict only if a deleting statement is skipped.

Closes #8122
Needed for #7202

NO_DOC=bugfix

Read lists (read set and other similar lists) are used only for
detecting a conflict when another transaction is committed.
Once a transaction is prepared (no matter with success or not)
those lists are no more needed. Moreover, in some part of code it
is expected that there can be no read set of already prepared tx.

So let's clean those lists once a transaction is prepared.

Closes #7945

NO_DOC=bugfix

There ware two functions - check_dup_clean and check_dup_dirty.
Merge them to one.

Also extract phantom checks from check_dup and call them explicitly.
That will additionally simplify check_dup and will allow to get rid
of temporary conflict trackers - memtx_tx_conflict. Note that this
kind of object will remain in memory monitoring by now. It will be
removed later.

No logical changes.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

If a transaction writes to a gap that is tracked by some another
transaction, the interval must be (usually) broken into parts,
while new story must be explicitly added to the read set of
reading transaction.

Now both read tracker and conflict trackers are set in this case.
But read tracker is enough in this case - when the writing tx
is prepared it will conflict reading transaction, that's all we
need.

Let's leave only read tracker.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Each story has two ends: the beginning and the end. For each
transaction both ends of a story could be visible or not.

Now there's a function that checks visibility of both ends of
a story. It can distinguish three cases: both ends are visible,
both ends are invisible, and the beginning is visible while the
end is not. The function returns true in the first and the last
cases; the actual case is clarified with an additional function
argument - visible_tuple, which is set to null in one of the
cases..

Let's make two different functions for checking visibility of
the beginning and the end of a story. Actually that is simple
split of function into two parts. The visible_tuple argument
will no longer be needed.

No logical changes.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

There's no harm but also no sense in it.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Hide structures and functions that are not required for API.

No logical changes.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Option memtx_tx_manager_use_mvcc_engine changes the behavior of
transaction execution workflow. Usually that is implemented as
direct check of memtx_tx_manager_use_mvcc_engine. But there are
places in the code that rely on the fact that some pointers are
set to not null if the engine is enabled. That's a bit confusing.

Let's always check for memtx_tx_manager_use_mvcc_engine option
when it's needed to determine which workflow must be executed.

Note that checking of memtx_tx_manager_use_mvcc_engine option is
more correct: in case of delete of nothing (delete statement when
a tuple was not found by given key) all the pointers including
old_tuple and new_tuple are null, while logically we still need
to use mvcc execution workflow. Note also that in this case the
mvcc engine does (and must do) almost nothing, so there was no
bug in the previous behaviour.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

No logical changes.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Since the fiber function is not expected to return (i.e., the inline
assembly is not expected to return), remove the dummy memory operand used
to indicate that the memory pointed to by the inline assembly pointer
operand is clobbered.

Closes #8125

NO_DOC=build diagnostic
NO_CHANGELOG=build diagnostic
NO_TEST=build diagnostic

Before the patch the LuaJIT integration workflow was dispatched to the
runner with the name given via <inputs.host> parameter. Unfortunately,
as a result of runners renaming we can't continue to dispatch the
workflow this way.

As a result of the patch there are two new workflow parameters:
<inputs.arch> to pass the host architecture name (i.e. x86_64 or ARM64)
and <inputs.os> to pass the OS family name (either Linux or macOS).
Considering two values we can choose the proper runner in LuaJIT
integration workflow. Besides, this change bring LuaJIT CI closer to
matrix usage for its integration workflow.

All three workflow parameters are not obligatory for now to avoid
tarantool/luajit CI break on both long-term and working branches. When
all branches are rebased on the new approach, <inputs.host> parameter
will be removed and both <inputs.arch> and <inputs.os> will become
obligatory.

Moreover, the new 'regular' label is also added to <runs-on> list, since
the new "lightweight" runners have been introduced to ghacts-shared-*
pool. There are a couple of LuaJIT tests that requires more memory than
provided by "lightweight" runners, so only "regular" ones need to be
chosen for LuaJIT integration testing.

Last but not least: attentive reader might notice there are strange
values used as a default for <inputs.host> as well as <inputs.arch> and
<inputs.os>. This is ugly hack required for the transition period, since
one can't use empty string or unknown label name within <runs-on> label
list. Hence 'self-hosted' looks like the most robust option for both old
and new behaviours.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Reviewed-by: Yaroslav Lobankov <y.lobankov@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

The status isn't used anywhere - to set the proper error when an aborted
transaction is attempted to be used, we check out transaction flags
(TXN_IS_CONFLICTED, TXN_IS_ABORTED_BY_YIELD, TXN_IS_ABORTED_BY_TIMEOUT).
Let's use TXN_ABORTED instead.

While we are at it, also set the transaction status to TXN_ABORTED when
a transaction is aborted by yield or timeout and use it instead of
checking flags where appropriate, since it's more convenient.

Follow-up #8123

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

We fail write statements if the current transaction was aborted by yield
or timeout. We should fail read-only statements in this case, as well.
Note, we already fail read-only statements if the current transaction
was aborted by conflict.

Closes #8123

NO_DOC=bug fix

Fixed pthread-related CMake checks. The checks code is built with
`-pedantic-errors` and it leads to errors of the following kind on
clang 15:

```
<...>/CMakeFiles/CMakeScratch/TryCompile-78KaOK/src.c:4:17: error: a
    function declaration without a prototype is deprecated in all
    versions of C [-Werror,-Wstrict-prototypes]
        int main() { pthread_setname_np(pthread_self(), ""); }
                ^
                 void
```

Fixed a warning in the SQL code (it's an error in Debug build):

```
<...>/src/box/sql/vdbeaux.c:170:13: error: variable 'n' set but not used
    [-Werror,-Wunused-but-set-variable]
        static int n = 0;
```

Fixed several warnings from lemon.c of the following kind:

```
<...>/extra/lemon.c:173:6: warning: a function declaration without a
    prototype is deprecated in all versions of C and is treated as a
    zero-parameter prototype in C2x, conflicting with a subsequent
    definition [-Wdeprecated-non-prototype]
void FindRulePrecedences();
     ^
<...>/extra/lemon.c:766:6: note: conflicting prototype is here
void FindRulePrecedences(struct lemon *xp)
```

See also https://github.com/tarantool/small/issues/57

Fixes #8110

NO_DOC=build fix
NO_TEST=build fix

Bump the small submodule with the following commit:

* https://github.com/tarantool/small/commit/5c3a0880ae4659cb662085563ff0b9eee40565fe

See https://github.com/tarantool/small/issues/57
Part of #8110

NO_DOC=build fix
NO_TEST=build fix
NO_CHANGELOG=to be added in a commit that fixes #8110