When we rollback a prepared statement that deletes an MVCC story, we need
to reset the deleted story's PSN.

Closes #7930

NO_DOC=bugfix

During transaction rollback, we unconditionally assign a PSN to it: we
should do this only when necessary, i.e., a transaction is RW and is not
already prepared.

Needed for #7930

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Currently, if transaction preparation fails, the transaction is left in an
inconsistent state: it has a PSN assigned to it, but its status is not
'prepared' — fix this by resetting its PSN.

Needed for #7930

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

During preparation of insert statements in MVCC, we define an old story and
abort all transactions that delete this story.

If there exists an older
story in the history chain, but the story is deleted by a prepared (not
necessarily committed) transaction, we consider that it de-facto does not
exist anymore — this logic is consistent, since during preparation of the
transaction deleting this story, the conflict resolution described above
was already done.

In this manner, there can be no more than one prepared statement deleting
a story at any point in time.

Closes #8104

NO_DOC=bugfix

Changelog: https://curl.se/changes.html#7_87_0

New release contains fixes for 7 security problems [1]:

https://curl.se/docs/CVE-2022-35252.html
https://curl.se/docs/CVE-2022-32221.html
https://curl.se/docs/CVE-2022-35260.html
https://curl.se/docs/CVE-2022-42915.html
https://curl.se/docs/CVE-2022-42916.html
https://curl.se/docs/CVE-2022-43551.html
https://curl.se/docs/CVE-2022-43552.html

Patch adds a new option ENABLE_WEBSOCKETS defined in curl build
infrastructure with it's default value used in 7.87.0.

1. https://curl.se/docs/releases.html

NO_DOC=libcurl submodule bump
NO_TEST=libcurl submodule bump

Closes #8150

Curl 7.87 uses CMake's keywords (for example GREATER_EQUAL [0])
available since CMake 3.7. However, we are still supporting old Ubuntu
version where CMake version lower than 3.7 is used. This patch adds a
script that enables CMake repository with newer CMake packages for
Ubuntu 16.04 (Xenial) and bumps a required version of CMake.

NOTE (regarding cmake3 package):

Commit 1a62d874 ("build: update CMake minimum version to 3.1") [1]
added an additional package requirement with "cmake3". This package has
been created in addition to package "cmake", because CMake 3 had
breaking changes [2]. Package "cmake3" has been provided only in Ubuntu
14.04 LTS (Trusty Tahr) [4], that will be EOLed in 2024, and CentOS 7,
that was EOLed in Aug 2020 and will have end of security support in Jun
2024 [5]. Latest version of package "cmake3" for Ubuntu 14.04 is 3.5.1
[3], so it is not worth to bump version of cmake3 in requirements and I
left it the same.

0. https://cmake.org/cmake/help/latest/command/if.html#greater-equal
1. https://github.com/tarantool/tarantool/commit/1a62d874db5f4780da5b35b6d4d0e3a296148920
2. https://cmake.org/cmake/help/latest/release/3.0.html#id4
3. https://launchpad.net/ubuntu/trusty/+package/cmake3
4. https://ubuntu.com/about/release-cycle
5. https://wiki.centos.org/About/Product

Needed for #8150

NO_CHANGELOG=see the next commit
NO_DOC=libcurl submodule bump
NO_TEST=libcurl submodule bump

Before the change all tarantool.compat options' tests were running on
the same instance, and because luatest uses multiple threads, they could
interfere with compat configuration at the same time as other test
checks default behavior. This could cause such tests to flack.

Now all options' tests are being run in a separate instance via
server:exec().

Closes #8033

NO_DOC=test fix
NO_CHANGELOG=test fix

In a few places visible to users and in iproto naming the term
"cluster" really means "replicaset". One of those places is a
part of public API - box.iproto.key.CLUSTER_UUID - which is not
yet released.

The commit renames "cluster" in those places as a preparation for
introduction of actual "cluster", like a set of replicasets. It
will start from introduction of cluster name in addition to
replicaset uuid/name.

There are places which still mention 'cluster', but their rename
would be breaking. It will be addressed in scope of a bigger
patchset.

Part of #5029

NO_CHANGELOG=Was not released

@TarantoolBot document
Title: Rename `IPROTO_CLUSTER_UUID` to `IPROTO_REPLICASET_UUID`

This is a name for one of the IProto keys. The key value doesn't
change and the protocol is still backward compatible. But better
rename it to `IPROTO_REPLICASET_UUID`, because in future
`IPROTO_CLUSTER_UUID` will most likely mean a different thing.

Proofread changelogs for 2.11.0-rc, part 2
Fix grammar, punctuation, and wording

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

The event is used by appliers as a better alternative to IPROTO_VOTE.
Besides, event subscribers receive exactly the same payload as the ones
sending IPROTO_VOTE. So there's no need to guard against subscription to
this particular event as long as IPROTO_VOTE isn't guarded.

Follow-up #5272

NO_DOC=no user-visible changes
NO_CHANGELOG=no user-visible changes
NO_TEST=tested by ee

As a result of the commit 98fcd437 ("ci:
add CMAKE_EXTRA_PARAMS to LuaJIT workflow") both <inputs.buildtype> and
<inputs.GC64> parameters have become obsolete. All jobs with LuaJIT
integration testing has started to use these in scope of the commit
tarantool/luajit@5b53850da30e532ced976e95af1f301667a6a272 ("ci: use
CMAKE_EXTRA_PARAMS in LuaJIT integration"). Hence, the value of
<inputs.CMAKE_EXTRA_PARAMS> has to be used to specify the build flavor,
so <inputs.buildtype> and <inputs.GC64> can be dropped later.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Reviewed-by: Yaroslav Lobankov <y.lobankov@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

As a result of the commit 1eb0a696 ("ci:
change runner dispatch for LuaJIT testing") <inputs.host> parameter has
become obsolete. The testing workflow has been updated in scope of the
commit tarantool/luajit@fcaecf8fb42ff8a35582fbd8d034eb6f3b9b5b68 ("ci:
use strategy matrix for integration workflow"). Hence, the only changes
required to finish the transition from <inputs.host> to <inputs.arch> +
<inputs.os> are the following:
* Drop <inputs.host> parameter from the LuaJIT integration workflow
* Make both <inputs.arch> and <inputs.os> parameters obligatory

Besides, there is no need to obtain the kernel name and the machine
hardware name in scope of the separate workflow step, since all info
need to be passed to .test.mk is already passed via workflow inputs.
Anyway, .test.mk need to be adjusted to the values used for the new
workflow parameters.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Reviewed-by: Yaroslav Lobankov <y.lobankov@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

The function remove_root_directory, which is used for obtaining module
names for per-module logging, throws an error when current working
directory is `/'.
Rewrite it to fix the bug and rename it to strip_cwd_from_path to make
the name more clear.

Closes #8158

NO_DOC=unreleased
NO_CHANGELOG=unreleased

See the docbot request for details.

Closes #5272

@TarantoolBot document
Title: new `bootstrap_strategy` configuration option

Default behaviour of replica set bootstrap, replica recovery when
connecting to remote nodes and replication reconfiguration is changed.
The new behaviour is controlled by the option `bootstrap_strategy`,
which has the default value "auto".

Now `replication_connect_quorum` configuration option takes no effect,
and the effective quorum value for each stage of configuration (quorum
of established connections, quorum of synced nodes) is determined
automatically.

On replica set bootstrap, the nodes will refuse to boot, unless a
majority is reached (this would mean replication_connect_quorum = 3,
when #box.cfg.repilcation is 4 or 5, for example, or
replication_connect_quorum = 2, when #box.cfg.replication is 2 or 3).
Moreover, the bootstrap leader will fail to boot unless it sees that
every connected node chose it as the bootstrap leader.

On new replica join to an existing cluster, the replica will fail to
boot only if it couldn't connect to anyone. As long as at least one
connection is established, the replica will try to join like before.

Moreover, the replica will check that its box.cfg.replication table
contains every registered node in the cluster, thus ensuring that it has
tried to connect to everyone and chose the best bootstrap leader
possible.

On replication reconfiguration on a working instance and recovery from
local WAL files, the node will try to connect to everyone specified in
box.cfg.replication. Any number of connections (even no connections)
will be deemed a success, but the replica will stay in orphan mode until
it is synced with everyone connected.

If you wish to return to the old behavior, a deprecated setting
`bootstrap_strategy` = "legacy" is left for now. With
`bootstrap_strategy` = "legacy", the node behaves exactly like before:
quorum for both connection and synchronisation is determined by
`replication_connect_quorum`, and neither bootstrap leader nor joining
replicas perform any additional checks on bootstrap.

The only observable behaviour of non-zero replication_sync_timeout is
that it delays box.cfg{replication=...} return until either the node is
synced with others or the timeout passes.

If the timeout passes without reaching sync, box.cfg{} is exited and the
node enters "orphan" state, in which it can't write anything until
either a reconfiguration happens or replicaset is finally synced.

While the previous box.cfg{} call is running (probably waiting for
replication_sync_timeout), the user can't issue another box.cfg{} call.

So basically, while giving no guarantees that the node exits box.cfg{}
in fully synced state, the timeout makes reconfiguration harder: even if
the user knows that the sync won't be achieved, he will have to wait
until the full timeout passes in order to reconfigure replication.

Let's make the default value of replication_sync_timeout 0 instead of
300 seconds. The user still may set the timeout to whatever he likes.
Besides, we have recently introduced box.ctl.on_recovery_state triggers,
which have a "synced" event, and this is the new recommended way to wait
until the node is synced with others.

Part-of #5272

@TarantoolBot document
Title: Changed default value for `box.cfg.replication_sync_timeout

The default value for `replication_sync_timeout` configuration option
was changed from 300 seconds to 0.

Now the instance appends a list of registered replica set members it
knows of to its ballot.

Prerequisite #5272

NO_CHANGELOG=not user-visible

@TarantoolBot document
Title: New fields in instance's ballot.

Instance's ballot (a response to IPROTO_VOTE sent on replica connect)
receives two new fields:
1) The uuid of the node this instance considers the bootstrap leader.
   Key: IPROTO_BALLOT_BOOTSTRAP_LEADER_UUID = 0x08
   Value: uuid, encoded as 36-byte string (like
   "bfd2b31c-b740-43e5-bf3c-28538a74c9a6").
2) An array of registered replica set members uuids.
   Key: IPROTO_BALLOT_REGISTERED_REPLICA_UUIDS = 0x09
   Value: a MP_ARRAY of uuids, each uuid encoded as a 36-byte string
   (like in an example above).

Note that bootstrap leader uuid is not set when an anonymous replica
registers, because technically it's not performing a bootstrap.

Prerequisite #5272

NO_DOC=appended to next commit's doc request
NO_CHANGELOG=not user-visible

Previously replicas chose the remote master to boot from by comparing
master ballot, which are received in response to IPROTO_VOTE request
right on connection init.

Such information is not enough in some scenarios. For example, when
implementing anonymous replicas and retrying relica join, we had to
restart all connections in order to get the latest ballot information.

Let's change that: make replica subscribe to the built-in
"internal.ballot" event instead of relying on request-response scheme of
IPROTO_VOTE. Now replicas always have up-to-date ballot information and
there is no need to reinitialize connections to update the ballots.

Introduce a new fiber running in tx thread for this purpose: applier
ballot watcher. The fiber subscribes on "internal.ballot" event and
watches it all the time while the connection to master is alive.

In case the master isn't aware of IPROTO_WATCH request or of
"internal.ballot" event, old behaviour is also implemented: ballot
watcher simply waits for IPROTO_VOTE response and exits.

The ballot watcher is started whenever replica tries to connect or
reconnect to the remote master and is cancelled whenever its parent
connection to the master is closed.

We do not put much effort into restarting the fiber and retrying to
connect in case it fails. For now ballot info is only used during
bootstrap, and not trying to keep the fiber alive at all costs
simplifies the code quite a lot.

Later on ballot subscriptions will play a more significant role in
choosing the bootstrap leader: replicas will re-check remote ballots
every now and then during the bootstrap leader election.

Part-of #5272

NO_CHANGELOG=internal change
NO_TEST=tested by existing replication tests
NO_DOC=internal change

Extract common connection initialization code in a helper.

It'll be used in the next commit by auxiliary fibers connecting to the
same master.

Part-of #5272

NO_CHANGELOG=refactoring
NO_TEST=refactoring
NO_DOC=refactoring

Extract ballot body decoding logic from xrow_decode_ballot, it will be
reused to decode "internal.ballot" event in the next commit.

Prerequisite #5272

NO_CHANGELOG=refactoring
NO_TEST=refactoring
NO_DOC=refactoring

Add a new builtin event carrying instance's ballot information (that is,
what this instance would normally send in reply to IPROTO_VOTE request).

The event will be watched by connecting replicas to find the bootstrap
leader.

Prerequisite #5272

NO_DOC=technically user-visible, but not intended for users
NO_CHANGELOG=see NO_DOC

In-scope-of #5272

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

luatest_helpers/cluster module was recently added to luatest trunk and
renamed to replica_set.
Let's update its name everywhere in gh_6260_add_builtin_events_test,
since this test will be amended in the following commits and the new
module name will be used.

In-scope-of #5272

NO_DOC=refactoring
NO_CHANGELOG=refactoring

box.iproto table with iproto features and constants was exported to Lua
in commit fe89aabe ("box: export IPROTO constants and features to
Lua").

Add the table to the whitelist of what's available even before box.cfg.

Prerequisite #5272
Closes #8053

NO_DOC=intermediate state wasn't released, no changes necessary
NO_CHANGELOG=see NO_DOC
NO_TEST=used in next commit's tests

Extract mp_sizeof_ballot_max() and mp_encode_ballot() helpers from
iproto_reply_vote(), since they will be used by builtin
"internal.ballot" event soon.

While I'm at it, fix mp_sizeof_ballot() calculation: add forgotten map
element and replace mp_sizeof_uint(UINt*_MAX) with sizes of actual
values to be encoded.

Prerequisite #5272

NO_CHANGELOG=refactoring
NO_TEST=refactoring
NO_DOC=refactoring

Prompt bookkeeping introduced in 66ca6252 is not thread-safe, whilst the
logging environment is multithreaded: leave this feature only in main
(transaction) thread.

Closes #8124

NO_CHANGELOG=<gh-7169 was not release yet>
NO_DOC=bugfix
NO_TEST=<hard to make Tarantool flood log from multiple threads using
         current test harness>

Both of the callbacks in the `print` wrapper are expected to be called, but
`print` may throw errors, e.g.,
`print(setmetatable({}, {__tostring = error})`, so we need to call it in a
protected environment and execute the 'after' callback even if `print`
throws.

Closes #8136

NO_CHANGELOG=<gh-7186 was not released yet>
NO_DOC=bugfix

The EE CMakeConfig.txt adds CE as a sub-directory. For this to work,
we should use project dir instead of cmake dir in CE cmake files,
see commit d8097325 ("cmake: align folders dependencies").

Fixes commit 14f93aee ("libunwind: improve incremental
build/rebuild").

Follow-up #8019

NO_DOC=build fix
NO_TEST=build fix
NO_CHANGELOG=build fix

ICU symbol renaming was disabled in EE build by commit
https://github.com/tarantool/tarantool-ee/commit/f51346d682e3afd93592023d0dedfb1e45167c7a
("static-build: disable symbols renaming for libicu"), because EE build
exports ICU symbols so that they can be used by Lua modules. It isn't
necessary in CE build, but since we're planning to reuse the CE cmake
config in the EE repository, we should do that.

Needed for https://github.com/tarantool/tarantool-ee/issues/185

NO_DOC=no functional changes
NO_TEST=no functional changes
NO_CHANGELOG=no functional changes

Split it so that it can be reused in the EE repository:

 - static-build/cmake/AddDependencyProjects.cmake
   Adds the external projects that are required to build tarantool.
   The project names are stored in the TARANTOOL_DEPENDS variable.

 - static-build/cmake/AddTarantoolProject.cmake
   Should be called after AddDependencyProjects.cmake, because it
   uses the TARANTOOL_DEPENDS variable. Adds the Tarantool external
   project and sets the TARANTOOL_BINARY to the path to the built
   tarantool binary.

 - static-build/cmake/AddTests.cmake
   Should be called after AddTarantoolProject.cmake, because it uses
   the TARANTOOL_BINARY variable. Adds cmake tests for the static
   binary.

Now, static-build/CMakeLists.txt just includes the three helper files.
The helper files are designed in such a way that they can be included
from the EE repository's CMakeLists.txt. We split the original config
into the three helper files, because in the EE repository, we need to
add extra dependency projects and extra tests.

While we are at it, we also move the cmake tests from
static-build/test/static-build to static-build/test and
static-build/test/CheckDependencies.cmake to
static-build/cmake/CheckDependencies.cmake.

This commit introduces no functional changes - it just moves the code.

Needed for https://github.com/tarantool/tarantool-ee/issues/185

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Currently we use ExternalProject_Add to build libunwind subproject. The
usecases it supports are not quite aligned with our. Configuration and
build steps are rerun in case download step sees updated sources. But
we use git submodules to fetch third-party projects and thus subproject
is not rebuild when it's files are changed on disk. Either because of
we doing some sort of experiments locally or because the new sources
are brought by git pull.

Actually libunwind has a CMakeList.txt so that we could try to build it
just as project subdirectory. But it requires quite a fresh CMake
version, currently supports only build on Visual Studio and probably is
not up to date given it's package version variables. Keeping
CMakeLists.txt in sync with main autotools build is additional
maintenance burden. So I'd like to use main autotools build.

So ext_project_autotools function is added to provide nicer build
integration with third-party projects which are build using autotools.
Actually incremental rebuild works only if CMake is at least 3.12 but
I think this should be true for developer installations.

Hopefully we can reuse the function in case we need to bundle more
subprojects with autotools builds.

Follow-up #5665

NO_DOC=build improvement
NO_TEST=build improvement
NO_CHANGELOG=build improvement

Every libunwind error during backtrace collection is reported with
`say_error`. Since commit 19abfd2a ("misc: get rid of fiber_gc")
backtraces are collected on each fiber gc allocation, of which there are
plenty.

For some reason (https://github.com/tarantool/tarantool/issues/7980)
each unw_step fails on mac, and an error is spammed to instance logs,
even though the backtrace is actually collected.

Silence the errors, since there is no much use for them anyway. And
silence all of them just to be consistent.

This doesn't close #7980, because that issue still needs a proper fix.
Although its severity is ameliorated now.

In-scope-of #7980

NO_DOC=bugfix
NO_CHANGELOG=bugfix
NO_TEST=nothing to test

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

Fix grammar, wording, and formatting

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

Fix grammar, wording, and formatting

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

* ci: use strategy matrix for integration workflow
* ci: change runner dispatch for LuaJIT testing

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
NO_CHANGELOG=ci

lbox_push_event_f and lbox_push_event_f callback functions used for
passing the statement between txn and space on/before replace Lua
triggers don't assume that the transaction may be aborted by yield
after the current statement began (this may happen if a trigger callback
yields). In this case, all statements in txn would be rolled back and
txn_current_stmt would return NULL, leading to a crash.

Let's fix this by checking if the transaction is still active and
raising an error immediately if it isn't, thus skipping Lua triggers.

Notes:
 - We merged lbox_pop_txn_stmt_and_check_format into lbox_pop_txn_stmt,
   because the latter is only called by the former.
 - Since lbox_push_event_f callback may now fail, we have to update
   lbox_trigger_run to handle it.

Closes #8027

NO_DOC=bug fix

A remote space object presented by a net.box connection mimics the API
of a local space object presented by box.space. Currently, it misses
information about sequences. Let's add it.

Note, we have to handle the case when the recently introduced
_vspace_sequence system space view is missing on the remote host.
To check that this works correctly, we reuse the 2.10.4 test data
created by commit 1c33484d ("box: add auth_history and
last_modified fields to _user space"). We also add the 'gen.lua'
that can be used to regenerated the data.

Closes #7858

NO_DOC=bug fix

Note, this patch will be backported to 2.10 so we add upgrade function
for 2.10.5, not for 2.11.0.

Needed for #7858

@TarantoolBot document
Title: Document `_space_sequence` and `_vspace_sequence` system spaces

The `_space_sequence` system space was added long time ago (in 1.7.5)
along with the `_sequence` and `_sequence_data` system spaces, but it
was never documented. The space is used to attach sequences to spaces
and has the following fields:
 1. 'id', type 'unsigned'. Space id.
 2. 'sequence_id', type 'unsigned'. Id of the attached sequence.
 3. 'is_generated', type 'boolean'. True if the sequence was created
    automatically (`space:create_index('pk', {sequence = true})`)
 4. 'field', type 'unsigned'. Id of the space field that is set using
    the attached sequence.
 5. 'path', type 'string'. Path to the data within the field that is set
    using the attached sequence.

The `_vspace_sequence` is a system space view of the `_space_sequence`
space that, like any other system space view, shows only rows accessible
by the current user. It will be introduced in Tarantool 2.10.5.

Since commit 85ebbcc0 ("box: reset system space formats for
bootstrap"), it's illegal to use field names in the upgrade script.
The commit missed one place: upgrade of collation strength. Fix it.

NO_DOC=unreleased
NO_CHANGELOG=unreleased
NO_TEST=checked by upgrade script tests