The function `vy_space_build_index`, which builds a new index on DDL,
calls `vy_scheduler_dump` on completion. If there's a checkpoint in
progress, the latter will wait on `vy_scheduler::dump_cond` until
`vy_scheduler::checkpoint_in_progress` is cleared. The problem is
`vy_scheduler_end_checkpoint` doesn't broadcast `dump_cond` when it
clears the flag. Usually, everything works fine because the condition
variable is broadcast on any dump completion, and vinyl checkpoint
implies a dump, but under certain conditions this may lead to a fiber
hang. Let's broadcast `dump_cond` in `vy_scheduler_end_checkpoint`
to be on the safe side.

While we are at it, let's also inject a dump delay to the original
test to make it more robust.

Closes #10267
Follow-up #10234

NO_DOC=bug fix

This commit introduces FETCH_SNAPSHOT_CURSOR feature, which is available
only in EE. The feature is not returned in response to IPROTO_ID and is
not shown in box.iproto.protocol_features in Community Edition. Its id
is shown only in box.iproto.feature, which is a list of all available
features in the current version.

Needed for tarantool/tarantool-ee#741

NO_CHANGELOG=minor

@TarantoolBot document
Title: Document iproto feature FETCH_SNAPSHOT_CURSOR

Root document: https://www.tarantool.io/en/doc/latest/reference/reference_lua/net_box/#net-box-connect

FETCH_SNAPSHOT_CURSOR feature requires cursor FETCH_SNAPSHOT on the
server. Its ID is IPROTO_FEATURE_FETCH_SNAPSHOT_CURSOR. IPROTO version
is 8 or more, Enterprise Edition is also required.

This commit introduces engine stubs that enable a new method
of fetching snapshots for anonymous replicas. Instead of using
the traditional read-view join approach, this update allows
file snapshot fetching. Note that file snapshot fetching
is only available in Tarantool EE.

Checkpoint fetching is done via IPROTO_IS_CHECKPOINT_JOIN,
IPROTO_CHECKPOINT_VCLOCK and IPROTO_CHECKPOINT_LSN fields.

If IPROTO_CHECKPOINT_JOIN is set to true, join will be done from
files: .snap for memtx, .run for vinyl, if false - from read view.

Checkpoint join allows to continue from the place, where client
stopped in case of snapshot fetching error. This allows to avoid
rebootstrap of an anonymous client. This can be done by specifying
CHECKPOINT_VCLOCK, which says from which file server should continue
join, client gets vclock at the beginning of the join. Specifying
CHECKPOINT_LSN allows to continue from some position in checkpoint.
Server sends all data >= CHECKPOINT_LSN.

If CHECKPOINT_VCLOCK is not specified, fetching is done from the latest
available checkpoint. If CHECKPOINT_LSN is not specified - start from
the beginning of the snap. So, specifying only IS_CHECKPOINT_JOIN
triggers fetching the latest checkpoint from files.

Needed for tarantool/tarantool-ee#741

NO_DOC=ee
NO_TEST=ee
NO_CHANGELOG=ee

This commit makes engine to send vclock without ignoring 0th component
during join, which is needed for checkpoint FETCH SNAPSHOT.

Currently engine join functions are invoked only from
relay_initial_join, which is done during JOIN or FETCH SNAPSHOT.
They respond with vclock of the read view we're going to send.

In the following commit checkpoint FETCH SNAPSHOT will be introduced,
which responds with vclock of the checkpoint, we're going to send.
Such vclock may include 0th component and it's crucial to send it to
a client, as in case of connection failure, client will send us the
same vclock and we'll have to use its signature to figure out, which
checkpoint client wants.

So, we have to send and receive 0th component of the vclock during
FETCH_SNAPSHOT. This commit also introduces decoding vclocks without
ignoring 0th component, as they'll be used in the following commit too.

Needed for tarantool/tarantool-ee#741

NO_DOC=internal
NO_TEST=ee
NO_CHANGELOG=internal

This commit renames xrow_encode_vlock to xrow_encode_vclock_ignore0
since the next commit will introduce encoding vclock without ignoring
0th component, which is needed during sending the response to fetch
snapshot request.

This commit also removes internal field inside the replication_request
structure, as the following commit will use 'vclock' for
encoding/decoding vclock without ignoring component.

Needed for tarantool/tarantool-ee#741

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

From now on during initial join memtx engine prepares vclock, raft and
limbo states, it also sends them during memtx_engine_join.

It's done in order to simplify the code of initial join, as in the
consequent commit checkpoint initial join will be introduced and we want
relay code to handle it the same as read-view join without confusing
conditions.

Needed for tarantool/tarantool-ee#741

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Before this commit raft and limbo states were written at the end of the
checkpoint, which makes it very costly to access them.

Checkpoint join needs to access limbo and raft state in order to send
them during JOIN_META stage. We cannot use the latest states, like it's
done for read-view snapshot fetching: states may be far ahead of the
data, written to the checkpoint, which we're going to send.

This commit moves raft and limbo states after data from the system
spaces but before user data. We cannot put them right at the beginning
of the snapshot, because then we'll have to patch recovery process,
which currently strongly relies on the fact, that system spaces are
at the beginning of the snapshot (this was done in order to apply force
recovery only for user data). If we patch recovery process, then old
versions, where it's unpatched, won't be able to recover from the
snapshots done by the newer version, compatibility of snapshots will be
broken.

The current change is not breaking, old Tarantool versions can restore
from the snapshot made by the newer one.

Needed for tarantool/tarantool-ee#741

NO_DOC=internal
NO_CHANGELOG=internal

Fix the following warnings (with ENABLE_READ_VIEW defined):

```
./perf/lua/column_scan_module.c:59:18: error: unused variable ‘index_id’ [-Werror=unused-variable]
   59 |         uint32_t index_id = luaL_checkinteger(L, 2);
      |                  ^~~~~~~~

./perf/lua/column_scan_module.c:149:18: error: unused variable ‘index_id’ [-Werror=unused-variable]
  149 |         uint32_t index_id = luaL_checkinteger(L, 2);
      |                  ^~~~~~~~
```

NO_DOC=perf test
NO_TEST=perf test
NO_CHANGELOG=perf test

After receiving async transaction from an old term applier_apply_tx
exits without unlocking the latch. If the same applier tries to
subscribe for replication, it fails with assertion, as the latch is
already locked.

Let's fix the function, which raises error so that it just sets
diag and returns -1.

Closes #10073

NO_DOC=bugfix
NO_CHANGELOG=no crash on release version

The test creates an empty space with 1000 nullable columns storing uint64
values. Then it initializes a datasets that consists of 10 columns and
1 million rows (row count and both column counts are configurable), then
it inserts the dataset into the space.

By default the test uses serial C API but one may switch to the Arrow API
for batch insertion (the feature is exclusive to the Enterprise Edition).

It's also possible to specify the engine and wal_mode to use (default are
memtx, write).

Needed for tarantool/tarantool-ee#712

NO_DOC=perf test
NO_TEST=perf test
NO_CHANGELOG=perf test

Needed for tarantool/tarantool-ee#712

NO_DOC=for enterprise edition
NO_TEST=for enterprise edition
NO_CHANGELOG=for enterprise edition

They are useful in C modules.

Needed for tarantool/tarantool-ee#712

@TarantoolBot document
Title: Update C API reference > Module lua/utils
Product: Tarantool
Root documents: https://www.tarantool.io/en/doc/latest/dev_guide/reference_capi/utils/

The following functions are missed in the documentation:

 * luaL_iscallable
 * luaL_iscdata
 * luaL_isnull
 * luaL_pushnull
 * luaT_call
 * luaT_checktuple
 * luaT_isdecimal
 * luaT_newdecimal
 * luaT_pushdecimal
 * luaT_toibuf
 * luaT_tolstring
 * luaT_tuple_encode
 * luaT_tuple_new

See also: https://github.com/tarantool/doc/issues/2011

Needed for tarantool/tarantool-ee#712

NO_TEST=EE
NO_DOC=internal
NO_CHANGELOG=internal

Needed for tarantool/tarantool-ee#712

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

There is no much sense in testing it, but it is sensitive to source code
changes, especially `ERRINJ_*_COUNTDOWN` injections, e.g. see commit
697123d0 ("box: use maximal space id instead of _schema.max_id").

Needed for tarantool/tarantool-ee#712

NO_DOC=test
NO_CHANGELOG=test

Now `sio_bind` function prints address into error message directly
instead of relying on `fd` used in `bind` that failed to execute.

`sio_bind` used `sio_socketname_to_buffer` for error message
effectively attempting printing address bound to `fd` while there
actually was an error in binding that address to that socket in the
first place.

Fixes #5925

NO_DOC=bugfix
NO_CHANGELOG=minor

This test checks, that when PROMOTE from the previous term is
encountered we immediately notice split-brain situation and break
replication without corrupting data.

Closes #9943

NO_DOC=test
NO_CHANGELOG=test

Can use the regular applier_apply_tx(), they do the same. The
latter is just more protective, but doesn't matter much in this
case if the code does a few latch locks.

The patch also drops an old test about double-received row panic
during final join. The logic is that absolutely the same situation
could happen during subscribe, but it was always filtered out by
checking replicaset.applier.vclock and skipping duplicate rows.

There doesn't seem to be a reason why final join must be any
different. It is, after all, same subscribe logic but the received
rows go into replica's initial snapshot instead of xlogs. Now it
even uses the same txn processing function applier_apply_tx().

The patch also moves `replication_skip_conflict` option setting
after bootstrap is finished. In theory, final join could deliver
a conflicting row and it must not be ignored. The problem is that
it can't be reproduced anyhow without illegal error injection
(which would corrupt something in an unrealistic way). But lets
anyway move it below bootstrap for clarity.

Follow-up #10113

NO_DOC=refactoring
NO_CHANGELOG=refactoring

No code besides box.cc can now update instance's vclock
explicitly. That is a protection against hacks like #9916.

Closes #10113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The goal is to make sure that no files except box.cc can change
instance_vclock_storage directly. That leads to all sorts of hacks
which in turn lead to bugs - #9916 is a good example.

Now applier on final join only sends rows into the journal. The
journal then is handled by box.cc where vclock is properly
updated.

Part of #10113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The function writes a single xrow into the journal in a blocking
way. It isn't so simple, so makes sense to keep as a function,
especially given that it will be used more in the next commit.

Part of #10113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Recovery journal uses word "recovery" to say that it works with
xlogs. For snapshot recovery there is bootstrap_journal. Lets use
it during local snapshot recovery.

The reasoning is that while right now there is no difference, in
next commits the recovery_journal will do more.

Part of #10113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Storing vclock of the instance in replicaset.vclock wasn't right.
It wasn't vclock of the whole replicaset. It was local to this
instance. There is no such thing as "replicaset vclock".

The patch moves it to box.h/cc.

Part of #10113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Applier during the registration waiting (for registering a new ID
or a name) could keep doing the master txns received before the
registration was started. They could still be inside WAL doing a
disk write, when the replica sends a register request.

Before this commit, it could cause an assertion failure in debug
and a double LSN error in release.

The reason was that during the registration waiting the applier
treated all incoming txns as "final join" txns. I.e. it wasn't
checking if those txns were already received, but not committed
yet.

During normal subscribe process the appliers (potentially
multiple) protect themselves from that by keeping track of the
vclocks which are already applied and also being applied right now
(replicaset.applier.vclock).

Such protection ensures that receiving same row from 2 appliers
wouldn't result into its double write. It also protects from the
case when a txn was received, goes to WAL, but then the applier
reconnects, resubscribes, and gets the same txn again - it
shouldn't be applied.

The patch makes so that the registration waiting after recovery
works like subscribe. Registration during recovery would mean
bootstrap via join. And outside of recovery it means the instance
is already running.

Closes #9916

NO_DOC=bugfix

As this fiber is made system in the commit bf620650 ("box: finish
client fibers on shutdown") we don not need the existing protection from
cancelling. So first remove it. Now make it managed on shutdown.

Note that we may have issues as we finish this fiber too early. The
tasks scheduled but not executed at this moment will never be executed.
So the tasks that be scheduled after fiber is finished. Now when we
don't use worker fiber for swim gc this will not cause leaks. And
leaking fd on Tarantool shutdown in fio is not a problem.

Closes #9722

NO_CHANGELOG=internal
NO_DOC=internal

Let's make sure box raft worker fiber is finished on Tarantool shutdown as
we are going to free fibers stacks. If fiber is not finished it's stack
may have references to objects on heap. Thus as fiber stack will be freed
we will have FP memory leaks under ASAN.

Part of #9722

NO_TEST=rely on existing tests
NO_CHANGELOG=internal
NO_DOC=internal

Let's make sure swim worker fiber is finished on Tarantool shutdown as
we are going to free fibers stacks. If fiber is not finished it's stack
may have references to objects on heap. Thus as fiber stack will be freed
we will have FP memory leaks under ASAN.

Let's make swim gc do not yield using asynchronuos deletion. This way we
will not use worker fiber for swim deletion. We are going to stop this
worker fiber before all swim object are collected.

Part of #9722

NO_CHANGELOG=internal
NO_DOC=internal

Let's make sure net.box system fiber is finished on Tarantool shutdown as
we are going to free fibers stacks. If fiber is not finished it's stack
may have references to objects on heap. Thus as fiber stack will be freed
we will have FP memory leaks under ASAN.

Part of #9722

NO_CHANGELOG=internal
NO_DOC=internal

Such fibers are cancelled during Tarantool shutdown the same way as
cancelled client fibers. This is internal API.

Part of #9722

NO_CHANGELOG=internal
NO_DOC=internal

Fiber pool shutdown is finishing all idle fibers. Any message processing
is finished earlier on client fiber shutdown.

We need some changes in shutdown order to make fiber pool shutdown.
First we need to move stopping of iproto threads from free to shutdown
step. The issue is we want to destroy "tx" endpoint which requires all
pipes connected to it to be destroyed first. There are pipes in iproto
threads that connected to "tx". Currently we destroy pipes in free step
and at this point as there is no event loop in tx thread
`cbus_endpoint_destroy` can't receive notifications that pipes are
destroyed.

Originally we put stopping of iproto threads to the free step because we
don't have client fibers shutdown. So it was convenient to have working
`net_pipe` so that client fibers can use iproto API without adding extra
logic to them. Now I guess it make sense to stop client fibers before
iproto shutdown. This is the second change in shutdown order.

There is another reason why we have iproto shutdown before client fiber
shutdown. In the process of iproto shutdown we close connections first
and then cancel all requests in progress. This way client do not receive
unexpected `FiberIsCancelled` errors in the process of server shutdown.
After the patch it not so. Well we may close connections as an extra
step before client fibers shutdown. But let's leave it this way.  Good
clients can subscribe to servere shutdown and prepare for it.  Otherwise
they may receive `FiberIsCancelled` for theier request which looks
sensible.

It is also makes sense now to move watcher and client fiber shutdown
to `box_shutdown` as we can both use watcher and create client fibers
without starting a box.

While at it also drop a note in code why we shutdown watcher before even
fiber clients shutdown.

Part of #9722

NO_CHANGELOG=internal
NO_DOC=internal

As we are going to check for memory leaks in ASAN we need to wait while
memtx gc finishes freeing tuples of dropped primary index or keys of
dropped functional indexes.

Part of #9722

NO_CHANGELOG=internal
NO_DOC=internal

It it same as ERROR_INJECT_YIELD() but if fiber is cancelled it will
execute given code.

Part of #9722

NO_TEST=internal
NO_CHANGELOG=internal
NO_DOC=internal

There may be more than one fiber waiting on `vy_scheduler::dump_cond`:

```
box.snapshot
  vinyl_engine_wait_checkpoint
    vy_scheduler_wait_checkpoint

space.create_index
  vinyl_space_build_index
    vy_scheduler_dump
```

To avoid hang, we should use `fiber_cond_broadcast`.

Closes #10233

NO_DOC=bug fix

This patch bumped small to the new version that does not trigger
UBSan with *_entry* macros and should support new oss-fuzz builder.

New commits:

* rlist: make its methods accept const arguments
* lsregion: introduce lsregion_to_iovec method
* rlist: make foreach_enrty_* macros not to use UB

Fixes: #10143

NO_DOC=small submodule bump
NO_TEST=small submodule bump
NO_CHANGELOG=small submodule bump

Changed default tarantool `offsetof` macro implementation so it don't
access members of null pointer in typeof that triggers UBsan.

Needed for #10143

NO_DOC=bugfix
NO_CHANGELOG=minor
NO_TEST=tested manually with fuzzer

This patch completely relaxes UUID checks and accepts an arbitrary
128-bit sequence as an UUID for binary data. String representations
still should match the grammars in RFC 4122, Section 3 [1] and RFC 9562,
Section 4 [2].

[1]: https://datatracker.ietf.org/doc/html/rfc4122#section-3
[2]: https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-format

Closes #5444

@TarantoolBot document
Title: uuid: relaxed UUID validation

[The UUID module documentation][1] mentions that Tarantool generates
UUIDs following the rules for RFC 4122,[version 4, variant 1][2]. It is
worth mentioning that the user can store an arbitrary 128-bit sequence
as an UUID for binary data. String representations still should match
the grammars in RFC 4122, [Section 3][3], and RFC 9562, [Section 4][4].

[1]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/uuid/
[2]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)
[3]: https://datatracker.ietf.org/doc/html/rfc4122#section-3
[4]: https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-format

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

* Correct fix for stack check when recording BC_VARG.
* test: remove inline suppressions of _TARANTOOL
* FFI: Fix ffi.alignof() for reference types.
* FFI: Fix sizeof expression in C parser for reference types.
* FFI: Allow ffi.metatype() for typedefs with attributes.
* FFI: Fix ffi.metatype() for non-raw types.
* Maintain chain invariant in DCE.
* build: introduce option LUAJIT_ENABLE_TABLE_BUMP
* ci: add tablebump flavor for exotic builds
* test: allow `jit.parse` to return aborted traces
* Handle all types of errors during trace stitching.
* Use generic trace error for OOM during trace stitching.
* Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check.
* cmake: set cmake_minimum_required only once
* cmake: fix warning about minimum required version
* ci: add a workflow for testing with AVX512 enabled
* test: introduce a helper read_file
* OSX/iOS/ARM64: Fix generation of Mach-O object files.
* OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
* build: introduce LUAJIT_USE_UBSAN option
* ci: enable UBSan for sanitizers testing workflow
* cmake: add the build directory to the .gitignore
* Prevent sanitizer warning in snap_restoredata().
* Avoid negation of signed integers in C that may hold INT*_MIN.
* Show name of NYI bytecode in -jv and -jdump.

Closes #9924
Closes #8473

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

In this case join will just hang. Instead let's raise an error in case
of Lua API and panic in case of C API.

Closes #10196

NO_DOC=minor