Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Updated mhash.test and mhash_bytemap.test to
activate TAP-compatible mode.

Part of #7618
NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

Part of #7618

NO_DOC=internal
NO_CHANGELOG=internal

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

The commit c1c77782 ("replication: fix bootstrap failing with
ER_READONLY") made applier retry connection infinitely upon receiving a
ER_READONLY error on join. At the time of writing that commit, this was
the only way to make join retriable. Because there were no retries in
scope of bootstrap_from_master. The join either succeeded or failed.

Later on, bootstrap_from_master was made retriable in commit
f2ad1dee ("replication: retry join automatically"). Now when
bootstrap_from_master fails, replica reconnects to all the remote nodes,
thus updating their ballots, chooses a new (probably different from the
previous approach) bootstrap leader, and retries booting from it.

The second approach is more preferable, and here's why. Imagine
bootstrapping a cluster of 3 nodes, A, B and C in a full-mesh topology.
B and C connect to all the remote peers almost instantly, and both
independently decide that B will be the bootstrap leader (it means it
has the smallest uuid among A, B, C).

At the same time, A can't connect to C. B bootstraps the cluster, and
joins C. After C is joined, A finally connects to C. Now A can choose a
bootstrap leader. It has an old B's ballot (smallest uuid, but not yet
booted) and C's ballot (already booted). This is because C's ballot is
received after cluster bootstrap, and B's ballot was received earlier
than that. So A believes C is a better bootstrap leader, and tries to
boot from it.

A will fail joining to C, because at the same time C tries to sync with
everyone, including A, and thus stays read-only. Since A retries joining
to the same instance over and over again, this situation makes the A and
C stuck forever.

Let's retry ER_READONLY on another level: instead of trying to join to
the same bootstrap leader over and over, try to choose a new bootstrap
leader and boot from it.

In the situation described above, this means that A would try to join to
C once, fail due to ER_READONLY, re-fetch new ballots from everyone and
choose B as a join master (now it has smallest uuid and is booted).

The issue was discovered due to linearizable_test.lua hanging
occasionally with the following output:
NO_WRAP
 No output during 40 seconds. Will abort after 320 seconds without output. List of workers not reporting the status:
- 059_replication-luatest [replication-luatest/linearizable_test.lua, None] at /tmp/t/059_replication-luatest/linearizable.result:0
[059] replication-luatest/linearizable_test.lua                       [ fail ]
[059] Test failed! Output from reject file /tmp/t/rejects/replication-luatest/linearizable.reject:
[059] TAP version 13
[059] 1..6
[059] # Started on Thu Sep 29 10:30:45 2022
[059] # Starting group: linearizable-read
[059] not ok 1	linearizable-read.test_wait_others
[059] #   ....11.0~entrypoint.531.dev/test/luatest_helpers/server.lua:104: Waiting for "readiness" on server server_1-q7berSRY4Q_E (PID 53608) timed out
[059] #   stack traceback:
[059] #   	....11.0~entrypoint.531.dev/test/luatest_helpers/server.lua:104: in function 'wait_for_readiness'
[059] #   	...11.0~entrypoint.531.dev/test/luatest_helpers/cluster.lua:92: in function 'start'
[059] #   	...t.531.dev/test/replication-luatest/linearizable_test.lua:50: in function <...t.531.dev/test/replication-luatest/linearizable_test.lua:20>
[059] #   	...
[059] #   	[C]: in function 'xpcall'
NO_WRAP

Part-of #7737

NO_DOC=bugfix

Usually, GitHub actions are named like `foo-bar` rather than `foo_bar`.
A few widely known examples: upload-artifact [1], download-artifact [2],
setup-python [3], setup-node [4]. So let's stick to this approach also.

[1] https://github.com/actions/upload-artifact
[2] https://github.com/actions/download-artifact
[3] https://github.com/actions/setup-python
[4] https://github.com/actions/setup-node

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

This patch fixed the assertion when using INDEXED BY with an index that
is at least the third in space.

Closes #5976

NO_DOC=bugfix

If the length of the tuple is greater than the number of fields in the
format, it is possible that the cursor in the VDBE will be overridden
with zeros.

Closes #5310

NO_DOC=bugfix

* FFI: Always fall back to metamethods for cdata length/concat.
* FFI: Add tonumber() specialization for failed conversions.
* build: introduce LUAJIT_ENABLE_CHECKHOOK option
* Fix overflow check in unpack().
* gdb: refactor iteration over frames while dumping stack
* gdb: adjust to support Python 2 (CentOS 7)

Closes #7458
Closes #7655
Needed for #7762
Part of #7230

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

dt.new() will raise a clear error on wrong timestamp type.

Closes #7273

NO_DOC=bugfix
NO_CHANGELOG=bugfix

For reasons described in #7231 HASH index 'GT' iterator type is deprecated:
print a warning exactly once about the deprecation.

Closes #7231

@TarantoolBot document
Title: memtx HASH index 'GT' iterator deprecation

memtx HASH index 'GT' iterator is deprecated since Tarantool 2.11
(tarantool/tarantool#7231) and will removed in a future release of
Tarantool: the user will get a warning when using it.

It's incorrect to create a tuple format when a read view is created,
because the tuple format registry isn't MT-safe so accessing a tuple
format created in the main thread from another thread by id (for
tuple_format(tuple)) may not work as expected. To address this issue,
we'll introduce a thread-local tuple format-registry. As a preparation
for this, let's create the tuple format when a read view is activated
by the thread that's going to use it (see the EE repository) while on
read view creation we now create only a field definition array necessary
to construct the format.

Needed for https://github.com/tarantool/tarantool-ee/issues/247

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

A read view must be activated in the thread that's going to use it.
This is required to support space upgrade and tuple format, which are
only used in EE. "Raw" index read view methods (returning raw message
pack data) don't require any activation by design so let's move read
view activate/deactivate logic completely to the EE repository.

Needed for https://github.com/tarantool/tarantool-ee/issues/247

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Such style is not recommented and going to be banned.

Closes #7715

NO_TEST=test refactoring
NO_DOC=test refactoring
NO_CHANGELOG= test refactoring

Commit 7f2c7609 ("box: add option to disable Lua FFI for read
operations") added an internal configuration option that switches all
index read operations from FFI to Lua C, which was necessary to
implement space upgrade in the EE repository. Turns out there was a
minor bug in that commit - when FFI is disabled, method usage errors
(raised when object.method is called instead of object:method) stop
working. Fix this bug and add an extensive test that checks that Lua C
and FFI implementations of index read operations are equivalent.

Fixes https://github.com/tarantool/tarantool-ee/issues/254

NO_DOC=bug fix
NO_CHANGELOG=bug manifests itself in only in EE

Concurrent transactions can try to insert tuples that intersect only by
parts of secondary index: in this case when one of them gets prepared, the
others get conflicted, but the committed story does not get retained
(because the conflicting statements are not added to the committed story's
delete statement list as opposed to primary index) and is lost after
garbage collection: retain stories if there is a newer uncommitted story
in the secondary indexes' history chain.

Closes #7712

NO_DOC=bugfix

In most places (schema.lua for example) we use box.error() to throw
error after error return from C function. Function is supposed to set
diag and box.error() will throw error based on the diag. For
example (crypto.lua):

    local ctx = ffi.C.crypto_stream_new(algo, mode, direction)
    if ctx == nil then
        box.error()
    end

This code is nice in case C function does not forget to set a diag.

But we have a different usage idiom of box.error().

    local result = pcall(module.cfg)
    if not result and oldvals ~= nil then
        rollback_module(module, oldcfg, keys, oldvals)
        return box.error()
    end

How it can go wrong? module.cfg is not necessary a C function. If it
is a Lua function it can throw error without diag. And if diag is not
set then box.error() is NOOP. So if there is a bug in code then Lua
generates error and it will be hided in this idiom. Or code can
intentionally throw Lua error() and again it will be hided. Or diag can
be set in some other place before and error will not be hided but
instead we will receive incorrect diagnosis.

One can argue that all these cases are some kind of bugs. But in this
case it will be useful to have correct diagnosis given it does not
involve much effort.

In short let's abandon pcall/box.error usage and use pcall/error(err)
instead.

Follow-up https://github.com/tarantool/tarantool-ee/issues/200

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Pagination for vinyl works exactly like in memtx: the feature is tested
by the existing test, which was providently created as multi-engine.

The idea is to set vy_read_iterator::last, which is used for restoring
the iterator after changes in the LSM-tree structure, to the iterator
position provided by the user at the time when the iterator is opened.
Note, we can't use vy_read_iterator::last for reporting the current
iterator position back to the user, because the iterator is closed on
the last iteration so as not to postpone freeing resources until
the Lua garbage collector kicks in, so we have to add a new member to
the vinyl_iterator struct for storing the current position and update
it after each call to vy_read_iterator_next().

Closes #7634

NO_DOC=later
NO_CHANGELOG=later

The test expects CE and running it in EE does not add value so let's
skip it.

NO_DOC=test fix
NO_CHANGELOG=test fix
NO_TEST=test fix

Add parts required to call actual flightrec reconfiguration.

Part of https://github.com/tarantool/tarantool-ee/issues/200

NO_DOC=in EE version
NO_CHANGELOG=in EE version

Function say_set_log_callback is not convenient for dynamic
configuration because we need to set callback only once per box run and
we can change flightrec log level many times. Let's make log callback
just a global variable.

Also give related items a bit more comprehensible names (yet not
generic).

Part of https://github.com/tarantool/tarantool-ee/issues/200

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Flightrec configuration parameters are reperated for init and check
functions. Also we are going to pass these parameters down the stack in
flightrec implementation patches.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

When using vclockset_psearch, the resulting vclock may be incomparable
to the search key. For example, with a vclock set { } (empty vclock),
{0: 1, 1: 10}, {0: 2, 1:11} vclockset_psearch(set, {0:2, 1: 9}) might
return {0: 1, 1: 10}, and not { }.
This is known and avoided in other places, for example
recover_remaining_wals(), where vclockset_match() is used instead.
vclockset_match() starts with the same result as vclockset_psearch() and
then unwinds the result until the first vclock which is less or equal to
the search key is found.

Having vclockset_psearch in wal_collect_garbage_f could lead to issues
even before local space changes became written to 0-th vclock component.
Once replica subscribes, its' gc consumer is set to the vclock, which
the replica sent in subscribe request. This vclock might be incomparable
with xlog vclocks of the master, leading to the same issue of
potentially deleting a needed xlog during gc.

Closes #7584

NO_DOC=bugfix

After this patch, it will be possible to drop tuple and field
constraints using the DROP CONSTRAINT statement.

Part of #6986

NO_DOC=will be added later
NO_CHANGELOG=will be added later

This patch introduces mp_format_on_region() and mp_vformat_on_region()
functions. These functions help to create an encoded value according to
a given format in a buffer allocated on region.

NO_DOC=Refactoring
NO_TEST=Refactoring
NO_CHANGELOG=Refactoring

The patch introduces C and Lua methods for extracting position
of tuple in index. Multikey and functional indexes are not supported.

Also this patch extends method index_create_iterator_after and adds
implementation of iterator_position for memtx tree. Options
after and fetch_pos are added to Lua select as well (both FFI and LuaC).
All the types of memtx tree indexes are supported (multikey, functional).

NO_CHANGELOG=see later commits
NO_DOC=see later commits

Closes #7633
Closes #7636

Currently, we set NULL tuple to iterator when it is exhausted. But we will
need this tuple for pagination, so let's leave last tuple in iterator.

This patch also renames iterator's field currrent to last to denote a new
property.

NO_TEST=no behaviour changes
NO_CHANGELOG=internal
NO_DOC=internal

Part of #7633

Currently, tuple_extract_key_raw is used only for primary key,
which is not nullable, so the problem was not observed before.
Its slowpath version chunk of the same size as tuple, but if
the tuple has trailing NULL fields, allocated chunk is not enough.
The patch fixes this problem.
Also, there is a special assert in this function which is poorly
located - let's move it to the end of the funciton.

Part of #7633

NO_CHANGELOG=internal
NO_DOC=internal