Usually, GitHub actions are named like `foo-bar` rather than `foo_bar`.
A few widely known examples: upload-artifact [1], download-artifact [2],
setup-python [3], setup-node [4]. So let's stick to this approach also.

[1] https://github.com/actions/upload-artifact
[2] https://github.com/actions/download-artifact
[3] https://github.com/actions/setup-python
[4] https://github.com/actions/setup-node

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

This patch fixed the assertion when using INDEXED BY with an index that
is at least the third in space.

Closes #5976

NO_DOC=bugfix

If the length of the tuple is greater than the number of fields in the
format, it is possible that the cursor in the VDBE will be overridden
with zeros.

Closes #5310

NO_DOC=bugfix

* FFI: Always fall back to metamethods for cdata length/concat.
* FFI: Add tonumber() specialization for failed conversions.
* build: introduce LUAJIT_ENABLE_CHECKHOOK option
* Fix overflow check in unpack().
* gdb: refactor iteration over frames while dumping stack
* gdb: adjust to support Python 2 (CentOS 7)

Closes #7458
Closes #7655
Needed for #7762
Part of #7230

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

dt.new() will raise a clear error on wrong timestamp type.

Closes #7273

NO_DOC=bugfix
NO_CHANGELOG=bugfix

For reasons described in #7231 HASH index 'GT' iterator type is deprecated:
print a warning exactly once about the deprecation.

Closes #7231

@TarantoolBot document
Title: memtx HASH index 'GT' iterator deprecation

memtx HASH index 'GT' iterator is deprecated since Tarantool 2.11
(tarantool/tarantool#7231) and will removed in a future release of
Tarantool: the user will get a warning when using it.

It's incorrect to create a tuple format when a read view is created,
because the tuple format registry isn't MT-safe so accessing a tuple
format created in the main thread from another thread by id (for
tuple_format(tuple)) may not work as expected. To address this issue,
we'll introduce a thread-local tuple format-registry. As a preparation
for this, let's create the tuple format when a read view is activated
by the thread that's going to use it (see the EE repository) while on
read view creation we now create only a field definition array necessary
to construct the format.

Needed for https://github.com/tarantool/tarantool-ee/issues/247

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

A read view must be activated in the thread that's going to use it.
This is required to support space upgrade and tuple format, which are
only used in EE. "Raw" index read view methods (returning raw message
pack data) don't require any activation by design so let's move read
view activate/deactivate logic completely to the EE repository.

Needed for https://github.com/tarantool/tarantool-ee/issues/247

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Such style is not recommented and going to be banned.

Closes #7715

NO_TEST=test refactoring
NO_DOC=test refactoring
NO_CHANGELOG= test refactoring

Commit 7f2c7609 ("box: add option to disable Lua FFI for read
operations") added an internal configuration option that switches all
index read operations from FFI to Lua C, which was necessary to
implement space upgrade in the EE repository. Turns out there was a
minor bug in that commit - when FFI is disabled, method usage errors
(raised when object.method is called instead of object:method) stop
working. Fix this bug and add an extensive test that checks that Lua C
and FFI implementations of index read operations are equivalent.

Fixes https://github.com/tarantool/tarantool-ee/issues/254

NO_DOC=bug fix
NO_CHANGELOG=bug manifests itself in only in EE

Concurrent transactions can try to insert tuples that intersect only by
parts of secondary index: in this case when one of them gets prepared, the
others get conflicted, but the committed story does not get retained
(because the conflicting statements are not added to the committed story's
delete statement list as opposed to primary index) and is lost after
garbage collection: retain stories if there is a newer uncommitted story
in the secondary indexes' history chain.

Closes #7712

NO_DOC=bugfix

In most places (schema.lua for example) we use box.error() to throw
error after error return from C function. Function is supposed to set
diag and box.error() will throw error based on the diag. For
example (crypto.lua):

    local ctx = ffi.C.crypto_stream_new(algo, mode, direction)
    if ctx == nil then
        box.error()
    end

This code is nice in case C function does not forget to set a diag.

But we have a different usage idiom of box.error().

    local result = pcall(module.cfg)
    if not result and oldvals ~= nil then
        rollback_module(module, oldcfg, keys, oldvals)
        return box.error()
    end

How it can go wrong? module.cfg is not necessary a C function. If it
is a Lua function it can throw error without diag. And if diag is not
set then box.error() is NOOP. So if there is a bug in code then Lua
generates error and it will be hided in this idiom. Or code can
intentionally throw Lua error() and again it will be hided. Or diag can
be set in some other place before and error will not be hided but
instead we will receive incorrect diagnosis.

One can argue that all these cases are some kind of bugs. But in this
case it will be useful to have correct diagnosis given it does not
involve much effort.

In short let's abandon pcall/box.error usage and use pcall/error(err)
instead.

Follow-up https://github.com/tarantool/tarantool-ee/issues/200

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Pagination for vinyl works exactly like in memtx: the feature is tested
by the existing test, which was providently created as multi-engine.

The idea is to set vy_read_iterator::last, which is used for restoring
the iterator after changes in the LSM-tree structure, to the iterator
position provided by the user at the time when the iterator is opened.
Note, we can't use vy_read_iterator::last for reporting the current
iterator position back to the user, because the iterator is closed on
the last iteration so as not to postpone freeing resources until
the Lua garbage collector kicks in, so we have to add a new member to
the vinyl_iterator struct for storing the current position and update
it after each call to vy_read_iterator_next().

Closes #7634

NO_DOC=later
NO_CHANGELOG=later

The test expects CE and running it in EE does not add value so let's
skip it.

NO_DOC=test fix
NO_CHANGELOG=test fix
NO_TEST=test fix

Add parts required to call actual flightrec reconfiguration.

Part of https://github.com/tarantool/tarantool-ee/issues/200

NO_DOC=in EE version
NO_CHANGELOG=in EE version

Function say_set_log_callback is not convenient for dynamic
configuration because we need to set callback only once per box run and
we can change flightrec log level many times. Let's make log callback
just a global variable.

Also give related items a bit more comprehensible names (yet not
generic).

Part of https://github.com/tarantool/tarantool-ee/issues/200

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Flightrec configuration parameters are reperated for init and check
functions. Also we are going to pass these parameters down the stack in
flightrec implementation patches.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

When using vclockset_psearch, the resulting vclock may be incomparable
to the search key. For example, with a vclock set { } (empty vclock),
{0: 1, 1: 10}, {0: 2, 1:11} vclockset_psearch(set, {0:2, 1: 9}) might
return {0: 1, 1: 10}, and not { }.
This is known and avoided in other places, for example
recover_remaining_wals(), where vclockset_match() is used instead.
vclockset_match() starts with the same result as vclockset_psearch() and
then unwinds the result until the first vclock which is less or equal to
the search key is found.

Having vclockset_psearch in wal_collect_garbage_f could lead to issues
even before local space changes became written to 0-th vclock component.
Once replica subscribes, its' gc consumer is set to the vclock, which
the replica sent in subscribe request. This vclock might be incomparable
with xlog vclocks of the master, leading to the same issue of
potentially deleting a needed xlog during gc.

Closes #7584

NO_DOC=bugfix

After this patch, it will be possible to drop tuple and field
constraints using the DROP CONSTRAINT statement.

Part of #6986

NO_DOC=will be added later
NO_CHANGELOG=will be added later

This patch introduces mp_format_on_region() and mp_vformat_on_region()
functions. These functions help to create an encoded value according to
a given format in a buffer allocated on region.

NO_DOC=Refactoring
NO_TEST=Refactoring
NO_CHANGELOG=Refactoring

The patch introduces C and Lua methods for extracting position
of tuple in index. Multikey and functional indexes are not supported.

Also this patch extends method index_create_iterator_after and adds
implementation of iterator_position for memtx tree. Options
after and fetch_pos are added to Lua select as well (both FFI and LuaC).
All the types of memtx tree indexes are supported (multikey, functional).

NO_CHANGELOG=see later commits
NO_DOC=see later commits

Closes #7633
Closes #7636

Currently, we set NULL tuple to iterator when it is exhausted. But we will
need this tuple for pagination, so let's leave last tuple in iterator.

This patch also renames iterator's field currrent to last to denote a new
property.

NO_TEST=no behaviour changes
NO_CHANGELOG=internal
NO_DOC=internal

Part of #7633

Currently, tuple_extract_key_raw is used only for primary key,
which is not nullable, so the problem was not observed before.
Its slowpath version chunk of the same size as tuple, but if
the tuple has trailing NULL fields, allocated chunk is not enough.
The patch fixes this problem.
Also, there is a special assert in this function which is poorly
located - let's move it to the end of the funciton.

Part of #7633

NO_CHANGELOG=internal
NO_DOC=internal

Currently, func_index_compare_with_key compares only functional key
of tuple. But when we will implement pagination, we will need to
find a tuple with a certain primary key. This commit fixes
the problem and introduces key_compare_and_skip_parts helper.

NO_CHANGELOG=internal
NO_DOC=internal

Part of #7633

To implement pagination, we need an entity which describes iterator
position in index with serialization and deserialization methods.
Extracted cmp_def is most appropriate, so the patch introduces
serialization and deserialization methods to pack key as position in index.

NO_CHANGELOG=internal
NO_DOC=internal

Closes #7635

The patch introduces new virtual method and extends existing one
for the needs of pagination. New functional is not implemented, stubs only.

Index virtual method create_iterator is extended with new argument pos,
If it is passed, iterator points to the first tuple after position,
described by this argument (or error is returned if index does not
support pagination). If pos is NULL, method has the same
behavior as before this patch.

Function index_create_iterator_after now calls create_iterator method
and index_create_iterator calls the same method but passes NULL as pos
argument.

Also the patch introduces a new virtual method of iterator -
iterator_position. It returnes position (extracted cmp_def without
MP_ARRAY header) of last fetched tuple, or NULL if no tuples were
fetched.

NO_TEST=no behavior changes
NO_CHANGELOG=internal
NO_DOC=internal

Part of #7633

NO_DOC=bugfix
NO_CHANGELOG=bugfix for unreleased bug

The idea is borrowed from [1]: hide and save prompt, user's input and
cursor position before writing to stdout/stderr and return everything
back afterwards.

Not every stdout/stderr write is handled this way: only tarantool's
logger (when it writes to stderr) and tarantool's print() Lua function
performs the prompt hide/show actions. For example,
`io.stdout:write(<...>)` Lua call or `write(STDOUT_FILENO, <...>)` C
call may mix readline's prompt with actual output. However the logger
and print() is likely enough for the vast majority of usages.

The readline's interactive search state (usually invoked by Ctrl+R) is
not covered by this patch. Sadly, I didn't find a way to properly save
and restore readline's output in this case.

Implementation details
----------------------

Several words about the allocation strategy. On the first glance it may
look worthful to pre-allocate a buffer to store prompt and user's input
data and reallocate it on demand. However rl_set_prompt() already
performs free() plus malloc() at each call[^1], so avoid doing malloc()
on our side would not change the picture much. Moreover, this code
interacts with a human, which is on many orders of magnitude slower that
a machine and will not notice a difference. So I decided to keep the
code simpler.

[^1]: Verified on readline 8.1 sources. However it worth to note that
      rl_replace_line() keeps the buffer and performs realloc() on
      demand.

The code is organized to make say and print modules calling some
callbacks without knowledge about its origin and dependency on the
console module (or whatever else module would implement this interaction
with readline). The downside here is that console needs to know all
places to set the callbacks. OTOH, it offers explicit list of such
callbacks in one place and, at whole, keep the relevant code together.

We can redefine the print() function from every place in the code, but I
prefer to make it as explicit as possible, so added the new internal
print.lua module.

We could redefine _G.print on demand instead of setting callbacks for a
function assigned to _G.print once. The downside here is that if a user
save/capture the old _G.print value, it'll use the raw print() directly
instead of our replacement. Current implementation seems to be more
safe.

Alternatives considered
-----------------------

I guess we can clear readline's prompt and user input manually and don't
let readline know that something was changed (and restore the
prompt/user input afterwards). It would save allocations and string
copying, but likely would lean on readline internals too much and repeat
some of its functionality. I considered this option as unstable and
declined.

We can redefine behavior for all writes to stdout and stderr. There are
different ways to do so:

1. Redefine libc's write() with our own implementation, which will call
   the original libc's write()[^2]. It is defined as a weak symbol in
   libc (at least in glibc), so there is no problem to do so.
2. Use pipe(), dup() and dup2() to execute our own code at
   STDOUT_FILENO, STDERR_FILENO writes.

[^2]: There is a good article about pitfalls on this road: [2]. It is
      about LD_PRELOAD, but I guess everything is very similar with
      wrapping libc's function from an executable.

In my opinion, those options are dangerous, because they implicitly
change behavior of a lot of code, which unlikely expects something of
this kind. The second option (use pipe()) adds more user space/kernel
space context switches, more copying and also would add possible
implicit fiber yield at any `write(STD*_FILENO, <...>)` call -- unlikely
all user's code is ready for that.

Fixes #7169

[1]: https://metacpan.org/dist/AnyEvent-ReadLine-Gnu/source/Gnu.pm
[2]: https://tbrindus.ca/correct-ld-preload-hooking-libc/

NO_DOC=this patch prevents mixing of output streams on a terminal and it
       is what a user actually expects; no reason to describe how bad
       would be his/her life without it

A basic (and pretty useless) example:

```lua
local it = require('test.luatest_helpers.interactive_tarantool')

local child = it.new()

child:execute_command('6 * 7')
local res = child:read_response()
t.assert_equals(res, 42)

child:close()
```

The module also contains `:read_line()`, `:assert_line()` helpers for
testing output directly: for example, when we need to catch a print()
from a background fiber. It provides has constants related to terminal's
control sequences.

A real usage can be seen in a next commit.

Part of #7169

NO_DOC=no user visible changes
NO_TEST=not applicable, it is a testing helper
NO_CHANGELOG=no user visible changes

Those callbacks will be set from the console module (which integrates
readline into tarantool), so the print() output won't mix with readline
prompting.

Part of #7169

NO_DOC=no user visible changes, it is internal API
NO_TEST=will be tested in a next commit together with the console
        callbacks itself
NO_CHANGELOG=no user visible changes

Those callbacks will be set from the console module (which integrates
readline into tarantool), so the stderr output won't mix with readline
prompting.

Part of #7169

NO_DOC=no user visible changes, it is internal API
NO_TEST=will be tested in a next commit together with the console
        callbacks itself
NO_CHANGELOG=no user visible changes

Caught the following error with UB sanitizer after trying to use the
31-st vclock component in one of the tests:

NO_WRAP
```
[040] +/opt/actions-runner/_work/tarantool/tarantool/src/lib/vclock/vclock.c:133:21: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
[040] +SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /opt/actions-runner/_work/tarantool/tarantool/src/lib/vclock/vclock.c:133:21 in
[040] Test "unit/vclock.test", conf: "None"
[040] 	failed, rerunning ...
[040] unit/vclock.test                                                [ fail ]
```
NO_WRAP

Let's fix this in all the affected places.

NO_DOC=refactoring
NO_TEST=already tested by unit/vclock.cc: test_minmax_ignore0()
NO_CHANGELOG=refactoring

We hold the following invariant in MVCC: the story at the top of the
history chain is present in index.

If a story is subject to be deleted from index and there is an older story
in the history chain, the older story starts to be at the top of the
history chain and is not present in index, which violates our invariant:
explicitly check for this case when evaluating whether a story can be
garbage collected and add an assertion to check the invariant above is not
violated.

Rollbacked stories need to be handled in a special way: they are
present at the end of some history chains and completely unlinked from
others (which also implies they are not present in the corresponding
indexes).

`memtx_tx_story_full_unlink` is called in two contexts: space deletion, in
which we delete all stories, and garbage collection step — the former case
can break the invariant described above, while the latter must preserve it,
hence add two different functions for the corresponding contexts.

Closes #7490

NO_CHANGELOG=<internal bugfix not user observable>
NO_DOC=<bugfix>

When we rollback a transaction statement, we relink its read trackers
to a newer story in the history chain, if present (6c990a7b), but we do not
handle the case when there is no newer story.

If there is an older story in the history chain, we can relink the
rollbacked story's reader to it, but if the rollbacked story is the
only one left, we need to retain it, because it stores the reader list
needed for conflict resolution — such stories are distinguished by the
rollbacked flag, and there can be no more than one such story located
strictly at the end of a given history chain (which means a story can be
fully unlinked from some indexes and present at the end of others).

There are several nuances we need to account for:

Firstly, such rollbacked stories must be impossible to read from an index:
this is ensured by `memtx_tx_story_is_visible`.

Secondly, rollbacked transactions need to be treated as prepared with
stories that have `add_psn == del_psn`, so that they are correctly deleted
during garbage collection.

After this logical change we have the following partially ordered set over
tuple stories:
———————————————————————————————————————————————————————> serialization time
|- - - - - - - -|— — — — — -|— — — — — |— — — — — — -|— — — — — — — -
| No more than  | Committed | Prepared | In-progress | One dirty
| one rollbacked|           |          |             | story in index
| story         |           |          |             |
|- - - - - - - -|— — — — — -| — — — — —|— — — — — — -|— — — — — — — —

Closes #7343

NO_DOC=bugfix

`struct memtx_story` has a `space` field, which is basically used
to identify that a tuple is unlinked from the history chain in
`memtx_tx_index_invisible_count_slow` (though this can be determined by its
presence in the index) and is used to get the space's index in
`memtx_tx_story_link_top` (though it can be  retrieved from the older
story's link field): remove this redundant field.

Needed for #7343

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>
NO_TEST=<refactoring>

When a space is deleted, all transactions need to be aborted and all their
stories need to be removed immediately out of order: currently we
artificially rollback statements — instead call this statement
removal to logically distinguish it from rollback. It differs in the sense
that the whole space's tuple history is teared down instead — no more
transaction managing is going to be done as opposed to rollback of an
individual transaction.

Needed for #7343

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Follow `memtx_tx_history_{add, prepare}_{insert, delete}` pattern: split
code responsible for rollbacking addition and deletion of a story into
separate functions.

Needed for #7343

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactorin

When a statement gets rollbacked, we need to remove delete statements
attached to the story it adds by relinking them and making them delete an
older story in the history chain: refactor this loop out into a separate
function.

Needed for #7343

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

If a statement becomes prepared, the story it adds must be 'sunk' to
the level of prepared stories: refactor this loop into a
separate function.

Needed for #7343

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Change error from "ER_QUORUM_WAIT: fiber is cancelled" to
FiberIsCancelled for consistency with other places that check for fiber
cancellation.
Same of "ER_QUORUM_WAIT: timed out". Change it to TimedOut.

While I'm at it, make box_wait_quorum tolerate spurious wake-ups.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring