It's really annoying to add a message after each check in a unit test.
Let's make this optional. If the message is omitted, "line %d" will be
used instead. Also, let's print the expression on failure because it
may be useful if exact sources are unavailable.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 62ae8bf3)

The ok/is/isnt macros expand to {} so they may be used without
a trailing semicolon. This is going to be fixed so let's add
missing semicolons.

NO_DOC=code cleanup
NO_CHANGELOG=code cleanup

(cherry picked from commit 4871b697)

There's no need to duplicate all unit test helpers for TAP compatible
tests. The only difference between them is that plan() prints the TAP
version so let's do just that.

NO_DOC=code cleanup
NO_CHANGELOG=code cleanup

(cherry picked from commit a235080c)

NO_DOC=seed corpus
NO_CHANGELOG=seed corpus
NO_TEST=seed corpus

(cherry picked from commit 4894863e)

NO_DOC=testing
NO_CHANGELOG=testing

(cherry picked from commit 4deadeb8)

NO_DOC=seed corpus
NO_CHANGELOG=seed corpus
NO_TEST=seed corpus

(cherry picked from commit 4b5fb953)

Examples of IPROTO decoding issues: #3900, #1928, #6781.
Patch adds a number of fuzzing tests that covers IPROTO decoding:

- xrow_decode_auth
- xrow_decode_begin
- xrow_decode_call
- xrow_decode_dml
- xrow_decode_id
- xrow_decode_raft
- xrow_decode_sql
- xrow_decode_watch
- xrow_greeting_decode

NO_DOC=testing
NO_CHANGELOG=testing

(cherry picked from commit 46cacf35)

Fuzzing test for LuaJIT generates random Lua programs and executes them.
We want to build a fuzzing test that will produce Lua programs that will
not contain semantic errors and will trigger as much as possible
components in LuaJIT.

This proposed patch introduces metrics that gathered after running the
test. LuaJIT metrics gathered using LuaJIT getmetrics module [1]. All
gathered metrics test will output after running with a finite number of
runs or finite duration of time (options `-runs` and `-max_total_time`)
or after sending SIGUSR1 to a test process.

```
$ ./build/test/fuzz/luaL_loadbuffer/luaL_loadbuffer_fuzzer -runs=1000

<snipped>

Done 1000 runs in 1 second(s)
Total number of samples: 1000
Total number of samples with errors: 438 (43%)
Total number of samples with recorded traces: 87 (8%)
Total number of samples with snap restores: 30 (3%)
Total number of samples with abort traces: 55 (5%)
```

1. https://www.tarantool.io/en/doc/latest/reference/tooling/luajit_getmetrics/#getmetrics-c-api

NO_CHANGELOG=testing
NO_DOC=testing

(cherry picked from commit 430fa6a2)

This refactoring will:

1. Move macros from a header to the source file.
Macros should be used in header only with undef to avoid redefinitions.
Undef directive is not useful since we want to use these macros in the
source file.

2. Remove `using namespace lua_grammar` from header.
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-using-directive

3. Moving serializer entry point and constant parameters into
luajit_fuzzer namespace.
It's a common practice in C++ to avoid name collisions.

4. Move serializer functions into anonymous namespace.
These functions are not a part of the interface so should have
static linkage.
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-unnamed2

5. Fix ConvertToStringDefault function.
It was logically wrong so it would generate an identifier `123` from
`*123`.

NO_CHANGELOG=internal
NO_DOC=fuzzer fix

(cherry picked from commit 56488e15)

LuaJIT fuzzer used to stop due to timeout caused by infinite cycles and
recursions. Counters were introduced for every cycle and function to
address LuaJIT fuzzer timeouts.

The idea is to add unique counters for every cycle and function to
ensure finite code execution, if it wasn't already. For while, repeat,
for cycles, local and global named, anonymous functions, counters will
be initialized before the code generated from protobuf, and checked
in the first body statement. An entry point for the serializer was
created to count cycles and functions for counter initialization.

The idea was taken from a paper "Program Reconditioning: Avoiding
Undefined Behaviour When Finding and Reducing Compiler Bugs" [1].

Here is an example of a change in serialized code made by this commit.

Before:
```lua
while (true) do
    foo = 'bar'
end
function bar()
    bar()
end
```

After:
```lua
counter_0 = 0
counter_1 = 0
while (true) do
    if counter_0 > 5 then
        break
    end
    counter_0 = counter_0 + 1
    foo = 'bar'
end
function bar()
    if counter_1 > 5 then
        return
    end
    counter_1 = counter_1 + 1
    bar()
end
```
Protobuf structures that reproduce the timeout problem were added to
the LuaJIT fuzzer corpus.

[1] https://www.doc.ic.ac.uk/~afd/homepages/papers/pdfs/2023/PLDI.pdf

NO_CHANGELOG=internal
NO_DOC=fuzzer fix

(cherry picked from commit 4d004bbe)

Cases in two switches had no breaks, so they were falling
through. Breaks were added to solve the problem. Code
generated by the LuaJIT fuzzer became more various.

NO_CHANGELOG=internal
NO_DOC=fuzzer fix

(cherry picked from commit 4430cac9)

Function `datetime_strptime` decodes string with datetime according to
specified format, it accepts a datetime struct, buffer with datetime and
string with format in arguments. Fuzzing test used static string
"iso8601" as a format and it blocked fuzzing test to cover functions
used by datetime_strptime under the hood. Fuzz introspector shows that
code coveraged by a test is quite low.

Patch updates the test to make it more effective: buffer with datetime
and format string are generated using FDP (Fuzzing Data Provider).

Test file extension was changed to .cc, because FuzzingDataProvider is
used and we need building it by C++ compiler.

Function `tnt_strptime` uses assert, that triggered by fuzzing tests.
Therefore it was replaced with to if..then.

1. https://storage.googleapis.com/oss-fuzz-introspector/tarantool/

Fixes #8490

NO_CHANGELOG=fuzzing test
NO_DOC=fuzzing test
NO_TEST=fuzzing test

(cherry picked from commit a1bd6e0b)

Corpus based on PUC Rio Lua tests imported from LuaJIT repository [1].

1. https://github.com/tarantool/luajit/tree/tarantool/test/PUC-Rio-Lua-5.1-tests

Follows up #4823

NO_CHANGELOG=corpus
NO_DOC=corpus
NO_TEST=corpus

(cherry picked from commit 890eb224)

Patch adds a LuaJIT fuzzer based on libprotobuf-mutator and LibFuzzer.
Grammar is described via messages in protobuf format, serializer is
applied to convert .proto format to string.

For displaying generated code on the screen during fuzzing set
the environment variable 'LPM_DUMP_NATIVE_INPUT'.

For displaying error messages from lua functions set
the environment variable 'LUA_FUZZER_VERBOSE'.

Note: UndefinedBehaviourSanitizer is unsupported by LuaJIT (see #8473),
so fuzzing test is disabled when CMake option ENABLE_UB_SANITIZER is
passed.

Closes #4823

NO_DOC=<fuzzing testing of LuaJIT>
NO_TEST=<fuzzing testing of LuaJIT>

(cherry picked from commit a287c853)

Added options for fuzzing and for getting more information
on debugging.

NO_CHANGELOG=<fuzzing options>
NO_DOC=<fuzzing options>
NO_TEST=<fuzzing options>

(cherry picked from commit 69f21e25)

Commit 2be74a65 ("test/cmake: add a function for generating unit
test targets") added a function for generating unit test targets in
CMake. This function makes code simpler and less error-prone.

Proposed patch adds a similar function for generating fuzzing test
targets in CMake.

NO_CHANGELOG=build infrastructure updated
NO_DOC=build infrastructure updated
NO_TEST=build infrastructure updated

(cherry picked from commit d9643bfd)

test/unit/guard.cc calls stack_break_f() recursively until the stack
overflows and a signal is fired, however it relies on undefined behavior
when compares pointers to local variables. Fixed by comparing
__builtin_frame_address() instead.

One of the examples of this UB is when ASAN allocates local variables on
fake stacks, in that case the test completes without the stack overflow.

Also this patch disables ASAN for stack_break_f() to keep the array on the
fiber stack (see the corresponding comment) and marks it as volatile to
avoid optimizing it away by the compiler.

Closes tarantool/tarantool-qa#323

NO_DOC=test fix
NO_CHANGELOG=test fix

(cherry picked from commit 05b696c7)

Since number type was introduced we can not assume if tuples are
equal by comparison then their sizes are equal too. So the place
the assumption is used is fixed.

Closes #8899

NO_DOC=bugfix

(cherry picked from commit f4de9faf)

Fixed the implementation of the box console.
Before this fix, result of `\set language` is shared between clients
via `console.connect`, despite the fact that clients have different
`box.session.id`. Now the parameter of the selected language is stored
by each client in his own `box.session.storage`.

Fixes #8817

NO_DOC=bugfix

(cherry picked from commit e4fda4b7)

Interactive console test helper could be used for remote connection too,
for that purpose prompt needs to be changed accordingly to connection
type.

This patch introduces getter and setter for the prompt. Prompt is
configured per session, so it is advised to create a new session
for each test case (e.g. with before_each()).

NO_CHANGELOG=test helper change
NO_DOC=test helper change

(cherry picked from commit ed86a729)

The ./test/luatest_helpers/interactive_tarantool.lua module is not
a luatest helper. So moving it to the ./test/ dir and removing empty
./test/luatest_helpers/.

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 5493db7c)

A basic (and pretty useless) example:

```lua
local it = require('test.luatest_helpers.interactive_tarantool')

local child = it.new()

child:execute_command('6 * 7')
local res = child:read_response()
t.assert_equals(res, 42)

child:close()
```

The module also contains `:read_line()`, `:assert_line()` helpers for
testing output directly: for example, when we need to catch a print()
from a background fiber. It provides has constants related to terminal's
control sequences.

A real usage can be seen in a next commit.

Part of #7169

NO_DOC=no user visible changes
NO_TEST=not applicable, it is a testing helper
NO_CHANGELOG=no user visible changes

(cherry picked from commit a9d96007)

As the underlying problem behind this injection is fixed in #7357 it can
be removed and `-i` flag could be used as initially intended.

Closes #7554
Requires #7357
NO_DOC=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 16d6e9d2)

The interactive mode has been ignored when stdin was not a tty and is no
more. Now results of another command can be handled by tarantool.
Before the patch:
```
$ echo 42 | tarantool -i
LuajitError: stdin:1: unexpected symbol near '42'
fatal error, exiting the event loop
```

After the patch:
```
$ echo 42 | tarantool -i
Tarantool 2.5.0-130-ge3cf64a6c
type 'help' for interactive help
tarantool> 42
---
- 42
...

```

Closes #5064

NO_DOC=bugfix

(cherry picked from commit 9965e3fe)

In #7309 a truncation of a space that was referenced by foreign
key from some other space was prohibited.

It appeared that this solution is too bothering since a user can't
truncate a space even if he truncated referring space before that.

Fix it by allowing space truncate if referring spaces are empty.
Also allow drop of the primary index in the same case with the
same reason: logically the index along with all space data is
not needed for consistency if there's no referring data.

Note that by design space truncate is implemented quite similar
to space drop. Both delete all indexes, from secondary to primary.
Since this patch allows deletion of the primary index (which is
the action that actually deletes all data from the space), this
patch changes the result of space drop too: the space remains
alive with no indexes, while before this patch it remained alive
with no secondary indexes but with present primary. In both cases
the behaviour is quite strange and must be fixed in #4348. To
make tests pass I had to perform drop in box.atomic manually.

Closes #8946

NO_DOC=bugfix

(cherry picked from commit 983a7ec2)

It is possible for interval to have days, hours, minutes and seconds
larger than INT_MAX (or less than INT_MIN). Before this patch, msgpack
decoding had failed to parse intervals with msgpack int64 and uint64.
int64_t should be enough to store any value allowed for datetime
intervals.

Closes #8887

NO_DOC=small bug fix

(cherry picked from commit 01c7ae11)

Before this patch, one couldn't create new datetime interval with
boundary value from Lua. At the same time, it was possible to create
such interval from Lua through addition and subtraction. C range
verification allow to create boundary value intervals, error message
also implies that they should be allowed. (See #8878 for more info.)

Closes #8878

NO_DOC=small bug fix

(cherry picked from commit b2a001cc)

This commit follows up the previous one (i.e. the backport of the commit
ff57f990 ("cmake: introduce FIBER_STACK_SIZE option")), since the
commit 98f62cd0 ("test: make fiber_stack.test tap-compatible") and
its siblings have not been backported to 2.10 in scope of #7618.

NO_DOC=fixup for d296f732
NO_CHANGELOG=fixup for d296f732

In scope of the commit 82f4b4a3 ("lib/core/fiber: Increase default
stack size") the default value of fiber stack size is increased up to
512 Kb (you can find the reasons in the aforementioned commit message
and in https://github.com/tarantool/tarantool/issues/3418 description).

Some of the tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT repo
(e.g. some cases with deep recursion in errors.lua or pm.lua) have
already been tweaked according to the limitations mentioned in
https://github.com/tarantool/tarantool/issues/5782, but the crashes
still occurs while running LuaJIT tests with ASan support enabled.

To make the testing routine more convenient, FIBER_STACK_SIZE option is
introduced to Tarantool CMake machinery. One can provide the size either
by raw digits (i.e. in bytes) or using Kb/Mb suffixes for convenience.

A couple of important nits:
* If the given value is not a multiple of 4Kb, CMake machinery adjusts
  it up to the nearest one greater than this value.
* If the adjusted value is less than 512Kb, configuration fails with the
  corresponding CMake fatal error.

Follows up #3418
Relates to #5782

@TarantoolBot document
Title: introduce FIBER_STACK_SIZE configuration option

To make managing of the default fiber stack size more convenient, the
corresponding CMake option is added.

**NB**: The stack size can't be less than 512Kb and if the given value
is not a multiple of 4Kb, CMake machinery adjusts it up to the nearest
one greater than this value.

(cherry picked from commit ff57f990)

This reverts commit 8aa2bb19.

In scope of the aforementioned commit changes from commit e4fda4b7
("box: fix shared lang between connected clients") are backported, but
the test in this patch uses test.interactive_tarantool.lua helper, that
is missing in release/2.10 branch. Applying all the related commits from
the master listed below doesn't fix the problem:
* commit a9d96007 ("test: add a helper for testing interactive
  mode")
* commit 5493db7c ("test: mv interactive_tarantool.lua to ./test/
  dir")
* commit ed86a729 ("test: add prompt setter to interactive helper")

Hence, simply revert the poisoned commit from the branch.

NO_DOC=revert
NO_TEST=revert
NO_CHANGELOG=revert

Test uses a popen module that starts tarantool process in background
mode. Tarantool process started in background mode forks a new process
and closes a parent, after that popen loses a PID of the started process
and `ph:kill()` and `ph:terminate()` doesn't work anymore. It leads to
non-terminated tarantool processes after running the test.

Patch fixes that by running `kill` using os.execute with a PID of
tarantool process written to a pid file.

Follows up #6128

NO_CHANGELOG=fix test
NO_DOC=fix test

(cherry picked from commit 88686227)

Fixed the implementation of the box console.
Before this fix, result of `\set language` is shared between clients
via `console.connect`, despite the fact that clients have different
`box.session.id`. Now the parameter of the selected language is stored
by each client in his own `box.session.storage`.

Fixes #8817

NO_DOC=bugfix

(cherry picked from commit e4fda4b7)

Previous attempt to fix flakiness in commit 6a2c73f8 ("test: fix
flakiness in gh_6128_background_mode_test") used a constant buffer size
in check_err_msg function. Tarantool 2.10 has a bit bigger log before a
desired message that other versions of Tarantool and it leads to a this
resulted in a truncated message ("entering the even" instead of
"entering the event loop"). Patch replaces check_err_msg()
implementation to grep_log used in luatest, it reads the whole log.

Also patch renames check_err_msg to check_msg, because "entering the
event loop" is not an error message.

Follows up #6128

NO_CHANGELOG=fix test
NO_DOC=fix test

(cherry picked from commit 1c8e7124)

Test runs an external process with tarantool that writes to a log file.
Then test reads that log file and searches a string with required
message in it (see function check_err_msg). Test was flaky on macOS and
I suspect it was happening due to a high log level - timeout was not
enough to wait message in the log file.

Patch decreases a log level to a default value and replaces io
functions with the similar alternatives in a fio module. Using
fio functions allows to not block fibers.

NO_CHANGELOG=test fix
NO_DOC=test fix

(cherry picked from commit 47380bb7)

The xlog reader Lua module uses the xlog_cursor_next_row, which decodes
the row header with xrow_header_decode. The latter silently ignores any
unknown fields, which complicates catching bugs when garbage is written
to a row header by mistake, for example, see #8783.

Let's parse a row header without using xrow_header_decode in the xlog
reader module, like we parse a row body, and output all unknown/invalid
keys as is.

To do that, we have to extend the xlog cursor API with the new method
xlog_cursor_next_row_raw that returns a pointer to the position in the
tx buffer where the next xrow is stored without advancing it. To avoid
a memory leak in case the caller fails to parse an xrow returned by this
function, we also have to move the call to xlog_tx_cursor_destroy from
xlog_tx_cursor_next_row to xlog_cursor_next_tx.

While we are at it,
 - Don't raise an error if a key type encountered in a row body is
   invalid (not an integer). Instead, silently ignore such keys.
 - Remove the useless body MsgPack validness check because we already
   check it after decoding the header.
 - Add error injection based tests to check all the corner cases.

NO_DOC=bug fix

(cherry picked from commit 8a25d170)

There's case when a transaction is rolled back from prepared
state. This happens when WAL fails, synchronized replication
confirmation failure or perhaps in other similar cases. By design
other RW transactions and transactions with READ_COMMITTED
isolation level are allowed to read prepared state. All these
transactions must be aborted in case of rollback of prepared
transaction since they definitely have read no more possible
database state.

This patch implements this abortion.

Closed #8654

NO_DOC=bugfix

(cherry picked from commit 54986902)

Rollback is rather complicated part if MVCC implementation that
is meant to handle two kinds of rollback:
* rollback from in-progress state, if box.rollback() is called.
* rollback from prepared state, when WAL fails.
Unfortunately the last one was not properly tested and surely
has at least one flaw. When an inserting transaction becomes
prepared its stories could be linked as deleted (via del_stmt
pointer) by other in-progress transactions in order to maintain
correct visibility. The problem is that in case of rollback of
such prepared transaction those links remained. Broken links
breaks the chain structure, and some older changes (that were
linked as deleted before that hapless preparation) can become
visible to other transaction.

Refactor, simplify a bit that part of code and fix the issue
described above; cover with tests.

Closes #8648

NO_DOC=bugfix

(cherry picked from commit 85569d9c)

Now there are three kinds of very close trackers:
* The transaction have read some tuple that is not committed and
  thus not visible. This kind is now stored as gap_item with
  is_nearby = false.
* The transaction made a select or range scan, reading a key or
  range between two adjacent tuples of the index. This kind is
  stored as gap_iteam with is_nearby = true.
* A transaction completed a full scan of unordered index. This
  kind is stored as full_scan_item.

All these trackers serve for the same thing: to record a fact
that a transaction read something but didn't see anything.

There are some problems with the current solution:
* gap_item with is_nearby = false has several unused members that
  just consume space.
* bool is_nearby flag for type descriptin is an ugly solution.
* full_scan_item is separated from logically close items.

This commit joins all these trackers under one base (that is
struct gap_item_base) and solves problems above.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit f8d97a2e)

Now read trackers are used both for cases when a transaction has
read an existing value and it has read nothing (read by key but
there was no visible tuple in this place). For latter case an
additional index_mask was used to identify from which index the
read was done. Along with that there was per-index interval gap
trackers.

This patch divides area of responsibility between read trackers
and gap tracker in the following way:
* Reads of existing visible values are stored in read trackers.
* Reads of non-existing or non-visible values are store in gap
  trackers.

This new approach allows to provide new invariants: gap trackers
are stored only at top of chain, and read trackers are stored
only at topmost committed story in chain.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 7b8b78be)

By a mistake in 8a565144 a shortcut was added to procedure
that handles gap write: it was considered that if the writing
transaction is the same as reading - there is no actual conflict
that must be stored further. That was a wrong decision: if such
a transaction yields and another transaction comes and commits
a value with the same key - the first one must go to conflicted
state since it has read no more possible state.

Another similar mistake was made in e6f5090c, where writing
after full scan of the same transaction was not tracked as read.
Obviously that was wrong: if some other transaction overwrites
the key and commits - this transaction must go to read view since
it did not see anything by this key which is not so anymore.

Fix it, reverting the first commit and an modifying the second and
add a test.

Closes #8326

NO_DOC=bugfix

(cherry picked from commit b41c4546)