Corpus based on PUC Rio Lua tests imported from LuaJIT repository [1].

1. https://github.com/tarantool/luajit/tree/tarantool/test/PUC-Rio-Lua-5.1-tests

Follows up #4823

NO_CHANGELOG=corpus
NO_DOC=corpus
NO_TEST=corpus

(cherry picked from commit 890eb224)

Patch adds a LuaJIT fuzzer based on libprotobuf-mutator and LibFuzzer.
Grammar is described via messages in protobuf format, serializer is
applied to convert .proto format to string.

For displaying generated code on the screen during fuzzing set
the environment variable 'LPM_DUMP_NATIVE_INPUT'.

For displaying error messages from lua functions set
the environment variable 'LUA_FUZZER_VERBOSE'.

Note: UndefinedBehaviourSanitizer is unsupported by LuaJIT (see #8473),
so fuzzing test is disabled when CMake option ENABLE_UB_SANITIZER is
passed.

Closes #4823

NO_DOC=<fuzzing testing of LuaJIT>
NO_TEST=<fuzzing testing of LuaJIT>

(cherry picked from commit a287c853)

Follows-up #4823

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

(cherry picked from commit 95d62cfc)

Added Google's 'libprotobuf-mutator' and 'protobuf' libraries
for developing grammar-based LuaJIT and SQL fuzzers based on
LibFuzzer.

It is needed to build protobuf module from source because
by default, the system-installed version of protobuf is used
by libprotobuf-mutator, and this version can be too old.

Part of #4823

NO_CHANGELOG=<dependencies>
NO_DOC=<dependencies>
NO_TEST=<dependencies>

(cherry picked from commit b11072a6)

Added options for fuzzing and for getting more information
on debugging.

NO_CHANGELOG=<fuzzing options>
NO_DOC=<fuzzing options>
NO_TEST=<fuzzing options>

(cherry picked from commit 69f21e25)

Commit 2be74a65 ("test/cmake: add a function for generating unit
test targets") added a function for generating unit test targets in
CMake. This function makes code simpler and less error-prone.

Proposed patch adds a similar function for generating fuzzing test
targets in CMake.

NO_CHANGELOG=build infrastructure updated
NO_DOC=build infrastructure updated
NO_TEST=build infrastructure updated

(cherry picked from commit d9643bfd)

test/unit/guard.cc calls stack_break_f() recursively until the stack
overflows and a signal is fired, however it relies on undefined behavior
when compares pointers to local variables. Fixed by comparing
__builtin_frame_address() instead.

One of the examples of this UB is when ASAN allocates local variables on
fake stacks, in that case the test completes without the stack overflow.

Also this patch disables ASAN for stack_break_f() to keep the array on the
fiber stack (see the corresponding comment) and marks it as volatile to
avoid optimizing it away by the compiler.

Closes tarantool/tarantool-qa#323

NO_DOC=test fix
NO_CHANGELOG=test fix

(cherry picked from commit 05b696c7)

Since number type was introduced we can not assume if tuples are
equal by comparison then their sizes are equal too. So the place
the assumption is used is fixed.

Closes #8899

NO_DOC=bugfix

(cherry picked from commit f4de9faf)

This commit introduces the key_compare_and_skip_parts function
that advances the keys on equal parts. These changes has been
extracted from commit ca832f27
("tuple_compare: compare not only functional key with key").

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=see the next commit

Runtime tuple formats are reusable, which means that a tuple format
returned by runtime_tuple_format_new may not be brand new, but actually
be used by a Lua object. As a result, if we call any function that may
trigger Lua GC between runtime_tuple_format_new and tuple_format_ref,
the tuple format may be deleted, leading to a use-after-free bug. This
is what happens in lbox_tuple_format_new. Fix this issue by taking a
reference to the format before pushing a cdata object to the Lua stack
in lbox_push_tuple_format.

The issue was fixed in the master branch by commit 28ec245d ("lua:
fix heap-use-after-free bug in tuple format constructor"). This isn't
a clean cherry-pick because the code changed quite a bit.

Closes #8889

NO_DOC=bug fix
NO_TEST=difficult to reproduce, found by ASAN

(cherry picked from commit 4123061b)

* ci: support coveralls
* cmake: add code coverage support
* test: run flake8 static analysis via CMake
* test: fix E741 errors by pycodestyle
* test: fix E722 errors by pycodestyle
* test: fix E711 errors by pycodestyle
* test: fix E502 errors by pycodestyle
* test: fix E501 errors by pycodestyle
* test: fix E305 errors by pycodestyle
* test: fix E303 errors by pycodestyle
* test: fix E302 errors by pycodestyle
* test: fix E301 errors by pycodestyle
* test: fix E275 errors by pycodestyle
* test: fix E251 errors by pycodestyle
* test: fix E231 errors by pycodestyle
* test: fix E203 errors by pycodestyle
* test: fix E201 and E202 errors by pycodestyle
* test: suppress E131 errors by pycodestyle
* test: fix E128 errors by pycodestyle
* test: fix E122 errors by pycodestyle
* gdb: fix Python <assert> statement usage

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
NO_CHANGELOG=LuaJIT submodule bump

The `__sanitizer_start_switch_fiber()` function takes a pointer as the
first argument to store the current fake stack if there is one (it is
necessary when stack-use-after-return detection is enabled). When leaving a
fiber definitely, NULL must be passed so that the fake stack is destroyed.

Before this patch, NULL was passed for dead fibers, however this is wrong
for dead fibers that are recycled and resumed. In such cases ASAN destroys
the fake stack, and the fiber crashes trying to use it in `fiber_yield()`
upon return from `coro_transfer()`.

Closes tarantool/tarantool-qa#321

NO_DOC=bugfix
NO_TEST=tested by test-release-asan workflow

(cherry picked from commit 72a6abee)

Fixed the implementation of the box console.
Before this fix, result of `\set language` is shared between clients
via `console.connect`, despite the fact that clients have different
`box.session.id`. Now the parameter of the selected language is stored
by each client in his own `box.session.storage`.

Fixes #8817

NO_DOC=bugfix

(cherry picked from commit e4fda4b7)

Interactive console test helper could be used for remote connection too,
for that purpose prompt needs to be changed accordingly to connection
type.

This patch introduces getter and setter for the prompt. Prompt is
configured per session, so it is advised to create a new session
for each test case (e.g. with before_each()).

NO_CHANGELOG=test helper change
NO_DOC=test helper change

(cherry picked from commit ed86a729)

The ./test/luatest_helpers/interactive_tarantool.lua module is not
a luatest helper. So moving it to the ./test/ dir and removing empty
./test/luatest_helpers/.

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 5493db7c)

A basic (and pretty useless) example:

```lua
local it = require('test.luatest_helpers.interactive_tarantool')

local child = it.new()

child:execute_command('6 * 7')
local res = child:read_response()
t.assert_equals(res, 42)

child:close()
```

The module also contains `:read_line()`, `:assert_line()` helpers for
testing output directly: for example, when we need to catch a print()
from a background fiber. It provides has constants related to terminal's
control sequences.

A real usage can be seen in a next commit.

Part of #7169

NO_DOC=no user visible changes
NO_TEST=not applicable, it is a testing helper
NO_CHANGELOG=no user visible changes

(cherry picked from commit a9d96007)

As the underlying problem behind this injection is fixed in #7357 it can
be removed and `-i` flag could be used as initially intended.

Closes #7554
Requires #7357
NO_DOC=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 16d6e9d2)

The interactive mode has been ignored when stdin was not a tty and is no
more. Now results of another command can be handled by tarantool.
Before the patch:
```
$ echo 42 | tarantool -i
LuajitError: stdin:1: unexpected symbol near '42'
fatal error, exiting the event loop
```

After the patch:
```
$ echo 42 | tarantool -i
Tarantool 2.5.0-130-ge3cf64a6c
type 'help' for interactive help
tarantool> 42
---
- 42
...

```

Closes #5064

NO_DOC=bugfix

(cherry picked from commit 9965e3fe)

Bump test-run to new version with the following improvements:

- luatest: fix detect tarantool crash at exit [1]
- Fix bug when lua script name truncated by dot [2]
- Raise an error and log it if test timeouts are set incorrectly [3]
- Pin PyYAML version to 5.3.1 [4]
- Add ability to set path to executable file [5]
- Migrate tarantoolctl from tarantool repository [6]
- Fix test-run crash when default server is crashed [7]
- Disable reproduce content printing [8]

[1] tarantool/test-run@be693d1
[2] tarantool/test-run@a6405f1
[3] tarantool/test-run@d34ecb0
[4] tarantool/test-run@704420e
[5] tarantool/test-run@0a70001
[6] tarantool/test-run@ad43d8f
[7] tarantool/test-run@b31329e
[8] tarantool/test-run@31f0ced

NO_DOC=test
NO_TEST=test
NO_CHANGELOG=test

(cherry picked from commit f4511948)

Run release build LTO testing inside a Docker container created from the
`tarantool/testing:ubuntu-jammy-clang16` image with Clang 16 installed.

Closes #318

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 9d0cb54f)

Run release build testing inside a Docker container created from the
`tarantool/testing:ubuntu-jammy-clang16` image with Clang 16 installed.

Closes #317

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 9134dabd)

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

(cherry picked from commit bb74d6c9)

Add code owners for CI-related script and
for github automation directory.

NO_CHANGELOG=no code changes
NO_TEST=no code changes
NO_DOC=no code changes

(cherry picked from commit 9234763a)

In order to improve tests of Tarantool core
assign dedicated team to perform review of each
and every change.

NO_CHANGELOG=no code changes
NO_TEST=no code changes
NO_DOC=no code changes

(cherry picked from commit 532bada7)

To make changelog preparation less stressful let's pass each and every
change to changelogs/ directly through the doc team.

NO_CHANGELOG=no code changes
NO_TEST=no code changes
NO_DOC=no code changes

(cherry picked from commit 2970bd57)

In #7309 a truncation of a space that was referenced by foreign
key from some other space was prohibited.

It appeared that this solution is too bothering since a user can't
truncate a space even if he truncated referring space before that.

Fix it by allowing space truncate if referring spaces are empty.
Also allow drop of the primary index in the same case with the
same reason: logically the index along with all space data is
not needed for consistency if there's no referring data.

Note that by design space truncate is implemented quite similar
to space drop. Both delete all indexes, from secondary to primary.
Since this patch allows deletion of the primary index (which is
the action that actually deletes all data from the space), this
patch changes the result of space drop too: the space remains
alive with no indexes, while before this patch it remained alive
with no secondary indexes but with present primary. In both cases
the behaviour is quite strange and must be fixed in #4348. To
make tests pass I had to perform drop in box.atomic manually.

Closes #8946

NO_DOC=bugfix

(cherry picked from commit 983a7ec2)

All LuaJIT related LSan warnings were suppressed in the scope of the
commit 985548e4 ("asan: suppress all
LSAN warnings related to LuaJIT"), since all compiler flags tweaks were
enclosed in LuaJIT CMake machinery. As a result of the commit in LuaJIT
submodule tarantool/luajit@a86e376 ("build: introduce LUAJIT_USE_ASAN
option") ASan and LSan support has been finally added to LuaJIT runtime,
so it was decided to remove LSan suppressions for LuaJIT functions.
Unfortunately, it was not so easy as it looked like.

At first, Lua global state is not closed properly at Tarantool instance
exit (see <tarantool_free> in src/main.cc and <tarantool_lua_free> in
src/lua/init.c for more info), so LSan false-positive leaks are detected
(for more info, see #3071). Hence, the original LSan suppression for
lj_BC_FUNCC is returned back (temporarily) until the aforementioned
issue is not resolved.

Furthermore, the internal LuaJIT memory allocator is not instrumented
yet, so to find any memory faults it's worth building LuaJIT with system
provided memory allocator (i.e. enable LUAJIT_USE_SYSMALLOC option).
However, again, since Tarantool doesn't finalize Lua universe the right
way, so running Tarantool testing routine with LUAJIT_USE_SYSMALLOC
enabled generates false-positive LSan leaks. Return back here to enable
LUAJIT_USE_SYSMALLOC, when #3071 is resolved.

Last but not least, the default value of fiber stack size is 512Kb, but
several tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT submodule
(e.g. some cases with deep recursion in errors.lua or pm.lua) have
already been tweaked according to the limitations mentioned in #5782,
but the crashes still occur while running LuaJIT tests with ASan support
enabled. Experiments once again confirm the notorious quote that "640 Kb
ought to be enough for anybody".

Anyway, LuaJIT tests are added to <test-release-asan> target in .test.mk
and LUAJIT_TEST_ENV is extended with required ASan and LSan options.

Follows up #5878

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit bacf4e56)

* ci: introduce testing workflow with sanitizers
* build: introduce LUAJIT_USE_ASAN option
* test: introduce test:done TAP helper
* memprof: remove invalid assertions
* ci: clean up workflow for exotic builds

Closes #5878

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
NO_CHANGELOG=LuaJIT submodule bump

Applier thread uses lsregion to allocate the messages for tx thread. The
messages are freed upon return to the applier thread using a
corresponding lsr_id.

Due to a typo, one of the lsregion allocations was made with a postfix
increment of lsr_id instead of the prefix one. Essentially, part of a
new message was allocated with an old lsr_id, and might be freed early
by a return of a previous message.

Fix this.

Closes #8848

NO_DOC=bugfix
NO_TEST=covered by asan in #8901
NO_CHANGELOG=bugfix

(cherry picked from commit 0d5bd6b7)

This improves diff hunk name detection. Needed for checkpatch to
correctly detect if patched code belongs to a function.

NO_DOC=git
NO_TEST=git
NO_CHANGELOG=git

(cherry picked from commit 642584fd)

This patch fixes an issue in generate_column_metadata(). Prior to this
patch, the number of variable-only expressions was counted incorrectly
when temporary memory was allocated on region to store their positions.
However, although this allocation was incorrect, this did not lead to
any problems due to the specifics of the region allocations.

This patch fixes this by removing the temporary memory allocation.

Closes #8763

NO_DOC=no user-visible changes
NO_TEST=no user-visible changes
NO_CHANGELOG=no user-visible changes

(cherry picked from commit d4f143ad)

It is possible for interval to have days, hours, minutes and seconds
larger than INT_MAX (or less than INT_MIN). Before this patch, msgpack
decoding had failed to parse intervals with msgpack int64 and uint64.
int64_t should be enough to store any value allowed for datetime
intervals.

Closes #8887

NO_DOC=small bug fix

(cherry picked from commit 01c7ae11)

* FFI: Fix recording of union initialization.
* Fix maxslots when recording BC_VARG, part 2.
* Fix maxslots when recording BC_VARG.
* Fix BC_UCLO insertion for returns.
* ci: update job concurrency group definition

Part of #8825

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

Before this patch, one couldn't create new datetime interval with
boundary value from Lua. At the same time, it was possible to create
such interval from Lua through addition and subtraction. C range
verification allow to create boundary value intervals, error message
also implies that they should be allowed. (See #8878 for more info.)

Closes #8878

NO_DOC=small bug fix

(cherry picked from commit b2a001cc)

A read source iterator stores statements in a vy_history object using
vy_history_append_stmt(). If a statement can be referenced, it's
reference counter is incremented. If it can't, i.e. it belongs to a
memory source, it's stored in a vy_history object without referencing.

This works fine because memory sources are append-only. A problem arises
only when we get to scanning disk sources. Since we yield while reading
disk, a dump task may complete concurrently dropping the memory sources
and possibly invalidating statements stored in the iterator history.
Although we drop the history accumulated so far and restart the
iteration from scratch in this case, there's still an issue that can
result in a use-after-free bug in vy_read_iterator_next().

The problem is that we access the current candidate for the next
statement while evaluating a disk source after a disk read. If 'next'
refers to a referenced statement, it's fine, but if it refers to a
statement from a memory source, it may cause use-after-free because
the memory source may be dropped during a disk read.

To fix this issue, let's make vy_history_append_stmt() copy statements
coming from memory sources. This should be fine performance-wise because
we copied memory statements eventually in vy_history_apply() anyway,
before returning them to the user.

Note that we also have to update vy_read_iterator_restore_mem() because
it implicitly relied on the fact that 'next' coming from a memory source
can't be freed by vy_mem_iterator_restore(), which cleans up the memory
source history. Now, it isn't true anymore so we have to temporarily
take a reference to 'next' explicitly.

Closes #8852

NO_DOC=bug fix
NO_TEST=tested by ASAN

(cherry picked from commit 0e5a3cc2)

This commit follows up the previous one (i.e. the backport of the commit
ff57f990 ("cmake: introduce FIBER_STACK_SIZE option")), since the
commit 98f62cd0 ("test: make fiber_stack.test tap-compatible") and
its siblings have not been backported to 2.10 in scope of #7618.

NO_DOC=fixup for d296f732
NO_CHANGELOG=fixup for d296f732

In scope of the commit 82f4b4a3 ("lib/core/fiber: Increase default
stack size") the default value of fiber stack size is increased up to
512 Kb (you can find the reasons in the aforementioned commit message
and in https://github.com/tarantool/tarantool/issues/3418 description).

Some of the tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT repo
(e.g. some cases with deep recursion in errors.lua or pm.lua) have
already been tweaked according to the limitations mentioned in
https://github.com/tarantool/tarantool/issues/5782, but the crashes
still occurs while running LuaJIT tests with ASan support enabled.

To make the testing routine more convenient, FIBER_STACK_SIZE option is
introduced to Tarantool CMake machinery. One can provide the size either
by raw digits (i.e. in bytes) or using Kb/Mb suffixes for convenience.

A couple of important nits:
* If the given value is not a multiple of 4Kb, CMake machinery adjusts
  it up to the nearest one greater than this value.
* If the adjusted value is less than 512Kb, configuration fails with the
  corresponding CMake fatal error.

Follows up #3418
Relates to #5782

@TarantoolBot document
Title: introduce FIBER_STACK_SIZE configuration option

To make managing of the default fiber stack size more convenient, the
corresponding CMake option is added.

**NB**: The stack size can't be less than 512Kb and if the given value
is not a multiple of 4Kb, CMake machinery adjusts it up to the nearest
one greater than this value.

(cherry picked from commit ff57f990)

This reverts commit 8aa2bb19.

In scope of the aforementioned commit changes from commit e4fda4b7
("box: fix shared lang between connected clients") are backported, but
the test in this patch uses test.interactive_tarantool.lua helper, that
is missing in release/2.10 branch. Applying all the related commits from
the master listed below doesn't fix the problem:
* commit a9d96007 ("test: add a helper for testing interactive
  mode")
* commit 5493db7c ("test: mv interactive_tarantool.lua to ./test/
  dir")
* commit ed86a729 ("test: add prompt setter to interactive helper")

Hence, simply revert the poisoned commit from the branch.

NO_DOC=revert
NO_TEST=revert
NO_CHANGELOG=revert

Test uses a popen module that starts tarantool process in background
mode. Tarantool process started in background mode forks a new process
and closes a parent, after that popen loses a PID of the started process
and `ph:kill()` and `ph:terminate()` doesn't work anymore. It leads to
non-terminated tarantool processes after running the test.

Patch fixes that by running `kill` using os.execute with a PID of
tarantool process written to a pid file.

Follows up #6128

NO_CHANGELOG=fix test
NO_DOC=fix test

(cherry picked from commit 88686227)

Fixed the implementation of the box console.
Before this fix, result of `\set language` is shared between clients
via `console.connect`, despite the fact that clients have different
`box.session.id`. Now the parameter of the selected language is stored
by each client in his own `box.session.storage`.

Fixes #8817

NO_DOC=bugfix

(cherry picked from commit e4fda4b7)