The checks in box.schema.index.create() and box.schema.index.alter()
were case sensitive, also it was possible to insert incorrect index
options directly into `box.space._index`. Fixed by adding checks
to memtx_space_check_index_def() and vinyl_space_check_index_def().

Closes #8937

NO_DOC=bugfix

(cherry picked from commit 4e25384b)

And remove unused index_def_check_xc().

As index_def_dup() never returns NULL anymore, change index_create() and
index_read_view_create() return type to `void` and update their callers.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit f6d61754)

Fuzzing test for LuaJIT generates random Lua programs and executes them.
We want to build a fuzzing test that will produce Lua programs that will
not contain semantic errors and will trigger as much as possible
components in LuaJIT.

This proposed patch introduces metrics that gathered after running the
test. LuaJIT metrics gathered using LuaJIT getmetrics module [1]. All
gathered metrics test will output after running with a finite number of
runs or finite duration of time (options `-runs` and `-max_total_time`)
or after sending SIGUSR1 to a test process.

```
$ ./build/test/fuzz/luaL_loadbuffer/luaL_loadbuffer_fuzzer -runs=1000

<snipped>

Done 1000 runs in 1 second(s)
Total number of samples: 1000
Total number of samples with errors: 438 (43%)
Total number of samples with recorded traces: 87 (8%)
Total number of samples with snap restores: 30 (3%)
Total number of samples with abort traces: 55 (5%)
```

1. https://www.tarantool.io/en/doc/latest/reference/tooling/luajit_getmetrics/#getmetrics-c-api

NO_CHANGELOG=testing
NO_DOC=testing

The `__sanitizer_start_switch_fiber()` function takes a pointer as the
first argument to store the current fake stack if there is one (it is
necessary when stack-use-after-return detection is enabled). When leaving a
fiber definitely, NULL must be passed so that the fake stack is destroyed.

Before this patch, NULL was passed for dead fibers, however this is wrong
for dead fibers that are recycled and resumed. In such cases ASAN destroys
the fake stack, and the fiber crashes trying to use it in `fiber_yield()`
upon return from `coro_transfer()`.

Closes tarantool/tarantool-qa#321

NO_DOC=bugfix
NO_TEST=tested by test-release-asan workflow

(cherry picked from commit 72a6abee)

The function can't be called on an unconfigured instance because it
needs IPROTO threads up and running. Let's raise an error to avoid
a crash.

Since we have two other places where we need to raise the same error
(box.session.su and box.__index), let's introduce the new code
ER_UNCONFIGURED for this error.

Closes #8975

NO_DOC=bug fix

(cherry picked from commit 4fd2686e)

Pin in cache the collation identifiers that are referenced by space format
and/or indexes, so that they can't be deleted.

Closes #4544

NO_DOC=bugfix

(cherry picked from commit d69aa687)

It was possible to delete a collation, which is in use by a key_def.

Part of #4544

NO_DOC=bugfix
NO_CHANGELOG=next commit

(cherry picked from commit 07beb340)

Since number type was introduced we can not assume if tuples are
equal by comparison then their sizes are equal too. So the place
the assumption is used is fixed.

Closes #8899

NO_DOC=bugfix

(cherry picked from commit f4de9faf)

Bump test-run to new version with the following improvements:

- luatest: fix detect tarantool crash at exit [1]
- Fix bug when lua script name truncated by dot [2]
- Raise an error and log it if test timeouts are set incorrectly [3]
- Pin PyYAML version to 5.3.1 [4]
- Add ability to set path to executable file [5]
- Migrate tarantoolctl from tarantool repository [6]
- Fix test-run crash when default server is crashed [7]
- Disable reproduce content printing [8]

[1] tarantool/test-run@be693d1
[2] tarantool/test-run@a6405f1
[3] tarantool/test-run@d34ecb0
[4] tarantool/test-run@704420e
[5] tarantool/test-run@0a70001
[6] tarantool/test-run@ad43d8f
[7] tarantool/test-run@b31329e
[8] tarantool/test-run@31f0ced

NO_DOC=test
NO_TEST=test
NO_CHANGELOG=test

(cherry picked from commit f4511948)

Run release build LTO testing inside a Docker container created from the
`tarantool/testing:ubuntu-jammy-clang16` image with Clang 16 installed.

Closes #318

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 9d0cb54f)

Run release build testing inside a Docker container created from the
`tarantool/testing:ubuntu-jammy-clang16` image with Clang 16 installed.

Closes #317

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 9134dabd)

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

(cherry picked from commit bb74d6c9)

Add code owners for CI-related script and
for github automation directory.

NO_CHANGELOG=no code changes
NO_TEST=no code changes
NO_DOC=no code changes

(cherry picked from commit 9234763a)

In order to improve tests of Tarantool core
assign dedicated team to perform review of each
and every change.

NO_CHANGELOG=no code changes
NO_TEST=no code changes
NO_DOC=no code changes

(cherry picked from commit 532bada7)

To make changelog preparation less stressful let's pass each and every
change to changelogs/ directly through the doc team.

NO_CHANGELOG=no code changes
NO_TEST=no code changes
NO_DOC=no code changes

(cherry picked from commit 2970bd57)

There must be a couple of rules:
* foreign key from non-temporary space to temporary space must be
  forbidden since after restart all existing links will be broken.
* foreign key from non-local space to local space must be forbidden
  on any replica all existing can be broken.

This patch implements the rules.

Closes #8936

NO_DOC=bugfix

(cherry picked from commit 7d23b339)

In #7309 a truncation of a space that was referenced by foreign
key from some other space was prohibited.

It appeared that this solution is too bothering since a user can't
truncate a space even if he truncated referring space before that.

Fix it by allowing space truncate if referring spaces are empty.
Also allow drop of the primary index in the same case with the
same reason: logically the index along with all space data is
not needed for consistency if there's no referring data.

Note that by design space truncate is implemented quite similar
to space drop. Both delete all indexes, from secondary to primary.
Since this patch allows deletion of the primary index (which is
the action that actually deletes all data from the space), this
patch changes the result of space drop too: the space remains
alive with no indexes, while before this patch it remained alive
with no secondary indexes but with present primary. In both cases
the behaviour is quite strange and must be fixed in #4348. To
make tests pass I had to perform drop in box.atomic manually.

Closes #8946

NO_DOC=bugfix

(cherry picked from commit 983a7ec2)

The test fails on osx_debug quite frequently with the following error:

NO_WRAP
> box/before_replace.test.lua                                     [ fail ]
>
> Test failed! Result content mismatch:
> --- box/before_replace.result	Mon Aug  7 10:36:06 2023
> +++ /tmp/t/rejects/box/before_replace.reject	Mon Aug  7 10:42:03 2023
> @@ -899,6 +899,7 @@
>  ...
>  for i = 1,17 do gen_inserts() end
>  ---
> +- error: fiber slice is exceeded
>  ...
>  test_run:cmd('restart server test')
>  s = box.space.test_on_schema_init
> @@ -906,7 +907,7 @@
>  ...
>  s:count()
>  ---
> -- 1
> +- 0
>  ...
>  -- For this test the number of invocations of the before_replace
>  -- trigger during recovery multiplied by the amount of return values
> @@ -921,7 +922,8 @@
>  -- the value is only increased by 1, and this change is visible only in memory.
>  s:get{1}[2] == 1 + 16999 + 17000 + 1 or s:get{1}[2]
>  ---
> -- true
> +- error: '[string "return s:get{1}[2] == 1 + 16999 + 17000 + 1 o..."]:1: attempt to
> +    index a nil value'
>  ...
>  test_run:cmd('switch default')
>  ---
NO_WRAP

Let's try to increase the max fiber slice up to 15 seconds to avoid
that, like we do in other potentially long tests. (The default value
is 1 second.)

NO_DOC=test fix
NO_CHANGELOG=test fix

(cherry picked from commit 371d4110)

This should suppress the following coverity issues:

https://scan7.scan.coverity.com/reports.htm#v39198/p13437/fileInstanceId=146712118&defectInstanceId=18978766&mergedDefectId=1563095
https://scan7.scan.coverity.com/reports.htm#v39198/p13437/fileInstanceId=146712113&defectInstanceId=18978750&mergedDefectId=1563094

While we are at it, let's use space_by_id instead of space_cache_find
because read view creation is a rare operation affecting all spaces so
caching the last space by id doesn't make any sense.

NO_DOC=code health
NO_TEST=code health
NO_CHANGELOG=code health

(cherry picked from commit f2886dd0)

All LuaJIT related LSan warnings were suppressed in the scope of the
commit 985548e4 ("asan: suppress all
LSAN warnings related to LuaJIT"), since all compiler flags tweaks were
enclosed in LuaJIT CMake machinery. As a result of the commit in LuaJIT
submodule tarantool/luajit@a86e376 ("build: introduce LUAJIT_USE_ASAN
option") ASan and LSan support has been finally added to LuaJIT runtime,
so it was decided to remove LSan suppressions for LuaJIT functions.
Unfortunately, it was not so easy as it looked like.

At first, Lua global state is not closed properly at Tarantool instance
exit (see <tarantool_free> in src/main.cc and <tarantool_lua_free> in
src/lua/init.c for more info), so LSan false-positive leaks are detected
(for more info, see #3071). Hence, the original LSan suppression for
lj_BC_FUNCC is returned back (temporarily) until the aforementioned
issue is not resolved.

Furthermore, the internal LuaJIT memory allocator is not instrumented
yet, so to find any memory faults it's worth building LuaJIT with system
provided memory allocator (i.e. enable LUAJIT_USE_SYSMALLOC option).
However, again, since Tarantool doesn't finalize Lua universe the right
way, so running Tarantool testing routine with LUAJIT_USE_SYSMALLOC
enabled generates false-positive LSan leaks. Return back here to enable
LUAJIT_USE_SYSMALLOC, when #3071 is resolved.

Last but not least, the default value of fiber stack size is 512Kb, but
several tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT submodule
(e.g. some cases with deep recursion in errors.lua or pm.lua) have
already been tweaked according to the limitations mentioned in #5782,
but the crashes still occur while running LuaJIT tests with ASan support
enabled. Experiments once again confirm the notorious quote that "640 Kb
ought to be enough for anybody".

Anyway, LuaJIT tests are added to <test-release-asan> target in .test.mk
and LUAJIT_TEST_ENV is extended with required ASan and LSan options.

Follows up #5878

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit bacf4e56)

* ci: introduce testing workflow with sanitizers
* build: introduce LUAJIT_USE_ASAN option
* test: introduce test:done TAP helper
* memprof: remove invalid assertions
* ci: clean up workflow for exotic builds

Closes #5878

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
NO_CHANGELOG=LuaJIT submodule bump

The is_new and is_old methods are the same for all compat options so
they should be defined in a metatable. A good thing about this change
is that it removes is_new and is_old from serialization:

* Before:

NO_WRAP
  tarantool> require('compat').yaml_pretty_multiline
  ---
  - is_new: 'function: 0x4175d6e8'
    is_old: 'function: 0x4175d790'
    brief: |
      Whether to encode in block scalar style all multiline strings or ones
      containing "\n\n" substring. The new behavior makes all multiline string output
      as single text block which is handier for the reader, but may be incompatible
      with some existing applications that rely on the old style.

      https://tarantool.io/compat/yaml_pretty_multiline
    current: default
    default: new
  ...
NO_WRAP

* After:

NO_WRAP
  tarantool> require('compat').yaml_pretty_multiline
  ---
  - current: default
    brief: |
      Whether to encode in block scalar style all multiline strings or ones
      containing "\n\n" substring. The new behavior makes all multiline string output
      as single text block which is handier for the reader, but may be incompatible
      with some existing applications that rely on the old style.

      https://tarantool.io/compat/yaml_pretty_multiline
    default: new
  ...
NO_WRAP

To achieve that, we have to remove the option name from the usage error
message but it seems to be okay because such errors shouldn't happen in
practice and the error message is clear enough to figure out what went
wrong.

Follow-up #8807

NO_DOC=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 75b5fd05)

Applier thread uses lsregion to allocate the messages for tx thread. The
messages are freed upon return to the applier thread using a
corresponding lsr_id.

Due to a typo, one of the lsregion allocations was made with a postfix
increment of lsr_id instead of the prefix one. Essentially, part of a
new message was allocated with an old lsr_id, and might be freed early
by a return of a previous message.

Fix this.

Closes #8848

NO_DOC=bugfix
NO_TEST=covered by asan in #8901
NO_CHANGELOG=bugfix

(cherry picked from commit 0d5bd6b7)

Currently if tarantool exits during relay's final join stage,
corresponding thread isn't terminated. This causes the flakiness
of the replicaset_ro_mostly.test.lua.

Let's reuse the same relay, in which subscribe cord is running, for
the final join stage. This way the cord will be cancelled during
replication_free().

Closes #8082

NO_DOC=not user-visible
NO_TEST=fix flaky test
NO_CHANGELOG=not user-visible

Co-authored-by: Sergey Petrenko <sergepetrenko@tarantool.org>

(cherry picked from commit 70a68836)

This improves diff hunk name detection. Needed for checkpatch to
correctly detect if patched code belongs to a function.

NO_DOC=git
NO_TEST=git
NO_CHANGELOG=git

(cherry picked from commit 642584fd)

This patch fixes an issue in generate_column_metadata(). Prior to this
patch, the number of variable-only expressions was counted incorrectly
when temporary memory was allocated on region to store their positions.
However, although this allocation was incorrect, this did not lead to
any problems due to the specifics of the region allocations.

This patch fixes this by removing the temporary memory allocation.

Closes #8763

NO_DOC=no user-visible changes
NO_TEST=no user-visible changes
NO_CHANGELOG=no user-visible changes

(cherry picked from commit d4f143ad)

It is possible for interval to have days, hours, minutes and seconds
larger than INT_MAX (or less than INT_MIN). Before this patch, msgpack
decoding had failed to parse intervals with msgpack int64 and uint64.
int64_t should be enough to store any value allowed for datetime
intervals.

Closes #8887

NO_DOC=small bug fix

(cherry picked from commit 01c7ae11)

`api_check()` is the LuaJIT internal assertion. To prevent inconsistency
during internal assertion changes (for example, during backporting), use
glibc's `assert()` instead.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

(cherry picked from commit e78afb3b)

* FFI: Fix recording of union initialization.
* Fix maxslots when recording BC_VARG, part 2.
* Fix maxslots when recording BC_VARG.
* Fix BC_UCLO insertion for returns.
* ci: update job concurrency group definition

Part of #8825

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

Before this patch, one couldn't create new datetime interval with
boundary value from Lua. At the same time, it was possible to create
such interval from Lua through addition and subtraction. C range
verification allow to create boundary value intervals, error message
also implies that they should be allowed. (See #8878 for more info.)

Closes #8878

NO_DOC=small bug fix

(cherry picked from commit b2a001cc)

This refactoring will:

1. Move macros from a header to the source file.
Macros should be used in header only with undef to avoid redefinitions.
Undef directive is not useful since we want to use these macros in the
source file.

2. Remove `using namespace lua_grammar` from header.
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-using-directive

3. Moving serializer entry point and constant parameters into
luajit_fuzzer namespace.
It's a common practice in C++ to avoid name collisions.

4. Move serializer functions into anonymous namespace.
These functions are not a part of the interface so should have
static linkage.
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-unnamed2

5. Fix ConvertToStringDefault function.
It was logically wrong so it would generate an identifier `123` from
`*123`.

NO_CHANGELOG=internal
NO_DOC=fuzzer fix

(cherry picked from commit 56488e15)

LuaJIT fuzzer used to stop due to timeout caused by infinite cycles and
recursions. Counters were introduced for every cycle and function to
address LuaJIT fuzzer timeouts.

The idea is to add unique counters for every cycle and function to
ensure finite code execution, if it wasn't already. For while, repeat,
for cycles, local and global named, anonymous functions, counters will
be initialized before the code generated from protobuf, and checked
in the first body statement. An entry point for the serializer was
created to count cycles and functions for counter initialization.

The idea was taken from a paper "Program Reconditioning: Avoiding
Undefined Behaviour When Finding and Reducing Compiler Bugs" [1].

Here is an example of a change in serialized code made by this commit.

Before:
```lua
while (true) do
    foo = 'bar'
end
function bar()
    bar()
end
```

After:
```lua
counter_0 = 0
counter_1 = 0
while (true) do
    if counter_0 > 5 then
        break
    end
    counter_0 = counter_0 + 1
    foo = 'bar'
end
function bar()
    if counter_1 > 5 then
        return
    end
    counter_1 = counter_1 + 1
    bar()
end
```
Protobuf structures that reproduce the timeout problem were added to
the LuaJIT fuzzer corpus.

[1] https://www.doc.ic.ac.uk/~afd/homepages/papers/pdfs/2023/PLDI.pdf

NO_CHANGELOG=internal
NO_DOC=fuzzer fix

(cherry picked from commit 4d004bbe)

Cases in two switches had no breaks, so they were falling
through. Breaks were added to solve the problem. Code
generated by the LuaJIT fuzzer became more various.

NO_CHANGELOG=internal
NO_DOC=fuzzer fix

(cherry picked from commit 4430cac9)

Follows-up #4823

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

(cherry picked from commit 95d62cfc)

Function `datetime_strptime` decodes string with datetime according to
specified format, it accepts a datetime struct, buffer with datetime and
string with format in arguments. Fuzzing test used static string
"iso8601" as a format and it blocked fuzzing test to cover functions
used by datetime_strptime under the hood. Fuzz introspector shows that
code coveraged by a test is quite low.

Patch updates the test to make it more effective: buffer with datetime
and format string are generated using FDP (Fuzzing Data Provider).

Test file extension was changed to .cc, because FuzzingDataProvider is
used and we need building it by C++ compiler.

Function `tnt_strptime` uses assert, that triggered by fuzzing tests.
Therefore it was replaced with to if..then.

1. https://storage.googleapis.com/oss-fuzz-introspector/tarantool/

Fixes #8490

NO_CHANGELOG=fuzzing test
NO_DOC=fuzzing test
NO_TEST=fuzzing test

(cherry picked from commit a1bd6e0b)

Fixes #8502
Needed for #8490

NO_DOC=bugfix
NO_TEST=covered by fuzzing test

(cherry picked from commit 783a7040)

Patch adds a LuaJIT fuzzer based on libprotobuf-mutator and LibFuzzer.
Grammar is described via messages in protobuf format, serializer is
applied to convert .proto format to string.

For displaying generated code on the screen during fuzzing set
the environment variable 'LPM_DUMP_NATIVE_INPUT'.

For displaying error messages from lua functions set
the environment variable 'LUA_FUZZER_VERBOSE'.

Note: UndefinedBehaviourSanitizer is unsupported by LuaJIT (see #8473),
so fuzzing test is disabled when CMake option ENABLE_UB_SANITIZER is
passed.

Closes #4823

NO_DOC=<fuzzing testing of LuaJIT>
NO_TEST=<fuzzing testing of LuaJIT>

(cherry picked from commit a287c853)

Added Google's 'libprotobuf-mutator' and 'protobuf' libraries
for developing grammar-based LuaJIT and SQL fuzzers based on
LibFuzzer.

It is needed to build protobuf module from source because
by default, the system-installed version of protobuf is used
by libprotobuf-mutator, and this version can be too old.

Part of #4823

NO_CHANGELOG=<dependencies>
NO_DOC=<dependencies>
NO_TEST=<dependencies>

(cherry picked from commit b11072a6)

Added options for fuzzing and for getting more information
on debugging.

NO_CHANGELOG=<fuzzing options>
NO_DOC=<fuzzing options>
NO_TEST=<fuzzing options>

(cherry picked from commit 69f21e25)

To achieve that, we bypass the read-only check for the _truncate system
space in box_process1() and perform it in the on_replace system trigger
instead, when we know which space is truncated.

Note, we have to move the check for insertion of a new record into the
_truncate system space before the read-only check in the on_replace
trigger callback; this is needed for initial recovery with a non-empty
_truncate space to work. While we are at it, let's use recovery_state to
make the check explicit.

Closes #5616

@TarantoolBot document
Title: Mention that temp and local spaces can be truncated in ro mode

DML operations on temporary and local spaces can be performed even if
the instance is in the read-only mode, but DDL operations (such as
`alter`) are forbidden in this case[^1]. Technically, `truncate` is
a DDL operation so initially it was forbidden as well. However, it
should be safe to perform this operation on a temporary or local space
because logically it only modifies the data stored in the space (like
DML) and it isn't replicated (see tarantool/tarantool#4263). So starting
from Tarantool 2.11.1 we allow users to truncate temporary spaces in the
read-only mode.

[^1]: https://www.tarantool.io/en/doc/latest/concepts/replication/repl_architecture/#replication-local

(cherry picked from commit 054526ac)