Currently, we close the transport from transport from
`luaT_netbox_transport_stop`, and we do not wait for the worker fiber to
stop. This causes several problems.

Firstly, the worker can switch context by yielding (`coio_wait`) or
entering the Lua VM (`netbox_on_state_change`). During a context switch,
the connection can get closed. When the connection is closed, its receive
buffer is reset. If there was some pending response that was partially
retrieved (e.g., a large select), then after resetting the buffer we will
read some inconsistent data. We must not allow this to happen, so let's
check for this case after returning from places where the worker can switch
context. In between closing the connection and cancelling the connection's
worker, an `on_disconnect` trigger can be called, which, in turn, can
also yield, returning control to the worker before it gets cancelled.

Secondly, when the worker enters the Lua VM, garbage collection can be
triggered and the connection owning the worker could get closed
unexpectedly to the worker.

The fundamental source of these problems is that we close the transport
before the worker's loop stops. Instead, we should close it after the
worker's loop stops. In `luaT_netbox_transport_stop`, we should only cancel
the worker, and either wait for the worker to stop, if we are not executing
on it, or otherwise throw an exception (`luaL_testcancel`) to stop the
worker's loop. The user will still have the opportunity to catch this
exception and prevent stoppage of the worker at his own risk. To safeguard
from this scenario, we will now keep the `is_closing` flag enabled once
`luaT_netbox_transport_stop` is called and never disable it.

There also still remains a special case of the connection getting garbage
collected, when it is impossible to stop the worker's loop, since we cannot
join the worker (yielding is forbidden from finalizers), and an exception
will not go past the finalizer. However, this case is safe, since the
connection is not going to be used by this point, so the worker can simply
stop on its own at some point. The only thing we need to account for is
that we cannot wait for the worker to stop: we can reuse the `wait` option
of `luaT_netbox_transport_stop` for this.

Closes #9621
Closes #9826

NO_DOC=<bugfix>

Co-authored-by: Vladimir Davydov <vdavydov@tarantool.org>
(cherry picked from commit fcf7f5c4)

Cherry pick note: Dropped gh_9621_netbox_worker_crash_test because
box.iproto.encode helpers aren't available on 2.11.

Also, remove unreleased/ entries.

NO_DOC=changelog
NO_TEST=changelog
NO_CHANGELOG=changelog

Before this patch MVCC engine expected that if index_replace
sets `result` to NULL then index_replace sets `successor` to
something (NULL or existing tuple, depending on index type).
That looked fine because by contract `successor` is set when
true insertion was happened.

Unfortunately it was not considered that in case of part with
`exclude_null` option in index the insertion can be silently
skipped and thus `successor` can be not set. The latter access
of it was actually an UB.

Fix it by explicit check of tuple_key_is_excluded and work on
this case correctly.

Note that logically `index_replace` should return a flag whether
the new tuple was filtered (excluded) by key_def. But on the other
hand this flag is required only for mvcc while the function is
already has lots of arguments and it's very cheap to determine
this flag right from memtx_tx, so I decided to make the most
simple patch.

NO_DOC=bugfix

(cherry picked from commit 14e21297)

The CI/CD builds are performed on VK Cloud virtual machines, so the
access to VK S3 is more reliable than to GitHub archives.

In fact, we experience periodical download problems with source archives
on GitHub in Tarantool Enterprise Edition builds in CI/CD and it is the
motivation to backup the archives on our side. The problems appear quite
frequently last few days.

The download problems are not on VK Cloud side and not on GitHub side.
The packet loss is somewhere in the middle. I don't know an exact reason
for now.

NO_DOC=no user-visible changes
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

(cherry picked from commit 03445e6b)

Bump metrics package submodule. There are 8 new commits, but only
two of them affects Tarantool:
- Add election_leader_idle metric [1];
- test: run log capture tests on a separate server [2];
the other 6 affect documentation and CI/CD scripts of the original repo.
The latter one is required to bump luatest in test-run [3], since
otherwise metrics log capture test fails on a new version.

1. https://github.com/tarantool/metrics/commit/ba9726d9b0cdfb22aa56d852f7fdc7b0aa22756a
2. https://github.com/tarantool/metrics/commit/3370f856efd4172bf24916e66e3846eeb01550a8
3. https://github.com/tarantool/test-run/pull/426

NO_DOC=doc is a part of submodule
NO_TEST=repo runs submodule tests, no new test files are added

(cherry picked from commit 50f74250)

* ci: bump version of actions/checkout
* test: fix typo in the link to the issue
* test: refactor CMake macro LibRealPath
* test: move LibRealPath to the separate module
* test: more cautious usage of LD_PRELOAD for ASan
* test: fix lj-802-panic-at-mcode-protfail GCC+ASan
* ci: execute LuaJIT tests with GCC 10 and ASAN
* cmake: replace prove with CTest
* Prevent down-recursion for side traces.
* Handle stack reallocation in debug.setmetatable() and
  lua_setmetatable().
* profilers: print user-friendly errors

Closes #5994
Closes #9595
Closes #9217
Closes #9656

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

New commits:

* matras: increase max capacity from 2^31 to 2^32 blocks
* matras: fix compilation on macOS
* small: introduce the new method small_alloc_info

Closes #9864

NO_TEST=small submodule bump
NO_DOC=I will update https://github.com/tarantool/doc/issues/3816

(cherry picked from commit 1e86e0a4)

This is the maximum possible capacity of a hash table with 32-bit record
identifiers and 8-element `LIGHT_GROW_INCREMENT`.

Needed for #9864

NO_DOC=see next commit
NO_CHANGELOG=see next commit

(cherry picked from commit f955ca0c)

This function contains a bitwise optimization that returns wrong result
when table size is greater than 2^31. E.g., if table size is 0xB0000000
and hash is 0, it returns 0x80000000 instead of 0. Fix it.

Needed for #9864

NO_DOC=see next commit
NO_CHANGELOG=see next commit

(cherry picked from commit d279368c)

Currently, the connection state is updated after calling triggers. However,
the triggers can, in turn, cause a new state change. The state will be
updated the state in the wrong order, and the original state change will
overwrite the state change from the trigger. To fix this, let's update the
connection state before calling any triggers.

Closes #9827

NO_DOC=<bugfix>

(cherry picked from commit bb38b059)

For the `on_connect` trigger, if the trigger execution fails and an
exception happens, the connection is terminated and its state changes to
'error'. It allows the following filtering semantic: the client checked
some condition from the trigger and decided that the connection does not
suite him — the exception is thrown to indicate that the connection should
be terminated.

Currently, the `on_schema_reload` trigger behaves the same way. However,
filtering a connection from the `on_schema_reload` trigger or waiting for a
schema update does not seem has neither a reasonable semantic, nor a valid
use case. Let's make the `on_schema_reload` trigger behave the same way as
the `on_disconnect` trigger, i.e, log the exception, but otherwise ignore
it.

Closes #9679

@TarantoolBot document
Title: `on_schema_reload` trigger of `net.box` connections behaviour update
Product: Tarantool
Since: 3.1
Root documents: https://www.tarantool.io/en/doc/latest/reference/reference_lua/net_box/#lua-function.conn.on_schema_reload

When an error is thrown from the `on_schema_reload` trigger, it now behaves
the same way as the `on_disconnect` trigger [^1]:
> If the trigger function causes an error, the error is logged but
otherwise is ignored.

[^1]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/net_box/#lua-function.conn.on_disconnect

(cherry picked from commit bbd8b684)

According to the documentation [1]:
> If the trigger function causes an error, the error is logged but
otherwise is ignored.

However, currently, the `on_disconnect` trigger behaves the same way as the
`on_connect` trigger, i.e., the connection is terminated and its state
changes to 'error'. Let's fix this inconsistency and log errors from the
`on_disconnect` trigger, but otherwise ignore them.

Closes #9677
Closes #9797

NO_DOC=<bugfix>

1. https://www.tarantool.io/en/doc/latest/reference/reference_lua/net_box/#lua-function.conn.on_disconnect

(cherry picked from commit 1d6d6a3a)

It has been decided to drop required devx team review for tests.

NO_DOC=codeowners
NO_TEST=codeowners
NO_CHANGELOG=codeowners

(cherry picked from commit 58f5ffe4)

Header "unit.h" contains `ok` and `is` macros used to check test cases.
The problem is such simple names can be used in C++ STL library headers
(it's OK because such short names can be hidden in a namespace), so when
including, for example, header "vector" after "unit.h", build can fail
because function declaration or definition in the C++ header will turn
into a macro invocation. I faced this problem building Tarantool on
MacOS with SDK of 14.4 version.

NO_TEST=fix build
NO_CHANGELOG=fix build
NO_DOC=fix build

(cherry picked from commit 025ba32f)

Currently, exclude_null option doesn't affect functional indexes at all.
It seems that we just forgot to check if tuple should be inserted to the
index - the patch simply adds missing check in replace and build_next
methods of functional memtx_tree index.

Closes #9732

NO_DOC=bugfix

(cherry picked from commit c56998fa)

In order to prevent the garbage collection of the discarded connection,
asynchronous requests must reference the connection object. We must
reference the connection object rather than the transport object, because
our garbage collection hook is attached to the former.

Closes #9629

NO_DOC=<bugfix>

(cherry picked from commit fb5bf51c)

This patch adds necessary cmake configurations for the
<evread.lua> module, so it can be used later to implement
human-readable error reporting in profile parsers.

Part of #9217

NO_DOC=LuaJIT submodule
NO_TEST=covered by the LuaJIT tests
NO_CHANGELOG=build

(cherry picked from commit e01fe8f7)

This commit increases a time to wait of the process termination. It may
take longer than 5 seconds, when tarantool is built with an address
sanitizer. The address sanitizer generates a report at the process
termination and it is not always a fast thing.

NO_DOC=test fix
NO_CHANGELOG=see NO_DOC

(cherry picked from commit 5260bc2a)

This patch reduces the number of tests in the in2.test.lua test file.
This patch also reduces the number of inserted values. This shouldn't
affect the test since it's not really an original Tarantool test, but it
will reduce the execution time of this test. Currently this test often
fails due to a timeout.

NO_DOC=test
NO_CHANGELOG=test

(cherry picked from commit 0ae908cd)

* test: set dependencies in BuildTestCLib macro
* Add 'cc' file type for saving bytecode.
* Fix C file generation in jit.bcsave.
* Throw any errors before stack changes in trace stitching.
* Fix recording of __concat metamethod.
* Check frame size limit before returning to a lower frame.
* build: purge sysprof.collapse module
* build: fix tool components handling
* memprof: refactor `heap_chunk` data structure
* memprof: remove unused arguments
* memprof: introduce the `--human-readable` option
* profilers: introduce event reader module
* ci: extend tarantool integration testing

Part of #9595
Part of #5994
Follows up #8700
Needed for #9217

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

After the commit 13ac5daf ("ci: fix step parameters for
reusable runs") it is now possible to create Tarantool
integration workflows for any repository in the Tarantool
organization. Considering this, we don't need to run Tarantool
tests under the old target for the LuaJIT integration in
.test.mk and we can leave only LuaJIT tests in this target for
the sake of exotic Tarantool builds testing.

NO_DOC=CI
NO_TEST=CI
NO_CHANGELOG=CI

Due to a regression introduced in c13b3a31, the worker fiber is started
synchronously, while it should be started asynchronously, in order for the
`wait_connected = false` option of `connect` to work correctly. We already
explicitly wait from Lua for the connection to become active via
`wait_state` when `wait_connected = true`.

Closes #9489

NO_DOC=<bugfix>

(cherry picked from commit 0502a1f5)

Fixed a bug when it was possible that the `on_relay_thread_start`
trigger handler would continue to use `data` allocated on the stack
of `box_collect_confirmed_vclock` after returning from it.

Closes #9505

NO_DOC=bugfix
NO_TEST=asan

(cherry picked from commit 40bd0eb1)

Bump version of the `codeql-action` action to v3 for fixing an annoying
warning that appears in GitHub WebUI:

    Node.js 16 actions are deprecated.
    Please update the following actions to use Node.js 20

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 5040fba9)

Bump version of the `upload-artifact` and `download-artifact` actions
to v4 for fixing an annoying warning that appears in GitHub WebUI:

    Node.js 16 actions are deprecated.
    Please update the following actions to use Node.js 20

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit b012117d)

Bump version of actions/checkout to v4.
Bump fixes an annoying warning that appears in Github WebUI:

Node.js 16 actions are deprecated. Please update the following actions
to use Node.js 20: actions/checkout@v3. For more information see:
https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.

NO_CHANGELOG=ci
NO_DOC=ci
NO_TEST=ci

(cherry picked from commit 8aff2ede)

In commit 13ac5daf ("ci: fix step parameters for reusable
runs") the integration workflow was made reusable, but
concurrency group pattern modification that was done for other
workflows made reusable in the same patch was forgotten. This
patch fixes the mentioned issue.

NO_DOC=CI
NO_TEST=CI
NO_CHANGELOG=CI

The #8350 was introduced by the commit b42302f5 ("lua-yaml: enable
aliasing for objects returned by __serialize") so the patch is
effectively reversed.

The idea is to call all object __serialize methods recursively before
finding references. The new serialization pass stores the mapping from
the original object to the serialized representation.

After this, the reference analysis pass and the encoding pass use this
mapping to replace original objects with the serialized representation.

As result, the reference analysis has a complete information about
objects and no references are missed.

Closes #8350
Closes #8310
Closes #8321

NO_DOC=bugfix

Co-authored-by: Nikolay Shirokovskiy <nshirokovskiy@tarantool.org>
(cherry picked from commit 610f5fb7)

Now the error message is allocated by `malloc' if it doesn't fit into the
static buffer.

Closes #4975

NO_DOC=bugfix

(cherry picked from commit c8da06ca)

It brings newer codespell version: 2.1.0. Ubuntu Focal offers 1.16.0.

Fixes tarantool/checkpatch#70

NO_DOC=CI adjustment
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

(cherry picked from commit 21a779e1)

This patch fixes three issues:
1. It changes the condition for workflows so they can be run not
only from the Tarantool repository but from any repository in the
Tarantool organization.

2. Reusable workflows substitute the `${{ github.workflow }}`
context variable with the name of their top-level workflow. This
behavior causes concurrency group clashes when several reusable
workflows are called from a single top-level workflow. This patch
adds an additional constant part to the concurrency group pattern
to solve the issue.

3. The checkout actions use the reference from the repository in
which the top-level workflow is located instead of the one where
the reusable workflow is located. This patch solves the issue by
passing the reference explicitly.

NO_DOC=CI
NO_TEST=CI
NO_CHANGELOG=CI

Some workflows are not relevant for integration testing.
This patch disables them.

NO_DOC=CI
NO_TEST=CI
NO_CHANGELOG=CI

If we run a static build with ASAN enabled via Clang 16, the build will
fail unless `libresolv` is in the white list of static dependencies.
It looks like it is a peculiarity of Clang 16 and higher.

Fixes #9740

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

(cherry picked from commit 8f6bc366)

There are two reasons for this changeset:

* The positive one: Tarantool supports -b and -j options to use LuaJIT
  modules since the commit bf8b76a4
  ("lua: proxy -j and -b flags"), so the related tests from lua-Harness
  suite can be partially (since -O option is still not implemented in
  Tarantool) enabled.

* The negative one: Tarantool diff-based tests for CLI interfaces are
  hard to maintain, if any change occurs in LuaJIT modules, since the
  aforementioned tests implement dumb comparison of the output, produced
  by the current CLI version against the expected one, managed by the
  .result file. Hence, to rule the tests related to the LuaJIT CLI
  interface in a more convenient way, the corresponding tests should be
  moved from the tests in the Tarantool repository to the tests in the
  LuaJIT repository.

The recent LuaJIT bump landed to the master in the scope of commit
0dcf6759 ("luajit: bump new version")
enables the nice checks implemented in the lua-Harness suite; this patch
removes the barely maintainable diff-based tests from this repository.

Follows up #5541

NO_DOC=test
NO_CHANGELOG=test

(cherry picked from commit 137e9156)

* cmake: introduce AddTestLib macro
* test: prepare lauxilarily libs for LuaJIT-tests
* test: separate LuaJIT helpers from ffi_util.inc
* test: enable <ffi_arith_ptr.lua> in LuaJIT-tests
* test: enable <ffi_bitfield.lua> in LuaJIT-tests
* test: enable <ffi_call.lua> in LuaJIT-tests
* test: enable <ffi_callback.lua> in LuaJIT-tests
* test: enable <ffi_const.lua> in LuaJIT-tests
* test: enable <ffi_convert.lua> in LuaJIT-tests
* test: enable <ffi_enum.lua> in LuaJIT-tests
* test: enable <ffi_gcstep_recursive.lua>
* test: enable <ffi_jit_arith.lua> in LuaJIT-tests
* test: enable <ffi_jit_call.lua> in LuaJIT-tests
* test: enable <ffi_jit_conv.lua> in LuaJIT-tests
* test: enable <ffi_lex_number.lua> in LuaJIT-tests
* test: enable <ffi_metatype.lua> in LuaJIT-tests
* test: enable <ffi_new.lua> in LuaJIT-tests
* test: enable <ffi_parse_array.lua> in LuaJIT-tests
* test: enable <ffi_parse_basic.lua> in LuaJIT-tests
* test: enable <ffi_parse_cdef.lua> in LuaJIT-tests
* test: enable <ffi_parse_struct.lua> LuaJIT test
* test: enable <ffi_tabov.lua> LuaJIT test
* test: enable <lightud.lua> LuaJIT test
* test: enable <api_call.lua> LuaJIT test
* test: enable <catch_wrap.lua> LuaJIT test
* test: enable <catch_cpp.lua> LuaJIT test
* test: introduce routine to build error message
* test: enable CLI-related lua-Harness tests back

Closes #7834
Part of #9398
Follows up #5541

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
NO_CHANGELOG=add new tests

Disable the app-tap/iconv.test.lua test on RED OS due to missing
character encodings (UTF-16, UTF-16BE, etc).

NO_DOC=test
NO_TEST=test
NO_CHANGELOG=test

Enable LTO in the rpm spec file for RedOS 7+. Otherwise, we are getting
a compilation error.

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

Google fuzzing efforts revealed yet another bound condition
we don't handle well in the `tnt_strptime` function:

- for format `%m%g%W`;
- and input string `07001`.

We failed with assertion failure:
```
  | datetime_strptime_fuzzer: ./src/lib/core/datetime.c:148: \
_Bool tm_to_datetime(struct tnt_tm *, struct datetime *): \
Assertion `mday >= 1 && mday <= 31' failed.
```

Closes #8525

NO_TEST=updated fuzzer corpus
NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 4043664d)

I got four fails on the given tests in a row on debug-asan job in CI for
tarantool-ee.

It seems, tarantool-ee is more sensitive to small timeouts, when the
address sanitizer slows down the execution. Or I'm just lucky.

Anyway, the given tests don't really need small timeouts: increasing it
doesn't break any test logic, doesn't increase duration of the test in a
successful case and doesn't increase it in case of a failure.

The tests are more stable after the change: I verified it locally by
running each of the tests in parallel many times on tarantool built with
enabled address sanitizer.

See the following commits for details about the given test cases and the
problems behind.

* commit 1fcfb8c2 ("app: start init script event loop explicitly")
* commit 786eb2ac ("main: don't break graceful shutdown on init
  script exit")

Follows up #9266
Follows up #9411

NO_DOC=test adjustment
NO_CHANGELOG=see NO_DOC

(cherry picked from commit dfca3c6c)