It has been decided to move freebsd testing to the nightly basis due
to lack of human and hardware resources to support per-commit testing.

Part of tarantool/infra#187

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

The approach to build SDK since 3.0 release series is changed and will
be under development for some period of time. So the `test-sdk` job is
just irrelevant now and expected to fail. Let's remove the job until it
is clear how to implement the integration testing in the scope of the
new approach for SDK build.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Also we now can free all resources allocated in iproto threads.

Part of #8423

NO_TEST=rely on existing tests
NO_CHANGELOG=internal
NO_DOC=internal

When shutdown is started iproto stop accepting new connections yet new
connections are still possible. One can use box.session.new() or use
box.cfg{} to start accepting connections again. This is a minor bug but
in the next patch we are going to free all iproto resources properly and
rely on the fact that we stop accepting new connections. (Before
iproto_free we call iproto_drop_connections to make sure all connections
are finished. But if we continue to accept new connections then
connection that established after dropping may not be finished.)

Part of #8423

NO_CHANGELOG=internal
NO_DOC=internal

To be able to do configuration changes in the iproto_on_shutdown_f in
the next patch.

Part of #8423

NO_TEST=refactoring
NO_CHANGELOG=refactoring
NO_DOC=refactoring

The patch updates curl module to the version 8.5.0 [1][2] that brings a
number of functional fixes and security fix for CVE-2023-46219 (HSTS
long file name clears contents), see description in [2], and updates
CMake module for building curl library.

Changes in CMake module:

- Option `CURL_DISABLE_HEADERS_API` was added and disabled by default [4].
- Option `CURL_DISABLE_BINDLOCAL` was added and disabled by default [5].
- Option `CURL_DISABLE_INSTALL` was added and disabled by default [6].

1. https://curl.se/changes.html#8_5_0
2. https://github.com/curl/curl/compare/curl-8_4_0...curl-8_5_0
3. https://curl.se/docs/CVE-2023-46219.html
4. https://github.com/curl/curl/commit/33493db2af2dc6d9910f5d7c702aae6f63b8a6a6
5. https://github.com/curl/curl/commit/20bb363f25151febe9026b41b8ad65df6db20b68
6. https://github.com/curl/curl/commit/aace27b0965c10394544d1dacc9c2cb2fe0de3d3

NO_DOC=libcurl submodule bump
NO_TEST=libcurl submodule bump

The commit "lib: add ability to disable auths individually" [1][2] in
Curl 8.3.0 removes CURL_DISABLE_CRYPTO_AUTH and introduces CMake options
for a number crypto protocols.

The patch reflects this change in Tarantool's build infrastructure.

1. https://github.com/curl/curl/commit/e92edfbef64448ef461117769881f3ed776dec4e
2. https://github.com/curl/curl/pull/11490

Follows up #9086

NO_CHANGELOG=third_party
NO_DOC=third_party
NO_TEST=third_party

The commit "cmake: improve OpenLDAP builds" [1][2] in Curl deletes CMake
option CURL_USE_OPENLDAP.

The patch reflects this change in Tarantool's build infrastructure.

1. https://github.com/curl/curl/commit/751e168d93b4a58f3fbbe2908c0041ae2f934329
2. https://github.com/curl/curl/pull/12024

Follows up #9086

NO_CHANGELOG=third_party
NO_DOC=third_party
NO_TEST=third_party

Commit 2e31751d ("cmake: introduce target checkpatch") introduced a
target 'checkpatch'. However, target name was incorrect in a dummy
target, that generated when checkpatch tool is not found.
The patch fixes that.

NO_CHANGELOG=cmake
NO_DOC=cmake
NO_TEST=cmake

This reverts commit b471fc97.

Tests for GC finalizers were patched in the luajit
submodule. The submodule was updated in commit 1a0bafd5
("luajit: bump new version").

NO_DOC=Revert CI hack
NO_TEST=Revert CI hack
NO_CHANGELOG=Revert CI hack

Remove all changelogs reported in release notes for 3.0.0.

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

New swim:on_member_event triggers use a slightly changed new trigger API
because it uses closures as triggers and a context can be passed.
Firstly, swim:on_member_event returns closure over the new trigger
instead of passed handler. Secondly, ctx can be passed as the second
argument - it is considered as an old trigger if the object is callable
and as ctx otherwise. Also, ctx can be passed as the third argument - it
is considered as name if it is string and as ctx otherwise. And, ctx can
be passed as the fourth argument. When one uses key-value API, ctx can
be passed with key "ctx".

Closes #6484
Closes #8659

NO_DOC=later

Triggers that were moved to the trigger registry accept trigger name as
an optional argument and allow to set triggers using new key-value API.
Let's provide triggers, that were not moved to the trigger registry yet,
with the same functionality. For this purpose the main trigger setter,
lbox_trigger_reset, is updated.

Backward compatibility is a bit broken after this patch. Firstly, all
the triggers were listed in order of their insertion, which is reversed
order of their actual layout and call. Now they are listed in direct
oreder of call.

The patch changes API for all the triggers that were not moved to the
trigger registry except for swim triggers - this case is a bit more
complicated because of context that can be passed to the swim triggers.

Note that argument `top` of `lbox_trigger_reset` is changed to `bottom`
because all passed `top`s became irrelevant with the introduction of the
third argument - changing it to `bottom` will help avoid a similar
problem in the future.

Along the way, make transactional triggers' error more descriptive and
add box.session.on_access_denied to the test - it wasn't tested by
mistake.

Part of #6484
Part of #8659

NO_CHANGELOG=later
NO_DOC=later

The patch rewrites Lua implementation of trigger list in C as a wrapper
over list of lbox triggers.

Backward compatibility is a bit broken after this patch. Firslty, despite
usually all the triggers are inserted to the beginning of the trigger list,
trigger list, implemented in Lua, inserted new triggers at the end. The
patch fixes this inconsistency. Also, triggers using lbox_trigger_reset
return nil when a trigger is deleted, and triggers using Lua implementation
return removed trigger in this case. The behavior is unified now - all
triggers return nil on deletion.

Part of #6484
Part of #8659

NO_CHANGELOG=internal
NO_DOC=internal

Commit 0ea1ba87 ("third_party: update libcurl from 8.3.0 to 8.4.0")
bumps libcurl version and bringed a regression: http client returns
nothing using connection with HTTP/2 protocol. The problem could be
reproduced by etcd-client, see description in issue [1]. The problem in
http client is due to commit "h2: testcase and fix for pausing h2
streams" [2] in libcurl. With reverted commit in libcurl the problem is
gone.

To work around the issue, commit sets HTTP 1.1 as default protocol in
http client.

Needed for #9547

1. https://github.com/tarantool/tarantool/issues/9547
2. https://github.com/curl/curl/commit/6b9a591bf7d82031f463373706d7de1cba0adee6

NO_CHANGELOG=workaround
NO_DOC=workaround
NO_TEST=etcd client's tests in Tarantool EE

Since on_shutdown triggers are fired asynchronously in their own fibers,
they are not referenced by `event_trigger_iterator`. That's why, if
on_shutdown trigger is deleted while it is running (or is scheduled to
run), it will be instantly removed, and when it is finished, segmentation
fault will happen on its finalizer (`func_adapter_end`). Let's reference
all on_shutdown triggers to prevent such situation - they will be deleted
soon along with the whole event subsystem.

Closes #9275

NO_CHANGELOG=bugfix for unreleased feature
NO_DOC=bugfix

The new function allows to reference all triggers from an event to
prevent them from being deleted. Blessed triggers will be freed only
when the whole event subsystem is freed. It will be used for on_shutdown
triggers.

Part of #9275

NO_CHANGELOG=internal
NO_DOC=internal

This patch introduces 3 new transaction-related events, that can be used
for setting user-defined triggers:

 * box.before_commit - triggered when a transaction is ready to commit;
 * box.on_commit - triggered when a transaction is committed;
 * box.on_rollback - triggered when a transaction is rolled back.

Each of them have 3 versions, e.g. `box.on_commit' event has:

 * box.on_commit - global version, called for all transactions;
 * box.on_commit.space.test - called for transactions that write to
   space "test";
 * box.on_commit.space[512] - called for transactions that write to
   space with id 512.

These triggers are implemented via the new trigger registry and work
independently from the core `on_commit' and `on_rollback' triggers.
One of the main advantages of the new triggers is that they can be set
for all transactions, rather than setting them within each transaction.

Space-specific triggers are called prior to global ones. If a trigger-
function fails, the remaining triggers for this event (including global)
are not executed.

If a space-specific trigger is added to the registry within an active
transaction, it may or may not be called on commit/rollback of this
transaction (the behavior is unspecified).

The trigger-function for each event may take an iterator parameter.
Similar to core triggers, the iterator goes through the effects of every
request that changed a space during the transaction.

`box.before_commit' trigger-function restrictions:

 * Yield is allowed (on memtx without mvcc it will abort the txn);
 * Allowed to write into database spaces;
 * If the function raises an error, the transaction is rolled back.

`box.on_commit' and `box.on_rollback' trigger-function restrictions:

 * Yield is forbidden (it will crash Tarantool);
 * The function should not access any database spaces;
 * If the function raises an error, the error is simply logged.

Closes #5717

NO_DOC=later (it depents on the documentation of the `trigger` module)

This patch adds push_iterator() virtual method of the `func_adapter' and
its implementation for Lua functions.

Needed for #5717

NO_DOC=internal
NO_CHANGELOG=internal

A user is unlikely interesting in intermediate changes. Let's just say
that the new declarative configuration is supported now.

NO_DOC=changelog changes
NO_TEST=see NO_DOC

There may be some confusion, so let's start with a background
information.

There are `TT_*` environment variables introduced in commit 1b330121
("box: set box.cfg options via environment variables"). They're
interpreted by the `box.cfg()` call.

There are `TT_*` environment variables introduced in commit 82b0cff9
("config: introduce env source"). They're interpreted by the declarative
configuration logic, when tarantool starts with the `--name <...>` CLI
option.

box.cfg's env variables have names deduced from box.cfg option names,
while config's env variable names are deduced from the config schema.

Some options have the same names here and there, for example
`TT_REPLICATION_ANON` (from `box.cfg.replication_anon` and
`replication.anon`). However, there are ones that have different names,
for example `TT_LISTEN` and `TT_IPROTO_LISTEN`.

Moreover, the declarative configuration has its own restrictions on the
configuration data. For example, `TT_IPROTO_LISTEN` is always a list of
URIs (like `[{"uri": <...>, "params": {<...>}}]`), not a single URI, not
a string, not a number. The declarative configuration has a certain
shape and doesn't allow polymorphic values.

Next, handling of box.cfg's variables by the old code in `load_cfg.lua`
doesn't work well with the declarative configuration flow.

The main reason is that the new configuration flow calls `box.cfg()`
with all the `box.cfg` values set, including default ones. If a user
removes an option from its config, it applies its default. On the same
time it instructs `box.cfg()` to don't read the corresponding
environment variables.

This commit offers a partial solution: it adds support of the most of
the box.cfg environment variables. The values are added into the
configuration data with the lowest priority: if the same value is set
in, for example, a file configuration, the file's value is preferred.

The following box.cfg's environment variables are not handled in this
commit.

* `TT_LOG`
* `TT_METRICS`
* `TT_INSTANCE_NAME`
* `TT_REPLICASET_NAME`
* `TT_CLUSTER_NAME`
* `TT_FORCE_RECOVERY`,
* `TT_READ_ONLY`
* `TT_BOOTSTRAP_LEADER`
* `TT_REPLICATION`
* `TT_REPLICATION_CONNECT_QUORUM`

Fixes #9485

NO_DOC=looks more like a bug fix or a kind of compatibility layer

This function makes it easier to run a code that can't be run directly
for some reason: for example, it needs the initialized database.

It is a wrapper around treegen and justrun.

Part of #9485

NO_DOC=testing helper change
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

`TT_LISTEN` and `TT_REPLICATION` environment variables were interpreted
by `box.cfg()` in a confusing way if query parameters with values are
present. For example, `localhost:3301?transport=plain` was interpreted
as the following map: `{['localhost:3301?transport'] = 'plain'}`. Later,
`box.cfg()` looks into this map for known URI fields like `login`,
`password`, `uri`, `host`, `service` and so on. It found nothing and
doesn't start a listening socket.

The reason of such a behaviour is that the environment value is
interpreted as a mapping in the `key=value,key=value` format, because
there is `=` in it.

The patch changes this behavior for an `key=value,key=value` environment
variable that contains `?` in a key: now such a value is not interpreted
as a mapping.

Note: Everything said above is also applicable to the so called
multilisten case: when several URIs are defined in the environment
variable. The following URI list is interpreted correctly now.

NOWRAP
```sh
export TT_LISTEN=localhost:3301?transport=plain,localhost:3302?transport=plain
```
NOWRAP

Note 2: Examples are given with the `plain` transport, which is default,
but the query parameters are the way to define TLS options. They're
supported in Tarantool Enterprise Edition, see [1].

Fixes #9539

NO_DOC=bugfix

[1]: https://www.tarantool.io/en/doc/latest/enterprise/security/#traffic-encryption

`httpc` module has two GC-finalizers: the first one for a Lua http
client (C function `luaT_httpc_cleanup`) and the second one for a Lua
http chunked requests (C function `luaT_httpc_io_cleanup`) introduced in
commit 417c6cb7 ("httpc: introduce stream input/output interface").
In a C implementation HTTP requests depends on structures of HTTP client
and there is a problem with destroying Lua objects in `httpc` module -
these GC-finalizers are not synchronized. This could lead to at least
two problems:

There is a race with GC-finalization that leads to use-after-free errors
when HTTP client is collected before collecting HTTP request. In a
stacktrace the problem looks as below:

```
0x55ca7d47652e in crash_collect+256
0x55ca7d476f6a in crash_signal_cb+100
0x7fb876c42520 in __sigaction+80
0x55ca7d641e51 in curl_slist_free_all+35
0x55ca7d441498 in httpc_request_delete+45
0x55ca7d4653f1 in httpc_io_destroy+27
0x55ca7d4674bc in luaT_httpc_io_cleanup+36
0x55ca7d4e00c7 in lj_BC_FUNCC+70
0x55ca7d4f8364 in gc_call_finalizer+668
0x55ca7d4f8946 in gc_finalize+1387
0x55ca7d4f91e2 in gc_onestep+864
0x55ca7d4f9716 in lj_gc_fullgc+276
...
```

Lua object `http.client` could be GC-collected when chunked HTTP request
is alive. This will lead to an error "IllegalParams: io: request must be
io" because we call a method when Lua object is already a `nil`.

```lua
local url = 'https://bronevichok.ru/'
local c = require('http.client').new()
local r = c:get(url, {chunked = true})
c = nil
collectgarbage()
collectgarbage()
r:read(1) -- IllegalParams: io: request must be io
```

The patch introduces two functions: `httpc_env_finish` and
`curl_env_finish`, that prepares curl and httpc environments for
destruction. HTTP client's GC finalizer now calls `httpc_env_finish`
instead of `httpc_env_destroy`, this prevents from destroying memory
that could be in use by HTTP requests. Additionally `httpc_env_finish`
sets a flag `cleanup`. HTTP environment destroying is called when flag
`cleanup` is set and a there are no active HTTP requests. The main idea
of the patch is a synchronization of destructors for HTTP client and
HTTP chunked requests. Unfortunately, GC will eventually collect HTTP
client object after calling its `__gc`. To prevent this we put a
reference to a Curl's userdata in Lua objects with HTTP chunked requests
and HTTP default client.

Fixes #9346
Fixes #9453

NO_DOC=bugfix

Bump curl version to 8.4.0 triggers a crash in Tarantool due to commit
"h2: testcase and fix for pausing h2 streams" [1]. The original
reproducer involves etcd and an etcd-client Lua module, running
etcd-client tests as a part of Tarantool integration testing is planned
to do in scope of [1].

However, the problem could be reproduced with a Lua code below:

```
local url = 'https://google.com/'

local c = require('http.client').new()

r1 = c:get(url, {chunked = true})
r1:read(1)
r2 = c:get(url, {chunked = true})
r2:read(1)
r3 = c:get(url, {chunked = true})
r3:read(1)
r4 = c:get(url, {chunked = true})
r4:read(1)

c = nil
collectgarbage()
collectgarbage()

r1:read(1)
r2:read(1)
r3:read(1)
r4:read(1)

collectgarbage()
collectgarbage()
```

According to Curl documentation, `curl_multi_cleanup` [1] must be called
before any easy handles are cleaned up. The patch adds a cleanup of easy
handles on running `curl_env_destroy`, right before calling
`curl_multi_cleanup`. The patch uses a function 'curl_multi_get_handles'
that returns all added easy handles introduced in Curl 8.4.0. Therefore
bump to 8.4.0 is required.

1. https://github.com/curl/curl/commit/6b9a591bf7d82031f463373706d7de1cba0adee6
2. https://curl.se/libcurl/c/curl_multi_cleanup.html

Fixes #9283

1. https://github.com/tarantool/tarantool/issues/9093

NO_DOC=bugfix
NO_TEST=no simple reproducer, covered by tests in etcd-client

FreeBSD instances in Tarantool CI have installed libcurl package (as a
dependency of Zabbix monitoring agent). Curl 8.4.0 introduces a new
function `curl_multi_get_handles` that is used in the following commit
in `src/curl.c`, but libcurl system package has no such symbol in
headers. On building on FreeBSD in Tarantool CI C compiler produces a
warning about implicit declaration of function, because it looks at
system headers by default and due to enabled CMake option
`-DENABLE_WERROR=ON` building has failed:

```
[ 63%] Building C object src/CMakeFiles/server.dir/title.c.o
/.cache/act/55d136250dd94303/hostexecutor/src/curl.c:266:17: error: implicit declaration of function 'curl_multi_get_handles' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
                CURL **list = curl_multi_get_handles(env->multi);
                              ^
/.cache/act/55d136250dd94303/hostexecutor/src/curl.c:266:17: note: did you mean 'curl_multi_add_handle'?
/usr/local/include/curl/multi.h:140:23: note: 'curl_multi_add_handle' declared here
CURL_EXTERN CURLMcode curl_multi_add_handle(CURLM *multi_handle,
                      ^
/.cache/act/55d136250dd94303/hostexecutor/src/curl.c:266:10: error: incompatible integer to pointer conversion initializing 'CURL **' (aka 'void **') with an expression of type 'int' [-Werror,-Wint-conversion]
                CURL **list = curl_multi_get_handles(env->multi);
                       ^      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 errors generated.
```

The patch fixes that by reordering headers passed to compiler, see [1].

1. https://cmake.org/cmake/help/latest/command/include_directories.html

Needed for #9283

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

The patch updates curl module to the version 8.4.0 [1] that brings a
number of functional fixes and security fix of SOCKS5 heap buffer
overflow (CVE-2023-38545), see description in [2] and commit
fb4415d8aee6 ("socks: return error if hostname too long for remote
resolve") in [3].

1. https://curl.se/changes.html#8_4_0
2. https://curl.se/docs/CVE-2023-38545.html
3. https://github.com/curl/curl/commit/fb4415d8aee6c1045be932a34fe6107c2f5ed147

NO_DOC=libcurl submodule bump
NO_TEST=libcurl submodule bump

Message below is printed every time on shutdown `httpd.py` when
`test/app-luatest/http_client_test.lua` is running by luatest without
capturing stdout:

```
BrokenPipeError: [Errno 32] Broken pipe exception handling
```

The patch suppress this exception by adding a handler for a signal
`SIGPIPE`.

NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing

NO_CHANGELOG=fixed typos
NO_DOC=fixed typos
NO_TEST=fixed typos

The patch propagates debug mode to building of third party components:
c-ares, libcurl, libeio, nghttp2, zstd. Other components enables debug
mode automatically once it is enabled in Tarantool build.

Curl has two similar options that enables debug mode, however they are
different: `ENABLE_CURLDEBUG` enable memory debugging and `ENABLE_DEBUG`
restricts code which is only compiled for debug enabled builds [1].

1. https://everything.curl.dev/internals/memory-debugging

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

Fixed a bug when the assertion in box_promote_qsync would fail.
The assertion is that at the moment when box_promote_qsync is
executed, no other promote is executed. It turned out that this
assertion is basically incorrect. Now after this patch the newly
elected leader is trying to repeat box_promote_qsync in
box_raft_update_synchro_queue until it fails due to the fact
that some other promotion is currently being executed.

Closes #9263

NO_DOC=bugfix

This commit changes the order of configuration appliers and move
`console` before `box_cfg`. The `console` applier is adjusted to
correctly interpret the configured socket file path before the first
box.cfg().

The main reason to change the order is to allow a user to call
`box.ctl.make_bootstrap_leader()` for a replicaset that starts in the
`replication.boostrap_strategy: supervised` mode.

Part of #8862

NO_DOC=The documentation already assumes that a control command may be
       issued on the console. No changes are required.

It is important to listen on a Unix domain socket, because its path
length is limited to 107 on Linux and 103 on Mac OS (in our
implementation -- in theory could be 108 and 104).

Part of #8862

NO_DOC=no public API changes
NO_CHANGELOG=see NO_DOC
NO_TEST=negligible change

These functions are to be re-used in the console applier code in the
next commit. Exporting them allows to eliminate a duplication of this
code.

Part of #8862

NO_DOC=no public API changes
NO_CHANGELOG=see NO_DOC
NO_TEST=no new behavior, just refactoring

It is needed to verify the `supervised` bootstrap strategy. We should
start the instances, issue `box.ctl.make_bootstrap_leader` and only than
wait for `config:info().status == 'ready'`.

Let's support this flow in the replicaset management helper.

Part of #8862

NO_DOC=testing helper change
NO_CHANGELOG=see NO_DOC

Sometimes a testing code needs to interpret parts of the configuration
and it is convenient to look at the global scope only without diving
into group, replicaset, instance options.

Let's allow to set a global scope option in the configuration builder
helper.

Part of #8862

NO_DOC=testing helper change
NO_CHANGELOG=see NO_DOC

Before this patch a replica adds an alert regarding inability to write
new credentials from config on startup. In most cases these new
credentials are applied on a master and the warning becomes obsolete.
The problem is that it is not removed at this point. A manual
`config:reload()` is needed to flush it.

It makes the warning more confusing than helpful. Let's remove it for
now.

We can return the alert back in a future, but we should do it together
with a logic that drops the alert if the needed data is received from a
master.

Part of #8862

NO_DOC=bugfix

This patch sets requirements for vshard version that is supported by
config module.

Part of #8862

NO_TEST=internal

@TarantoolBot document
Title: config: supported vshard version

The vshard version supported by the config module cannot be less than
0.1.25. Note, that vshard will only be loaded if any of instances have
sharding storage role or sharding router role.

* FFI: Fix dangling reference to CType in carith_checkarg().
* FFI: Fix dangling reference to CType. Improve checks.
* FFI: Fix dangling reference to CType.
* FFI: Ensure returned string is alive in ffi.typeinfo().
* FFI: Fix missing cts->L initialization in argv2ctype().
* Abstract out on-demand loading of FFI library.
* test: fix flaky finalizer error handler tests
* test: adjust lua-Harness test error assertion
* Fix snapshot PC when linking to BC_JLOOP that was a BC_RET*.
* snap: check J->pc is within its proto bytecode
* Fix HREFK forwarding vs. table.clear().
* Fix FOLD rule for BUFHDR append.
* Prevent CSE of a REF_BASE operand across IR_RETF.
* test: rewrite sysprof test using managed execution
* test: disable buffering for the C test engine

Part of #9145

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

To fix #9507 before release. We need more time to prepare a good patch
dealing with the issue.

This reverts commit 26acba83.

NO_TEST=fixing ASAN run of existing test
NO_CHANGELOG=issue is only in unreleased version
NO_DOC=bugfix