The commit adds msgpack object to func_adapter. It is required to pass
xrow object to recovery triggers.

Part of #8859

NO_CHANGELOG=internal
NO_DOC=internal

Before this patch privileges for space, function and sequence alone
were disabled. Only permission for universe could be granted. It was
done because there wasn't a way to grant or revoke a privilege when
subjecting object hasn't been created yet.

This patch allows grants and revokes for objects that will be created
after the applier. These actions will be postponed and executed via
on_commit trigger after the object is registered in the according system
space of box.space._space/_func/_sequence.

Now permission can be granted to space/function/sequence from within the
config.

Part of #8967

NO_DOC=documentation request will be filed manually for the whole
       credentials

Before this patch reload_success_case() didn't allow to pass arguments
to verify_2() in the same way as to verify(). Now they can be passed
via opts.verify_args_2.

Part of #8967

NO_DOC=test helper upgrade
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

This patch sets the aforementioned option to new. This is a breaking
change that makes the behaviour of the stored C functions called locally
and remotely via iproto consistent.

Closes #4799
Relates to #8576

NO_DOC=see previous commit

With this option enabled (new), the multiresults returned by a stored C
function via iproto aren't wrapped in the additional msgpack array (old).

Due to new behaviour some renames are performed:
* `port_c_dump_msgpack()` -> `port_c_dump_msgpack_wrapped()`, since this
  is dump format with additional msgpack array encoded.
* `port_c_dump_msgpack16()` -> `port_c_dump_msgpack()`, since this
  format is now the default new format of a msgpack dump.

The behaviour of the C port msgpack dumping depends on the
`c_func_iproto_multireturn` option:
* uses `port_c_dump_msgpack()` if set to true (new),
* uses `port_c_dump_msgpack_wrapped()` otherwise (old).

Needed for #4799

@TarantoolBot document
Title: Document `c_func_iproto_multireturn` compat option

Please create a documentation page for the new compat option:
https://tarantool.io/compat/c_func_iproto_multireturn

In the new behaviour, the multiresults returned by a stored C function
via iproto aren't wrapped in the additional msgpack array (old).

```
tarantool> compat.c_func_iproto_multireturn = 'old'
---
...

tarantool> net_box.connect(box.cfg.listen):call('myclib.cfunc')
---
- [true, -1]
...

tarantool> compat.c_func_iproto_multireturn = 'new'
---
...

tarantool> net_box.connect(box.cfg.listen):call('myclib.cfunc')
---
- true
- -1
...

```

The new behaviour is consistent with the local call of the function
via `box.func`:

```
tarantool> box.func['myclib.cfunc']:call()
---
- true
- -1
...

```

Assume you have a stored C function that returns values like the
following:

```c
char *position = mp_encode_bool(buffer, true);
box_return_mp(ctx, buffer, position);
/* ... */
position = mp_encode_int(buffer, -1);
box_return_mp(ctx, buffer, position);
```

If you want to preserve the format of the returned array for your C
functions, when the `c_func_iproto_multireturn` option is set to "new",
you should add the additional wrapping, like the following:

```c
char *position = mp_encode_array(buffer_with_results, n_results);
position = mp_encode_bool(position, true);
/* ... */
position = mp_encode_int(position, -1);
box_return_mp(ctx, buffer_with_results, position);
```

The amount of `box_return_mp()` calls indicates the number of values to
be returned.

Also, you should update its usage via `box.func` if there is any.

This patch sets the `tarantool` package as a dependency for
`tarantool-dev` and `tarantool-devel` packages.

Follows up #8771

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

Before this commit, packages were generated in one iteration because
their contents were identical. Now, we need to put some files in the
different paths depending on the package format. The simplest way to
resolve this issue is to split creating packages into separate steps,
one for RPM and one for DEB.

Follows up #8771
Closes tarantool/infra#188

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

Follows up #8771
Part of tarantool/infra#188

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

luarocks version updated to version 3.9.2

Closes #6597

NO_DOC=The engine has been updated, the functionality has not changed
NO_TEST=see NO_DOC

Before this patch if one called `vinyl`, `sql`, `gc` and `memory`
functions from box.info() instance crashed. It's interesting that
`replication_anon` functions worked ok.
This patch fixes that crashes.

Closes #9173

NO_DOC=bugfix

Part of https://github.com/tarantool/tarantool-ee/issues/564

NO_DOC=The documentation request is to be added as part of Tarantool
       Enterprise Edition patchset.
NO_CHANGELOG=see NO_DOC
NO_TEST=To be tested in Tarantool Enterprise Edition.

This message doesn't give any useful information for `tarantool
--failover` run, because it always starts background fibers for
monitoring purposes. Let's suppress the message in the case.

Part of https://github.com/tarantool/tarantool-ee/issues/564

NO_DOC=It is a small visual improvement, there is nothing to document
       here.
NO_CHANGELOG=It is part of Tarantool Enterprise Edition feature.
NO_TEST=Doesn't worth it.

The option enables a built-in failover coordinator script, which reads a
configuration from a file (pointed by --config option) or from etcd,
connects to all the instances, polls them for their status and performs
automatic actions to ensure that there is an active leader in each
replicaset (if possible).

The failover coordinator code is part of Tarantool Enterprise Edition.

Part of https://github.com/tarantool/tarantool-ee/issues/564

NO_DOC=The documentation request is to be added as part of Tarantool
       Enterprise Edition patchset.
NO_CHANGELOG=see NO_DOC
NO_TEST=To be tested in Tarantool Enterprise Edition.

It allows to reuse the code that collects the cluster configuration
without checks against particular instance name.

Part of https://github.com/tarantool/tarantool-ee/issues/564

NO_DOC=refactoring
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

I want to use it outside of the configdata code. It seems appropriate to
expose it from the instance config schema, because it is a general
purpose transformation of instance config data.

Part of https://github.com/tarantool/tarantool-ee/issues/564

NO_DOC=refactoring
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

Our split-brain detection machinery relies among other things on all
nodes tracking the synchro queue confirmed lsn. This tracking was only
added together with the split-brain detection. Only the synchro queue
owner tracked the confirmed lsn before.

This means that after an upgrade all the replicas remember the latest
confirmed lsn as 0, and any PROMOTE/DEMOTE request from the queue owner
is treated as a split brain.

Let's fix this and only enable split-brain detection on the replica set
once the schema version is updated. Thanks to the synchro queue freeze
on restart, this can only happen after a new PROMOTE or DEMOTE entry is
written by one of the nodes, and thus the correct confirmed lsn
is propagated with this PROMOTE/DEMOTE to all the cluster members.

Closes #8996

NO_DOC=bugfix

This patch introduces a new CLI option: --force-recovery.

Closes #8876

@TarantoolBot document
Title: --force-recovery CLI option

The --force-recovery CLI option is another way to set the box.cfg{}
force_recovery option. The priority of this option is higher than the
priority of the TT_FORCE_RECOVERY environment variable, but less than
the priority of explicitly setting the box.cfg{} force_recovery option.

* test: fix fix-mips64-spare-side-exit-patching
* test: fix `fillmcode()` generator helper
* MIPS: Fix "bad FP FLOAD" assertion.
* Handle table unsinking in the presence of IRFL_TAB_NOMM.
* Fix handling of instable types in TNEW/TDUP load forwarding.
* Fix frame for more types of on-trace error messages.
* Fix frame for on-trace out-of-memory error.
* Fix predict_next() in parser (again).
* Always exit after machine code page protection change fails.

Closes #562
Part of #8825

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

Allocations from fiber region and heap should never fail so it's okay to
use xmalloc for them.

This fixes the following coverity report complaining about unchecked
region_alloc result:

https://scan7.scan.coverity.com/reports.htm#v39198/p13437/fileInstanceId=149497639&defectInstanceId=19021351&mergedDefectId=1525560

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The goal of this patch is to make the xlog writer API less confusing.
It does the following changes:

 - Rename xlog_rename to xlog_materialize. The function strips the
   .inprogress suffix from an xlog file name. Usually it's called after
   the xlog was sealed (by writing the EOF marker) and closed but it may
   also be called before xlog_close in case the xlog file is going to be
   appended to (used in WAL).

 - Make xlog_sync private. It's called by xlog_close so there's no point
   in calling it manually. Make xlog_close fail if it fails to sync the
   file.

 - Make xlog_close flush the buffer so that the caller doesn't need to
   call xlog_flush manually before closing an xlog file.

 - Drop the reuse_fd argument of xlog_close. Instead, introduce a new
   function xlog_close_reuse_fd that works exactly like xlog_close
   except it doesn't close the fd and returns it in the out argument.

 - Introduce xlog_discard. It's supposed to be called on an incomplete
   xlog file to close its fd and unlink it.

Rework all the places where we write xlog files in accordance with the
new API. Note about memtx snapshot writer: use xlog_materialize and
xlog_discard instead of coio_rename and coio_unlink.

The new API (especially the xlog_discard helper) facilitates
introduction of thorough (secure) file deletion because it significantly
reduces the number of places in the code where xlog files are deleted.

Needed for tarantool/tarantool-ee#540

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The goal of the script is to assist with removing changelog entries that
are already included into a release of another development branch.

For example, if a bug fix is released as part of 2.11.3, there is no
reason to highlight that it is fixed in 3.0.0. It is better to describe
changes comparing to the latest present 2.11 release.

The script was initially written for personal use and there are obvious
points for improvements. However, it seems that it already simplifies
some tasks, so I'm going to share it as is and improve later. I'll leave
the relevant issue open.

Part of #9135

NO_DOC=The commit adds a development tool and doesn't touch anything in
       tarantool itself.
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

Proposed ASAN implementation of region allocator does not support double
reservation for the sake of simplicity. Every reservation is supposed to
be followed by one or more allocations.

This restiction does not work well with mpstream currently. The issue is
mpstream_init/mpstream_reserve do reservation of size 0. For example In
case of region slab of min order is reserved (a chunk of memory of page
size currently). If the first data we want to write to mpstream is
larger then the reservation done then we make reservation again.

Let's get rid of this reservation at the beginning as it is suboptimal
behaviour. Moreover let's get rid of mpstream_reset as mpstream_init
is lightweight and we can create a new mpstream instead of reusing
exiting.

Also while we at it avoid allocation of 0 size in mpstream_flush as it
is done in mpstream_reserve_slow (see 3.0.0-alpha3-19-g8159347d0 "misc:
avoid allocations of size 0 for region" for details).

NO_TEST=internal
NO_CHANGELOG=internal
NO_DOC=internal

Download the readline and libiconv archives from the backup storage to
avoid network issues with accessing https://ftp.gnu.org in the future.

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

Regular region implementation supports allocations of size 0 with no
extra efforts. It returns a non-NULL pointer in this case. However in
case of ASAN friendly implementation it will require a special care for
this case. Instead let's avaid allocations if size 0 for region.

Also use xregion_ macros for allocations. Our current policy is to panic
on OOM on runtime allocations.

Part of tarantool/tarantool#7327

NO_TEST=internal
NO_CHANGELOG=internal
NO_DOC=internal

When a replica is restarted, it doesn't immediately switch to the
'follow' state. First it needs to sync. Let's wait for the 'follow'
state a bit to fix the following failure:

NO_WRAP
[039] replication-luatest/gh_9049_schema_downgrade_t>                 [ fail ]
[039] Test failed! Output from reject file /tmp/t/rejects/replication-luatest/gh_9049_schema_downgrade.reject:
[039] Tarantool version is 3.0.0-alpha3-18-g6892907bad9d
[039] TAP version 13
[039] 1..1
[039] # Started on Thu Sep 21 13:55:09 2023
[039] # Starting group: replication-luatest.gh_9049_schema_downgrade
[039] not ok 1  replication-luatest.gh_9049_schema_downgrade.test_schema_downgrade
[039] #   ...st/replication-luatest/gh_9049_schema_downgrade_test.lua:33: expected: "follow"
[039] #   actual: "sync"
[039] #   stack traceback:
[039] #         ...st/replication-luatest/gh_9049_schema_downgrade_test.lua:31: in function 'replication-luatest.gh_9049_schema_downgrade.test_schema_downgrade'
[039] #         ...
[039] #         [C]: in function 'xpcall'
[039] #   artifacts:
[039] #   replica -> /tmp/t/039_replication-luatest/artifacts/replica-AgoXGWUXxvnc
[039] #   master -> /tmp/t/039_replication-luatest/artifacts/master-ZUTtyXyAX9YF
[039] # Ran 1 tests in 1.768 seconds, 0 succeeded, 1 failed
NO_WRAP

Fixes commit 71de4b2c ("box: fix schema downgrade replication").

NO_DOC=test fix
NO_CHANGELOG=test fix

The new option is backed by `box.cfg.auth_retries`. It is available only
in Enterprise Edition builds.

Needed for tarantool/tarantool-ee#541

NO_DOC=will be added to Enterprise Edition
NO_CHANGELOG=will be added to Enterprise Edition

Required to suppress the ASAN leak detector.

Closes #9158

NO_DOC=ASAN
NO_TEST=ASAN
NO_CHANGELOG=ASAN

Before this patch credentials applier used to just skip if Tarantool
was in Read Only mode. Now it starts a fiber that waits for instance
to be switched to Read Write mode and then applies in the background.

Part of #8967

NO_DOC=documentation request will be filed manually for the whole
       credentials

We decided not to merge entries from Tarantool CE changelog to
Tarantool EE changelog. Instead we now add a link to CE release notes
to EE release notes. Update the script accordingly.

NO_DOC=tools
NO_TEST=tools
NO_CHANGELOG=tools

If MP_ERROR map contains two MP_ERROR_STACK keys, then the second call to
`error_set_prev(effect, cur)' will crash, because `effect' is NULL, but
`err == NULL' is false, because it is assigned on the first iteration.
This patch raises an error if more than one MP_ERROR_STACK key is present.

NO_DOC=bugfix

Closes #9136

Wraps multistatement DDL functions into begin/commit block if no
transaction is active.

The functions are:
- `box.schema.space.drop`
- `box.schema.index.create`
- `box.schema.index.drop`
- `box.schema.index.alter`
- `box.schema.sequence.drop`
- `box.schema.func.drop`
- `box.schema.user.create`
- `box.schema.user.drop`
- `box.schema.role.drop`

Added tests for atomicity of each transactioned function except
the `box.schema.role.drop`, which is implicitly tested with the
`box.schema.user.drop` test, and the `box.schema.index.drop`,
which is impossible to test in this flavor without using error
injection.

Updated the tests modified in tarantool/tarantool#8947, because
the space drop is atomic now.

Closes #4348

NO_DOC=bugfix

Sequences can only be attached to integer fields of a space. The
corresponding check is done in both schema.lua and alter.cc. The
problem is that in order to make the index alter transactional,
we have to perform the `_index` space tuple replace before the
existed sequence is unattached from the space (because the index
alter may be yielding). And the check existing in the alter.cc
won't allow us to do that in some cases.

Since the check already exists in `box.schema.index.alter`, it's
decided to drop it from the `_index` `on_replace` trigger.

The downside of the solution is that uncareful manual altering the
index can silently put the space into an inconsistent condition:
the next insert into the space can either fail or silently succeed
whereas it's not supposed operation for the space. Example:

```lua
box.cfg{}
s = box.schema.space.create('s')
pk = s:create_index('pk', {parts = {{1, 'unsigned'}}, sequence = true})

-- Manual altering the index without dropping the sequence.
box.space._index:update({s.id, pk.id}, {{'=', 6, {{0, 'string'}}}})

-- This will fail: passed integer (from sequence) in string field.
s:insert({box.NULL})

-- Manual altering the index without dropping the sequence again.
box.space._index:update({s.id, pk.id}, {{'=', 6, {{0, 'scalar'}}}})

-- This will succeed: passed integer (from sequence) in scalar field.
-- Though it shouldn't have been done at all: one should not pass
-- NULL in the field which is indexed by the primary key.
s:insert({box.NULL})
```

NO_DOC=no visible changes
NO_TEST=no visible changes
NO_CHANGELOG=no visible changes

Before this commit disable of functional index caused index rebuild
which does not allow the box.schema.space.drop function to be
atomic. Since effectively the disabled functional index rebuild is
a no-op, the "rebuild" step was omitted and effective changes are
applied instead.

NO_TEST=no functional changes
NO_DOC=no functional changes
NO_CHANGELOG=no functional changes

This patch is intended to resolve the following issue that appears while
calling `cmake` for submodules from time to time:

    fatal: No tags can describe '<sha>'.
    Try --always, or create some tags.

This happens because the `actions/checkout` GitHub action fetches the
repository with the `--unshallow` option, but it doesn't do the same for
submodules. So this leads to the error above while using `git describe`
for submodules.

This is fixed by converting a shallow submodule to a complete one.

NO_DOC=ci
NO_TEST=circuit
NO_CHANGELOG=ci

The patch updates curl module to the version 8.3.0 [1] and updates a
CMake module for building curl library.

Changes in CMake module:

- Option `CURL_STATICLIB` is gone and replaced with `BUILD_STATIC_LIBS`.
- Option `CURL_USE_GNUTLS` was added and disabled by default.
- NSS library support was removed and option `CURL_USE_NSS` has been
  removed as well.
- Option `CMAKE_UNITY_BUILD` was added and disabled by default.
- Option `CURL_DISABLE_FORM_API` was added and disabled by default. It is
  in fact depends on `CURL_DISABLE_MIME`, but anyway disabled explicitly.

Changelog: https://curl.se/changes.html#8_3_0

1. https://github.com/curl/curl/releases/tag/curl-8_3_0

NO_DOC=libcurl submodule bump
NO_TEST=libcurl submodule bump

Fixes #9086

The commit populates submodule session with an event for each public
trigger - they will be used for user-defined triggers. Old triggers are
not removed because they are used for internal purposes - audit from EE
uses them, for instance. Also, the commit introduces a test for all
triggers that use old API. It checks only session triggers now - all the
other triggers will be included there after they will be moved to the
trigger registry.

In order to have an opportunity to move user-defined triggers to trigger
registry, the commit introduces helper luaT_event_reset_trigger that
supports two different ways to set triggers - key-value API and positional
API. The first one is a new API that allows to set or delete trigger only
by name. The second API is implemented for the sake of backward
compatibility - it mimics behavior of function lbox_trigger_reset, which
was used to modify triggers, but sets the triggers to event from trigger
registry instead of separate trigger list. Also, positional API allows
to pass a name as the third argument - in this case the function will
set or delete trigger by passed name - the first argument is a new
trigger (or nil, if the function is called to delete a trigger), the
second argument will be ignored in this case.

The new helper supports not only functions - any callable object can be
used as a trigger.

Part of #6484
Part of #8657

NO_CHANGELOG=later
NO_DOC=later

Sql fuzzer uses box module depending on event subsystem, which was
not initialized by mistake. It didn't cause any problems because box
does not use events yet, but it will use them when triggers will be
moved to the trigger registry, so let's initialize event in sql fuzzer.

Part of #8657

NO_CHANGELOG=test
NO_DOC=test

Boolean type is required by box.session.on_auth triggers.

Part of #8657

NO_CHANGELOG=internal
NO_DOC=internal

With this patch, mp_check sets diag with detailed information about
the MsgPack decoding error. The diag includes the error reason, which is
appended to the error message, and the data offset, which is stored in
an error payload field. Possible error reasons are: truncated input,
junk after input, illegal code, invalid extension. In case of truncated
input, the error also includes the trunc_count payload field, which is
set to the number of missing MsgPack values. Failing to decode a MsgPack
extension adds two payload fields, ext_type and ext_len, and also an
error cause, which is set by the extension decoder. For all extensions
except the error extension, the error cause is just "cannot unpack FOO".
For the error extension, it includes a detailed cause pointing to the
error in the MsgPack data stored in the extension.

Currently, the mp_check error is reported only by the Lua msgapck
decoder while other mp_check users (e.g. xrow decoder) override it.
We may improve on this in future.

Examples:

tarantool> require('msgpack').decode('\x94\xc0')
---
- error: Invalid MsgPack - truncated input
...

tarantool> box.error.last():unpack()
---
- offset: 2
  code: 20
  base_type: ClientError
  type: ClientError
  trunc_count: 3
  message: Invalid MsgPack - truncated input
  trace:
  - file: ./src/box/msgpack.c
    line: 151
...

tarantool> require('msgpack').decode(string.char(
         >   130,  39, 170, 114, 101, 116, 117, 114, 110,  32,  46,
         >    46,  46,  33, 145, 199,  74,   3, 129,   0, 145, 134,
         >     0, 171,  67, 108, 105, 101, 110, 116,  69, 114, 114,
         >   111, 114,   1, 170,  99, 111, 110, 102, 105, 103,  46,
         >   108, 117,  97,   2, 211,   0,   0,   0,   0,   0,   0,
         >     0, 201,   3, 173,  85, 110, 107, 110, 111, 119, 110,
         >    32, 101, 114, 114, 111, 114,   4, 211,   0,   0,   0,
         >     0,   0,   0,   0,   0,   5, 211,   0,   0,   0,   0,
         >     0,   0,   0,   0))
---
- error: Invalid MsgPack - invalid extension
...

tarantool> box.error.last():unpack()
---
- code: 20
  base_type: ClientError
  prev: Invalid MsgPack - cannot unpack error
  message: Invalid MsgPack - invalid extension
  ext_len: 74
  ext_type: 3
  trace:
  - file: ./src/box/msgpack.c
    line: 161
  type: ClientError
  offset: 18
...

tarantool> box.error.last().prev:unpack()
---
- code: 20
  base_type: ClientError
  type: ClientError
  prev: Invalid MsgPack - MP_ERROR_LINE value must be MP_UINT
  message: Invalid MsgPack - cannot unpack error
  trace:
  - file: ./src/box/msgpack.c
    line: 126
...

tarantool> box.error.last().prev.prev:unpack()
---
- offset: 30
  code: 20
  base_type: ClientError
  type: ClientError
  message: Invalid MsgPack - MP_ERROR_LINE value must be MP_UINT
  trace:
  - file: ./src/box/mp_error.cc
    line: 350
...

Closes #7968

NO_DOC=error reporting improvement

Just use xmalloc for allocating exception objects.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring