`vy_apply_result_does_cross_pk()` must be called after the new tuple
format is validated, otherwise it may crash in case the new tuple has
fields conflicting with the primary key definition.

While we are at it, fix the operation cursor (`ups_ops`) not advanced
on this kind of error. This resulted in skipped `upsert` statements
following an invalid `upsert` statement in a transaction.

Closes #10099

NO_DOC=bug fix

Bump test-run to new version with the following improvements:

- Calculate parallel jobs based on available CPUs [1]
- Bump luatest to 1.0.1-15 (--list-test-cases) [2]
- luatest: detox test searching code [3]
- luatest: allow to run test cases in parallel [4]

[1] tarantool/test-run@182aa77
[2] tarantool/test-run@1fbbf9a
[3] tarantool/test-run@3b0ccd0
[4] tarantool/test-run@dd00063

NO_DOC=test
NO_TEST=test
NO_CHANGELOG=test

Disable workaround for LuaJIT profiling tests on aarch64 runners due to
the following error:

    mount: /tmp/luajit-test-vardir: mount failed: Operation not permitted

Looks like it happens because our aarch64 runners are LXD containers.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

This reverts commit ef3152cd.

We have solved the issues with aarch64 runners, so we can enable back
aarch64 jobs.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

If a secondary index is altered in such a way that its key parts are
extended with the primary key parts, rebuild isn't required because
`cmp_def` doesn't change, see `vinyl_index_def_change_requires_rebuild`.
In this case `vinyl_index_update_def` will try to update `key_def` and
`cmp_def` in-place with `key_def_copy`. This will lead to a crash
because the number of parts in the new `key_def` is greater.

We can't use `key_def_dup` instead of `key_def_copy` there because
there may be read iterators using the old `key_def` by pointer so
there's no other option but to force rebuild in this case.

The bug was introduced in commit 64817066 ("vinyl: use update_def
index method to update vy_lsm on ddl").

Closes #10095

NO_DOC=bug fix

The metamethod is a way to key_def length introspection.

Closes #10111

@TarantoolBot document
Title: key_def length introspection

To check key_def length (parts count) there is a standard lua operator
`#` (`__len` metamethod).

Example:

```lua
function is_full_pkey(space, key)
    return #space.index[0].parts == #key
end
```

A DML request (insert, replace, update) can yield while reading from
the disk in order to check unique constraints. In the meantime the index
can be dropped. The DML request can't crash in this case thanks to
commit d3e12369 ("vinyl: abort affected transactions when space is
removed from cache"), but the DDL operation can because:
 - It unreferences the index in `alter_space_commit`, which may result
   in dropping the LSM tree with `vy_lsm_delete`.
 - `vy_lsm_delete` may yield in `vy_range_tree_free_cb` while waiting
   for disk readers to complete.
 - Yielding in commit triggers isn't allowed (crashes).

We already fixed a similar issue when `index.get` crashed if raced
with index drop, see commit 75f03a50 ("vinyl: fix crash if space is
dropped while space.get is reading from it"). Let's fix this issue in
the same way - by taking a reference to the LSM tree while checking
unique constraints. To do that it's enough to move `vy_lsm_ref` from
`vinyl_index_get` to `vy_get`.

Also, let's replace `vy_slice_wait_pinned` with an assertion checking
that the slice pin count is 0 in `vy_range_tree_free_cb` because
`vy_lsm_delete` must not yield.

Closes #10094

NO_DOC=bug fix

`tuple_hash_field()` doesn't advance the MsgPack cursor after hashing
a tuple field with the type `double`, which can result in crashes both
in memtx (while inserting a tuple into a hash index) and in vinyl
(while writing a bloom filter on dump or compaction).

The bug was introduced by commit 51af059c ("box: compare and hash
msgpack value of double key field as double").

Closes #10090

NO_DOC=bug fix

The problem is found by @ochaton.

NO_DOC=bugfix
NO_CHANGELOG=not a public API

Bump test-run to new version with the following improvements:

- Bump luatest to 1.0.1-14-gdfee2f3 [1]
- Adjust test result report width to terminal size [2]
- dispatcher: lift pipe buffer size restriction [3]
- flake8: fix E721 do not compare types [4]

[1] tarantool/test-run@84ebae5
[2] tarantool/test-run@1724211
[3] tarantool/test-run@81259c4
[4] tarantool/test-run@1037299

We also have to fix several tests that check that script with luatest
assertions have empty stderr output. test-run brings Luatest which
logs assertions at 'info' level.

Note that gh_8433_raft_is_candidate_test is different. Original
assertion involves logging huge tables that have somewhere closed
sockets inside. And 'socket.__tostring' currently raises error for
closed sockets.

We need to fix gh_6819_iproto_watch_not_implemented_test also to
account error created on loading `luatest` on `server:exec` after
the commit "Add trace check for error assertions".

Part of #9914

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

In scope of the #9914 issue we are setting error trace for API to the
caller frame. Let's leverage existing diff tests to check error trace.
Just check error trace when error is raised when evaluating expression
in console which is used in diff tests.

The check is done for test build only and only for Lua modules specified
in `tarantool._internal.trace_check_is_required`.

Part of #9914

NO_TEST=internal
NO_CHANGELOG=internal
NO_DOC=internal

Why these modules? Initially in the scope of #9914 we only want
to fix trace for `schema.lua` but there is an issue. In the next
patch we changing `console.lua` so that for existing diff test the
trace is checked (for specified modules).

In that patch we add a wrapper function around evaluated expression. So
that argument checking functions like `luaL_checklstring` start to refer
wrapper's ``fn`` in error instead of ``?``. We decided drop the usage of
such checkers in code covered by diff tests. Once we touch a module in
the scope this change we also fix all non box errors to box ones with
proper level.

Part of #9914

NO_CHANGELOG=incomplete
NO_DOC=incomplete

In scope of the #9914 issue we are setting error trace for API to the
caller frame. For the API written in LuaC without any Lua code around
it is easy task. Just make `luaT_error` set the proper trace.

By the way test we don't mess up trace for code evaluated thru netbox.

Part of #9914

NO_CHANGELOG=incomplete
NO_DOC=incomplete

Here is just of bunch utility functions that used in the patch that make
several modules throw box.error with trace set to the caller place. That
patch is just a boring huge switch to box.error and setting proper level
on error creation. Factor out utility functions so that they and their
tests are not get lost.

Part of #9914

NO_CHANGELOG=internal
NO_DOC=internal

Remote replica's vclock is given to master to send data starting
from that position. The master does that, but, in order to find
the relevant position in local WAL to start from, the master must
ignore the local rows. Consider them all already "sent". For that
the master replaces the remote vclock[0] with the local vclock[0].
That makes xlog cursor skip all the local rows.

The problem is that this vclock was taken by relay as is, like if
it was truly reported by the replica. It was even saved as the
"last received ACK". Which clearly isn't the case.

When a real ACK was received, it didn't contain anything in
vclock[0], and yet relay "saw" that the previous ACK has
vclock[0] > 0. That looked like the replica went backwards without
even closing connection, which isn't possible. That made the relay
crash from cringe (on assert).

The fix is not to save the local vclock[0] in the last received
ACK.

For GC and xlog cursor the hack is still needed. An option how to
make it easier was to set vclock[0] to INT64_MAX to just never
even bother with any local rows, but that didn't work. Some
assumptions in other places seem to depend on having a proper
local LSN in these places.

Closes #10047

NO_CHANGELOG=the bug wasn't released
NO_DOC=bugfix

It wasn't clear which of them are inputs and which are outputs.
The patch explicitly marks the input vclocks as const. It makes
the code a bit easier to read inside of relay.cc knowing that
these vclocks shouldn't change.

Alongside "replica_clock" in subscribe is renamed to
"start_vclock". To make it consistent with relay_final_join(), and
to signify that technically it doesn't have to be a replica
vclock. It isn't really. Box.cc alters the replica's vclock before
giving it to relay, which means it is no longer "replica clock".

In scope of #10047

NO_TEST=refactoring
NO_CHANGELOG=refactoring
NO_DOC=refactoring

GC consumer creation and destroy seemed to only happen in box.cc
with one exception in relay_subscribe(). Lets move it out for
consistency. Now relay can only notify GC consumers, but can't
manage them.

That also makes it harder to misuse the GC by passing some wrong
vclock to it, similar to what was happening in #10047.

In scope of #10047

NO_TEST=refactoring
NO_CHANGELOG=refactoring
NO_DOC=refactoring

The function takes the burden of explaining why this hack about
setting local component in a remote vclock is needed. It also
creates a new vclock, not alters an existing one. This is to
signify that the vclock is no longer what was received from a
remote host.

Otherwise it is too easy to actually mistreat this mutant vlock as
a remote vclock. That btw did happen and is fixed in following
commits.

In scope of #10047

NO_TEST=refactoring
NO_CHANGELOG=refactoring
NO_DOC=refactoring

The source files for built-in Lua modules are generally placed on the
same level: in `src/lua` or in `src/box/lua`, disregarding whether
they're public or internal.

The recently introduced `experimental.connpool` built-in module is
placed in the `experimental` subdirectory.

This commit moves `src/box/lua/experimental/connpool.lua` to
`src/box/lua/connpool.lua` to follow the existing file structure.
Public, internal and experimental modules are all on the same level now.

The `connpool` module is still experimental and
`require('experimental.connpool')` is needed to use it.

This commit doesn't change the code of the module.

NO_DOC=no code changes
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

Currently, the demoted leader sees that nobody has requested a vote in the
newly persisted term (because it has just written it without voting, and
nobody had time to see the new term yet), and hence votes for itself,
becoming the most probable winner of the next elections.

To prevent this from happening, let's forbid the demoted leader to be a
candidate in the next elections using `box_raft_leader_step_off`.

Closes #9855

NO_DOC=<bugfix>

Co-authored-by: Serge Petrenko <sergepetrenko@tarantool.org>

Suggested by Nikita Zheleztsov in the scope of #9855.

Needed for #9855

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>
NO_TEST=<refactoring>

Co-authored-by: Nikita Zheleztsov <n.zheleztsov@proton.me>

Logically, we call triggers after running statements. These triggers can
make significant changes (for instance, DDL triggers), so, for consistency,
we should call the statement's `on_rollback` triggers before rolling back
the statement. This also adheres to the logic that transaction
`on_rollback` triggers are called before rolling back individual
transaction statements.

One particular bug that this patch fixes is rolling back of DDL on the
`_space` space. DDL is essentially a replace operation on the `_space`
space, which also invokes the `on_replace_dd_space` trigger. In this
trigger, among other things, we swap the indexes of the original space,
`alter->old_space`, which is equal to the corresponding transaction
`stmt->space`, with the indexes of the newly created space,
`alter->new_space`:
https://github.com/tarantool/tarantool/blob/de80e0264f7deb58ea86ef85b37b92653a803430/src/box/alter.cc#L1036-L1047

If then a rollback happens, we first rollback the replace operation, using
`stmt->space`, and only after that do we swap back the indexes in
`alter_space_rollback`:
https://github.com/tarantool/tarantool/blob/de80e0264f7deb58ea86ef85b37b92653a803430/src/box/memtx_engine.cc#L659-L669
https://github.com/tarantool/tarantool/blob/de80e0264f7deb58ea86ef85b37b92653a803430/src/box/alter.cc#L916-L925

For DDL on the _space space, the replace operation and DDL occur on the
same space. This means that during rollback of the replace, we will try to
do a replace in the empty indexes that were created for `alter->new_space`.
Not only does this break the replace operation, but also the newly inserted
tuple, which remains in the index, gets deleted, and access to it causes
undefined behavior (heap-use-after-free).

As part of the work on this patch, tests of rollback of DDL on system
spaces which use `on_rollback` triggers were enumerated:
* `_sequence` — box/sequence.test.lua;
* `_sequence_data` — box/sequence.test.lua;
* `_space_sequence` — box/sequence.test.lua;
* `_trigger` — sql/ddl.test.lua, sql/errinj.test.lua;
* `_collation` — engine-luatest/gh_4544_collation_drop_test.lua,
                 box/ddl_collation.test.lua;
* `_space` — box/transaction.test.lua, sql/ddl.test.lua;
* `_index` — box/transaction.test.lua, sql/ddl.test.lua;
* `_cluster` — box/transaction.test.lua;
* `_func` — box/transaction.test.lua, box/function1.test.lua;
* `_priv` — box/errinj.test.lua,
            box-luatest/rollback_ddl_on__priv_space_test.lua;
* `_user` — box/transaction.test.lua,
            box-luatest/gh_4348_transactional_ddl_test.lua.

Closes #9893

NO_DOC=<bugfix>

In scope of #9893 we are going to run statement `on_rollback` triggers
before rolling back the corresponding statement. During rollback of DDL in
the `_priv` space, the database is accessed from `user_reload_privs` to
reload user privileges, so we need it to account for the current statement
being rolled back: i.e., the new tuple that was introduced (if any) must
not be used, while the old tuple (if any) must be used.

Needed for #9893

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>

It's used to introduce new data structures in the Tarantool EE.

NO_DOC=no functional changes
NO_TEST=no functional changes
NO_CHANGELOG=no functional changes

It could fail in ASAN build. Can't tell why just there.

The main reason was that in a topology server1 + server2->server3
one of the cases
- did a txn on server1,
- then enabled server2->server3 replication,
- then waited for server2->server3 sync,
- and instantly assumed the txn reached server3.

Surely it not always did. At the server2->server3 sync the txn
might not had reached server2 itself yet.

The fix is as simple as explicitly ensure the txn is on server2
before waiting server2->server3 sync.

Another potential for flakiness was that the default timeout in
luatest.helpers.retrying is super low, just 5 seconds. The patch
manually bumps it to 60 seconds to be sure any future failures
wouldn't be related to too small timeout.

Closes #10031

NO_DOC=test
NO_CHANGELOG=test

We have 2 options to set for example `IllegalParams` error in C code:

`diag_set(IllegalParams, ...)`
    and
`diag_set(ClientError, ER_ILLEGAL_PARAMS, ...)`

We will have different error messages and error types in these 2 cases.
Let's leave only `IllegalParams` and add an assertion for the second case.

Also let's instantiate `IllegalParams` error from Lua in expressions
like `box.error(box.error.ILLEGAL_PARAMS)`.

Part of #9914

NO_CHANGELOG=refactoring
NO_DOC=refactoring

Since the commit 5a031fb6 ("exception: drop OOM handling") we
use xmalloc for allocating exceptions so we don't need to check
for NULL.

NO_TEST=refactoring
NO_CHANGELOG=refactoring
NO_DOC=refactoring

Bump the OpenSSL library version to 3.2.1 and remove OpenSSL patches which
are already present in the updated library version.

Disable modules in OpenSSL configuration to make sure the OpenSSL 3.0
legacy provider is compiled into the library.

Closes #7502

NO_DOC=<dependency bump>
NO_TEST=<dependency bump>

Co-authored-by: Sergey Bronnikov <sergeyb@tarantool.org>

An attempt to use a non-table module as a role now reports a more
descriptive error:

> Unable to use module <...> as a role: expected table, got <...>

Fixes #10049

NO_DOC=The error is reported as before this commit as well as after it,
       but the error message is changed. The error message is not part
       of the API.

Fixes #10044

@TarantoolBot document
Title: Configuration status is shown in `box.info.config` now

There is the `config:info([version])` method, but in order to access it
over iproto an application developer should add something like the
following into the application code:

```lua
_G.config = require('config')
```

It is not convenient, at least because it requires an attention from the
application developer and it can't be solved solely by an administrator.

Now, the `config:info('v2')` result is reported in the `config` field of
the `box.info` table. It is accessible over iproto if appropriate
privileges are granted for a calling user.

Fix several problems that led to unexpected results:
* Handle error thrown by box.clt.promote. Not very often but
  it could fail ruining the test. Fixed by retry.
* It's not so obvious but RO server after successful call of
  box.ctl.promote for some time remains RO. On the other hand
  both net.replicaset and the test rely box.info.ro to define
  the leader, for in their view the actual switch to the new
  leader happens some (perhaps different) time after call to
  box.ctl.promote. That asynchrony sometimes led to unexpected
  results. Fixed by waiting of RW status.
* The fix above (wait for RW of the new leader) introduced new
  problem: in case with sequential leader change and broadcast
  of new values, the new leader has good chances (but not 100%)
  to deliver old broadcasted value to the watcher. Fix it by
  splitting the sequence to the leader change with verification
  and broadcast with verification.
* The similar problem happens when a test starts. It may happen
  that all instances are RO, so the initial broadcast on actual
  leader is run with wrong value (that a replica must use).
  Fix it by waiting for a leader to become RW.

Follow-up #9823

NO_DOC=fix flaky test
NO_CHANGELOG=fix flaky test

This patch tests update splice operation more thoroughly:
* All variants of splice: delete/insert/replace/noop.
* All variants of position: negative/positive/zero/OOB.
* One operation and several sequential operations.
* Tuple update, space update, space upsert.
* String of varbinary field/arg.

Follow up #9997
Closes #10032

NO_DOC=tests
NO_CHANGELOG=tests

@TarantoolBot document
Title: splice update operation now supports varbinary

Just in case splice is an update operation that can modify string
field, deleting a part of it and inserting something instead.
```
tarantool> box.tuple.new{'1234567'}:update{{':', 1, 2, 3, '!'}}
---
- ['1!567']
...
```
In the this example splice operation ':' takes field 1 of tuple,
takes position 2 in string, removes 3 symbols and inserts '!' to
that position.

Both deletion and insertion can naturally degrade to no-op.

Before this patch a splice operation required both field and
inserting argument to be strings.

This patch allows the field and the argument to be varbinary.
```
tarantool> varb = require('varbinary')
tarantool> t = box.tuple.new{varb.new('1234567')}
tarantool> t:update{{':', 1, 2, 3, varb.new('!')}}
---
- [!!binary MSE1Njc=]
...

```

That also allows to insert strings into varbinary fields and
insert varbinary data into string. The actual field type after
update such operation remains the same, so updated string will
be string, while updated varbinary will be varbinary.

Closes #9997

Part of #9914

@TarantoolBot document
Title: Add box.error.is
Product: Tarantool
Since: 3.2

The function checks whether argument passed is box.error.

tarantool> box.error.is(box.error.new(box.error.UNKNOWN))
---
- true
...

tarantool> box.error.is('foo')
---
- false
...

The level argument has same meaning as when error created or raised
with table/error argument. Examples:

```lua
e = box.error.new(box.error.SOME_ERROR, <error arguments>, level)
box.error(box.error.SOME_ERROR, <error arguments>, level)
```

Follows up #9792
Part of #9914

NO_DOC=minor

We are experiencing some issues with aarch64 runners at this moment, so
we have to disable aarch64 packaging job until the issue is resolved.
Other aarch64 jobs can be disabled via GitHub UI, so let's do it there.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

```
/home/shiny/dev/tarantool/src/lib/core/coio_task.c:114:58:
	error: ‘calloc’ sizes specified with ‘sizeof’ in the earlier argument
	and not in the later argument [-Werror=calloc-transposed-args]
  114 |         struct cord *cord = (struct cord *)calloc(sizeof(struct cord), 1);
```

NO_TEST=build fix
NO_CHANGELOG=build fix
NO_DOC=build fix

When upgrading a space, attribute `has_optional_parts` of indexes can be
changed. So in order to correctly index both old and new tuples we should
set new min_field_count value to the minimal min_field_count of old and
new formats. Actual value will be set when space upgrade completes.

Part of tarantool/tarantool-ee#698
Part of tarantool/tarantool-ee#750

NO_TEST=in ee
NO_CHANGELOG=in ee
NO_DOC=bugfix

Implement 'np' (next prefix) and 'pp' (previous prefix) iterators.
They work only in memtx tree and in a nutshell searches for
strings with greater ('np') or less ('pp') prefix of size as in
given key, comparing with given key.

Closes #9994

@TarantoolBot document
Title: 'np' and 'pp' (next/previous prefix) iterators

Now there are two more iterators available: 'np' (next prefix)
and 'pp' (previous prefix). They work only in memtx tree. Also,
if the last part of key is not a string, they degrade to 'gt'
and 'lt' iterators.

These iterators introduce special comparison of the last part of
key (if it is a string). In terms of lua, if s is the search part,
and t is the corresponding tuple part, 'np' iterator searches for
the first tuple with string.sub(t, 1, #s) > s, while 'pp' searches
for the last tuple with string.sub(t, 1, #s) < s.

Comparison of all other parts of the key remains normal.

As usual, these iterators are available both in select and pairs,
in index and space methods.

Similar to all other tree iterators, they change only initial
search of selection. Once the first tuple found, the rest are
selected sequentially in direct (for 'np') or reverse (for 'pp')
order of the index.

For example:
```
tarantool> s:select{}
---
- - ['a']
  - ['aa']
  - ['ab']
  - ['b']
  - ['ba']
  - ['bb']
  - ['c']
  - ['ca']
  - ['cb']
...

tarantool> s:select({'b'}, {iterator = 'np'})
---
- - ['c']
  - ['ca']
  - ['cb']
...

tarantool> s:select({'b'}, {iterator = 'pp'})
---
- - ['ab']
  - ['aa']
  - ['a']
...
```

wal_queue_max_size took effect only after the initial box.cfg call,
meaning that users with non-zero `replication_sync_timeout` still synced
using the default 16 Mb queue size. In some cases the default was too
big and the same issues described in #5536 arose.

Fix this.

Closes #10013

NO_DOC=bugfix