Rework "running tests with built-in package" assert since now checks is
a callable table package with subpackages instead of a single function.

NO_DOC=No tagged version since checks initial embedding
NO_CHANGELOG=No tagged version since checks initial embedding

(cherry picked from commit a8bc6312)

This patch replaces the type for some int and uint32_t values with
size_t to avoid problems with integer overflow.

Closes tarantool/security#119

NO_DOC=bugfix

(cherry picked from commit 60a187d7)

Skip unstable tests of `replication-luatest/linearizable_test.lua` due
to tarantool-qa#277.

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 442ee729)

The fixed tests often failed with

  fail | 2023-03-01 15:54:30.550 [3724975] main/103/server_instance.lua
  F> can't initialize storage: unlink, called on fd 63,
  aka unix/:(socket), peer of unix/:(socket): Address already in use

We fixed a similar issue in commit 3d3e9dea ("test: fix flaky
box-luatest/gh_7917_log_row_on_recovery_error_test") by using unique
instance names. Let's do the same here.

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 7ac2685b)

Server API from the luatest has been changed: `server:clean()` method
has been removed. Use `server:drop()` instead.

Follows up tarantool/luatest#296

NO_DOC=test fix
NO_TEST=test fix
NO_CHANGELOG=test fix

(cherry picked from commit efa60df0)

Bump test-run to new version with the following improvements:

- Bump luatest to 0.5.7-33-g8523e5c [1]

[1] tarantool/test-run@7db594d

Part of tarantool/luatest#296

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 80238639)

Tuples that have been inserted into system spaces after the _space
definition has been inserted into _space have the same format as the
space into which they were inserted. However, previously inserted tuples
have an incomplete tuple format with parts missing. One piece that is
missing information is the JSON token, which is used to determine the
number of fields.

The tarantoolsqlIdxKeyCompare() function contains the correct checks for
the case when the number of fields from the format is equal to or less
than the fieldno field, but uses tuple_format_field() before this check,
resulting in an assertion.

This patch forces tarantoolsqlIdxKeyCompare() to call
tuple_format_field() only after checking that fieldno is less than
field_count.

Closes #8418

NO_DOC=Bugfix in debug.

(cherry picked from commit 2252fa7b)

* Add an extra step that should be considered as a workaround for the
  runner agent (ChristopherHX/github-act-runner) that is used on FreeBSD
  machines. Without it, env.ARTIFACT_EXISTS will be undefined for some
  reason.

* In the s3-upload-artifact action change `if: env.ARTIFACT_EXISTS` to
  `if: env.ARTIFACT_EXISTS == 'true'` to be compatible with the FreeBSD
  runner agent.

Follows up tarantool/multivac#116

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 561c4f16)

Fix grammar, punctuation, and wording

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit b6e31f96)

This patch adds the step to all workflows with test-run tests to save
test artifacts to S3 for providing artifact link from Multivac Grafana
dashboard.

Resolves tarantool/multivac#116

NO_DOC=CI
NO_TEST=CI
NO_CHANGELOG=CI

Co-authored-by: Yaroslav Lobankov <y.lobankov@tarantool.org>
(cherry picked from commit 001ef4a6)

This patch adds a local GitHub action to save test artifacts to S3.
Print a warning if the provided artifact path doesn't exist.

Part of tarantool/multivac#116

NO_DOC=CI
NO_TEST=CI
NO_CHANGELOG=CI

Co-authored-by: Yaroslav Lobankov <y.lobankov@tarantool.org>
(cherry picked from commit 1e3529dc)

Original test was flaky and passed even if the problem was not fixed in
Release build and more rarely in Debug. Let's improve this test to be more
reliable.

Closes tarantool/tarantool-qa#203

NO_CHANGELOG=test
NO_DOC=test

(cherry picked from commit bf4494a0)

Currently, we check that tuple passed as position fits space format.
However, for pagination purposes, it's enough to validate only parts of
tuple that are used in cmp_def. And, it allows not to use space format -
we will need it in read view. So the patch replaces tuple validation with
validation of its cmp_def parts only.

Closes #8511
Part of tarantool/tarantool-ee#285

NO_DOC=bugfix

(cherry picked from commit 275130ff)

We are going to validate only key parts of passed tuple in pagination.
That is why the patch introduces helper that allows to do it with raw
version of tuple.

Part of #8511
Part of tarantool/tarantool-ee#285

NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit f8bb4ec4)

There used to be a rare error when failed to connect via tarantoolctl to
listening cartridge console. It was caused by unclear
console.local_print() contract. Starting from gh-7031 fix, the function
assumed string-only arguments, while in some cases cdata error was
passed.

Now console.local_print() prints all non-string arguments as is, without
modifying potential local_eos.

Closes #8374

NO_DOC=bugfix
NO_TEST=very hard to test

(cherry picked from commit 796e4b5b)

Part of tarantool/tt#286

NO_TEST=third-party dependency
NO_DOC=third-party dependency

(cherry picked from commit 10f7109c)

Follows up #8488

NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing

(cherry picked from commit 64427eec)

Follows up #8488

NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing

(cherry picked from commit 1e584e89)

Seed corpus based on test data used in regression tests.
Dictionary was created using fuzzing test after 10^6 test executions.

Follows up #6731
Fixes #8488

NO_CHANGELOG=fuzzing corpus
NO_DOC=fuzzing corpus
NO_TEST=fuzzing corpus

(cherry picked from commit 46725004)

Seed corpus based on test data used in regression tests.
Dictionary was created using fuzzing test after 10^6 test executions.

Follows up #6731
Part of #8488

NO_CHANGELOG=fuzzing corpus
NO_DOC=fuzzing corpus
NO_TEST=fuzzing corpus

(cherry picked from commit b0b11131)

metrics.cfg is a module configuration entrypoint [1]. After this patch,
box.cfg{metrics = cfg} triggers metrics.cfg. Parameter is dynamic.

This patch also enables all metrics by default. To disable them, use
box.cfg{metrics = {include = 'none'}}. Since callbacks are triggered
manually (when user calls `invoke_callbacks()` or
`collect{invoke_callbacks=true}`), it should not affect
the performance.

1. https://www.tarantool.io/en/doc/latest/book/monitoring/api_reference/#lua-function.metrics.cfg

Closes #7725

@TarantoolBot document
Title: embedded metrics configuration

This patch add new box.cfg dynamic parameter: metrics. It is a table
with the same effect as in metrics.cfg. metrics.cfg has its own
documentation, see [1].

1. https://www.tarantool.io/en/doc/latest/book/monitoring/api_reference/#lua-function.metrics.cfg

(cherry picked from commit 8f95eaa5)

tarantool/metrics [1] is a lua module (distributed as a separate rock)
for metrics aggregation and export. After this patch, it will be a part
of the tarantool binary.

1. https://github.com/tarantool/metrics

Part of #7725

@TarantoolBot document
Title: embedded metrics

Now tarantool has metrics module on its board. metrics is a lua module
previously distributed as a separate rock which is widely used by
tarantool applications. Metrics has its own documentation section:
https://www.tarantool.io/en/doc/latest/book/monitoring/ , but it doesn't
yet mention anything about the embedding.

(cherry picked from commit 82ebbb35)

Bump test-run to new version with the following improvements:

- Introduce flaky tests statistics [1]

[1] tarantool/test-run@7729a4c

Part of tarantool/tarantool-qa#299

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit f05e683c)

There was a problem with the leader's relay continuing to ping the
remote followers even when the leader's tx thread is hung. This tricked
the followers into thinking the leader is alive and well, even though it
couldn't serve any new requests.

The problem was partially fixed by commit 56571d83 ("raft: make
followers notice leader hang"): that commit made relay thread stop
sending heartbeats in case tx thread is unresponsive.

Up to now we didn't differentiate between heartbeats and data rows: the
receipt of both was considered a sign the master is alive. So if some
replicas are not up to date with the master, they will continue thinking
it's alive until they are fully synced and notice there are no more
heartbeats from it.

In order to fix this, stop treating all data as heartbeats and start
sending heartbeats on top of an active replication stream.

Closes #7515

NO_DOC=bugfix

(cherry picked from commit ee49c51a)

`box.stat.memtx` is a table that contains the 'tx' function. This is
confusing because other stat entries are callable: `box.stat.net()`,
`box.stat.vinyl()`, `box.stat.sql()`.

Let's make `box.stat.memtx` callable for consistency. The function
returns a table with the only field 'tx'. Note, we can't drop
`box.stat.memtx.tx()` without breaking backward compatibility so we now
return `box.stat.memtx().tx` when it's called.

Also, let's use `info_handler` instead of pushing statistics directly to
Lua for better encapsulation.

Needed for https://github.com/tarantool/tarantool-ee/issues/143
Closes #8448

NO_DOC=updated https://github.com/tarantool/doc/issues/2801

(cherry picked from commit 89a60d82)

Now we use _space:max() instead of max_id for space id generation, so
it is not used anymore. The patch removes max_id from bootstrap snapshot,
adds upgrade and downgrade scripts.

Closes #5997

@TarantoolBot document
Title: Update the description of _schema
Root document: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_space/_schema/

Since tarantool 2.11.1, there is no max_id field in space _schema.

(cherry picked from commit 5324d66f)

Currently, _schema.max_id is used to generate sequentially growing
space ids. The main drawback of this approach is that generated space id
can be not unique if one mixes implicit and explicit space ids. Let's use
actual maximal space id to generate a new one, and scan for free id if
overflow happened.

Closes #8036

NO_DOC=bugfix

(cherry picked from commit 697123d0)

Disable to downgrade in this case.

Closes #8457

NO_DOC=bugfix

(cherry picked from commit fada9ca3)

JIT has been disabled for these 4 tests on arm64 to avoid failing due to
side-effects of constant rematerialization:
* <app-tap/datetime.test.lua>
* <box-luatest/gh_6539_log_user_space_empty_or_nil_select_test.lua>
* <box-luatest/pagination_netbox_test.lua>
* <engine-luatest/pagination_test.lua>

The problem was solved via the commit
15e62a67 ("luajit: bump new version").
So, enable JIT compilation for these tests back.

Closes #6599
Closes #7739
Closes #8011

NO_CHANGELOG=tests
NO_DOC=tests

(cherry picked from commit 915e0b3a)

Follow-up #8463

NO_DOC=style fix
NO_CHANGELOG=style fix
NO_TEST=style fix

(cherry picked from commit 64532551)

Bump test-run to new version with the following improvements:

- Bump luatest to 0.5.7-29-geef05dd [1]

[1] tarantool/test-run@cc3c38e

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 8ac35039)

The `lua_add_key_u64()` function pushes an `uint64_t` value using
`lua_pushinteger()`, which accepts `int64_t` argument. A value >= 2^63
will be interpreted as a negative value on all architectures we're
supporting. However, technically it is implementation defined behavior
(see n1256, 6.3.1.3.3).

It is not a problem, in fact, because the function is used only to
report `http_client:stat()` statistics and because values beyond 2^63-1
are unreachable in practice.

OTOH, it is easy to eliminate the undefined behavior by replacing
`lua_pushinteger()` with our own helper function, which accepts
`uint64_t`: `luaL_pushuint64()`.

The values above 10^14 - 1 are now pushed as `cdata<uint64_t>`. Lower
values are pushed as `number` just like before the commit.

Reported-in: https://github.com/tarantool/security/issues/103

NO_DOC=The type of values in the statistics is not specified explicitly
       in the documentation (not obligated to be `number`) and it is
       quite common for Tarantool to return a value of `cdata<int64_t>`
       or `cdata<uint64_t>` type for an integer with a large absolute
       value.
NO_CHANGELOG=see NO_DOC
NO_TEST=It is hard to reach so large values externally (send 2^63
        requests) and it doesn't look worthful to introduce an error
        injection/a internal API to test it. `luaL_pushuint64()` is
        covered by the module API test.

(cherry picked from commit 3dbbf2d3)

The index directory is created on demand since commit c00ba8e7
("xlog: make log directory if needed") and removed when it becomes
empty. There's no need to create it when an index is created anymore.

Follow-up #8441

NO_DOC=bugfix

(cherry picked from commit 9b50c095)

When vinyl space is dropped, its files are left on the file system
until GC removes them. At the moment GC removes only run files,
but not the root directory. These empty directories are never
removed and occupy 4KB on ext-family file systems each. In a case
of many dropped vinyl spaces it can become a serious disk space
and inode leak. Current commit makes gc always remove root directory
if there are no runs in it.

Closes #8441

NO_DOC=bugfix

(cherry picked from commit ee3c1964)

`func_opts_reg` definition misses a `OPT_END` termintator item. This
leads to UB on iterating it. Particularly when `func_opts_reg` is used
as argument to `opts_decode` in `func_def_new_from_tuple`.

Closes #8463

NO_DOC=bugfix

(cherry picked from commit a652b03f)

The function is unused and still triggers some static analysis
warnings. Let's drop it.

itoa() became unused with removal of fdprintf(), so let's drop it as
well.

Closes tarantool/security#113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 01220555)

Casting a uint64_t greater than INT64_MAX to int64_t is
implementation-defined behaviour, according to the C standard. Let's
avoid that.

In both cases fixed `len` is uint32_t and `ibuf_used(in)` returns a
size_t (aka uint64_t on every platform that we care about).
Hence the result of the subtraction is uint64_t and better use
it directly. Besides, `coio_breadn_timeout` also takes a size_t.

While I'm at it, let's actually change `len` to be uint64_t:
`mp_decode_uint()` returns that anyway.

Closes tarantool/security#108
Closes tarantool/security#109

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit cc2d765a)

Due to a typo some big numbers were coded as MP_(U)INT.

Since msgpackffi is used in selectffi, which is used for memtx,
that could lead to strange select results with big number keys.

Closes #6119

NO_DOC=bugfix

(cherry picked from commit 67a586e1)

This patch fixes incorrect conversion of an integer greater than
INT64_MAX or less than 0 to decimal during SQL arithmetic operations.

Closes #8460

NO_DOC=bugfix

(cherry picked from commit 1e660dcf)