Currently test can fail if in developer environment .inputrc is custom.

Follow-up #4317

NO_TEST=test fix
NO_DOC=test fix
NO_CHANGELOG=test fix

Remove parts that used before flightrec reader API is added.

- fix test_max_record_size case of prbuf unit test to use reader from
  file
- drop prbuf reader from buffer in memory
- drop prbuf Lua FFI interface

Follow up: https://github.com/tarantool/tarantool-ee/issues/319

NO_DOC=internal
NO_CHANGELOG=internal

Found by EE CI in release asan in flightrec test which uses prbuf.
Change prbuf test so that the issue is tested in CE where the prbuf code
resides.

Follow-up https://github.com/tarantool/tarantool-ee/issues/319

NO_DOC=bug fix
NO_CHANGELOG=unreleased bug fix

Closes #8501

@TarantoolBot document
Title: Document memtx read view statistics

New entries have been added to `box.stat.memtx()` output:

```
tarantool> box.stat.memtx().data
---
- garbage: 0
  total: 24986
  read_view: 0
...

tarantool> box.stat.memtx().index
---
- read_view: 0
  total: 933888
...
```

`data` shows how much memory is allocated for memtx tuples:
 - `data.total` - total amount of memory allocated for data tuples.
   This includes `data.read_view` and `data.garbage` plus tuples that
   are actually stored in memtx spaces.
 - `data.read_view` - memory held for read views.
 - `data.garbage` - memory that is unused and scheduled to be freed
   (freed lazily on memory allocation).

`index` shows how much memory is allocated for memtx index extents:
 - `index.total` - total amount of memory allocated for indexing data.
   This includes `index.read_view` plus memory used for indexing tuples
   that are actually stored in memtx spaces.
 - `index.read_view` - memory held for read views.

All numbers are given in bytes.

`data.read_view` and `index.read_view` include memory allocated both for
system read views (snapshot, replication) and user read views (EE-only).
They should be non-zero only if there are open read views. To list all
open read views, use `box.read_view.list()`.

The commit adds new read view methods needed for pagination. Also, the
check that tuple is not nil is dropped for the sake of consistency - we
will omit this check in read view because it is incomplete anyway and
the error message looks fine without it.

Part of tarantool/tarantool-ee#285

NO_CHANGELOG=in EE
NO_DOC=in EE

Since we are going to implement pagination for read_view, we need to
factor out iterator_position logics to avoid duplication.

Part of tarantool/tarantool-ee#285

NO_TEST=refactoring
NO_CHANGELOG=refactoring
NO_DOC=refactoring

Currently, we check that tuple passed as position fits space format.
However, for pagination purposes, it's enough to validate only parts of
tuple that are used in cmp_def. And, it allows not to use space format -
we will need it in read view. So the patch replaces tuple validation with
validation of its cmp_def parts only.

Closes #8511
Part of tarantool/tarantool-ee#285

NO_DOC=bugfix

We are going to validate only key parts of passed tuple in pagination.
That is why the patch introduces helper that allows to do it with raw
version of tuple.

Part of #8511
Part of tarantool/tarantool-ee#285

NO_CHANGELOG=internal
NO_DOC=internal

This includes:
- Call flightrec Lua C initialization.
- Stop exporting in public API flightrec configuration function. It
is intended for private use.
- Export iproto_key_translation for representing requests/response
records.

Part of https://github.com/tarantool/tarantool-ee/issues/319

NO_TEST=preparation for EE changes
NO_CHANGELOG=preparation for EE changes
NO_DOC=preparation for EE changes

It is intended to be used in EE. As we introduce this function it is
a good moment to drop level_to_char and level_to_string. They have too
little functionality.

Part of https://github.com/tarantool/tarantool-ee/issues/319

NO_CHANGELOG=internal
NO_DOC=internal

There is an issue in prbuf_prepare on branch when we need to
wrap and allocate space for the record starting from the beginning of
the data area.

In the process we set end offset to 0. This means empty buffer. So if
application is crashed after such prepare but before commit buffer will
have no records. This is a bug.

But actually we don't have to reset end to 0 unless we preparing for
very big record which will occupy almost all buffer. In this case there
is really no records left after preparation. Otherwise if we don't
change end offset buffer will be valid and records will be there.

Part of https://github.com/tarantool/tarantool-ee/issues/319

NO_DOC=bugfix
NO_CHANGELOG=minor

New prbuf reader API is intended to be used in flight recorder reader
API implementation.

Currently prbuf has API to read records memory buffer. It is used for
testing purposes only. We aim to drop this reader eventually. This patch
in particular switch prbuf unit test to use new reader API.

This patch also cleanups prbuf unit test besides merely adding tests
for new and missing cases. This includes:

- Using structured test plan. Now test plan is not some number which
updated after running the test. With structure it can be written down
deliberately.
- Buffer/payload sizes in main test are tuned to test all possible
unused space at the end of the buffer cases.
- Check actual number of records in buffer. Otherwise we may not notice
having no records at all for example.
- Factor out common parts to helper functions/tests.
- Print actual object in tests where it may help tests debugging.

Part of https://github.com/tarantool/tarantool-ee/issues/319

NO_DOC=internal
NO_CHANGELOG=internal

Part of https://github.com/tarantool/tarantool-ee/issues/319

NO_CHANGELOG=internal
NO_DOC=internal

There used to be a rare error when failed to connect via tarantoolctl to
listening cartridge console. It was caused by unclear
console.local_print() contract. Starting from gh-7031 fix, the function
assumed string-only arguments, while in some cases cdata error was
passed.

Now console.local_print() prints all non-string arguments as is, without
modifying potential local_eos.

Closes #8374

NO_DOC=bugfix
NO_TEST=very hard to test

Follows up #8194

NO_CHANGELOG=ci
NO_DOC=ci
NO_TEST=ci

- remove obsoleted parameters
- return non-zero exit code when warnings are found. Doxygen has an
  option that set exit code to non-zero on warnings, available in
  Doxygen 1.9.0+, see [1] and [2].

1. https://github.com/doxygen/doxygen/issues/8023
2. https://www.doxygen.nl/manual/config.html

NO_CHANGELOG=doxygen config
NO_DOC=doxygen config
NO_TEST=doxygen config

Follows up #8194

- remove < and > around function names to make them resolvable
- place @sa and \sa to the new lines and remove round parenthesis around
- add missed dots
- added comments for box_tuple_upsert, box_tuple_update, luaT_pusherror
  etc.
- replace \example tags with \code and \endcode, because \example
  requires a file name with example, when \code allows to place a code
  inline
- comments for macros alignof and likely/unlikely looks a bit ugly,
  but I couldn't find a better way to resolve doxygen warning
- remove dashes after @param
- corrected comments for box_tuple_update() and box_tuple_upsert()

Closes #8194

NO_CHANGELOG=fix doxygen
NO_DOC=fix doxygen
NO_TEST=fix doxygen

We need MemtxAllocator stats for reporting read view memory usage, see
https://github.com/tarantool/tarantool-ee/issues/143. Let's also use
them for reporting data memory usage in box.info.memory() for
consistency and add a test.

It looks more correct that using allocator stats for reporting data
memory usage because box.info.memory() is supposed to show high level
stats. For low level allocator stats we have box.slab.info().

Note that a memtx tuple may be held by tuple_bless so we have to reset
it by creating a new tuple in Lua before checking memory usage.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

We account three metrics:

 - used_total - total size of allocated memory
 - used_rv - size of memory held for read views
 - used_gc - size of memory freed on demand

We'll need them to report memtx read view statistics.

While we are at it, switch the test to TAP and drop the result file.

Needed for https://github.com/tarantool/tarantool-ee/issues/143

NO_DOC=internal
NO_CHANGELOG=internal

Fix grammar, punctuation, and wording

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

Part of tarantool/tt#286

NO_TEST=third-party dependency
NO_DOC=third-party dependency

Follows up #8488

NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing

Follows up #8488

NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing

Seed corpus based on test data used in regression tests.
Dictionary was created using fuzzing test after 10^6 test executions.

Follows up #6731
Fixes #8488

NO_CHANGELOG=fuzzing corpus
NO_DOC=fuzzing corpus
NO_TEST=fuzzing corpus

Seed corpus based on test data used in regression tests.
Dictionary was created using fuzzing test after 10^6 test executions.

Follows up #6731
Part of #8488

NO_CHANGELOG=fuzzing corpus
NO_DOC=fuzzing corpus
NO_TEST=fuzzing corpus

metrics.cfg is a module configuration entrypoint [1]. After this patch,
box.cfg{metrics = cfg} triggers metrics.cfg. Parameter is dynamic.

This patch also enables all metrics by default. To disable them, use
box.cfg{metrics = {include = 'none'}}. Since callbacks are triggered
manually (when user calls `invoke_callbacks()` or
`collect{invoke_callbacks=true}`), it should not affect
the performance.

1. https://www.tarantool.io/en/doc/latest/book/monitoring/api_reference/#lua-function.metrics.cfg

Closes #7725

@TarantoolBot document
Title: embedded metrics configuration

This patch add new box.cfg dynamic parameter: metrics. It is a table
with the same effect as in metrics.cfg. metrics.cfg has its own
documentation, see [1].

1. https://www.tarantool.io/en/doc/latest/book/monitoring/api_reference/#lua-function.metrics.cfg

tarantool/metrics [1] is a lua module (distributed as a separate rock)
for metrics aggregation and export. After this patch, it will be a part
of the tarantool binary.

1. https://github.com/tarantool/metrics

Part of #7725

@TarantoolBot document
Title: embedded metrics

Now tarantool has metrics module on its board. metrics is a lua module
previously distributed as a separate rock which is widely used by
tarantool applications. Metrics has its own documentation section:
https://www.tarantool.io/en/doc/latest/book/monitoring/ , but it doesn't
yet mention anything about the embedding.

Patch replaces "sql_value" and "struct sql_value" with "struct Mem" and
renames sqlValueFree to mem_delete to be consistent with functions names
for "struct Mem".

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Function argument uses a name that is a reserved word in a C++ [1].
Patch renames it to avoid clash when building with C++ compiler.

1. https://en.cppreference.com/w/cpp/keyword

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

We need matras statistics for reporting read view memory usage, see
https://github.com/tarantool/tarantool-ee/issues/143. Let's also use
them for reporting index memory usage in box.info.memory() for
consistency and add a test.

While we are at it, drop unused memtx_index_extent_pool declaration,
which is a leftover from commit 3d138884 ("memtx: move all global
variables to engine").

NO_DOC=refactoring
NO_CHANGELOG=refactoring

Simply forward the new matras_stats argument to matras_create().
Currently, it's only used in tests.

While we are at it:
 - Drop custom alloc/free func types in favor of matras types since
   now container constructors depend on matras_stats anyway.
 - Rearrange arguments order to group allocator-related arguments
   together.
 - Drop rtree_init return value (it's always 0).

Needed for https://github.com/tarantool/tarantool-ee/issues/143

NO_DOC=refactoring
NO_CHANGELOG=refactoring

This commit pulls matras statistics.

Needed for https://github.com/tarantool/tarantool-ee/issues/143

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Bump test-run to new version with the following improvements:

- Introduce flaky tests statistics [1]

[1] tarantool/test-run@7729a4c

Part of tarantool/tarantool-qa#299

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

There was a problem with the leader's relay continuing to ping the
remote followers even when the leader's tx thread is hung. This tricked
the followers into thinking the leader is alive and well, even though it
couldn't serve any new requests.

The problem was partially fixed by commit 56571d83 ("raft: make
followers notice leader hang"): that commit made relay thread stop
sending heartbeats in case tx thread is unresponsive.

Up to now we didn't differentiate between heartbeats and data rows: the
receipt of both was considered a sign the master is alive. So if some
replicas are not up to date with the master, they will continue thinking
it's alive until they are fully synced and notice there are no more
heartbeats from it.

In order to fix this, stop treating all data as heartbeats and start
sending heartbeats on top of an active replication stream.

Closes #7515

NO_DOC=bugfix

`box.stat.memtx` is a table that contains the 'tx' function. This is
confusing because other stat entries are callable: `box.stat.net()`,
`box.stat.vinyl()`, `box.stat.sql()`.

Let's make `box.stat.memtx` callable for consistency. The function
returns a table with the only field 'tx'. Note, we can't drop
`box.stat.memtx.tx()` without breaking backward compatibility so we now
return `box.stat.memtx().tx` when it's called.

Also, let's use `info_handler` instead of pushing statistics directly to
Lua for better encapsulation.

Needed for https://github.com/tarantool/tarantool-ee/issues/143
Closes #8448

NO_DOC=updated https://github.com/tarantool/doc/issues/2801

Corpus based on PUC Rio Lua tests imported from LuaJIT repository [1].

1. https://github.com/tarantool/luajit/tree/tarantool/test/PUC-Rio-Lua-5.1-tests

Follows up #4823

NO_CHANGELOG=corpus
NO_DOC=corpus
NO_TEST=corpus

Patch adds a LuaJIT fuzzer based on libprotobuf-mutator and LibFuzzer.
Grammar is described via messages in protobuf format, serializer is
applied to convert .proto format to string.

For displaying generated code on the screen during fuzzing set
the environment variable 'LPM_DUMP_NATIVE_INPUT'.

For displaying error messages from lua functions set
the environment variable 'LUA_FUZZER_VERBOSE'.

Note: UndefinedBehaviourSanitizer is unsupported by LuaJIT (see #8473),
so fuzzing test is disabled when CMake option ENABLE_UB_SANITIZER is
passed.

Closes #4823

NO_DOC=<fuzzing testing of LuaJIT>
NO_TEST=<fuzzing testing of LuaJIT>

Added options for fuzzing and for getting more information
on debugging.

NO_CHANGELOG=<fuzzing options>
NO_DOC=<fuzzing options>
NO_TEST=<fuzzing options>

Added Google's 'libprotobuf-mutator' and 'protobuf' libraries
for developing grammar-based LuaJIT and SQL fuzzers based on
LibFuzzer.

It is needed to build protobuf module from source because
by default, the system-installed version of protobuf is used
by libprotobuf-mutator, and this version can be too old.

Part of #4823

NO_CHANGELOG=<dependencies>
NO_DOC=<dependencies>
NO_TEST=<dependencies>

Now we use _space:max() instead of max_id for space id generation, so
it is not used anymore. The patch removes max_id from bootstrap snapshot,
adds upgrade and downgrade scripts.

Closes #5997

@TarantoolBot document
Title: Update the description of _schema
Root document: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_space/_schema/

Since tarantool 2.11.1, there is no max_id field in space _schema.