The total number of read views that we can possibly (not necessarily
simultaneously) ever create is limited by UINT32_MAX, because we use
uint32_t for read view versioning and read view version must never wrap
around.

If read views were only used for making snapshots or joining replicas,
this would be fine, because even if we made a snaphost every second
(which is hardly possible), it'd take more than one hundred years for
the read view version to wrap around. However, if read views could be
created by users (which is our ultimate goal), they could get created as
often as every millisecond, which would reduce the wrap around window
down to one month, which is unacceptable.

Let's fix this issue by reusing the most recent read views in case it
was created less than 100 ms ago. The algorithm is described in the
comments to the code.

Closes #7189

NO_DOC=internal
NO_CHANGELOG=internal

Once we start reusing read views, we won't be able to allocate and free
a tuple to trigger garbage collection in tests, because the tuple may
get attached to the last read view. Let's make the collect_garbage()
method public and use it in tests.

While we are at it, let's also rewrite the destroy() method using
collect_garbage().

Needed for #7189

NO_DOC=refactoring
NO_CHANGELOG=refactoring

Irrespective of whether there's an open read view or not, we always free
memtx tuples that come from temporary spaces immediately (see #3432).
This is acceptable if read views are only used for snapshotting or
replication, but to reuse the read view infrastructure for user read
views, we need to delay deletion of temporary tuples until all read
views that may access them have been closed.

The idea is to maintain independent lists of tuple garbage collection
arrays for temporary and normal tuples. If a read view doesn't need to
access temporary tuples, we create one garbage collection array for it,
otherwise we create two garbage collection arrays. When we free a tuple,
we choose a garbage collection array for it looking at its type.

Closes #7412

NO_DOC=internal
NO_CHANGELOG=internal

Currently, tuples are never garbage collected if the number of open read
views stays above zero, even if they can't possibly be accessed from any
read view (e.g. were freed before the oldest read view was created).
This commit fixes this issue by introducing per read view tuple garbage
collection lists. The algorithm is described in the comments to the
code.

Closes #7185

NO_DOC=internal
NO_CHANGELOG=internal

Because we will use this version for user read views, not just
snapshots. Comments are updated as well.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

To support accessing tuple fields by name, we create a runtime tuple
format for each space read view, using the space field names to
initialize the field dictionary.

Note, we can't reuse the space tuple format as is, because it allocates
tuples from the engine arena, which is single-threaded, while a read
view may be used from any thread, not just tx. The runtime arena is
single-threaded as well, but we will make it per-thread in future.

Note, we can't even reuse tuple field dictionary - we create a new one
using the space definition instead - because the dictionary is in fact
mutable - it may be changed when the space definition is altered, see
tuple_dictionary_swap.

Good news is runtime tuple formats are reusable so if we create several
read views of the same space, they will all use the same tuple format.

Since this feature isn't required for snapshots/replication, we add
a read view option to enable it - read_view_opts::needs_field_names.

Needed for https://github.com/tarantool/tarantool-ee/issues/207

NO_DOC=internal
NO_TEST=ee
NO_CHANGELOG=internal

Currently, the helper is used only for creation of a tuple format for
Lua (needed for net.box schema). Later on, we will reuse this helper for
creating tuple formats for user read views.

Needed for https://github.com/tarantool/tarantool-ee/issues/207

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This will let us access space read view format (added later) from
the code that uses an index read view.

Needed for https://github.com/tarantool/tarantool-ee/issues/207

NO_DOC=internal
NO_TEST=ee
NO_CHANGELOG=internal

We store a pointer to index read view in each implementation class,
anyway. Let's move it to the base class - this way we'll be able to
access space read view format (added later) from the code that uses
a read view iterator.

Needed for https://github.com/tarantool/tarantool-ee/issues/207

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

'public' role and thus guest and any newly created user has 'write'
privilege to _collation space. It is not enough to modify _collation as
one also needs 'create' privilege to add a collation and 'drop privilege
to delete one. So it is safe. Yet looks like we don't have any purpuse
of exposing _collation now.

NO_DOC=bugfix

Non privileged user (thru public role) has write access to _truncate
table in order to be able to perform truncates on it's tables. Normally
it should be able to modify records only for the tables he has write
access. Yet now due to bootstrap check it is not so.

Closes tarantool/security#5

NO_DOC=bugfix

Simple part is a part without any extra key besides 'field' and 'type'.
Let's make a check in try_simplify_index_parts itself.

NO_TEST=refactoring
NO_DOC=refactoring
NO_CHANGELOG=refactoring

If index parts are specified using old syntax like:

	parts = {1, 'number', 2, 'string'},

then (except if parts count is 1) index options set in space format
are not taken into account. Solution is to continue after parsing 1.6.0
style parts so to use code that check format options.

Closes #7614

NO_DOC=bugfix

`make api` generates `src/module.h`. `make module_api` is the target for
compiling a library for testing of the module API:
`test/app-tap/module_api.so`. It depends on the `api` target.

Both works for generating `module.h`, but `make api` don't do extra
unneeded actions.

NO_DOC=Not a user visible change.
NO_TEST=It is CI workflow. It would be too strong to test each part of
        the testing/deployment infrastructure.
NO_CHANGELOG=Not a user visible change.

 - Mix-in EE Lua sources if ENABLE_READ_VIEW is set.
 - Raise an error on attempt to access box.read_view in CE.

Needed for https://github.com/tarantool/tarantool-ee/issues/139

NO_DOC=internal
NO_CHANGELOG=internal

We need to know space names to export a database read view to Lua.

Needed for https://github.com/tarantool/tarantool-ee/issues/139

NO_DOC=internal
NO_TEST=ee
NO_CHANGELOG=internal

Currently internal tarantool fibers can be cancelled from the user's app,
which can lead to critical errors.

Let's mark these fibers as a system ones in order to be sure that they
won't be cancelled from the Lua world.

Closes #7448
Closes #7473

NO_DOC=minor change

There are a number of internal system fibers which are not supposed to
be cancelled.

Let's introduce `FIBER_IS_SYSTEM` flag that will indicate, if the fiber
can be explicitly killed. If this flag is set, killing functions will
just ignore cancellation request.

This commit introduce blocking system fiber cancelling only from the Lua
public API, as it is more important to have it right. The prohibition to
cancel fibers from C API will be introduced later.

Related to #7448
Part of #7473

NO_DOC=internal
NO_TEST=will be added in subsequent commit
NO_CHANGELOG=internal

The `ubuntu-18.04` environment is deprecated, so let's switch to
`ubuntu-latest` where it is safe. For more details see [1].

[1] https://github.com/actions/virtual-environments/issues/6002

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Benchmark is implemented using Google Benchmark lib. Here's benchmark
settings:
 - values: we use structure (tuple) containing pointer to heap memory
           and size (all payload is of the same size - 32 bytes);
 - keys: unsigned char (first byte in the tuple memory);
 - hash function: FNV-1a;
 - value comparator: std::memcmp();
 - value count: 10k - 100k - 1M

Before each test we prepare vector of tuples storing truly random
values.

Here's the list of results obtained on my PC (i7-8700 12 X 4600 MHz):

Insertions: ~20-12M per second;
Find (no misses): ~58-16M* per second (find by key gives the same result);
Find (many misses): ~84-30M per second;
Iteration with dereference: ~450M per second;
Insertions after erase: ~50-17M* per second;
Find after erase: ~52-17M* per second (the same as without erase);
Delete: ~32-8M* per second.

* The first value is for 10k values in hash table; second - is for 1M.

Just to have some baseline here results for quite similar benchmark for
std::unordered_map (it is also included in source file):

Insertions: ~26-8M per second;
Find (no misses): ~44-11M per second;
Iteration with dereference: ~265-56M per second;
Find after erase: ~37-13M per second.

Part of #7338

NO_TEST=<Benchmark>
NO_DOC=<Benchmark>
NO_CHANGELOG=<Benchmark>

There are a lot of pretty things introduced in 14 standard,
so let's use it.

NO_DOC=<Build change>
NO_TEST=<Build change>
NO_CHANGELOG=<Build change>

It's useful and can be used in all performance tests, so let's move it
to a separate header.

NO_TEST=<Refactoring>
NO_DOC=<Refactoring>
NO_CHANGELOG=<Refactoring>

This patch introduces the SQL_EXPR functions. These functions are SQL
functions that consist of only one expression. Currently, the only way
to use these functions is as a function in a CHECK constraint, since
there is no interface to define the function's arguments.

Part of #6986

NO_DOC=will be added later
NO_CHANGELOG=will be added later

This commit introduces a new parse rule for compiling an unresolved
single expression. This simplifies the current implementation of
sql_expr_compile() and is useful for generating SQL expressions that can
be used in the core check constraint. This rule is for internal use
only.

Part of #6986

NO_DOC=will be added later
NO_TEST=refactoring
NO_CHANGELOG=will be added later

This patch introduces get_msgpack() method for port_c. After this patch
C-functions could be used for check-constraints.

Needed for #6986

NO_DOC=will be added later
NO_CHANGELOG=will be added later

Needed for #6896

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

In commit (35334ca1) qsort was fixed but unfortunately a small
typo was introduced. Due to that typo the qsort made its job wrong.

Fix the problem and add unit test for qsort.

Unfortunately the test right from the issue runs extremely long,
so it should go to long-tests.

Closes #7605

NO_DOC=bugfix

It is unused and misleading. Let's remove them so that now we have
single entry point for log subsystem - `say()`.

NO_DOC=<Refactoring>
NO_CHANGELOG=<Refactoring>
NO_TEST=<Refactoring>

Let's introduce on_log_level static variable which is assumed to be
configured in `say_set_log_callback()`. on_log_level is assumed to be
log level of `log->on_log` callback (i.e. if entry to be logger features
higher log level - it is simply skipped). Note that now casual log_level
is calculated as MAX(level, on_log_level) since log_level is the single
guard for passing execution flow to `log_vsay()` where both things (to
be precise on_log callback invocation and ordinary logging) happens.

This change is required since if log_level has lower magnitude than
on_log_level - on_log callback will be skipped.

NO_DOC=<Internal change>
NO_TEST=<Internal change>
NO_CHANGELOG=<Internal change>

This patch fixes a number of issues with trigger_clear() while the
trigger list is being run:
1) clearing the next-to-be-run trigger doesn't prevent it from being run
2) clearing the next-to-be-run trigger causes an infinite loop or a
   crash
3) swapping trigger list head before the last trigger is run causes an
   infinite loop or a crash (see space_swap_triggers() in alter.cc, which
   had worked all this time by miracle: space _space on_replace trigger
   swaps its own head during local recovery, and that had only worked
   because the trigger by luck was the last to run)

This is fixed by adding triggers in a separate run list on trigger_run.
This list may be iterated by `rlist_shift_entry`, which doesn't suffer
from any of the problems mentioned above.

While being bad in a number of ways, old approach supported practically
unlimited number of concurrent trigger_runs for the same trigger list.
The new approach requires the trigger to be in as many run lists as
there are concurrent trigger_runs, which results in quite a big
refactoring.

Add a luatest-based test and a unit test.

Closes #4264

NO_DOC=bugfix

struct trigger is about to get a new field, and it's mandatory that this
field is specified in all initializers. Let's introduce a macro to avoid
adding every new field to all the initializers and at the same time keep
the benefits of static initialization.

Also while we're at it fix `lbox_trigger_reset` setting all trigger
fileds manually.

Part-of #4264

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring

Make trigger_fiber_run return an error, when it occurs, so that the
calling code decides how to log it.
Also, while I'm at it, simplify trigger_fiber_run's code a bit.

In-scope-of #4264

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

cord_exit should be always called in the exiting thread. It's a single
place to call all the thread-specific module deinitalization routines.

In-scope-of #4264

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Unit test compilation with `#define UNIT_TAP_COMPATIBLE 1` might fail
with an error complaining that <stdarg.h> is not included. Fix this.

In-scope-of #4264

NO_CHANGELOG=testing stuff
NO_DOC=testing stuff

Our triggers support recursive invocation: for example, an on_replace
trigger on a space may do a replace in the same space.

However, this is not tested and might get broken easily. Let's add a
corresponding test.

In-scope-of #4264

NO_DOC=testing
NO_CHANGELOG=testing

We will use it in the read view Lua API. For memtx tree and hash
indexes, we add a stub function with a possibility of override if
ENABLE_READ_VIEW is set. For the sequence data space, we raise an
error if this function is called, because we aren't planning to
implement it in the EE repository.

Needed for https://github.com/tarantool/tarantool-ee/issues/197

NO_DOC=internal
NO_TEST=ee
NO_CHANGELOG=internal

We aren't planning to support all iterator types for _sequence read
views, but once we implement a Lua API for read views, the user will
still be able to create a read view of this space. So let's replace
the assertions with a graceful error.

Needed for https://github.com/tarantool/tarantool-ee/issues/197

NO_DOC=internal
NO_TEST=ee
NO_CHANGELOG=internal

Tuples stored in a read view might not have a format, because we
overwrite the tuple format id when we free a tuple. So for anything more
sophisticated than a full-scan, we need to use special comparators that
don't access the tuple format. To address that, let's provide a way to
reset tree and hash key_def if ENABLE_READ_VIEW is defined.

The stub functions simply reset the read view key_def to NULL, because
it isn't supposed to be used in the CE version.

Needed for https://github.com/tarantool/tarantool-ee/issues/197

NO_DOC=internal
NO_TEST=ee
NO_CHANGELOG=internal

For memtx tree and hash indexes: if the ENABLE_READ_VIEW macro is
defined, include a source file with the read view iterator
implementation instead of defining it directly. We don't need to support
any iterator types except ALL in the CE repository, because user read
views will be available only in EE. All other iterator types will be
defined in the EE repository. We include source files (not headers),
because to implement a read view iterator, we need access to index
internals, which are defined in memtx_hash.cc and memtx_tree.cc.

The included implementation source file is supposed to define
the {tree,hash}_read_view_iterator_start function, which positions the
iterator to the given key. It's called directly by create_iteartor index
read view method.

For the memtx tree index, the iterator implementation needs to know the
iteration key (for handling EQ/REQ requests) so we also add the key to
the iterator struct in this commit.

Needed for https://github.com/tarantool/tarantool-ee/issues/197

NO_DOC=internal
NO_TEST=ee
NO_CHANGELOG=internal

We can't return tuples retrieved from a consistent index read view as
is. We need to filter out dirty tuples and decompress compressed tuples.
Let's introduce a helper function for this. Currently, this helper
function is used only in the ALL iterator implementation of the tree and
hash indexes. Later, we will use it in all other iterator types, which
will be implemented in the EE repository.

Needed for https://github.com/tarantool/tarantool-ee/issues/197

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring