The latter flag is a bit wider: it reveals not only inserting
statements after deleting by the same transaction, but also
replacing and deleting statements after all kinds of previois
changes but the same transaction. This extended behavior will
be used in further commits.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

It was an ugly solution when MVCC engine requires outside engine
to set this flag which is not convenient.

Remove it and use mode arguments to set up proper read trackers.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The function memtx_tx_story_delete is expected to delete fully
unlinked stories and thus should not try to unlink something by
itself. So remove unlink and add asserts instead.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Now there are three kinds of very close trackers:
* The transaction have read some tuple that is not committed and
  thus not visible. This kind is now stored as gap_item with
  is_nearby = false.
* The transaction made a select or range scan, reading a key or
  range between two adjacent tuples of the index. This kind is
  stored as gap_iteam with is_nearby = true.
* A transaction completed a full scan of unordered index. This
  kind is stored as full_scan_item.

All these trackers serve for the same thing: to record a fact
that a transaction read something but didn't see anything.

There are some problems with the current solution:
* gap_item with is_nearby = false has several unused members that
  just consume space.
* bool is_nearby flag for type descriptin is an ugly solution.
* full_scan_item is separated from logically close items.

This commit joins all these trackers under one base (that is
struct gap_item_base) and solves problems above.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Now read trackers are used both for cases when a transaction has
read an existing value and it has read nothing (read by key but
there was no visible tuple in this place). For latter case an
additional index_mask was used to identify from which index the
read was done. Along with that there was per-index interval gap
trackers.

This patch divides area of responsibility between read trackers
and gap tracker in the following way:
* Reads of existing visible values are stored in read trackers.
* Reads of non-existing or non-visible values are store in gap
  trackers.

This new approach allows to provide new invariants: gap trackers
are stored only at top of chain, and read trackers are stored
only at topmost committed story in chain.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

In further commit this list will be used for tracking all read
gaps, not only 'nearby'. Since this rename has rather huge
diff, let's make it in separate commit.

No logical changes.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

In our SQL implementation temporary spaces are used. They come to
MVCC engine in two variants - NULL or ephemeral. In both cases
MVCC engine must not do anything, there are several checks for
that in different parts of code.

Normalize these checks and make them similar to each other.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The only place where this static function is used is more general
static function - memtx_tx_track_read_story. This is a bit
confusing - usually slow variant stand for public inline 'fast'
method.

So merge both functions in one.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

By a mistake in 8a565144 a shortcut was added to procedure
that handles gap write: it was considered that if the writing
transaction is the same as reading - there is no actual conflict
that must be stored further. That was a wrong decision: if such
a transaction yields and another transaction comes and commits
a value with the same key - the first one must go to conflicted
state since it has read no more possible state.

Another similar mistake was made in e6f5090c, where writing
after full scan of the same transaction was not tracked as read.
Obviously that was wrong: if some other transaction overwrites
the key and commits - this transaction must go to read view since
it did not see anything by this key which is not so anymore.

Fix it, reverting the first commit and an modifying the second and
add a test.

Closes #8326

NO_DOC=bugfix

There's a special 'point hole' mechanism in mvcc transactional
manager that manages point gap reads by full key when no raw
tuple was found in the index. For instance, it's the only way
to collect gap reads for non-tree indexes.

Once a new tuple is inserted to the index, the read records are
transferred to the normal read set in the corresponding story.
Actually after that the 'point hole' record in no more needed.

So let's remove it.

While we are here, drop unused point_holes_size, improve names
and comments.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Before this patch there were several different places in the code
that deal with referencing tuple in space, setting in_index member
and marking the story as retained or not. But logically all above
is about the same - about placing a story to the top of a chain,
i.e. the first story in version list to which index points.

This commit refactors these things a bit. This mostly relates to
two functions - memtx_tx_story_new and memtx_tx_story_link_top.

Changes in memtx_tx_story_new are based on the fact that if a story
is created by tuple, it is or immediately will be at the top of
chain. Considering this we can omit argument `is_referenced_to_pk`
and always create a story ready to be in top of chain. If a story
is already in the top - nothing else is needed; if it is to become
the top - memtx_tx_story_link_top must be called after.

Further, linking to top of chain is needed exactly in two cases:
* if a story just created by memtx_tx_story_new must become a top
* if a chain is reordered involving the top story (the top and the
  next stories are swapped)
These two cases are logically very close but still different.
Even more, previously there were two functions for that:
memtx_tx_story_link_top_light and memtx_tx_story_link_top
correspondingly. This commit introduces one function for that
(although with one more argument) that also incapsulates
activities about referencing tuples and marking stories as
retained.

After this patch the rules are logical and simple:
* if a tuple is inserted - call _story_new and _link_top(.. true).
* if a story of existing clean tuple is needed - call _story_new.
* if a chain is reordered involving top story - _link_top(.. false).

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Remove runtime allocation error handling and use panic-on-fail
versions of allocation functions. Reasons for that:
* Memory error handling was never tested an probably doesn't work.
* Some return codes was ignored so the code had obvious flaws.
* Rollback in case of memory error made some code overcomplicated.

Part of #8648
Part of #8654

NO_DOC=no new functionality added
NO_TEST=no new functionality added
NO_CHANGELOG=no new functionalily added

Conflict trackers are used to store information that if some
transaction is committed then some another transaction must be
aborted. This happens when the first transaction writes some
key while the other reads the same key. On the other hand there
are another trackers - read trackers - that are designed to
handle exactly the same situation. That's why conflict trackers
can be simply replaced with read trackers.

That would allow to remove conflict trackers as not needed
anymore.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

If addition of a tuple is rolled back while the corresponding
story is needed for something else (for example it stores a read
set of another transaction) - the story cannot be deleted.
Now there's a special flag `rollbacked` that is set to true
for such stories, and the flag must be considered in places
where history chains are scanned. That approach also requires
psn to be set for rolled-back transactions, which surprisingly
not as simple as it to say. All that makes the code complicated
and hard to maintain.

There's another approach for managing rolled back stories: simply
set their del_psn to a low enough value (lower than any existing
transaction's PSN) and (if necessary) push them to the end of
history chain. Such a story would be invisible to any transaction
due to already existing mechanisms, that's what is needed.

In order to provide "low enough" del_psn it will be natural to
assign real PSN starting from some predefined value, so any value
below that predefined value will be less that any existing PSN and
thus "low enough".

Implement this more simple approach.

Part of #8648
Part of #8654

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

That's strange, but in this test in a group of simple test cases
there are test cases that checks replaces, updates and deletes,
but occasionally there's no test case that checks inserts.

Fix it and add simple test cases for inserts.

No logical changes.

Part of #8648
Part of #8654

NO_DOC=new test case
NO_CHANGELOG=new test case

It is useful for options like `log_level` (with values like `5` or
`'info'`) or `synchro_quorum` (with values like `5` or `'N / 2 + 1'`).

This is a temporary solution. I want to introduce an explicit union
schema node later.

Fixes #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

This is a shortcut for an array of unique string values from the given
list of allowed values. Useful for schema nodes declarations that
describes permissions such as {'read'}, {'read', 'write', 'execute'}
and so on.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

This is a shortcut for a string scalar with the given allowed values.
Suitable for declarations of options like `wal_mode`: `none`, `write`,
`fsync`.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

The function interprets typeless data from an environment variable
as a value of the given type. Raises an error if it is not possible.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

The method allows to walk over the schema and process scalar, map and
array nodes somehow.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

This method is useful to merge several configuration data from different
sources. For example, a config formed from environment variables, a
config read from a local file, a config from etcd.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

This method adds default values from the `default` annotation for missed
fields. The annotation is supported for scalar values. The default can
be conditional, it is controlled by the `apply_default_if` annotation.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

The method allows to perform hierarchical data transformations using
schema information. For example, apply a function to all string scalars.
Or replace all missed values with their defaults from the schema.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

It allows to walk over the hierarchical data and process it with
knowledge about corresponding schema nodes.

In conjunction with user-provided annotations (stored in the schema
nodes) it offers a powerful way to slice the data in different ways.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

It allows to construct a hierarchical data step-by-step or modify it
easily. Just like a field assignment operation for a flat data.

The function validates the value that is to be assigned, so data
constructed using a sequence of `:set()` calls is valid.

It performs the same path validation as `<schema object>:get()` to prevent
indexing of a scalar value or assigning an unknown record's field.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

It allows to acquire a nested data with several convenience properties.

* Optional chaining semantic: `<schema object>:get(data, 'foo.bar')`
  works similarly to TypeScript's `data?.foo?.bar` operator.
* Path validation against the schema: it prevents attempt to get an
  unknown field and attempt to index a scalar value.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

It allows to add a specific validator for a particular schema node.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

The annotation allows to restrict values accepted by a schema node. It
effectively implements a enumeration type.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

The config module should ensure that a received configuration data
corresponds to a given schema before start to process it.

This commit implements a simple validation procedure.

There are possible optimization opportunities, such as generation a
validation code by a given schema.

No optimization tricks are used here, because the configuration
validation occurs at the configuration update. It is not a frequent
operation.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

It is the initial commit for the schema-aware data manipulation module.

Just add constructors here. Methods will be added in further commits.

The future config module will validate and manipulate hierarchical
configuration data. The schema module is a set of utilities for the
config module.

Part of #8725

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

It was disabled because ligomp.a on some AArch64 systems (e.g. CentOS)
is compiled without PIC support. Now Tarantool doesn't depend on ligomp,
so it's safe to turn on PIC for AArch64 systems.

Follow-up #7536

NO_DOC=build
NO_TEST=build

It was disabled because dlsym tests failed when PIC is on (#7640).
However, later it turned out that the issue is not related to PIC, and
LuaJIT was turned off for such tests by commit 67e79b15 ("test: turn
LuaJIT off in tests with dlsym"). Now it's safe to turn on PIC for FreeBSD.

Follow-up #7536

NO_DOC=build
NO_TEST=build

OpenMP is no longer used since commit 4f617b70 ("box: introduce
memtx_sort_threads config parameter"). All dependencies on libgomp
should be removed.

Follow-up #7689

NO_DOC=build
NO_TEST=build

The 2.11.0 release is out now.

It reverts PR #8367.

NO_DOC=no code changes
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

Historically, we encode strings that contain invalid or non-printable
utf-8 sequences in YAML as binary base64 blobs. We do that because of
limitations/bugs of the YAML encoder, which refuses to encode invalid
utf-8 strings. To work around this issue, we introduced the helper
utf8_check_printable, which is basically a copy of yaml_check_utf8,
and treat strings for which it fails as binary data (MP_BIN).

This commit updates the YAML submodule to the version where all known
issues with encoding invalid/unprintable utf-8 strings are fixed and
removes special treatment of such strings (drops utf8_check_printable).
Now unprintable or invalid utf-8 sequences are emitted as code points,
e.g. '\xFF' or '\uFFFF'. This change is a pre-requisite for introducing
the new varbinary type to Lua. Without it plain strings would be
implicitly converted to varbinary after decoding/encoding them in YAML,
which would be confusing.

Closes #8756

NO_DOC=bug fix

The fselect space/index method returns select results in a pretty format
that looks like a table in the console, which encodes the results in the
YAML format. Before the yaml_pretty_multiline compat option was added,
multi-line strings emitted by the console looked mangled so the fselect
method returns a table of strings instead of a multi-line string for the
output to look good. Moreover, it also prepends the strings with the
special zero-width space character (U+200B) to employ a bug in the YAML
formatter: this character isn't printed but it makes the YAML formatter
emit the string without quotes. This hack won't work when #8756 is fixed
because we'll update the YAML formatter to output all non-printable
character as code points.

Now, that we have yaml_pretty_multiline compat option available and
enabled by default, we don't need these hacks anymore. Let's patch
fselect to return the result as a multi-line string and drop the
'use_nbsp' option, which was used to enable or disable the zero-width
space character hack. Let's also drop the 'print' option because one can
now print the fselect results directly.

Needed for #8756

NO_DOC=fselect is undocumented

The future config module works with hierarchical data a lot and needs a
way to show a table as a part a log message. The config module issues
log messages at different log levels and it is undesirable to perform
JSON encoding for messages that will be filtered out by the current log
level.

It is complicated to achieve the goal using the `log` module. Let's add
a wrapper as the temporary solution. The wrapper can be eliminated after
solving the problems in the `log` module that are linked in the
wrapper's code.

The wrapper also simplifies debugging of the config module itself.

NO_DOC=the module is for internal use from the config code
NO_CHANGELOG=see NO_DOC

I'm going to add the `src/box/lua/config/utils/schema.lua` file and
other *.lua files into the `src/box/lua/config` directory.

Let's adjust gitignore for generated *.lua.c file.

NO_DOC=gitignore
NO_TEST=gitignore
NO_CHANGELOG=gitignore

It was decided to include the `osx_debug.yml` and `osx_release.yml`
workflows to the default tests run (without the `full-ci` label).
Now we can get test results for macOS faster and without an extra
load on CI.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

The `space` table of `net.box` stream objects shouldn't have its own
fields, since it used to lookup spaces and those would interfere its
`__index` metamethod, but currently, it erroneously has `_stream` and
`_stream_space_cache` — replace them with upvalues.

Also fixed the same problem for the `index` table of `space` wrapper
objects returned by `__index` metamethod of `net.box` objects `space`
table.

Closes #8598

NO_DOC=bugfix