Before this commit all the peers of the given replicaset were obligated
to have suitable URI to connect -- either iproto.advertise.peer or one
of iproto.listen URIs.

Now replication.peers is successfully constructed if at least one
suitable URI is constructed (except URI of the given instance itself).

However, if an unknown user is used in the iproto.advertise.peer option
of any peer, it is considered as an error (startup failure), just like
before.

The testing code for unsuccessful cases of replication.peers
autobuilding is rewritten to reflect that just one bad peer doesn't
cause a startup failure anymore. Now it also needs stripping of log
messages, because the peers autobuilding tells about unsuitable URIs.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

This failover mode enables automatic leader election on a replicaset.
Assigning a leader manually (`leader` option) and assigning RO/RW mode
(`database.mode` option) are forbidden in this failover mode.

Configuration example:

```yaml
replication:
  failover: election         # !!

groups:
  group-001:
    replicasets:
      replicaset-001:
        instances:
          instance-001: {}
          instance-002: {}
          instance-003: {}
```

All the replicaset instances are so called candidates by default: they
can vote for a leader, they can be elected as a leader. However, it is
possible to set `replicaset.election_mode` to 'voter', 'off' or
'manual'. See more detailed description in the box_cfg applier comments
in the code and in the box.cfg() options documentation [1].

Note: Unlike box.cfg()'s option 'election_mode', the
replication.election_mode = "off" forces the instance to be read-only
(if failover = "election" is enabled). box.cfg() call has no cluster
configuration and it doesn't know, whether the given replicaset is
managed by the built-in election algorithm or an external coordinator.

The election failover may be used to improve cluster's availability.

[1]: https://www.tarantool.io/en/doc/latest/reference/configuration/#cfg-replication-election-mode

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

This failover mode allows to set a leader for a replicaset by its
instance name instead of per-instance database.mode option. For example:

```yaml
replication:
  failover: manual           # !!

groups:
  group-001:
    replicasets:
      replicaset-001:
        leader: instance-001 # !!
        instances:
          instance-001: {}
          instance-002: {}
          instance-003: {}
```

The "manual" failover mode doesn't allow several leaders in a
replicaset. If it is desired, use the "off" failover mode.

The "manual" mode doesn't perform a proper leader switching at the
moment. An administrator should resign the old leader (by unsetting the
leader option or setting it to `null`), wait till all the data arrives
to the upcoming leader and then configure it as the new leader.

The proper leader switching is subject of a future work. Also, the next
commit will add replicaset.failover = "election", which handles such
problems.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

It has the only one value for now: 'off'. Two other ones are planned:
'manual' and 'election'.

The 'off' value means exactly what is implemented now: the database.mode
option controls whether a particular instance should serve write
requests.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

If the given instance has no existing snapshot, it means that it is not
registered in the replicaset and will attempt to join as a new replica.
It has no chance to succeed if all the instances in the replicaset are
in the read-only mode: neither of them can register the new instance.

Report an error and fail the startup in the case to make the situation
clear.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

If the startup may take a long time (reads of an existing snapshot or
fetches it from a remote master), then

* If there are other instances in the replicaset, force the read-only
  mode on the database startup[^1].
* After the database startup, re-read the configuration and apply the
  fresh one (now without any RO/RW mangling).

If the startup shouldn't take a long time[^2], the configuration is
read and applied once.

[^1]: The read-only mode is not forced if there is no existing snapshot.
      It may lead to missing bootstrap leader in the replicaset. See
      details in the comment in the box_cfg applier code.
[^2]: There is the only case that we consider as such: startup of an
      instance without an existing snapshot without other instances in
      the same replicaset.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC
NO_TEST=it is not trivial to verify automatically

Enabling read-write by default is unsafe for an instance in a replicaset
with more than one instance. The only meaningful default here is
read-only.

On the other hand, a signleton instance (the only one in its replicaset)
almost always started in the read-write mode in practice.

Let's use these values as defaults for these situations.

The name of the option is changed from `rw` (boolean) to `mode` (enum
with allowed values `'ro'` and `'rw'`). We agreed on the enum after a
long controversy whether it should be `ro` or `rw`.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

In brief:

* client -- for external clients
* peer -- for connections within the cluster, in particular for replicas
* sharding -- for routers and a rebalancer

See the instance_config.lua file for the details.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

The following new syntax variants are introduced for `iproto.advertise`.

* `user@` -- use the given user, a password from the `credentials`
  section and `host:port` from `iproto.listen`
* `user:pass@` -- use the given user and password, use `host:port` from
  `iproto.listen`
* `user@host:port` -- use the given user, host and port, use a password
  from the `credentials` section

It allows to don't repeat the same information in different places of
the config.

The `test_no_advertise_*` test cases are generalized: now the same
boilerplate code is used for these and new cases.

The `test.luatest_helpers.server` util gains ability to parse all the
new `iproto.advertise` variants.

The replicaset and etcd configuration examples are updated to use the
`user@` syntax.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

There are a few nuances in using the `iproto.listen` option as a peer
URI at replicaset building (constructing a list of peer URIs).

* `iproto.listen` is a comma separated list of URIs: the single URI is
  just a special case.
* `iproto.listen` may contain `0.0.0.0` or `::` host and/or zero port.
  It is perfectly valid for listening, but doesn't give enough
  information to connect.

Let's handle `iproto.listen` appropriately: split it the URI list, walk
over the URIs and check whether the URI is suitable to connect. The
first suitable URI is used as the peer URI.

If no suitable URI is found, raise an error (it means a startup failure
if occurs at a startup).

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

Aside of a fast feedback for a user that feeds an incorrect
configuration, this validation simplifies further box_cfg applier code:
we can assume that the URI list is valid.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

The following values shouldn't be accepted as an advertise URI:

* Not an URI.
* Comma separated list of URIs.
* An URI with IPv4/IPv6 INADDR_ANY host (`0.0.0.0` or `::`).
* An URI with zero TCP port.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

We can't just pass it over and start the instance:

1. If it is an initial bootstrap (there is no data for the given
   instance yet), the instance will form its own replicaset instead of
   joining into the existing one.
2. If it is a startup of an existing instance, the local data may be
   outdated and serving requests is undesirable.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

In scope of #4693, we now create runtime tuple formats from format data, so
we need to adapt space read views accordingly: they use a field definition
array for creating tuple formats and for accessing field names and types —
for the latter case we will also allocate a field definition array
separately.

Follows-up #4693

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Previous attempt to fix flakiness in commit 6a2c73f8 ("test: fix
flakiness in gh_6128_background_mode_test") used a constant buffer size
in check_err_msg function. Tarantool 2.10 has a bit bigger log before a
desired message that other versions of Tarantool and it leads to a this
resulted in a truncated message ("entering the even" instead of
"entering the event loop"). Patch replaces check_err_msg()
implementation to grep_log used in luatest, it reads the whole log.

Also patch renames check_err_msg to check_msg, because "entering the
event loop" is not an error message.

Follows up #6128

NO_CHANGELOG=fix test
NO_DOC=fix test

This patch reworks the way the meta is populated. This is done to
incrementally populate the metadata, instead of set the metadata at the
end of reading data from the source. This allows to get the correct meta
in cases where getting data from the source failed.

Follow-up #8789

NO_DOC=feature not yet released
NO_TEST=tested in EE
NO_CHANGELOG=feature not yet released

The bug was fixed in the libyaml repository. This commit just updates
the submodule and adds a test.

Closes #8782

NO_DOC=bug fix

Function execution permissions should only be checked on constraint
creation.

So when the function is used to check a tuple access rights don't
have to be checked on each call for the current user.

Closes #7873

NO_DOC=bugfix

Function execution permissions should only be checked on functional
index creation and on functional index function set.

So when the function is used by key_list_iterator its rights don't
have to be checked on each call for the current user.

Part of #7873

NO_DOC=bugfix
NO_CHANGELOG=see the next commit

When a space is altered, we abort all in-progress transactions and delete
all stories related to that space: the problem is we don't delete the
stories' read gaps, which are also linked to the stories' transactions,
which get cleaned up on transaction destruction — this, in turn, results in
heap-use-after-free. To fix this, clean up stories' read gap in
`memtx_on_space_delete` — we don't do this in `memtx_tx_story_delete` since
it expects the story to not have any read gaps (see
`memtx_tx_story_gc_step`).

Tested this patch manually against Nick Shirokovskiy's experimental
small-ASAN integration branch.

Closes #8781

NO_DOC=bugfix
NO_TEST=<already covered by existing tests, but was not detectable by ASAN>

The created packages can be found in the job artifacts.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Co-authored-by: Yaroslav Lobankov <y.lobankov@tarantool.org>

This target is a wrapper on the command to build DEB and RPM packages
with a statically compiled Tarantool binary inside. It just runs the
`./static-build/make_packages.sh` script with the properly defined
VERSION env variable.

Example of usage:

    $ make -f .pack.mk package-static

NO_DOC=make
NO_TEST=make
NO_CHANGELOG=make

This patch adds facility to make DEB and RPM packages with a statically
compiled Tarantool binary inside. The build is performed in a Docker
container, using PackPack docker image (centos-7) and CPack.

The packpack/packpack:centos-7 image has all the necessary dependencies
for building Tarantool and quite old glibc 2.17 which theoretically
provides compatibility of the created packages with any distro where
glibc >= 2.17.

The build can be run with the command below:

    $ VERSION=3.0.0 ./static-build/make_packages.sh

In the `static_build` directory, there will be the following packages:

    tarantool_3.0.0-1_amd64.deb
    tarantool-dev_3.0.0-1_amd64.deb
    tarantool-3.0.0-1.x86_64.rpm
    tarantool-devel-3.0.0-1.x86_64.rpm

`tarantool_3.0.0-1_amd64.deb`, `tarantool-3.0.0-1.x86_64.rpm` are
packages with the Tarantool server binary inside.

`tarantool-dev_3.0.0-1_amd64.deb`, `tarantool-devel-3.0.0-1.x86_64.rpm`
are packages with the Tarantool server development files inside.

NO_DOC=build
NO_TEST=build

Co-authored-by: Yaroslav Lobankov <y.lobankov@tarantool.org>

Introduce `box.tuple.format` object, a Lua wrapper around tuple format:
these objects own a tuple format, which is almost equivalent to
`space:format`, except for check constraints and foreign key constraints
being disabled (though they appear to be present for compatibility with
`space:format`).

Add an option table argument to `box.tuple.new` with 'format' option,
allowing to create formatted spaceless tuples.

Closes #4693

@TarantoolBot document
Title: Formats for standalone tuples and `box_tuple_new_vararg` compat opt

A new box.tuple.format library was added, with a tuple format constructor
(`new`) and a tuple format validator (`is`).

New tuple format objects (userdata) were added, which can be used with the
same format clause as for the `space:format` method (except that check
constraints and foreign keys are disabled for them):
NO_WRAP
```lua
f = box.tuple.format.new(box.space._space:format())
f = box.tuple.format.new{{name = 'field1', type = 'string', is_nullable = true,
                          nullable_action = 'none', collation = 'unicode_uk_s2',
                          default = 'UPPER("string")',
                          constraint = {ck = 'box.schema.user.info'},
                          foreign_key = {fk = {space = '_space', field = 'name'}}},
                          {name = 'field2', nullable_action = 'ignore',
                          foreign_key = {fk = {space = '_space', field = 1}}}}
```
NO_WRAP

Format objects have several introspection methods: `:pairs`, `:ipairs`,
`totable`, and also have a `__serialize` metamethod — these methods return
the original (i.e., user-provided) format clause. `:pairs` is an alias to
`ipairs` (since the format clause is an array by nature), and the `totable`
method is an alias to the `__serialize` metamethod, which returns an array
of field definitions.

Format objects also have a `:tostring` method, which simply returns a
"box.tuple.format" literal.

The standalone tuple constructor, `box.tuple.new` was extended with an
options parameter which currently has one available option, `format`
(default value is `nil`, i.e., no format). The format option is either a
tuple format object previously created using `box.tuple.format.new` or a
format clause.

Examples of standalone tuple creation with formats:
NO_WRAP
```lua
box.tuple.new({1}, {format = {{name = 'field', type = 'number'}}})
box.tuple.new({1}, {format = {{'field', type = 'number'}}})
box.tuple.new({1}, {format = {{'field', 'number'}}})

f = box.tuple.format.new({{name = 'field', type = 'number'}})
box.tuple.new({}, {format = f})
box.tuple.new({1}, {format = f})
box.tuple.new({'str'}, {format = f})
-- error: Tuple field 1 (field) type does not match one required by operation: expected number, got string
box.tuple.new({'str'}, {format = f})
```
NO_WRAP

See also the design document https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#a33e2d7418d249679969e5f21ef2832c

A new `box_tuple_new_vararg` compatibility option was introduced: a new
page needs to be created for it (https://tarantool.io/compat/box_tuple_new_vararg)

This option controls whether `box.tuple.new` should interpret an argument
list as an array of tuple fields (i.e., vararg, old behaviour), or as a
value plus a tuple format (new default behaviour). The value can be either
a scalar, an array or a box tuple. The old behaviour does not allow
creating formatted standalone tuples.

Old behaviour examples:
```lua
box.tuple.new(1)
box.tuple.new{1}
box.tuple.new(1, 2, 3)
box.tuple.new{1, 2, 3}
-- This won't create a formatted tuple: the format option will become the
-- second tuple field.
box.tuple.new({1, 2, 3}, {format = box.tuple.format.new{{'field'}}})
```

New behaviour examples:
```lua
box.tuple.new(1)
box.tuple.new(1, {format = box.tuple.format.new{{'field'}}})
box.tuple.new{1}
box.tuple.new({1}, {format = box.tuple.format.new{{'field'}}})
box.tuple.new(1, 2, 3) -- error
box.tuple.new(1, 2, 3, {format = box.tuple.format.new{{'field'}}}) -- error
box.tuple.new{1, 2, 3}
box.tuple.new({1, 2, 3}, {format = box.tuple.format.new{{'field'}}})
```

See also the design document https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#6f74f0c70005463b8438830edd1a0117.

Previously, reusable tuple formats were only used for ephemeral spaces and
by `net.box` (which only used 'name' definitions), so non-space tuple
format definitions were ignored in tuple format comparison and hash
functions, but in scope of #4693 reusable tuple formats will be
interchangeable with space formats, so now we need to account for these
too.

Needed for #4693

NO_CHANGELOG=internal
NO_DOC=internal

In scope of #4693 we need to save the MsgPack encoding of the original
(i.e, user-provided) format clause in the tuple format structure for
serialization to Lua and IPROTO (#8147, #8633): since tuple formats for
spaces are created from space definitions, we need to also save the MsgPack
encoding in the latter.

Since we need to pass the format clause MsgPack encoding to the runtime
tuple format constructor, refactor it to perform field definition decoding
in-place. We cannot use default field definition array decoding for
external formats (67578d1f), so introduce a new field name decoder for this
case.

Needed for #4693

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

userdata is the preferred type for Lua objects wrapping C structures, but
tuple formats were made cdata for no good reason: change them to userdata.

Needed for #4693

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Since we are going to make public Lua tuple format objects, introduce a new
`tuple_format` submodule and move the code related to these objects there.

Needed for #4693

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring

Test runs an external process with tarantool that writes to a log file.
Then test reads that log file and searches a string with required
message in it (see function check_err_msg). Test was flaky on macOS and
I suspect it was happening due to a high log level - timeout was not
enough to wait message in the log file.

Patch decreases a log level to a default value and replaces io
functions with the similar alternatives in a fio module. Using
fio functions allows to not block fibers.

NO_CHANGELOG=test fix
NO_DOC=test fix

The xlog reader Lua module uses the xlog_cursor_next_row, which decodes
the row header with xrow_header_decode. The latter silently ignores any
unknown fields, which complicates catching bugs when garbage is written
to a row header by mistake, for example, see #8783.

Let's parse a row header without using xrow_header_decode in the xlog
reader module, like we parse a row body, and output all unknown/invalid
keys as is.

To do that, we have to extend the xlog cursor API with the new method
xlog_cursor_next_row_raw that returns a pointer to the position in the
tx buffer where the next xrow is stored without advancing it. To avoid
a memory leak in case the caller fails to parse an xrow returned by this
function, we also have to move the call to xlog_tx_cursor_destroy from
xlog_tx_cursor_next_row to xlog_cursor_next_tx.

While we are at it,
 - Don't raise an error if a key type encountered in a row body is
   invalid (not an integer). Instead, silently ignore such keys.
 - Remove the useless body MsgPack validness check because we already
   check it after decoding the header.
 - Add error injection based tests to check all the corner cases.

NO_DOC=bug fix

To avoid garbage written to xlog.

Closes #8783

NO_DOC=bug fix
NO_TEST=next commit

These are misc tests that can be related to the issue. Not sure all of
them do not work before the patch set for the issue. It is nice to have
them and be sure everything works fine.

Closes #8658

NO_DOC=minor

We can replace the last `xrow_update_err_double` occurrence with
`xrow_update_err_no_such_field`. The last is correct one for this place
as `xrow_update_op_do_field_##op_type` is called only for non terminal
paths. Thus if XUPDATE_SCALAR is encountered then the field referenced
does not exist.

Part of #8658

NO_DOC=minor
NO_CHANGELOG=minor

This will help to do further updates into into/inside affected field.
It does not make much difference from the performance POV but allows to
reuse existing code.

Part of #8658

NO_CHANGELOG=later
NO_DOC=minor

This way after branching the operation will be presented by XUPDATE_SCALAR
in the parent array/map so that we can apply multiple operations to the
same field.

Part of #8658

NO_CHANGELOG=later
NO_DOC=minor

Current representation allows only single splice. Rope in turn allows
for arbitrary number of updates and have good asymptotic. Using only
rope gives 4% degradation of single update of the string field (full
time of update operation is measured). Thus rope is started to use
from the second update into the same string field.

Part of #8658

NO_DOC=internal
NO_CHANGELOG=internal

Now when groundwork is done it is time to unblock multiple updates to
the same field.

Note that along the way we fix one more case. For example 'bar' update
on path '[2].a.b.c.d' and then setting on the path '[2].a.b.c'.
Currently this leads to cryptic 'can not update map by non-string key'
error. The issue is if new path is shorter than old path of the bar
update which is to be branched then we exit path comparison loop when
new_toke.type == JSON_TOKEN_END.

Note that current code for handling double update of the same field for
the bar case is not optimal. It will produce route node (optional),
array or map node and node for field itself in the update tree. Instead
we can have just same bar node which apply 2 scalar updates like in case
of terminal update in map or array. This optimization is work to be
done.

Part of #8658

NO_DOC=minor
NO_CHANGELOG=later

Now result is stored in operation itself. It is inconvenient if we are
going to apply multiple operations on the same field. To keep the said
result let's introduce scalar type that can hold all possible scalar
including strings.

By the way it is good time to get rid of `new_field_len` field. It is
excessive now.

Part of #8658

NO_TEST=refactoring
NO_DOC=refactoring
NO_CHANGELOG=refactoring

It is actually not a scalar as we can set arrays/maps which further
holds arrays/maps and so on and so on. Let's make it XUPDATE_NOP
instead. XUPDATE_NOP is basically a state of array/map item without
changes so we can easily make further field updates after '=' update.

For this purpose we need to untie tail data from updated field data
by introducing tail data pointer.

We can do updates to/into newly inserted field before this patch. Yet let's
require it explicitly here by tests for insert operation.

Part of #8658

NO_DOC=minor
NO_CHANGELOG=later

Panic if we fail to allocate internal temporary objects on region. We do
not test allocation failures and this should normally happen also
 (see #3534).

Part of #8658

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup