The following values shouldn't be accepted as an advertise URI:

* Not an URI.
* Comma separated list of URIs.
* An URI with IPv4/IPv6 INADDR_ANY host (`0.0.0.0` or `::`).
* An URI with zero TCP port.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

We can't just pass it over and start the instance:

1. If it is an initial bootstrap (there is no data for the given
   instance yet), the instance will form its own replicaset instead of
   joining into the existing one.
2. If it is a startup of an existing instance, the local data may be
   outdated and serving requests is undesirable.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

In scope of #4693, we now create runtime tuple formats from format data, so
we need to adapt space read views accordingly: they use a field definition
array for creating tuple formats and for accessing field names and types —
for the latter case we will also allocate a field definition array
separately.

Follows-up #4693

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Previous attempt to fix flakiness in commit 6a2c73f8 ("test: fix
flakiness in gh_6128_background_mode_test") used a constant buffer size
in check_err_msg function. Tarantool 2.10 has a bit bigger log before a
desired message that other versions of Tarantool and it leads to a this
resulted in a truncated message ("entering the even" instead of
"entering the event loop"). Patch replaces check_err_msg()
implementation to grep_log used in luatest, it reads the whole log.

Also patch renames check_err_msg to check_msg, because "entering the
event loop" is not an error message.

Follows up #6128

NO_CHANGELOG=fix test
NO_DOC=fix test

This patch reworks the way the meta is populated. This is done to
incrementally populate the metadata, instead of set the metadata at the
end of reading data from the source. This allows to get the correct meta
in cases where getting data from the source failed.

Follow-up #8789

NO_DOC=feature not yet released
NO_TEST=tested in EE
NO_CHANGELOG=feature not yet released

The bug was fixed in the libyaml repository. This commit just updates
the submodule and adds a test.

Closes #8782

NO_DOC=bug fix

Function execution permissions should only be checked on constraint
creation.

So when the function is used to check a tuple access rights don't
have to be checked on each call for the current user.

Closes #7873

NO_DOC=bugfix

Function execution permissions should only be checked on functional
index creation and on functional index function set.

So when the function is used by key_list_iterator its rights don't
have to be checked on each call for the current user.

Part of #7873

NO_DOC=bugfix
NO_CHANGELOG=see the next commit

When a space is altered, we abort all in-progress transactions and delete
all stories related to that space: the problem is we don't delete the
stories' read gaps, which are also linked to the stories' transactions,
which get cleaned up on transaction destruction — this, in turn, results in
heap-use-after-free. To fix this, clean up stories' read gap in
`memtx_on_space_delete` — we don't do this in `memtx_tx_story_delete` since
it expects the story to not have any read gaps (see
`memtx_tx_story_gc_step`).

Tested this patch manually against Nick Shirokovskiy's experimental
small-ASAN integration branch.

Closes #8781

NO_DOC=bugfix
NO_TEST=<already covered by existing tests, but was not detectable by ASAN>

The created packages can be found in the job artifacts.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Co-authored-by: Yaroslav Lobankov <y.lobankov@tarantool.org>

This target is a wrapper on the command to build DEB and RPM packages
with a statically compiled Tarantool binary inside. It just runs the
`./static-build/make_packages.sh` script with the properly defined
VERSION env variable.

Example of usage:

    $ make -f .pack.mk package-static

NO_DOC=make
NO_TEST=make
NO_CHANGELOG=make

This patch adds facility to make DEB and RPM packages with a statically
compiled Tarantool binary inside. The build is performed in a Docker
container, using PackPack docker image (centos-7) and CPack.

The packpack/packpack:centos-7 image has all the necessary dependencies
for building Tarantool and quite old glibc 2.17 which theoretically
provides compatibility of the created packages with any distro where
glibc >= 2.17.

The build can be run with the command below:

    $ VERSION=3.0.0 ./static-build/make_packages.sh

In the `static_build` directory, there will be the following packages:

    tarantool_3.0.0-1_amd64.deb
    tarantool-dev_3.0.0-1_amd64.deb
    tarantool-3.0.0-1.x86_64.rpm
    tarantool-devel-3.0.0-1.x86_64.rpm

`tarantool_3.0.0-1_amd64.deb`, `tarantool-3.0.0-1.x86_64.rpm` are
packages with the Tarantool server binary inside.

`tarantool-dev_3.0.0-1_amd64.deb`, `tarantool-devel-3.0.0-1.x86_64.rpm`
are packages with the Tarantool server development files inside.

NO_DOC=build
NO_TEST=build

Co-authored-by: Yaroslav Lobankov <y.lobankov@tarantool.org>

Introduce `box.tuple.format` object, a Lua wrapper around tuple format:
these objects own a tuple format, which is almost equivalent to
`space:format`, except for check constraints and foreign key constraints
being disabled (though they appear to be present for compatibility with
`space:format`).

Add an option table argument to `box.tuple.new` with 'format' option,
allowing to create formatted spaceless tuples.

Closes #4693

@TarantoolBot document
Title: Formats for standalone tuples and `box_tuple_new_vararg` compat opt

A new box.tuple.format library was added, with a tuple format constructor
(`new`) and a tuple format validator (`is`).

New tuple format objects (userdata) were added, which can be used with the
same format clause as for the `space:format` method (except that check
constraints and foreign keys are disabled for them):
NO_WRAP
```lua
f = box.tuple.format.new(box.space._space:format())
f = box.tuple.format.new{{name = 'field1', type = 'string', is_nullable = true,
                          nullable_action = 'none', collation = 'unicode_uk_s2',
                          default = 'UPPER("string")',
                          constraint = {ck = 'box.schema.user.info'},
                          foreign_key = {fk = {space = '_space', field = 'name'}}},
                          {name = 'field2', nullable_action = 'ignore',
                          foreign_key = {fk = {space = '_space', field = 1}}}}
```
NO_WRAP

Format objects have several introspection methods: `:pairs`, `:ipairs`,
`totable`, and also have a `__serialize` metamethod — these methods return
the original (i.e., user-provided) format clause. `:pairs` is an alias to
`ipairs` (since the format clause is an array by nature), and the `totable`
method is an alias to the `__serialize` metamethod, which returns an array
of field definitions.

Format objects also have a `:tostring` method, which simply returns a
"box.tuple.format" literal.

The standalone tuple constructor, `box.tuple.new` was extended with an
options parameter which currently has one available option, `format`
(default value is `nil`, i.e., no format). The format option is either a
tuple format object previously created using `box.tuple.format.new` or a
format clause.

Examples of standalone tuple creation with formats:
NO_WRAP
```lua
box.tuple.new({1}, {format = {{name = 'field', type = 'number'}}})
box.tuple.new({1}, {format = {{'field', type = 'number'}}})
box.tuple.new({1}, {format = {{'field', 'number'}}})

f = box.tuple.format.new({{name = 'field', type = 'number'}})
box.tuple.new({}, {format = f})
box.tuple.new({1}, {format = f})
box.tuple.new({'str'}, {format = f})
-- error: Tuple field 1 (field) type does not match one required by operation: expected number, got string
box.tuple.new({'str'}, {format = f})
```
NO_WRAP

See also the design document https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#a33e2d7418d249679969e5f21ef2832c

A new `box_tuple_new_vararg` compatibility option was introduced: a new
page needs to be created for it (https://tarantool.io/compat/box_tuple_new_vararg)

This option controls whether `box.tuple.new` should interpret an argument
list as an array of tuple fields (i.e., vararg, old behaviour), or as a
value plus a tuple format (new default behaviour). The value can be either
a scalar, an array or a box tuple. The old behaviour does not allow
creating formatted standalone tuples.

Old behaviour examples:
```lua
box.tuple.new(1)
box.tuple.new{1}
box.tuple.new(1, 2, 3)
box.tuple.new{1, 2, 3}
-- This won't create a formatted tuple: the format option will become the
-- second tuple field.
box.tuple.new({1, 2, 3}, {format = box.tuple.format.new{{'field'}}})
```

New behaviour examples:
```lua
box.tuple.new(1)
box.tuple.new(1, {format = box.tuple.format.new{{'field'}}})
box.tuple.new{1}
box.tuple.new({1}, {format = box.tuple.format.new{{'field'}}})
box.tuple.new(1, 2, 3) -- error
box.tuple.new(1, 2, 3, {format = box.tuple.format.new{{'field'}}}) -- error
box.tuple.new{1, 2, 3}
box.tuple.new({1, 2, 3}, {format = box.tuple.format.new{{'field'}}})
```

See also the design document https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#6f74f0c70005463b8438830edd1a0117.

Previously, reusable tuple formats were only used for ephemeral spaces and
by `net.box` (which only used 'name' definitions), so non-space tuple
format definitions were ignored in tuple format comparison and hash
functions, but in scope of #4693 reusable tuple formats will be
interchangeable with space formats, so now we need to account for these
too.

Needed for #4693

NO_CHANGELOG=internal
NO_DOC=internal

In scope of #4693 we need to save the MsgPack encoding of the original
(i.e, user-provided) format clause in the tuple format structure for
serialization to Lua and IPROTO (#8147, #8633): since tuple formats for
spaces are created from space definitions, we need to also save the MsgPack
encoding in the latter.

Since we need to pass the format clause MsgPack encoding to the runtime
tuple format constructor, refactor it to perform field definition decoding
in-place. We cannot use default field definition array decoding for
external formats (67578d1f), so introduce a new field name decoder for this
case.

Needed for #4693

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

userdata is the preferred type for Lua objects wrapping C structures, but
tuple formats were made cdata for no good reason: change them to userdata.

Needed for #4693

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Since we are going to make public Lua tuple format objects, introduce a new
`tuple_format` submodule and move the code related to these objects there.

Needed for #4693

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring

Test runs an external process with tarantool that writes to a log file.
Then test reads that log file and searches a string with required
message in it (see function check_err_msg). Test was flaky on macOS and
I suspect it was happening due to a high log level - timeout was not
enough to wait message in the log file.

Patch decreases a log level to a default value and replaces io
functions with the similar alternatives in a fio module. Using
fio functions allows to not block fibers.

NO_CHANGELOG=test fix
NO_DOC=test fix

The xlog reader Lua module uses the xlog_cursor_next_row, which decodes
the row header with xrow_header_decode. The latter silently ignores any
unknown fields, which complicates catching bugs when garbage is written
to a row header by mistake, for example, see #8783.

Let's parse a row header without using xrow_header_decode in the xlog
reader module, like we parse a row body, and output all unknown/invalid
keys as is.

To do that, we have to extend the xlog cursor API with the new method
xlog_cursor_next_row_raw that returns a pointer to the position in the
tx buffer where the next xrow is stored without advancing it. To avoid
a memory leak in case the caller fails to parse an xrow returned by this
function, we also have to move the call to xlog_tx_cursor_destroy from
xlog_tx_cursor_next_row to xlog_cursor_next_tx.

While we are at it,
 - Don't raise an error if a key type encountered in a row body is
   invalid (not an integer). Instead, silently ignore such keys.
 - Remove the useless body MsgPack validness check because we already
   check it after decoding the header.
 - Add error injection based tests to check all the corner cases.

NO_DOC=bug fix

To avoid garbage written to xlog.

Closes #8783

NO_DOC=bug fix
NO_TEST=next commit

These are misc tests that can be related to the issue. Not sure all of
them do not work before the patch set for the issue. It is nice to have
them and be sure everything works fine.

Closes #8658

NO_DOC=minor

We can replace the last `xrow_update_err_double` occurrence with
`xrow_update_err_no_such_field`. The last is correct one for this place
as `xrow_update_op_do_field_##op_type` is called only for non terminal
paths. Thus if XUPDATE_SCALAR is encountered then the field referenced
does not exist.

Part of #8658

NO_DOC=minor
NO_CHANGELOG=minor

This will help to do further updates into into/inside affected field.
It does not make much difference from the performance POV but allows to
reuse existing code.

Part of #8658

NO_CHANGELOG=later
NO_DOC=minor

This way after branching the operation will be presented by XUPDATE_SCALAR
in the parent array/map so that we can apply multiple operations to the
same field.

Part of #8658

NO_CHANGELOG=later
NO_DOC=minor

Current representation allows only single splice. Rope in turn allows
for arbitrary number of updates and have good asymptotic. Using only
rope gives 4% degradation of single update of the string field (full
time of update operation is measured). Thus rope is started to use
from the second update into the same string field.

Part of #8658

NO_DOC=internal
NO_CHANGELOG=internal

Now when groundwork is done it is time to unblock multiple updates to
the same field.

Note that along the way we fix one more case. For example 'bar' update
on path '[2].a.b.c.d' and then setting on the path '[2].a.b.c'.
Currently this leads to cryptic 'can not update map by non-string key'
error. The issue is if new path is shorter than old path of the bar
update which is to be branched then we exit path comparison loop when
new_toke.type == JSON_TOKEN_END.

Note that current code for handling double update of the same field for
the bar case is not optimal. It will produce route node (optional),
array or map node and node for field itself in the update tree. Instead
we can have just same bar node which apply 2 scalar updates like in case
of terminal update in map or array. This optimization is work to be
done.

Part of #8658

NO_DOC=minor
NO_CHANGELOG=later

Now result is stored in operation itself. It is inconvenient if we are
going to apply multiple operations on the same field. To keep the said
result let's introduce scalar type that can hold all possible scalar
including strings.

By the way it is good time to get rid of `new_field_len` field. It is
excessive now.

Part of #8658

NO_TEST=refactoring
NO_DOC=refactoring
NO_CHANGELOG=refactoring

It is actually not a scalar as we can set arrays/maps which further
holds arrays/maps and so on and so on. Let's make it XUPDATE_NOP
instead. XUPDATE_NOP is basically a state of array/map item without
changes so we can easily make further field updates after '=' update.

For this purpose we need to untie tail data from updated field data
by introducing tail data pointer.

We can do updates to/into newly inserted field before this patch. Yet let's
require it explicitly here by tests for insert operation.

Part of #8658

NO_DOC=minor
NO_CHANGELOG=later

Panic if we fail to allocate internal temporary objects on region. We do
not test allocation failures and this should normally happen also
 (see #3534).

Part of #8658

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

Apple's libunwind for AArch64 returns the Instruction Pointer with the
Pointer Authentication Codes (bits 47-63) even though Tarantool is compiled
for arm64 (not arm64e) architecture, so we have to strip them out [1].
Although there is the ptrauth_strip macro for this purpose, it works only
if compilation target is arm64e (not arm64) [2].

1. https://developer.apple.com/documentation/security/preparing_your_app_to_work_with_pointer_authentication#3042105
2. https://github.com/dotnet/runtime/issues/42955#issuecomment-886910180

Closes #8074
Closes tarantool/tarantool-qa#308
Closes tarantool/tarantool-qa#309

NO_DOC=bugfix

Implementation notes:
 - The varbinary type is implemented as VLS cdata so we can't use
   the existing luaL_pushcdata and luaL_checkcdatas helpers for
   pushing an object of this type to Lua stack. Instead, we copied
   the implementation from the Lua JIT internals.
 - We already have the code handling `MP_BIN` fields in all built-in
   serializers. We just need to patch it to convert the data to/from
   a varbinary object instead of a plain string.
 - We updated the tuple.tostring method to set the NOWRAP base64
   encoder flag when dumping binary blobs. The flag was apparently
   omitted by mistake because we mask all other new line characters
   while converting a tuple to a string.
 - The box/varbinary_type test was rewritten using the luatest
   framework with all the FFI code needed to insert binary data
   replaced with the new varbinary object.
 - We have to update quite a few SQL tests involving varbinary type
   because binary blobs are now returned as varbinary objects, not
   as plain strings, as they used to be.

Closes #1629

@TarantoolBot document
Title: Document the varbinary type

The new module `varbinary` was introduced. The module implements the
following functions:
 - `varbinary.new` - constructs a varbinary object from a plain string
   or cdata pointer and size (to be used with the `buffer` module).
 - `varbinary.is` - returns true if the argument is a varbinary object.

```Lua
local bin = varbinary.new('data')
assert(varbinary.is(bin))
assert(not varbinary.is('data'))
```

Like a plain string, a varbinary object stores arbitrary data. Unlike
a plain string, it's encoded as a binary blob by the built-in encoders
that support the binary type (MsgPack, YAML). (Actually, encoding binary
blobs with the proper type is the main goal of the new type.)

```
tarantool> '\xFF\xFE'
---
- "\xFF\xFE"
...

tarantool> varbinary.new('\xFF\xFE')
---
- !!binary //4=
...

tarantool> msgpack.encode('\xFF\xFE')
---
- "\xA2\xFF\xFE"
...

tarantool> msgpack.encode(varbinary.new('\xFF\xFE'))
---
- "\xC4\x02\xFF\xFE"
...
```

Note, the JSON format doesn't support the binary type so a varbinary
object is still encoded as a plain string:

```
tarantool> json.encode('\xFF\xFE')
---
- "\"\xFF\xFE\""
...

tarantool> json.encode(varbinary.new('\xFF\xFE'))
---
- "\"\xFF\xFE\""
...
```

The built-in decoders now decode binary data fields (fields with the
'binary' tag in YAML; the `MP_BIN` type in MsgPack) to a varbinary
object by default:

```
tarantool> varbinary.is(msgpack.decode('\xC4\x02\xFF\xFE'))
---
- true
...

tarantool> varbinary.is(yaml.decode('!!binary //4='))
---
- true
...
```

This also implies that the data stored in the database under the
'varbinary' field type is now returned to Lua not as a plain string,
but as a varbinary object. It's possible to revert to the old behavior
by toggling the new compat option `binary_data_decoding` because this
change may break backward compatibility:

```
tarantool> compat.binary_data_decoding = 'old'
---
...

tarantool> varbinary.is(msgpack.decode('\xC4\x02\xFF\xFE'))
---
- false
...

tarantool> varbinary.is(yaml.decode('!!binary //4='))
---
- false
...
```

Please create a documentation page for the new compat option:
https://tarantool.io/compat/binary_data_decoding

A varbinary object implements the following meta-methods:
- `__len` - returns the length of the binary data, in bytes.
- `__tostring` - returns the data in a plain string.
- `__eq` - returns true if the varbinary object contains
  the same data as another varbinary object or a string.

```Lua
local bin = varbinary.new('foo')
assert(#bin == 3)
assert(tostring(bin) == 'foo')
assert(bin == 'foo')
assert(bin ~= 'bar')
assert(bin == varbinary.new('foo'))
assert(bin ~= varbinary.new('bar'))
```

There are no string manipulation methods, like `string.sub` or
`string.match`. If you need to match a substring in a varbinary
object, you have to convert it to a string first.

For more details, see the [design document][1].

[1]: https://www.notion.so/tarantool/varbinary-in-Lua-a0ce453dcf5a46e3bc421bf80d4cc276

Let's drop yaml/b64 in favor of the base64 encoder used everywhere else
in the Tarantool source code.

yaml/b64 is also used by serialize_lua to print MP_BIN values. Let's
print MP_BIN values as MP_STR there. This doesn't have any user-visible
changes because since commit 890a821c ("yaml: don't encode
unprintable strings as binary blobs") luaL_tofield never creates MP_BIN
values. However, when we introduce the varbinary type to Lua, we will
use the MP_BIN value type for it, and printing it in the Lua format as a
string with unprintable characters escaped is going to be less confusing
than encoding it in base64 without any tags or markers.

While we're at it, let's use the luaL_field.sval.data in the encoders
instead of extracting the string from the Lua stack again.

Needed for #1629

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

There's base64_bufsize for calculating the buffer size needed for
base64_encode but there's no such function for base64_decode - one's
supposed to pass a buffer > 3/4 of the input size. This is confusing.
Let's rename base64_bufsize to base64_encode_bufsize and introduce
base64_decode_bufsize.

While we're at it, move base64_encode_bufsize body from base64.h to
base64.c because otherwise the linker fails if this function, which is
currently declared as extern inline, is used in lyaml.cc.

Needed for #1629

NO_DOC=refactoring
NO_CHANGELOG=refactoring

This patch allows additional config features to be added to EE.

Closes #8789

NO_DOC=will be added later
NO_TEST=will be added in EE code
NO_CHANGELOG=will be added later

This patch integrates config module with the Tarantool --config option.

Part of #8789

NO_DOC=will be added later
NO_CHANGELOG=will be added later

Our macOS runners have such a TMPDIR value. It breaks
`config-luatest/basic_test.lua`, because net.box seems unable to connect
a Unix domain socket using an URI with a double slash in the middle.

See the comment in the code for details.

NO_DOC=testing helper change
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

This patch introduces a config test helper that simplifies the creation
of a replica set.

NO_DOC=testing helper
NO_CHANGELOG=see NO_DOC

NO_DOC=testing helper
NO_CHANGELOG=see NO_DOC

It makes the helper a bit more convenient to use it tests.

NO_DOC=testing helper
NO_TEST=see NO_DOC
NO_CHANGELOG=see NO_DOC

This patch implements broadcastion of the current statusi.

NO_DOC=will be added later
NO_CHANGELOG=will be added later