ASAN-friendly implementation poisons memory after allocation with
ibuf_alloc so we need to fix existing places in code where we access
memory after allocation.

Part of ibuf implementation is inline functions in headers. Thus ibuf
implementation in Lua reimplement this parts. We add poison to these
inline functions in ASAN-friedly implementation so we need add same poison
in Lua implementation.

Part of #7327

NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 4f542bb7)

We are going to include generated small_config.h into small allocator
headers (currently it is only included in small source files).
core/memory.h depends on small headers and salad/heap.h depends on
core/memory.h. As a result we need to provide a way for salad/heap.h
users to find small_config.h header.

Instead let's drop dependency from core/memory.h as we only use it for
typeof definition.

Part of #7327

NO_CHANGELOG=code cleanup
NO_DOC=code cleanup

(cherry picked from commit d01609a4)

If leak sanitizer reaches the memory protected from read with mprotect
it exhibits all sorts of odd behaviour. It can hang, can crash, can
return errors with no leak backtraces.

We use mprotect to create guard zones at the end of fiber stack so if
stack is overflowed we get a signal and crash. We take protection off
when fiber is destroyed. Unfortunately we do not destroy cords (and its
fibers) which cancelled through cord_cancel_and_join. This is going to
be addressed in patch for issue #8423 ("Get rid of pthread_cancel()").
Until that moment let's disable protection for ASAN builds.

Note that we did not hit this behaviour before because LSAN only scans
memory allocated using malloc and regular slab cache uses mmap to get
memory.

Part of #7327

NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 2ee15793)

Regularly fiber stack slab is page aligned. So upper stack border is
page aligned too when stack grows down. But with ASAN friendly slab
cache implementation this border is not page aligned. As a result
madvise call on stack may zero memory beyond stack slab which will cause
heap corruption. In debug build corruption is detected by assertion:

NO_WRAP
 >  Fatal glibc error: malloc.c:2593 (sysmalloc): assertion failed: (old_top
 >  == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >=
 >  MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize
 >  - 1)) == 0)
NO_WRAP

Interestingly enough the issue can not be investigated using ASAN. The
memory is zeroed by kernel code which is not instrumented so it is
invisible for sanitizer.

Looks like non-ASAN builds are not affected. Even if stack_size is
not page aligned the slab allocated for stack is page aligned. Thus
memory zeroing will be inside the slab and there will be no memory
corruption.

Also when stack grows up lower stack border in not aligned even with
regular small implementation. So madvise call will fail with EINVAL as
it is required that start address is page aligned. We ignore the error
though. Let's fix this issue too while we at it.

Let's introduce fiber_madvise_aligned to align madvise range with proper
direction before calling madvise(2). To justify its usage note that
besides fixing the issues described above, in case of stack growing down
fiber->stack is page aligned and in case of stack growing up
fiber->stack + fiber->stack_size is page aligned.

Part of #7327

NO_TEST=tested by ASAN (debug build)
NO_CHANGELOG=has effect only with newly introduced ASAN friendly slab cache
NO_DOC=has effect only with newly introduced ASAN friendly slab cache

(cherry picked from commit 130c7807)

The unpoison was added in the initial commit 1.7.2-68-gafd229393 that
supported ASAN. It is not clear why do we need it as we don't poison
stack memory manually.

Part of #7327

NO_TEST=removing unfunctional code
NO_CHANGELOG=removing unfunctional code
NO_DOC=removing unfunctional code

(cherry picked from commit 0784f7b7)

This patch removes some deprecated code. This code had no user-visible
effect, but caused problems when running the test with ASAN enabled.

Closes #8761

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit d63a4bf2)

Regular region implementation supports allocations of size 0 with no
extra efforts. It returns a non-NULL pointer in this case. However in
case of ASAN friendly implementation it will require a special care for
this case. Instead let's avaid allocations if size 0 for region.

Also use xregion_ macros for allocations. Our current policy is to panic
on OOM on runtime allocations.

Part of tarantool/tarantool#7327

NO_TEST=internal
NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 8159347d)

Small library currently depends on Tarantool core through 'exception.h'.
This is not the way to go. Let's drop this dependency and instead of
moving _xc functions to Tarantool repo we can just stop using them. Our
current policy is to panic on OOM in case of runtime allocation.

Part of #7327

NO_DOC=<OOM behaviour is not documented>
NO_CHANGELOG=<no OOM expectations>
NO_TEST=<no test harness for checking OOM>

(cherry picked from commit 3fccfc8f)

They are rather noisy. Also delete debug log on arena creation. These
two make sense only with each other.

Part of #7327

NO_TEST=internal
NO_DOC=internal
NO_CHANGELOG=internal

(cherry picked from commit 0dc37356)

Panic if we fail to allocate internal temporary objects on region. We do
not test allocation failures and this should normally happen also
 (see #3534).

Part of #8658

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

(cherry picked from commit b1a03a49)

This patch replaces region_*() functions with xregion_*() functions.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 1ba84fe3)

This patch removes the 'size' argument from macros, as it was only used
to set an error on failure, which is not possible for x* versions. In
addition, both macros now cast the value to the specified type, as is
done in the original macros.

Closes #8522

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

(cherry picked from commit ae02f0cd)

This patch fixes SQL memory leaks found by static analyzers and SQL
fuzzer.

Part of tarantool/security#120

NO_DOC=fix for memleak
NO_TEST=fix for memleak
NO_CHANGELOG=fix for memleak

(cherry picked from commit cd173ce5)

Proposed ASAN implementation of region allocator does not support double
reservation for the sake of simplicity. Every reservation is supposed to
be followed by one or more allocations.

This restriction does not work well with mpstream currently. The issue is
mpstream_init/mpstream_reserve do reservation of size 0. For example In
case of region slab of min order is reserved (a chunk of memory of page
size currently). If the first data we want to write to mpstream is
larger then the reservation done then we make reservation again.

Let's get rid of this reservation at the beginning as it is suboptimal
behaviour. Moreover let's get rid of mpstream_reset as mpstream_init
is lightweight and we can create a new mpstream instead of reusing
exiting.

Also while we at it avoid allocation of 0 size in mpstream_flush as it
is done in mpstream_reserve_slow (see 3.0.0-alpha3-19-g8159347d0 "misc:
avoid allocations of size 0 for region" for details).

NO_TEST=internal
NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 3b1de78d)

This way we will have access to build info in those modules. In
particularly build.asan flag is going to be used in buffer.lua in scope
of #7327.

Part of #7327

NO_TEST=internal
NO_DOC=internal
NO_CHANGELOG=internal

(cherry picked from commit f58cc96f)

We already use this info in one of the test and going to use it more.

Part of #7327

@TarantoolBot document
Title: new tarantool.build.asan flag

It is `true` if `ENABLE_ASAN` build option is set and `false` otherwise.

(cherry picked from commit 23012356)

The check_param and check_param_table Lua helpers are defined in
box/lua/schema.lua but used across the whole code base. The problem is
we can't use them in files that are loaded before box/lua/schema.lua,
like box/lua/session.lua. Let's move them to a separate source file
lua/utils.lua to overcome this limitation. Also, let's add some tests.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit d8d267c5)

Configuring log modules work differently with log.cfg and box.cfg:
box.cfg{log_modules=...} overwrites the current config completely while
log.cfg{modules=...} overwrites the currently config only for the
specified modules. Let's fix this inconsistency by making log.cfg behave
exactly as box.cfg.

Closes #7962

NO_DOC=bug fix

(cherry picked from commit c13e59a5)

We install a signal handler that prints the stack trace on SIGSEGV,
SIGBUS, SIGILL, SIGFPE. The signal handler uses the current stack.
This works fine for most issues, but not for stack overflow, because
the latter makes the current stack unusable, leading to a crash in
the signal handler. Let's install an alternative signal stack in each
thread so that we can print the stack trace on stack overflow.

Note that we skip this for ASAN because it installs its own signal
stack. (Installing a custom stack would result in a crash.)

Closes #9222

NO_DOC=bug fix

(cherry picked from commit cb8e903b)

The motivation is to reduce time slip on Tarantool startup before
running init scripts. Internal ev time is set in fiber_init/ev_default_loop
and is not get updated until starting event loop. This causes
timeouts slip up to 0.3 in debug ASAN build in init script (see #9261).

Let's run event loop right at the beginning of the run_script_f before
executing any script. This way besides updating internal ev time we make
an explicit place of starting script event loop. Currently it is started
lazily when config script yields.

This will fix CI for PR https://github.com/tarantool/tarantool-ee/pull/572
for debug ASAN workflow.

We can also remove start_loop condition. It does not make sense now. It
was added in the commit 3a851430 ("Fix tarantool -e "os.exit()"
hang") but since then we start to stop event loop after handling
os.exit().

Also this fixes #9266. The issue is we don't have an event loop to run
on shutdown triggers if -e command line expression add such a trigger
and then call os.exit().

Follow-up #7327
Closes #9266

NO_DOC=bugfix

(cherry picked from commit 1fcfb8c2)

Tarantool supports two console protocols: text and binary. The binary
protocol is implemented with IPROTO EVAL request so the console module
reuses the net.box module to establish and maintain a binary connection.
Currently, instead of passing the original URI specified by the user to
net.box.connect as is, the console module parses the URI and passes the
host and port. As a result, extra information that may be specified in
URI parameters is lost. This prevents the user from connecting to the
binary console using the SSL transport because to use the SSL transport
the user must specify transport=ssl URI parameter.

Needed for tarantool/tarantool-ee#567

NO_DOC=no visible changes in CE
NO_TEST=no visible changes in CE
NO_CHANGELOG=no visible changes in CE

(cherry picked from commit 33e72567)

During building an index in background, some transaction can perform a dml
request that affects space size (e.g. a replace), but the size will remain
the same, because bsize is moved from the old space to the new space in
memtx_space_prepare_alter() prior to space_execute_dml(). Fix this issue by
calling space_finish_alter() in alter_space_do().
In fact, this patch partially reverts commit 9ec3b1a4 ("alter: zap
space_vtab::commit_alter").

NO_DOC=bugfix

Closes #9247

(cherry picked from commit 54a42186)

Before this patch, if an index was created due to a column's UNIQUE
constraint or a column's PRIMARY KEY constraint before adding a
collation, and if the column's fieldno was not equal to the index's
position in space->index, the collation would not be assigned to the
index.

Also, this patch fixes an assertion in debug build for the case when an
index with more that one field was created before a collation was added.

Closes #9229

NO_DOC=bugfix

(cherry picked from commit 65608d87)

Required to suppress the ASAN leak detector.

Closes #9158

NO_DOC=ASAN
NO_TEST=ASAN
NO_CHANGELOG=ASAN

(cherry picked from commit bf62170f)

Force recovery first tries to collect all rows of a transaction into a
single list, and only then applies those rows.

The problem was that it collected rows based on the row replica_id. For
local rows replica_id is set to 0, but actually such rows can be part
of a transaction coming from any instance.

Fix recovery of such rows

Follow-up #8746
Follow-up #7932

NO_DOC=bugfix
NO_CHANGELOG=the broken behaviour couldn't be seen due to bug #8746

(cherry picked from commit 85df1c96)

In order to preserve transaction boundaries over replication, Tarantool
writes a global NOP row after the last transaction row, if this row
happens to be local. This is done to make sure that the is_commit flag,
which is set only in the last transaction row, reaches the replica. This
wouldn't happen if the last row was local.

This workaround works fine for transactions completely authored by one
instance: when both global and local rows come from operations of a
single master.

However, it's possible to append local rows to a remote master's
transaction on a replica. For example, one can use on_replace triggers
to write to replica's local space on each new transaction coming from
master.

In this case essentially a global NOP entry is added at the end of a
remote master's transaction. This leads to several problems.

First of all, this bumps replica's LSN, which is counter-intuitive,
given that the replica might even be read-only. Besides, in a star
topology this leads to master being unable to connect to the replica
later on due to their vclocks becoming incompatible.

Secondly, even if replication channel between master and replica is
bidirectional, it creates a new row which should be replicated from
replica to master, but at the same time is the last row of the master's
transaction. Once master receives this row, it breaks its connection to
replica due to transaction boundary violation (the last row of the
transaction is received without its beginning).

Adding a NOP row became extraneous since the previous commit, which made
relay find transaction boundaries by itself.

Closes #8958

NO_DOC=bugfix

(cherry picked from commit f5e52b2c)

Some time ago we started writing transaction boundaries to WAL and
respecting them in the replication stream: replicas wait for a full
transaction receipt before applying it.

However, during all these changes relay remained transaction-agnostic:
it simply read single rows from WAL and sent them over to the receiver.

This lead to a handful of ugly crutches: for example, tsn is not always
equal to the lsn of the first global row of the transaction: if the
first row is local, tsn is deduced from the first global row of the
transaction.

Also a dummy NOP was appended to the end of a transaction ending by a
local row, so that is_commit flag wasn't lost by the replication.

Let's make relay read a full transaction, filter out all the unnecessary
rows, set the transaction boundaries accordingly and then send the
transaction at once.

Since in relay a single fiber sends data to the remote peer, there is no
chance for a heartbeat to get in between rows of a single transaction:
they're all sent at once. Hence the deletion of a corresponding guard
`relay->is_sending_tx`.

Prerequisite #8958

NO_DOC=internal change
NO_CHANGELOG=internal change
NO_TEST=covered by existing tests

(cherry picked from commit f96782b5)

Transaction boundaries were not updated correctly for transactions in
which local space writes were made from a replication trigger. Existing
transaction boundaries and row flags from the master were written as is
on the replica. Actually, the replica should recalculate transaction
boundaries and even WAIT_SYNC/WAIT_ACK flags.

Transaction boundaries should be recalculated when a replica appends a
local write at the end of the master's transaction, and
WAIT_SYNC/WAIT_ACK should be overwritten when nopifying synchronous
transactions coming from an old term.

The latter fix has uncovered the bug in skipping outdated synchronous
transactions: if one replica replaces a transaction from an old term
with NOPs and then passes that transaction to the other replica, the
other replica raises a split brain error. It believes the NOPs are an
async transaction form an old term. This worked before the fix, because
the rows were written with the original WAIT_ACK = true bit. Now this
is fixed properly: we allow fully NOP async tranasctions from the old
term.

Closes #8746

NO_DOC=bugfix
NO_CHANGELOG=covered by the next commit

(cherry picked from commit 099cb2da)

If non-terminal symbol is referenced in C code then destructor for
expression is not called. Thus we don't need to duplicate. Otherwise we
got a memory leak.

See https://www.sqlite.org/cgi/src/doc/trunk/doc/lemon.html#destructor

Close #9159

NO_DOC=bugfix
NO_TEST=tested by debug ASAN CI (to be turned on)

(cherry picked from commit 36ef3fb4)

In this case we don't have knowledge how to downgrade correctly.

Close #9182

NO_DOC=bugfix

(cherry picked from commit cbe6a4da)

mp_compare_decimal_any_number erroneously assumed that any float or
double from which a decimal can't be created is either infinite or NaN.
This is not true. Any float greater than 1e38 can't fit into our decimal
representation. When such a float got compared to a decimal, an
assertion fired, which was wrong. Luckily, on release build the
comparison was correct. Only the assertion is wrong. Fix it.

Closes #8472

NO_DOC=bugfix

(cherry picked from commit f1b23896)

Our split-brain detection machinery relies among other things on all
nodes tracking the synchro queue confirmed lsn. This tracking was only
added together with the split-brain detection. Only the synchro queue
owner tracked the confirmed lsn before.

This means that after an upgrade all the replicas remember the latest
confirmed lsn as 0, and any PROMOTE/DEMOTE request from the queue owner
is treated as a split brain.

Let's fix this and only enable split-brain detection on the replica set
once the schema version is updated. Thanks to the synchro queue freeze
on restart, this can only happen after a new PROMOTE or DEMOTE entry is
written by one of the nodes, and thus the correct confirmed lsn
is propagated with this PROMOTE/DEMOTE to all the cluster members.

Closes #8996

NO_DOC=bugfix

(cherry picked from commit a844bd37)

With this option enabled (new), the multiresults returned by a stored C
function via iproto aren't wrapped in the additional msgpack array (old).

Due to new behaviour some renames are performed:
* `port_c_dump_msgpack()` -> `port_c_dump_msgpack_wrapped()`, since this
  is dump format with additional msgpack array encoded.
* `port_c_dump_msgpack16()` -> `port_c_dump_msgpack()`, since this
  format is now the default new format of a msgpack dump.

The behaviour of the C port msgpack dumping depends on the
`c_func_iproto_multireturn` option:
* uses `port_c_dump_msgpack()` if set to true (new),
* uses `port_c_dump_msgpack_wrapped()` otherwise (old).

Needed for #4799

@TarantoolBot document
Title: Document `c_func_iproto_multireturn` compat option

Please create a documentation page for the new compat option:
https://tarantool.io/compat/c_func_iproto_multireturn

In the new behaviour, the multiresults returned by a stored C function
via iproto aren't wrapped in the additional msgpack array (old).

```
tarantool> compat.c_func_iproto_multireturn = 'old'
---
...

tarantool> net_box.connect(box.cfg.listen):call('myclib.cfunc')
---
- [true, -1]
...

tarantool> compat.c_func_iproto_multireturn = 'new'
---
...

tarantool> net_box.connect(box.cfg.listen):call('myclib.cfunc')
---
- true
- -1
...

```

The new behaviour is consistent with the local call of the function
via `box.func`:

```
tarantool> box.func['myclib.cfunc']:call()
---
- true
- -1
...

```

Assume you have a stored C function that returns values like the
following:

```c
char *position = mp_encode_bool(buffer, true);
box_return_mp(ctx, buffer, position);
/* ... */
position = mp_encode_int(buffer, -1);
box_return_mp(ctx, buffer, position);
```

If you want to preserve the format of the returned array for your C
functions, when the `c_func_iproto_multireturn` option is set to "new",
you should add the additional wrapping, like the following:

```c
char *position = mp_encode_array(buffer_with_results, n_results);
position = mp_encode_bool(position, true);
/* ... */
position = mp_encode_int(position, -1);
box_return_mp(ctx, buffer_with_results, position);
```

The amount of `box_return_mp()` calls indicates the number of values to
be returned.

Also, you should update its usage via `box.func` if there is any.

(cherry picked from commit 96ee6d9b)

luarocks version updated to version 3.9.2

Closes #6597

NO_DOC=The engine has been updated, the functionality has not changed
NO_TEST=see NO_DOC

(cherry picked from commit 1dc8cd81)

Before this patch if one called `vinyl`, `sql`, `gc` and `memory`
functions from box.info() instance crashed. It's interesting that
`replication_anon` functions worked ok.
This patch fixes that crashes.

Closes #9173

NO_DOC=bugfix

(cherry picked from commit d85556c9)

If MP_ERROR map contains two MP_ERROR_STACK keys, then the second call to
`error_set_prev(effect, cur)' will crash, because `effect' is NULL, but
`err == NULL' is false, because it is assigned on the first iteration.
This patch raises an error if more than one MP_ERROR_STACK key is present.

NO_DOC=bugfix

Closes #9136

(cherry picked from commit 990aeee9)

Some downgrade operations are performed with disabled system space
triggers because they were prohibited recently (creation of SQL built-in
functions) or never allowed (dropping a system space). This works fine
on the instance running downgrade but apparently fails on replicas.

To fix this issue, let's disable the checks the operations that prevent
downgrade in the following scenarios:
 - in the fiber that is currently running a schema upgrade or downgrade;
 - in the applier fiber so that it can replicate changes done by upgrade
   or downgrade on the master;
 - during recovery so that DDL records written to the WAL can be
   replayed.

We already have all the necessary infrastructure in-place - we use it
for allowing DDL operations with an old schema for upgrade.

Closes #9049

NO_DOC=bug fix

(cherry picked from commit 71de4b2c)

NOTE: We don't have the commit that disables DDL operations with an old
schema in 2.11 so we have to backport bits of it from 3.0, see commit
97c2c9a4 ("box: disable DDL with old schema").

If `strlen(name)` is 1, `value_size` is 1, and `extra` is 0, then 15 bytes
are allocated for `struct error_field` in error_payload_prepare(). However,
the size of this structure is 16 because of the padding for the alignment.
Thus TRASH() in error_payload_destroy() writes 1 byte beyond the structure.

Closes #9098

NO_DOC=bugfix

(cherry picked from commit 454ffd13)

An attempt to print a dead fiber raised a fatal error, which is quite
unexpected. This patch updates __tostring metamethod of fiber_object so
that it pushes the "fiber: <fid> (dead)" string instead of the error.
The __serialize metamethod is patched similarly.

Closes #4265

NO_DOC=bugfix

(cherry picked from commit 3421a3bd)

Historically, we encode strings that contain invalid or non-printable
utf-8 sequences in YAML as binary base64 blobs. We do that because of
limitations/bugs of the YAML encoder, which refuses to encode invalid
utf-8 strings. To work around this issue, we introduced the helper
utf8_check_printable, which is basically a copy of yaml_check_utf8,
and treat strings for which it fails as binary data (MP_BIN).

This commit updates the YAML submodule to the version where all known
issues with encoding invalid/unprintable utf-8 strings are fixed and
removes special treatment of such strings (drops utf8_check_printable).
Now unprintable or invalid utf-8 sequences are emitted as code points,
e.g. '\xFF' or '\uFFFF'. This change is a pre-requisite for introducing
the new varbinary type to Lua. Without it plain strings would be
implicitly converted to varbinary after decoding/encoding them in YAML,
which would be confusing.

Closes #8756

NO_DOC=bug fix

(cherry picked from commit 890a821c)