If MP_ERROR map contains two MP_ERROR_STACK keys, then the second call to
`error_set_prev(effect, cur)' will crash, because `effect' is NULL, but
`err == NULL' is false, because it is assigned on the first iteration.
This patch raises an error if more than one MP_ERROR_STACK key is present.

NO_DOC=bugfix

Closes #9136

(cherry picked from commit 990aeee9)

Some downgrade operations are performed with disabled system space
triggers because they were prohibited recently (creation of SQL built-in
functions) or never allowed (dropping a system space). This works fine
on the instance running downgrade but apparently fails on replicas.

To fix this issue, let's disable the checks the operations that prevent
downgrade in the following scenarios:
 - in the fiber that is currently running a schema upgrade or downgrade;
 - in the applier fiber so that it can replicate changes done by upgrade
   or downgrade on the master;
 - during recovery so that DDL records written to the WAL can be
   replayed.

We already have all the necessary infrastructure in-place - we use it
for allowing DDL operations with an old schema for upgrade.

Closes #9049

NO_DOC=bug fix

(cherry picked from commit 71de4b2c)

NOTE: We don't have the commit that disables DDL operations with an old
schema in 2.11 so we have to backport bits of it from 3.0, see commit
97c2c9a4 ("box: disable DDL with old schema").

If `strlen(name)` is 1, `value_size` is 1, and `extra` is 0, then 15 bytes
are allocated for `struct error_field` in error_payload_prepare(). However,
the size of this structure is 16 because of the padding for the alignment.
Thus TRASH() in error_payload_destroy() writes 1 byte beyond the structure.

Closes #9098

NO_DOC=bugfix

(cherry picked from commit 454ffd13)

An attempt to print a dead fiber raised a fatal error, which is quite
unexpected. This patch updates __tostring metamethod of fiber_object so
that it pushes the "fiber: <fid> (dead)" string instead of the error.
The __serialize metamethod is patched similarly.

Closes #4265

NO_DOC=bugfix

(cherry picked from commit 3421a3bd)

Historically, we encode strings that contain invalid or non-printable
utf-8 sequences in YAML as binary base64 blobs. We do that because of
limitations/bugs of the YAML encoder, which refuses to encode invalid
utf-8 strings. To work around this issue, we introduced the helper
utf8_check_printable, which is basically a copy of yaml_check_utf8,
and treat strings for which it fails as binary data (MP_BIN).

This commit updates the YAML submodule to the version where all known
issues with encoding invalid/unprintable utf-8 strings are fixed and
removes special treatment of such strings (drops utf8_check_printable).
Now unprintable or invalid utf-8 sequences are emitted as code points,
e.g. '\xFF' or '\uFFFF'. This change is a pre-requisite for introducing
the new varbinary type to Lua. Without it plain strings would be
implicitly converted to varbinary after decoding/encoding them in YAML,
which would be confusing.

Closes #8756

NO_DOC=bug fix

(cherry picked from commit 890a821c)

Currently, the zero-width space Unicode character is silently ignored by
the YAML encoder but with commit https://github.com/tarantool/libyaml/commit/351108e1ffd5c6296349fa9ab87f91ef8a84c4e7
("Allow to emit invalid utf-8 scalar strings"), it will print it as
a unicode code point, as it should. The fselect helper uses this
character to prevent the YAML encoder from quoting the output string.
Actually, quoting is triggered by usage of spaces and vertical bars.
We already replace spaces with NBSP (U+00A0). Let's also replace
vertical bars with Latin Letter Dental Click (U+01C0). It looks the
same but has no special meaning to YAML.

In the master branch this issue was fixed by removing the use_nbsp
option of fselect completely and using multi-line strings instead,
see commit f76d3c69 ("box: make fselect return a multi-line string
instead of a table"). We can't do that in 2.11 because there the
yaml_pretty_multiline compat option is disabled by default and without
it multi-line strings look ugly in YAML.

Needed for #8756

NO_DOC=no user-visible changes
NO_CHANGELOG=no user-visible changes

Fiber flags are initialized after fiber stack creation. As result
currently check for custom stack in fiber_stack_watermark_create does
not work. This leads to heap-buffer-overflow on putting watermark
if custom stack size is less than FIBER_STACK_SIZE_WATERMARK.

Close #9026

NO_DOC=bugfix

(cherry picked from commit 750e4890)

`msg` is used after it is freed in iproto_msg_delete.

Close #9037

NO_TEST=tested by ASAN
NO_DOC=bugfix

(cherry picked from commit 4916389a)

When fiber region is freed/destroyed and ENABLE_BACKTRACE is set then
`fiber_on_gc_truncate` callback is called. At this time both `used`
argument and `fiber->gc_initial_size` are equal to 0. Thus
`fiber->first_alloc_bt` is accessed which is already freed.

With a bad luck freeing fiber region can put slab back into slab arena.
So writing after free can change memory used by another thread.

Closes #9020

NO_TEST=tested by ASAN
NO_DOC=bugfix

(cherry picked from commit c480a867)

`*type_out` was set to uninitialized value for `field->type == MP_EXT`.
This was introduced by commit 9f9142d6 ("box: cleanup on tuple encoding
failure")

Closes #9023

NO_DOC=bugfix
NO_CHANGELOG=not user-visible

(cherry picked from commit bbfaef3b)

The maximum length of box.cfg{} string parameters is now 512 instead of
256 before.

NO_DOC=no need to document
NO_TEST=will be added in EE

(cherry picked from commit f5511d45)

Memory is leaked in the following scenario:
- MP_ERROR_STACK with 2 errors is passed to error_unpack_unsafe():
  1. A correct MP_MAP with MP_ERROR_* fields;
  2. Something unexpected, e.g. MP_INT;
- This first call to mp_decode_error_one() allocates memory for the first
  error in error_build_xc() -> `new ClientError()`;
- The second call to mp_decode_error_one() returns NULL, and
  error_unpack_unsafe() returns NULL too. Memory from the previous step
  is leaked.

Closes #8921

NO_DOC=bugfix

(cherry picked from commit b367fb98)

part_count was checked in index_def_check(), which was called too late.
Before that check:
1. `malloc(sizeof(*part_def) * part_count)` can fail for huge part_count;
2. key_def_new() can crash for zero part_count because of out of bound
   access in:

NO_WRAP
   - #1 key_def_contains_sequential_parts (def=0x5555561a2ef0) at src/box/tuple_extract_key.cc:26
   - #2 key_def_set_extract_func (key_def=0x5555561a2ef0) at src/box/tuple_extract_key.cc:442
   - #3 key_def_set_func (def=0x5555561a2ef0) at src/box/key_def.c:162
   - #4 key_def_new (parts=0x7fffc4001350, part_count=0, for_func_index=false) at src/box/key_def.c:320
NO_WRAP

Closes #8688

NO_DOC=bugfix

(cherry picked from commit ef9e3320)

test/unit/guard.cc calls stack_break_f() recursively until the stack
overflows and a signal is fired, however it relies on undefined behavior
when compares pointers to local variables. Fixed by comparing
__builtin_frame_address() instead.

One of the examples of this UB is when ASAN allocates local variables on
fake stacks, in that case the test completes without the stack overflow.

Also this patch disables ASAN for stack_break_f() to keep the array on the
fiber stack (see the corresponding comment) and marks it as volatile to
avoid optimizing it away by the compiler.

Closes tarantool/tarantool-qa#323

NO_DOC=test fix
NO_CHANGELOG=test fix

(cherry picked from commit 05b696c7)

Runtime tuple formats are reusable, which means that a tuple format
returned by runtime_tuple_format_new may not be brand new, but actually
be used by a Lua object. As a result, if we call any function that may
trigger Lua GC between runtime_tuple_format_new and tuple_format_ref,
the tuple format may be deleted, leading to a use-after-free bug. This
is what happens in lbox_tuple_format_new. Fix this issue by taking a
reference to the format before pushing a cdata object to the Lua stack
in lbox_push_tuple_format.

The issue was fixed in the master branch by commit 28ec245d ("lua:
fix heap-use-after-free bug in tuple format constructor"). This isn't
a clean cherry-pick because the code changed quite a bit.

Closes #8889

NO_DOC=bug fix
NO_TEST=difficult to reproduce, found by ASAN

Examples of IPROTO decoding issues: #3900, #1928, #6781.
Patch adds a number of fuzzing tests that covers IPROTO decoding:

- xrow_decode_auth
- xrow_decode_begin
- xrow_decode_call
- xrow_decode_dml
- xrow_decode_id
- xrow_decode_raft
- xrow_decode_sql
- xrow_decode_watch
- xrow_greeting_decode

NO_DOC=testing
NO_CHANGELOG=testing

(cherry picked from commit 46cacf35)

The checks in box.schema.index.create() and box.schema.index.alter()
were case sensitive, also it was possible to insert incorrect index
options directly into `box.space._index`. Fixed by adding checks
to memtx_space_check_index_def() and vinyl_space_check_index_def().

Closes #8937

NO_DOC=bugfix

(cherry picked from commit 4e25384b)

And remove unused index_def_check_xc().

As index_def_dup() never returns NULL anymore, change index_create() and
index_read_view_create() return type to `void` and update their callers.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit f6d61754)

The `__sanitizer_start_switch_fiber()` function takes a pointer as the
first argument to store the current fake stack if there is one (it is
necessary when stack-use-after-return detection is enabled). When leaving a
fiber definitely, NULL must be passed so that the fake stack is destroyed.

Before this patch, NULL was passed for dead fibers, however this is wrong
for dead fibers that are recycled and resumed. In such cases ASAN destroys
the fake stack, and the fiber crashes trying to use it in `fiber_yield()`
upon return from `coro_transfer()`.

Closes tarantool/tarantool-qa#321

NO_DOC=bugfix
NO_TEST=tested by test-release-asan workflow

(cherry picked from commit 72a6abee)

The function can't be called on an unconfigured instance because it
needs IPROTO threads up and running. Let's raise an error to avoid
a crash.

Since we have two other places where we need to raise the same error
(box.session.su and box.__index), let's introduce the new code
ER_UNCONFIGURED for this error.

Closes #8975

NO_DOC=bug fix

(cherry picked from commit 4fd2686e)

Pin in cache the collation identifiers that are referenced by space format
and/or indexes, so that they can't be deleted.

Closes #4544

NO_DOC=bugfix

(cherry picked from commit d69aa687)

It was possible to delete a collation, which is in use by a key_def.

Part of #4544

NO_DOC=bugfix
NO_CHANGELOG=next commit

(cherry picked from commit 07beb340)

Since number type was introduced we can not assume if tuples are
equal by comparison then their sizes are equal too. So the place
the assumption is used is fixed.

Closes #8899

NO_DOC=bugfix

(cherry picked from commit f4de9faf)

There must be a couple of rules:
* foreign key from non-temporary space to temporary space must be
  forbidden since after restart all existing links will be broken.
* foreign key from non-local space to local space must be forbidden
  on any replica all existing can be broken.

This patch implements the rules.

Closes #8936

NO_DOC=bugfix

(cherry picked from commit 7d23b339)

In #7309 a truncation of a space that was referenced by foreign
key from some other space was prohibited.

It appeared that this solution is too bothering since a user can't
truncate a space even if he truncated referring space before that.

Fix it by allowing space truncate if referring spaces are empty.
Also allow drop of the primary index in the same case with the
same reason: logically the index along with all space data is
not needed for consistency if there's no referring data.

Note that by design space truncate is implemented quite similar
to space drop. Both delete all indexes, from secondary to primary.
Since this patch allows deletion of the primary index (which is
the action that actually deletes all data from the space), this
patch changes the result of space drop too: the space remains
alive with no indexes, while before this patch it remained alive
with no secondary indexes but with present primary. In both cases
the behaviour is quite strange and must be fixed in #4348. To
make tests pass I had to perform drop in box.atomic manually.

Closes #8946

NO_DOC=bugfix

(cherry picked from commit 983a7ec2)

This should suppress the following coverity issues:

https://scan7.scan.coverity.com/reports.htm#v39198/p13437/fileInstanceId=146712118&defectInstanceId=18978766&mergedDefectId=1563095
https://scan7.scan.coverity.com/reports.htm#v39198/p13437/fileInstanceId=146712113&defectInstanceId=18978750&mergedDefectId=1563094

While we are at it, let's use space_by_id instead of space_cache_find
because read view creation is a rare operation affecting all spaces so
caching the last space by id doesn't make any sense.

NO_DOC=code health
NO_TEST=code health
NO_CHANGELOG=code health

(cherry picked from commit f2886dd0)

The is_new and is_old methods are the same for all compat options so
they should be defined in a metatable. A good thing about this change
is that it removes is_new and is_old from serialization:

* Before:

NO_WRAP
  tarantool> require('compat').yaml_pretty_multiline
  ---
  - is_new: 'function: 0x4175d6e8'
    is_old: 'function: 0x4175d790'
    brief: |
      Whether to encode in block scalar style all multiline strings or ones
      containing "\n\n" substring. The new behavior makes all multiline string output
      as single text block which is handier for the reader, but may be incompatible
      with some existing applications that rely on the old style.

      https://tarantool.io/compat/yaml_pretty_multiline
    current: default
    default: new
  ...
NO_WRAP

* After:

NO_WRAP
  tarantool> require('compat').yaml_pretty_multiline
  ---
  - current: default
    brief: |
      Whether to encode in block scalar style all multiline strings or ones
      containing "\n\n" substring. The new behavior makes all multiline string output
      as single text block which is handier for the reader, but may be incompatible
      with some existing applications that rely on the old style.

      https://tarantool.io/compat/yaml_pretty_multiline
    default: new
  ...
NO_WRAP

To achieve that, we have to remove the option name from the usage error
message but it seems to be okay because such errors shouldn't happen in
practice and the error message is clear enough to figure out what went
wrong.

Follow-up #8807

NO_DOC=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 75b5fd05)

Applier thread uses lsregion to allocate the messages for tx thread. The
messages are freed upon return to the applier thread using a
corresponding lsr_id.

Due to a typo, one of the lsregion allocations was made with a postfix
increment of lsr_id instead of the prefix one. Essentially, part of a
new message was allocated with an old lsr_id, and might be freed early
by a return of a previous message.

Fix this.

Closes #8848

NO_DOC=bugfix
NO_TEST=covered by asan in #8901
NO_CHANGELOG=bugfix

(cherry picked from commit 0d5bd6b7)

Currently if tarantool exits during relay's final join stage,
corresponding thread isn't terminated. This causes the flakiness
of the replicaset_ro_mostly.test.lua.

Let's reuse the same relay, in which subscribe cord is running, for
the final join stage. This way the cord will be cancelled during
replication_free().

Closes #8082

NO_DOC=not user-visible
NO_TEST=fix flaky test
NO_CHANGELOG=not user-visible

Co-authored-by: Sergey Petrenko <sergepetrenko@tarantool.org>

(cherry picked from commit 70a68836)

This patch fixes an issue in generate_column_metadata(). Prior to this
patch, the number of variable-only expressions was counted incorrectly
when temporary memory was allocated on region to store their positions.
However, although this allocation was incorrect, this did not lead to
any problems due to the specifics of the region allocations.

This patch fixes this by removing the temporary memory allocation.

Closes #8763

NO_DOC=no user-visible changes
NO_TEST=no user-visible changes
NO_CHANGELOG=no user-visible changes

(cherry picked from commit d4f143ad)

It is possible for interval to have days, hours, minutes and seconds
larger than INT_MAX (or less than INT_MIN). Before this patch, msgpack
decoding had failed to parse intervals with msgpack int64 and uint64.
int64_t should be enough to store any value allowed for datetime
intervals.

Closes #8887

NO_DOC=small bug fix

(cherry picked from commit 01c7ae11)

`api_check()` is the LuaJIT internal assertion. To prevent inconsistency
during internal assertion changes (for example, during backporting), use
glibc's `assert()` instead.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

(cherry picked from commit e78afb3b)

Before this patch, one couldn't create new datetime interval with
boundary value from Lua. At the same time, it was possible to create
such interval from Lua through addition and subtraction. C range
verification allow to create boundary value intervals, error message
also implies that they should be allowed. (See #8878 for more info.)

Closes #8878

NO_DOC=small bug fix

(cherry picked from commit b2a001cc)

Function `datetime_strptime` decodes string with datetime according to
specified format, it accepts a datetime struct, buffer with datetime and
string with format in arguments. Fuzzing test used static string
"iso8601" as a format and it blocked fuzzing test to cover functions
used by datetime_strptime under the hood. Fuzz introspector shows that
code coveraged by a test is quite low.

Patch updates the test to make it more effective: buffer with datetime
and format string are generated using FDP (Fuzzing Data Provider).

Test file extension was changed to .cc, because FuzzingDataProvider is
used and we need building it by C++ compiler.

Function `tnt_strptime` uses assert, that triggered by fuzzing tests.
Therefore it was replaced with to if..then.

1. https://storage.googleapis.com/oss-fuzz-introspector/tarantool/

Fixes #8490

NO_CHANGELOG=fuzzing test
NO_DOC=fuzzing test
NO_TEST=fuzzing test

(cherry picked from commit a1bd6e0b)

Fixes #8502
Needed for #8490

NO_DOC=bugfix
NO_TEST=covered by fuzzing test

(cherry picked from commit 783a7040)

To achieve that, we bypass the read-only check for the _truncate system
space in box_process1() and perform it in the on_replace system trigger
instead, when we know which space is truncated.

Note, we have to move the check for insertion of a new record into the
_truncate system space before the read-only check in the on_replace
trigger callback; this is needed for initial recovery with a non-empty
_truncate space to work. While we are at it, let's use recovery_state to
make the check explicit.

Closes #5616

@TarantoolBot document
Title: Mention that temp and local spaces can be truncated in ro mode

DML operations on temporary and local spaces can be performed even if
the instance is in the read-only mode, but DDL operations (such as
`alter`) are forbidden in this case[^1]. Technically, `truncate` is
a DDL operation so initially it was forbidden as well. However, it
should be safe to perform this operation on a temporary or local space
because logically it only modifies the data stored in the space (like
DML) and it isn't replicated (see tarantool/tarantool#4263). So starting
from Tarantool 2.11.1 we allow users to truncate temporary spaces in the
read-only mode.

[^1]: https://www.tarantool.io/en/doc/latest/concepts/replication/repl_architecture/#replication-local

(cherry picked from commit 054526ac)

A read source iterator stores statements in a vy_history object using
vy_history_append_stmt(). If a statement can be referenced, it's
reference counter is incremented. If it can't, i.e. it belongs to a
memory source, it's stored in a vy_history object without referencing.

This works fine because memory sources are append-only. A problem arises
only when we get to scanning disk sources. Since we yield while reading
disk, a dump task may complete concurrently dropping the memory sources
and possibly invalidating statements stored in the iterator history.
Although we drop the history accumulated so far and restart the
iteration from scratch in this case, there's still an issue that can
result in a use-after-free bug in vy_read_iterator_next().

The problem is that we access the current candidate for the next
statement while evaluating a disk source after a disk read. If 'next'
refers to a referenced statement, it's fine, but if it refers to a
statement from a memory source, it may cause use-after-free because
the memory source may be dropped during a disk read.

To fix this issue, let's make vy_history_append_stmt() copy statements
coming from memory sources. This should be fine performance-wise because
we copied memory statements eventually in vy_history_apply() anyway,
before returning them to the user.

Note that we also have to update vy_read_iterator_restore_mem() because
it implicitly relied on the fact that 'next' coming from a memory source
can't be freed by vy_mem_iterator_restore(), which cleans up the memory
source history. Now, it isn't true anymore so we have to temporarily
take a reference to 'next' explicitly.

Closes #8852

NO_DOC=bug fix
NO_TEST=tested by ASAN

(cherry picked from commit 0e5a3cc2)

In scope of the commit 82f4b4a3 ("lib/core/fiber: Increase default
stack size") the default value of fiber stack size is increased up to
512 Kb (you can find the reasons in the aforementioned commit message
and in https://github.com/tarantool/tarantool/issues/3418 description).

Some of the tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT repo
(e.g. some cases with deep recursion in errors.lua or pm.lua) have
already been tweaked according to the limitations mentioned in
https://github.com/tarantool/tarantool/issues/5782, but the crashes
still occurs while running LuaJIT tests with ASan support enabled.

To make the testing routine more convenient, FIBER_STACK_SIZE option is
introduced to Tarantool CMake machinery. One can provide the size either
by raw digits (i.e. in bytes) or using Kb/Mb suffixes for convenience.

A couple of important nits:
* If the given value is not a multiple of 4Kb, CMake machinery adjusts
  it up to the nearest one greater than this value.
* If the adjusted value is less than 512Kb, configuration fails with the
  corresponding CMake fatal error.

Follows up #3418
Relates to #5782

@TarantoolBot document
Title: introduce FIBER_STACK_SIZE configuration option

To make managing of the default fiber stack size more convenient, the
corresponding CMake option is added.

**NB**: The stack size can't be less than 512Kb and if the given value
is not a multiple of 4Kb, CMake machinery adjusts it up to the nearest
one greater than this value.

(cherry picked from commit ff57f990)

In the original commit 5f6d367c ("compat: add is_new and
is_old to options") `compat.<option_name>:is_new()` and `:is_old()`
were introduced, but by mistake they contained different usage
messages. This patch updates `:is_new()` usage msg to more
informative one from `:is_old()`.

Follows up #8807

NO_CHANGELOG=changelog from 5f6d367c is valid
NO_DOC=doc from 5f6d367c is valid

(cherry picked from commit f590ec22)

It used to be somewhat complicated to check the effective value
of a compat option, because `<option_name>.current` could contain
'default' state.
This patch introduces helper functions that take care of that.

The following alternatives were considered:
* `compat.<option_name>.effective` - it is excessive in the presence
  if `current` and `default`, and is visible in serialization
* `compat.<option_name>.get()` - while it is a function, it does only
  half of the work required, user still has to compare result to 'new'

Closes #8807

@TarantoolBot document
Title: Add `:is_new/old()` helpers to tarantool.compat options

`compat.<option_name>.current` can be 'new', 'old' or 'default',
thus when it is default there must be an additional check if
`compat.<option_name>.default` is 'new'. It is handier to have a
helper to deal with that instead of complicated `if`:
* check if effective value is 'new' before the patch:
  ```lua
  if compat.<option_name>.current == 'new' or
          (compat.<option_name>.current == 'default' and
           compat.<option_name>.default == 'new') then
      ...
  end
  ```
* after the patch:
  ```lua
  if compat.<option_name>:is_new() then
      ...
  end
  ```

Please update [tutorial on using compat], maybe add an example to
[Listing options details].

[tutorial on using compat]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/compat/compat_tutorial/
[Listing options details]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/compat/compat_tutorial/#listing-options-details

(cherry picked from commit 5f6d367c)