Old: "Failed to allocate 2147483648 bytes in hash_table for key"
New: "Failed to allocate 16384 bytes in hash_table for key"

ERRINJ_INDEX_ALLOC cannot be used to test this error, because it fails
earlier, so ERRINJ_HASH_INDEX_REPLACE is introduced.

Follow-up #3594

NO_DOC=minor
NO_CHANGELOG=minor

(cherry picked from commit 0a8043d1)

The issue is fixed in the `small' submodule by the following commit:

* matras: fix matras_view::block_count overflow

Closes #3594

NO_TEST=The test requires more than 64 GB of RAM.

@TarantoolBot document
Title: Document maximum number of tuples in hash index
Product: Tarantool
Root document: https://www.tarantool.io/en/doc/latest/book/box/limitations/

Number of tuples in hash index: 2147483648

(cherry picked from commit 7eaa2d23)

The bug was fixed in the small library:
 - slab: fix NULL ptr deref in assertion in slab_get
   https://github.com/tarantool/small/commit/ef77efacd452cb90caea2caf22d266f791c95ec3
 - slab: fix uint32_t overflow in slab_capacity
   https://github.com/tarantool/small/commit/77203600a7c645d97bce56f901eec25de0b29d6e

The small library submodule was updated in commit ebafd684
("small: bump version").

Closes #9218

NO_DOC=changelog
NO_TEST=changelog

(cherry picked from commit 1fc6b4a1)

This is part of master commit bd4c6675 ("memtx: use MemtxAllocator
stats for box.info.memory"). The test is failed without the patch in
ASAN build.

Follow-up #7327

NO_CHANGELOG=test fix
NO_DOC=test fix

Message body resides in one of rotating input buffers for the
connection. When we don't need message body anymore we advance the
reading end of the input buffer by the size of the message. But message
processing order can differ from the order of messages in the wire.
Thus this advancing a bit hacky. Let's instead mark the data in the
input buffer as read when we process all the messages in the input
buffer. We cannot reuse the buffer any earlier anyway.

Follow-up #7327

NO_TEST=refactoring
NO_CHANGELOG=refactoring
NO_DOC=refactoring

(cherry picked from commit 9e918278)

The API functions additionally poison related data in ASAN build.

Follow-up #7327

NO_TEST=refactoring
NO_CHANGELOG=refactoring
NO_DOC=refactoring

(cherry picked from commit 73832668)

The new macro is like assert, but it evaluates the checked expression
even in the release mode.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

(cherry picked from commit bfd2a20d)

The test is quite a flacky in debug ASAN CI workflow. The issue is test
check upper boundary of it's execution time. I run many instances of
this test on in parallel and got average time of 40s for memtex and 70s
for vinyl.

The time quota is already changed by the commit 84cb1e04 ("sql:
increase time quota for selectG test on vinyl") for laptops with HDD.
I did not check execution time for HDD though. I guess the bottleneck
for debug ASAN is CPU.

Follow-up #7327

NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 703d11fe)

This patch also includes:
  - misc trivial fixes for ASAN discovered issues
  - minor adaptations for ASAN friendly allocators

Closes #7327

NO_DOC=internal
NO_CHANGELOG=internal

(cherry picked from commit 446201b8)

With new ASAN-friendly small implementation unit/fiber_stack.c test
start to fail. The issue is leak sanitizer reports a leak. This is an
expected leak of test for mprotect failure on fiber stack destruction.
Let's tell sanitizer to ignore this case.

By the way let's drop test code for temporary redirecting stderr. It is
outdated as test is TAP-compatible. It was a PITA as due to this
redirection there was no leak report only error exit code.

Part of #7327

NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 41844ffb)

With ASAN-friendly small allocators there are a lot test failures due
to leak reports which are gone if JIT is off.

Fortunately all the reports related to a few functions. Let's suppress
temporarily such reports.

Part of #7327

NO_TEST=internal
NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit f0feb923)

When SMALL_MALLOC_IMPL is defined and ASAN-friendly allocators are used
the arena allocator is not used at all as we not allocate memory
directly from there. And other ASAN-friendly allocators are not allocate
from it too. Thus box.slab.info().arena_size == 0. Same for usage
of runtime arena box.runtime.info().used.

Also usage with ASAN-friendly lsregion is a bit different as it does
not account for size of alignment padding. Thus we need to adapt
box.stat.vinyl().memory.level0 tests. Approach is to check for lower
and upper limit instead of checking for exact values.

Part of #7327

NO_DOC=test changes
NO_CHANGELOG=test changes

(cherry picked from commit 1436eb41)

ASAN-friendly implementation poisons memory after allocation with
ibuf_alloc so we need to fix existing places in code where we access
memory after allocation.

Part of ibuf implementation is inline functions in headers. Thus ibuf
implementation in Lua reimplement this parts. We add poison to these
inline functions in ASAN-friedly implementation so we need add same poison
in Lua implementation.

Part of #7327

NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 4f542bb7)

We are going to include generated small_config.h into small allocator
headers (currently it is only included in small source files).
core/memory.h depends on small headers and salad/heap.h depends on
core/memory.h. As a result we need to provide a way for salad/heap.h
users to find small_config.h header.

Instead let's drop dependency from core/memory.h as we only use it for
typeof definition.

Part of #7327

NO_CHANGELOG=code cleanup
NO_DOC=code cleanup

(cherry picked from commit d01609a4)

If leak sanitizer reaches the memory protected from read with mprotect
it exhibits all sorts of odd behaviour. It can hang, can crash, can
return errors with no leak backtraces.

We use mprotect to create guard zones at the end of fiber stack so if
stack is overflowed we get a signal and crash. We take protection off
when fiber is destroyed. Unfortunately we do not destroy cords (and its
fibers) which cancelled through cord_cancel_and_join. This is going to
be addressed in patch for issue #8423 ("Get rid of pthread_cancel()").
Until that moment let's disable protection for ASAN builds.

Note that we did not hit this behaviour before because LSAN only scans
memory allocated using malloc and regular slab cache uses mmap to get
memory.

Part of #7327

NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 2ee15793)

Regularly fiber stack slab is page aligned. So upper stack border is
page aligned too when stack grows down. But with ASAN friendly slab
cache implementation this border is not page aligned. As a result
madvise call on stack may zero memory beyond stack slab which will cause
heap corruption. In debug build corruption is detected by assertion:

NO_WRAP
 >  Fatal glibc error: malloc.c:2593 (sysmalloc): assertion failed: (old_top
 >  == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >=
 >  MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize
 >  - 1)) == 0)
NO_WRAP

Interestingly enough the issue can not be investigated using ASAN. The
memory is zeroed by kernel code which is not instrumented so it is
invisible for sanitizer.

Looks like non-ASAN builds are not affected. Even if stack_size is
not page aligned the slab allocated for stack is page aligned. Thus
memory zeroing will be inside the slab and there will be no memory
corruption.

Also when stack grows up lower stack border in not aligned even with
regular small implementation. So madvise call will fail with EINVAL as
it is required that start address is page aligned. We ignore the error
though. Let's fix this issue too while we at it.

Let's introduce fiber_madvise_aligned to align madvise range with proper
direction before calling madvise(2). To justify its usage note that
besides fixing the issues described above, in case of stack growing down
fiber->stack is page aligned and in case of stack growing up
fiber->stack + fiber->stack_size is page aligned.

Part of #7327

NO_TEST=tested by ASAN (debug build)
NO_CHANGELOG=has effect only with newly introduced ASAN friendly slab cache
NO_DOC=has effect only with newly introduced ASAN friendly slab cache

(cherry picked from commit 130c7807)

The unpoison was added in the initial commit 1.7.2-68-gafd229393 that
supported ASAN. It is not clear why do we need it as we don't poison
stack memory manually.

Part of #7327

NO_TEST=removing unfunctional code
NO_CHANGELOG=removing unfunctional code
NO_DOC=removing unfunctional code

(cherry picked from commit 0784f7b7)

ASAN small object allocator implementation has a bit different pattern
on quota leasing on allocating memory. So we may need to allocate more
objects to hit the quota etc.

Part of #7327

NO_CHANGELOG=test tuning
NO_DOC=test tuning

(cherry picked from commit d456a986)

This patch removes some deprecated code. This code had no user-visible
effect, but caused problems when running the test with ASAN enabled.

Closes #8761

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit d63a4bf2)

Regular region implementation supports allocations of size 0 with no
extra efforts. It returns a non-NULL pointer in this case. However in
case of ASAN friendly implementation it will require a special care for
this case. Instead let's avaid allocations if size 0 for region.

Also use xregion_ macros for allocations. Our current policy is to panic
on OOM on runtime allocations.

Part of tarantool/tarantool#7327

NO_TEST=internal
NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 8159347d)

Small library currently depends on Tarantool core through 'exception.h'.
This is not the way to go. Let's drop this dependency and instead of
moving _xc functions to Tarantool repo we can just stop using them. Our
current policy is to panic on OOM in case of runtime allocation.

Part of #7327

NO_DOC=<OOM behaviour is not documented>
NO_CHANGELOG=<no OOM expectations>
NO_TEST=<no test harness for checking OOM>

(cherry picked from commit 3fccfc8f)

They are rather noisy. Also delete debug log on arena creation. These
two make sense only with each other.

Part of #7327

NO_TEST=internal
NO_DOC=internal
NO_CHANGELOG=internal

(cherry picked from commit 0dc37356)

Panic if we fail to allocate internal temporary objects on region. We do
not test allocation failures and this should normally happen also
 (see #3534).

Part of #8658

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

(cherry picked from commit b1a03a49)

This patch replaces region_*() functions with xregion_*() functions.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 1ba84fe3)

This patch removes the 'size' argument from macros, as it was only used
to set an error on failure, which is not possible for x* versions. In
addition, both macros now cast the value to the specified type, as is
done in the original macros.

Closes #8522

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

(cherry picked from commit ae02f0cd)

This patch fixes SQL memory leaks found by static analyzers and SQL
fuzzer.

Part of tarantool/security#120

NO_DOC=fix for memleak
NO_TEST=fix for memleak
NO_CHANGELOG=fix for memleak

(cherry picked from commit cd173ce5)

Proposed ASAN implementation of region allocator does not support double
reservation for the sake of simplicity. Every reservation is supposed to
be followed by one or more allocations.

This restriction does not work well with mpstream currently. The issue is
mpstream_init/mpstream_reserve do reservation of size 0. For example In
case of region slab of min order is reserved (a chunk of memory of page
size currently). If the first data we want to write to mpstream is
larger then the reservation done then we make reservation again.

Let's get rid of this reservation at the beginning as it is suboptimal
behaviour. Moreover let's get rid of mpstream_reset as mpstream_init
is lightweight and we can create a new mpstream instead of reusing
exiting.

Also while we at it avoid allocation of 0 size in mpstream_flush as it
is done in mpstream_reserve_slow (see 3.0.0-alpha3-19-g8159347d0 "misc:
avoid allocations of size 0 for region" for details).

NO_TEST=internal
NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 3b1de78d)

This way we will have access to build info in those modules. In
particularly build.asan flag is going to be used in buffer.lua in scope
of #7327.

Part of #7327

NO_TEST=internal
NO_DOC=internal
NO_CHANGELOG=internal

(cherry picked from commit f58cc96f)

We already use this info in one of the test and going to use it more.

Part of #7327

@TarantoolBot document
Title: new tarantool.build.asan flag

It is `true` if `ENABLE_ASAN` build option is set and `false` otherwise.

(cherry picked from commit 23012356)

The check_param and check_param_table Lua helpers are defined in
box/lua/schema.lua but used across the whole code base. The problem is
we can't use them in files that are loaded before box/lua/schema.lua,
like box/lua/session.lua. Let's move them to a separate source file
lua/utils.lua to overcome this limitation. Also, let's add some tests.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit d8d267c5)

We hit #3807 in release/2.11 for release ASAN build with ASAN-friendly
small allocators.

Follow-up #7327

NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 3fbd7fcb)

Configuring log modules work differently with log.cfg and box.cfg:
box.cfg{log_modules=...} overwrites the current config completely while
log.cfg{modules=...} overwrites the currently config only for the
specified modules. Let's fix this inconsistency by making log.cfg behave
exactly as box.cfg.

Closes #7962

NO_DOC=bug fix

(cherry picked from commit c13e59a5)

We install a signal handler that prints the stack trace on SIGSEGV,
SIGBUS, SIGILL, SIGFPE. The signal handler uses the current stack.
This works fine for most issues, but not for stack overflow, because
the latter makes the current stack unusable, leading to a crash in
the signal handler. Let's install an alternative signal stack in each
thread so that we can print the stack trace on stack overflow.

Note that we skip this for ASAN because it installs its own signal
stack. (Installing a custom stack would result in a crash.)

Closes #9222

NO_DOC=bug fix

(cherry picked from commit cb8e903b)

The motivation is to reduce time slip on Tarantool startup before
running init scripts. Internal ev time is set in fiber_init/ev_default_loop
and is not get updated until starting event loop. This causes
timeouts slip up to 0.3 in debug ASAN build in init script (see #9261).

Let's run event loop right at the beginning of the run_script_f before
executing any script. This way besides updating internal ev time we make
an explicit place of starting script event loop. Currently it is started
lazily when config script yields.

This will fix CI for PR https://github.com/tarantool/tarantool-ee/pull/572
for debug ASAN workflow.

We can also remove start_loop condition. It does not make sense now. It
was added in the commit 3a851430 ("Fix tarantool -e "os.exit()"
hang") but since then we start to stop event loop after handling
os.exit().

Also this fixes #9266. The issue is we don't have an event loop to run
on shutdown triggers if -e command line expression add such a trigger
and then call os.exit().

Follow-up #7327
Closes #9266

NO_DOC=bugfix

(cherry picked from commit 1fcfb8c2)

This patch fixes issue:

$ tarantoolctl rocks --version 1>/dev/null
Warning: failed to load command module luarocks.cmd.help

NO_DOC=bugfix
NO_CHANGELOG=not released yet

(cherry picked from commit d6ae403e)

Tarantool supports two console protocols: text and binary. The binary
protocol is implemented with IPROTO EVAL request so the console module
reuses the net.box module to establish and maintain a binary connection.
Currently, instead of passing the original URI specified by the user to
net.box.connect as is, the console module parses the URI and passes the
host and port. As a result, extra information that may be specified in
URI parameters is lost. This prevents the user from connecting to the
binary console using the SSL transport because to use the SSL transport
the user must specify transport=ssl URI parameter.

Needed for tarantool/tarantool-ee#567

NO_DOC=no visible changes in CE
NO_TEST=no visible changes in CE
NO_CHANGELOG=no visible changes in CE

(cherry picked from commit 33e72567)

During building an index in background, some transaction can perform a dml
request that affects space size (e.g. a replace), but the size will remain
the same, because bsize is moved from the old space to the new space in
memtx_space_prepare_alter() prior to space_execute_dml(). Fix this issue by
calling space_finish_alter() in alter_space_do().
In fact, this patch partially reverts commit 9ec3b1a4 ("alter: zap
space_vtab::commit_alter").

NO_DOC=bugfix

Closes #9247

(cherry picked from commit 54a42186)

These tests fail after the commit [1] has been added to the Luatest:

- app-luatest/gh_8083_fatal_signal_handler_test.lua
- app-luatest/gh_8445_crash_during_crash_report_test.lua
- box-luatest/gh_7434_yield_in_on_shutdown_trigger_test.lua

The issue is due to lack of necessary directories:

    sh: 1: cd: can't cd to /tmp/t/001_app-luatest/server-XXX

Just update tests on the simple `fio` module instead `luatest.server`.

[1] tarantool/luatest@7d1358c

NO_CHANGELOG=internal
NO_DOC=internal

(cherry picked from commit 23b61351)

Bump test-run to new version with the following improvements:

- luatest: bump luatest to 0.5.7-48-g18859f6 [1]
- Adapt use luatest with new --no-clean option [2]
- luatest: bump luatest to 0.5.7-49-g9c7710e [3]

[1] tarantool/test-run@aa3b34d
[2] tarantool/test-run@8ebb3aa
[3] tarantool/test-run@82542d3

NO_DOC=test
NO_TEST=test
NO_CHANGELOG=test

(cherry picked from commit f4bc53e8)

The test start to fail in CI on osx_debug (x86_64) workflow

```
[033]  	*** test_buffer_foreach_copy_number ***
[033] -ok 13 - prbuf(size=256, payload=16, iterations=16) has been validated
[033] -ok 14 - prbuf(size=256, payload=16, iterations=32) has been validated
[033] -ok 15 - prbuf(size=256, payload=16, iterations=64) has been validated
[033] +ok 13 - prbuf(size=256, payload=4294967312, iterations=16) has been validated
[033] +ok 14 - prbuf(size=256, payload=4294967312, iterations=32) has been validated
[033] +ok 15 - prbuf(size=256, payload=4294967312, iterations=64) has been validated
[033]  	*** test_buffer_foreach_copy_number: done ***
```

NO_CHANGELOG=test fix
NO_DOC=test fix

(cherry picked from commit 4a868563)