Tarantool supports two console protocols: text and binary. The binary
protocol is implemented with IPROTO EVAL request so the console module
reuses the net.box module to establish and maintain a binary connection.
Currently, instead of passing the original URI specified by the user to
net.box.connect as is, the console module parses the URI and passes the
host and port. As a result, extra information that may be specified in
URI parameters is lost. This prevents the user from connecting to the
binary console using the SSL transport because to use the SSL transport
the user must specify transport=ssl URI parameter.

Needed for tarantool/tarantool-ee#567

NO_DOC=no visible changes in CE
NO_TEST=no visible changes in CE
NO_CHANGELOG=no visible changes in CE

User password is stored in a system space is a form of hash when
'chap-sha1' auth type is set, and in a form of hash with salt when
'pap-sha256' is set.

Now, if a user is set inside config, and the current auth type is
different from the type the users password is stored in, the password
hash will be regenerated.

Part of #8967

NO_DOC=documentation request will be filed manually for the whole
       credentials

With #8906 the object types mentioned above were introduced. They control
access to code execution over IPROTO.
This patch adds such object types support to credentials applier. Now
'execute' can be granted to a user or role for 'lua_eval', 'lua_call' and
'sql'. Note that similar to 'universe', objects can't be specified in
the config, only 'all' is allowed.

Part of #8967

NO_DOC=documentation request will be filed manually for the whole
       credentials

The new parameters will be implemented in Tarantool Enterprise Edition.
This commit just adds configuration stubs.

Needed for tarantool/tarantool-ee#502
Needed for tarantool/tarantool-ee#503

NO_DOC=stubs for enterprise edition
NO_CHANGELOG=stubs for enterprise edition

Let's obtain box.cfg parameters from audit_log_init with cfg_get instead
of passing them as arguments, like we do with the security module, for
example. This simplifies addition of new audit log parameters.

Needed for tarantool/tarantool-ee#502
Needed for tarantool/tarantool-ee#503

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

During building an index in background, some transaction can perform a dml
request that affects space size (e.g. a replace), but the size will remain
the same, because bsize is moved from the old space to the new space in
memtx_space_prepare_alter() prior to space_execute_dml(). Fix this issue by
calling space_finish_alter() in alter_space_do().
In fact, this patch partially reverts commit 9ec3b1a4 ("alter: zap
space_vtab::commit_alter").

NO_DOC=bugfix

Closes #9247

Consider the following example:
```
tarantool -e "box.cfg{} require('console').start()"
```
When a local console is exited by pressing Ctrl+D, Tarantool seemingly
freezes - console stops to work, typed characters are not echoed. But
the event loop is not stopped because there are background fibers running.
This patch adds a message that Ctrl+C should be pressed in such a case.

Closes #7017

NO_DOC=minor

Attaching triggers to space id instead of space object is a significant
pitfall. The users who haven't discovered new triggers may not expect
the triggers of a dropped space will be fired by a new one. So let's
drop triggers that were set with old API along with the space.

All the tests, changed because of described above breaking change, are
restored.

Closes #9223

NO_DOC=later

The patch provides event triggers with flags and introduces the first
one - EVENT_TRIGGER_IS_TEMPORARY. When the new
event_remove_temporary_triggers method is called, all the temporary
triggers are removed from the event.

Part of #9223

NO_CHANGELOG=internal
NO_DOC=internal

This patch also includes:
  - misc trivial fixes for ASAN discovered issues
  - minor adaptations for ASAN friendly allocators

Closes #7327

NO_DOC=internal
NO_CHANGELOG=internal

With new ASAN-friendly small implementation unit/fiber_stack.c test
start to fail. The issue is leak sanitizer reports a leak. This is an
expected leak of test for mprotect failure on fiber stack destruction.
Let's tell sanitizer to ignore this case.

By the way let's drop test code for temporary redirecting stderr. It is
outdated as test is TAP-compatible. It was a PITA as due to this
redirection there was no leak report only error exit code.

Part of #7327

NO_CHANGELOG=internal
NO_DOC=internal

ASAN-friendly implementation poisons memory after allocation with
ibuf_alloc so we need to fix existing places in code where we access
memory after allocation.

Part of ibuf implementation is inline functions in headers. Thus ibuf
implementation in Lua reimplement this parts. We add poison to these
inline functions in ASAN-friedly implementation so we need add same poison
in Lua implementation.

Part of #7327

NO_CHANGELOG=internal
NO_DOC=internal

We are going to include generated small_config.h into small allocator
headers (currently it is only included in small source files).
core/memory.h depends on small headers and salad/heap.h depends on
core/memory.h. As a result we need to provide a way for salad/heap.h
users to find small_config.h header.

Instead let's drop dependency from core/memory.h as we only use it for
typeof definition.

Part of #7327

NO_CHANGELOG=code cleanup
NO_DOC=code cleanup

If leak sanitizer reaches the memory protected from read with mprotect
it exhibits all sorts of odd behaviour. It can hang, can crash, can
return errors with no leak backtraces.

We use mprotect to create guard zones at the end of fiber stack so if
stack is overflowed we get a signal and crash. We take protection off
when fiber is destroyed. Unfortunately we do not destroy cords (and its
fibers) which cancelled through cord_cancel_and_join. This is going to
be addressed in patch for issue #8423 ("Get rid of pthread_cancel()").
Until that moment let's disable protection for ASAN builds.

Note that we did not hit this behaviour before because LSAN only scans
memory allocated using malloc and regular slab cache uses mmap to get
memory.

Part of #7327

NO_CHANGELOG=internal
NO_DOC=internal

Regularly fiber stack slab is page aligned. So upper stack border is
page aligned too when stack grows down. But with ASAN friendly slab
cache implementation this border is not page aligned. As a result
madvise call on stack may zero memory beyond stack slab which will cause
heap corruption. In debug build corruption is detected by assertion:

NO_WRAP
 >  Fatal glibc error: malloc.c:2593 (sysmalloc): assertion failed: (old_top
 >  == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >=
 >  MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize
 >  - 1)) == 0)
NO_WRAP

Interestingly enough the issue can not be investigated using ASAN. The
memory is zeroed by kernel code which is not instrumented so it is
invisible for sanitizer.

Looks like non-ASAN builds are not affected. Even if stack_size is
not page aligned the slab allocated for stack is page aligned. Thus
memory zeroing will be inside the slab and there will be no memory
corruption.

Also when stack grows up lower stack border in not aligned even with
regular small implementation. So madvise call will fail with EINVAL as
it is required that start address is page aligned. We ignore the error
though. Let's fix this issue too while we at it.

Let's introduce fiber_madvise_aligned to align madvise range with proper
direction before calling madvise(2). To justify its usage note that
besides fixing the issues described above, in case of stack growing down
fiber->stack is page aligned and in case of stack growing up
fiber->stack + fiber->stack_size is page aligned.

Part of #7327

NO_TEST=tested by ASAN (debug build)
NO_CHANGELOG=has effect only with newly introduced ASAN friendly slab cache
NO_DOC=has effect only with newly introduced ASAN friendly slab cache

The unpoison was added in the initial commit 1.7.2-68-gafd229393 that
supported ASAN. It is not clear why do we need it as we don't poison
stack memory manually.

Part of #7327

NO_TEST=removing unfunctional code
NO_CHANGELOG=removing unfunctional code
NO_DOC=removing unfunctional code

The reason check is different for ASAN and regular versions of obuf.

Part of #7327

NO_DOC=internal
NO_CHANGELOG=internal
NO_TEST=<will be tested by asan-debug CI>

The commit moves on_shutdown triggers to the trigger registry. The
triggers set by C API and internal triggers remain unchanged - only Lua
user triggers are affected.

Changelog entry of #8657 is populated with box.ctl triggers and is
slightly improved.

Closes #8657

NO_DOC=later

Function trigger_fiber_run is used only for on_shutdown triggers and
uses internal structure run_list. This structure is another list but all
the triggers are popped from run_list instead of iteration because this
approach is safe when triggers are deleted from the list that is being run.
Also, new triggers are not inserted to run_list.

Since we are running only on_shutdown triggers, which won't be used after
they are fired, we can move all the triggers to an internal trigger
list (so that no new triggers will appear) and pop them instead of
iteration. So let's remove function trigger_fiber_run and run on_shutdown
core triggers in a new special function. Later, this new function will run
triggers from on_shutdown event as well.

Part of #8657

NO_TEST=no behavior changes
NO_CHANGELOG=later
NO_DOC=later

The patch moves all triggers from box.ctl to module trigger instead of
on_shutdown trigger - they are run in separate fibers, which makes it more
difficult to move it to the event subsystem, so it will be moved there
in a separate commit.

Also, box_raft_on_broadcast triggers are renamed to box_raft_on_election.
Despite they are fired on broadcast, the only place they are installed
along the whole tarantool organization is box.ctl.on_election.

NO_DOC=later
NO_CHANGELOG=later

Part of #8657

Before this patch, if an index was created due to a column's UNIQUE
constraint or a column's PRIMARY KEY constraint before adding a
collation, and if the column's fieldno was not equal to the index's
position in space->index, the collation would not be assigned to the
index.

Also, this patch fixes an assertion in debug build for the case when an
index with more that one field was created before a collation was added.

Closes #9229

NO_DOC=bugfix

Because of inlining rules some parts of comparators aren't optimized
properly by the gcc compiler, this causes a regression introduced by
the sort order implementation.

This patch introduces inline hints for the compiler in order to
mitigate the regression.

perf/tuple.cc test results (RelWithDebInfo, time in nanoseconds):

                             Tiger Lake

gcc 11.4.0:

                             Base    After #8915      Patched
       tuple_tuple_compare   40.5    41.5 (+2.5%)     39.4 (-2.7%)
  tuple_tuple_compare_hint   43.0    33.5 (-22.1%)    35.9 (-16.5%)

clang 14.0.0:

                             Base    After #8915      Patched
       tuple_tuple_compare   25.7    25.1 (-2.3%)     25.7
  tuple_tuple_compare_hint   33.1    32.5 (-1.8%)     33.1

                                Zen 3

gcc 11.4.0:

                             Base    After #8915      Patched
       tuple_tuple_compare   18.9    22.85 (+20.9%)   19.4 (+2.6%)
  tuple_tuple_compare_hint   24.25   22.95 (-5.4%)    23.5 (-3.1%)

clang 14.0.0:

                             Base    After #8915      Patched
       tuple_tuple_compare   17.3    17.0 (-1.7%)     17.0 (-1.7%)
  tuple_tuple_compare_hint   20.3    20.1 (-1.0%)     20.1 (-1.0%)

Closes #9216

NO_DOC=no code modification
NO_TEST=no code modification
NO_CHANGELOG=no code modification

The new option is backed by `box.cfg.secure_erasing`. It is available
only in Enterprise Edition builds.

Needed for tarantool/tarantool-ee#540

NO_DOC=will be added to Enterprise Edition
NO_CHANGELOG=will be added to Enterprise Edition

We call xdir_collect_inprogress() at startup to clean up the xlog
directory of files left from the previous run. Let's rename it to
xdir_remove_temporary_files() and make it delete all files for which
the new callback function xlog_file_is_temporary() returns true. By
default, the callback returns true only for .inprogress files but it
can be overridden to make xdir_remove_temporary_files() delete other
kinds of files. This is required for thorough file deletion.

Needed for tarantool/tarantool-ee#540

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This commit introduces the xlog_remove_file() function that removes
a file by name and logs the error on failure. We use this function
everywhere we delete xlog files so that there's a single place where we
call unlink(). We also factor out the core functionality to a callback
function that can be overridden. This will help us implement thorough
file deletion.

Needed for tarantool/tarantool-ee#540

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Currently, vy_run_remove_files calls coio several times under the hood -
once per each run file and data directory. Apart from being inefficient,
this also prevents us from adding some extra logic for thorough file
deletion. So let's perform all the operations in a single coio call.

Needed for tarantool/tarantool-ee#540

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The structure is no longer used, so it is dropped.

Follow-up #9112

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This patch introduces variations of DROP CONSTRAINT with a declared
constraint type.

Closes #9112

@TarantoolBot document
Title: upgrade of DROP CONSTRAINT

Now, instead of just `ALTER TABLE table DROP CONSTRAINT constraint;` we
have 8 operator variants:
1) Statement to drop PRIMARY KEY, UNIQUE, tuple FOREIGN NEY or tuple
CHECK constraints:
```
ALTER TABLE tab_name DROP CONSTRAINT constr_name;
```

This statement cannot drop a constraint if `constr_name` matches
more than one constraint.

2) Statement to drop field FOREIGN NEY or field CHECK constraints:
```
ALTER TABLE tab_name DROP CONSTRAINT field_name.constr_name;
```

This statement cannot drop a constraint if `constr_name` matches
more than one constraint for the `field_name` field.

3) Statement to drop PRIMARY KEY constraint:
```
ALTER TABLE tab_name DROP CONSTRAINT constr_name PRIMARY KEY;
```

4) Statement to drop UNIQUE constraint:
```
ALTER TABLE tab_name DROP CONSTRAINT constr_name UNIQUE;
```

5) Statement to drop tuple FOREIGN KEY constraint:
```
ALTER TABLE tab_name DROP CONSTRAINT constr_name FOREIGN KEY;
```

6) Statement to drop tuple CHECK constraint:
```
ALTER TABLE tab_name DROP CONSTRAINT constr_name CHECK;
```

7) Statement to drop field FOREIGN KEY constraint:
```
ALTER TABLE tab_name DROP CONSTRAINT field_name.constr_name FOREIGN KEY;
```

8) Statement to drop field CHECK constraint:
```
ALTER TABLE tab_name DROP CONSTRAINT field_name.constr_name CHECK;
```

This patch prohibits DROP CONSTRAINT if more than one constraint matches
a given name.

Part of #9112

NO_DOC=will be added later
NO_CHANGELOG=will be added later

This patch introduces "ALTER TABLE table_name DROP CONSTRAINT
field_name.constraint_name" which can be used to drop field constraints.
Also, after this patch, field constraints cannot be dropped using
"ALTER TABLE table_name DROP CONSTRAINT constraint_name;".

Part of #9112

NO_DOC=will be added later
NO_CHANGELOG=will be added later

This patch replaces region_alloc() by xregion_alloc() in
mp_vformat_on_region().

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Introducing ASAN-friendly small allocators slows down execution notably.
As a result several tests start to fail due to hitting max slice limit.
I guess we don't interested if fibers in ASAN build grabs control for
too long as we have release build run in CI anyway.

Some tests set max slice limit explicitly to some large value thus
overwriting default infinity value for ASAN. Unfortunately this large
value is not large enough for ASAN. Let's set some really large value.

Part of #7327

NO_CHANGELOG=internal
NO_DOC=internal

The tuple format and access subsystems have static variables holding their
states which don't get reset during cleanup: initialize them explicitly in
`*_init` functions — that way we can re-initialize these subsystems
multiple times (e.g., when setting up and tearing down benchmarks). Opted
for initializing them in ``*_init` functions rather than resetting them in
`*_free` functions for logical consistency.

Needed for #6964

NO_CHANGELOG=cleanup fix
NO_DOC=cleanup fix
NO_TEST=cleanup fix

Force recovery first tries to collect all rows of a transaction into a
single list, and only then applies those rows.

The problem was that it collected rows based on the row replica_id. For
local rows replica_id is set to 0, but actually such rows can be part
of a transaction coming from any instance.

Fix recovery of such rows

Follow-up #8746
Follow-up #7932

NO_DOC=bugfix
NO_CHANGELOG=the broken behaviour couldn't be seen due to bug #8746

In order to preserve transaction boundaries over replication, Tarantool
writes a global NOP row after the last transaction row, if this row
happens to be local. This is done to make sure that the is_commit flag,
which is set only in the last transaction row, reaches the replica. This
wouldn't happen if the last row was local.

This workaround works fine for transactions completely authored by one
instance: when both global and local rows come from operations of a
single master.

However, it's possible to append local rows to a remote master's
transaction on a replica. For example, one can use on_replace triggers
to write to replica's local space on each new transaction coming from
master.

In this case essentially a global NOP entry is added at the end of a
remote master's transaction. This leads to several problems.

First of all, this bumps replica's LSN, which is counter-intuitive,
given that the replica might even be read-only. Besides, in a star
topology this leads to master being unable to connect to the replica
later on due to their vclocks becoming incompatible.

Secondly, even if replication channel between master and replica is
bidirectional, it creates a new row which should be replicated from
replica to master, but at the same time is the last row of the master's
transaction. Once master receives this row, it breaks its connection to
replica due to transaction boundary violation (the last row of the
transaction is received without its beginning).

Adding a NOP row became extraneous since the previous commit, which made
relay find transaction boundaries by itself.

Closes #8958

NO_DOC=bugfix

Some time ago we started writing transaction boundaries to WAL and
respecting them in the replication stream: replicas wait for a full
transaction receipt before applying it.

However, during all these changes relay remained transaction-agnostic:
it simply read single rows from WAL and sent them over to the receiver.

This lead to a handful of ugly crutches: for example, tsn is not always
equal to the lsn of the first global row of the transaction: if the
first row is local, tsn is deduced from the first global row of the
transaction.

Also a dummy NOP was appended to the end of a transaction ending by a
local row, so that is_commit flag wasn't lost by the replication.

Let's make relay read a full transaction, filter out all the unnecessary
rows, set the transaction boundaries accordingly and then send the
transaction at once.

Since in relay a single fiber sends data to the remote peer, there is no
chance for a heartbeat to get in between rows of a single transaction:
they're all sent at once. Hence the deletion of a corresponding guard
`relay->is_sending_tx`.

Prerequisite #8958

NO_DOC=internal change
NO_CHANGELOG=internal change
NO_TEST=covered by existing tests

Transaction boundaries were not updated correctly for transactions in
which local space writes were made from a replication trigger. Existing
transaction boundaries and row flags from the master were written as is
on the replica. Actually, the replica should recalculate transaction
boundaries and even WAIT_SYNC/WAIT_ACK flags.

Transaction boundaries should be recalculated when a replica appends a
local write at the end of the master's transaction, and
WAIT_SYNC/WAIT_ACK should be overwritten when nopifying synchronous
transactions coming from an old term.

The latter fix has uncovered the bug in skipping outdated synchronous
transactions: if one replica replaces a transaction from an old term
with NOPs and then passes that transaction to the other replica, the
other replica raises a split brain error. It believes the NOPs are an
async transaction form an old term. This worked before the fix, because
the rows were written with the original WAIT_ACK = true bit. Now this
is fixed properly: we allow fully NOP async tranasctions from the old
term.

Closes #8746

NO_DOC=bugfix
NO_CHANGELOG=covered by the next commit

If non-terminal symbol is referenced in C code then destructor for
expression is not called. Thus we don't need to duplicate. Otherwise we
got a memory leak.

See https://www.sqlite.org/cgi/src/doc/trunk/doc/lemon.html#destructor

Close #9159

NO_DOC=bugfix
NO_TEST=tested by debug ASAN CI (to be turned on)

Part of https://github.com/tarantool/tarantool-ee/issues/564

NO_DOC=The documentation request is to be added as part of Tarantool
       Enterprise Edition patchset.
NO_CHANGELOG=see NO_DOC
NO_TEST=To be tested in Tarantool Enterprise Edition.

The new 'supervised' failover mode uses an external failover agent to
make decisions regarding leadership in a replicaset. This is a feature
of Tarantool Enterprise Edition.

This commit adds a new `replication.failover` value `supervised`, adds
corresponding instance startup code and necessary configuration
validation.

The most interesting part is how to start all the instances in RO, but
if the replicaset is not bootstrapped yet, start one instance in RW to
perform the replicaset bootstrap. See comments in applier/box_cfg.lua
for details.

Part of https://github.com/tarantool/tarantool-ee/issues/564

NO_DOC=The documentation request is to be added as part of Tarantool
       Enterprise Edition patchset.
NO_CHANGELOG=see NO_DOC
NO_TEST=The overall logic of this mode is to be tested in Tarantool
        Enterprise Edition.