Each story has two ends: the beginning and the end. For each
transaction both ends of a story could be visible or not.

Now there's a function that checks visibility of both ends of
a story. It can distinguish three cases: both ends are visible,
both ends are invisible, and the beginning is visible while the
end is not. The function returns true in the first and the last
cases; the actual case is clarified with an additional function
argument - visible_tuple, which is set to null in one of the
cases..

Let's make two different functions for checking visibility of
the beginning and the end of a story. Actually that is simple
split of function into two parts. The visible_tuple argument
will no longer be needed.

No logical changes.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

There's no harm but also no sense in it.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Hide structures and functions that are not required for API.

No logical changes.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Option memtx_tx_manager_use_mvcc_engine changes the behavior of
transaction execution workflow. Usually that is implemented as
direct check of memtx_tx_manager_use_mvcc_engine. But there are
places in the code that rely on the fact that some pointers are
set to not null if the engine is enabled. That's a bit confusing.

Let's always check for memtx_tx_manager_use_mvcc_engine option
when it's needed to determine which workflow must be executed.

Note that checking of memtx_tx_manager_use_mvcc_engine option is
more correct: in case of delete of nothing (delete statement when
a tuple was not found by given key) all the pointers including
old_tuple and new_tuple are null, while logically we still need
to use mvcc execution workflow. Note also that in this case the
mvcc engine does (and must do) almost nothing, so there was no
bug in the previous behaviour.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

No logical changes.

Part of #8122

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Since the fiber function is not expected to return (i.e., the inline
assembly is not expected to return), remove the dummy memory operand used
to indicate that the memory pointed to by the inline assembly pointer
operand is clobbered.

Closes #8125

NO_DOC=build diagnostic
NO_CHANGELOG=build diagnostic
NO_TEST=build diagnostic

Before the patch the LuaJIT integration workflow was dispatched to the
runner with the name given via <inputs.host> parameter. Unfortunately,
as a result of runners renaming we can't continue to dispatch the
workflow this way.

As a result of the patch there are two new workflow parameters:
<inputs.arch> to pass the host architecture name (i.e. x86_64 or ARM64)
and <inputs.os> to pass the OS family name (either Linux or macOS).
Considering two values we can choose the proper runner in LuaJIT
integration workflow. Besides, this change bring LuaJIT CI closer to
matrix usage for its integration workflow.

All three workflow parameters are not obligatory for now to avoid
tarantool/luajit CI break on both long-term and working branches. When
all branches are rebased on the new approach, <inputs.host> parameter
will be removed and both <inputs.arch> and <inputs.os> will become
obligatory.

Moreover, the new 'regular' label is also added to <runs-on> list, since
the new "lightweight" runners have been introduced to ghacts-shared-*
pool. There are a couple of LuaJIT tests that requires more memory than
provided by "lightweight" runners, so only "regular" ones need to be
chosen for LuaJIT integration testing.

Last but not least: attentive reader might notice there are strange
values used as a default for <inputs.host> as well as <inputs.arch> and
<inputs.os>. This is ugly hack required for the transition period, since
one can't use empty string or unknown label name within <runs-on> label
list. Hence 'self-hosted' looks like the most robust option for both old
and new behaviours.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Reviewed-by: Yaroslav Lobankov <y.lobankov@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

The status isn't used anywhere - to set the proper error when an aborted
transaction is attempted to be used, we check out transaction flags
(TXN_IS_CONFLICTED, TXN_IS_ABORTED_BY_YIELD, TXN_IS_ABORTED_BY_TIMEOUT).
Let's use TXN_ABORTED instead.

While we are at it, also set the transaction status to TXN_ABORTED when
a transaction is aborted by yield or timeout and use it instead of
checking flags where appropriate, since it's more convenient.

Follow-up #8123

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

We fail write statements if the current transaction was aborted by yield
or timeout. We should fail read-only statements in this case, as well.
Note, we already fail read-only statements if the current transaction
was aborted by conflict.

Closes #8123

NO_DOC=bug fix

Fixed pthread-related CMake checks. The checks code is built with
`-pedantic-errors` and it leads to errors of the following kind on
clang 15:

```
<...>/CMakeFiles/CMakeScratch/TryCompile-78KaOK/src.c:4:17: error: a
    function declaration without a prototype is deprecated in all
    versions of C [-Werror,-Wstrict-prototypes]
        int main() { pthread_setname_np(pthread_self(), ""); }
                ^
                 void
```

Fixed a warning in the SQL code (it's an error in Debug build):

```
<...>/src/box/sql/vdbeaux.c:170:13: error: variable 'n' set but not used
    [-Werror,-Wunused-but-set-variable]
        static int n = 0;
```

Fixed several warnings from lemon.c of the following kind:

```
<...>/extra/lemon.c:173:6: warning: a function declaration without a
    prototype is deprecated in all versions of C and is treated as a
    zero-parameter prototype in C2x, conflicting with a subsequent
    definition [-Wdeprecated-non-prototype]
void FindRulePrecedences();
     ^
<...>/extra/lemon.c:766:6: note: conflicting prototype is here
void FindRulePrecedences(struct lemon *xp)
```

See also https://github.com/tarantool/small/issues/57

Fixes #8110

NO_DOC=build fix
NO_TEST=build fix

Bump the small submodule with the following commit:

* https://github.com/tarantool/small/commit/5c3a0880ae4659cb662085563ff0b9eee40565fe

See https://github.com/tarantool/small/issues/57
Part of #8110

NO_DOC=build fix
NO_TEST=build fix
NO_CHANGELOG=to be added in a commit that fixes #8110

Add `IPROTO_UNKNOWN` command code for overriding the unknown request
handler.

Change request type variable types related to IPROTO to `uint32_t`.

Add request handler hash table to transaction thread and request handler
set to IPROTO threads for storing overridden request handlers: TX thread
notifies IPROTO threads about overridden request handlers using IPROTO
configuration message.

If a given request handler is overridden, the IPROTO thread does not
preprocess it and sends the package immediately over a dedicated route. If
later it is necessary to fallback to the system handler, the message
decoding and dispatching is done in the TX thread.

Add new `box.iproto.override` method to Lua and `box_iproto_override` to C
API, which allow setting IPROTO request handler callbacks.

Closes #7901

@TarantoolBot document
Title: Document overriding IPROTO request handlers feature
For the API description and usage examples, see:
* [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#0f84694523214c0e9bf2f3d75cccace4);
* tarantool/tarantool#7901.

Factor out the logic of IPROTO message decoding into a separate function.

Needed for #7901

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring

If the output buffer savepoint passed to `tx_end_msg` is the same as the
connection's current writing position, it means the response is empty, so
it makes no sense to record it.

Needed for #7901

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Add an optional dictionary field to MsgPack object which can be used during
indexing for aliasing string keys.

Needed for #7901

NO_CHANGELOG=<internal feature>
NO_DOC=<internal feature>

* cmake: introduce CheckUnwindTables helper
* x64/LJ_GC64: Fix type-check-only variant of SLOAD.
* LJ_GC64: Fix ir_khash for non-string GCobj.
* gdb: support full-range 64-bit lightuserdata

Relates to #6481
Part of #7230

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

Use an array to convert a hex digit into a number instead of
`isdigit()` and `tolower()`.

Eliminate the `isxdigit()` check by reusing the same array lookup.

Results on perf/lua/uri_escape_unescape.lua show 2.4x average boost
(from 2.1x to 2.5x).

Noise in the results: 4% in baseline, 11% in the new version (the
maximal difference between min and max results). The average
linear/standard deviation is within 3%.

Measured on i7-10510U, acpi-cpufreq driver (no intel_pstate),
no TurboBoost, userspace governor, 1.80GHz, HyperThreading. The workload
is bound to a single logical core using `taskset`.

NO_DOC=no behavior change
NO_TEST=no behavior change
NO_CHANGELOG=the function is new and not yet released

Change runner label 'macos-11' to 'macos-11-self-hosted' and 'macos-12'
to 'macos-12-self-hosted' to distinguish between self-hosted and
GitHub-hosted runners.
We want to use only self-hosted macOS runners because of test-run problems
with python3.11 on GitHub-hosted macOS runners.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Closes #8043

NO_DOC=bug fix

The coverage workflow is a part of the default testing, so there is no
sense to run this workflow when the 'full-ci' label is set. Moreover, it
cancelled the run of the default testing and started the run of the full
testing due to workflow `concurrency`.

By default, a workflow only runs when a `pull_request` event's activity
type is `opened`, `synchronize`, or `reopened`. That's why there is no
sense in the following construction:

  pull_request:
    types: [ opened, reopened, synchronize ]

So just removed the line related to event's activity type.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Add workflow for integration testing of ddl
and crud modules.

Resolves tarantool/tarantool#6619
Resolves tarantool/tarantool#6620

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

This patch removes unused variables that were not caught by the compiler
due to MAYBE_UNUSED or conversion to void.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Also pretty printer was introduced for struct tuple, so it can be
printed with 'p' command as well. 'tt-tuple' command allows to specify
some additional options (see 'help tt-tuple')

Closes #7729

NO_DOC=gdb extension
NO_CHANGELOG=gdb extension
NO_TEST=gdb extension

Supported extensions:
- decimal
- uuid
- datetime
- error
- compression (only shows compression type and raw/compressed data size)
- interval

This functionality is similar to 'mp_str' function, but 'mp_str' can be
invoked from gdb only when a process is alive, so to be able to print
MsgPack during post-mortem analysis of core dump, 'mp_str' and all
subsequent functions that decode and convert MsgPack into human-readable
string were adapted to Python and put into the extension.

In order to simplify implementation and maintenance of the gdb-extension
implementation of the most of functions is kept as close to possible to
their 'C'-counterparts (original 'C'-name can be found as a comment at
the end of 'def' string)

Part of #7729

NO_DOC=gdb extension
NO_CHANGELOG=gdb extension
NO_TEST=gdb extension

This patch introduces new keyword SEQSCAN and new restrictions on
SELECTs. These restrictions are disabled by default.

Closes #7747

@TarantoolBot document
Title: SEQSCAN

Now scanning SELECT will not run and will throw an error if the new
SEQSCAN keyword is not used for scanned spaces. This change only affects
SELECT and does not affect UPDATE and DELETE. A SELECT is recognized as
a scanning SELECT if `EXPLAIN QUERY PLAN SELECT ...` indicates that the
SELECT `scans` rather than `searches`.

For example, if we have spaces created with these queries:
```
CREATE TABLE t(i INT PRIMARY KEY, a INT);
CREATE TABLE s(i INT PRIMARY KEY, a INT);
```

Then these queries will throw an error:
```
SELECT * FROM t;
SELECT * FROM t WHERE a > 1;
SELECT * FROM t WHERE i + 1 = 5;
SELECT * FROM t, s;
SELECT * FROM t JOIN s;
```

And these will not:
```
SELECT * FROM t WHERE i > 1;
SELECT * FROM SEQSCAN t;
SELECT * FROM SEQSCAN t WHERE i + 1 = 5;
SELECT * FROM SEQSCAN t, SEQSCAN s;
SELECT * FROM SEQSCAN t JOIN SEQSCAN s;
```

Scanning can be allowed or disallowed by default. To do this, a new
session setting is introduced: `sql_seq_scan`. The default value for
setting is `true`, i.e. scanning is allowed. When set to `false`, the
scanning SELECTs will throw a `scanning is not allowed` error.

This commit adds a few configuration options, function stubs, and error
codes that will be used to perform extra security checks in EE:

 * box.cfg.auth_delay. Type: double. Default: 0. Unit: seconds.

   Description: If authentication of a user fails, the next
   authentication attempt for the same user will fail with
   ER_AUTH_DELAY error if called before box.cfg.auth_delay passes.

   Implementation: Error will be raised by security_check_auth_pre()
   called by authenticate() right before checking the challenged
   password. Authentication failures will be accounted per user in
   session_on_auth trigger.

 - box.cfg.disable_guest. Type: boolean. Default: false.

   Description: If set, an attempt to perform any request except
   'auth', 'ping', 'id', or 'vote' over iproto without authentication
   or authenticated as guest will raise ER_AUTH_REQUIRED error.

   Implementation: Error will be raised by security_check_session()
   called by tx_check_msg(), which in turn is called before starting
   to process any message received over iproto.

 - box.cfg.password_lifetime_days. Type: number. Default: 0. Unit: days.

   Description: If > 0, an attempt to authenticate as a user that hasn't
   reset the password for more than box.cfg.password_lifetime_days will
   fail with ER_PASSWORD_EXPIRED error.

   Implementation: Error will be raised by security_check_auth_post()
   called by authenticate() right after successfully authenticating
   the user. Note, we can't raise the error in security_check_auth_pre()
   because that would enable user enumeration.

All the new options are dynamic. The option values will be stored and
used in C code so we'll have to define a configuration callback for them
in EE: box.internal.cfg_set_security. Also note that since the new
options configure authentication behavior, they should be set before
the box port is opened: we'll do that in security_cfg() called by
box_storage_init().

Needed for https://github.com/tarantool/tarantool-ee/issues/299
Needed for https://github.com/tarantool/tarantool-ee/issues/300
Needed for https://github.com/tarantool/tarantool-ee/issues/301

NO_DOC=ee
NO_TEST=ee
NO_CHANGELOG=ee

In contrast to all other request handlers, replication request handlers
don't check the schema version. Strictly speaking, this is incorrect -
if it's specified it must always be checked. It was fine, because we
don't set the schema version in replication requests. However, in the
future we're planning to add more checks in tx_check_msg, which will
have to be performed for all requests, including replication. So let's
call tx_check_msg before processing replication requests - it shouldn't
hurt to check the schema version if available.

Needed for https://github.com/tarantool/tarantool-ee/issues/301

NO_DOC=ee
NO_TEST=ee
NO_CHANGELOG=ee

After accepting an iproto message in tx, we call tx_check_schema to
check if the schema version supplied by the user matches the actual
one. If it doesn't, we bail out early with ER_WRONG_SCHEMA_VERSION.

We need to add more checks that should be done before continuing with
processing a message in tx. tx_check_schema looks like a perfect place
for it. Let's rename it to tx_check_msg and pass an accepted message
instead of the schema version to it.

Needed for https://github.com/tarantool/tarantool-ee/issues/301

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Closes #7931

@TarantoolBot document
Title: Document a percent-encoding of params passed to http client

The HTTP client has `params` option, where a user may provide a table of
query parameters (added in #6832). Those parameters are encoded into a
`?foo=bar&tweedledoo=tweedledee` string verbatim. If a name or a value of
a query parameter contains `&`, `=` (or any another symbol with specific
meaning in the URI query component), the query may be interpreted
incorrectly by a server. Now key and values passed in a table as
`params` option are percent-encoded and then encoded to a query string.
This will be made automatically. Percent-encoding depends on used HTTP
method: with `GET`, `HEAD` and `DELETE` parameters `uri.QUERY_PART`
are used and with other HTTP method `uri.FORM_URLENCODED` is used.

Commit "uri: encode table with http params to a string" (b31aec89)
introduced two functions `params()` and `encode_kv()` that encodes a
table with http key-value parameters to a http query string. However,
keys and values could be interpreted wrong by http server if they
contains reserved symbols that have special meaning for http server.

Patch adds escaping for key-values before encoding params to query
string.

Needed for #7931

NO_CHANGELOG=internal
NO_DOC=internal

Patch replaces encoding and decoding functions written in Lua with
functions implemented in C.

Performance of Lua implementation (before the patch):

```
uri.escape   152.37  runs/sec
uri.unescape 263.44  runs/sec
```

Performance of C implementation (after the patch):

```
uri.escape   4983.03  runs/sec
uri.unescape 4197.19  runs/sec
```

Follows up #3682

NO_CHANGELOG=see previous commit
NO_DOC=see previous commit

Co-authored-by: Alexander Turenko <alexander.turenko@tarantool.org>

Added a simple benchmark for URI escape/unescape.

Part of #3682

NO_DOC=documentation is not required for performance test
NO_CHANGELOG=performance test
NO_TEST=performance test

Closes #3682

@TarantoolBot document
Title: Document a new functions to encode and decode parts of URI

New functions `uri.escape()` and `uri.unescape()` have been introduced.
First one allows to escape symbols to a string and second one to
unescape symbols to a string according to RFC 3986 [1].

Examples:

```
tarantool> uri.escape("тарантул")
---
- '%D1%82%D0%B0%D1%80%D0%B0%D0%BD%D1%82%D1%83%D0%BB'
...

tarantool> uri.unescape("%D1%82%D0%B0%D1%80%D0%B0%D0%BD%D1%82%D1%83%D0%BB")
---
- тарантул
...

```

`uri.escape()` accepts a string that will be encoded and optionally a
table with encoding options: string with unreserved symbols that will
*not* be encoded and boolean option that enables/disables encoding of a
space characters as '+'. By default `uri.escape()` uses a set of
unreserved symbols defined in RFC 3986 ("2.3. Unreserved Characters")
and encoding of space characters as '+' is disabled. Table with default
encoding options is defined as `uri.RFC3986`.

`uri.unescape()` accepts a string that will be decoded and optionally a
table with decoding options: string with unreserved symbols (these
symbols are actually unused by decoding function) and boolean option
that enables/disables decoding of '+' as a space character. Table with
default decoding options is defined as `uri.RFC3986`.

See detailed description in RFC "http: add percent-encoding/decoding of
query string in request" [2].

NO_WRAP
1. Uniform Resource Identifier (URI): Generic Syntax
   https://datatracker.ietf.org/doc/html/rfc3986
2. https://www.notion.so/tarantool/http-add-percent-encoding-decoding-of-query-string-in-request-76a2425a4c4744a1a72643527a4fe7f7
NO_WRAP

Add translation table for `box.iproto.key` constants encoding to simplify
packet assembly.

Add new `box.iproto.send` method to Lua and `box_iproto_send` function to C
API, which allow sending arbitrary IPROTO packets, using active IPROTO
sessions. Packets are sent asynchronously using Kharon.

Add `xregion_join` to the `xalloc` API.

Change gh-7894 test: instead of simply comparing `box.iproto` table to the
reference table, iterate over `box.iproto` and check that corresponding
non-{function, thread, userdata} type values exist in the reference table.

Closes #7897

@TarantoolBot document
Title: Document sending arbitrary IPROTO packets feature
For the API description and usage examples, see:
* [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#a2cc04da89d34fad8f8564c150cd9977);
* tarantool/tarantool#7897.

The translation table (and the future msgpack object dictionary) uses
string keys coming from Lua and hence using `lua_hash{string}` as the hash
function is beneficial.

Needed for #7897

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

`test_key_def_dup` uses `box_key_def_dump_parts` which allocates memory on
the fiber region, but the test does not clean it up: fix that by creating
a region savepoint and truncating the fiber region in the end of the test.

Needed for #7897

NO_CHANGELOG=<test fix>
NO_DOC=<test>

In some cases we need to convert a string to lower case (e.g., when
normalizing an upper-case constant): add helper functions that do this
in-place or by creating a copy of the original string.

Needed for #7897

NO_CHANGELOG=internal
NO_DOC=internal

Currently replicaset state machine tracking the number of connected,
loading and synced appliers may perform unnecessary decrementing of
their count. On debug version this may lead to assertion failure.
Here's the way it may happen:
  1. Any kind of exception occurs in applier thread and leads to
     invoking its destructor (applier_thread_data_destroy), which
     is set with scoped guard;
  2. Cbus call is made in order to remove the corresponding applier
     from the thread. According to the fact that cbus_call is
     synchronous, we yield, waiting for the result from the applier
     thread.
  3. During yielding user calls reconfiguration, which invokes
     replicaset_update. Old appliers are pruned: for every replica
     trigger on changing state machine counter is deleted after which
     we stop fiber and wait its join.
  4. If the first replica in replicaset_foreach is not the errored
     one and the errored fiber wakes up during yielding with
     fiber_join, then zero decrementing happens.

Let's clear the above mentioned triggers for all replicas at the
first place and only after that stop and join their applier fibers.

Closes #7590

NO_DOC=bugfix

Before the change there was an unexpected behavior when using
channel:close(), as it closed the channel entirely and discarded all
unread events.

This commit introduces graceful channel close option in tarantool.compat
(gh-7000) that allows to select new or old behavior.

With the new behavior `close()` marks channel as closed for writing.
Only when all events are extracted, the channel is closed entirely. If
there are no events in the channel, it is closed as usual.

Document that describes new API can be found on notion (private):
https://www.notion.so/fiber-channel-graceful-close-53b2788ed1f144598c4c0e1229c2eb69

Requires #7060
Requires #8007
Closes #7746
See also #7000

@TarantoolBot document
Title: new compat option fiber_channel_close_mode

New behavior is gracefully closing fiber channel by marking it
read-only, instead of destroying. Full API description can be found
on notion (private):
https://www.notion.so/tarantool/Fiber-channel-graceful-close-53b2788ed1f144598c4c0e1229c2eb69

A jiggle in tarantool's initialization code can lead to trace numbers
above 9. We can either:

* accept trace numbers above 9 in the test
* or drop traces from the initialization code in the test and assume
  that the new trace will have number 1.

Igor Munkin suggested to stick with the second approach to avoid
dependency on the initialization code.

NO_DOC=test fixup, no user visible changes
NO_CHANGELOG=see NO_DOC

Co-authored-by: Igor Munkin <imun@tarantool.org>