Add a missed check to on_replace_dd_truncate, similar to
on_replace_dd_space and on_replace_dd_index.

Closes #7309

NO_DOC=bugfix

The problem is that even if we block all signals on all
threads except the main thread, the signals still can be
delivered to other threads (#7206). And another problem
is that user can spawn his own thread and not block
signals.

That is why the patch introduces tt_sigaction function that
guarantees that all signals will be handled only by the main
thread. We use this helper in clock_lowres module.
This is supposed to solve the problem, described in #7408.

NO_CHANGELOG=internal
NO_DOC=internal

Added a function (see the API in the documentation request below), which
reflects the `tuple[json_path]` Lua API (see #1285).

Part of #7228

@TarantoolBot document
Title: tuple: access a field using JSON path via module API

The following function is added into the module API:

```c
/**
 * Return a raw tuple field in the MsgPack format pointed by
 * a JSON path.
 *
 * The JSON path includes the outmost field. For example, "c" in
 * ["a", ["b", "c"], "d"] can be accessed using "[2][2]" path (if
 * index_base is 1, as in Lua). If index_base is set to 0, the
 * same field will be pointed by the "[1][1]" path.
 *
 * The first JSON path token may be a field name if the tuple
 * has associated format with named fields. A field of a nested
 * map can be accessed in the same way: "foo.bar" or ".foo.bar".
 *
 * The return value is valid until the tuple is destroyed, see
 * box_tuple_ref().
 *
 * Return NULL if the field does not exist or if the JSON path is
 * malformed or invalid. Multikey JSON path token [*] is treated
 * as invalid in this context.
 *
 * \param tuple a tuple
 * \param path a JSON path
 * \param path_len a length of @a path
 * \param index_base 0 if array element indexes in @a path are
 *        zero-based (like in C) or 1 if they're one-based (like
 *        in Lua)
 * \retval a pointer to a field data if the field exists or NULL
 */
API_EXPORT const char *
box_tuple_field_by_path(box_tuple_t *tuple, const char *path,
			uint32_t path_len, int index_base);
```

In Lua field's numbers are counted from base 1, however currently
space:format() and space.foreign_key return zero-based foreign key
fields, which leads to an error on space:format(space:format()).

Closes #7350

NO_DOC=bugfix

Currently a foreign_key field in the `format` argument, passed to
normalize_format, can be changed inside normalize_foreign_key_one.
Fix this by using a local copy of def.field.

NO_DOC=bugfix
NO_CHANGELOG=minor bug

It is inconvenient to create self-referencing FK constraints, as the
space ID will only be generated during space creation. This is
especially useful for SQL, since the format for the space is created
at compile time, and the space ID is only obtained at run time.

Closes #7200

@TarantoolBot document
Title: Describe foreign keys referring to the same space
Since: 2.11
Root document: https://www.tarantool.io/en/doc/latest/book/box/data_model/#foreign-keys

It is possible to create a foreign key that refers to the same space (a
child space equals to the parent space).
To do that, omit `space` in the `foreign_key` parameter, or set it to
the id or to the name of the current space.

According to man 3 exec: "The first argument, by convention, should point
to the filename associated with the file being executed.". Using empty
string as the first argument while calling `true` program, provided by
coreutils led to error message being printed to stderr, which failed the
test.

This patch passes 'true' as the first argument.

Closes #7452.

NO_DOC=test
NO_CHANGELOG=test

Currently net.box's methods doesn't check types of passed options.
This can lead to Lua's internal errors, which are not self-explaining.

Let's add this functionality and raise errors with meaningful messages
in case of incorrect options.

Closes #6063
Closes #6530

NO_DOC=Not a visible change

Test is needed to be updated every time net.box's internals are patched
(e.g. in case of adding new functions of deleting old ones), as the order
of suggested autocomplete options changes.

Let's replace `assert_equals` with `assert_items_equals`, which is not
order dependent.

NO_DOC=test
NO_CHANGELOG=test

Created simplistic unit test for checking boundary conditions
for `datetime_unpack` function.

NO_DOC=internal
NO_CHANGELOG=internal

Introduced fuzzing testing for datetime decoder.

NO_DOC=internal
NO_CHANGELOG=internal

NO_DOC=refactoring
NO_CHANGELOG=refactoring

This patch fixes the ctrl+c behavior in multiline mode.
The old behavior was ignoring sigint in this mode.
The new behavior is quitting this mode and printing the new
default prompt.
Added a new test for covering the mentioned case.

Follows up #2717
Closes #7109

NO_DOC=BUGFIX

This commit replaces enter/leave_delayed_free_mode() MemtxAllocator
methods with open/close_read_view(). The open_read_view() method
returns a pointer to the new read view, which is supposed to be pssed
to close_read_view() when the read view is no longer needed. The new
API will allow us to optimize garbage collection, see #7185.

Currently, open_read_view() always returns nullptr and both ReadView
and memtx_read_view_opts are empty structs. We will extend them in
the future.

Closes #7364

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Checking #7185 with functional tests is complicated. Let's add a unit
test for this.

Needed for #7185

NO_DOC=test
NO_CHANGELOG=test

Now we can make interactive transactions like in case of
local or txt remote consoles if peer supports streams.

Closes #7413

NO_DOC=minor change

The issue is if in binary remote console a error is thrown in expression
like "box.begin() error('something') box.commit()" then it is
overwritten by iproto check for active transactions at the end of eval.
The solution is to rollback active transaction in this case in console
code before iproto check.

Let's also assert in tests a behaviour that after successful transaction
begin it stays active in next evaluations (successful or not) until
explicit rollback.

Closes #7288

NO_DOC=minor change

Starting server is failed after chdir(3) for example if we call
box.cfg{} with work_dir set.

NO_DOC=fix testing harness
NO_TEST=fix testing harness
NO_CHANGELOG=fix testing harness

We keep per server directories there too, not only socket files.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The ; separator was missed after select (check the first hunk) and as
a result the rollback was not executed. So the next test was failed merely
due to the unfinished transaction of the previous test.

We also need to fix the third test to use a different tuple.
Also fix a couple of comments.

NO_DOC=test fix
NO_TEST=test fix
NO_CHANGELOG=test fix

If MP_ERROR contains an error message with invalid UTF-8 sequences,
encode it in Base64 to obtain printable string.

Closes #6781
Closes #6934

NO_DOC=bugfix

The upgrade script first tries to determine if the node is booted from
old snaps not recoverable on current Tarantool versions. If this is the
case, it sets up special triggers so that snaps are automatically
converted to a suitable format.

This happens before box.cfg{}, so the workdir is not set at this point
in time, and the upgrade script should take configured work_dir into
account explicitly. Fix this.

Closes #7232

NO_DOC=bugfix

Prior to this patch, some opcodes could use a pointer to struct space
that was set during parsing. However, the pointer to struct space is not
really something that defines spaces. A space can be identified by its
ID or name. In most cases, specifying space by pointer works fine, but
truncate() changes the pointer to space, resulting in a sigfault for
prepared statements using the above opcodes. To avoid this problem, a
new opcode has been introduced. This opcode uses the space ID to
determine the pointer to the struct space at runtime and stores it in
the MEM, which is later used in the mentioned opcodes.

Closes #7358

NO_DOC=bugfix

Functional index keys are allocated and freed with MemtxAllocator's
alloc and free methods. In contrast to tuples, which are allocated and
freed with alloc_tuple and free_tuple, freeing a functional index key
happens immediately, irrespective of whether there's a snapshot in
progress or not. It's acceptable, because snapshot only uses primary
indexes, which can't be functional. However, to reuse the snapshot
infrastructure for creating general purpose user read views, we will
need to guarantee that functional index keys stay alive until all read
views using them are closed.

To achieve that, this commit turns functional index keys into tuples,
which automatically makes them linger if there's an open read view.
We use the same global tuple format for allocating functional keys,
because the key format is checked in key_list_iterator_next.

Closes #7376

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Currently the latch doesn't guarantee the order in which it is acquired
by fibers, which requested it. E.g. it is possible to wake up spuriously
a fiber which is yielding in the latch_lock, it will be removed from
l->queue by fiber_make_ready, then it will be inserted to l->queue
again, but for this time, to the head of the list instead of its
original place in the queue.

Fix this by using latch_waiter structure, which is linked into l->queue.

Part of #7166

@TarantoolBot document
Title: Update box_latch_lock description
Since: 2.11

Add "Locks are acquired in the strict order as they were requested." to
the box_latch_lock description in C API reference - Module latch.

This patch fixes the following error while tarantool build on
FreeBSD 13.1:

    [100%] Linking CXX executable watcher.test
    ld: error: undefined symbol: set_sigint_cb
    >>> referenced by console.c:261 (lua/console.c:261)
    >>>               console.c.o:(lbox_console_readline) in
                        archive ../../src/box/libbox.a
    >>> referenced by console.c:342 (lua/console.c:342)
    >>>               console.c.o:(lbox_console_readline) in
                        archive ../../src/box/libbox.a
    >>> referenced by console.c:352 (lua/console.c:352)
    >>>               console.c.o:(lbox_console_readline) in
                        archive ../../src/box/libbox.a
    >>> referenced 1 more times
    c++: error: linker command failed with exit code 1
        (use -v to see invocation)
    gmake[2]: *** [test/unit/CMakeFiles/watcher.test.dir/build.make:152:
        test/unit/watcher.test] Error 1
    gmake[1]: *** [CMakeFiles/Makefile2:10528:
        test/unit/CMakeFiles/watcher.test.dir/all] Error 2
    gmake: *** [Makefile:156: all] Error 2

NO_DOC=minor changes
NO_TEST=minor changes
NO_CHANGELOG=minor changes

The gh_6565 test doesn't stop the hot standby replica it started,
because the replica should fail to initialize and exit eventually
anyway. However, if the replica lingers until the next test due to
https://github.com/tarantool/test-run/issues/345, the next test may
successfully connect to it, which is likely to lead to a failure,
because UNIX socket paths used by luatest servers are not randomized.

For example, here gh_6568 test fails after gh_6565, because it uses the
same alias for the test instance ('replica'):

NO_WRAP
[008] vinyl-luatest/gh_6565_hot_standby_unsupported_>                 [ pass ]
[008] vinyl-luatest/gh_6568_replica_initial_join_rem>                 [ fail ]
[008] Test failed! Output from reject file /tmp/t/rejects/vinyl-luatest/gh_6568_replica_initial_join_removal_of_compacted_run_files.reject:
[008] TAP version 13
[008] 1..1
[008] # Started on Fri Jul  8 15:30:47 2022
[008] # Starting group: gh-6568-replica-initial-join-removal-of-compacted-run-files
[008] not ok 1  gh-6568-replica-initial-join-removal-of-compacted-run-files.test_replication_compaction_cleanup
[008] #   builtin/fio.lua:242: fio.pathjoin(): undefined path part 1
[008] #   stack traceback:
[008] #         builtin/fio.lua:242: in function 'pathjoin'
[008] #         ...ica_initial_join_removal_of_compacted_run_files_test.lua:43: in function 'gh-6568-replica-initial-join-removal-of-compacted-run-files.test_replication_compaction_cleanup'
[008] #         ...
[008] #         [C]: in function 'xpcall'
[008] replica | 2022-07-08 15:30:48.311 [832856] main/103/default.lua F> can't initialize storage: unlink, called on fd 30, aka unix/:(socket), peer of unix/:(socket): Address already in use
[008] # Ran 1 tests in 0.722 seconds, 0 succeeded, 1 errored
NO_WRAP

Let's fix this by explicitly killing the hot standby replica. Since it
could have exited voluntarily, we need to use pcall, because server.stop
fails if the instance is already dead.

This issue is similar to the one fixed by commit 85040161 ("test:
stop server started by vinyl-luatest/update_optimize test").

NO_DOC=test
NO_CHANGELOG=test

Handle status header response like 'HTTP/2 200' with version without
dot.

Closes #7319

NO_DOC=bugfix

We use LuaJIT 'bit' module for bitwise operations. Due to platform
interoperability it truncates arguments to 32bit and returns signed
result. Thus on granting rights using bit.bor to admin user which
have 0xffffffff rights (from bootstrap snapshot) we get -1 as a result.
This leads to type check error given in issue later in execution.

Closes #7226

NO_DOC=minor bugfix

Let's hide all the logic regarding delayed freeing of memtx tuples to
MemtxAllocator and provide memtx_engine with methods for allocating and
freeing tuples (not memtx_tuples, just generic tuples). All the tuple
and snapshot version manipulation stuff is now done entirely in
MemtxAllocator.

This is a preparation for implementing a general-purpose tuple read view
API in MemtxAllocator, see #7364.

Note, since memtx_engine now deals with the size of a regular tuple,
which is 4 bytes less than the size of memtx_tuple, this changes the
size reported by OOM messages and the meaning of memtx_max_tuple_size,
which now limits the size of a tuple, not memtx_tuple.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This patch fixes a bug where the ANY field type was replaced by the
SCALAR field type in the ephemeral space used in ORDER BY.

Closes #7345

NO_DOC=bugfix

After this patch, the result type of arithmetic between two unsigned
values will be INTEGER.

Closes #7295

NO_DOC=bugfix

This makes the test more real-life, and allows not to bother in the
child process with the memory allocated prior to fork.

Closes #7370

NO_DOC=test fix
NO_CHANGELOG=test fix

Since "x64/LJ_GC64: Fix fallback case of asm_fuseloadk64()."
(42853793ec3e6e36bc0f7dff9d483d64ba0d8d28) is backported into
tarantool/luajit trunk, box/bitset.test.lua and box/function1.test.lua
tests are no more fragile.

Follows up tarantool/tarantool-qa#234
Follows up tarantool/tarantool-qa#235

NO_DOC=test changes
NO_CHANGELOG=test changes
NO_TEST=test changes

This patch fixes `box-py/args.test.py` test and allows it to work
against tarantool installed from a package.

Closes tarantool/tarantool-qa#246

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

This patch fixes `app-tap/tarantoolctl.test.lua` test and allows it to
work against tarantool installed from a package.

Part of tarantool/tarantool-qa#246

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

This patch fixes `gh-1700-abort-recording-on-fiber-switch.test.lua` test
and allows it to work against tarantool installed from a package.

Part of tarantool/tarantool-qa#246

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

This patch adds the new `make` test targets to run unit and functional
tests independending on each other. In some cases it can be useful.

New test targets:

* `test-unit` - run unit tests and exit after the first failure
* `test-unit-force` - run unit tests
* `test-func` - run functional tests and exit after the first failure
* `test-func-force` - run functional tests

Note, tests for 'small' lib are considered as unit tests as well.

Part of tarantool/tarantool-qa#246

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

If readline 'show-mode-in-prompt' is on then test fails because it does
not handle prefix added to prompt in this mode. Let's use default
(compiled in) readline configuration instead of the one provided by
user or system config.

NO_DOC=test changes
NO_CHANGELOG=test changes
NO_TEST=test changes

Current implementation of tracking statements that delete a story has a
flaw, consider the following example:

tx1('box.space.s:replace{0, 0}') -- statement 1

tx2('box.space.s:replace{0, 1}') -- statement 2
tx2('box.space.s:delete{0}') -- statement 3
tx2('box.space.s:replace{0, 2}') -- statement 4

When statement 1 is prepared, both statements 2 and 4 will be linked to the
delete statement list of {0, 0}'s story, though, apparently, statement 4
does not delete {0, 0}.

Let us notice the following: statement 4 is "pure" in the sense that, in
the transaction's scope, it is guaranteed not to replace any tuple — we
can retrieve this information when we check where the insert statement
violates replacement rules, use it to determine "pure" insert statements,
and skip them later on when, during preparation of insert statements, we
handle other insert statements which assume they do not replace anything
(i.e., have no visible old tuple).

On the contrary, statements 1 and 2 are "dirty": they assume that they
replaced nothing (i.e., there was no visible tuple in the index) — when one
of them gets prepared — the other one needs to be either aborted or
relinked to replace the prepared tuple.

We also need to fix relinking of delete statements from the older story
(in terms of the history chain) to the new one during preparation of insert
statements: a statement needs to be relinked iff it comes from a different
transaction (to be precise, there must, actually, be no more than one
delete statement from the same transaction).

Additionally, add assertions to verify the invariant that the story's
add (delete) psn is equal to the psn of the add (delete) statement's
transaction psn.

Closes #7214
Closes #7217

NO_DOC=bugfix