Currently, if a snapshot contains some correct entries, but doesn't
include system spaces, Tarantool crashes with segmentation fault, or
for Debug build: void diag_raise(): Assertion `e != NULL' failed.
This happens because memtx_engine_recover_snapshot returns -1, while
diag is not set. Let's panic instead of a crash.

Closes #7800

NO_DOC=bugfix

(cherry picked from commit e0a9aed4)

0xc1 isn't a valid MsgPack header, but it was allowed by mp_check.
As a result, msgpack.decode crashed while trying to decode it.
This commit updates the msgpuck library to fix this issue.

Closes #7818

NO_DOC=bug fix

(cherry picked from commit ced405af)

Closes #7797

NO_DOC=security fix
NO_TEST=security fix

(cherry picked from commit dd7d46af)

getenv() return values cannot be trusted, because an attacker might set
them. For instance, we shouldn't expect, that getenv() returns a value
of some sane size.

Another problem is that getenv() returns a pointer to one of
`char **environ` members, which might change upon next setenv().

Introduce a wrapper, getenv_safe(), which returns the value only when
it fits in a buffer of a specified size, and copies the value onto the
buffer. Use this wrapper everywhere in our code.

Below's a slightly decorated output of `grep -rwn getenv ./src --include
*.c --include *.h --include *.cc --include *.cpp --include *.hpp
--exclude *.lua.c` as of 2022-10-14.
`-` marks invalid occurences (comments, for example),
`*` marks the places that are already guarded before this patch,
`X` mars the places guarded in this patch, and
`^` marks places fixed in the next commit:

NO_WRAP
```
* ./src/lib/core/coio_file.c:509:	const char *tmpdir = getenv("TMPDIR");
X ./src/lib/core/errinj.c:75: const char *env_value = getenv(inj->name);
- ./src/proc_title.c:202: * that might try to hang onto a getenv() result.)
- ./src/proc_title.c:241:	* is mandatory to flush internal libc caches on getenv/setenv
X ./src/systemd.c:54: sd_unix_path = getenv("NOTIFY_SOCKET");
* ./src/box/module_cache.c:300: const char *tmpdir = getenv("TMPDIR");
X ./src/box/sql/os_unix.c:1441: azDirs[0] = getenv("SQL_TMPDIR");
X ./src/box/sql/os_unix.c:1446: azDirs[1] = getenv("TMPDIR");
* ./src/box/lua/console.c:394: const char *envvar = getenv("TT_CONSOLE_HIDE_SHOW_PROMPT");
^ ./src/box/lua/console.lua:771: local home_dir = os.getenv('HOME')
^ ./src/box/lua/load_cfg.lua:1007: local raw_value = os.getenv(env_var_name)
X ./src/lua/init.c:575: const char *path = getenv(envname);
X ./src/lua/init.c:592: const char *home = getenv("HOME");
* ./src/find_path.c:77: snprintf(buf, sizeof(buf) - 1, "%s", getenv("_"));
```
NO_WRAP

Part-of #7797

NO_DOC=security

(cherry picked from commit b86395ff)

This patch fixes the issue described in issue #5310 when the tuple
format has more fields than the space format. This solution is more
general than the solution in 89057a21.

Follow-up #5310
Closes #4666

NO_DOC=bugfix

(cherry picked from commit 5a38c5c9)

Since our diagnostics use the `__FILE__` macro, they provide absolute
paths, which is kind of redundant and inconsistent: replace them with
relative ones.

As for debugging information, replacing absolute paths with relative ones
also requires an extra command to tell the debugger where to find the
source files, which is not convenient for developers: provide a new
`DEV_BUILD` option (turned off by default), which replaces absolute paths
with relative ones in debugging information if turned off.

Strip the prefix map flags from compiler flags exported to tarantool via
`src/trvia/config.h`.

Closes #7808

NO_DOC=<verbosity>
NO_TEST=<verbosity>

(cherry picked from commit 256da010)

Fixes bug with determination of debug build (tarantool/test-run#352).

NO_CHANGELOG=<internal submodule version update>
NO_TEST=<submodule version update>
NO_DOC=<submodule version update>

(cherry picked from commit e554e481)

Setting hardening compiler flags is used in three places: default build,
static build and enterprise build — refactor it into a separate module.

Follow-up e6abe1c9

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

(cherry picked from commit dd51a2fa)

e6abe1c9 passes compiler flags to dependencies via a `<project>_build` macro
parameter, which is, firstly, inconvenient, and, secondly, as a result, not
all dependencies got the required compiler flags passed: use global
variables instead and pass these flags to skipped dependencies.

Follow-up e6abe1c9

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

(cherry picked from commit c6794757)

`add_compile_flags` macro relied on undefined behaviour: instead of
unsetting the `_lang` and `_flag` variables, it tried to unset their
values — fix this.

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

(cherry picked from commit 4aa9f154)

Information about test code coverage is useful on code review.
This patch enables code coverage job in default ci.

NO_CHANGELOG=not a user-visible change
NO_DOC=not a user-visible change
NO_TEST=ci

(cherry picked from commit 66e388e8)

If we raise different errors in case of entering an invalid password and
entering the login of a non-existent user during authorization, it will
open the door for an unauthorized person to enumerate users.
So let's unify raised errors in the cases described above.

Closes #tarantool/security#16

NO_DOC=security fix

(cherry picked from commit 5c62f01b)

The _vfunc system space is the sysview for the _func system space.
However, the _vfunc format is different from the _func format. This
patch makes the _vfunc format the same as the _func format.

Closes #7822

NO_DOC=bugfix

(cherry picked from commit 707da125)

We used to ignore timezone difference (in `tzoffset`) for
datetime subtraction operation:

```
tarantool> datetime.new{tz='MSK'} - datetime.new{tz='UTC'}
---
- +0 seconds
...

tarantool> datetime.new{tz='MSK'}.timestamp -
           datetime.new{tz='UTC'}.timestamp
---
- -10800
...
```

Now we accumulate tzoffset difference in the minute component
of a resultant interval:

```
tarantool> datetime.new{tz='MSK'} - datetime.new{tz='UTC'}
---
- -180 minutes
...
```

Closes #7698

NO_DOC=bugfix

(cherry picked from commit 0daed8d5)

We did not take into consideration the fact that
as result of date/time arithmetic we could get
in a different timezone, if DST boundary has been
crossed during operation.

```
tarantool> datetime.new{year=2008, month=1, day=1,
			tz='Europe/Moscow'} +
	   datetime.interval.new{month=6}
---
- 2008-07-01T01:00:00 Europe/Moscow
...
```

Now we resolve tzoffset at the end of operation if
tzindex is not 0.

Fixes #7700

NO_DOC=bugfix

(cherry picked from commit 6ca07285)

In PR #7791 the coverage stuff is slightly updated. The changes are
related to excluding some directories where the coverage will not be
calculated. The patch itself is straightforward enough, but I can see
the problems with uploading the results to coveralls.io:

NO_WRAP
    Using lcov file: ./coverage.info
    FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

    <--- Last few GCs --->

    [2683200:0x327c000]    27836 ms: Scavenge 2033.4 (2038.7) -> 2033.2 (2044.2) MB, 6.8 / 0.0 ms  (average mu = 0.258, current mu = 0.243) allocation failure
    [2683200:0x327c000]    27854 ms: Scavenge 2036.7 (2044.2) -> 2037.1 (2044.4) MB, 15.6 / 0.0 ms  (average mu = 0.258, current mu = 0.243) allocation failure
    [2683200:0x327c000]    27871 ms: Scavenge 2037.2 (2044.4) -> 2036.7 (2055.9) MB, 17.1 / 0.0 ms  (average mu = 0.258, current mu = 0.243) allocation failure

    <--- JS stacktrace --->

    ==== JS stack trace =========================================

        0: ExitFrame [pc: 0x140dc19]
    Security context: 0x2481918808d1 <JSObject>
        1: encode(aka encode) [0x23ec5e514a11] [/opt/actions-runner/_work/_actions/coverallsapp/github-action/v1.1.2/node_modules/qs/lib/utils.js:~118] [pc=0x193ecd3d5d82](this=0x35d34e4804b1 <undefined>,0x1e5895380119 <Very long string[60387428]>)
        2: stringify(aka stringify) [0x23ec5e5174c9] [/opt/actions-runner/_work/_actions/coverallsapp/github-action/v1.1.2/n...

     1: 0xa1a640 node::Abort() [/opt/actions-runner/externals/node12/bin/node]
     2: 0xa1aa4c node::OnFatalError(char const*, char const*) [/opt/actions-runner/externals/node12/bin/node]
     3: 0xb9a62e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/opt/actions-runner/externals/node12/bin/node]
     4: 0xb9a9a9 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/opt/actions-runner/externals/node12/bin/node]
     5: 0xd57c25  [/opt/actions-runner/externals/node12/bin/node]
     6: 0xd582b6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/opt/actions-runner/externals/node12/bin/node]
     7: 0xd64b75 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/opt/actions-runner/externals/node12/bin/node]
     8: 0xd65a25 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/opt/actions-runner/externals/node12/bin/node]
     9: 0xd670cf v8::internal::Heap::HandleGCRequest() [/opt/actions-runner/externals/node12/bin/node]
    10: 0xd15b35 v8::internal::StackGuard::HandleInterrupts() [/opt/actions-runner/externals/node12/bin/node]
    11: 0x1071176 v8::internal::Runtime_StackGuard(int, unsigned long*, v8::internal::Isolate*) [/opt/actions-runner/externals/node12/bin/node]
    12: 0x140dc19  [/opt/actions-runner/externals/node12/bin/node]
NO_WRAP

I have tried to use the latest version of the action (1.1.3), but I have
got the same issue.

It looks like it is a `coverallsapp/github-action` issue due to issue
in JavaScript due to inefficient amount of memory. The corresponding
bug [1] was filed against the repo.

So I suggest switching to the console utility `coveralls-lcov`. I did
some internal testing and it works fine.

[1] https://github.com/coverallsapp/github-action/issues/133

Fixes tarantool/tarantool-qa#278

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 761574e3)

This patch fixed the assertion when JOIN uses index of unsupported type.

Closes #5678

NO_DOC=bugfix

(cherry picked from commit fd780129)

Currently if a non-string type is passed to luaT_key_def_set_part,
lua_tolstring returns null-pointer type_name, which is passed to
a string formatting function in diag_set.

Closes #5222

NO_DOC=bugfix

(cherry picked from commit 5215f3f3)

Don't accept an empty string or leading part of "str" or "num" as a
valid field type.

Closes #5940

NO_DOC=Partial field types weren't documented

Co-authored-by: Alexander Turenko <alexander.turenko@tarantool.org>
(cherry picked from commit 2dbaf9c2)

Since the function is actually an eval, by default there should
be no execute access right in public role.

Closes tarantool/security#14

NO_DOC=bugfix

(cherry picked from commit 815788c8)

By design after patching update.lua, introducing new data version,
a developer needs to update bootstrap.snap, which would have the
new data version.

Generation of bootstrap.snap is a bit tricky, there's a manual:
https://github.com/tarantool/tarantool/wiki/How-to-generate-new-bootstrap-snapshot

This commit presents a simple script that automates the process.
Just run it in tarantool build directory and check its output.
Check --help for script options.

Follow-up of tarantool/security#14

NO_DOC=internal tool
NO_TEST=internal tool
NO_CHANGELOG=internal tool

(cherry picked from commit 0922816e)

Fix a simple typo that caused the problem.

Closes #7645

NO_DOC=bugfix

(cherry picked from commit 6e450424)

This patch introduces new rules to determine type of NULLIF() built-in
function.

Closes #6990

@TarantoolBot document
Title: New rules to determine type of result of NULLIF

The type of the result of NULLIF() function now matches the type of the
first argument.

(cherry picked from commit 805cbaa7)

This patch introduces new rules to determine type of CASE operation.

Part of #6990

@TarantoolBot document
Title: New rules to determine type of result of CASE

New rules are applied to determine the type of the CASE operation. If
all values are NULL with no type, or if a bind variable exists among
the possible results, then the type of CASE is ANY. Otherwise, all NULL
values with no type are ignored, and the type of CASE is determined
using the following rules:
1) if all values of the same type, then type of CASE is this type;
2) otherwise, if any of the possible results is of one of the
incomparable types, then the type of CASE is ANY;
3) otherwise, if any of the possible results is of one of the
non-numeric types, then the type of CASE is SCALAR;
4) otherwise, if any of the possible results is of type NUMBER, then the
type of CASE is NUMBER;
5) otherwise, if any of the possible results is of type DECIMAL, then
the type of CASE is DECIMAL;
6) otherwise, if any of the possible results is of type DOUBLE, then the
type of CASE is DOUBLE;
7) otherwise the type of CASE is INTEGER.

(cherry picked from commit 90f64460)

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit e5e14cd6)

If the length of the tuple is greater than the number of fields in the
format, it is possible that the cursor in the VDBE will be overridden
with zeros.

Closes #5310

NO_DOC=bugfix

(cherry picked from commit 89057a21)

Usually, GitHub actions are named like `foo-bar` rather than `foo_bar`.
A few widely known examples: upload-artifact [1], download-artifact [2],
setup-python [3], setup-node [4]. So let's stick to this approach also.

[1] https://github.com/actions/upload-artifact
[2] https://github.com/actions/download-artifact
[3] https://github.com/actions/setup-python
[4] https://github.com/actions/setup-node

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 3c4a5056)

This patch fixed the assertion when using INDEXED BY with an index that
is at least the third in space.

Closes #5976

NO_DOC=bugfix

(cherry picked from commit 22c65f96)

* FFI: Always fall back to metamethods for cdata length/concat.
* FFI: Add tonumber() specialization for failed conversions.
* build: introduce LUAJIT_ENABLE_CHECKHOOK option
* Fix overflow check in unpack().
* gdb: refactor iteration over frames while dumping stack
* gdb: adjust to support Python 2 (CentOS 7)

Closes #7458
Closes #7655
Needed for #7762
Part of #7230

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

dt.new() will raise a clear error on wrong timestamp type.

Closes #7273

NO_DOC=bugfix
NO_CHANGELOG=bugfix

(cherry picked from commit 8d4cbf44)

Such style is not recommented and going to be banned.

Closes #7715

NO_TEST=test refactoring
NO_DOC=test refactoring
NO_CHANGELOG= test refactoring

(cherry picked from commit ce784430)

Some luatest framework tests use Lua `assert`s, which are incomprehensible
when failed (the only information provided is 'assertion failed!'),
making debugging difficult: replace them with luatest `assert`s and their
context-specific varieties.

NO_CHANGELOG=<code health>
NO_DOC=<code health>

(cherry picked from commit 1d9645f7)

Commit 7f2c7609 ("box: add option to disable Lua FFI for read
operations") added an internal configuration option that switches all
index read operations from FFI to Lua C, which was necessary to
implement space upgrade in the EE repository. Turns out there was a
minor bug in that commit - when FFI is disabled, method usage errors
(raised when object.method is called instead of object:method) stop
working. Fix this bug and add an extensive test that checks that Lua C
and FFI implementations of index read operations are equivalent.

Fixes https://github.com/tarantool/tarantool-ee/issues/254

NO_DOC=bug fix
NO_CHANGELOG=bug manifests itself in only in EE

(cherry picked from commit e8d9a7cb)

Concurrent transactions can try to insert tuples that intersect only by
parts of secondary index: in this case when one of them gets prepared, the
others get conflicted, but the committed story does not get retained
(because the conflicting statements are not added to the committed story's
delete statement list as opposed to primary index) and is lost after
garbage collection: retain stories if there is a newer uncommitted story
in the secondary indexes' history chain.

Closes #7712

NO_DOC=bugfix

(cherry picked from commit 7b0baa57)

Generate changelog for 2.10.3 release.
Also, clean changelogs/unreleased folder.

NO_DOC=no code changes
NO_TEST=no code changes
NO_CHANGELOG=no code changes

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

Fix wording, punctuation, and formatting.

NO_CHANGELOG=changelog
NO_DOC=changelog
NO_TEST=changelog

When using vclockset_psearch, the resulting vclock may be incomparable
to the search key. For example, with a vclock set { } (empty vclock),
{0: 1, 1: 10}, {0: 2, 1:11} vclockset_psearch(set, {0:2, 1: 9}) might
return {0: 1, 1: 10}, and not { }.
This is known and avoided in other places, for example
recover_remaining_wals(), where vclockset_match() is used instead.
vclockset_match() starts with the same result as vclockset_psearch() and
then unwinds the result until the first vclock which is less or equal to
the search key is found.

Having vclockset_psearch in wal_collect_garbage_f could lead to issues
even before local space changes became written to 0-th vclock component.
Once replica subscribes, its' gc consumer is set to the vclock, which
the replica sent in subscribe request. This vclock might be incomparable
with xlog vclocks of the master, leading to the same issue of
potentially deleting a needed xlog during gc.

Closes #7584

NO_DOC=bugfix

(cherry picked from commit c63bfb9a)

We hold the following invariant in MVCC: the story at the top of the
history chain is present in index.

If a story is subject to be deleted from index and there is an older story
in the history chain, the older story starts to be at the top of the
history chain and is not present in index, which violates our invariant:
explicitly check for this case when evaluating whether a story can be
garbage collected and add an assertion to check the invariant above is not
violated.

Rollbacked stories need to be handled in a special way: they are
present at the end of some history chains and completely unlinked from
others (which also implies they are not present in the corresponding
indexes).

`memtx_tx_story_full_unlink` is called in two contexts: space deletion, in
which we delete all stories, and garbage collection step — the former case
can break the invariant described above, while the latter must preserve it,
hence add two different functions for the corresponding contexts.

Closes #7490

NO_CHANGELOG=<internal bugfix not user observable>
NO_DOC=<bugfix>

(cherry picked from commit c8eccfbb)