decimal_unpack() and hence mp_decode_decimal() didn't check length of
digits array, overflowing the output structure.

Fix that and a couple of typos in comments as well.

Closes https://github.com/tarantool/security/issues/17

NO_DOC=security
NO_CHANGELOG=security

(cherry picked from commit 142c324e)

Some internal modules have been recently copied to luatest repo [1,2,3]
and now they can be safely removed, and the corresponding functionality
from luatest can be used instead.

Affected modules:

- test/luatest_helpers/misc.lua
- test/luatest_helpers/fiber.lua
- test/luatest_helpers/proxy/*

[1] https://github.com/tarantool/luatest/pull/247
[2] https://github.com/tarantool/luatest/pull/248
[3] https://github.com/tarantool/luatest/pull/255

Closes tarantool/luatest#238
Closes tarantool/luatest#251

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 54bd77ad)

Bump test-run to new version with the following improvements:

- Bump luatest to 0.5.7-6-gee1f5c1 [1]
- Fix detection of debug builds [2]

[1] https://github.com/tarantool/test-run/pull/354
[2] https://github.com/tarantool/test-run/pull/356

Part of tarantool/luatest#238
Part of tarantool/luatest#251

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

(cherry picked from commit 002b1ffc)

Drop unused code and arguments to fix the warning:

src/lua/log.lua:223:11: variable log_type is never accessed

Follow-up commit 21c6fa6e ("say: enable json log format with
syslog").

NO_DOC=luacheck
NO_TEST=luacheck
NO_CHANGELOG=luacheck

All the heavy lifting was done in the previous commit that dropped
the syslog formatter. This commit just removes the checks that forbid
using syslog with json.

Closes #7860

@TarantoolBot document
Title: JSON log format can now be used with syslog

The configuration reference says that syslog is incompatible with
the JSON log format:

https://www.tarantool.io/en/doc/latest/reference/configuration/#confval-log_format

This isn't true anymore. If JSON is used with syslog, then a JSON
message body is appended to the syslog header.

See https://github.com/tarantool/tarantool/issues/7860.

(cherry picked from commit ec1747b4)

All messages written to syslog must have a header, which contains the
current time, message severity, and process identity, so we have a
special formatter function for syslog. As a result, the syslog log
destination is incompatible with the json log format, which has its own
formatter function. This is confusing, because there's nothing that
prevents us from writing json in the log entry body to syslog - we just
need to prepend the message with a proper header.

As a preparation for befriending json and syslog, let's get rid of the
syslog formatter function and instead write the header right in log_vsay
in case logs are written to syslog. We just need to strip the duplicate
information from the log entry in say_format_plain so as not to print
the time and pid twice.

Needed for #7860

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 11ac72bd)

We have three places in say.c where we peform exactly the same steps to
obtain the current time. Let's add a helper function to avoid code and
comments duplication.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit dd825873)

It's a global althoug static variable so better use say_ prefix so as
not to mix it with a local variable.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 0920ab39)

`dlsym` is known to be buggy in FreeBSD. See #7640.

NO_DOC=internal
NO_CHANGELOG=internal

(cherry picked from commit 67e79b15)

The test is expected the replication is broken by the time of the
server switch. Apparently, switch won the race, so the expectation
is changed for waiting.

Closes https://github.com/tarantool/tarantool-qa/issues/271

NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing

(cherry picked from commit 0e181b35)

If a complex (tuple) foreign key is added along with a new field name
that participates in that foreign key, the foreign key does not work
correctly. This happens because the new name of a local field is added
to new_space->def->dict only in ModifySpace::alter, while before that,
new_space->def->dict points to the old dictionary with old names (see
ModifySpace::alter_def). So when local_field_no is initialized earlier
in alter_space_do -> space_create -> tuple_constraint_fkey_init, it's
unable to find the new field number in the old dictionary.

Fix this by moving `new_def->dict = alter->old_space->def->dict;` from
ModifySpace::alter_def() to ModifySpace::alter(). Note that, as before,
we refer to the old dictionary from the new space (just put new names
into it), because that dict is referenced by existing tuple formats.

Closes #7652

NO_DOC=bugfix

(cherry picked from commit 4aaf9049)

`say_logrotate` does not rotate logs synchronously. It posts tasks to
coio which executes them in it's pool thread. On application exit we
destroy logs calling `log_destroy`. This function waits for rotate task
to finish because if it will free resources immediately then unfinished
rotate task can have use-after-free issues. Waiting crashes because at
this moment event loop is not running which is required for
`fiber_cond_wait` to work.

Note that if there is no crash then we will hang forever waiting in
`log_destroy` because event loop is not running and
`log_rotate_async_cb` will never be executed.

Let's use mutexes and conditions instead. It solves both problems with
crash and hang. Thanks Vladimir Davydov for idea.

Fixes #4450

NO_DOC=bugfix

(cherry picked from commit eed09192)

Fixes tarantool/tarantool-qa#282

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 16585472)

Fixes tarantool/tarantool-qa#281

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 3c0b800a)

Fix the following warning:

NO_WRAP
    Node.js 12 actions are deprecated. For more information see:
    https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.
    Please update the following actions to use Node.js 16: actions/checkout
NO_WRAP

Fixes tarantool/tarantool-qa#279

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 847e5e0c)

Fix the following warning:

NO_WRAP
    Node.js 12 actions are deprecated. For more information see:
    https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.
    Please update the following actions to use Node.js 16: actions/upload-artifact
NO_WRAP

Fixes tarantool/tarantool-qa#280

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 90feddff)

This commit fixes BEGIN, COMMIT, and ROLLBACK counters in the box.stat()
output. Before this commit, they always showed 0. Now, they report
the number of started, committed, and rolled back transactions,
respectively.

Closes #7583

NO_DOC=bug fix

(cherry picked from commit 70969d1d)

Currently, if a snapshot contains some correct entries, but doesn't
include system spaces, Tarantool crashes with segmentation fault, or
for Debug build: void diag_raise(): Assertion `e != NULL' failed.
This happens because memtx_engine_recover_snapshot returns -1, while
diag is not set. Let's panic instead of a crash.

Closes #7800

NO_DOC=bugfix

(cherry picked from commit e0a9aed4)

0xc1 isn't a valid MsgPack header, but it was allowed by mp_check.
As a result, msgpack.decode crashed while trying to decode it.
This commit updates the msgpuck library to fix this issue.

Closes #7818

NO_DOC=bug fix

(cherry picked from commit ced405af)

Closes #7797

NO_DOC=security fix
NO_TEST=security fix

(cherry picked from commit dd7d46af)

getenv() return values cannot be trusted, because an attacker might set
them. For instance, we shouldn't expect, that getenv() returns a value
of some sane size.

Another problem is that getenv() returns a pointer to one of
`char **environ` members, which might change upon next setenv().

Introduce a wrapper, getenv_safe(), which returns the value only when
it fits in a buffer of a specified size, and copies the value onto the
buffer. Use this wrapper everywhere in our code.

Below's a slightly decorated output of `grep -rwn getenv ./src --include
*.c --include *.h --include *.cc --include *.cpp --include *.hpp
--exclude *.lua.c` as of 2022-10-14.
`-` marks invalid occurences (comments, for example),
`*` marks the places that are already guarded before this patch,
`X` mars the places guarded in this patch, and
`^` marks places fixed in the next commit:

NO_WRAP
```
* ./src/lib/core/coio_file.c:509:	const char *tmpdir = getenv("TMPDIR");
X ./src/lib/core/errinj.c:75: const char *env_value = getenv(inj->name);
- ./src/proc_title.c:202: * that might try to hang onto a getenv() result.)
- ./src/proc_title.c:241:	* is mandatory to flush internal libc caches on getenv/setenv
X ./src/systemd.c:54: sd_unix_path = getenv("NOTIFY_SOCKET");
* ./src/box/module_cache.c:300: const char *tmpdir = getenv("TMPDIR");
X ./src/box/sql/os_unix.c:1441: azDirs[0] = getenv("SQL_TMPDIR");
X ./src/box/sql/os_unix.c:1446: azDirs[1] = getenv("TMPDIR");
* ./src/box/lua/console.c:394: const char *envvar = getenv("TT_CONSOLE_HIDE_SHOW_PROMPT");
^ ./src/box/lua/console.lua:771: local home_dir = os.getenv('HOME')
^ ./src/box/lua/load_cfg.lua:1007: local raw_value = os.getenv(env_var_name)
X ./src/lua/init.c:575: const char *path = getenv(envname);
X ./src/lua/init.c:592: const char *home = getenv("HOME");
* ./src/find_path.c:77: snprintf(buf, sizeof(buf) - 1, "%s", getenv("_"));
```
NO_WRAP

Part-of #7797

NO_DOC=security

(cherry picked from commit b86395ff)

This patch fixes the issue described in issue #5310 when the tuple
format has more fields than the space format. This solution is more
general than the solution in 89057a21.

Follow-up #5310
Closes #4666

NO_DOC=bugfix

(cherry picked from commit 5a38c5c9)

Since our diagnostics use the `__FILE__` macro, they provide absolute
paths, which is kind of redundant and inconsistent: replace them with
relative ones.

As for debugging information, replacing absolute paths with relative ones
also requires an extra command to tell the debugger where to find the
source files, which is not convenient for developers: provide a new
`DEV_BUILD` option (turned off by default), which replaces absolute paths
with relative ones in debugging information if turned off.

Strip the prefix map flags from compiler flags exported to tarantool via
`src/trvia/config.h`.

Closes #7808

NO_DOC=<verbosity>
NO_TEST=<verbosity>

(cherry picked from commit 256da010)

Fixes bug with determination of debug build (tarantool/test-run#352).

NO_CHANGELOG=<internal submodule version update>
NO_TEST=<submodule version update>
NO_DOC=<submodule version update>

(cherry picked from commit e554e481)

Setting hardening compiler flags is used in three places: default build,
static build and enterprise build — refactor it into a separate module.

Follow-up e6abe1c9

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

(cherry picked from commit dd51a2fa)

e6abe1c9 passes compiler flags to dependencies via a `<project>_build` macro
parameter, which is, firstly, inconvenient, and, secondly, as a result, not
all dependencies got the required compiler flags passed: use global
variables instead and pass these flags to skipped dependencies.

Follow-up e6abe1c9

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

(cherry picked from commit c6794757)

`add_compile_flags` macro relied on undefined behaviour: instead of
unsetting the `_lang` and `_flag` variables, it tried to unset their
values — fix this.

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

(cherry picked from commit 4aa9f154)

Information about test code coverage is useful on code review.
This patch enables code coverage job in default ci.

NO_CHANGELOG=not a user-visible change
NO_DOC=not a user-visible change
NO_TEST=ci

(cherry picked from commit 66e388e8)

If we raise different errors in case of entering an invalid password and
entering the login of a non-existent user during authorization, it will
open the door for an unauthorized person to enumerate users.
So let's unify raised errors in the cases described above.

Closes #tarantool/security#16

NO_DOC=security fix

(cherry picked from commit 5c62f01b)

The _vfunc system space is the sysview for the _func system space.
However, the _vfunc format is different from the _func format. This
patch makes the _vfunc format the same as the _func format.

Closes #7822

NO_DOC=bugfix

(cherry picked from commit 707da125)

We used to ignore timezone difference (in `tzoffset`) for
datetime subtraction operation:

```
tarantool> datetime.new{tz='MSK'} - datetime.new{tz='UTC'}
---
- +0 seconds
...

tarantool> datetime.new{tz='MSK'}.timestamp -
           datetime.new{tz='UTC'}.timestamp
---
- -10800
...
```

Now we accumulate tzoffset difference in the minute component
of a resultant interval:

```
tarantool> datetime.new{tz='MSK'} - datetime.new{tz='UTC'}
---
- -180 minutes
...
```

Closes #7698

NO_DOC=bugfix

(cherry picked from commit 0daed8d5)

We did not take into consideration the fact that
as result of date/time arithmetic we could get
in a different timezone, if DST boundary has been
crossed during operation.

```
tarantool> datetime.new{year=2008, month=1, day=1,
			tz='Europe/Moscow'} +
	   datetime.interval.new{month=6}
---
- 2008-07-01T01:00:00 Europe/Moscow
...
```

Now we resolve tzoffset at the end of operation if
tzindex is not 0.

Fixes #7700

NO_DOC=bugfix

(cherry picked from commit 6ca07285)

In PR #7791 the coverage stuff is slightly updated. The changes are
related to excluding some directories where the coverage will not be
calculated. The patch itself is straightforward enough, but I can see
the problems with uploading the results to coveralls.io:

NO_WRAP
    Using lcov file: ./coverage.info
    FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

    <--- Last few GCs --->

    [2683200:0x327c000]    27836 ms: Scavenge 2033.4 (2038.7) -> 2033.2 (2044.2) MB, 6.8 / 0.0 ms  (average mu = 0.258, current mu = 0.243) allocation failure
    [2683200:0x327c000]    27854 ms: Scavenge 2036.7 (2044.2) -> 2037.1 (2044.4) MB, 15.6 / 0.0 ms  (average mu = 0.258, current mu = 0.243) allocation failure
    [2683200:0x327c000]    27871 ms: Scavenge 2037.2 (2044.4) -> 2036.7 (2055.9) MB, 17.1 / 0.0 ms  (average mu = 0.258, current mu = 0.243) allocation failure

    <--- JS stacktrace --->

    ==== JS stack trace =========================================

        0: ExitFrame [pc: 0x140dc19]
    Security context: 0x2481918808d1 <JSObject>
        1: encode(aka encode) [0x23ec5e514a11] [/opt/actions-runner/_work/_actions/coverallsapp/github-action/v1.1.2/node_modules/qs/lib/utils.js:~118] [pc=0x193ecd3d5d82](this=0x35d34e4804b1 <undefined>,0x1e5895380119 <Very long string[60387428]>)
        2: stringify(aka stringify) [0x23ec5e5174c9] [/opt/actions-runner/_work/_actions/coverallsapp/github-action/v1.1.2/n...

     1: 0xa1a640 node::Abort() [/opt/actions-runner/externals/node12/bin/node]
     2: 0xa1aa4c node::OnFatalError(char const*, char const*) [/opt/actions-runner/externals/node12/bin/node]
     3: 0xb9a62e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/opt/actions-runner/externals/node12/bin/node]
     4: 0xb9a9a9 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/opt/actions-runner/externals/node12/bin/node]
     5: 0xd57c25  [/opt/actions-runner/externals/node12/bin/node]
     6: 0xd582b6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/opt/actions-runner/externals/node12/bin/node]
     7: 0xd64b75 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/opt/actions-runner/externals/node12/bin/node]
     8: 0xd65a25 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/opt/actions-runner/externals/node12/bin/node]
     9: 0xd670cf v8::internal::Heap::HandleGCRequest() [/opt/actions-runner/externals/node12/bin/node]
    10: 0xd15b35 v8::internal::StackGuard::HandleInterrupts() [/opt/actions-runner/externals/node12/bin/node]
    11: 0x1071176 v8::internal::Runtime_StackGuard(int, unsigned long*, v8::internal::Isolate*) [/opt/actions-runner/externals/node12/bin/node]
    12: 0x140dc19  [/opt/actions-runner/externals/node12/bin/node]
NO_WRAP

I have tried to use the latest version of the action (1.1.3), but I have
got the same issue.

It looks like it is a `coverallsapp/github-action` issue due to issue
in JavaScript due to inefficient amount of memory. The corresponding
bug [1] was filed against the repo.

So I suggest switching to the console utility `coveralls-lcov`. I did
some internal testing and it works fine.

[1] https://github.com/coverallsapp/github-action/issues/133

Fixes tarantool/tarantool-qa#278

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit 761574e3)

This patch fixed the assertion when JOIN uses index of unsupported type.

Closes #5678

NO_DOC=bugfix

(cherry picked from commit fd780129)

Currently if a non-string type is passed to luaT_key_def_set_part,
lua_tolstring returns null-pointer type_name, which is passed to
a string formatting function in diag_set.

Closes #5222

NO_DOC=bugfix

(cherry picked from commit 5215f3f3)

Don't accept an empty string or leading part of "str" or "num" as a
valid field type.

Closes #5940

NO_DOC=Partial field types weren't documented

Co-authored-by: Alexander Turenko <alexander.turenko@tarantool.org>
(cherry picked from commit 2dbaf9c2)

Since the function is actually an eval, by default there should
be no execute access right in public role.

Closes tarantool/security#14

NO_DOC=bugfix

(cherry picked from commit 815788c8)

By design after patching update.lua, introducing new data version,
a developer needs to update bootstrap.snap, which would have the
new data version.

Generation of bootstrap.snap is a bit tricky, there's a manual:
https://github.com/tarantool/tarantool/wiki/How-to-generate-new-bootstrap-snapshot

This commit presents a simple script that automates the process.
Just run it in tarantool build directory and check its output.
Check --help for script options.

Follow-up of tarantool/security#14

NO_DOC=internal tool
NO_TEST=internal tool
NO_CHANGELOG=internal tool

(cherry picked from commit 0922816e)

Fix a simple typo that caused the problem.

Closes #7645

NO_DOC=bugfix

(cherry picked from commit 6e450424)

This patch introduces new rules to determine type of NULLIF() built-in
function.

Closes #6990

@TarantoolBot document
Title: New rules to determine type of result of NULLIF

The type of the result of NULLIF() function now matches the type of the
first argument.

(cherry picked from commit 805cbaa7)