In case the space format is changed after inserting tuples, the new
format contains new fields, and a new index is created that works with
those new fields, it is possible to get an assertion during SELECT. This
problem was fixed in this path.

Closes #5183

NO_DOC=bugfix

The async transactions should always wait for LSN from the master after
switch to replica.

Closes tarantool/tarantool-qa#274

NO_CHANGELOG=testing
NO_DOC=testing

The bug was fixed in https://github.com/tarantool/msgpuck/pull/24.

This commit just bumps the submodule.

Closes https://github.com/tarantool/security/issues/18

NO_DOC=bug fix

Add a forgotten changelog entry.

Follow-up https://github.com/tarantool/security/issues/17

NO_DOC=changelog
NO_TEST=changelog

decimal_unpack() and hence mp_decode_decimal() didn't check length of
digits array, overflowing the output structure.

Fix that and a couple of typos in comments as well.

Closes https://github.com/tarantool/security/issues/17

NO_DOC=security
NO_CHANGELOG=security

Appeared that ASAN build could return nil values from child
process executed via popen. Which sporadically produces
error messages:

  not ok 1 ..console_debugger_session.test_interactive_debugger_session
  #   ...ntool/test/app-luatest/console_debugger_session_test.lua:11:
  # attempt to index local 's' (a nil value)
  #   stack traceback:

Due to used timeouts we are ok to receive nil at one of a loop
cycle - we simply repeat read to popen object. So handle this
case gracefully.

NO_DOC=bugfix
NO_CHANGELOG=internal

SQL select1:1-8.4 test didn't have any rationale in modern tarantool SQL
and had been turned off since 2017.

Closes: #5737
NO_DOC=test
NO_CHANGELOG=test

All the heavy lifting was done in the previous commit that dropped
the syslog formatter. This commit just removes the checks that forbid
using syslog with json.

Closes #7860

@TarantoolBot document
Title: JSON log format can now be used with syslog

The configuration reference says that syslog is incompatible with
the JSON log format:

https://www.tarantool.io/en/doc/latest/reference/configuration/#confval-log_format

This isn't true anymore. If JSON is used with syslog, then a JSON
message body is appended to the syslog header.

See https://github.com/tarantool/tarantool/issues/7860.

All messages written to syslog must have a header, which contains the
current time, message severity, and process identity, so we have a
special formatter function for syslog. As a result, the syslog log
destination is incompatible with the json log format, which has its own
formatter function. This is confusing, because there's nothing that
prevents us from writing json in the log entry body to syslog - we just
need to prepend the message with a proper header.

As a preparation for befriending json and syslog, let's get rid of the
syslog formatter function and instead write the header right in log_vsay
in case logs are written to syslog. We just need to strip the duplicate
information from the log entry in say_format_plain so as not to print
the time and pid twice.

Needed for #7860

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

We have three places in say.c where we peform exactly the same steps to
obtain the current time. Let's add a helper function to avoid code and
comments duplication.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

It's a global althoug static variable so better use say_ prefix so as
not to mix it with a local variable.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Some internal modules have been recently copied to luatest repo [1,2,3]
and now they can be safely removed, and the corresponding functionality
from luatest can be used instead.

Affected modules:

- test/luatest_helpers/misc.lua
- test/luatest_helpers/fiber.lua
- test/luatest_helpers/proxy/*

[1] https://github.com/tarantool/luatest/pull/247
[2] https://github.com/tarantool/luatest/pull/248
[3] https://github.com/tarantool/luatest/pull/255

Closes tarantool/luatest#238
Closes tarantool/luatest#251

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

Bump test-run to new version with the following improvements:

- Bump luatest to 0.5.7-6-gee1f5c1 [1]
- Fix detection of debug builds [2]

[1] https://github.com/tarantool/test-run/pull/354
[2] https://github.com/tarantool/test-run/pull/356

Part of tarantool/luatest#238
Part of tarantool/luatest#251

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

tarantool.debug.getsources() used to provide
lua sources only for libserver lua modules,
and was not providing sources for src/box/lua/*
modules.

Now this code works:
```lua
local tnt = require 'tarantool'
local code1 = tnt.debug.getsources('box/session')
local code1 = tnt.debug.getsources('@builtin/box/session..lua')
```

Closes #7839

NO_DOC=bugfix
NO_CHANGELOG=later

Refactor `tarantool_debug_getsource` implementation to
use mhash-based `mh_strnptr_find_str` function for
fast lookup in the hash map using given key.

Established set of function wrapping this functionality:

- `builtin_modcache_init` for creating map;
- `builtin_modcache_free` as destructor for map;
- `builtin_modcache_put` for adding new pair of
  (modname, lua code) into this hash map;
- `builtin_modcache_find` for fast lookup given
   short module name.

NB! We do not use keys in a form `@builtin/%s.lua` - we use
short module names instead.

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring, older tests should still work

`dlsym` is known to be buggy in FreeBSD. See #7640.

NO_DOC=internal
NO_CHANGELOG=internal

See [1] for some details on why the code for log configuration needs
some care. In short log config validity checks are spread thru many
places, on some code paths we use one checks and on other code paths
other checks. And we make repetitive validity checks many times in
runtime on single configure call.

We can also reuse code for setting default values, checking options
type and resetting values to default.

- As a side effect of refactoring one can now reset values to default thru
  `log.cfg()` so now `log.cfg()` is on par with `box.cfg` in this respect.

- This patch also drops conversion `log_level` from string to number.

  Before (shorten):

    tarantool> box.cfg{log_level='warn'}
    tarantool> box.cfg.log_level
    - info
    tarantool> log.cfg.level
    - 5

  Also:
    tarantool> log.cfg{level='info'}
    tarantool> log.cfg.level
    - 5
    tarantool> box.cfg{}
    tarantool> box.cfg.log_level
    - 5

  After patch if `log_level`/`level` is given as string than it is saved
  and returned as string too. I guess it should not affect users but looks
  more handy.

- Also fixed issue with inconsistent setting `log_nonblock` thru
  `box.cfg()` and `log.cfg()`. In former case `nil` means setting default
  depending on logger type. In the latter case `nil` meant setting
  `nonblock` to `false`.

- Also patch fixes #7447.

Closes #7447.

[1] PR for this refactoring
https://github.com/tarantool/tarantool/pull/7454

NO_DOC=refactoring/tiny API improvemnent

It is same as luaL_dosting but also sets fiber diag.

Part of #7447

NO_DOC=internal
NO_CHANGELOG=internal

If a complex (tuple) foreign key is added along with a new field name
that participates in that foreign key, the foreign key does not work
correctly. This happens because the new name of a local field is added
to new_space->def->dict only in ModifySpace::alter, while before that,
new_space->def->dict points to the old dictionary with old names (see
ModifySpace::alter_def). So when local_field_no is initialized earlier
in alter_space_do -> space_create -> tuple_constraint_fkey_init, it's
unable to find the new field number in the old dictionary.

Fix this by moving `new_def->dict = alter->old_space->def->dict;` from
ModifySpace::alter_def() to ModifySpace::alter(). Note that, as before,
we refer to the old dictionary from the new space (just put new names
into it), because that dict is referenced by existing tuple formats.

Closes #7652

NO_DOC=bugfix

The test is expected the replication is broken by the time of the
server switch. Apparently, switch won the race, so the expectation
is changed for waiting.

Closes https://github.com/tarantool/tarantool-qa/issues/271

NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing

It was disabled in the commit ccfef5fd ("memtx: ignore
slab_alloc_granularity < machine word size") to postpone addressing
unaligned objects issues.

Add PACKED attribute to inform compiler of unaligned objects. We also
need to introduce `struct stailq_entry_ptr` to be able to inform compiler
that `last` pointer unaligned too.

Closes #7422

NO_DOC=internal
NO_CHANGELOG=internal

`say_logrotate` does not rotate logs synchronously. It posts tasks to
coio which executes them in it's pool thread. On application exit we
destroy logs calling `log_destroy`. This function waits for rotate task
to finish because if it will free resources immediately then unfinished
rotate task can have use-after-free issues. Waiting crashes because at
this moment event loop is not running which is required for
`fiber_cond_wait` to work.

Note that if there is no crash then we will hang forever waiting in
`log_destroy` because event loop is not running and
`log_rotate_async_cb` will never be executed.

Let's use mutexes and conditions instead. It solves both problems with
crash and hang. Thanks Vladimir Davydov for idea.

Fixes #4450

NO_DOC=bugfix

Fixes tarantool/tarantool-qa#282

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Fixes tarantool/tarantool-qa#281

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Fix the following warning:

NO_WRAP
    Node.js 12 actions are deprecated. For more information see:
    https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.
    Please update the following actions to use Node.js 16: actions/checkout
NO_WRAP

Fixes tarantool/tarantool-qa#279

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Fix the following warning:

NO_WRAP
    Node.js 12 actions are deprecated. For more information see:
    https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.
    Please update the following actions to use Node.js 16: actions/upload-artifact
NO_WRAP

Fixes tarantool/tarantool-qa#280

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

This commit fixes BEGIN, COMMIT, and ROLLBACK counters in the box.stat()
output. Before this commit, they always showed 0. Now, they report
the number of started, committed, and rolled back transactions,
respectively.

Closes #7583

NO_DOC=bug fix

Currently, if a snapshot contains some correct entries, but doesn't
include system spaces, Tarantool crashes with segmentation fault, or
for Debug build: void diag_raise(): Assertion `e != NULL' failed.
This happens because memtx_engine_recover_snapshot returns -1, while
diag is not set. Let's panic instead of a crash.

Closes #7800

NO_DOC=bugfix

We need this option to implement a basic MT-safe read view C API
in the EE repository.

Closes #7815

NO_DOC=internal
NO_CHANGELOG=internal
NO_TEST=will be added to the EE repository

We need the options to optionally disable tuple decompression in memtx.

Needed for #7815

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

memtx_read_view_opts contains the only flag include_temporary_tuples,
which is set to read_view_opts.enable_temporary_spaces. Let's drop this
intermediary structure and pass read_view_opts to the allocator instead.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

needs_FOO looks awkward for a flag name. Let's rename the flags to
enable_FOO. The new names will also look better in conjunction with the
disable_decompression flag that will be added in the scope of #7815.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

0xc1 isn't a valid MsgPack header, but it was allowed by mp_check.
As a result, msgpack.decode crashed while trying to decode it.
This commit updates the msgpuck library to fix this issue.

Closes #7818

NO_DOC=bug fix

So one can easily check current box status.

NO_DOC=minor change

Closes #7255

cord() can't be NULL anymore - it's created on demand if not created
explicitly with cord_create().

Follow-up #7814

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

We're planning to introduce a basic C API for user read views (EE-only).
Like all other box C API functions, the new API functions will use the
existing box error C API for reporting errors. The problem is that
a read view created using C API should be usable from user threads
(started with the pthread lib) while the box error C API doesn't work
in user threads, because those threads don't have the cord pointer
initialized (a diagnostic area is stored in a cord object).

To address this issue, let's create a new cord object automatically on
first use of cord() if it wasn't created explicitly. Automatically
created object is destroyed at thread exit (to achieve that, we use
the C++ RAII concept).

Closes #7814

NO_DOC=The C API documentation doesn't say anything about threads.
       Let's keep it this way for now. We're planning to introduce
       a new C API to work with threads in C modules. We'll update
       the doc when it's ready.

Closes #7797

NO_DOC=security fix
NO_TEST=security fix

getenv() return values cannot be trusted, because an attacker might set
them. For instance, we shouldn't expect, that getenv() returns a value
of some sane size.

Another problem is that getenv() returns a pointer to one of
`char **environ` members, which might change upon next setenv().

Introduce a wrapper, getenv_safe(), which returns the value only when
it fits in a buffer of a specified size, and copies the value onto the
buffer. Use this wrapper everywhere in our code.

Below's a slightly decorated output of `grep -rwn getenv ./src --include
*.c --include *.h --include *.cc --include *.cpp --include *.hpp
--exclude *.lua.c` as of 2022-10-14.
`-` marks invalid occurences (comments, for example),
`*` marks the places that are already guarded before this patch,
`X` mars the places guarded in this patch, and
`^` marks places fixed in the next commit:

NO_WRAP
```
* ./src/lib/core/coio_file.c:509:	const char *tmpdir = getenv("TMPDIR");
X ./src/lib/core/errinj.c:75: const char *env_value = getenv(inj->name);
- ./src/proc_title.c:202: * that might try to hang onto a getenv() result.)
- ./src/proc_title.c:241:	* is mandatory to flush internal libc caches on getenv/setenv
X ./src/systemd.c:54: sd_unix_path = getenv("NOTIFY_SOCKET");
* ./src/box/module_cache.c:300: const char *tmpdir = getenv("TMPDIR");
X ./src/box/sql/os_unix.c:1441: azDirs[0] = getenv("SQL_TMPDIR");
X ./src/box/sql/os_unix.c:1446: azDirs[1] = getenv("TMPDIR");
* ./src/box/lua/console.c:394: const char *envvar = getenv("TT_CONSOLE_HIDE_SHOW_PROMPT");
^ ./src/box/lua/console.lua:771: local home_dir = os.getenv('HOME')
^ ./src/box/lua/load_cfg.lua:1007: local raw_value = os.getenv(env_var_name)
X ./src/lua/init.c:575: const char *path = getenv(envname);
X ./src/lua/init.c:592: const char *home = getenv("HOME");
* ./src/find_path.c:77: snprintf(buf, sizeof(buf) - 1, "%s", getenv("_"));
```
NO_WRAP

Part-of #7797

NO_DOC=security

This patch fixes the issue described in issue #5310 when the tuple
format has more fields than the space format. This solution is more
general than the solution in 89057a21.

Follow-up #5310
Closes #4666

NO_DOC=bugfix

Since our diagnostics use the `__FILE__` macro, they provide absolute
paths, which is kind of redundant and inconsistent: replace them with
relative ones.

As for debugging information, replacing absolute paths with relative ones
also requires an extra command to tell the debugger where to find the
source files, which is not convenient for developers: provide a new
`DEV_BUILD` option (turned off by default), which replaces absolute paths
with relative ones in debugging information if turned off.

Strip the prefix map flags from compiler flags exported to tarantool via
`src/trvia/config.h`.

Closes #7808

NO_DOC=<verbosity>
NO_TEST=<verbosity>