Since commit f6ea7180 ("Try to load several variants of libssl.")
the digest module uses an internal version of SHA1. Back then, we didn't
link the OpenSSL library. Instead, we tried to load it dynamically.
Since on some distributions the library could be missing, it was decided
to implement an internal version of SHA1, see #405.

However, since commit 59a55740 ("Link against libssl and libcrypto.
Issue #1382") we link the OpenSSL library unconditionally so there's no
need in having an internal implementation of SHA1. Let's drop it and
switch the digest module to the version of SHA1 implemented by the
crypto module using OpenSSL.

Part of #7987

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

We added an internal version of SHA1, because initially we didn't link
the OpenSSL library. However, since commit 59a55740 ("Link against
libssl and libcrypto. Issue #1382") we do so there's no need in it.

Part of #7987

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

Scramble is used only by the chap-sha1 authentication mechanism,
which has a proper API. There's no need in a standalone scramble lib.

Part of #7987

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

Authentication salt is expected to be 20 byte long (SCRAMBLE_SIZE).
However, IPROTO sends 36 bytes (IPROTO_SALT_SIZE). Let's add a few
assertions ensuring that we never pass less than 20 bytes to
authentication methods.

Part of #7987

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

box_cfg_xc() initializes a lot of storage-specific modules like
gc, engine, schema, etc.

These modules were supposed to be destroyed by box_free(), but it
didn't happen if an exception was thrown during box_cfg_xc().

For example, memtx engine might be initialized and even have a
snapshot thread running, then SIGTERM would arrive, and the
process would try to free all initialized modules skipping the
box-ones.

It sometimes resulted in fiber_free() being called, all fiber
stacks freed, and then the snapshot thread ending. Its join was
already started in memtx_engine_wait_checkpoint(), it already had
an on-exit callback prepared. The callback was stored on the stack
of one of the deleted fibers in a struct cord_on_exit. The struct
was garbage and the callback invocation would crash.

The patch makes the storage-specific modules be destroyed even if
box.cfg{} didn't end but those modules were initialized.

The test is a bit more complicated. Just testing that the process
can finish without crashes wouldn't be possible:

- Luatest server:stop() doesn't waitpid() the child process. As a
  result the exit code is just lost. Even if the process would
  crash somewhere in the end, it wouldn't be detected.

- On MacOS a crash in any non-main thread is ignored because all
  threads but the main one have all the signals blocked (see
  tt_pthread_create()). On Mac apparently the blockage of
  SIGSEGV/SIGABRT or some other signal causes their turning into
  mach exceptions which are simply not handled atm. As a result,
  the "crashed" thread is just frozen and the main thread ends
  fine (in a test for this particular bug) (#8023).

Closes #7743

NO_DOC=bugfix

It is a wrapper around pthread cancel and join. It was repeated
many times and was dangerous, because left cord.id set. An
accidental attempt to cord_join/cojoin() such cord would lead to
UB then.

The patch introduces a function which encapsulates the blocking
cancellation. It is going to be used in a next patch to count the
number of cords in the process. Which in turn is needed for a new
test.

The counter is atomic in case some cords would be created not by
the main cord.

There are now also more sanity checks against accidental attempts
to join the same cord twice.

Needed for #7743

NO_DOC=internal
NO_CHANGELOG=internal

Added tarantool.compat option that allows yaml.encode() to encodes all
strings containing a newline in a more convenient block scalar style.
This is a breaking change, programs that rely on precise lyaml encoding
may fail if the new behavior is selected.

Closes #3012
Requires #7060
Requires #8007

@TarantoolBot document
Title: YAML formatter improvement

Now yaml.encode() can encode strings with a newline in block scalar style,
see https://yaml.org/spec/1.2-old/spec.html#style/block/literal and
https://github.com/tarantool/tarantool/wiki/compat%3Ayaml_pretty_multiline

old:
```
tarantool> compat.yaml_pretty_multiline = 'old'
---
...

tarantool> return "Title: xxx\n- Item 1\n- Item 2\n"
---
- 'Title: xxx

  - Item 1

  - Item 2

  '
...

```

new:
```
tarantool> compat.yaml_pretty_multiline = 'new'
---
...

tarantool> return "Title: xxx\n- Item 1\n- Item 2\n"
---
- |
  Title: xxx
  - Item 1
  - Item 2
...

```

For unknown reason in upstream lua_cjson '/' was escaped
while according to the standard [rfc4627] it is unnecessary and is
questionably compatible with other implementations.

It was decided that the change will be introduced using
tarantool.compat (gh-7000). The patch adds json_escape_forward_slash
option to compat and its logic in lua_cjson and msgpuck.

Requires #7060
Requires #8007
Fixes #6200
See also #7000

@TarantoolBot document
Title: new compat option json_escape_forward_slash

In the new behavior forward slash is not escaped in `json.encode()`
and msgpack:

```
tarantool> compat.json_escape_forward_slash = 'new'
---
...

tarantool> json.encode('/')
---
- '"/"'
...

tarantool> compat.json_escape_forward_slash = 'old'
---
...

tarantool> json.encode('/')
---
- '"\/"'
...
```

The `recursive_rmdir` can start execution while tarantool instance
started by the tarantoolctl is in the process of shutdown. As a result
the instance pid file can be available at the name collection, but by
the time of its removal the instance deletes it by itself.

NO_DOC=test
NO_CHANGELOG=test

Closes #3211

NO_TEST=Documentation

@TarantoolBot document
Title: Per-module log level
Root document: https://www.tarantool.io/en/doc/latest/reference/configuration/#logging

Since version 2.11 it is possible to specify the log level for each
module separately, e.g.:

box.cfg {
    log_level = 5,
    log_modules = {
        ['foo.bar'] = 1,
        expirationd = 'debug'
    }
}

The name of a module is determined automatically during the execution
of `require('log')` in the module's source code. The name is derived
from its filename, including a part of the path. Also it is possible
to create a logger with a custom name by using `log.new()`.

---

Root document: https://www.tarantool.io/en/doc/latest/reference/reference_lua/log/
New function: log.new(name)
Creates a new logger with a custom name.

Parameter `name`:
Type: string
Optional: false

Example:
box.cfg{log_level='error', log_modules={my_module='info'}}
log = require('log')
my_log = log.new('my_module')
my_log.info('Info')

Now the name of a module, from which the logging function was called,
is determined automatically during the execution of `require('log')`
in the module's source code. This is implemented by overriding the
built-in `require` function.

Part of #3211

NO_DOC=See next commit
NO_CHANGELOG=See next commit

Co-authored-by: AnastasMIPT <beliaev.ab@tarantool.org>

Now it is possible to specify the log level for each module separately,
e.g.:

box.cfg {
    log_level = 5,
    log_modules = {
        ['foo.bar'] = 1,
        expirationd = 'debug'
    }
}

Part of #3211

NO_DOC=See next commit
NO_CHANGELOG=See next commit

Co-authored-by: AnastasMIPT <beliaev.ab@tarantool.org>

It allows to create a new instance of a log module, with a custom name:
local my_log = require('log').new('my_module')

The name is added to the log message after fiber name:
YYYY-MM-DD hh:mm:ss.ms [PID]: CORD/FID/FIBERNAME/MODULENAME LEVEL> MSG

Part of #3211

NO_DOC=See next commit
NO_CHANGELOG=See next commit

Co-authored-by: AnastasMIPT <beliaev.ab@tarantool.org>

This commit introduces an abstraction for the authentication code so
that one can easily add new methods. To add a new method, one just needs
to define a set of authentication callbacks in a struct auth_method and
register it with auth_method_register.

The IPROTO_AUTH and _user.auth formats were initially designed with
extensibility in mind: both take the authentication method name
(currently, only 'chap-sha1' is supported) so no changes to the schema
are required.

Note that although 'chap-sha1' is now implemented in its own file
src/box/auth_chap_sha1.c, we don't merge src/scramble.c into it.
This will be done later, in the scope of #7987.

Since we call authentication plug-ins "methods" (not "mechanisms"),
let's rename BOX_USER_FIELD_AUTH_MECH_LIST to BOX_USER_FIELD_AUTH while
we are at it. Anyway, the corresponding field of the _user system space
is called 'auth' (not 'auth_mech_list').

Closes #7986

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

We will need to store some extra data in the user_def struct to support
different authentication mechanisms. Let's introduce convenient helpers
for allocating and freeing this struct so that we don't have to patch
all the places in the code where it's allocated or freed when we extend
the struct.

While we are at it, switch to grp_alloc, shorted the license text, and
replace include guards with pragma.

Needed for #7986

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

C++ features aren't really needed there. Let's drop exceptions and
convert to C to simplify further development.

While we are at it, shorten the License text, replace ifdef guards
with pragma, and clean up the include list.

Needed for #7986

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This option enables emission of the DWARF CFI (Call Frame Information)
directives to the assembler. It is enabled by default on most compilers,
but on GCC 7 for AArch64 and older it wasn't [1], so turn it on explicitly.
When enabled, the compiler emits .cfi_* directives that are required for
the stack unwinding, and defines __GCC_HAVE_DWARF2_CFI_ASM.

Part of #7960

[1] https://gcc.gnu.org/pipermail/gcc-patches/2018-March/495549.html

NO_DOC=internal
NO_CHANGELOG=internal
NO_TEST=Tested by test/app-luatest/fiber_parent_backtrace_test.lua

This feature was disabled due to a crash in libunwind. After commit
5b08d71a ("libunwind: use latest release v1.6.2 as a base") the
crash is gone.

Closes #7960

NO_DOC=internal
NO_CHANGELOG=internal
NO_TEST=<Leak backtraces are tested by test/unit/fiber.cc; the crash in
libunwind is observable on test/box-luatest/gh_6310_grant_rw_access_on_
_session_settings_space_to_public_role_test.lua>

An encoder function may only fail if it fails to allocate memory from
fiber->gc. The amount of memory it allocates is fairly small. It's used
as an extra stack to return the encoded data. This should never fail,
because there's no hard limit for runtime memory (and we're not planning
to ever add one). Let's make all encoder functions return void and drop
the untested checks for OOM.

To avoid confusion, let's make xrow_header_encode, xrow_encode_dml, and
xrow_to_iovec return the number of io vectors in a new out argument,
because a return value of type int is usually used to indicate an error.

See also #3534

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

* Prepare workspace before actions/checkout to prevent
  git-related workflow failures.
* Replace tarantool/actions/cleanup, because it solves
  just the same problem.

Resolves tarantool/tarantool-qa#285

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Since we are going to introduce zlib compression in enterprise version,
we need to find zlib package there, so let's search for zlib package
only if it has not been found before.

NO_TEST=cmake
NO_CHANGELOG=no behavior changes
NO_DOC=no behavior changes

It was decided not to proceed with support for these distros.

Closes tarantool/infra#162

NO_DOC=ci/cd
NO_TEST=ci/cd
NO_CHANGELOG=ci/cd

In order to send IPROTO packets using `box_iproto_send` (#7897) we need to
have some session identifier source in the C API: for now, export an analog
of `box.session.id`.

Closes #7895

@TarantoolBot document
Title: Document export of current session's identifier to C API
For the API description and usage examples, see:
* [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#0900ff9cb6b148378ce0b185d3f628b9);
* tarantool/tarantool#7895.

Currently, we stop without checking the authentication request body if
the specified user doesn't exist. As a result, one can check if a user
exists by sending a malformed authentication request: for an existing
user, the server will reply with ER_INVALID_MSGPACK error while if the
user doesn't exist, the error will be set to ER_CREDS_MISMATCH. To fix
this, let's parse the authentication request body even if the user
doesn't exist.

Closes https://github.com/tarantool/security/issues/21

NO_DOC=bug fix

The triggers are supposed to run on any authentication attempt,
successful or not. Without it, the admin may not notice a malefactor
enumerating user names.

Closes #8017

NO_DOC=bug fix

The guest can't have a non-empty password - it's explicitly prevented by
the _user trigger callback:

https://github.com/tarantool/tarantool/blob/2ed06651c317f2d46631b0d55e4b88f2c8a6c0a0/src/box/alter.cc#L2930-L2936

So there's no need to check the guest user password in authenticate.

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

1. Fixed modification of a variable passed to the logging function
   (both log formats)

   tarantool> a = {foo = 'bar', file = 'c://autorun.bat'}
   tarantool> log.info(a)
   [...]
   tarantool> a
   (before) - foo: bar
    (after) - foo: bar
              file: c://autorun.bat

2. Fixed the drop of fields with reserved internal names (plain log format)

   tarantool> log.info({foo = 'bar', pid = 666})
   (before) {"foo":"bar"}
    (after) {"foo":"bar","pid":666}

3. Now if 'message' field is absent, it is set to the first field of the
   table (JSON log format)

   tarantool> log.info({'Hello, world', key = 'value'})
   (before) {[...], "1":"Hello, world", "key":"value", [...]}
    (after) {[...], "message":"Hello, world", "key":"value", [...]}

4. Fixed assertion on a table without string keys (JSON log format)

   If a Lua table, passed to say(), contains only integer indexes,
   json.encode() will encode it as an array rather than a map, thereafter
   say_format_json() will produce improperly formatted JSON string, or get
   assertion failure. Fix it by encoding all kinds of tables as maps.

Closes #3853
Closes #7955

NO_DOC=Mostly bug fixes, the format of the JSON log is not documented.

Turn LuaJIT off for this test on AArch64 in the scope of the issue [1].

Also increase grep_log_bytes. Currently the message is 235 byte long,
which is pretty close to 256. If one more item is added to the traceback,
the test will fail.

```
C> Potentially long select from space 'test_memtx' (512)
 stack traceback:
        builtin/box/schema.lua:2528: in function 'check_select_safety'
        builtin/box/schema.lua:2545: in function 'select'
        eval:1: in main chunk
        [C]: at 0x556e8ffd5c
```

[1] Related to #8011
Closes tarantool/tarantool-qa#264

NO_DOC=test fix
NO_CHANGELOG=test fix

In the scope of this commit check for instruction/line hooks for
compiled code is turned on. As a result a special pad is emitted to the
trace validating whether any debug hook is fired while this trace is
being executed.

Closes #7762

NO_DOC=internal

Reviewed-by: Sergey Kaplun <skaplun@tarantool.org>
Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

* Fix narrowing of unary minus.
* Don't compile math.modf() anymore.
* OSX/ARM64: Fix external unwinding.
* Fix build with busybox grep.
* BSD: Fix build with BSD grep.
* OSX/ARM64: Disable unwind info.
* ARM64: Reorder interpreter stack frame and fix unwinding.
* OSX/ARM64: Disable external unwinding for now.
* OSX: Fix build by hardcoding external frame unwinding.
* Cleanup and enable external unwinding for more platforms.
* test: remove TAP side effects in getmetrics tests
* test: relax JIT setup in misc.getmetrics test
* test: relax JIT setup in lj-430-maxirconst test
* GC64: enable sysprof support

Closes #6096
Closes #6976
Closes #7919
Relates to #781
Relates to #7762
Part of #7230

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

Update luacheckrc to suppress all warnings from lua loop vars with `_`
prefix. This allows to make loops more verbose.

Closes #8006

NO_CHANGELOG=luacheckrc fix
NO_DOC=luacheckrc fix
NO_TEST=luacheckrc fix

Bump test-run to new version with the following improvements:

- Bump luatest to 0.5.7-18-gcb28d93 [1]

[1] tarantool/test-run#364

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

This patch will enable the CodeQL analyzer for PRs to the
master branch.

NO_TEST=no code changes
NO_DOC=no code changes
NO_CHANGELOG=no code changes

As a followup to #6200 and a result of #6912, Tarantool compatibility
module was implemented to simplify introduction of debatable behavior
changes.

Closes #7000
Needed for #6200
See also #6912

@TarantoolBot document
Title: Tarantool compatibility module

Compat module is basically a global options table with additional
verbose interface and helper functions. There are tree stages of
changing a behavior:
 - old behavior by default
 - new behavior by default
 - new behavior is frozen and the old behavior is removed

on first two stages user can toggle options via interface and change the
behavior according to his needs, on the last stage old behavior is
removed from codebase and option is marked as obsolete.
As compat is a global instance, options can be hardcoded into it or
added in runtime e.g. by external module.

There should be a link to tarantool.wiki/compat on the doc page. There
are pages for each existing option with detailed description, list of
known compatibility issues and some tips on detecting problems in codebase.

Tutorial can be found here:
https://github.com/tarantool/tarantool/wiki/compat%3Atutorial

Full API description is here (private):
https://www.notion.so/tarantool/tarantool-compat-introduce-compatibility-module-to-tarantool-8cb33e23932b490384c05f1cdfbf7baa

Extend the interface of `luamp_encode` to accept a string to integer hash
table for translation of first-level `MP_MAP` keys.

Needed for #7897

NO_CHANGELOG=internal feature
NO_DOC=internal feature

Hash tables mapping strings to integers often occur: factor out definition
of `mh_strnu32_t` —  it will be reused in scope of #7897.

Add `mh_strnu32_find_str`: a helper for looking up a key in the
`mh_strnu32_t` hash table, similar to `mh_strnptr_find_str`.

Needed for #7897

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Add `__index` metamethod and `get` method to `msgpack.object` which perform
indexation of MsgPack data stored in the object similar to `tuple` from
`box.tuple`: `__index` resolves collisions in favor of
`msgpack.object` methods, whereas `get` ignores methods.

Closes #7898

@TarantoolBot document
Title: Document `msgpack.object:index`
For the API description and usage exmaples, see:
* [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#73867e0f33c844f3822e62acdc4e7814);
* tarantool/tarantool#7898.

Some internal modules have been recently copied to luatest repo [1,2]
and now they can be safely removed, and the corresponding functionality
from luatest can be used instead.

Affected modules:

- test/luatest_helpers/server.lua

[1] tarantool/luatest#258
[2] tarantool/luatest#266

Closes tarantool/luatest#239

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

Bump test-run to new version with the following improvements:

- Bump luatest to 0.5.7-17-g1387aa8 [1]

[1] tarantool/test-run#360

Part of tarantool/luatest#239

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

Now we base on some unreleased state of libunwind. This is by itself not
a good practice. Yet the main motivation is that in the current version of
libunwind fast path for backtrace detection on x86_64 does not work.
I guess this is because of libunwind/libunwind@400b3f819ad4 (" x86_64: Stop
aliasing RSP and CFA"). See libunwind/libunwind#440. Fortunately this
commit is not present it the latest release.

Using fast or slow path has a great impact on performance of debug build
where collecting fiber gc allocation backtrace is turned on by default.
It surely depends on frequency and pattern of allocation. Test
sql/delete3 depends on backtrace performance most dramatically. On some
installations collecting backtraces slowed down the test up to 10 times.

If fast path is available then collecting backtrace does not affect
performance in a noticeable way.

I propose not to keep autotools products in the libunwind fork repo as
it is done previously. LOG_CONFIGURE is removed because it somehow
incompatible with using && in CONFIGURE_COMMAND command and not critical
to the build.

Also add autotools build dependencies for packpack package specs.
Currently not all packpack images have autotools preinstalled so this is
required for build to success. Anyway we'b better have precise build
requirements in build specs.

Follow-up #5665

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal