The built-in modules overriding functionality will be implemented as a
Lua loader. It must be in effect, when built-in modules are loading, so
setup the loaders earlier.

This commit doesn't change any user visible behavior, but it marks a
minor problem with a filename assigned to the loaded Lua code (seen in
error messages and `debug.getinfo()`) to fix it later.

Part of #7774

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST

The `minifio` module is created specifically to use in code that needs
file operations, but works before the `fio` module is initialized.

The loaders module will be loaded at early loading stage to make the
override loader working from very start and allow to override most of
the built-in modules.

Part of #7774

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST

The minifio module is supposed to be used in tarantool's code that
potentially works at early initialization stage.

The loaders.lua module needs several file manipulation functions and
it'll be moved to the early initialization stage (see the previous
commit for the idea).

Next commits will use minifio instead of fio in loaders.lua and will
move minifio and loaders at the early loading stage.

The list of changes in the functions:

* fio.pathjoin() uses `error(<...>, 0)` to don't prefix the error
  message with `internal.minifio.lua` -- a user is unlikely interested
  how fio is split to files internally.
* An obsoleted comment from fio.abspath() is dropped (it is obsoleted by
  commit 583e8ba2 ("fio: new approach for error handling")).

cwd(), pathjoin() and abspath() are moved to `minifio` and exposed from
`fio`. dirname() is duplicated: `minifio`'s implementation uses
ffi.new(), `fio`'s implementation uses cord_ibuf_take()/cord_ibuf_put().

## Alternatives considered

In brief: no really good option, but the implemented one looks as the
best one.

### Copy to loaders.lua

First option is to copy those four functions right into loaders.lua. It
requires a slight adaptation:

- cwd(): reimplement using ffi.
- pathjoin(): just copy.
- abspath(): copy and use own cwd() and pathjoin().
- dirname(): copy and use ffi.new() instead of
  cord_ibuf_take()/cord_ibuf_put().

All the functions would be maintained in two places that is error-prone.
It would be good to reduce amount of copies of the same/similar code.

### Add minifio.lua

Okay, let's assume we created own file for the four functions. Can we
avoid adding minifio.c for cwd()? There are two ways (spoiler: both are
bad).

* We can initialize the C part of `fio` before minifio.lua and use it
  here. But it would be highly counter-intuitive to use `fio` in
  `minifio`.
* We can reimplement cwd() on ffi, but we'll need to duplicate abspath()
  to use `minifio`'s cwd() function.

We definitely need `minifio.c`.

### Add minifio.c and minifio.lua

This option is implemented in this commit. The only function that is
duplicated is dirname().

### Mitigate dirname() duplication

There are two ways.

It is possible to add a dependency on the `buffer` module and use
cord_ibuf_take()/cord_ibuf_put() in `minifio`. However it would mean
that `buffer` shouldn't depend on other built-in modules. It is logical
for `minifio`, which is created specifically to load at early stage, but
counter-intuitive for `buffer`. If `buffer` will depend on another
built-in module in a future, a developer will need to play around
'right' order of loading.

We can also move dirname()'s implementation to the C part of `minifio`,
use C's cord_ibuf_take()/cord_ibuf_put() and expose the function as to
`minifio` as well as to `fio`.

The latter is a good option, but I don't bother much about the
copy-paste, because the function body has 7 SLoC.

### Move into Lua C API

We can implement all the four function using the Lua C API in fio.c and
initialize the C part of `fio` before loading of loaders.lua.

It would look more clean in some sense: all the file manipulation
functions are in the `fio` module. However it is also error-prone,
because nothing would stop a future developer to use some fio.foobar()
function, which is actually loaded after loaders.lua, before loading
Lua's part of `fio`.

An explicit splitting to early/usual stage looks safer.

Part of #7774

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST

This commit continues the series of preliminary commits for implementing
the built-in module overriding (so called dual-life modules, see #7774
for the problem statement).

The core idea of the future functionality is that if there is the
`override.foo` module on the filesystem, it automatically replaces the
built-in module `foo`.

This machinery will be implemented as an extra [loader][1], so it'll
work only after loading of the loaders. In turn, it means that we need
to load the loaders as early as possible to allow to replace all (or at
least most of) the built-in modules.

Tarantool loads built-in Lua code file-per-file and it is hard to
eliminate all dependencies from the src/lua/init.lua code.

This commit extracts the loaders code into its own file. Following
commits will eliminate dependencies on other built-in modules and move
loading of the loaders to the early stage.

The init.lua file contains initialization code from different domains
and it worth to split it further. I'll not do that in this series.

Part of #7774

[1]: https://www.lua.org/manual/5.1/manual.html#pdf-package.loaders

NO_TEST=no user visible changes
NO_CHANGELOG=see NO_TEST
NO_DOC=see NO_TEST

- Drop testing for macOS 11 since macOS 13 is now available

- Add missing testing for macOS 12:
  - debug build (x86_64)
  - debug, release, and static-cmake builds (aarch64)

- Add testing for macOS 13:
  - debug, release, release-lto, and static-cmake builds (x86_64)
  - debug, release, release-lto, and static-cmake builds (aarch64)

Closes #6739
Closes tarantool/tarantool-qa#301

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

With previous commit rawset is not required anymore because internally
we start to use raw_cfg.

Follows up #2867

NO_CHANGELOG=code health
NO_DOC=code health
NO_TEST=code health

Tarantool has a special table 'box.cfg' that includes configuration
parameters. User could view that table, but it's direct modification had
no effect - after assigning a new value it is actually "updated", but
actual value remains the same. Such behaviour is a counterintuitive for
our users and provides a bad experience.

Proposed patch change this behaviour: new value assigned to parameter
via direct access to table box.cfg raise an error.

Before the patch:

tarantool> box.cfg{}
<snipped>
tarantool> box.cfg.read_only=true
---
...

tarantool>

After the patch:

NO_WRAP
tarantool> box.cfg{}
<snipped>
tarantool> box.cfg.read_only=true
---
- error: 'builtin/box/load_cfg.lua:973: Use box.cfg{read_only = true} for update'
...

tarantool>
NO_WRAP

Closes #2867

@TarantoolBot document
Title: Document changed behaviour on setting options to box.cfg directly

Tarantool has a special table 'box.cfg' that includes configuration
parameters. User could view that table, but it's direct modification has
no effect - after assigning a new value it is actually "updated" but
actual value remains the same. Such behaviour is a counterintuitive for
our users and provides a bad experience.

Now new value assigned to parameter via direct access to table box.cfg
raise an error.

tls_construct_ctos_session_ticket() has a potential
NULL pointer dereference.

Closes tarantool/security#54

NO_DOC=security
NO_TEST=security
NO_CHANGELOG=security

Now, delete in ephemeral space is obviously incorrect - if we try to
delete a tuple, which is not present in index, NULL dereference will
happen. Fortunately, ephemeral spaces are used for internal purposes
only, so, most likely, this never happens. Let's fix this part not to
confuse code analyzers.

Closes https://github.com/tarantool/security/issues/38

NO_TEST=shouldn't normally happen
NO_CHANGELOG=shouldn't normally happen
NO_DOC=shouldn't normally happen

Added bounds check after conversion of a string key to int to avoid
potential out-of-bounds access.

Closes tarantool/security#45

NO_TEST=trivial
NO_CHANGELOG=internal
NO_DOC=internal

Sometimes, we only need to test static-build, e.g. when we apply a patch
to a third party sub-project. Let's introduce a new label to run the ci
checks faster in this case.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

We're going to add a whole bunch of them. Putting them all in
a sub-directory will help keeping the file tree organized.

Note, we have to update .gitignore so that the patches/ sub-directory
is ignored only at the top level (it's used by quilt).

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

This is useful for example for the analysis of performance complaints
from users, when they claim that one version of Tarantool is slower
than another, in fact comparing debug and release builds.

NO_DOC=minor change
NO_TEST=minor change

This update pulls the following commits

 * Constify mp_char2escape
   https://github.com/tarantool/msgpuck/commit/28a7421cf7fa538a0180c79bd9c12ee0dd8c12eb

   This is code cleanup.

 * Don't escape forward slash in mp_snprint
   https://github.com/tarantool/msgpuck/commit/e05a538d076509063240a00f2e703ede7f803a87

   This commit disables escaping of the forward slash character in
   mp_snprint, because the function produces JSON-like string and
   according to the JSON spec, we don't need to escape it.

Follow-up #8117

NO_DOC=submodule update
NO_TEST=submodule update
NO_CHANGELOG=minor change

This commit adds the json_escape_forward_slash variable and a tweak for
it that is now used by the compat module. The new variable configures
whether '/' is escaped when encoded in JSON.

Note, the old implementation was quite messy so we had to rework it:

 - Drop luaL_serializer::encode_escape_forward_slash. This was an
   undocumented serializer option implemented only by the JSON
   serializer and used only by the compat module. Now, we use the
   json_escape_forward_slash global tweak for all JSON serializers
   instead, because making this tweak configurable per serializer
   doesn't make much sense.

 - Don't use mp_char2escape for escaping JSON characters. Historically,
   we used this table defined in libmsgpuck in the json_escape utility
   function. It equals the escape table used by the JSON encoder so it
   looks more reasonable to use the JSON encoder escape table in
   json_escape. This commit moves the JSON encoder escape table to
   util.c and adds an inline utility function json_escape_char which is
   now used by the JSON encoder and json_escape.

 - Drop an extra JSON escape table with '/' escaped. We had two escape
   tables in JSON, one with escaped '/' and another with unescaped '/'.
   Duplicating tables like this is error-prone and ugly. Let's use one
   table with '/' unescaped and check the json_escape_forward_slash flag
   in json_escape_char instead. The cost of this check is negligible
   performance-wise.

This commit also drops the lua/compat.c source file, because it isn't
used anymore.

While we are at it, remove any mentions of MsgPack from the changelog
entry for the json_escape_forward_slash compat option, because it isn't
relevant anymore.

Closes #8117

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This commits adds the yaml_pretty_multiline variable and a tweak for it
that is now used by the compat module. The new variable configures
whether all multiline strings are encoded in the block scalar style or
only those that contain a '\n\n' substring.

Part of #8117

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This commit adds a tweak for the fiber_channel_close_mode variable
and makes the compat module use the new tweak instead of switching
the variable directly.

Note that we don't drop fiber_channel_set_close_mode function, because
it's still used in unit tests.

Part of #8117

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

This commit adds the new 'internal.tweaks' Lua module for toggling
internal tweaks. The module is implemented as a Lua table with the
__index and __newindex metamethods overridden, which makes it possible
to get/set a tweak value with a simple assignment, for example:

  tarantool> tweaks = require('internal.tweaks')
  ---
  ...

  tarantool> tweaks.fiber_channel_close_mode = 'graceful'
  ---
  ...

  tarantool> tweaks.fiber_channel_close_mode
  ---
  - graceful
  ...

(Note, currently there are no tweaks available; tweaks are added in the
following commits.)

Getting a value of an unknown tweak returns nil. Setting a value of an
unknown tweak raises 'No such option' error. Setting an invalid value
for an existing tweak raises 'Invalid value' error. The module also
implements __serialize and __autocomplete metamethods that return all
tweaks and their values in a table, for example:

  tarantool> tweaks
  ---
  - fiber_channel_close_mode: forceful
    yaml_pretty_multiline: false
    json_escape_forward_slash_default: true
  ...

Closes #7883
Needed for #8117

NO_DOC=internal
NO_CHANGELOG=internal

This commit adds an internal C API for registering tweaks. A tweak is
an object that provides a convenient setter/getter API for a global C
variable.

To register a tweak, use a TWEAK_XXX macro at the global level in
a C source file:

  static int my_var;
  TWEAK_INT(my_var);

The name of a tweak equals the name of the underlying variable
("my_var" in the example above).

To set/get a tweak value, use the tweak_set and tweak_get functions:

  struct tweak_value val;
  tweak_get("my_var", &val);
  val.ival = 42;
  tweak_set("my_var", &val);

The tweak_value struct is a variant that can contain one of three types:
int, bool, and string, one per each available tweak types:

  TWEAK_BOOL(bool_var);
  TWEAK_INT(int_var);
  TWEAK_ENUM(enum_name, enum_var);

The TWEAK_ENUM macro is special, as it also requires the name of the
enum type to be passed. When a enum tweak value is exported/imported,
it is converted to a string using the STR2ENUM macro.

It's also possible to iterate over all registered tweaks using the
tweak_foreach() function.

The tweak registry is a simple hash table mapping tweak names to
tweak objects, which in turn point to underlying variables. Tweaks
are registered at startup using the constructor function attribute.

Part of #7883
Needed for #8117

NO_DOC=internal
NO_CHANGELOG=internal

The test started hanging after commit d560fb3f ("Revert "replication:
set default replication_sync_timeout to 0"") because it still expected a
zero replication_sync_timeout. This wasn't caught by PR's full-ci for
some reason.

Follow-up #8223

NO_DOC=test fix
NO_CHANGELOG=test fix

Increase changelog verbosity, add examples of how to achieve old
behavior.

Follow-up #5272

NO_DOC=added in original commit
NO_TEST=changelog

Instead of setting the `replication_sync_timeout` default to new value
(0) unconditionally, add a compat option which will control the default
value. The compat option is named "box_cfg_replication_sync_timeout"
and is "old" by default, meaning default `replication_sync_timeout` is
300.

The user has to set it to "new" before the initial box.cfg call for the
default value to change to 0.

Follow-up #5272

NO_DOC=amended an existing doc issue, see https://github.com/tarantool/doc/issues/3295

This reverts commit 67cb4e4e.

The commit introduced new behaviour unconditionally and by default,
which's considered a breaking change. It was decided to add a compat
option for this change. So revert the commit.

NO_DOC=amended an existing doc issue manually. See https://github.com/tarantool/doc/issues/3295

Bitset index size calculation uses the cardinality of the 'flag' bitset,
but when the bitset index is empty, i.e., uninitialized, the
'flag' bitset is not allocated, hence we should simply return 0.

Closes #5809

NO_DOC=bugfix

After adding the autorequiring luatest [1,2], there is no need to use
the following approach now:

```
local t = require('luatest')
local g = t.group()

server:exec(function()
    local t = require('luatest') -- duplicate
    t.assert(...)
end)
```

Modern approach looks like:

```
local t = require('luatest')
local g = t.group()

-- `t` already available in the remote server
server:exec(function() t.assert(...) end)

-- also it works with any variable
local my_custom_t = require('luatest')

server:exec(function()
    my_custom_t.assert(...) -- already available
end)
```

[1] tarantool/luatest#277
[2] tarantool/luatest#289

Part of tarantool/luatest#233

NO_DOC=test fix
NO_TEST=test fix
NO_CHANGELOG=test fix

We didn't take into consideration the fact, that precision
value passed to control the width of nanoseconds part in
datetime_object:format could be more than maximum positive
value, integer may have. Currently it leads to segfault.

```
tarantool> require('datetime').new{}:format('%2147483648f')
```

We should check errno in order to find out, if overflow
occurs. The problem is the fact, that `width` variable must
have int type due to snprintf requirements ("%*d") and
strtol returns long. Errno won't be set if returned value
is in bounds [INT_MAX, LONG_MAX], but it will overflow
int resulting in inconsistent behavior.

So, let's save the result of strotl to the temp value.
If this value doesn't belong to the above-mentioned set,
or errno was set, we assign to `width` maximum value, it
may have: 9.

Closes tarantool/security#31

NO_DOC=bugfix

The function was replaced by luaT_newmodule(). See commit e3cf5a5d
("lua: add built-in module registration function") for the motivation
and details.

Now all luaL_register_module() usages are eliminated (including
Tarantool Enterprise source code) and we can safely drop it.

Part of #7774
Follows up PR #8173

NO_DOC=no user visible changes: the function is internal
NO_TEST=see NO_DOC
NO_CHANGELOG=see NO_DOC

To improve the stability of the tests, let's use unix sockets for iproto
connection instead of ports.

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

During creation of a function definition, an error condition for spaces is
diagnosed for wrong options: change it to its own error code.

While we are here, cover missing function definition error conditions.

Closes #7972

NO_DOC=bugfix

Bump test-run to new version with the following improvements:

- Bump luatest to 0.5.7-25-g9e117e6 [1]

[1] tarantool/test-run@7c77fcb

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

Fixing a bug with nodes in 'manual' election mode bumping the term
excessively revealed a hang in election_pre_vote test. Turns out the
test passed thanks to the previous buggy behaviour.

The following behaviour is expected: when a node is configured in manual
election mode, calling box.ctl.promote() on it should make it bump term
once, try to gather votes and fail on timeout.

Once the extra term bump on timeout was removed in commit 5765fdc4
("raft: fix 'manual' nodes bumping the term excessively"),
box.ctl.promote() without a quorum started hanging.

Let's return the correct behaviour: 'manual' nodes should transition
back to follower if an election timeout passes after the promotion
without any term outcome.

Enable the test_promote_no_quorum testcase of election_pre_vote test
back, since it's fixed now.

Follow-up #8168
Closes #8217

NO_DOC=bugfix
NO_CHANGELOG=changes not released behaviour

The `tarantool_version` symbol identifies the binary as Tarantool (see also
https://github.com/tarantool/tarantool/blob/f991f7c0be73558f0710f0af871d07e8bd506efe/tools/tarabrt.sh#L179-L180

).
It is not exported and thus can be optimized away by LTO — add it to the
exports list.

Closes #8129

Acked-by: Aleksandr Lyapunov <alyapunov@tarantool.org>

NO_CHANGELOG=<internal change>
NO_DOC=<internal change>
NO_TEST=<no convenient way to test devtools>

There used to be ';' as a default local End Of Stream in Lua, which
could be confusing for a human. Now the default local EOS is '' and it
can be configured per session with `\set output lua,local_eos=`.
Works on both local and remote consoles.

Closes #7031

@TarantoolBot document
Title: configure local End Of Stream symbol in Lua console

The old behavior was:
```
tarantool> \set output lua
true;
tarantool> 123
123;
```

Now default EOS is '', you can set it like this:
```
tarantool> \set output lua
true
tarantool> 123
123
tarantool> \set output lua,local_eos=#
true#
tarantool> 123
123#
tarantool> \set output lua,local_eos=
true
tarantool> 123
123
```

If you don't provide a symbol, it would be configured to ''.

Refactor parse_output function to handle several opts at once in
`\set output` command.

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring

Interactive console test helper could be used for remote connection too,
for that purpose prompt needs to be changed accordingly to connection
type.

This patch introduces getter and setter for the prompt. Prompt is
configured per session, so it is advised to create a new session
for each test case (e.g. with before_each()).

NO_CHANGELOG=test helper change
NO_DOC=test helper change

box.schema.upgrade is called automatically by box.cfg if the instance
is rw and box.cfg.replication is unset. This may be unexpected and
dangerous (the upgrade operation is irreversible). Let's drop auto
schema upgrade.

A look into the history of auto schema upgrade:

The feature was first introduced by commit 29a197cf ("box: auto
upgrade to 1.7.5") to automatically create the _truncate system space so
that space.truncate continues to work after update. Later on, we figured
out that auto-upgrade shouldn't be called if the instance is read-only,
see commit 6cfefcd4 ("Do not auto upgrade schema if read only"), or
configures replication, see commit 582a85d4 ("box: disable schema
auto upgrade for replication"). This turned this initially neat and
simple feature into a monstrous creature. We should have zapped it long
time ago.

Closes #8207

NO_DOC=auto schema upgrade wasn't documented

Bump test-run to new version with the following improvements:

- Bump luatest to 0.5.7-24-g8de38b3 [1] [2]
- Add ignore_zero option to get_vclock [3]

[1] https://github.com/tarantool/test-run/pull/370
[2] https://github.com/tarantool/test-run/pull/368
[3] https://github.com/tarantool/test-run/pull/367

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

The field will not be shown to a user in the interactive console.

NO_CHANGELOG=There are no releases with the field.
NO_DOC=The field is internal and so it is not present in the
       documentation (and shouldn't).

Reported-by: Mons Anderson <mons@cpan.org>

The memtx transaction manager MVCC invariant violation described in
c8eccfbb could actually lead to a bug described in #7394, since
`space:count` inherently relies on this invariant: add a test for this
case.

Closes #7394

NO_CHANGELOG=<bug was fixed in c8eccfbb>
NO_DOC=bugfix

A tuple update with the first operation creating a new field
somewhere deep in the tuple and the second operation trying to go
into that new field could crash. This happened because the route
branching function xrow_update_route_branch() missed this case.

It can be detected when see that the bar path is already fully
used (the next JSON token is END), and the new operation's path
is still not END.

Closes #8216

NO_DOC=bugfix