There must be a couple of rules:
* foreign key from non-temporary space to temporary space must be
  forbidden since after restart all existing links will be broken.
* foreign key from non-local space to local space must be forbidden
  on any replica all existing can be broken.

This patch implements the rules.

Closes #8936

NO_DOC=bugfix

The test fails on osx_debug quite frequently with the following error:

NO_WRAP
> box/before_replace.test.lua                                     [ fail ]
>
> Test failed! Result content mismatch:
> --- box/before_replace.result	Mon Aug  7 10:36:06 2023
> +++ /tmp/t/rejects/box/before_replace.reject	Mon Aug  7 10:42:03 2023
> @@ -899,6 +899,7 @@
>  ...
>  for i = 1,17 do gen_inserts() end
>  ---
> +- error: fiber slice is exceeded
>  ...
>  test_run:cmd('restart server test')
>  s = box.space.test_on_schema_init
> @@ -906,7 +907,7 @@
>  ...
>  s:count()
>  ---
> -- 1
> +- 0
>  ...
>  -- For this test the number of invocations of the before_replace
>  -- trigger during recovery multiplied by the amount of return values
> @@ -921,7 +922,8 @@
>  -- the value is only increased by 1, and this change is visible only in memory.
>  s:get{1}[2] == 1 + 16999 + 17000 + 1 or s:get{1}[2]
>  ---
> -- true
> +- error: '[string "return s:get{1}[2] == 1 + 16999 + 17000 + 1 o..."]:1: attempt to
> +    index a nil value'
>  ...
>  test_run:cmd('switch default')
>  ---
NO_WRAP

Let's try to increase the max fiber slice up to 15 seconds to avoid
that, like we do in other potentially long tests. (The default value
is 1 second.)

NO_DOC=test fix
NO_CHANGELOG=test fix

This should suppress the following coverity issues:

https://scan7.scan.coverity.com/reports.htm#v39198/p13437/fileInstanceId=146712118&defectInstanceId=18978766&mergedDefectId=1563095
https://scan7.scan.coverity.com/reports.htm#v39198/p13437/fileInstanceId=146712113&defectInstanceId=18978750&mergedDefectId=1563094

While we are at it, let's use space_by_id instead of space_cache_find
because read view creation is a rare operation affecting all spaces so
caching the last space by id doesn't make any sense.

NO_DOC=code health
NO_TEST=code health
NO_CHANGELOG=code health

In #7309 a truncation of a space that was referenced by foreign
key from some other space was prohibited.

It appeared that this solution is too bothering since a user can't
truncate a space even if he truncated referring space before that.

Fix it by allowing space truncate if referring spaces are empty.
Also allow drop of the primary index in the same case with the
same reason: logically the index along with all space data is
not needed for consistency if there's no referring data.

Note that by design space truncate is implemented quite similar
to space drop. Both delete all indexes, from secondary to primary.
Since this patch allows deletion of the primary index (which is
the action that actually deletes all data from the space), this
patch changes the result of space drop too: the space remains
alive with no indexes, while before this patch it remained alive
with no secondary indexes but with present primary. In both cases
the behaviour is quite strange and must be fixed in #4348. To
make tests pass I had to perform drop in box.atomic manually.

Closes #8946

NO_DOC=bugfix

Closes #8926

@TarantoolBot document
Title: Document `inherit_fds` option of `popen.new`

The new option takes an array of file descriptor numbers that should be
left open in the child process if the `close_fds` flag is set. If the
`close_fds` flag isn't set, the option has no effect because in this
case none of the parent's file descriptors are closed anyway.

The option may be useful for establishing a communication channel
between a Tarantool instance and a process spawned by it with
`popen.new`, for example, using `socket.socketpair`.

Allocation from malloc and the fiber region never fail so we can use x*
variants to simplify code and improve coverage.

NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

@TarantoolBot document
Title: Remove `OutOfMemory` error from `popen.new` and `popen.read`

These functions don't raise the `OutOfMemory` exception anymore so all
mentions of it should be removed from the documentation:

 - https://www.tarantool.io/en/doc/latest/reference/reference_lua/popen/#popen-new
 - https://www.tarantool.io/en/doc/latest/reference/reference_lua/popen/#popen-read

Remove hard-coded compiler version for the `test-release-asan` target
in the .test.mk file.

NO_DOC=make
NO_TEST=make
NO_CHANGELOG=make

This patch introduces all metrics configuration.

Part of #8861

NO_DOC=tarantool/doc#3544 links the most actual schema,
       no need to update the issue.

Bump the metrics submodule to 1.0.0-3-4865675c

NO_DOC=metrics submodule bump
NO_TEST=metrics submodule bump
NO_CHANGELOG=metrics submodule bump

All LuaJIT related LSan warnings were suppressed in the scope of the
commit 985548e4 ("asan: suppress all
LSAN warnings related to LuaJIT"), since all compiler flags tweaks were
enclosed in LuaJIT CMake machinery. As a result of the commit in LuaJIT
submodule tarantool/luajit@a86e376 ("build: introduce LUAJIT_USE_ASAN
option") ASan and LSan support has been finally added to LuaJIT runtime,
so it was decided to remove LSan suppressions for LuaJIT functions.
Unfortunately, it was not so easy as it looked like.

At first, Lua global state is not closed properly at Tarantool instance
exit (see <tarantool_free> in src/main.cc and <tarantool_lua_free> in
src/lua/init.c for more info), so LSan false-positive leaks are detected
(for more info, see #3071). Hence, the original LSan suppression for
lj_BC_FUNCC is returned back (temporarily) until the aforementioned
issue is not resolved.

Furthermore, the internal LuaJIT memory allocator is not instrumented
yet, so to find any memory faults it's worth building LuaJIT with system
provided memory allocator (i.e. enable LUAJIT_USE_SYSMALLOC option).
However, again, since Tarantool doesn't finalize Lua universe the right
way, so running Tarantool testing routine with LUAJIT_USE_SYSMALLOC
enabled generates false-positive LSan leaks. Return back here to enable
LUAJIT_USE_SYSMALLOC, when #3071 is resolved.

Last but not least, the default value of fiber stack size is 512Kb, but
several tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT submodule
(e.g. some cases with deep recursion in errors.lua or pm.lua) have
already been tweaked according to the limitations mentioned in #5782,
but the crashes still occur while running LuaJIT tests with ASan support
enabled. Experiments once again confirm the notorious quote that "640 Kb
ought to be enough for anybody".

Anyway, LuaJIT tests are added to <test-release-asan> target in .test.mk
and LUAJIT_TEST_ENV is extended with required ASan and LSan options.

Follows up #5878

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

It is convenient to access configuration using `config:get()` from the
application script (`app.file` or `app.module`).

However, before this commit, it was not possible, because the
configuration was not considered as applied before the application
script is loaded.

Now, the script loading is moved into the post-apply phase.

The resulting sequence of steps on startup/reload is the following.

* collect configuration information (from all the sources)
* <if the previous step is failed, set `check_errors` status and break>
* apply the configuration (call all the appliers)
* <if the previous step is failed, set `check_errors` status and break>
* <set the new successful status: `ready` or `check_warnings`>
* call post-apply hooks (including application script loading)
* <if the previous step is failed, set `check_errors` status and break>
* <set the new successful status: `ready` or `check_warnings`>

I would like to briefly comment the changes in the tests.

* `app_test.lua`: added the check for the new behavior (call
  config:get() from the app script)
* `appliers_test.lua`: fixed the applier call (`.apply` ->
  `.post_apply`)
* `config_test.lua`: fixed status observed in the app script
  (`*_in_progress` -> `ready`)

Part of #8862

NO_DOC=reflected in https://github.com/tarantool/doc/issues/3544

The declarative config has the `app` section with following three
parameters:

* `app.file` -- the path to the script file
* `app.module` -- the same, but the script is searched using the Lua
  search paths (and loaded using `require`)
* `app.cfg` -- a user provided configuration (arbitrary map)

This commit adds a few success/failure cases. The goal is to have simple
test examples to add more ones easier in a future.

Implemented several test helpers for typical scenarios that can be used
outside of the application script test.

Part of #8862

NO_DOC=testing improvements
NO_CHANGELOG=see NO_DOC

The usual environment configuration source is useful for parametrized
run:

```
TT_MEMTX_MEMORY=<...> tarantool --name <...> --config <...>
```

However, sometimes a user may need to set a default value, which doesn't
rewrite one that is provided in the configuration. Now, it is possible
to do using the environment variables with the `_DEFAULT` suffix.

```
TT_MEMTX_MEMORY_DEFAULT=<...> tarantool --name <...> --config <...>
```

This feature may be especially useful for wrappers that run tarantool
internally with some default paths for data directories, socket files,
pid file. We likely will use it in the `tt` tool.

Part of #8862

NO_DOC=added into https://github.com/tarantool/doc/issues/3544 manually

It simplifies code reusage in the following commit, which adds the
second environment config source.

Part of #8862

NO_DOC=no user-visible changes, just code refactoring
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

Bump test-run to new version with the following improvements:

- luatest: fix detect tarantool crash at exit [1]
- Fix bug when lua script name truncated by dot [2]
- Raise an error and log it if test timeouts are set incorrectly [3]
- Pin PyYAML version to 5.3.1 [4]
- Add ability to set path to executable file [5]
- Migrate tarantoolctl from tarantool repository [6]
- Fix test-run crash when default server is crashed [7]
- Disable reproduce content printing [8]

[1] tarantool/test-run@be693d1
[2] tarantool/test-run@a6405f1
[3] tarantool/test-run@d34ecb0
[4] tarantool/test-run@704420e
[5] tarantool/test-run@0a70001
[6] tarantool/test-run@ad43d8f
[7] tarantool/test-run@b31329e
[8] tarantool/test-run@31f0ced

NO_DOC=test
NO_TEST=test
NO_CHANGELOG=test

* ci: introduce testing workflow with sanitizers
* build: introduce LUAJIT_USE_ASAN option
* test: introduce test:done TAP helper
* memprof: remove invalid assertions
* ci: clean up workflow for exotic builds

Closes #5878

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump
NO_CHANGELOG=LuaJIT submodule bump

The is_old compat option method is more reliable than comparing
the current option value to 'old' directly because it may be set to
'default', in which case we also have to check the default value.

Follow-up #8807

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The is_new and is_old methods are the same for all compat options so
they should be defined in a metatable. A good thing about this change
is that it removes is_new and is_old from serialization:

* Before:

NO_WRAP
  tarantool> require('compat').yaml_pretty_multiline
  ---
  - is_new: 'function: 0x4175d6e8'
    is_old: 'function: 0x4175d790'
    brief: |
      Whether to encode in block scalar style all multiline strings or ones
      containing "\n\n" substring. The new behavior makes all multiline string output
      as single text block which is handier for the reader, but may be incompatible
      with some existing applications that rely on the old style.

      https://tarantool.io/compat/yaml_pretty_multiline
    current: default
    default: new
  ...
NO_WRAP

* After:

NO_WRAP
  tarantool> require('compat').yaml_pretty_multiline
  ---
  - current: default
    brief: |
      Whether to encode in block scalar style all multiline strings or ones
      containing "\n\n" substring. The new behavior makes all multiline string output
      as single text block which is handier for the reader, but may be incompatible
      with some existing applications that rely on the old style.

      https://tarantool.io/compat/yaml_pretty_multiline
    default: new
  ...
NO_WRAP

To achieve that, we have to remove the option name from the usage error
message but it seems to be okay because such errors shouldn't happen in
practice and the error message is clear enough to figure out what went
wrong.

Follow-up #8807

NO_DOC=refactoring
NO_CHANGELOG=refactoring

Run release build LTO testing inside a Docker container created from the
`tarantool/testing:ubuntu-jammy-clang16` image with Clang 16 installed.

Closes #318

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Run release build testing inside a Docker container created from the
`tarantool/testing:ubuntu-jammy-clang16` image with Clang 16 installed.

Closes #317

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

Add code owners for CI-related script and
for github automation directory.

NO_CHANGELOG=no code changes
NO_TEST=no code changes
NO_DOC=no code changes

The instance config schema was modified in commit 8ee2b0d8
("config: start singleton instance in RW by default"): the `database.rw`
(boolean) parameter was replaced by the `database.mode` parameter (enum
of `ro` and `rw`).

However, in the same commit, the default mode of an instance that is the
only one in its replicaset was modified: it starts in the read-write
mode by default. As result, the parameter is actually unneeded in the
minimal configuration example.

These changes were not reflected in the 'instance not found' error
message, which contains the minimal configuration example. It is fixed
here.

Part of #8862
Follows up #8810

NO_DOC=nothing to document, it is just an error message
NO_CHANGELOG=see NO_DOC
NO_TEST=there is a test case in config-luatest/reload_test.lua that
        triggers the given error; it doesn't needs updating, because it
        checks the start of the error message

This patch reverts the temporary fix introduced in commit
53c94bc7 ("test: disable Lua JIT in
app-luatest/http_client_test") since the issues with invalid traces
generation for vararg functions are resolved, so JIT machinery can be
enabled back then.

Follows up #8718
Relates to #8516

NO_DOC=test
NO_CHANGELOG=test

In order to improve tests of Tarantool core
assign dedicated team to perform review of each
and every change.

NO_CHANGELOG=no code changes
NO_TEST=no code changes
NO_DOC=no code changes

The following box.cfg options were described in instance_config
with defaults similar to ones in box.cfg:
* auth_type
* auth_delay
* disable_guest
* password_lifetime_days
* password_min_length
* password_enforce_uppercase
* password_enforce_lowercase
* password_enforce_digits
* password_enforce_specialchars
* password_history_length

Part of #8861

NO_DOC=tarantool/doc#3544 links the most actual schema,
       no need to update the issue.

Applier thread uses lsregion to allocate the messages for tx thread. The
messages are freed upon return to the applier thread using a
corresponding lsr_id.

Due to a typo, one of the lsregion allocations was made with a postfix
increment of lsr_id instead of the prefix one. Essentially, part of a
new message was allocated with an old lsr_id, and might be freed early
by a return of a previous message.

Fix this.

Closes #8848

NO_DOC=bugfix
NO_TEST=covered by asan in #8901
NO_CHANGELOG=bugfix

The test has two groups, in front of each of which, a cluster is
created. Sometimes, it fails on the first test in every group:
either ro_reason is explicitly checked and it equals to `orphan`,
not nil, or we try to write to a still read-only instance and get
the error, that it's orphan.

The problem is the fact, that we don't wait until the connections
are established in a cluster. Let's add waiting for full mesh
in cluster creation.

Closes tarantool/tarantool-qa#320

NO_CHANGELOG=test fix
NO_DOC=test fix

Modify `actions/pack-and-deploy/action.yml` for creating static build
packages and deploying them to the repository via RWS service. Modify
`workflows/static_build_packaging.yml` for using this action.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Set variable `OUTPUT_DIR` in the .pack.mk file to the directory where
static build packages will be stored. This directory will be used for
the deployment.

Add target `deploy-static` for deploying deb and rpm packages via RWS.

NO_DOC=make
NO_TEST=make
NO_CHANGELOG=make

Update artifact paths in the static_build_packaging.yml workflow due to
fixing the issue with `OUTPUT_DIR` in the previous commit.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Before these changes, it was impossible to define a custom absolute
path for `OUTPUT_DIR` when calling the static-build/make_packages.sh
script. The build failed due to missing directory inside the container.
Also, set the default value for `OUTPUT_DIR` to the absolute path from
where the static-build/make_packages.sh script runs instead of relative
path to the static-build directory.

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

Currently if tarantool exits during relay's final join stage,
corresponding thread isn't terminated. This causes the flakiness
of the replicaset_ro_mostly.test.lua.

Let's reuse the same relay, in which subscribe cord is running, for
the final join stage. This way the cord will be cancelled during
replication_free().

Closes #8082

NO_DOC=not user-visible
NO_TEST=fix flaky test
NO_CHANGELOG=not user-visible

Co-authored-by: Sergey Petrenko <sergepetrenko@tarantool.org>

The normalize_position() internal function is called by select/pairs to
prepare a position specified by the caller in the options.after argument
to be passed to C over FFI. The function encodes the position passed as
a table or a tuple in a temporary buffer obtained with cord_ibuf_take().
The problem is the cord buffer is already taken by the calling function
(select/pairs) to encode the search key. As a result, cord_ibuf_take()
allocates a new temporary buffer with malloc.

Let's avoid this extra buffer allocation by passing the taken buffer to
normalize_position(). Let's also remove the out arguments because they
are always set to iterator_pos and iterator_pos_end and instead set
iterator_pos and iterator_pos_end directly. To make this behavior clear,
we rename normalize_position() to iterator_pos_set(). We also remove
box.error() from this function and instead make it return true/false so
that the caller can release the buffer on failure.

perf/lua/box_select.lua test results:

                      base          patched
              get_0    159      153 (-  3%)
              get_1    243      243 (+  0%)
           select_0    227      227 (+  0%)
           select_1    343      336 (-  2%)
           select_5   2323     2305 (-  0%)
          select_10   6832     6866 (+  0%)
     select_after_0    687      558 (- 18%)
     select_after_1    746      524 (- 29%)
     select_after_5   2917     2704 (-  7%)
    select_after_10   5033     4979 (-  1%)
 select_fetch_pos_0    239      233 (-  2%)
 select_fetch_pos_1    418      413 (-  1%)
 select_fetch_pos_5   2492     2496 (+  0%)
select_fetch_pos_10   5121     5179 (+  1%)
            pairs_0    320      314 (-  1%)
            pairs_1   1060     1054 (-  0%)
            pairs_5   1466     1459 (-  0%)
           pairs_10   3749     3913 (+  4%)
      pairs_after_0    788      670 (- 14%)
      pairs_after_1   1813     1611 (- 11%)
      pairs_after_5   2764     2638 (-  4%)
     pairs_after_10   4150     4098 (-  1%)
          tuple_pos    116      117 (+  0%)

NO_DOC=perf improvement
NO_TEST=perf improvement

The test runs get, select, pairs space methods with various arguments in
a loop and prints the average method run time in nanoseconds (lower is
better).

Usage:

  tarantool box_select.lua

Output format:

  <test-case> <run-time>

Example:

  $ tarantool box_select.lua --pattern 'get|select_%d$'
  get_0 155
  get_1 240
  select_0 223
  select_1 335
  select_5 2321

Options:

  --pattern <string>  run only tests matching the pattern; use '|'
                      to specify more than one pattern, for example,
                      'get|select'
  --read_view         use a read view (EE only)

Apart from the test, this patch also adds a script that compares test
results:

  $ tarantool box_select.lua --pattern get > base
  $ tarantool box_select.lua --pattern get > patched1
  $ tarantool box_select.lua --pattern get > patched2
  $ tarantool compare.lua base patched1 patched2
         base          patched1          patched2
  get_0   149       303 (+103%)       147 (-  1%)
  get_1   239       418 (+ 74%)       238 (-  0%)

NO_DOC=perf test
NO_TEST=perf test
NO_CHANGELOG=perf test

This improves diff hunk name detection. Needed for checkpatch to
correctly detect if patched code belongs to a function.

NO_DOC=git
NO_TEST=git
NO_CHANGELOG=git

This patch introduces all flightrec options.

Part of #8861

NO_DOC=Was already described before.

Part of #8861

NO_DOC=Was already described before.

This patch fixes an issue in generate_column_metadata(). Prior to this
patch, the number of variable-only expressions was counted incorrectly
when temporary memory was allocated on region to store their positions.
However, although this allocation was incorrect, this did not lead to
any problems due to the specifics of the region allocations.

This patch fixes this by removing the temporary memory allocation.

Closes #8763

NO_DOC=no user-visible changes
NO_TEST=no user-visible changes
NO_CHANGELOG=no user-visible changes

It is possible for interval to have days, hours, minutes and seconds
larger than INT_MAX (or less than INT_MIN). Before this patch, msgpack
decoding had failed to parse intervals with msgpack int64 and uint64.
int64_t should be enough to store any value allowed for datetime
intervals.

Closes #8887

NO_DOC=small bug fix