If you want to get information or get csw (Context SWitch) of some fiber
you need to call fiber.info(), but it creates table with information about
all the fibers. This patch introduces fiber_object:info() and
fiber_object:csw() - functions to solve this problem.

Closes #5799

@TarantoolBot document
Title: introduce fiber_object:info() and fiber_object:csw()
```
-- fiber_object:info() is the same as fiber.info(), but show information only
about one alive fiber.
-- fiber_object:csw() show csw (Context SWitch) of alive fiber.
```

This patch moves MEM types from the 'u32 flags' field to the new
'enum mem_type type' field. Now, we can be sure that only one type is
set for MEM. In addition, it is now easier to distinguish MAP and ARRAY
from VARBINARY, and this makes it easier to add extension types - UUID
and DECIMAL.

Closes #4906

After this patch, the mem_is_bin() function will return 'true' only if
the value that the MEM contains is of type VARBINARY. This patch also
adds the mem_is_bin_ext() function, which is used to check if a MEM
contains value of type VARBINARY or value of types that are currently
considered VARBINARY extensions - MAP and ARRAY.

Part of #4906

This patch adds proper initialization for the MEM, which is used in the
aggregate functions min() and max().

Part of #4906

Lua json module used to have a global buffer for all encodings. It
was reused by each next encode().

This was not correct, because during encode() might happen a GC
step, which might call encode() again and spoil the global buffer.

The same problem was already fixed for the global static buffer in
scope of #5632. Similarly to that time, the patch makes Lua json
module use cord_ibuf to prevent "concurrent" usage of the buffer
data. The global buffer is deleted.

According to a few microbenchmarks it didn't affect the perf
anyhow.

Core part of the patch is strbuf changes. Firstly, its destruction
is now optional, cord_ibuf can free itself on a next yield.
Secondly, its reallocation algorithm is kept intact - ibuf is used
as an allocator, not as the buffer itself. This is done so as not
to be too intrusive in the third party module which might need an
upgrade to the upstream in the future.

Closes #6050

Script used to run Jepsen tests uses Terraform to prepare test environment.
When I made this script I had a wrong assumption about working directory
used by Terraform. Due to this sometimes job with Jepsen tests
failed on cleanup with message:

```
Error: Could not load plugin

Plugin reinitialization required. Please run "terraform init".

Plugins are external binaries that Terraform uses to access and manipulate
resources. The configuration provided requires plugins which can't be located,
don't satisfy the version constraints, or are otherwise incompatible.

Terraform automatically discovers provider requirements from your
configuration, including providers used in child modules. To see the
requirements and constraints, run "terraform providers".

Failed to instantiate provider
"registry.terraform.io/terraform-providers/openstack" to obtain schema:
unknown provider "registry.terraform.io/terraform-providers/openstack"
```

Terraform documentation describes how Terraform uses working directories [1]
and there are at least to ways to resolve an issue. First one is using always
one directory before running terraform subcommands. Second one is using option
`-chdir` in all terraform commands [2].

1. https://www.terraform.io/docs/cli/init/index.html
2. https://www.terraform.io/docs/cli/commands/index.html#switching-working-directory-with-chdir

Closes #6089

Prior to this patch, UUID was not part of SCALAR. However, this should
be changed to comply with the RFC "Consistent Lua/SQL types".

Closes #6042

@TarantoolBot document
Title: UUID is now part of SCALAR

The UUID field type is now part of the SCALAR field type. This means
that now values of type UUID can be inserted into the SCALAR field, and
these values can participate in the sorting of the SCALAR fields. The
order is as follows: boolean < number < string < varbinary < uuid.

This allows `xlog` Lua module to decode appropriate
types into symbolic form.

For example with the patch we should see raft and
promote types in output.

 | $ tarantoolctl cat 00000000000000000004.xlog
 | ---
 | HEADER:
 |   lsn: 2
 |   group_id: 1
 |   type: RAFT
 |   timestamp: 1621541912.4588
 | BODY:
 |   0: 3
 |   1: 4
 | ---
 | HEADER:
 |   lsn: 1
 |   replica_id: 4
 |   type: PROMOTE
 |   timestamp: 1621541912.4592
 | BODY:
 |   2: 0
 |   3: 0
 |   83: 3

Fixes #6088

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Now test-run reports memory occupied by tests and their duration (see
[1] and [2]).

Aside of this, there is a tiny code health change around waiting for
`box.info.status` value: it should not affect a visible test-run
behaviour ([3]).

[1]: https://github.com/tarantool/test-run/issues/277
[2]: https://github.com/tarantool/test-run/issues/286
[3]: https://github.com/tarantool/test-run/issues/301

commit fbc04dfb
("static-build: use hashes of downloaded files") added hashes for
ExternalProject_Add() to avoid extra downloads on project rebuild.
Changes for libicu and libiconv were missed and this patch fixes it.

Follows-up #5761

Co-authored-by: Sergey Bronnikov <sergeyb@tarantool.org>

Since commit c9d88d5f ('Fix #984: add
jit.* library to the binary') all required modules implemented in Lua
are bundled (i.e. compiled into the executable as a C literal) into
Tarantool binary. While making Tarantool work on ARM64 platforms, it
turned out the arch-specific module (namely, jit/dis_arm64.lua) is not
bundled. Within this patch the missing sources are added and jit.dump
works fine on ARM64 hosts as a result.

Part of #5983
Relates to #5629
Follows up #984

Reviewed-by: Sergey Kaplun <skaplun@tarantool.org>
Reviewed-by: Sergey Ostanevich <sergos@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

* FFI/ARM64/OSX: Fix vararg call handling.
* OSX/iOS: Handle iOS simulator and ARM64 Macs.
* build: pass sysroot to MacOS SDK

Closes #6065
Closes #6066
Part of #5629
Relates to #5983

Workflow 'debug_coverage' produces and uploads results to 'coveralls.io'
web site. Message about it can be shown in PR within each run was done.
This patch adds the ability to send message in available PR otherwise it
is skipped. Also added 'coverage.info' file to artifacts list.

Found that used 'coverallsapp/github-action' action checks if the
trigger was 'pull_request' used to run the workflow [1]. And only in
this way it writes results message to PR. Previously 'pull_request'
trigger was blocked to avoid duplication with 'push' trigger. To make
able to run workflow by any single trigger and to send message to PR
if it exits, it was implemented the following logic:

  - run workflows on both triggers 'push' and 'pull_request';
  - for workflow with 'push' trigger check if PR exists then skip all
    jobs otherwise continue running;
  - for workflow with 'pull_request' trigger continue running;

To avoid of issue coverallsapp/github-action#55 [2], sources checkout
should be based on 2nd version and later.

Closes #5644

[1]: https://github.com/coverallsapp/github-action/blob/master/src/run.ts#L38
[2]: https://github.com/coverallsapp/github-action/issues/55#issuecomment-644927165

The problem that is fixed by this test-run update is the following. A
test that is listed in the 'release_disabled' field in a suite.ini file
of a 'core = app' or a 'core = unittest' test suite was disabled at all,
not only for the release tarantool build. A 'core = tarantool' test
suite was not affected by the problem.

Disabled sql-tap/debug_mode_only.test.lua test, because it does not work
anymore. We plan to revisit this test in the scope of #3694.

See https://github.com/tarantool/test-run/issues/199

There are a few issues concerning build of coro on Mac M1.

Firstly, Mach-O assembler does not support .type directive. So let's put
it under ifndef __APPLE__ preprocessing guard.

Secondly, according to OS X calling convention symbols (at least those
which are not exported) are mangled with underscore ("_") prefix:

Undefined symbols for architecture arm64:
  "_coro_startup", referenced from:
      _coro_create in libcoro.a(coro.c.o)

The same concerns "abort" function call. So to call function from assembly
inline we have to specify exact name (i.e. prefixed with _).

Part of #5983

This test-run update offers several changes.

One changeset renews test-run dependencies (see [1]):

* 27e097e Eliminate six dependency
* 0868e26 Drop unused dependencies
* e698240 Relax dependency version requirements

Another one fixes --reproduce mode (see [2]):

* 9fe1cc9 Fix --reproduce mode for fragile tests

[1]: https://github.com/tarantool/test-run/pull/288
[2]: https://github.com/tarantool/test-run/issues/233

This fixes a nit in commit 6af47377
("fiber: use uint64_t for fiber IDs").

Since lua_pushfstring doesn't support %llu format [1] it
breaks the test

app-tap/gh-1700-abort-recording-on-fiber-switch.test.lua

[1] https://www.lua.org/manual/5.1/manual.html#lua_pushfstring



Follows up #5846

Reviewed-by: Igor Munkin <imun@tarantool.org>
Reviewed-by: Alexander V. Tikhonov <avtikhon@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Signed-off-by: Igor Munkin <imun@tarantool.org>

Update of an absent field could crash when had 2 operations in
reversed order both on not specified fields.

The problem was that the rope item was created incorrectly.

Rope item is a range of array fields and consists of 2 parts:
xrow_update_field and a tail. The xrow field should describe
exactly one array field (first in the given range) and its update
operation. When no operation, it is NOP.

The tail includes the rest of the array fields not affected by
any operations yet.

But in the code it was so the rope item was created with its xrow
field covering multiple array fields, and with zero tail. As a
result, split of that item didn't work because it can't split an
xrow field.

The bug was in the nil-autofill for absent array fields. The patch
fixes it so the newly created item with nils has its xrow field
containing one nil, and the other nils in the tail. This allows to
split the item correctly if necessary.

Closes #6069

The position of first argument to check from is 6, not 0.
Looks like our tests with CFORMAT was simply not working
since commit 7d12d66e (Refactor macroses for compiler
builtins and function attributes).

Lets turn them all back.

Follow-up #5846

Reported-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

 - vclock_get returns int64_t
 - vclock_sum returns int64_t

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Use explicit cast for 64 bit number.

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Use explicit conversion for 64 bit number.

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Drop redundant "%s" from format.

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

 - The string length specificator expects length as
   an integer value
 - Use matching quoting for key print out

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Drop redundant "%s" argument.

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Use explicit "%llu" for uint64_t type.

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

 - drop redundant %p
 - use %zd for size_t

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Historically we use uint32_t for fiber IDs. And these IDs were
wrapping in time, especially if instance is running for a long
period. Strictly speaking this is not very convenient because if
some external tool gonna track fibers by their IDs it might get
confused (or miss) by IDs wrapping.

Lets rather switch to wide integers and fixup outputs (such as
snprintf callers, Lua's fid reports and etc). This allows us to
simplify code a bit and forget about IDs wrapping at all.

Same time wrong output specificators resulted in weird informal
lines

 | main/-244760339/cartridge.failover.task I> Instance state changed

Thus changing IDs type forced us to review all printouts and fix
formatting to not confuse users.

Closes #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

relay_subscribe_f() remembered old recovery pointer, which might be
replaced by relay_restart_recovery() if a raft message is delivered during
cbus_process() loop in relay_send_is_raft_enabled().

Fix the issue by removing the alias altogether and referencing relay->r
directly to prevent any further mistakes.

Closes #6031

We had various places in box.cc and relay.cc which counted processed
rows and yielded every now and then. These yields didn't cover cases,
when recovery has to position inside a long WAL file:

For example, when tarantool exits without leaving an empty WAL file
which will be used to recover instance vclock on restart. In this case
the instance freezes while processing the last available WAL in order
to recover the vclock.

Another issue is with replication. If a replica connects and needs data
from the end of a really long WAL, recovery will read up to the needed
position without yields, making relay disconnect by timeout.

In order to fix the issue, make recovery decide when a yield should
happen. Once recovery decides so, it calls a xstream callback,
schedule_yield. Currently schedule_yield is fired once recovery
processes (either skips or writes) WAL_ROWS_PER_YIELD rows.

schedule_yield either yields right away, in case of relay, or saves the
yield for later, in case of local recovery, because it might be in the
middle of a transaction.

Closes #5979

In case if we call fiber_join() over the non joinable fiber
we trigger an assert and crash execution (on debug build).

On release build the asserts will be zapped and won't cause
problems but there is an another one -- the target fiber will
cause double fiber_reset() calls which in result cause to
unregister_fid() with id = 0 (not causing crash but definitely
out of intention) and we will drop stack protection which
might be not ours anymore.

Since we're not allowed to break API on C level lets just
panic early in case of such misuse, it is a way better than
continue operating with potentially screwed data in memory.

Fixes #6046

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

No need for additional variable here.

In-scope-of #6046

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

This patch fixes inaccuracy in Tarantool build configuration introduced
by commit 07c83aab ('build: adjust
LuaJIT build system'). All those MacOS-related tweaks for __PAGEZERO
size and preferred load address for the bundle are necessary only for
builds with 32-bit GC area on 64-bit host. The only case fitting these
conditions is x86_64 with no LUAJIT_ENABLE_GC64. All other 64-bit builds
use 64-bit GC area unconditionally.

Part of #5983
Needed for #5629
Follows up #4862

Reviewed-by: Sergey Kaplun <skaplun@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>
Reviewed-by: Sergey Ostanevich <sergos@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

They were not deleted ever. Worked fine for DDL and replication,
for which they were introduced in the first place, because these
triggers are on the region memory.

But didn't work when the triggers became available in the public
API, because these are allocated on the heap. As a result, all the
box.on_commit() and box.on_rollback() triggers leaked.

The patch ensures all the on_commit/on_rollback triggers are
destroyed.

The statement triggers on_commit/on_rollback are left intact since
they are private and never need deletion, but the patch adds
assertions in case they ever would need to be destroyed.

Another option was to force all the commit and rollback triggers
clear themselves. For example, in case of commit all the on_commit
triggers must clear themselves, and the rollback triggers are
destroyed. Vice versa when a rollback happens. This would allow
not to manually destroy on_commit triggers in case of commit. But
it didn't work because the Lua triggers all work via a common
runner lbox_trigger_run(), which can't destroy its argument in
most of the cases (space:on_replace, :before_replace, ...). It
could be patched but requires to add some work to the Lua triggers
affecting all of them, which in total might be not better.

Closes #6025

Regex for validating version was expecting a single
character (digit) for version `patch`, but it's not correct.
This patch fixes test behaviour for tarantool 1.10.10

Close #6039

The previous commit made fiber_wakeup() safe to use on the current
fiber. Leverage the new behaviour everywhere in the source code to
remove all checks f != fiber() before fiber_wakeup(f) calls.

Follow-up #5292

fiber.wakeup() in Lua and fiber_wakeup() in C could lead to a
crash or undefined behaviour when called on the currently running
fiber.

In particular, if after wakeup was started a new fiber in a
blocking way (fiber.create() and fiber_start()) it would crash in
debug build, and lead to unknown results in release.

If after wakeup was made a sleep with non-zero timeout or an
infinite yield (fiber_yield()), the fiber woke up in the same
event loop iteration regardless of any timeout or other wakeups.
It was a spurious wakeup, which is not expected in most of the
places internally.

The patch makes the wakeup nop on the current fiber making it safe
to use anywhere.

Closes #5292
Closes #6043

@TarantoolBot document
Title: fiber.wakeup() in Lua and fiber_wakeup() in C are nop on self
In Lua `fiber.wakeup()` being called on the current fiber does not
do anything, safe to use. The same for `fiber_wakeup()` in C.

 - add comments about modes
 - there is no need to carry NULL in wal_mode_STRS
 - use designated assignment in wal_mode_STRS

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Acked-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

There is no need to test @confirm_lsn since we're
continue the list traversing in any case.

I keep "continue" here to be consistent with other "if"s.

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Acked-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

There is no need to test @confirm_lsn at all because
even with value -1 we're to continue iterating the queue.
Lets drop it and save a branch.

I keep "continue" here to be consistent with other "if"s.

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Acked-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>