Found that on self-hosted runners where CentOS 7 is the base OS,
'send-telegram-notify' action creates message with syntax error:

  --------------'\n't'\n'```'\n'')) ; \
  ^
  SyntaxError: unexpected character after line continuation character

It happened because of extra quotes at '\n' while it had to be \n.
To avoid of it the same message changes must be done as for OSX
hosts are doing. These changes should be done when self-hosted
runners uses RHEL or Fedora as base OS.

After commit:

  58fe0fcb ('github-ci: avoid of use container tags in actions')

We began to use not the docker containers, but native github hosts.
To avoid of permissions fails on native github actions runners apt
command must run using sudo. Added flag '-n|--non-interactive' to
sudo command to avoid prompting the user for input of any kind which
could hang it. Added '-y' flag to apt update command to accept changes.

For now python3 is used as the default python on all OS and it is
needed to enable it in send-telegram-notify action.

Found issue:

  Traceback (most recent call last):
    File "<string>", line 3, in <module>
  AttributeError: module 'urllib' has no attribute 'quote_plus'

In Python 3 quote_plus included into urllib.parse.

Check documentaion [1]:

  Note The urllib module has been split into parts and renamed in Python 3 to urllib.request, urllib.parse, and urllib.error.

Check the same issue [2].

This patch changes use of all needed routines just from 'urllib'.

Closes tarantool/tarantool-qa#112

[1]: https://docs.python.org/2/library/urllib.html
[2]: https://github.com/web2py/web2py/issues/1822

box.ctl.promote() used to assume that the last synchronous entry is
already written to WAL by the time it's called. This is not the case
when promote is executed on the limbo owner. The last synchronous entry
might still be en route to WAL.

In order to fix the issue, wait until all the limbo entries are written
to disk via wal_sync(). After this happens, it's safe to proceed to
gathering quorum in promote.

Closes #6032

Occasionally, test/app-tap/gh-5040-inter-mode-isatty-via-errinj.test.lua
failed because it used output file with the same name as
test/app-tap/gh-4983-tnt-e-assert-false-hangs.test.lua and the last one
didn't remove file after usage.

Added removal of output file to test for 4983 and also changed file names
to distinguish outputs of these tests better in case of failure.

Fixes tarantool/tarantool-qa#122

There are several updates in test-run:

* Unit testing and coverage collecting for test-run ([1]).
  (Plus a fixup for `--disable-schema-upgrade` for Python 3.5.)
* Allow to miss a timeout in test_run:wait_log(): it is 60 seconds by
  default ([2]).
* pretest_clean: add built-in `UUID()` SQL function ([3]).

[1]: https://github.com/tarantool/test-run/pull/283
[2]: https://github.com/tarantool/test-run/pull/306
[3]: https://github.com/tarantool/test-run/pull/307

It is strange to create a new fiber and see that it has yielded 100
times, when in fact it never actually did it.

The patch makes fiber->csw = 0 for each created fiber.

Follow-up #5799

If you want to get information or get csw (Context SWitch) of some fiber
you need to call fiber.info(), but it creates table with information about
all the fibers. This patch introduces fiber_object:info() and
fiber_object:csw() - functions to solve this problem.

Closes #5799

@TarantoolBot document
Title: introduce fiber_object:info() and fiber_object:csw()
```
-- fiber_object:info() is the same as fiber.info(), but show information only
about one alive fiber.
-- fiber_object:csw() show csw (Context SWitch) of alive fiber.
```

This patch moves MEM types from the 'u32 flags' field to the new
'enum mem_type type' field. Now, we can be sure that only one type is
set for MEM. In addition, it is now easier to distinguish MAP and ARRAY
from VARBINARY, and this makes it easier to add extension types - UUID
and DECIMAL.

Closes #4906

After this patch, the mem_is_bin() function will return 'true' only if
the value that the MEM contains is of type VARBINARY. This patch also
adds the mem_is_bin_ext() function, which is used to check if a MEM
contains value of type VARBINARY or value of types that are currently
considered VARBINARY extensions - MAP and ARRAY.

Part of #4906

This patch adds proper initialization for the MEM, which is used in the
aggregate functions min() and max().

Part of #4906

Lua json module used to have a global buffer for all encodings. It
was reused by each next encode().

This was not correct, because during encode() might happen a GC
step, which might call encode() again and spoil the global buffer.

The same problem was already fixed for the global static buffer in
scope of #5632. Similarly to that time, the patch makes Lua json
module use cord_ibuf to prevent "concurrent" usage of the buffer
data. The global buffer is deleted.

According to a few microbenchmarks it didn't affect the perf
anyhow.

Core part of the patch is strbuf changes. Firstly, its destruction
is now optional, cord_ibuf can free itself on a next yield.
Secondly, its reallocation algorithm is kept intact - ibuf is used
as an allocator, not as the buffer itself. This is done so as not
to be too intrusive in the third party module which might need an
upgrade to the upstream in the future.

Closes #6050

Script used to run Jepsen tests uses Terraform to prepare test environment.
When I made this script I had a wrong assumption about working directory
used by Terraform. Due to this sometimes job with Jepsen tests
failed on cleanup with message:

```
Error: Could not load plugin

Plugin reinitialization required. Please run "terraform init".

Plugins are external binaries that Terraform uses to access and manipulate
resources. The configuration provided requires plugins which can't be located,
don't satisfy the version constraints, or are otherwise incompatible.

Terraform automatically discovers provider requirements from your
configuration, including providers used in child modules. To see the
requirements and constraints, run "terraform providers".

Failed to instantiate provider
"registry.terraform.io/terraform-providers/openstack" to obtain schema:
unknown provider "registry.terraform.io/terraform-providers/openstack"
```

Terraform documentation describes how Terraform uses working directories [1]
and there are at least to ways to resolve an issue. First one is using always
one directory before running terraform subcommands. Second one is using option
`-chdir` in all terraform commands [2].

1. https://www.terraform.io/docs/cli/init/index.html
2. https://www.terraform.io/docs/cli/commands/index.html#switching-working-directory-with-chdir

Closes #6089

Prior to this patch, UUID was not part of SCALAR. However, this should
be changed to comply with the RFC "Consistent Lua/SQL types".

Closes #6042

@TarantoolBot document
Title: UUID is now part of SCALAR

The UUID field type is now part of the SCALAR field type. This means
that now values of type UUID can be inserted into the SCALAR field, and
these values can participate in the sorting of the SCALAR fields. The
order is as follows: boolean < number < string < varbinary < uuid.

This allows `xlog` Lua module to decode appropriate
types into symbolic form.

For example with the patch we should see raft and
promote types in output.

 | $ tarantoolctl cat 00000000000000000004.xlog
 | ---
 | HEADER:
 |   lsn: 2
 |   group_id: 1
 |   type: RAFT
 |   timestamp: 1621541912.4588
 | BODY:
 |   0: 3
 |   1: 4
 | ---
 | HEADER:
 |   lsn: 1
 |   replica_id: 4
 |   type: PROMOTE
 |   timestamp: 1621541912.4592
 | BODY:
 |   2: 0
 |   3: 0
 |   83: 3

Fixes #6088

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Now test-run reports memory occupied by tests and their duration (see
[1] and [2]).

Aside of this, there is a tiny code health change around waiting for
`box.info.status` value: it should not affect a visible test-run
behaviour ([3]).

[1]: https://github.com/tarantool/test-run/issues/277
[2]: https://github.com/tarantool/test-run/issues/286
[3]: https://github.com/tarantool/test-run/issues/301

commit fbc04dfb
("static-build: use hashes of downloaded files") added hashes for
ExternalProject_Add() to avoid extra downloads on project rebuild.
Changes for libicu and libiconv were missed and this patch fixes it.

Follows-up #5761

Co-authored-by: Sergey Bronnikov <sergeyb@tarantool.org>

Since commit c9d88d5f ('Fix #984: add
jit.* library to the binary') all required modules implemented in Lua
are bundled (i.e. compiled into the executable as a C literal) into
Tarantool binary. While making Tarantool work on ARM64 platforms, it
turned out the arch-specific module (namely, jit/dis_arm64.lua) is not
bundled. Within this patch the missing sources are added and jit.dump
works fine on ARM64 hosts as a result.

Part of #5983
Relates to #5629
Follows up #984

Reviewed-by: Sergey Kaplun <skaplun@tarantool.org>
Reviewed-by: Sergey Ostanevich <sergos@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

* FFI/ARM64/OSX: Fix vararg call handling.
* OSX/iOS: Handle iOS simulator and ARM64 Macs.
* build: pass sysroot to MacOS SDK

Closes #6065
Closes #6066
Part of #5629
Relates to #5983

Workflow 'debug_coverage' produces and uploads results to 'coveralls.io'
web site. Message about it can be shown in PR within each run was done.
This patch adds the ability to send message in available PR otherwise it
is skipped. Also added 'coverage.info' file to artifacts list.

Found that used 'coverallsapp/github-action' action checks if the
trigger was 'pull_request' used to run the workflow [1]. And only in
this way it writes results message to PR. Previously 'pull_request'
trigger was blocked to avoid duplication with 'push' trigger. To make
able to run workflow by any single trigger and to send message to PR
if it exits, it was implemented the following logic:

  - run workflows on both triggers 'push' and 'pull_request';
  - for workflow with 'push' trigger check if PR exists then skip all
    jobs otherwise continue running;
  - for workflow with 'pull_request' trigger continue running;

To avoid of issue coverallsapp/github-action#55 [2], sources checkout
should be based on 2nd version and later.

Closes #5644

[1]: https://github.com/coverallsapp/github-action/blob/master/src/run.ts#L38
[2]: https://github.com/coverallsapp/github-action/issues/55#issuecomment-644927165

The problem that is fixed by this test-run update is the following. A
test that is listed in the 'release_disabled' field in a suite.ini file
of a 'core = app' or a 'core = unittest' test suite was disabled at all,
not only for the release tarantool build. A 'core = tarantool' test
suite was not affected by the problem.

Disabled sql-tap/debug_mode_only.test.lua test, because it does not work
anymore. We plan to revisit this test in the scope of #3694.

See https://github.com/tarantool/test-run/issues/199

There are a few issues concerning build of coro on Mac M1.

Firstly, Mach-O assembler does not support .type directive. So let's put
it under ifndef __APPLE__ preprocessing guard.

Secondly, according to OS X calling convention symbols (at least those
which are not exported) are mangled with underscore ("_") prefix:

Undefined symbols for architecture arm64:
  "_coro_startup", referenced from:
      _coro_create in libcoro.a(coro.c.o)

The same concerns "abort" function call. So to call function from assembly
inline we have to specify exact name (i.e. prefixed with _).

Part of #5983

This test-run update offers several changes.

One changeset renews test-run dependencies (see [1]):

* 27e097e Eliminate six dependency
* 0868e26 Drop unused dependencies
* e698240 Relax dependency version requirements

Another one fixes --reproduce mode (see [2]):

* 9fe1cc9 Fix --reproduce mode for fragile tests

[1]: https://github.com/tarantool/test-run/pull/288
[2]: https://github.com/tarantool/test-run/issues/233

This fixes a nit in commit 6af47377
("fiber: use uint64_t for fiber IDs").

Since lua_pushfstring doesn't support %llu format [1] it
breaks the test

app-tap/gh-1700-abort-recording-on-fiber-switch.test.lua

[1] https://www.lua.org/manual/5.1/manual.html#lua_pushfstring



Follows up #5846

Reviewed-by: Igor Munkin <imun@tarantool.org>
Reviewed-by: Alexander V. Tikhonov <avtikhon@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Signed-off-by: Igor Munkin <imun@tarantool.org>

Update of an absent field could crash when had 2 operations in
reversed order both on not specified fields.

The problem was that the rope item was created incorrectly.

Rope item is a range of array fields and consists of 2 parts:
xrow_update_field and a tail. The xrow field should describe
exactly one array field (first in the given range) and its update
operation. When no operation, it is NOP.

The tail includes the rest of the array fields not affected by
any operations yet.

But in the code it was so the rope item was created with its xrow
field covering multiple array fields, and with zero tail. As a
result, split of that item didn't work because it can't split an
xrow field.

The bug was in the nil-autofill for absent array fields. The patch
fixes it so the newly created item with nils has its xrow field
containing one nil, and the other nils in the tail. This allows to
split the item correctly if necessary.

Closes #6069

The position of first argument to check from is 6, not 0.
Looks like our tests with CFORMAT was simply not working
since commit 7d12d66e (Refactor macroses for compiler
builtins and function attributes).

Lets turn them all back.

Follow-up #5846

Reported-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

 - vclock_get returns int64_t
 - vclock_sum returns int64_t

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Use explicit cast for 64 bit number.

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Use explicit conversion for 64 bit number.

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Drop redundant "%s" from format.

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

 - The string length specificator expects length as
   an integer value
 - Use matching quoting for key print out

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Drop redundant "%s" argument.

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Use explicit "%llu" for uint64_t type.

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

 - drop redundant %p
 - use %zd for size_t

Follow-up #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Historically we use uint32_t for fiber IDs. And these IDs were
wrapping in time, especially if instance is running for a long
period. Strictly speaking this is not very convenient because if
some external tool gonna track fibers by their IDs it might get
confused (or miss) by IDs wrapping.

Lets rather switch to wide integers and fixup outputs (such as
snprintf callers, Lua's fid reports and etc). This allows us to
simplify code a bit and forget about IDs wrapping at all.

Same time wrong output specificators resulted in weird informal
lines

 | main/-244760339/cartridge.failover.task I> Instance state changed

Thus changing IDs type forced us to review all printouts and fix
formatting to not confuse users.

Closes #5846

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

relay_subscribe_f() remembered old recovery pointer, which might be
replaced by relay_restart_recovery() if a raft message is delivered during
cbus_process() loop in relay_send_is_raft_enabled().

Fix the issue by removing the alias altogether and referencing relay->r
directly to prevent any further mistakes.

Closes #6031

We had various places in box.cc and relay.cc which counted processed
rows and yielded every now and then. These yields didn't cover cases,
when recovery has to position inside a long WAL file:

For example, when tarantool exits without leaving an empty WAL file
which will be used to recover instance vclock on restart. In this case
the instance freezes while processing the last available WAL in order
to recover the vclock.

Another issue is with replication. If a replica connects and needs data
from the end of a really long WAL, recovery will read up to the needed
position without yields, making relay disconnect by timeout.

In order to fix the issue, make recovery decide when a yield should
happen. Once recovery decides so, it calls a xstream callback,
schedule_yield. Currently schedule_yield is fired once recovery
processes (either skips or writes) WAL_ROWS_PER_YIELD rows.

schedule_yield either yields right away, in case of relay, or saves the
yield for later, in case of local recovery, because it might be in the
middle of a transaction.

Closes #5979

In case if we call fiber_join() over the non joinable fiber
we trigger an assert and crash execution (on debug build).

On release build the asserts will be zapped and won't cause
problems but there is an another one -- the target fiber will
cause double fiber_reset() calls which in result cause to
unregister_fid() with id = 0 (not causing crash but definitely
out of intention) and we will drop stack protection which
might be not ours anymore.

Since we're not allowed to break API on C level lets just
panic early in case of such misuse, it is a way better than
continue operating with potentially screwed data in memory.

Fixes #6046

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

No need for additional variable here.

In-scope-of #6046

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>