The patch refactors the algorithm of finding a join-master (in
replicaset_find_join_master()) to use scores instead of multiple
iterations with different criteria.

The original code was relatively fine as long as it had only
one parameter to change - whether should it skip
`box.cfg{read_only = true}` nodes.

Although it was clear that it was "on the edge" of acceptable
complexity due to a second non-configurable parameter whether a
replica is in read-only state regardless of its config.

It is going to get more complicated when the algorithm will take
into account the third parameter whether an instance is
bootstrapped.

Then it should make decisions like "among bootstrapped nodes try
to prefer instances not having read_only=true, and not being in
read-only state". The easiest way to do so is to use
scores/weights incremented according to the instance's parameters
matching certain "good points".

Part of #5613

The new field reports whether the instance has finished its
bootstrap/recovery, or IOW has finished box.cfg().

The new field will help in fixing #5613 so as not to try to join
to a replicaset via non-bootstrapped instances if there are
others.

The problem is that otherwise, if all nodes are booted but
are read-only, new instances bootstrap their own independent
replicaset. It would be better to just fail and terminate the
process than do such a bizarre action.

Part of #5613

Is_loading in the ballot used to mean the following: "the instance
did not finish its box.cfg() or has read_only = true". Which is
quite a strange property.

For instance, it was 'true' even if the instance is not really
loading anymore but has read_only = true.

The patch renames it to 'is_ro' (which existed here before, but
also with a wrong meaning).

Its behaviour is slightly changed to report the RO state of the
instance. Not its read_only. This way it incorporates all the
possible RO conditions. Such as not finished bootstrap, having
read_only = true, being a Raft follower, and so on.

The patch is done in scope of #5613 where the ballot is going to
be extended and used a bit differently in the join-master search
algorithm.

Part of #5613

Rename the member to show its actual meaning. It is not the
real RO state of the instance. Only how it is configured.

It can happen that the instance is read_only = false, but still is
in RO state due to other reasons.

The patch is done in scope of #5613 where the ballot is going to
be extended and used a bit differently in the join-master search
algorithm.

Part of #5613

Firstly, rename it to replicaset_find_join_master(). Now, when
there is Raft with a concept of an actual leader, the function
name becomes confusing.

Secondly, do not access ballot member in struct applier in such a
long way - save the ballot pointer on the stack. This is going to
become useful when in one of the next patches the ballot will be
used more.

Part of #5613

If Raft state machine sees the current leader has explicitly
resigned from its role, it starts a new election round right away.

But in the code starting a new round there was an assumption that
there is no a volatile state. There was, in fact.

The patch makes the election start code use the volatile state to
bump the term. It should be safe, because the other nodes won't
receive it anyway until the new term is persisted.

There was an alternative - do not schedule new election until the
current WAL write ends. It wasn't done, because would achieve the
same (the term would be bumped and persisted) but with bigger a
latency.

Another reason is that if the leader would appear and resign
during WAL write on another candidate, in the end of its WAL write
the latter would see 0 leader and would think this term didn't
have one yet. And would try to elect self now, in the current
term. It makes little sense, because it won't win - the current
term had already had a leader and the majority of votes is
already taken.

Closes #6129

After this patch, the tt_uuid_from_strl() function will check the length
of the given string before converting it to a UUID.

Follow up #5886

This patch introduces a new SQL built-in function UUID().

Closes #5886

@TarantoolBot document
Title: SQL built-in function UUID()

SQL built-in function UUID() takes zero or one argument. If no argument
is specified, a UUID v4 is generated. If the version of the UUID to
generate is specified as an argument, the function returns the new UUID
of the given version. Currently only version 4 of UUID is supported.

This patch introduces UUID to SQL. UUID is now available as a new field
type.

Part of #5886

@TarantoolBot document
Title: Field type UUID is now available in SQL

The UUID field type is now available in SQL. This means that we can
create spaces and indexes with UUID, use it in SELECT, UPDATE and
DELETE. UUID can be accepted and returned by built-in functions and
user-defined functions.

According to the comparison rules, there will be no implicit casting in
the comparison. This rule also applies to UUID values: if a value is not
part of a SCALAR field, it cannot be compared to a value of any other
type. If the value is in a SCALAR field, it can be compared to any other
scalar value according to the comparison rules for a SCALAR field.

In case a UUID value is used in an operation that is not a comparison,
it can be implicitly converted to STRING or VARBINARY.

If a STRING or VARBINARY value is used in an operation that is not a
comparison, it can be implicitly converted to a UUID.

UUID value can always be explicitly converted to STRING or VARBINARY.

A STRING or VARBINARY value can be explicitly converted to a UUID if it
conforms to the UUID standard.

This patch allows VARBINARY to be returned for user-defined LUA
functions. However, there are currently no values that can be
interpreted as VARBINARY by the serializer, so the only way to get a
VARBINARY result for user-defined LUA functions is to return a UUID or
DECIMAL. Both types are not supported by SQL and are treated as
VARBINARY.

Closes #6024

This patch allows VARBINARY to be returned for user-defined C functions.
There is currently no support for UUID and DECIMAL in SQL, so they are
also treated as VARBINARY.

Part of #6024

In 0e37af31 an optimization eliminating INSERT+DELETE and DELETE+INSERT
statements by the same key in write set was introduced. It is fine until
it comes for secondary index build. While we are building secondary
index we save current lsn, set on_replace trigger forwarding new
requests to the secondary index and copy row-by-row tuples (to be more
precise keys) to secondary index until lsn of tuple is less than the one
we preserved at the start. Now, if during index build we execute request
replacing key that hasn't been already transferred to secondary index,
we will get missing key in secondary index since:
a) In on_replace trigger replace is split into DELETE+INSERT and
   eliminated by mentioned optimization (the same concerns simple pair
   of DELETE+INSERT requests made in single transaction - so that they
   get into one write set);
b) It is skipped in the main loop transferring tuples from PK to SK
   since lsn of modified tuples is greater than saved lsn.

In this respect, we may get missing tuples in secondary index.
The proposed solution is quite trivial: we are able to track that index
is still being created (see previous commit) so we won't apply
INSERT+DELETE annihilation if index build is not finished.

Closes #6045

It tells whether LSM tree is currently being constructed, or is already
built and committed.

Needed for #6045

Remote node doing the subscribe might be from a different
replicaset.

Before this patch the subscribe would be retried infinitely
because the node couldn't be found in _cluster, and the master
assumed it must have joined to another node, and its ID should
arrive shortly (ER_TOO_EARLY_SUBSCRIBE).

The ID would never arrive, because the node belongs to another
replicaset.

The patch makes so the master checks if the peer lives in the same
replicaset. Since it is doing a subscribe, it must have joined
already and should have a valid replicaset UUID, regardless of
whether it is anonymous or not.

Correct behaviour is to hard cut this peer off immediately,
without retries.

Closes #6094
Part of #5613

This update offers one tiny change in the pretest_clean functionaly:
it'll not remove jit.dis_arm64 from the package.loaded table anymore.

Relates to #5983

Sometimes jobs on CI with Jepsen tests failed on installation
dependencies:

```
sudo -S -u root bash -c "cd /; env DEBIAN_FRONTEND=noninteractive apt-get install -y --force-yes apt-transport-https libzip4 ntpdate faketime"

STDIN:
null

STDOUT:
Reading package lists...
Building dependency tree...
Reading state information...

STDERR:
W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to locate package libzip4
E: Unable to locate package ntpdate
E: Unable to locate package faketime
```

Problem looks as a flaky, I couldn't reproduce it locally.  I suspect
the root cause is an infrastructure problem and to get more details
about it I have enabled debug options in apt-get and added
`set -o errexit` as it is recommended in documentation [1]
(see Note section).

1. https://www.terraform.io/docs/language/resources/provisioners/remote-exec.html#argument-reference

Part of: https://github.com/tarantool/jepsen.tarantool/issues/87

It is possible that a new async transaction is added to the limbo
when there is an in-progress CONFIRM WAL write for all the pending
sync transactions.

Then when CONFIRM WAL write is done, it might see that the limbo
now in the first place contains an async transaction not yet
written to WAL. A suspicious situation - on one hand the async
transaction does not have any blocking sync txns before it and
can be considered complete, on the other hand its WAL write is not
done and it is not complete.

Before this patch it resulted into a crash - limbo didn't consider
the situation possible at all.

Now when CONFIRM covers a not yet written async transactions, they
are removed from the limbo and are turned to plain transactions.

When their WAL write is done, they see they no more have
TXN_WAIT_SYNC flag and don't even need to interact with the limbo.

It is important to remove them from the limbo right when the
CONFIRM is done. Because otherwise their limbo entry may be not
removed at all when it is done on a replica. On a replica the
limbo entries are removed only by CONFIRM/ROLLBACK/PROMOTE. If
there would be an async transaction in the first position in the
limbo queue, it wouldn't be deleted until next sync transaction
appears.

This replica case is not possible now though. Because all synchro
entries on the applier are written in a blocking way. Nonetheless
if it ever becomes non-blocking, the code should handle it ok.

Closes #6057

Currently `log` module accepts only numeric values of
logging levels. I turn `box.cfg` interface supports
symbolic names (such as 'fatal', 'crit' and etc).

Thus we should support the same in `log` module.

Closes #5882

Reported-by: Alexander Turenko <alexander.turenko@tarantool.org>
Acked-by: Alexander Turenko <alexander.turenko@tarantool.org>
Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Found that commit message may consists of special characters which can
be used be Markdown as commands, like '`' or '```'. To avoid of it these
characters must be changed to some predefined names like for:

'\' - BACKSLASH
'`' - BACKTICK

Also added filter block to avoid of other not known symbols which we
could miss. This block converts commit message to HTML and then takes
only text from it.

Found that on self-hosted runners where CentOS 7 is the base OS,
'send-telegram-notify' action creates message with syntax error:

  --------------'\n't'\n'```'\n'')) ; \
  ^
  SyntaxError: unexpected character after line continuation character

It happened because of extra quotes at '\n' while it had to be \n.
To avoid of it the same message changes must be done as for OSX
hosts are doing. These changes should be done when self-hosted
runners uses RHEL or Fedora as base OS.

After commit:

  58fe0fcb ('github-ci: avoid of use container tags in actions')

We began to use not the docker containers, but native github hosts.
To avoid of permissions fails on native github actions runners apt
command must run using sudo. Added flag '-n|--non-interactive' to
sudo command to avoid prompting the user for input of any kind which
could hang it. Added '-y' flag to apt update command to accept changes.

For now python3 is used as the default python on all OS and it is
needed to enable it in send-telegram-notify action.

Found issue:

  Traceback (most recent call last):
    File "<string>", line 3, in <module>
  AttributeError: module 'urllib' has no attribute 'quote_plus'

In Python 3 quote_plus included into urllib.parse.

Check documentaion [1]:

  Note The urllib module has been split into parts and renamed in Python 3 to urllib.request, urllib.parse, and urllib.error.

Check the same issue [2].

This patch changes use of all needed routines just from 'urllib'.

Closes tarantool/tarantool-qa#112

[1]: https://docs.python.org/2/library/urllib.html
[2]: https://github.com/web2py/web2py/issues/1822

box.ctl.promote() used to assume that the last synchronous entry is
already written to WAL by the time it's called. This is not the case
when promote is executed on the limbo owner. The last synchronous entry
might still be en route to WAL.

In order to fix the issue, wait until all the limbo entries are written
to disk via wal_sync(). After this happens, it's safe to proceed to
gathering quorum in promote.

Closes #6032

Occasionally, test/app-tap/gh-5040-inter-mode-isatty-via-errinj.test.lua
failed because it used output file with the same name as
test/app-tap/gh-4983-tnt-e-assert-false-hangs.test.lua and the last one
didn't remove file after usage.

Added removal of output file to test for 4983 and also changed file names
to distinguish outputs of these tests better in case of failure.

Fixes tarantool/tarantool-qa#122

There are several updates in test-run:

* Unit testing and coverage collecting for test-run ([1]).
  (Plus a fixup for `--disable-schema-upgrade` for Python 3.5.)
* Allow to miss a timeout in test_run:wait_log(): it is 60 seconds by
  default ([2]).
* pretest_clean: add built-in `UUID()` SQL function ([3]).

[1]: https://github.com/tarantool/test-run/pull/283
[2]: https://github.com/tarantool/test-run/pull/306
[3]: https://github.com/tarantool/test-run/pull/307

It is strange to create a new fiber and see that it has yielded 100
times, when in fact it never actually did it.

The patch makes fiber->csw = 0 for each created fiber.

Follow-up #5799

If you want to get information or get csw (Context SWitch) of some fiber
you need to call fiber.info(), but it creates table with information about
all the fibers. This patch introduces fiber_object:info() and
fiber_object:csw() - functions to solve this problem.

Closes #5799

@TarantoolBot document
Title: introduce fiber_object:info() and fiber_object:csw()
```
-- fiber_object:info() is the same as fiber.info(), but show information only
about one alive fiber.
-- fiber_object:csw() show csw (Context SWitch) of alive fiber.
```

This patch moves MEM types from the 'u32 flags' field to the new
'enum mem_type type' field. Now, we can be sure that only one type is
set for MEM. In addition, it is now easier to distinguish MAP and ARRAY
from VARBINARY, and this makes it easier to add extension types - UUID
and DECIMAL.

Closes #4906

After this patch, the mem_is_bin() function will return 'true' only if
the value that the MEM contains is of type VARBINARY. This patch also
adds the mem_is_bin_ext() function, which is used to check if a MEM
contains value of type VARBINARY or value of types that are currently
considered VARBINARY extensions - MAP and ARRAY.

Part of #4906

This patch adds proper initialization for the MEM, which is used in the
aggregate functions min() and max().

Part of #4906

Lua json module used to have a global buffer for all encodings. It
was reused by each next encode().

This was not correct, because during encode() might happen a GC
step, which might call encode() again and spoil the global buffer.

The same problem was already fixed for the global static buffer in
scope of #5632. Similarly to that time, the patch makes Lua json
module use cord_ibuf to prevent "concurrent" usage of the buffer
data. The global buffer is deleted.

According to a few microbenchmarks it didn't affect the perf
anyhow.

Core part of the patch is strbuf changes. Firstly, its destruction
is now optional, cord_ibuf can free itself on a next yield.
Secondly, its reallocation algorithm is kept intact - ibuf is used
as an allocator, not as the buffer itself. This is done so as not
to be too intrusive in the third party module which might need an
upgrade to the upstream in the future.

Closes #6050

Script used to run Jepsen tests uses Terraform to prepare test environment.
When I made this script I had a wrong assumption about working directory
used by Terraform. Due to this sometimes job with Jepsen tests
failed on cleanup with message:

```
Error: Could not load plugin

Plugin reinitialization required. Please run "terraform init".

Plugins are external binaries that Terraform uses to access and manipulate
resources. The configuration provided requires plugins which can't be located,
don't satisfy the version constraints, or are otherwise incompatible.

Terraform automatically discovers provider requirements from your
configuration, including providers used in child modules. To see the
requirements and constraints, run "terraform providers".

Failed to instantiate provider
"registry.terraform.io/terraform-providers/openstack" to obtain schema:
unknown provider "registry.terraform.io/terraform-providers/openstack"
```

Terraform documentation describes how Terraform uses working directories [1]
and there are at least to ways to resolve an issue. First one is using always
one directory before running terraform subcommands. Second one is using option
`-chdir` in all terraform commands [2].

1. https://www.terraform.io/docs/cli/init/index.html
2. https://www.terraform.io/docs/cli/commands/index.html#switching-working-directory-with-chdir

Closes #6089

Prior to this patch, UUID was not part of SCALAR. However, this should
be changed to comply with the RFC "Consistent Lua/SQL types".

Closes #6042

@TarantoolBot document
Title: UUID is now part of SCALAR

The UUID field type is now part of the SCALAR field type. This means
that now values of type UUID can be inserted into the SCALAR field, and
these values can participate in the sorting of the SCALAR fields. The
order is as follows: boolean < number < string < varbinary < uuid.

This allows `xlog` Lua module to decode appropriate
types into symbolic form.

For example with the patch we should see raft and
promote types in output.

 | $ tarantoolctl cat 00000000000000000004.xlog
 | ---
 | HEADER:
 |   lsn: 2
 |   group_id: 1
 |   type: RAFT
 |   timestamp: 1621541912.4588
 | BODY:
 |   0: 3
 |   1: 4
 | ---
 | HEADER:
 |   lsn: 1
 |   replica_id: 4
 |   type: PROMOTE
 |   timestamp: 1621541912.4592
 | BODY:
 |   2: 0
 |   3: 0
 |   83: 3

Fixes #6088

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Now test-run reports memory occupied by tests and their duration (see
[1] and [2]).

Aside of this, there is a tiny code health change around waiting for
`box.info.status` value: it should not affect a visible test-run
behaviour ([3]).

[1]: https://github.com/tarantool/test-run/issues/277
[2]: https://github.com/tarantool/test-run/issues/286
[3]: https://github.com/tarantool/test-run/issues/301

commit fbc04dfb
("static-build: use hashes of downloaded files") added hashes for
ExternalProject_Add() to avoid extra downloads on project rebuild.
Changes for libicu and libiconv were missed and this patch fixes it.

Follows-up #5761

Co-authored-by: Sergey Bronnikov <sergeyb@tarantool.org>

Since commit c9d88d5f ('Fix #984: add
jit.* library to the binary') all required modules implemented in Lua
are bundled (i.e. compiled into the executable as a C literal) into
Tarantool binary. While making Tarantool work on ARM64 platforms, it
turned out the arch-specific module (namely, jit/dis_arm64.lua) is not
bundled. Within this patch the missing sources are added and jit.dump
works fine on ARM64 hosts as a result.

Part of #5983
Relates to #5629
Follows up #984

Reviewed-by: Sergey Kaplun <skaplun@tarantool.org>
Reviewed-by: Sergey Ostanevich <sergos@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

* FFI/ARM64/OSX: Fix vararg call handling.
* OSX/iOS: Handle iOS simulator and ARM64 Macs.
* build: pass sysroot to MacOS SDK

Closes #6065
Closes #6066
Part of #5629
Relates to #5983

Workflow 'debug_coverage' produces and uploads results to 'coveralls.io'
web site. Message about it can be shown in PR within each run was done.
This patch adds the ability to send message in available PR otherwise it
is skipped. Also added 'coverage.info' file to artifacts list.

Found that used 'coverallsapp/github-action' action checks if the
trigger was 'pull_request' used to run the workflow [1]. And only in
this way it writes results message to PR. Previously 'pull_request'
trigger was blocked to avoid duplication with 'push' trigger. To make
able to run workflow by any single trigger and to send message to PR
if it exits, it was implemented the following logic:

  - run workflows on both triggers 'push' and 'pull_request';
  - for workflow with 'push' trigger check if PR exists then skip all
    jobs otherwise continue running;
  - for workflow with 'pull_request' trigger continue running;

To avoid of issue coverallsapp/github-action#55 [2], sources checkout
should be based on 2nd version and later.

Closes #5644

[1]: https://github.com/coverallsapp/github-action/blob/master/src/run.ts#L38
[2]: https://github.com/coverallsapp/github-action/issues/55#issuecomment-644927165