Part of #6241

This patch moves response_decoder table (which decodes only errors
despite its name) from Lua to C. This is a step towards rewriting
performance-critical parts of net.box in C.

Since error_unpack_unsafe is not used by FFI anymore, remove it from
exports.h

Part of #6241

Found that Github actions/checkout@v2, when Github Actions triggers
a job upon the tag push, it makes the following calls [1]:

  git fetch origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*
  git fetch origin +$HASH:refs/tags/$TAG

While the first call is fine, the second one spoils the tag and makes
it non-annotated.

Later, Tarantool calls git describe without --tags flag, and the
spoiled tag is ignored. This patch handles it by fetching all the
tags once again explicitly.

Also explained the next step, where we remove the tag. The Github
Actions may runs twice the same job on branch push and on new tag
if it was created on this push. To avoid of fails on saving the
commonly named packages, we drop a tag that points to a current
commit (if any) on a job triggered by pushing to a branch (it will
produce the package based on previous tag) and leave the new tag
in job triggered by tag.

To check this change was used commands to create dump commit and tag
the new version. Also made twice push with extra flag for tags:

  git commit -m"Dump commit" --allow-empty
  git tag -a 2.4.8 -m ''
  git push && git push --tags

In this way would be run 2 jobs for each package:

  1. Build package on testing branch. It will produce package with
     name 2.4.7.x (because of the exception mentioned above).
  2. Build package on tag 2.4.8. It will produce package with name
     2.4.8.x.

Closes tarantool/tarantool-qa#119

[1]: https://github.com/actions/checkout/issues/290

At some point ER_TUPLE_FOUND error message was extended to contain
old tuple and new tuple. It seems that after rebase this change
was ignored in MVCC code that used previous format. This patch
fixes that.

Closes #6247

This patch removes functions that become unused due to changes of
implicit and explicit cast rules.

Closes #4470

This patch removes deprecated implicit cast from OP_MakeRecord opcode,
which were used in some rare cases, for example during IN operation
with subselect as right-value.

Closes #4230
Part of #4470

Since the OP_Seek* opcodes now work using the new implicit casting
rules, we don't need to call OP_ApplyType before them. This patch
removes such calls.

Part of #4230
Part of #4470

This patch changes the Seek* opcodes that are used to search in space
using index. After the redesign, searches using these opcodes work
according to the new implicit casting rules. However, currently implicit
cast in these opcodes is not invoked since there is OP_ApplyType before
them. Unnecessary OP_ApplyType calls will be removed in next patch.

Part of 4230
Part of 4470

After this patch, the new rules will be applied to implicit cast during
comparison where index is not used. Essentially it means that implicit
cast from STRING to number during such comparisons was removed.

Part of #4230
Part of #4470

After this patch, the new rules will be applied to implicit cast during
assignment. According to these rules, all scalar values can be cast to
SCALAR, all numeric values can be cast to NUMBER, and any numeric value
can be cast to another numeric type only if the conversion is exact.
No other implicit cast is allowed.

Part of #4470

Fix the test failing occasionally with the following result mismatch:

[001] replication/election_qsync.test.lua             memtx           [ fail ]
[001]
[001] Test failed! Result content mismatch:
[001] --- replication/election_qsync.result	Thu Jul 15 17:15:48 2021
[001] +++ var/rejects/replication/election_qsync.reject	Thu Jul 15 20:46:51 2021
[001] @@ -145,8 +145,7 @@
[001]   | ...
[001]  box.space.test:select{}
[001]   | ---
[001] - | - - [1]
[001] - |   - [2]
[001] + | - - [2]
[001]   | ...
[001]  box.space.test:drop()
[001]   | ---
[001]

The issue happened because row [1] wasn't delivered to the 'default'
instance from the 'replica' at all. The test does try to wait for [1] to
be written to WAL and replicated, but sometimes it fails to wait until
this event happens:

box.ctl.promote() is issued asynchronously once the instance becomes the
Raft leader. So issuing `box.ctl.wait_rw()` doesn't guarantee that the
replica has already written the PROMOTE (the limbo is initially
unclaimed so replica becomes writeable as soon as it becomes the Raft
leader).

Right after `wait_rw()` we wait for lsn propagation and for 'default'
instance to reach replica's lsn. It may happen that lsn propagation
happens due to PROMOTE being written to WAL, and not row [1].

When this is the case, the 'default' instance doesn't receive row [1] at
all, resulting in the test error shown above.

Fix the issue by waiting for the promotion to happen explicitly.

Part of #5430

Introduce a new journal entry, DEMOTE. The entry has the same meaning as
PROMOTE, with the only difference that it clears limbo ownership instead
of transferring it to the issuer.

Introduce `box.ctl.demote`, a counterpart to `box.ctl.promote`, which
clears the limbo ownership (when elections are off) via writing the DEMOTE, or
simply makes this instance step down from the leader's role (when elections
are enabled in any mode).

A new request was necessary instead of simply writing PROMOTE(origin_id
= 0), because origin_id is deduced from row.replica_id, which cannot be
0 for replicated rows (it's always equal to instance_id of the row
originator).

Closes #6034

@TarantoolBot document
Title: box.ctl.demote

`box.ctl.demote()` is a counterpart to `box.ctl.promote()` which works
as follows:
 - when `box.cfg.election_mode` is not 'off': make the instance give up
   leadership.
 - when `box.cfg.election_mode` is 'off': write a DEMOTE entry to WAL.

`box.ctl.demote()` may only be issued on the synchronous transaction
queue owner (i.e. leader when elections are enabled).

A DEMOTE request (DEMOTE = 32) copies PROMOTE behaviour (it clears the
limbo as well), but clears the synchronous transaction queue  ownership instead
of assigning it to a new instance.

Forbid limbo ownership transition without an explicit promote.
Make it so that synchronous transactions may be committed only after it
is claimed by some instance via a PROMOTE request.

Make everyone but the limbo owner read-only even when the limbo is
empty.

Part-of #6034

@TarantoolBot document
Title: synchronous replication changes

`box.info.synchro.queue` receives a new field: `owner`. It's a replica
id of the instance owning the synchronous transaction queue.

Once some instance owns the queue, every other instance becomes
read-only. When the queue is unclaimed, e.g.
`box.info.synchro.queue.owner` is `0`, everyone may be writeable, but
cannot create synchronous transactions.

In order to claim or re-claim the queue, you have to issue
`box.ctl.promote()` on the instance you wish to promote.

When elections are enabled, the instance issues `box.ctl.promote()`
automatically once it wins the elections, no additional actions are
required.

Part of #6034

There are two use cases for box_promote(): it's either invoked manually
or after a won election round. In the latter case promote() is much
simpler.

Part-of #6034

The failure itself was fixed in 68de8753
(raft: replace raft_start_candidate with _promote), let's add a
regression test now.

Follow-up #3055

It was allowed to call promote on any instance, even when it's already
the limbo owner (iow Raft leader, when elections are enabled).

This doesn't break anything, when elections are disabled, but is rather
strange.

When elections are enabled, in contrary, calling promote() should be a
no-op on the leader. Otherwise it would make the leader read-only until
it wins the next election round, which's quite inconvenient.

Part-of #6034

When called without elections, promote resulted in multiple
PROMOTE entries for the same term. This is not correct, because all
the promotions for the same term except the first one would be ignored
as already seen.

Part-of #6034

box_promote() is a monster. It does a lot of different things based on
flags: try_wait and run_elections. The flags themselves depend on the
node's Raft state and the lunar calendar.

Moreover, there are multiple cancellation points and places where
external state may have changed and needs a re-check.

Things are going to get even worse with the introduction of box.ctl.demote().

So it's time to split up box_promote() into reasonable parts, each doing
exactly one thing.

This commit mostly addresses the multiple cancellation points issue,
so that promote() doesn't look like a huge pile of if(something_changed)
blocks. Some other functions will look like that instead.

Part of #6034

Make raft_new_term() always bump the current term, even when Raft is
disabled.

Part-of #6034

Make Raft nodes send out their latest persisted term to joining
replicas.

This is needed to avoid the situation when txn_limbo-managed 'promote
greatest term' is greater than current Raft term. Otherwise the
following may happen: replica joins off some instance and receives its
latest limbo state. The state includes "greatest term seen" and makes
limbo filter out any data coming from instances with smaller terms.
Imagine that master this replica has joined from dies before replica has
a chance to subscribe to it. Then it doesn't receive its current Raft
term and start elections at smallest term possible, 2 (when there are no
suitable Raft nodes besides the replica).

Once the elections in a small term number are won, a ton of problems
arises: starting with filtering out PROMOTE requests for "old" term and
nop-ifying any data coming from terms smaller than "greatest term seen".

Prerequisite #6034

A joining instance may never receive the latest PROMOTE request, which
is the only source of information about the limbo owner. Send out the
latest limbo state (e.g. the latest applied PROMOTE request) together
with the initial join snapshot.

Prerequisite #6034

The new META stage is part of server's response to a join request.
It's marked by IPROTO_JOIN_META and IPROTO_JOIN_SNAPSHOT requests and goes
before the actual snapshot data.

Prerequisite #6034

@TarantoolBot document
Title: new protocol stage during JOIN

A new stage is added to the stream of JOIN rows coming from master.
The stage is marked with a bodyless row with type
IPROTO_JOIN_META = 71
Once all the rows from the stage are sent out, the JOIN continues as
before (as a stream of snapshot rows). The end of META stage is marked
with a row of type IPROTO_JOIN_SNAPSHOT = 72

The stage contains the rows that are necessary for instance
initialization (current Raft term, current state of synchronous
transaction queue), but do not belong to any system space.

iproto_write_error() used to be blocking until the commit
4dac37a6 (iproto: remove
iproto_write_error_blocking())
Actually, it should block until the error is written to socket, because
some tests (vinyl/errinj.test.lua, for example) rely on that.

Do not make iproto_write_error() blocking in release builds for safety
reasons, as stated in commit above. But make it blocking in debug for
testing sake.

Part-of #6034

The replica's version will be needed once sending limbo and election
state snapshot is implemented.

Prerequisite #6034

@TarantoolBot document
Title: New field in JOIN request

JOIN request now comes with a new field: replica's version.
The field uses IPROTO_SERVER_VERSION key.

Previously PROMOTE entries, just like CONFIRM and ROLLBACK were only
stored in WALs. This is because snapshots consist solely of confirmed
transactions, so there's nothing to CONFIRM or ROLLBACK.

PROMOTE has gained additional meaning recently: it pins limbo ownership
to a specific instance, rendering everyone else read-only. So now
PROMOTE information must be stored in snapshots as well.

Save the latest limbo state (owner id and latest confirmed lsn) to the
snapshot as a PROMOTE request.

Prerequisite #6034

txn_limbo_process() used to filter out promote requests whose term was
equal to the greatest term seen. This wasn't correct for PROMOTE entries
with term 1.

Such entries appear after box.ctl.promote() is issued on an instance
with disabled elections. Every PROMOTE entry from such an instance has
term 1, but should still be applied. Fix this in the patch.

Also, when an outdated PROMOTE entry with term smaller than already
applied from some replica arrived, it wasn't filtered at all. Such a
situation shouldn't be possible, but fix it as well.

Part-of #6034

Tarantool used to send out raft state on subscribe only when raft was
enabled. This was a safeguard against partially-upgraded clusters, where
some nodes had no clue about Raft messages and couldn't handle them
properly.

Actually, Raft state should be sent out always. For example, promote
will be changed to bump Raft term even when Raft is disabled, and it's
important that everyone in cluster has the same term for the sake of promote
at least.

So, send out Raft state to every subscriber with version >= 2.6.0
(that's when Raft was introduced).
Do the same for Raft broadcasts. They should be sent only to replicas
with version >= 2.6.0

Closes #5438

On_election triggers are fired asynchronously after any Raft event with
a broadcast, they are run in a worker fiber, so it's allowed to yield
inside them, unlike Raft's on_update triggers we already had.

Closes #5819

@TarantoolBot document
Title: document triggers on election state change

A new function to register triggers is added, `box.ctl.on_election()`.
Triggers registered via this function are run asynchronously every time
a visible change in `box.info.election` table appears.
No parameters are passed to the trigger, it may check what's changed by
looking at `box.info.election` and `box.info.synchro`.

After this patch, arithmetic operations will only accept numeric values.
For the "%" operation, the rules have become even stricter, now it
accepts only INTEGER and UNSIGNED values.

Part of #4470
Closes #5756

After this patch, bitwise operations will only accept UNSIGNED and
positive INTEGER values as operands. The result of the bitwise operand
will be UNSIGNED.

Part of #4470
Closes #5364

Prior to this patch, if a non-NULL-terminated string was cast to
BOOLEAN, the conversion always failed. Casting to BOOLEAN is now
independent of NULL termination.

Part of #4470

This patch removes explicit cast of VARBINARY values to numeric types.

Part of #4470
Closes #4772
Closes #5852

This patch removes explicit cast of BOOLEAN values to numeric types and
explicit cast of numeric values to BOOLEAN.

Part of #4470

This opcode was used to convert INTEGER values to REAL. It is not
necessary in Tarantool and causes errors.

Due to OP_Realify two type of errors appeared:
1) In some cases in trigger INTEGER may be converted to DOUBLE.
For example:
box.execute("CREATE TABLE t (i NUMBER PRIMARY KEY, n NUMBER);")
box.execute("CREATE TRIGGER t AFTER INSERT ON t FOR EACH ROW BEGIN UPDATE t SET n = new.n; END;")
box.execute("INSERT INTO t VALUES (1, 1);")
box.execute("SELECT i / 2, n / 2 FROM t;")

Result:
tarantool> box.execute("SELECT i / 2, n / 2 FROM t;")
---
- metadata:
  - name: COLUMN_1
    type: number
  - name: COLUMN_2
    type: number
  rows:
  - [0, 0.5]
...

2) If SELECT uses GROUP BY then it may return DOUBLE instead of INTEGER.
For example:
box.execute("CREATE TABLE t (i NUMBER PRIMARY KEY, n NUMBER);")
box.execute("INSERT INTO t VALUES (1,1);")
box.execute("SELECT i / 2, n / 2 FROM t GROUP BY n;")

Result:
tarantool> box.execute("SELECT i / 2, n / 2 FROM t GROUP BY n;")
---
- metadata:
  - name: COLUMN_1
    type: number
  - name: COLUMN_2
    type: number
  rows:
  - [0.5, 0.5]
...

This patch removes OP_Realify, after which these errors disappear.

Closes #5335

Prior to this patch when DOUBLE value that less than 0.0 and greater
than -1.0 was cast to INTEGER, it was considered to be negative number
though the result was 0. This patch fixes this, so now such DOUBLE
value will be properly cast to INTEGER and UNSIGNED.

Closes #6225

This patch adds changelog entry for commit
6a420c25 ('luajit: bump new version')
and commit 20c0a12a ('test: enable JIT
for Lua Fun chain iterator back').

Follows up #5118
Follows up #4252
Follows up #5049

Signed-off-by: Igor Munkin <imun@tarantool.org>

This patch reverts the temporary fix introduced in commit
5fa7ded2 ("test: disable JIT for Lua Fun
chain iterator") since the issues with invalid traces generation for
<fun.chain> iterator are resolved and JIT can be enabled back then.

Follows up #4252
Follows up #5118
Relates to #5049

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
Reviewed-by: Sergey Kaplun <skaplun@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

* Detect inconsistent renames even in the presence of sunk values.

Closes #5118
Follows up #4252