The new META stage is part of server's response to a join request.
It's marked by IPROTO_JOIN_META and IPROTO_JOIN_SNAPSHOT requests and goes
before the actual snapshot data.

Prerequisite #6034

@TarantoolBot document
Title: new protocol stage during JOIN

A new stage is added to the stream of JOIN rows coming from master.
The stage is marked with a bodyless row with type
IPROTO_JOIN_META = 71
Once all the rows from the stage are sent out, the JOIN continues as
before (as a stream of snapshot rows). The end of META stage is marked
with a row of type IPROTO_JOIN_SNAPSHOT = 72

The stage contains the rows that are necessary for instance
initialization (current Raft term, current state of synchronous
transaction queue), but do not belong to any system space.

iproto_write_error() used to be blocking until the commit
4dac37a6 (iproto: remove
iproto_write_error_blocking())
Actually, it should block until the error is written to socket, because
some tests (vinyl/errinj.test.lua, for example) rely on that.

Do not make iproto_write_error() blocking in release builds for safety
reasons, as stated in commit above. But make it blocking in debug for
testing sake.

Part-of #6034

The replica's version will be needed once sending limbo and election
state snapshot is implemented.

Prerequisite #6034

@TarantoolBot document
Title: New field in JOIN request

JOIN request now comes with a new field: replica's version.
The field uses IPROTO_SERVER_VERSION key.

Previously PROMOTE entries, just like CONFIRM and ROLLBACK were only
stored in WALs. This is because snapshots consist solely of confirmed
transactions, so there's nothing to CONFIRM or ROLLBACK.

PROMOTE has gained additional meaning recently: it pins limbo ownership
to a specific instance, rendering everyone else read-only. So now
PROMOTE information must be stored in snapshots as well.

Save the latest limbo state (owner id and latest confirmed lsn) to the
snapshot as a PROMOTE request.

Prerequisite #6034

txn_limbo_process() used to filter out promote requests whose term was
equal to the greatest term seen. This wasn't correct for PROMOTE entries
with term 1.

Such entries appear after box.ctl.promote() is issued on an instance
with disabled elections. Every PROMOTE entry from such an instance has
term 1, but should still be applied. Fix this in the patch.

Also, when an outdated PROMOTE entry with term smaller than already
applied from some replica arrived, it wasn't filtered at all. Such a
situation shouldn't be possible, but fix it as well.

Part-of #6034

Tarantool used to send out raft state on subscribe only when raft was
enabled. This was a safeguard against partially-upgraded clusters, where
some nodes had no clue about Raft messages and couldn't handle them
properly.

Actually, Raft state should be sent out always. For example, promote
will be changed to bump Raft term even when Raft is disabled, and it's
important that everyone in cluster has the same term for the sake of promote
at least.

So, send out Raft state to every subscriber with version >= 2.6.0
(that's when Raft was introduced).
Do the same for Raft broadcasts. They should be sent only to replicas
with version >= 2.6.0

Closes #5438

On_election triggers are fired asynchronously after any Raft event with
a broadcast, they are run in a worker fiber, so it's allowed to yield
inside them, unlike Raft's on_update triggers we already had.

Closes #5819

@TarantoolBot document
Title: document triggers on election state change

A new function to register triggers is added, `box.ctl.on_election()`.
Triggers registered via this function are run asynchronously every time
a visible change in `box.info.election` table appears.
No parameters are passed to the trigger, it may check what's changed by
looking at `box.info.election` and `box.info.synchro`.

After this patch, arithmetic operations will only accept numeric values.
For the "%" operation, the rules have become even stricter, now it
accepts only INTEGER and UNSIGNED values.

Part of #4470
Closes #5756

After this patch, bitwise operations will only accept UNSIGNED and
positive INTEGER values as operands. The result of the bitwise operand
will be UNSIGNED.

Part of #4470
Closes #5364

Prior to this patch, if a non-NULL-terminated string was cast to
BOOLEAN, the conversion always failed. Casting to BOOLEAN is now
independent of NULL termination.

Part of #4470

This patch removes explicit cast of VARBINARY values to numeric types.

Part of #4470
Closes #4772
Closes #5852

This patch removes explicit cast of BOOLEAN values to numeric types and
explicit cast of numeric values to BOOLEAN.

Part of #4470

This opcode was used to convert INTEGER values to REAL. It is not
necessary in Tarantool and causes errors.

Due to OP_Realify two type of errors appeared:
1) In some cases in trigger INTEGER may be converted to DOUBLE.
For example:
box.execute("CREATE TABLE t (i NUMBER PRIMARY KEY, n NUMBER);")
box.execute("CREATE TRIGGER t AFTER INSERT ON t FOR EACH ROW BEGIN UPDATE t SET n = new.n; END;")
box.execute("INSERT INTO t VALUES (1, 1);")
box.execute("SELECT i / 2, n / 2 FROM t;")

Result:
tarantool> box.execute("SELECT i / 2, n / 2 FROM t;")
---
- metadata:
  - name: COLUMN_1
    type: number
  - name: COLUMN_2
    type: number
  rows:
  - [0, 0.5]
...

2) If SELECT uses GROUP BY then it may return DOUBLE instead of INTEGER.
For example:
box.execute("CREATE TABLE t (i NUMBER PRIMARY KEY, n NUMBER);")
box.execute("INSERT INTO t VALUES (1,1);")
box.execute("SELECT i / 2, n / 2 FROM t GROUP BY n;")

Result:
tarantool> box.execute("SELECT i / 2, n / 2 FROM t GROUP BY n;")
---
- metadata:
  - name: COLUMN_1
    type: number
  - name: COLUMN_2
    type: number
  rows:
  - [0.5, 0.5]
...

This patch removes OP_Realify, after which these errors disappear.

Closes #5335

Prior to this patch when DOUBLE value that less than 0.0 and greater
than -1.0 was cast to INTEGER, it was considered to be negative number
though the result was 0. This patch fixes this, so now such DOUBLE
value will be properly cast to INTEGER and UNSIGNED.

Closes #6225

This patch adds changelog entry for commit
6a420c25 ('luajit: bump new version')
and commit 20c0a12a ('test: enable JIT
for Lua Fun chain iterator back').

Follows up #5118
Follows up #4252
Follows up #5049

Signed-off-by: Igor Munkin <imun@tarantool.org>

This patch reverts the temporary fix introduced in commit
5fa7ded2 ("test: disable JIT for Lua Fun
chain iterator") since the issues with invalid traces generation for
<fun.chain> iterator are resolved and JIT can be enabled back then.

Follows up #4252
Follows up #5118
Relates to #5049

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
Reviewed-by: Sergey Kaplun <skaplun@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

* Detect inconsistent renames even in the presence of sunk values.

Closes #5118
Follows up #4252

Currently, we use len() to find length of string and then we use this value to calculate width of column. The problem is len() returns size of string in bytes, not number of characters in string. That is why alignment of space:fselect() output is being broken by multibyte characters. To avoid this, use utf8.len() instead of len().

The alignment will be still broken, if there are characters with variable length in space.

Closes #6165

decimal conversion to string in Lua used decimal_str() function.
The function is not safe to use in preemptive context like Lua,
where any attempt to push something onto the Lua stack might
trigger GC, which in turn might invoke any other code.

It is not safe because uses the static buffer, which is global and
cyclic. Newer allocations can override the old data without any
warning.

The same problem was fixed for tt_uuid_str() and uuids in
box.info in one of the previous commits.

The patch adds a new function lua_pushdecimalstr() which does not
use the static buffer. It is now used to push decimals safely on a
Lua stack.

Follow up #5632
Follow up #6050
Closes #6259

It saves decimal as a string into an externally passed buffer.
This will be used by places which can not use the static buffer
returned by decimal_str().

Part of #6259

To be consistent with tt_uuid_str() and tt_uuid_to_string().
_str() returns a string. _to_string() copies it into an externally
passed buffer.

Part of #6259

box.info.uuid, box.info.cluster.uuid, and box.info.* replica UUIDs
used tt_uuid_str() function. The function is not safe to use in
preemptive context like Lua, where any attempt to push something
onto the Lua stack might trigger GC, which in turn might invoke
any other code.

It is not safe because uses the static buffer, which is global and
cyclic. Newer allocations can override the old data without any
warning.

Follow up #5632
Follow up #6050
Part of #6259

The function safely pushes tt_uuid as a string on a Lua stack.
Safety means that it does not use tt_uuid_str() which stores the
result into the global static buffer and can not be used in Lua
context.

The static buffer is not safe to use in Lua and Lua C because
during a static string push onto a Lua stack the GC might be
started and it can spoil the buffer.

Part of #6259

This patch moves method_decoder table from Lua to C. This is a step
towards rewriting performance-critical parts of net.box in C.

Part of #6241

Before the patch, instead of checking stdout,
the return result is checked.

Affected test: "check answers in case of call".
Affected test case: "check 'eval' stdout for 'good_script ok_script.lua'".

* test: disable interactive mode assertions on BSD
* test: update lua-Harness to c4451fe
* test: support tarantool cli in lua-Harness
* test: backport lua-Harness directory detection
* test: support Tarantool in lua-Harness
* test: refactor with _dofile
* test: refactor with _retrieve_progname
* test: use CI friendly variables in lua-Harness
* test: rename lua-Harness tap to test_assertion
* test: port lua-Harness to Test.Assertion

Closes #5970
Part of #4473

Currently, an mpstream is initialized with the Lua error handler in
netbox_prepare_request, which is used by all encoding methods, including
netbox_encode_auth. The latter will be moved to C, along with iproto
request handlers, where we will have to use a different error handler.
Let's create an mpstream in netbox_encode_method and netbox_encode_auth
instead. For now, they do the same, but once we move the code to C they
will use different error handlers.

Part of #6241

This patch moves method_encoder table from Lua to C. This is a step
towards rewriting performance-critical parts of net.box in C.

Part of #6241

Request context only stores tuple format or nil, which is used for
decoding a response. Rename it appropriately.

Part of #6241

Replica registration works via looking for the smallest not
occupied ID in _cluster and inserting it into the space.

It works not so good when mvcc is enabled. In particular, if more
than 1 replica try to register at the same time, they might get
the same replica_id because don't see changes of each other until
the registration in _cluster is complete.

This in the end leads to all replicas failing the registration
except one with the 'duplicate key' error (primary index in
_cluster is replica ID).

The patch makes the replicas occupy their ID before they commit it
into _cluster. And new replica ID search now uses the replica ID
map instead of _cluster iterator.

This way the registration works like before - like MVCC does not
exist which is fine.

Closes #5601

We'll need them to reimplement parts of net.box in C.

Part of #6241

luamp_encode copies its argument to the top of Lua stack unless it's
already on the top so netbox_encode_{update,upsert} encode key/tuple and
ops in reverse order to avoid the copy. There's no much point in it,
because copying an argument in Lua stack costs nearly nothing - it's
just a pointer copy. I ran a simple net.box test doing upserts and saw
no difference with and without this optimization. Let's remove it,
because it's easier to rewrite parts of net.box in C without it.

Part of #6241

We need this to port performance-critical parts of net.box to C, where
it's easier and more efficient to look up encoders/decoders by integer
id rather than by string method name.

Part of #6241

The only difference between the two is that the latter ensures that the
response body has no more than one tuple. There's no point in this
check, because get/min/max never return more than one tuple in
Tarantool.

Note, since decode_get was the only method returning an error, we don't
need to set request.errno after calling a decoder anymore.

Part of #6241

No need to have it there, since we can call it directly.

Part of #6241

request.id and request.errno are set appropriately whenever a connection
is closed so there's no need to check if worker_fiber is running. We do
not want to have this check in request methods so that we can move its
implementation to C, where worker_fiber isn't (and won't be) available.

Part of #6241

request.discard() doesn't wake up fibers blocked in
request.wait_result() as a result they hang until timeout.
Fix this and add a test.

Closes #6250

Referenced a wrong ticket :-(

Follow-up 22b49404 ("net.box: fix console connection breakage when
request is discarded").

Follow-up 22b49404 ("net.box: fix console connection breakage when
request is discarded").