See https://github.com/tarantool/checkpatch

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

 - Remove lz4 from link libraries, because it can be linked via
   EXTRA_CORE_LINK_LIBRARIES. It'd be more correct, because the
   open-source doesn't use lz4. Follow-up commit 55f968b4 ("msgpack:
   add stubs for compressed data support").

 - Add OpenSSL include directories unconditionally, because they are
   used in the open-source build since commit e5ec324d ("Move
   OpenSSL initialization to lib/core/ssl").

NO_DOC=cmake
NO_CHANGELOG=cmake

Limbo terms tracking is shared between appliers and when
one of appliers is waiting for write to complete inside
journal_write() routine, an other may need to access read
term value to figure out if promote request is valid to
apply. Due to cooperative multitasking access to the terms
is not consistent so we need to be sure that other fibers
read up to date terms (ie written to the WAL).

For this sake we use a latching mechanism, when one fiber
takes a lock for updating other readers are waiting until
the operation is complete.

For example here is a call graph of two appliers

applier 1
---------
applier_apply_tx
  (promote term = 3
   current max term = 2)
  applier_synchro_filter_tx
  apply_synchro_row
    journal_write
      (sleeping)

at this moment another applier comes in with obsolete
data and term 2

                              applier 2
                              ---------
                              applier_apply_tx
                                (term 2)
                                applier_synchro_filter_tx
                                  txn_limbo_is_replica_outdated -> false
                                journal_write (sleep)

applier 1
---------
journal wakes up
  apply_synchro_row_cb
    set max term to 3

So the applier 2 didn't notice that term 3 is already seen
and wrote obsolete data. With locking the applier 2 will
wait until applier 1 has finished its write.

We introduce the following helpers:

1) txn_limbo_begin: which takes a lock
2) txn_limbo_commit and txn_limbo_rollback which simply release
   the lock but have different names for better semantics
3) txn_limbo_process is a general function which uses x_begin
   and x_commit helper internally
4) txn_limbo_apply to do a real job over processing the
   request, it implies that txn_limbo_begin been called

Testing such in-flight condition won't be easy so we introduce
"box.info.synchro.queue.busy" field to report if limbo is
currently latched and processing a sync request.

@TarantoolBot document
Title: synchronous replication changes

`box.info.synchro.queue` gets a new `busy` field. It is set to
`true` when there is a synchronous transaction is processing
but not yet complete. Thus any other incoming synchronous
transactions will be delayed until active one is finished.

Part-of #6036

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

To test if latch is locked.

Part-of #6036

NO_DOC=Internal helper for next patch
NO_CHANGELOG=Internal helper for next patch

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

To build EE tests, we need to add some new CMake variables:

 - Set EXTRA_TEST_SUITES_{BINARY,SOURCE}_DIR to the binary/source directory
   that contains extra test suites.
 - Set EXTRA_TEST_SUITES to the list of extra test suites to build.

NO_DOC=cmake
NO_CHANGELOG=cmake

When a request is redirected, old headers are of no use anymore and
should be dropped. We can detect a redirect by checking the value of
CURLINFO_REDIRECT_COUNT.

Note about the test: we change the redirect reply to 'redirecting...',
so that its length differs from 'hello world'.

Close #6101

NO_DOC=bug fix

NO_DOC=test
NO_CHANGELOG=test

NO_DOC=test
NO_CHANGELOG=test

In some cases, when calling a user-defined function, some of the memory
allocated earlier could be lost. This patch fixes this problem.

Closes #6789

NO_DOC=No need for doc in this case

* ci: introduce GitHub Actions

NO_DOC=LuaJIT submodule bump
NO_CHANGELOG=no visible changes

This commit add stub functions for tuple compression. Now after new
tuple allocation, we check if space format contains compression for
some fields and if it so, we compress new allocated tuple. When we
return tuple from space to user (from box_select or using C API), we
check if tuple is compressed and if it is so decompress tuple.
In opensource version user unable to set compression for tuple fields,
so all stub functions are unreachable.

Closes #2695

NO_CHANGELOG=stubs for enterprise version
NO_DOC=stubs for enterprise version

Currently `index_build` function used only for build secondary
indexes in memtx engine. Move and rename it to simplify subsequent
implementation of tuple compression.

Part of #2695

NO_CHANGELOG=refactoring
NO_DOC=refactoring

Currently `index_get` and `iterator_next` functions return
result tuple as is, without any transformation, because
tuple is stored in the space in the same format in which
it is returned to the user. Later after tuple compression
implementation (and maybe some other features) tuple should
be transformed before return it to user. Rework `index_get`
and `iterator_next` functions - now they are expected to
return the tuple in format appropriate for user. Implement
`index_get_raw` and `iterator_next_raw` functions, which
return tuple as is.

Part of #2695

NO_CHANGELOG=internal changes
NO_DOC=internal changes

When implementing tuple compression, we need to change memtx
indexes vtab dynamically, when we change space format. For
this purpose we implement helpful functions for memtx index
vtab selection.

Part of #2695

NO_CHANGELOG=refactoring
NO_DOC=refactoring

When implementing tuple compression, it is planned
to use templates to select index vtab. Templates
enabled only in c++, so we convert all associated
files from *.c to *.cc.

Part of #2695

NO_CHANGELOG=refactoring
NO_DOC=refactoring

Currently memtx use stmt->old_tuple and stmt->new_tuple
for rollback, but later when tuple compression will be
implemented we need special place to store tuples, which
was inserted or deleted during transaction, because
stmt->old_tuple and stmt->new_tuple are used in on_replace
triggers and should be decompressed.

Part of #2695

NO_CHANGELOG=internal changes
NO_DOC=internal changes

The purpose of this commit is code refactoring, such that all
the work which is common to insert/replace/delete/update/upsert
functions in memtx space is done in one place.

Part of #2695

NO_CHANGELOG=refactoring
NO_DOC=refactoring

Implement ability to set compression for tuple fields. Compression type
for tuple fields is set in the space format, and can be set during space
creation or during setting of a new space format.
```lua
format = {{name = 'x', type = 'unsigned', compression = 'none'}}
space = box.schema.space.create('memtx_space', {format = format})
space:drop()
space = box.schema.space.create('memtx_space')
space:format(format)
```
For opensource build only one compression type ('none') is
supported. This type of compression means its absence, so
it doesn't affect something.

Part of #2695

NO_CHANGELOG=stubs for enterprise version
NO_DOC=stubs for enterprise version

Library `tuple` depends on the `coll` library, since `key_def.c` file
from the `tuple` library use functions from the `coll` library. This
usually does not cause problems, since the `coll` library links with
tarantool and resolves symbols in `tuple` library. For tests it is not
a problem, because functions from the `tuple` library, which depends
from the `coll` library are `static inline` and they they are not used
in tests. But adding not `static inline` function in `tuple` library
which depends on `coll` library leads to unresolved symbols in tests.
Fix this explicitly adding `coll` library as a target link library
for `tuple` library. Also remove `${ICU_LIBRARIES}` from target link
libraries of `tuple` library because `tuple` library use ICU only via
`coll` library.

NO_CHANGELOG=cmake
NO_DOC=cmake

This commit add stub functions for compressed data
support. Add new MP_EXT type MP_COMPRESSION. Despite
the fact that this type is added to open source version
all functions for working with it are implemented only
in enterprise version of tarantool. In open source version
add stubs for appropriate functions.

Part of #2695

NO_CHANGELOG=stubs for enterprise version
NO_DOC=stubs for enterprise version

Currently, we don't check tuple format in before_replace triggers,
that's why some bugs happen if we don't use the triggers correctly.

Let's check tuple format before execution of before_replace triggers
and after each before_replace trigger. The check will be disabled
during recovery for backward compatibility.

Closes #6780

NO_DOC=bug fix

The test checks key validation in before_replace trigger, but it
uses replace() to check it, and, as we plan to check tuple format in
before_replace trigger, we need to use something that works with
key, not tuple: delete(), for example.

Part of #6780

NO_DOC=refactoring
NO_CHANGELOG=refactoring

Technically the box region always was fiber local. However we had an
idea to make it global in a future, so we left a room in the guarantees
stated by the documentation.

Now I think that we should make it officially fiber local. If it is
global, one can't use it across fiber yields (say, to collect something
from a network).

Also I don't like so vague guarantees. Just hard to use an object, which
is either global or fiber local. Need to interpret a code in the head
twice.

Follows up commit c6117909 ('module api:
expose box region').

NO_DOC=I will add it to https://github.com/tarantool/doc/issues/1611
NO_CHANGELOG=no behaviour changes

Follows up #5273

A secondary index creation proceeds as follows:
 1. Build the new index by inserting statements from the primary index,
    see vinyl_space_build_index().
 2. Dump the new index and wait for the dump to complete.
 3. Commit the index creation record to the WAL.

While the new index is being dumped at step 2, new statements may be
inserted into the space. We need to insert those statements during
recovery, see vy_build_recover(). We identify such statements by
comparing LSN to vy_lsm::dump_lsn, see vy_build_recover_stmt().

It might occur that the newly built index is empty while the primary
index memory level isn't - if all statements cancel each other. In this
case, the secondary index won't be dumped during creation and its
dump_lsn will be set to -1, see the vy_lsm_is_empty() check in
vinyl_space_build_index(). This would break the assumption made on
recovery: that all statements with LSN > vy_lsm::dump_lsn should be
inserted into the secondary index. If a statement like this isn't
compatible with the new index, we will get a crash trying to insert it.

Let's fix this issue by skipping vy_build_recover() in case the new
secondary index was never dumped.

Closes #6778

NO_DOC=bug fix

This patch allows LIMIT to be used with a different sort order in
ORDER BY. This is a temporary solution and should be changed after
issue #3309 is fixed.

NO_DOC=It is fix of existing feature

Closes #6664

Return a more meaningful message about a malformed opts to
select, count and pairs methods of index object. Do not fail
with luajit error.

Also allow to pass specific iterator as a box.index.{ALL,...} directly.
For example, `s:pairs(nil, box.index.GT)`.

Fixes #6501

NO_DOC=bug fix

Currently, the library is initialized in lib/crypto. Let's move it to
lib/core/ssl, because:
 - lib/crypto links lib/core/ssl so it looks more logical to have
   OpenSSL initialized in the core lib.
 - We need to extend the OpenSSL initialization code in the EE build,
   which doesn't affect lib/crypto, but replaces lib/core/ssl.

While we are at it, let's also allow the EE build to link extra
libraries to lib/core by setting EXTRA_CORE_LINK_LIBRARIES.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

Useful to run some tests only on static build.

@TarantoolBot document
Title: Document tarantool.build.linking

It shows linking type (dynamic or static):

```
tarantool> require('tarantool').build.linking
---
- dynamic
...
```

Please update the tarantool module documentation:

https://www.tarantool.io/en/doc/latest/reference/reference_lua/tarantool/

Previous behaviour was quite illogical and unexpected. If user
once defined strict mode at start of tests it's expected that all
subtests also will have strict mode. However before this patch it
wasn't so. And wasn't clear at start why enabled strict mode
didn't work.
After this patch subtests strict mode will be the same as for
parent. This behaviour wasn't tested anyhow and wasn't documented.

Follow-up #4125

@TarantoolBot document
Title: clarify 'strict' behaviour in tap tests
Defined for root tap object strict mode will be the same for all
subtests.

Example:
```lua
t = require('tap').test('123')
t.strict = true

t:is_deeply({a = box.NULL}, {}) -- false

t:test('subtest', function(t)
    t:is_deeply({a = box.NULL}, {}) -- also false
end)
```

Before this patch comparison of two `false` values produced
incorrect result. That because nil and false are consieded in the
same way by if operator. This patch fixes an issue and introduces
corresponding tests.

Closes #6386

NO_DOC=bugfix

* Enabled UNIT_TAP_COMPATIBLE mode in datetime.c
* Got rid of unnecessary datetime.result

Followup to #5000, #6731, #6796
NO_DOC=internal
NO_CHANGELOG=internal

TAP (Test Anything Protocol) output must indicate that
this is TAP13 in the first line. See specification [1].

[1] https://testanything.org/tap-version-13-specification.html



Introducing in unit.h UNIT_TAP_COMPATIBLE define to activate TAP-compliant
mode, which may be used by single test to generate TAP output, and get rid
of result file.

So one could enable new mode this way

  /* New test. */
  #define UNIT_TAP_COMPATIBLE 1
  #include <unit.h>

While omitting of this define would still use older (incompatible) mode.

  /* Old test. */
  #include <unit.h>

Co-authored-by: Sergey Bronnikov <sergeyb@tarantool.org>

Followup to #5000, #6731, #6796
NO_DOC=internal
NO_CHANGELOG=internal

Msgpuck is a submodule for tarantool and is needed while packaging.
Moved function of mp_encode_str0() from mp_error.cc into a common
place is msgpuck.h.

Needed for packaging workflow in tarantool/tarantool:
tarantool/tarantool#6260

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes

The luatest_helpers.lua module confuses developers because luatest
itself has the helpers.lua module as well. It is unclear which one
should be used and when because it is hard to remember what stuff
provides each of them.

Actually, luatest_helpers.lua has functions that can be safely moved to
more proper places (luatest_helpers/server.lua & instances/default.lua).

The `get_vclock` function logically belongs to server stuff, so it was
moved to server.lua and slightly changed (`eval` call was replaced by
`exec`).

The `wait_vclock` function logically belongs to server stuff as well, so
it was moved to server.lua without changes.

The `instance_uri` function logically belongs to server stuff as well,
so it was moved to server.lua and slightly changed (it was renamed into
`build_instance_uri` and redundant `instance_id` param was dropped).

The `default_cfg`, `env_cfg`, `box_cfg` functions logically belongs to
instance stuff, so they were moved to default.lua without changes.

Thus, the luatest_helpers.lua module was dropped and tests using this
module were updated.

Closes tarantool/luatest#192

NO_DOC=testing stuff
NO_CHANGELOG=testing stuff

The luatest_helpers/asserts.lua module confuses developers because
luatest itself has the asserts.lua module as well. It is unclear which
one should be used and when because it is hard to remember what stuff
provides each of them.

Actually, luatest_helpers/asserts.lua has functions that are not really
about asserting and can be safely moved to more proper places like the
luatest_helpers/server.lua and luatest_helpers/cluster.lua modules.

The `assert_server_follow_upstream` function logically belongs to server
stuff, so it was moved to server.lua and slightly changed (renamed into
`assert_follows_upstream` and `eval` calls were replaced by `exec`).

The `wait_fullmesh` function logically belongs to cluster stuff, so it
was moved to cluster.lua and slightly changed (now it accepts one param
that can include wait timeout and delay, otherwise, defaults applied).

Thus, the luatest_helpers/asserts.lua module was dropped and tests using
this module were updated.

Part of tarantool/luatest#192

NO_DOC=testing stuff
NO_CHANGELOG=testing stuff

We have recently dropped 'checksum' machinery in test-run
(tarantool/test-run#321) and now there is no need to keep
checksums in suite.ini config files. So removing them.

Closes tarantool/tarantool-qa#152

* test: adapt tests checking loading bytecode files
* test: adapt disabled tests from PUC-Rio
* test: adapt tests checking traceback in tail call
* test: adapt test checking global environment

Closes #5687
Closes #5691
Closes #5703
Closes #5713
Part of #5870

NO_DOC=LuaJIT submodule bump
NO_CHANGELOG=no visible changes

Bump test-run to new version with the following improvements:

  - drop 'checksum' machinery (tarantool/test-run#316)
  - rerun fragile tests only with status == 'fail'

NO_DOC=testing stuff
NO_CHANGELOG=testing stuff

The function of the question (`tarantoolsqlIdxKeyCompare()`) compares a
tuple with a key. It extracts necessary fields from the tuple and runs
the compare function, which is basically `memcmp()` of two buffers (for
`varbinary` values -- numeric comparisons are different).

In the Debug build the function also re-verifies the result using a
separate comparison code. And here we compare two `memcmp()` return
values.

We shouldn't do it directly, because `memcmp()` only guarantees sign of
the result. Please, consider the linked issue for details.

NO_DOC=only debug build is affected
NO_CHANGELOG=only debug build is affected
NO_TEST=hard to create a stable reproducer

Fixes #6849

The broadcast doesn't need to go into the network, but
it will be needed locally for a built-in event. So
called "box.election".

Part of #6260

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes