tuple_format_new has lots of arguments, all of them necessary
indeed. But a small analysss showed that almost always there are
only two kinds of usage of that function: with lots of zeros as
arguments and lots of values taken from space_def.

Make two versions of tuple_format_new:
simple_tuple_format_new, with all those zeros omitted, and
space_tuple_format_new, that takes space_def as an argument.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

Introduce field constraints - limitaions for particular fields.
Each constraint must refer to a function in _func space. For the
first step we expect lua functions with body there.

Field constraint checks are very close to field type checks, so
it's natural to implement them in tuple formats. On the other hand
tuple formats belong to tuple library, that does not include
functions (func.h/c etc), so constraints are split into two parts:
- a part in tuple library that implements arbitrary constraints
 with pointers to functions that actually check constraints.
- a part in box library which uses the part above, sets particular
 check functions and handles alter etc.

There are two not-so-obvious problems that are solved here:
 - Functions in _func space must be preserved while used by such
 constraints. Func pinning is used for this purpose.
 - During initial recovery constraits are created before _func
 space recovery, so we have to pospone constraint initialization.

One can set up constraint for any field in tuple format with one
or several functions that must be present in _func space:
space:format{name='a', constraint='func1'}
space:format{name='a', constraint={name1='func1'}}
space:format{name='a', constraint={name1='func1', name2='func2'}}

So constraint(s) can be set by one function name or by lua table
with function name values. Each consraints has a name that can be
specified directly (with string key in table) or imlicitly set to
the name of function.

The check function receives two arguments: the checking value and
the name of the constraint. Also the name of the failed constraint
is present in raised exception.

The only way to pass the constraint is to return true from its
function. All other values and exception are treated as failure
(exeptions are also logged).

NO_DOC=see later commits
NO_CHANGELOG=see later commits

gpr_alloc is a small library that is designed for simplification
of allocation of several objects in one memory block. It could be
anything, but special attention is given to string objects, that
are arrays of chars.

Typical usage consist of two phases: gathering total needed size
of memory block and creation of objects in given block.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

There are cases when we need to be sure that a function is not
deleted and/or removed from func cache. For example constraints:
they must hold a pointer to struct func while it's very hard to
determine whether there'a constraint that points to given func.

Implement func pin/unpin for this purpose. You can pin a func to
declare that the func must not be deleted. To have to unpin it
when the func is not needed anymore.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

Some non-important fixes that are not related to any issue.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

fselect is designed to make selects results in console better
readable for human eyes. The usage is expected to be rare and
by developers only so there's no performance requirements.

Now the API of fselect is:
s:fselect(<key>, <select+fselect option map>, <fselect option map>)

But sometimes there are cases when only some columns are needed
for analysis, while all other columns consume space on screen.

So make an option in fselect that allows to choose which columns
to output (like some other databases allows).

Add an option 'columns' in fselect options along with other its
options with list of columns (by number or name). Allow lua tables
and comma-separated-strings for that.

If options argument in fselect is string, interpret is as columns
options.

NO_DOC=There's no doc about fselect yet, I'll append these change
to the existing doc ticket
NO_CHANGELOG=minor changes in minor feature

Added support to netbox's `stream` and `future` objects using the
__autocomplete metamethod.

Closes #6305
NO_DOC=internal

This patch updates the logic in lua_rl_complete() where __index from
a metatable apparently expected to be a table, not a function. It is
rather simple for a table object, but some of them are userdata, so
an __autocomplete method is introduced to be sure it returns a table
with all names suitable for tabcompletion.

Part of #6305
NO_DOC=internal
NO_CHANGELOG=part of a fix

When the deferred DELETE optimization was introduced, it was enabled for
all Vinyl spaces. Though the optimization  should improve write
performance, at the same time it may result in significant degradation
of reads. Let's disable this optimization by default and add a space
option to enable it.

Closes #4501

@TarantoolBot document
Title: Document defer_deletes space option

The new option is a boolean flag that only makes sense for Vinyl spaces
that have secondary indexes. Setting the flag results in deferring
generation of DELETE statements for secondary indexes till compaction of
the primary index. It should speed up writes, because it eliminates a
lookup in the space for REPLACE and DELETE operations. At the same time,
it may result in degradation of reads from secondary indexes, because it
entails an additional lookup in the primary index per each "phantom"
tuple (a tuple that was deleted in the primary index, but not yet in the
secondary index, and hence should be skipped on read).

Example:

```lua
box.schema.space.create('test', {
    engine = 'vinyl',
    defer_deletes = true,
})
```

If omitted on space creation, this option is set to the value of
`box.cfg.vinyl_defer_deletes`, which is false by default.

Tarantool console quits if you type Ctrl+C.
This patch fixes console behavior on sending SIGINT.
Console discards the current input on typing Ctrl+C
and invites user to the new line.

In daemon mode the process will exit after receiving SIGINT.

Test gh-2717 should be skipped on release build, cause it
uses error injection which is enabled only on debug mode.

Fixes #2717

@TarantoolBot document
Title: Use Ctrl+C to discard the input in console

The signal SIGINT discards the input in console mode.
When tarantool executes with -e flag or runs as a daemon, SIGINT
kills the process and tarantool complains about it in log.

During DDL operations triggers of old space and new created space
are swapped. This leads to crash in case if this swap occurs from
space on_replace triggers. This patch banned all DDL operations
from on_replace triggers.

Closes #6920

@TarantoolBot document
Title: Ban DDL operations from space on_replace triggers

Previously user can set function for space on_replace trigger,
which performs DDL operation. This may leads to tarantool crash,
that's why this patch bans DDL operations from on_replace triggers.
All this operations fails with error: "Space on_replace trigger
does not support DDL operations".

This patch fixes an assertion or segmentation fault when getting the
value of MP_EXT via netbox.

Closes #6766

NO_DOC=Bugfix

netbox_perform_request throws a general ClientError exception on
time-out error: it should throw the more suitable and POSIX-compliant
TimedOut exception which sets errno to ETIMEDOUT.

Closes #6144

NO_DOC=There is no mention of what kind of error is returned on timeout
in the net.box documentation.

Part of #6773

NO_DOC=Doc-request will be added in another commit.
NO_CHANGELOG=Changelog will be added in another commit.

This patch introduces basic DATETIME support in SQL. After this patch,
it will be allowed to select DATETIME values from spaces, insert them
into spaces, and use them in functions. CAST() from DATETIME to STRING,
SCALAR and ANY is also supported.

Part of #6773

NO_DOC=Doc-request will be added in another commit.
NO_CHANGELOG=Changelog will be added in another commit.

NO_CHANGELOG=feature was not released

Closes #6823

@TarantoolBot document
Title: Document msgpack object iterator take_array method

The new method copies the given number of msgpack values starting from
the iterator cursor position to a new msgpack array object. On success
it returns the new msgpack object and advances the iterator cursor. If
there isn't enough values to decode, it raises a Lua error and leaves
the iterator cursor unchanged.

This function could be implemented in Lua like this:

```lua
function take_array(iter, count)
    local array = {}
    for _ = 1, count do
        table.insert(array, iter:take())
    end
    return msgpack.object(array)
end
```

Usage example:

```lua
local mp = msgpack.object({10, 20, 30, 40})
local it = mp:iterator()
it:decode_array_header()       -- returns 4
it:decode()                    -- returns 10
local mp2 = it:take_array(2)
mp2:decode()                   -- returns {20, 30}
it:decode()                    -- returns 40
```

Normalization was unnecessary in negative interval values
close to 0. This led to extra second in result, like:

```
tarantool> date.now() - date.now()
---
- -1.000026000 seconds
...
```

We do need to normalize when assign to datetime values, but not for
intermediate intervals.

Closes #6882

NO_DOC=bugfix

The problem is memtx_tuple_new() says address of memtx_tuple, while
memtx_tuple_delete() says address of an actual tuple (part of memtx_tuple).
Different addresses in log may be confusing so the patch fixes the problem.

Closes #6634

NO_CHANGELOG=no visible changes
NO_DOC=no visible changes

Limbo terms tracking is shared between appliers and when
one of appliers is waiting for write to complete inside
journal_write() routine, an other may need to access read
term value to figure out if promote request is valid to
apply. Due to cooperative multitasking access to the terms
is not consistent so we need to be sure that other fibers
read up to date terms (ie written to the WAL).

For this sake we use a latching mechanism, when one fiber
takes a lock for updating other readers are waiting until
the operation is complete.

For example here is a call graph of two appliers

applier 1
---------
applier_apply_tx
  (promote term = 3
   current max term = 2)
  applier_synchro_filter_tx
  apply_synchro_row
    journal_write
      (sleeping)

at this moment another applier comes in with obsolete
data and term 2

                              applier 2
                              ---------
                              applier_apply_tx
                                (term 2)
                                applier_synchro_filter_tx
                                  txn_limbo_is_replica_outdated -> false
                                journal_write (sleep)

applier 1
---------
journal wakes up
  apply_synchro_row_cb
    set max term to 3

So the applier 2 didn't notice that term 3 is already seen
and wrote obsolete data. With locking the applier 2 will
wait until applier 1 has finished its write.

We introduce the following helpers:

1) txn_limbo_begin: which takes a lock
2) txn_limbo_commit and txn_limbo_rollback which simply release
   the lock but have different names for better semantics
3) txn_limbo_process is a general function which uses x_begin
   and x_commit helper internally
4) txn_limbo_apply to do a real job over processing the
   request, it implies that txn_limbo_begin been called

Testing such in-flight condition won't be easy so we introduce
"box.info.synchro.queue.busy" field to report if limbo is
currently latched and processing a sync request.

@TarantoolBot document
Title: synchronous replication changes

`box.info.synchro.queue` gets a new `busy` field. It is set to
`true` when there is a synchronous transaction is processing
but not yet complete. Thus any other incoming synchronous
transactions will be delayed until active one is finished.

Part-of #6036

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

To build EE tests, we need to add some new CMake variables:

 - Set EXTRA_TEST_SUITES_{BINARY,SOURCE}_DIR to the binary/source directory
   that contains extra test suites.
 - Set EXTRA_TEST_SUITES to the list of extra test suites to build.

NO_DOC=cmake
NO_CHANGELOG=cmake

When a request is redirected, old headers are of no use anymore and
should be dropped. We can detect a redirect by checking the value of
CURLINFO_REDIRECT_COUNT.

Note about the test: we change the redirect reply to 'redirecting...',
so that its length differs from 'hello world'.

Close #6101

NO_DOC=bug fix

NO_DOC=test
NO_CHANGELOG=test

NO_DOC=test
NO_CHANGELOG=test

Implement ability to set compression for tuple fields. Compression type
for tuple fields is set in the space format, and can be set during space
creation or during setting of a new space format.
```lua
format = {{name = 'x', type = 'unsigned', compression = 'none'}}
space = box.schema.space.create('memtx_space', {format = format})
space:drop()
space = box.schema.space.create('memtx_space')
space:format(format)
```
For opensource build only one compression type ('none') is
supported. This type of compression means its absence, so
it doesn't affect something.

Part of #2695

NO_CHANGELOG=stubs for enterprise version
NO_DOC=stubs for enterprise version

Currently, we don't check tuple format in before_replace triggers,
that's why some bugs happen if we don't use the triggers correctly.

Let's check tuple format before execution of before_replace triggers
and after each before_replace trigger. The check will be disabled
during recovery for backward compatibility.

Closes #6780

NO_DOC=bug fix

The test checks key validation in before_replace trigger, but it
uses replace() to check it, and, as we plan to check tuple format in
before_replace trigger, we need to use something that works with
key, not tuple: delete(), for example.

Part of #6780

NO_DOC=refactoring
NO_CHANGELOG=refactoring

A secondary index creation proceeds as follows:
 1. Build the new index by inserting statements from the primary index,
    see vinyl_space_build_index().
 2. Dump the new index and wait for the dump to complete.
 3. Commit the index creation record to the WAL.

While the new index is being dumped at step 2, new statements may be
inserted into the space. We need to insert those statements during
recovery, see vy_build_recover(). We identify such statements by
comparing LSN to vy_lsm::dump_lsn, see vy_build_recover_stmt().

It might occur that the newly built index is empty while the primary
index memory level isn't - if all statements cancel each other. In this
case, the secondary index won't be dumped during creation and its
dump_lsn will be set to -1, see the vy_lsm_is_empty() check in
vinyl_space_build_index(). This would break the assumption made on
recovery: that all statements with LSN > vy_lsm::dump_lsn should be
inserted into the secondary index. If a statement like this isn't
compatible with the new index, we will get a crash trying to insert it.

Let's fix this issue by skipping vy_build_recover() in case the new
secondary index was never dumped.

Closes #6778

NO_DOC=bug fix

This patch allows LIMIT to be used with a different sort order in
ORDER BY. This is a temporary solution and should be changed after
issue #3309 is fixed.

NO_DOC=It is fix of existing feature

Closes #6664

Return a more meaningful message about a malformed opts to
select, count and pairs methods of index object. Do not fail
with luajit error.

Also allow to pass specific iterator as a box.index.{ALL,...} directly.
For example, `s:pairs(nil, box.index.GT)`.

Fixes #6501

NO_DOC=bug fix

Currently, the library is initialized in lib/crypto. Let's move it to
lib/core/ssl, because:
 - lib/crypto links lib/core/ssl so it looks more logical to have
   OpenSSL initialized in the core lib.
 - We need to extend the OpenSSL initialization code in the EE build,
   which doesn't affect lib/crypto, but replaces lib/core/ssl.

While we are at it, let's also allow the EE build to link extra
libraries to lib/core by setting EXTRA_CORE_LINK_LIBRARIES.

NO_DOC=refactoring
NO_CHANGELOG=refactoring

Useful to run some tests only on static build.

@TarantoolBot document
Title: Document tarantool.build.linking

It shows linking type (dynamic or static):

```
tarantool> require('tarantool').build.linking
---
- dynamic
...
```

Please update the tarantool module documentation:

https://www.tarantool.io/en/doc/latest/reference/reference_lua/tarantool/

Previous behaviour was quite illogical and unexpected. If user
once defined strict mode at start of tests it's expected that all
subtests also will have strict mode. However before this patch it
wasn't so. And wasn't clear at start why enabled strict mode
didn't work.
After this patch subtests strict mode will be the same as for
parent. This behaviour wasn't tested anyhow and wasn't documented.

Follow-up #4125

@TarantoolBot document
Title: clarify 'strict' behaviour in tap tests
Defined for root tap object strict mode will be the same for all
subtests.

Example:
```lua
t = require('tap').test('123')
t.strict = true

t:is_deeply({a = box.NULL}, {}) -- false

t:test('subtest', function(t)
    t:is_deeply({a = box.NULL}, {}) -- also false
end)
```

Before this patch comparison of two `false` values produced
incorrect result. That because nil and false are consieded in the
same way by if operator. This patch fixes an issue and introduces
corresponding tests.

Closes #6386

NO_DOC=bugfix

* Enabled UNIT_TAP_COMPATIBLE mode in datetime.c
* Got rid of unnecessary datetime.result

Followup to #5000, #6731, #6796
NO_DOC=internal
NO_CHANGELOG=internal

TAP (Test Anything Protocol) output must indicate that
this is TAP13 in the first line. See specification [1].

[1] https://testanything.org/tap-version-13-specification.html



Introducing in unit.h UNIT_TAP_COMPATIBLE define to activate TAP-compliant
mode, which may be used by single test to generate TAP output, and get rid
of result file.

So one could enable new mode this way

  /* New test. */
  #define UNIT_TAP_COMPATIBLE 1
  #include <unit.h>

While omitting of this define would still use older (incompatible) mode.

  /* Old test. */
  #include <unit.h>

Co-authored-by: Sergey Bronnikov <sergeyb@tarantool.org>

Followup to #5000, #6731, #6796
NO_DOC=internal
NO_CHANGELOG=internal

Msgpuck is a submodule for tarantool and is needed while packaging.
Moved function of mp_encode_str0() from mp_error.cc into a common
place is msgpuck.h.

Needed for packaging workflow in tarantool/tarantool:
tarantool/tarantool#6260

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes

The luatest_helpers.lua module confuses developers because luatest
itself has the helpers.lua module as well. It is unclear which one
should be used and when because it is hard to remember what stuff
provides each of them.

Actually, luatest_helpers.lua has functions that can be safely moved to
more proper places (luatest_helpers/server.lua & instances/default.lua).

The `get_vclock` function logically belongs to server stuff, so it was
moved to server.lua and slightly changed (`eval` call was replaced by
`exec`).

The `wait_vclock` function logically belongs to server stuff as well, so
it was moved to server.lua without changes.

The `instance_uri` function logically belongs to server stuff as well,
so it was moved to server.lua and slightly changed (it was renamed into
`build_instance_uri` and redundant `instance_id` param was dropped).

The `default_cfg`, `env_cfg`, `box_cfg` functions logically belongs to
instance stuff, so they were moved to default.lua without changes.

Thus, the luatest_helpers.lua module was dropped and tests using this
module were updated.

Closes tarantool/luatest#192

NO_DOC=testing stuff
NO_CHANGELOG=testing stuff

The luatest_helpers/asserts.lua module confuses developers because
luatest itself has the asserts.lua module as well. It is unclear which
one should be used and when because it is hard to remember what stuff
provides each of them.

Actually, luatest_helpers/asserts.lua has functions that are not really
about asserting and can be safely moved to more proper places like the
luatest_helpers/server.lua and luatest_helpers/cluster.lua modules.

The `assert_server_follow_upstream` function logically belongs to server
stuff, so it was moved to server.lua and slightly changed (renamed into
`assert_follows_upstream` and `eval` calls were replaced by `exec`).

The `wait_fullmesh` function logically belongs to cluster stuff, so it
was moved to cluster.lua and slightly changed (now it accepts one param
that can include wait timeout and delay, otherwise, defaults applied).

Thus, the luatest_helpers/asserts.lua module was dropped and tests using
this module were updated.

Part of tarantool/luatest#192

NO_DOC=testing stuff
NO_CHANGELOG=testing stuff

We have recently dropped 'checksum' machinery in test-run
(tarantool/test-run#321) and now there is no need to keep
checksums in suite.ini config files. So removing them.

Closes tarantool/tarantool-qa#152

The broadcast doesn't need to go into the network, but
it will be needed locally for a built-in event. So
called "box.election".

Part of #6260

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes