The test assumes that a version string looks like this
2.9.0-123-gabcabcababc. We want to append a flow string
after <major>.<minor>.<patch>. Fix the test accordingly.

Needed for #6183

This patch removes SQL built-in functions from _func. These functions
could be called directly from Lua, however all they did was returned an
error. After this patch, no SQL built-in functions can be called
directly from LUA.

Part of #6106

Updated:

  box/net.box_reconnect_after_gh-3164.test.lua gh-5081
  replication/errinj.test.lua                  gh-3870
  replication/qsync_basic.test.lua             gh-5355
  replication/anon.test.lua                    gh-5381
  replication/status.test.lua                  gh-5409
  replication/election_qsync.test.lua          gh-5430

Added new:

  box-py/iproto.test.py                             gh-qa-132
  replication/gh-5435-qsync-clear-synchro-queue-co> gh-qa-129
  replication/gh-5445-leader-inconsistency.test.lua gh-qa-129
  replication/gh-3055-election-promote.test.lua     gh-qa-127
  replication/election_basic.test.lua               gh-qa-133

This patch introduces a new SQL built-in function UUID().

Closes #5886

@TarantoolBot document
Title: SQL built-in function UUID()

SQL built-in function UUID() takes zero or one argument. If no argument
is specified, a UUID v4 is generated. If the version of the UUID to
generate is specified as an argument, the function returns the new UUID
of the given version. Currently only version 4 of UUID is supported.

Regex for validating version was expecting a single
character (digit) for version `patch`, but it's not correct.
This patch fixes test behaviour for tarantool 1.10.10

Close #6039

ER_TUPLE_FOUND message shows only space and index, let's also show old
tuple and new tuple.

This commit changes error message in code and in tests. Test sql/checks
and sql-tap/aler remain the same due to problems in showing their old
and new tuples in error message.

Closes #5567

Response object is a unpacked msgpack structure, returned by Tarantool.
Nor msgpack-python [1] that used in tarantool-python nor msgpack [2]
itself cannot guarantee an order of keys in unpacked dictionaries.
Therefore we have different keys order with running tests under Python 2
and Python 3, for example box-py/call.test.py. To workaround a problem
proposed a conversion of dictionaries in a tuple to lists before
printing Response object.

Requires changes in tarantool-python [3].

1. https://github.com/msgpack/msgpack-python/pull/164
2. https://github.com/msgpack/msgpack/issues/215
3. https://github.com/tarantool/tarantool-python/pull/186

Part of #5652
Follows up #5538

Test output with running on Python 2 and Python 3 is not the same.

[009] box-py/iproto.test.py                                           [ fail ]
[009]
[009] Test failed! Result content mismatch:
[009] --- box-py/iproto.result  Sat Mar 13 17:46:54 2021
[009] +++ /rw_bins/test/var/rejects/box-py/iproto.reject        Sat Mar 13 17:50:51 2021
[009] @@ -29,7 +29,7 @@
[009]  IPROTO_UPDATE
[009]  query {'IPROTO_CODE': 4} {'IPROTO_SPACE_ID': 280}
[009]  True
[009] -query {'IPROTO_CODE': 4} {'IPROTO_SPACE_ID': 280, 'IPROTO_KEY': (1,)}
[009] +query {'IPROTO_CODE': 4} {'IPROTO_KEY': (1,), 'IPROTO_SPACE_ID': 280}
[009]  True
[009]
[009]
[009]
[009] Last 15 lines of Tarantool Log file [Instance "box"][/rw_bins/test/var/009_box-py/box.log]:

1. https://github.com/tarantool/tarantool/pull/5751/checks?check_run_id=2103097418

Part of #5652
Follows up #5538

Part of #5652

Found more flaky tests running testings with real parallel even for
tests in fragile lists.

Update the test-run submodule with updating its tarantool-python
submodule with the Response.__str__() change, which unifies behaviour
between Python 2 and Python 3. We lean on this in the
box-py/call.test.py test. See more in [1].

Updated the box-py/iproto.test.py test, because it is confused by the
new IPROTO_SQL_INFO_* constants, introduced in the tarantool-python
connector in [2].

Part of https://github.com/tarantool/test-run/issues/20
Part of #5652

[1]: https://github.com/tarantool/test-run/pull/264
[2]: https://github.com/tarantool/tarantool-python/commit/9d3573e36199ee5ebc360f9017190ada9726bb23

Closes #5456

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Reviewed-by: Igor Munkin <imun@tarantool.org>

Co-authored-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Co-authored-by: Igor Munkin <imun@tarantool.org>

The largest change in Python 3 is the handling of strings.
In Python 2, the str type was used for two different
kinds of values - text and bytes, whereas in Python 3,
these are separate and incompatible types.
Patch converts strings to byte strings where it is required
to make tests compatible with Python 3.

Part of #5538

In Python 2.x calling items() makes a copy of the keys that you can
iterate over while modifying the dict. This doesn't work in Python 3.x
because items() returns an iterator instead of a list and Python 3 raise
an exception "dictionary changed size during iteration". To workaround
it one can use list to force a copy of the keys to be made.

Part of #5538

- convert print statement to function. In a Python 3 'print' becomes a
function, see [1]. Patch makes 'print' in a regression tests compatible with
Python 3.
- according to PEP8, mixing using double quotes and quotes in a project looks
inconsistent. Patch makes using quotes with strings consistent.
- use "format()" instead of "%" everywhere

1. https://docs.python.org/3/whatsnew/3.0.html#print-is-a-function

Part of #5538

After this patch, the persistent functions "box.schema.user.info" and
"LUA" will have the same rights as the user who executed them.

The problem was that setuid was unnecessarily set. Because of this,
these functions had the same rights as the user who created them.
However, they must have the same rights as the user who used them.

Fixes tarantool/security#1

In test-run implemented the new format of the fragile lists based on
JSON format set as fragile option in 'suite.ini' files per each suite:

   fragile = {
        "retries": 10,
        "tests": {
            "bitset.test.lua": {
                "issues": [ "gh-4095" ],
                "checksums": [ "050af3a99561a724013995668a4bc71c", "f34be60193cfe9221d3fe50df657e9d3" ]
            }
        }}

Added ability to check results file checksum on tests fail and
compare with the checksums of the known issues mentioned in the
fragile list.

Also added ability to set 'retries' option, which sets the number
of accepted reruns of the tests failed from 'fragile' list that
have checksums on its fails.

Closes #5050

Part of #4967

Fragiled flaky tests from parallel runs to avoid
of flaky fails in regular testing:

  box-py/snapshot.test.py                ; gh-4514
  replication/misc.test.lua              ; gh-4940
  replication/skip_conflict_row.test.lua ; gh-4958
  replication-py/init_storage.test.py    ; gh-4949
  vinyl/stat.test.lua                    ; gh-4951
  xlog/checkpoint_daemon.test.lua        ; gh-4952

Part of #4953

This patch introduces support of stacked errors in IProto protocol and
in net.box module.

Closes #1148

@TarantoolBot document
Title: Stacked error diagnostic area

Starting from now errors can be organized into lists. To achieve this
Lua table representing error object is extended with .prev field and
e:set_prev(err) method. .prev field returns previous error if any exist.
e:set_prev(err) method expects err to be error object or nil and sets
err as previous error of e. For instance:
```
e1 = box.error.new({code = 111, reason = "cause"})
e2 = box.error.new({code = 111, reason = "cause of cause"})

e1:set_prev(e2)
assert(e1.prev == e2) -- true
```
Cycles are not allowed for error lists:
```
e2:set_prev(e1)
- error: 'builtin/error.lua: Cycles are not allowed'
```
Nil is valid input to :set_prev() method:
```
e1:set_prev(nil)
assert(e1.prev == nil) -- true
```
Note that error can be 'previous' only to the one error at once:
```
e1:set_prev(e2)
e3:set_prev(e2)
assert(e1.prev == nil) -- true
assert(e3.prev == e2) -- true
```
Setting previous error does not erase its own previous members:
```
-- e1 -> e2 -> e3 -> e4
e1:set_prev(e2)
e2:set_prev(e3)
e3:set_prev(e4)
e2:set_prev(e5)
-- Now there are two lists: e1->e2->e5 and e3->e4
assert(e1.prev == e2) -- true
assert(e2.prev == e5) -- true
assert(e3.prev == e4) -- true
```
Alternatively:
```
e1:set_prev(e2)
e2:set_prev(e3)
e3:set_prev(e4)
e5:set_prev(e3)
-- Now there are two lists: e1->e2 and e5->e3->e4
assert(e1.prev == e2) -- true
assert(e2.prev == nil) -- true
assert(e5.prev == e3) -- true
assert(e3.prev == e4) -- true
``
Stacked diagnostics is also supported by IProto protocol. Now responses
containing errors always (even if there's only one error to be returned)
include new IProto key: IPROTO_ERROR_STACK (0x51). So, body corresponding to
error response now looks like:
```
MAP{IPROTO_ERROR : string, IPROTO_ERROR_STACK : ARRAY[MAP{ERROR_CODE : uint, ERROR_MESSAGE : string}, MAP{...}, ...]}
```
where IPROTO_ERROR is 0x31 key, IPROTO_ERROR_STACK is 0x52, ERROR_CODE
is 0x01 and ERROR_MESSAGE is 0x02.
Instances of older versions (without support of stacked errors in
protocol) simply ignore unknown keys and still rely only on IPROTO_ERROR
key.

This patch creates _session_settings system space. This space is
used to view and change session settings. This space is one of the
special spaces that have a "service" engine. The main idea of this
space is that it will not store tuples, but when you call the
get() or select() methods, it creates tuples on the fly. Because
of this, even if the same setting is asked, the returned tuples
will be different. In addition, this space allows you to change
the setting value using the update() method, in which case it
directly changes the setting value.

There are no settings at the moment, some will be added in the
next patch.

Part of #4511

Unicode_ci collation breaks the general
rule for objects naming, so we remove it
in version 2.3.1

Closes #4561

Rows_per_wal option was deprecated because it can be covered by
wal_max_size. In order not to complicate WAL code with that
option's support this commit drops it completely.

In some tests the option was used to create several small xlog
files. Now the same is done via wal_max_size. Where it was
needed, number of rows per wal is estimated as wal_max_size / 50.
Because struct xrow_header size ~= 50 not counting paddings and
body.

Note, file box/configuration.result was deleted here, because it
is a stray result file, and it contained the rows_per_wal option
mentioning. Its test was dropped much earlier in
fdc3d1dd.

Closes #3762

Now it is possible to disable and enable ck constraints in LUA.
This option is persistent. All ck constraints are constructed
in enabled state when Tarantool is configured. This ability
may be usefulwhen processed data is verified and constraints
validation is not required. For instance, during casual recovery
process there's no need to provide any checks since data is
assumed to be consistent.

Persisting is_enabled flag is an important feature.
If the option is not stored the following scenario is
possible:
- the option is turned off
- data is changed so that a constraint is violated
- the system is restarted while the option state is lost
- there is no way (even in theory) to discover it and find that
  data is incorrect.

Part of #4244

This patch does two things: renames existing scalar min/max
functions and reserves names for them in NoSQL cache.

Moreover it is an important step to get rid of function's name
overloading required for replace FuncDef cache with Tarantool's
function cache.

Closes #4405
Needed for #2200, #4113, #2233

@TarantoolBot document
Title: Scalar functions MIN/MAX are renamed to LEAST/GREATEST

The MIN/MAX functions are typically used only as aggregate
functions in other RDBMS(MSSQL, Postgress, MySQL, Oracle) while
Tarantool's SQLite legacy code use them also in meaning
GREATEST/LEAST scalar function. Now it fixed.

Closes #1260

@TarantoolBot document
Title: introduce func indexes in memtx
Now you can define a func_index using a registered persistent
function.

There are restrictions for function and key definition for
a functional index:
 - the referenced function must be persistent, deterministic
   and must return a scalar type or an array.
 - you must define key parts which describe the function return value
 - the function must return data which types match the
   defined key parts
 - the function may return multiple keys; this would be a multikey
   functional index; each key entry is indexed separately;
 - for multikey functional indexes, the key definition should
   start with part 1 and cover all returned key parts
 - key parts can't use JSON paths.
 - the function used for the functional index can not access tuple
   fields by name, only by index.

Functional index can't be primary.
It is not possible to change the used function after a functional
index is defined on it. The index must be dropped first.

Each key returned by functional index function (even when it is a
single scalar) must be returned as a table i.e. {1} and must
match the key definition.

To define a multikey functional index, create a function with
opts = {is_multikey = true} and return a table of keys.

Example:
s = box.schema.space.create('withdata')
s:format({{name = 'name', type = 'string'},
          {name = 'address', type = 'string'}})
pk = s:create_index('name', {parts = {1, 'string'}})
lua_code = [[function(tuple)
                local address = string.split(tuple[2])
                local ret = {}
                for _, v in pairs(address) do
			table.insert(ret, {utf8.upper(v)})
		end
                return ret
             end]]
box.schema.func.create('address', {body = lua_code,
                       is_deterministic = true, is_sandboxed = true,
                       opts = {is_multikey = true}})
idx = s:create_index('addr', {unique = false,
                     func = 'address',
                     parts = {{1, 'string', collation = 'unicode_ci'}}})
s:insert({"James", "SIS Building Lambeth London UK"})
s:insert({"Sherlock", "221B Baker St Marylebone London NW1 6XE UK"})
idx:select('Uk')
---
- - ['James', 'SIS Building Lambeth London UK']
  - ['Sherlock', '221B Baker St Marylebone London NW1 6XE UK']
...

Closes #4182
Closes #4219
Needed for #1260

@TarantoolBot document
Title: Persistent Lua functions

Now Tarantool supports 'persistent' Lua functions.
Such functions are stored in snapshot and are available after
restart.
To create a persistent Lua function, specify a function body
in box.schema.func.create call:
e.g. body = "function(a, b) return a + b end"

A Lua persistent function may be 'sandboxed'. The 'sandboxed'
function is executed in isolated environment:
  a. only limited set of Lua functions and modules are available:
    -assert -error -pairs -ipairs -next -pcall -xpcall -type
    -print -select -string -tonumber -tostring -unpack -math -utf8;
  b. global variables are forbidden

Finally, the new 'is_deterministic' flag allows to mark a
registered function as deterministic, i.e. the function that
can produce only one result for a given list of parameters.

The new box.schema.func.create interface is:
box.schema.func.create('funcname', <setuid = true|FALSE>,
	<if_not_exists = true|FALSE>, <language = LUA|c>,
	<body = string ('')>, <is_deterministic = true|FALSE>,
	<is_sandboxed = true|FALSE>, <comment = string ('')>)

This schema change is also reserves names for sql builtin
functions:
    TRIM, TYPEOF, PRINTF, UNICODE, CHAR, HEX, VERSION,
    QUOTE, REPLACE, SUBSTR, GROUP_CONCAT, JULIANDAY, DATE,
    TIME, DATETIME, STRFTIME, CURRENT_TIME, CURRENT_TIMESTAMP,
    CURRENT_DATE, LENGTH, POSITION, ROUND, UPPER, LOWER,
    IFNULL, RANDOM, CEIL, CEILING, CHARACTER_LENGTH,
    CHAR_LENGTH, FLOOR, MOD, OCTET_LENGTH, ROW_COUNT, COUNT,
    LIKE, ABS, EXP, LN, POWER, SQRT, SUM, TOTAL, AVG,
    RANDOMBLOB, NULLIF, ZEROBLOB, MIN, MAX, COALESCE, EVERY,
    EXISTS, EXTRACT, SOME, GREATER, LESSER, SOUNDEX,
    LIKELIHOOD, LIKELY, UNLIKELY,
    _sql_stat_get, _sql_stat_push, _sql_stat_init, LUA

A new Lua persistent function LUA is introduced to evaluate
LUA strings from SQL in future.

This names could not be used for user-defined functions.

Example:
lua_code = [[function(a, b) return a + b end]]
box.schema.func.create('summarize', {body = lua_code,
		is_deterministic = true, is_sandboxed = true})
box.func.summarize
---
- aggregate: none
  returns: any
  exports:
    lua: true
    sql: false
  id: 60
  is_sandboxed: true
  setuid: false
  is_deterministic: true
  body: function(a, b) return a + b end
  name: summarize
  language: LUA
...
box.func.summarize:call({1, 3})
---
- 4
...

@kostja: fix style, remove unnecessary module dependencies,
add comments

This reverts commit 4e3470ce.

The RFC did not pass review yet.

Closes #4182
Needed for #1260

@TarantoolBot document
Title: Persistent Lua functions

Now Tarantool supports 'persistent' Lua functions.
Such functions are stored in snapshot and are available after
restart.
To create a persistent Lua function, specify a function body
in box.schema.func.create call:
e.g. body = "function(a, b) return a + b end"

A Lua persistent function may be 'sandboxed'. The 'sandboxed'
function is executed in isolated environment:
  a. only limited set of Lua functions and modules are available:
    -assert -error -pairs -ipairs -next -pcall -xpcall -type
    -print -select -string -tonumber -tostring -unpack -math -utf8;
  b. global variables are forbidden

Finally, the new 'is_deterministic' flag allows to mark a
registered function as deterministic, i.e. the function that
can produce only one result for a given list of parameters.

The new box.schema.func.create interface is:
box.schema.func.create('funcname', <setuid = true|FALSE>,
	<if_not_exists = true|FALSE>, <language = LUA|c>,
	<body = string ('')>, <is_deterministic = true|FALSE>,
	<is_sandboxed = true|FALSE>, <comment = string ('')>)

Example:
lua_code = [[function(a, b) return a + b end]]
box.schema.func.create('sum', {body = lua_code,
		is_deterministic = true, is_sandboxed = true})
box.func.sum
---
- is_sandboxed: true
  is_deterministic: true
  id: 2
  setuid: false
  body: function(a, b) return a + b end
  name: sum
  language: LUA
...
box.func.sum:call({1, 3})
---
- 4
...

Add "_vcollation" sysview to read it from net.box. This
sysview is always readable, except when a user doesn't have
"public" role.

Needed for #3941

@TarantoolBot document
Title: box.space._vcollation

_vcollation is a system space that represents a virtual view.
The structure of its tuples is identical to that of _collation.
Tuples of this sysview is always readable, except when the user
doesn't have "public" role.

This patch introduces a new system space to persist check
constraints. The format of the new system space is

_ck_constraint (space id = 364)
[<space id> UINT, <constraint name> STR,
 <is_deferred>BOOL, <language>STR, <code>STR]

A CK constraint is local for a space, so every pair
<space id, CK name> is unique
(it is also the PK in the _ck_constraint space).

After insertion into this space, a new instance describing check
constraint is created. Check constraint holds an exspression AST.
While space features some check constraints, it isn't allowed to
be dropped. The :drop() space method firstly deletes all check
constraints and then removes an entry from the _space.

Because the space alter, the index alter and the space truncate
operations cause space recreation process, a new
RebuildCkConstrains object is introduced. This alter object
compiles a new ck constraint object, replaces and removes
an existent instances atomically (but if the assembly of some
ck constraint object fails, nothing is changed).
In fact, in scope of this patch we don't really need to recreate
a ck_constraint object in such situations (it is enough to patch
space_def pointer in AST tree like we did it before, but we are
going to recompile a VDBE that represents ck constraint in
further patches, and that operation is not safe).

The main motivation for these changes is an ability to support
ADD CHECK CONSTRAINT operation in the future. CK constraints are
easier to manage as self-sustained objects: such change is
managed with atomic insertion(unlike the current architecture).

Finally, the xfer optimization is disabled now if some space have
ck constraints. In following patches this xfer optimisation
becomes impossible, so there is no reason to rewrite this code
now.

Needed for #3691

Rather than passing 'sequence_part' along with 'sequence' on index
create/alter, pass a table with the following fields:

 - id: sequence id or name
 - field: auto increment field id or name or path in case of json index

If id is omitted, the sequence will be auto-generated (equivalent to
'sequence = true'). If field is omitted, the first indexed field is
used. Old format, i.e. passing false/true or sequence name/id instead of
a table is still supported.

Follow-up #4009

Closes #4009

@TarantoolBot document
Title: Sequence can now be set for an index part other than the first

Initially one could attach a sequence (aka autoincrement) only to the
first index part. Now it's possible to attach a sequence to any primary
index part. The part still must be integer though.

Syntax:

```
box.schema.space.create('test')
box.space.test:create_index('primary', {
    parts = {{1, 'string'}, {2, 'unsigned'}, {3, 'unsigned'}},
    sequence = true, sequence_part = 2
})
box.space.test:insert{'a', box.null, 1} -- inserts {'a', 1, 1}
```

Note, `sequence_part` option is 1-base.

If `sequence_part` is omitted, 1 is used, which assures backward
compatibility with the original behavior.

One can also attach a sequence to another index part using
`index.alter` (the code below continues the example above):

```
box.space.test.index.primary:alter{sequence_part = 3}
box.space.test:insert{'a', 1, box.null, 'x'} -- inserts {'a', 1, 2, 'x'}
```

The primary reason why this change is needed is that yaml.load() w/o an
explicit loader was banned in Gentoo Linux for recent pyyaml versions;
see [1].

We don't use the pyyaml feature that allows to construct a Python object
based on a yaml tag, so safe_load() fit our needs.

See also related changes in test-run and tarantool-python ([2], [3], [4]).

[1]: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=79ba924d94cb0cf8559565178414c2a1d687b90c
[2]: https://github.com/tarantool/test-run/commit/38400e91c600677fb661154d00459d660fa9880d
[3]: https://github.com/tarantool/test-run/commit/89808d60eb3b5130e227fc1a7866f2ad5a197bea
[4]: https://github.com/tarantool/tarantool-python/commit/350771d240a18eec188a53e8c696028b41baa13f

These tables won't be used anymore and should be deleted.

Note, this patch breaks backward compatibility between 2.1.1 and
2.1.2, but that's okay as 2.1.1 was beta and we didn't recommend
anyone to use it.

Part of #2843
Follow up #4069

Adds a new set of default collations.
The collation 'unicode_ky_s1' supports the difference
between Cyrillic letters 'Е' and 'Ё'. The standard case insensitive
collation ('unicode_ci') doesn't distinguish these letters.
Adds tests for a new collations.

Closes #4007

Before the patch, collations with no strength set used
tertiary strength. But it was not easy to understand it,
because box.space._collation:select{} would return
... [1, 'unicode', 1, 'ICU', '', {}] ... for such collations.
After the patch default value is set explicitly, so
user would observe : ... [1, 'unicode', 1, 'ICU', '',
{strength='tertiary'}] ...

Closes #3573

@TarantoolBot document
Title: default collation strength is explicit tertiary now
Before the patch we already have tertiary strength is default
strength for collations, but it was implicit:
[1, 'unicode', 1, 'ICU', '', {}]
After the patch it's just become explicit:
1, 'unicode', 1, 'ICU', '', {'strength' = 'tertiary'}]

Also please fix this https://tarantool.io/en/doc/2.1/book/box/data_model/#collations
There is line saying: "unicode collation observes all weights,
from L1 to Ln (identical)" It was not true and now this fact
would just become obvious.

Tarantool could not start from the snapshot created by version
2.1.0 because the new version 2.1.1 does not support the
index.opts.sql index opt and stops the execution.
Introduced a special state OPT_DEF_LEGACY macro to ignore legacy
options and introduced migration code in upgrade.lua.

Fixed issues:

- box-py/iproto.test.py
  1) Fixed receive_response() to wait for whole response.
  2) Clean up _cluster space.

- replication-py/cluster.test.py
  1) Clean up _cluster space.

- replication-py/multi.test.py
  1) Removed vclock checking because it fails if previous test make some
     DML and vclock was incremented. Looks like it was used for debug
     and is not part of this test case.
  2) Fixed typo in 'Synchronize' block.

The following test sequences did fail due to unexpected IDs in _cluster
space:

- [box-py/iproto.test.py, null]
- [box-py/bootstrap.test.py, null]

- [replication-py/cluster.test.py, null]
- [replication-py/multi.test.py, null]

Part of #3232

They aren't needed there as we reset them anyway once the snapshot is
replayed on initial bootstrap. OTOH having them results in calling
replica_{set,clear}_id twice on initial bootstrap, which will look weird
when I patch them to log the ids. So let's remove them from the initial
snapshot. This makes the initial bootstrap impossible to recover from as
it is, but that shouldn't be an issue since one can always bootstrap a
new instance in a normal way. This also allows us to make cluster uuid
truly immutable (currently, one can update it with REPLACE).