The only error type having an errno as a part of it was
SystemError (and its descendants SocketError, TimedOut, OOM, ...).
That was used in logs (SystemError::log() method), and exposed to
Lua (if type was SystemError, an error object had 'errno' field).

But actually errno might be useful not only there. For example,
box.info.replication exposes the latest error message of
applier/relay as 'message' field of 'upstream/downstream' fields,
lacking errno description.

Before the patch it was impossible to obtain an errno code from C,
because it was necessary to check whether an error has SystemError
type, cast to SystemError class, and call SystemError::get_errno()
method.

Now errno is available as a part of struct error object (available
from C), and is not 0 for system errors.

Part of #4402

Box.session.su() raised 'SystemError' when a user was not found
due to a too long user name. That was obviously wrong, because
SystemError is always something related to libraries (standard,
curl, etc), and it has an errno code.

Now a ClientError is raised.

Functions are stored in lists inside module objects. Module
objects are stored in a hash table, where key is a package name.
But the key was a pointer at one of module's function definition
object. Therefore, when that function was deleted, its freed
package name memory was still in the hash key, and could be
accessed, when another function was deleted.

Now module does not use memory of its functions, and keep a copy
of the package name.

fiber.top() fills in statistics every event loop iteration,
so if it was just enabled, fiber.top() returns zero in fiber cpu
usage statistics because total time consumed by the main thread was
not yet accounted for.
Same stands for viewing top() results for a freshly created fiber:
its metrics will be zero since it hasn't lived a full ev loop iteration
yet.
Fix this by delaying the test till top() results are meaningful and add
minor refactoring.

Follow-up #2694

When fiber EMA is 0 and first non-zero observation is added to it, we assumed
that EMA should be equal to this observation (i.e. average value should
be the same as the observed one). This breaks the following invariant:
sum of clock EMAs of all fibers equals clock EMA of the thread.
If one of the fibers is just spawned and has a big clock delta, it
will assign this delta to its EMA, while the thread will calculate the
new EMA as 15 * EMA / 16 + delta / 16, which may lead to a situation
when fiber EMA is greater than cord EMA.

This caused occasional test failures:
```
[001] Test failed! Result content mismatch:
[001] --- app/fiber.result	Mon Nov 18 17:00:48 2019
[001] +++ app/fiber.reject	Mon Nov 18 17:33:10 2019
[001] @@ -1511,7 +1511,7 @@
[001]  -- not exact due to accumulated integer division errors
[001]  sum_avg > 99 and sum_avg < 101 or sum_avg
[001]  ---
[001] -- true
[001] +- 187.59585601717
[001]  ...
[001]  tbl = nil
[001]  ---

```

Follow-up #2694

Unify all the members related to fiber's clock statistics into struct
clock_stat and all the members related to cord's knowledge of cpu state
and clocks to struct cpu_stat.
Reset stats of all alive fibers on fiber.top_enable().

Follow-up #2694

The problem appears after 6c627af3
('test: tarantoolctl: verify delayed box.cfg()'), where the test case
was changed and it doesn't more assume an error at the instance start.
So we need to stop it to prevent a situation when instances are stay
after `make test`.

Fixes #4600.
Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>

Before commit 03f85d4c ('app: fix
boolean handling in argparse module') the module does not expect a value
after a 'boolean' argument. However there was the problem: a 'boolean'
argument can be passed only at end of an argument list, otherwise it
wrongly consumes a next argument and gives a confusing error message.

The mentioned commit fixes this behaviour in the following way: it still
allows to pass a 'boolean' argument at end of the list w/o a value, but
requires a value ('true', 'false', '1', '0') if a 'boolean' argument is
not at the end to be provided using {'--foo=true'} or {'--foo', 'true'}
syntax.

Here this behaviour is changed: a 'boolean' argument does not assume an
explicitly passed value despite its position in an argument list. If a
'boolean' argument appears in the list, then argparse.parse() returns
`true` for its value (a list of `true` values in case of 'boolean+'
argument), otherwise it will not be added to the result.

This change also makes the behaviour of long (--foo) and short (-f)
'boolean' options consistent.

The motivation of the change is simple: it is easier and more natural to
type, say, `tarantoolctl cat --show-system 00000000000000000000.snap`
then `tarantoolctl cat --show-system true 00000000000000000000.snap`.

This commit adds several new test cases, but it does not mean that we
guarantee that the module behaviour will not be changed around some
corner cases, say, handling of 'boolean+' arguments. This is internal
module.

Follows up #4076.
Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>

ck_constraint_def_new_from_tuple is used in
on_replace_dd_ck_constraint therefore it has to be cleared from
exceptions. Now it doesn't throw any more. It's usages are updated.
Some _xc functions, not needed any more, are removed.

Part of #4247
Conflicts:
	src/box/alter.cc

fk_constraint_check_dup_links is used in
on_replace_dd_fk_constraint therefore it has to be cleared from
exceptions. Now it doesn't throw any more. It's usages are updated.

Part of #4247

During replacement of tuple_field_bool_xc with it's non-xc version
turned out that it might be called even if there is not enough fields
in processed tuple. Now it is fixed.

Part of #4247

fk_constraint_def_new_from_tuple is used in
on_replace_dd_fk_constraint therefore it has to be cleared from
exceptions. Now it doesn't throw any more. It means we also need
to clear from exceptions it's subsidiary function: decode_fk_links.
Their usages are updated. Some _xc functions, not needed any more,
are removed.

Part of #4247

coll_id_def_new_from_tuple is used in on_replace_dd_collation
therefore it has to be cleared from exceptions. Now it doesn't
throw any more. It's usages are updated.

Part of #4247

user_def_new_from_tuple is used in on_replace_dd_user &
user_cache_alter_user therefore it has to be cleared from
exceptions. Now it doesn't throw any more. It means we also need
to clear from exceptions it's subsidiary function:
user_def_fill_auth_data. Their usages are updated.

Part of #4247

alter_space_new doesn't throw anymore. It's usages are updated.

Part of #4247

user_has_data is used in on_replace_dd_user therefore it has to be
cleared from exceptions. Now it doesn't throw any more. It means
we also need to clear from exceptions it's subsidiary function:
space_has_data. Their usages are updated.

Part of #4247

space_def_new_from_tuple is used in on_replace_dd_space therefore
it has to be cleared from exceptions. Now it doesn't throw any
more. It means we also need to clear from exceptions it's
subsidiary functions: space_opts_decode, field_def_decode &
space_format_decode. Their usages are updated.

Part of #4247

   284  int
   285  json_path_cmp(const char *a, int a_len, const char *b, int b_len,
   286                int index_base)
   287  {

  ...

   304          /* Paths a and b must be valid. */
   305          assert(rc_b == 0 && rc_b == 0);

Obviously (according to the comment) author implied that both rc_a == 0
and rc_b == 0. Let's fix this small typo.

Before the patch a struct xrow_update_field object didn't account
array header in its .size and .data members. Indeed, it was not
needed, because anyway updates could be only 'flat'.
For example, consider the tuple:

    [mp_array, mp_uint, mp_uint, mp_uint]
              ^                         ^
             pos1                      pos2

Struct xrow_update_field.size and .data accounted memory from
pos1 to pos2, without the array header. Number of fields was
stored inside a rope object. This is why it made no sense to keep
array header pointer.

But now updates are going to be not flat, and not only for array.
There will be an update tree. Each node of that tree will describe
update of some part of a tuple.

Some of the nodes will need to know exact borders of their
children, including headers. It is going to be used for fast
copying of neighbours of such children. Consider an example.

Tuple with one field consisting of nested maps:

    tuple = {}
    tuple[1] = {
        a = {
            b = {
                c = {
                    d = {1, 2, 3}
                }
            }
        }
    }

Update:

    {{'+', '[1].a.b.c.d[1]', 1}, {'+', '[1].a.b.c.d[2]', 1}}

To update such a tuple a simple tree will be built:

            root: [ [1] ]
                     |
 isolated path: [ 'a.b.c' ]
                     |
      leaves: [ [1] [2] [3] ]
                +1  +1   -

Root node keeps the whole tuple borders. It is a rope with single
field.
This single field is a deeply updated map. Such deep multiple
updates with long common prefixes are stored as an isolated path
+ map/array in the end. Here the isolated path is 'a.b.c'. It
ends with the terminal array update.

Assume, that operations are applied and it is time to save the
result. Save starts from the root.
Root rope will encode root array header, and will try to save the
single field. The single field is an isolated update. It needs to
save everything before old {1,2,3}, the new array {2,2,3}, and
everything after the old array. The simplest way to do it - know
exact borders of the old array {1,2,3} and memcpy all memory
before and after.

This is exactly what this patch allows to do. Everything before
xrow_update_field.data, and after xrow_update_field.data + .size
can be safely copied, and is not related to the field. To copy
adjacent memory it is not even needed to know field type.
Xrow_update_field.data and .size have the same meaning for all
field types.

Part of #1261

They are needed in incoming JSON updates, which are going to
solve a task of comparison of two JSON paths, their simultaneous
parsing, and digging into a tuple.

json_token_cmp() existed before this patch, but it was trying to
compare parent pointers too, which is not needed in the JSON
updates, since they won't use JSON trees.

Needed for #1261

There were some _xc functions used in triggers. Now they all are
replaced with their non _xc versions. If corresponding _xc version
hadn't had any other usages, it was removed.

Part of #4247

sequence_field_from_tuple is used in set_space_sequence &
on_replace_dd_space_sequence therefore it has to be cleared from
exceptions. Now it doesn't throw any more. It's usages are updated.

Part of #4247

sequence_def_new_from_tuple is used in on_replace_dd_sequence
therefore it has to be cleared from exceptions. Now it doesn't
throw any more. It's usages are updated.

Part of #4247

Bootstrap and recovery work on behalf of admin. Without the
universe access they are not able to even fill system spaces with
data.

It is better to forbid this ability until someone made their
cluster unrecoverable.

The admin user has universal privileges before bootstrap or
recovery are done. That allows to, for example, bootstrap from a
remote master, because to do that the admin should be able to
insert into system spaces, such as _priv.

But after the patch on online credentials update was implemented
(#2763, 48d00b0e) the admin could
loose its universal access if, for example, a role was granted to
him before universal access was recovered.

That happened by two reasons:

    - Any change in access rights, even in granted roles, led to
      rebuild of universal access;

    - Any change in access rights updated the universal access in
      all existing sessions, thanks to #2763.

What happened: two tarantools were started. One of them master,
granted 'replication' role to admin. Second node, slave, tried to
bootstrap from the master. The slave created an admin session and
started loading data. After it loaded 'grant replication role to
admin' command, this nullified admin universal access everywhere,
including this session. Next rows could not be applied.

Closes #4606

After the issue #4537 fixed for the data segment size limit,
the temporary blocked tests because of it unblocked.

Part of #4271

func_def_new_from_tuple is used in on_replace_dd_func therefore
it has to be cleared from exceptions. Now it doesn't throw any
more. It means we also need to clear from exceptions it's
subsidiary function func_def_get_ids_from_tuple.
Their usages are updated.

Part of #4247

index_def_new_from_tuple is used in on_replace_dd_index therefore
it has to be cleared from exceptions. Now it doesn't throw any
more. It means we also need to clear from exceptions it's
subsidiary functions: index_def_check_sequence,
index_def_check_tuple, index_opts_decode, func_index_check_func.
Their usages are updated.

Part of #4247

Implement a new function in Lua fiber library: top(). It returns a table
containing fiber cpu usage stats. The table has two entries:
"cpu_misses" and "cpu". "cpu" itself is a table listing all the alive
fibers and their cpu consumtion.
The patch relies on CPU timestamp counter to measure each fiber's time
share.

Closes #2694

@TarantoolBot document
Title: fiber: new function `fiber.top()`

`fiber.top()` returns a table of all alive fibers and lists their cpu
consumption. Let's take a look at the example:
```
tarantool> fiber.top()
---
- cpu:
    107/lua:
      instant: 30.967324490456
      time: 0.351821993
      average: 25.582738345233
    104/lua:
      instant: 9.6473633128437
      time: 0.110869897
      average: 7.9693406131877
    101/on_shutdown:
      instant: 0
      time: 0
      average: 0
    103/lua:
      instant: 9.8026528631511
      time: 0.112641118
      average: 18.138387232255
    106/lua:
      instant: 20.071174377224
      time: 0.226901357
      average: 17.077908441831
    102/interactive:
      instant: 0
      time: 9.6858e-05
      average: 0
    105/lua:
      instant: 9.2461986412164
      time: 0.10657528
      average: 7.7068458630827
    1/sched:
      instant: 20.265286315108
      time: 0.237095335
      average: 23.141537169257
  cpu_misses: 0
...

```
The two entries in a table returned by `fiber.top()` are
`cpu_misses` and `cpu`.

`cpu` itself is a table whose keys are strings containing fiber ids and
names.
The three metrics available for each fiber are:
1) instant (per cent),
which indicates the share of time fiber was executing during the
previous event loop iteration
2) average (per cent), which is calculated as an exponential moving
average of `instant` values over all previous event loop iterations.
3) time (seconds), which estimates how much cpu time each fiber spent
processing during its lifetime.

More info on `cpu_misses` field returned by `fiber.top()`:
`cpu_misses` indicates the amount of times tx thread detected it was
rescheduled on a different cpu core during the last event loop
iteration.
fiber.top() uses cpu timestamp counter to measure each fiber's execution
time. However, each cpu core may have its own counter value (you can
only rely on counter deltas if both measurements were taken on the same
core, otherwise the delta may even get negative).
When tx thread is rescheduled to a different cpu core, tarantool just
assumes cpu delta was zero for the latest measurement. This loweres
precision of our computations, so the bigger `cpu misses` value the
lower the precision of fiber.top() results.

Fiber.top() doesn't work on arm architecture at the moment.

Please note, that enabling fiber.top() slows down fiber switching by
about 15 per cent, so it is disabled by default.
To enable it you need to issue `fiber.top_enable()`.
You can disable it back after you finished debugging  using
`fiber.top_disable()`.
"Time" entry is also added to each fibers output in fiber.info()
(it duplicates "time" entry from fiber.top().cpu per fiber).
Note, that "time" is only counted while fiber.top is enabled.

Before the patch update was implemented as a set of operations
applicable for arrays only. It was ok until field names and JSON
paths appearance, because tuple is an array on the top level.

But now there are four reasons to allow more complex updates of
tuple field internals by JSON paths:

  - tuple field access by JSON path is allowed so for consistency
    JSON paths should be allowed in updates as well;

  - JSON indexes are supported. JSON update should be able to
    change an indexed field without rewriting half of a tuple, and
    its full replacement;

  - Tarantool is going to support documents in storage so JSON
    path updates is one more step forward;

  - JSON updates are going to be faster and more compact in WAL
    than get + in-memory Lua/connector update + replace (or update
    of a whole tuple field).

The patch reworks the current update code in such a way, that now
update is not just an array of operations, applied to tuple's top
level fields. Now it is a tree, just like tuples are.

The concept is to build a tree of xrow_update_field objects. Each
updates a part of a tuple. Leafs in the tree contain update
operations, specified by a user, as xrow_update_op objects.

To make the code support and understanding simpler, the patch
splits update implementation into several independent
files-modules for each type of an updated field. One file
describes how to update an array field, another file - how to
update a map field, etc. This commit introduces only array. Just
because it was already supported before the patch. Next commits
will introduce more types one by one.

Besides, the patch makes some minor changes, not separable from
this commit:

  - The big comment about xrow updates in xrow_update.c is
    updated. Now it describes the tree-idea presented above;

  - Comments were properly aligned by 66 symbols in all the moved
    or changed code. Not affected code is kept as is so as not to
    increase the diff even more;

  - Added missing comments to moved or changed structures and
    their attributes such as struct xrow_update,
    struct xrow_update_op_meta, struct xrow_update_op.

  - Struct xrow_update_field was significantly reworked. Now it is
    not just a couple of pointers at tuple's top level array. From
    now it stores type of the updated field, range of its source
    data in the original tuple, and a subtree of other update
    fields applied to the original data.

  - Added missing comments to some functions which I moved and
    decided worth commenting alongside, such as
    xrow_update_op_adjust_field_no(), xrow_update_alloc().

  - Functions xrow_update_op_do_f, xrow_update_op_read_arg_f,
    xrow_update_op_store_f are separated from struct xrow_update,
    so as they could be called on any updated field in the tree.
    From this moment they are methods of struct xrow_update_op.
    They take an op as a first argument (like 'this' in C++), and
    are applied to a given struct xrow_update_field.

Another notable, but not separable, change is a new naming schema
for the methods of struct xrow_update_field and struct
xrow_update_op. This is motivated by the fact that struct
xrow_update_field now has a type, and might be not a terminal.

There are now two groups of functions. Generic functions working
with struct xrow_update_field of any type:

    xrow_update_field_sizeof
    xrow_update_field_store
    xrow_update_op_do_field_<operation>

And typed functions:

    xrow_update_<type>_sizeof
    xrow_update_<type>_store
    xrow_update_op_do_<type>_<operation>

Where
    operation = insert/delete/set/arith ...
         type = array/map/bar/scalar ...

Common functions are used when type of a field to update is not
known in advance. For example, when an operation is applied to one
of fields of an array, it is not known what a type this field has:
another array, scalar, not changed field, map, etc. Common
functions do nothing more than just a switch by field type to
choose a more specific function.

Typed functions work with a specific type. They may change the
given field (add a new array element, replace it with a new value,
...), or may forward an operation deeper in case they see that its
JSON path is not fully passed yet.

Part of #1261

That patch finishes transformation of tuple_update public API to
xrow_update.

Part of #1261

Tuple_update is a too general name for the updates implemented
in these files. Indeed, a tuple can be updated from Lua, from
SQL, from update microlanguage. Xrow_update is a more specific
name, which is already widely used in tuple_update.c.

Part of #1261

When invalid command is passed we should send an error message to a
client. Instead a nil dereference occurs that causes abnormal exit of a
console.

This is the regression from 96dbc49d
('box/console: Refactor command handling').

Reported-by: Mergen Imeev <imeevma@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>

Added build + test jobs in GitLab-CI and build + test + deploy jobs on
Travis-CI for CentOS 8.

Updated testing dependencies in the RPM spec to follow the new Python 2
package naming scheme that was introduced in CentOS 8: it uses
'python2-' prefix rather then 'python-'.

CentOS 8 does not provide python2-gevent and python2-greenlet packages,
so they were pushed to https://packagecloud.io/packpack/backports
repository. This repository is enabled in our build image
(packpack/packpack:el-8) by default. Those dependencies are build-time,
so nothing was changed for a user. The source RPM packages were gathered
from https://cbs.centos.org

.

Disabled app-tap/pwd.test.lua on CentOS 8 due to systemd-nss issue,
which was not worked around properly. Filed #4592 to resolved it in the
future.

Eliminated libunwind runtime dependency (and libunwind-devel build
dependency) on CentOS 8, because the base system does not provide it.
fiber.info() backtraces and printing of a backtrace after a crash will
not be available on this system. Hopefully we'll fix it in the future,
filed #4611 on this.

Closes #4543

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
Reviewed-by: Igor Munkin <imun@tarantool.org>

After ea5929db ('build: fix OpenSSL
linking problems on FreeBSD') we set CFLAGS explicitly (possibly to an
empty value) when invoking a configure script for curl. When this
parameter is set the script does not use a value of environment variable
CFLAGS.

Before this commit LDFLAGS environment variable can affect build of curl
submodule. This can lead to a problem when a user or a tool set CFLAGS
and LDFLAGS both and some linker flag assumes that some compilation flag
is present. Here we set empty LDFLAGS explicitly to avoid using of the
environment variable.

A distributive build tool such as rpmbuild or emerge usually sets CFLAGS
and LDFLAGS. The problem with incompatible compiler / linker options has
been reveal under rpmbuild on CentOS 8 with hardened build enabled
(which is so when backtraces are disabled).

It is not clear whether we should follow environment variables or values
determined by CMake for CFLAGS, CPPFLAGS and LDFLAGS when building a
submodule (such as luajit and curl). Let's decide about this later.

Part of #4543.

Reviewed-by: Alexander V. Tikhonov <avtikhon@tarantool.org>
Reviewed-by: Igor Munkin <imun@tarantool.org>

Type was displayed in error messages, was returned in
meta headers, and a type string is a result of
typeof() SQL function.

Typeof() always returns lower case type string; meta
contained upper case type; error messages contained
both.

It was necessary to choose one case for everything,
and the lower one was chosen. It allows not to break
typeof() function which actually might be used by
someone.

There was a bug that netbox at any schema update called
on_connect() triggers. This was due to overcomplicated logic of
handling of changes in the netbox state machine. On_connect() was
fired each time the machine entered 'active' state, even if its
previous states were 'active' and then 'fetch_schema'. The latter
state can be entered many times without reconnects.

Another bug was about on_disconnect() - it could be fired even if
the connection never entered active state. For example, if its
first 'fetch_schema' has failed.

Now there is an explicit flag showing the machine connect state.
The triggers are fired only when it is changed, on 'active' and on
any error states. Intermediate states (fetch_schema, auth) do not
matter anymore.

Thanks @mtrempoltsev for the initial investigation and a draft
fix.

Closes #4593

This patch fixes memory leak in lbox_tuple_format_new().

Closes #4588

Box.session.su() worked like following: check user
existence, create its credentials on the stack, check
the function, call the function, destroy the
credentials, restore the old credentials.

After creating the credentials on the stack the
function check could raise a Lua error. It led to the
credentials object not being destroyed. As a result,
user.credentials_list was pointing at invalid memory.

Now there is no errors between creating the temporary
credentials and its destruction.

Closes #4597

This function is supposed to return NULL on an error.
For exceptions there is user_find_by_name_xc.