index_def_new_from_tuple is used in on_replace_dd_index therefore
it has to be cleared from exceptions. Now it doesn't throw any
more. It means we also need to clear from exceptions it's
subsidiary functions: index_def_check_sequence,
index_def_check_tuple, index_opts_decode, func_index_check_func.
Their usages are updated.

Part of #4247

Implement a new function in Lua fiber library: top(). It returns a table
containing fiber cpu usage stats. The table has two entries:
"cpu_misses" and "cpu". "cpu" itself is a table listing all the alive
fibers and their cpu consumtion.
The patch relies on CPU timestamp counter to measure each fiber's time
share.

Closes #2694

@TarantoolBot document
Title: fiber: new function `fiber.top()`

`fiber.top()` returns a table of all alive fibers and lists their cpu
consumption. Let's take a look at the example:
```
tarantool> fiber.top()
---
- cpu:
    107/lua:
      instant: 30.967324490456
      time: 0.351821993
      average: 25.582738345233
    104/lua:
      instant: 9.6473633128437
      time: 0.110869897
      average: 7.9693406131877
    101/on_shutdown:
      instant: 0
      time: 0
      average: 0
    103/lua:
      instant: 9.8026528631511
      time: 0.112641118
      average: 18.138387232255
    106/lua:
      instant: 20.071174377224
      time: 0.226901357
      average: 17.077908441831
    102/interactive:
      instant: 0
      time: 9.6858e-05
      average: 0
    105/lua:
      instant: 9.2461986412164
      time: 0.10657528
      average: 7.7068458630827
    1/sched:
      instant: 20.265286315108
      time: 0.237095335
      average: 23.141537169257
  cpu_misses: 0
...

```
The two entries in a table returned by `fiber.top()` are
`cpu_misses` and `cpu`.

`cpu` itself is a table whose keys are strings containing fiber ids and
names.
The three metrics available for each fiber are:
1) instant (per cent),
which indicates the share of time fiber was executing during the
previous event loop iteration
2) average (per cent), which is calculated as an exponential moving
average of `instant` values over all previous event loop iterations.
3) time (seconds), which estimates how much cpu time each fiber spent
processing during its lifetime.

More info on `cpu_misses` field returned by `fiber.top()`:
`cpu_misses` indicates the amount of times tx thread detected it was
rescheduled on a different cpu core during the last event loop
iteration.
fiber.top() uses cpu timestamp counter to measure each fiber's execution
time. However, each cpu core may have its own counter value (you can
only rely on counter deltas if both measurements were taken on the same
core, otherwise the delta may even get negative).
When tx thread is rescheduled to a different cpu core, tarantool just
assumes cpu delta was zero for the latest measurement. This loweres
precision of our computations, so the bigger `cpu misses` value the
lower the precision of fiber.top() results.

Fiber.top() doesn't work on arm architecture at the moment.

Please note, that enabling fiber.top() slows down fiber switching by
about 15 per cent, so it is disabled by default.
To enable it you need to issue `fiber.top_enable()`.
You can disable it back after you finished debugging  using
`fiber.top_disable()`.
"Time" entry is also added to each fibers output in fiber.info()
(it duplicates "time" entry from fiber.top().cpu per fiber).
Note, that "time" is only counted while fiber.top is enabled.

Before the patch update was implemented as a set of operations
applicable for arrays only. It was ok until field names and JSON
paths appearance, because tuple is an array on the top level.

But now there are four reasons to allow more complex updates of
tuple field internals by JSON paths:

  - tuple field access by JSON path is allowed so for consistency
    JSON paths should be allowed in updates as well;

  - JSON indexes are supported. JSON update should be able to
    change an indexed field without rewriting half of a tuple, and
    its full replacement;

  - Tarantool is going to support documents in storage so JSON
    path updates is one more step forward;

  - JSON updates are going to be faster and more compact in WAL
    than get + in-memory Lua/connector update + replace (or update
    of a whole tuple field).

The patch reworks the current update code in such a way, that now
update is not just an array of operations, applied to tuple's top
level fields. Now it is a tree, just like tuples are.

The concept is to build a tree of xrow_update_field objects. Each
updates a part of a tuple. Leafs in the tree contain update
operations, specified by a user, as xrow_update_op objects.

To make the code support and understanding simpler, the patch
splits update implementation into several independent
files-modules for each type of an updated field. One file
describes how to update an array field, another file - how to
update a map field, etc. This commit introduces only array. Just
because it was already supported before the patch. Next commits
will introduce more types one by one.

Besides, the patch makes some minor changes, not separable from
this commit:

  - The big comment about xrow updates in xrow_update.c is
    updated. Now it describes the tree-idea presented above;

  - Comments were properly aligned by 66 symbols in all the moved
    or changed code. Not affected code is kept as is so as not to
    increase the diff even more;

  - Added missing comments to moved or changed structures and
    their attributes such as struct xrow_update,
    struct xrow_update_op_meta, struct xrow_update_op.

  - Struct xrow_update_field was significantly reworked. Now it is
    not just a couple of pointers at tuple's top level array. From
    now it stores type of the updated field, range of its source
    data in the original tuple, and a subtree of other update
    fields applied to the original data.

  - Added missing comments to some functions which I moved and
    decided worth commenting alongside, such as
    xrow_update_op_adjust_field_no(), xrow_update_alloc().

  - Functions xrow_update_op_do_f, xrow_update_op_read_arg_f,
    xrow_update_op_store_f are separated from struct xrow_update,
    so as they could be called on any updated field in the tree.
    From this moment they are methods of struct xrow_update_op.
    They take an op as a first argument (like 'this' in C++), and
    are applied to a given struct xrow_update_field.

Another notable, but not separable, change is a new naming schema
for the methods of struct xrow_update_field and struct
xrow_update_op. This is motivated by the fact that struct
xrow_update_field now has a type, and might be not a terminal.

There are now two groups of functions. Generic functions working
with struct xrow_update_field of any type:

    xrow_update_field_sizeof
    xrow_update_field_store
    xrow_update_op_do_field_<operation>

And typed functions:

    xrow_update_<type>_sizeof
    xrow_update_<type>_store
    xrow_update_op_do_<type>_<operation>

Where
    operation = insert/delete/set/arith ...
         type = array/map/bar/scalar ...

Common functions are used when type of a field to update is not
known in advance. For example, when an operation is applied to one
of fields of an array, it is not known what a type this field has:
another array, scalar, not changed field, map, etc. Common
functions do nothing more than just a switch by field type to
choose a more specific function.

Typed functions work with a specific type. They may change the
given field (add a new array element, replace it with a new value,
...), or may forward an operation deeper in case they see that its
JSON path is not fully passed yet.

Part of #1261

That patch finishes transformation of tuple_update public API to
xrow_update.

Part of #1261

Tuple_update is a too general name for the updates implemented
in these files. Indeed, a tuple can be updated from Lua, from
SQL, from update microlanguage. Xrow_update is a more specific
name, which is already widely used in tuple_update.c.

Part of #1261

When invalid command is passed we should send an error message to a
client. Instead a nil dereference occurs that causes abnormal exit of a
console.

This is the regression from 96dbc49d
('box/console: Refactor command handling').

Reported-by: Mergen Imeev <imeevma@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>

Added build + test jobs in GitLab-CI and build + test + deploy jobs on
Travis-CI for CentOS 8.

Updated testing dependencies in the RPM spec to follow the new Python 2
package naming scheme that was introduced in CentOS 8: it uses
'python2-' prefix rather then 'python-'.

CentOS 8 does not provide python2-gevent and python2-greenlet packages,
so they were pushed to https://packagecloud.io/packpack/backports
repository. This repository is enabled in our build image
(packpack/packpack:el-8) by default. Those dependencies are build-time,
so nothing was changed for a user. The source RPM packages were gathered
from https://cbs.centos.org

.

Disabled app-tap/pwd.test.lua on CentOS 8 due to systemd-nss issue,
which was not worked around properly. Filed #4592 to resolved it in the
future.

Eliminated libunwind runtime dependency (and libunwind-devel build
dependency) on CentOS 8, because the base system does not provide it.
fiber.info() backtraces and printing of a backtrace after a crash will
not be available on this system. Hopefully we'll fix it in the future,
filed #4611 on this.

Closes #4543

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>
Reviewed-by: Igor Munkin <imun@tarantool.org>

After ea5929db ('build: fix OpenSSL
linking problems on FreeBSD') we set CFLAGS explicitly (possibly to an
empty value) when invoking a configure script for curl. When this
parameter is set the script does not use a value of environment variable
CFLAGS.

Before this commit LDFLAGS environment variable can affect build of curl
submodule. This can lead to a problem when a user or a tool set CFLAGS
and LDFLAGS both and some linker flag assumes that some compilation flag
is present. Here we set empty LDFLAGS explicitly to avoid using of the
environment variable.

A distributive build tool such as rpmbuild or emerge usually sets CFLAGS
and LDFLAGS. The problem with incompatible compiler / linker options has
been reveal under rpmbuild on CentOS 8 with hardened build enabled
(which is so when backtraces are disabled).

It is not clear whether we should follow environment variables or values
determined by CMake for CFLAGS, CPPFLAGS and LDFLAGS when building a
submodule (such as luajit and curl). Let's decide about this later.

Part of #4543.

Reviewed-by: Alexander V. Tikhonov <avtikhon@tarantool.org>
Reviewed-by: Igor Munkin <imun@tarantool.org>

Type was displayed in error messages, was returned in
meta headers, and a type string is a result of
typeof() SQL function.

Typeof() always returns lower case type string; meta
contained upper case type; error messages contained
both.

It was necessary to choose one case for everything,
and the lower one was chosen. It allows not to break
typeof() function which actually might be used by
someone.

There was a bug that netbox at any schema update called
on_connect() triggers. This was due to overcomplicated logic of
handling of changes in the netbox state machine. On_connect() was
fired each time the machine entered 'active' state, even if its
previous states were 'active' and then 'fetch_schema'. The latter
state can be entered many times without reconnects.

Another bug was about on_disconnect() - it could be fired even if
the connection never entered active state. For example, if its
first 'fetch_schema' has failed.

Now there is an explicit flag showing the machine connect state.
The triggers are fired only when it is changed, on 'active' and on
any error states. Intermediate states (fetch_schema, auth) do not
matter anymore.

Thanks @mtrempoltsev for the initial investigation and a draft
fix.

Closes #4593

This patch fixes memory leak in lbox_tuple_format_new().

Closes #4588

Box.session.su() worked like following: check user
existence, create its credentials on the stack, check
the function, call the function, destroy the
credentials, restore the old credentials.

After creating the credentials on the stack the
function check could raise a Lua error. It led to the
credentials object not being destroyed. As a result,
user.credentials_list was pointing at invalid memory.

Now there is no errors between creating the temporary
credentials and its destruction.

Closes #4597

This function is supposed to return NULL on an error.
For exceptions there is user_find_by_name_xc.

Some guest user privileges were not revoked in the
end.

Func_delete() called credentials_destroy() after
func->vtab->destroy(). But appeared, that vtab->destroy() is
actually delete, and it frees the func object. Now the func's
owner credentials are destroyed before the function is freed.

Closes #4597
Follow up #2763

Explicit cast uses uppercase, and the patch makes the
implicit cast the same.

Upper case is the standard according to SQL standard
2011, cast specification 6.13, general rules 11.e:

    General rules

    11) If TD is variable-length character string or
        large object character string, then let MLTD
        be the maximum length in characters of TD.

        e) If SD (source type) is boolean, then Case:

            i)   If SV is True and MLTD is not less
                 than 4, then TV is 'TRUE'.

            ii)  If SV is False and MLTD is not less
                 than 5, then TV is 'FALSE'.

            iii) Otherwise, an exception condition is
                 raised: data exception — invalid
                 character value for cast.

Part of #4462

Before the patch LENGTH didn't take boolean argument
into account. Now it does and treats like any other
non-string argument - stringify and calculate length.

It is worth mentioning, that in future LENGTH will
discard any non-string argument, see #3929.

Part of #4462

Argparse module stores unspecified parameter values as boolean
true. It led to a problem, that a command line '--value' with
'value' defined as a number or a string, showed a strange error
message:

    Expected number/string, got "true"

Even though a user didn't pass any value. Now it shows 'nothing'
instead of '"true"'. That is clearer.

Follow up #4076

There was a complaint that tarantoolctl --show-system option is
very hard to use. It incorrectly parsed passed values, and
provided strange errors.

    tarantoolctl cat --show-system true
    Bad input for parameter "show-system". Expected boolean, got "true"

    tarantoolctl cat --show-system 1
    Bad input for parameter "show-system". Expected boolean, got "1"

    tarantoolctl cat --show-system=true
    Bad input for parameter "show-system". Expected boolean, got "true"

First of all, appeared that the complaining people didn't read
documentation in 'tarantoolctl --help'. It explicitly says, that
'--show-system' should go after a file name, and does not have a value.

Secondly, even having taken the documentation into account, the
errors indeed look ridiculous. 'Expected boolean, got "true"'
looks especially weird.

The problem appeared to be with argparse module, how it parses
boolean parameters, and how stores parameter values not specified
in a command line.

All parameters were parsed into a dictionary: parameter name ->
value. If a name is alone (no value), then it is boolean true.
Otherwise it was always a string value. An attempt to specify
an explicit parameter value 'true' led to storing string 'true'
in that dictionary.

Consequential check for boolean parameters was trivial:
type(value) == 'boolean', which was obviously wrong, and didn't
pass for 'true' string, but passed for an empty value.

Closes #4076

Credentials is a cache of user universal privileges. And that
cache can become outdated in case user privs were changed after
creation of the cache.

The patch makes user update all its credentials caches with new
privileges, via a list of all creds.

That solves a couple of real life problems:

- If a user managed to connect after box.cfg started listening
port, but before access was granted, then he needed a reconnect;

- Even if access was granted, a user may connect after box.cfg
listen, but before access *is recovered* from _priv space. It
was not possible to fix without a reconnect. And this problem
affected replication.

Closes #2763
Part of #4535
Part of #4536

@TarantoolBot document
Title: User privileges update affects existing sessions and objects
Previously if user privileges were updated (via
`box.schema.user.grant/revoke`), it was not reflected in already
existing sessions and objects like functions. Now it is.

For example:
```
        box.cfg{listen = 3313}
        box.schema.user.create('test_user', {password = '1'})
        function test1() return 'success' end

        c = require('net.box').connect(box.cfg.listen, {
                user = 'test_user', password = '1'
        })
        -- Error, no access for this connection.
        c:call('test1')

        box.schema.user.grant('test_user', 'execute', 'universe')
        -- Now works, even though access was granted after
        -- connection.
        c:call('test1')
```

A similar thing happens now with `box.session.su` and functions
created via `box.schema.func.create` with `setuid` flag.

In other words, now user privileges update is reflected
everywhere immediately.

(cherry picked from commit 06dbcec597f14fae6b3a7fa2361f2ac513099662)

Struct credentials is a cache of user's universal privileges. It
is static and is never changed after creation. That is a problem.
If a user privileges are updated, it is not reflected in his
existing credentials caches.

This patch reworks credentials API so as now this struct is not
just a container for several numbers. It is an object with
standard methods like create(), destroy(). A credentials object
still is not updated together with its source user, but now at
least the API allows to fix that.

Next patch will link all struct credentials of a user into a list
via which the user will be able to keep the credentials up to
date.

Part of #2763

(cherry picked from commit a8c3ebdbfc97b72832ebc5d87b681a310cce9589)

Added --exclude option (#54).

First reason - update is a too general name. Tarantool
has SQL update, update in Lua, configuration update.
So name 'just update' can't be used.

'Xrow update' fits because it works directly with
MessagePack tuple internals and because the updates
are persisted in WAL in that format.

Second reason, without which the first one would not
matter - next patches are going to split
tuple_update.c into multiple module files. That will
make some structures and functions of tuple_update.c
be declared in header files, what makes them a public
API of xrow update.

Public API methods should be prefixed with their
subsystem name, and here it is 'xrow_update_'.

Part of #1261

Rope is a library to define a custom rope data
structure with specified type of a stored value, and
some rope functions such as split, alloc.

It is possible to choose a unique name for a defined
rope structure. It was implemented as

    #define rope_api(x) rope_##rope_name##_##x
    #define rope rope_##rope_name

But with such rope_api definition it was always
expanded to `rope_rope_name_<x value>`. So rope_name
was basically a constant 'rope_name' regardless what
was defined under it.

The patch fixes it and makes name generation just like
bps_tree.h.

Additionally, the name template is changed a bit, now
it is

    <rope_name>_ + rope + _<method>

instead of

    <rope> + _<rope_name>_ + <method>

It just appeared to look better. For example, consider
rope name 'xrow_update' and method 'size':

    new: xrow_update_rope_size()
    old: rope_xrow_update_size()

The second name would be generated by the old template
and looks wrong. The new name not only looks better,
but also conforms with our code style.

Now errors are returned using errcode in triggers
and this functions wasn't updated to do so.
It doesn't throw, so it set to return 0 unconditionally.

This allows to overcome problems when CMake chooses one toolchain to
build tarantool, but a library (libluajit.a or libcurl.a) is built using
another (incompatible) toolchain.

Fixes #4587.

FreeBSD has OpenSSL as part of the base system: libraries are located in
/usr/lib, headers are in /usr/include. However a user may install the
library into /usr/local/{lib,include} from ports / pkg. In this case
tarantool did choose /usr/local version, while libcurl will pick up a
base system library. This is fixed by passing --with-ssl option with an
argument (/usr/local or /usr if custom -DOPENSSL_ROOT_DIR=<...> is not
passed).

Now the behaviour is the following. If -DOPENSSL_ROOT_DIR=<...> is
passed, then try to use OpenSSL from it. Otherwise find the library in
/usr/local and then in /usr. This is right as for tarantool's crypto
module as well as for libcurl submodule.

There is a flaw here: a user is unable to choose a base system library
if a ports / pkg version of OpenSSL is installed. The reason here is
that tarantool's crypto module depends on other libraries and
-I/usr/local/include may be added to build options. I have no good
solution for that, so `cmake . -DOPENSSL_ROOT_DIR=/usr` will give a
warning on FreeBSD and `gmake` likely will fail if libraries are of
different versions (see cmake/os.cmake comments for more information).
See also a [discussion][1] in FreeBSD community about all those /usr and
/usr/local problems.

There were two other problems that may fail tarantool build on FreeBSD:
they are fixed in this commit and described below.

First, libcurl's configure script chooses GCC by default if it exists
(say, installed from ports / pkg). It is unexpected behaviour when
tarantool sources itself are built with clang. Now it is fixed by
passing a compiler explicitly to the libcurl's configure script: the
library will use base system clang by default or one that a user pass to
tarantool's cmake.

Side note: GCC has /usr/local/include in its default headers search
paths; libcurl's configure script chooses GCC as a compiler and OpenSSL
from a base system by default (when CC and --with-ssl=<...> are not set)
that leads to OpenSSL header / library mismatch. It is the primary
reason of the build fail that was fixed in
1f2338bd ('build: FreeBSD packages
installation'). It is not much relevant anymore, because we don't try to
link with a base system OpenSSL if /usr/local one exists (however if it
is asked explicitly with -DOPENSSL_ROOT_DIR=<...> we'll do, but will
give a warning). Anyway, it is important to know such details if we'll
change build scripts in a future.

Second, backtraces are not supported on FreeBSD, but were enabled if
libunwind headers is found. This leads to an error on cmake stage,
because of inability to find a right library (this is a bug). Now we
disable backtraces on FreeBSD by default even if libunwind is found. See

When CC is passed to libcurl's configure script, the new problem opens
on Mac OS. CMake chooses XCode toolchain by default (at least on a
particular system where I tried it), which requires -isysroot=<SDK_PATH>
option to be passed to a preprocessor and a compiler in order to find
system headers. See [2] for more information.

[1]: https://wiki.freebsd.org/WarnerLosh/UsrLocal
[2]: https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes#3035623

Follows up #4490.

schema_find_grants is used in some triggers therefore it has to be
cleared from exceptions. Now it doesn't throw any more.
It's usages are updated.

Part of #4247

modify_priv, revoke_priv & on_replace_dd_priv triggers are
cleared from exceptions. A list of functions: priv_def_check,
priv_def_create_from_tuple, user_grant_priv, user_reload_privs,
rebuild_effective_grants, grant_revoke, role_check, role_grant,
role_revoke, priv_grant, access_check_ddl & txn_alter_trigger_new
were also refactored to achieve it. Their usages are updated.
user_find_xc is removed as far as it is not needed anymore.

Part of #4247

replica_check_id is used in on_replace_dd_cluster trigger
therefore it has to be cleared from exceptions. Now it doesn't
throw any more. It's usages are updated.

Part of #4247

`tnt_raise` and `diag_raise` are now properly replaced with
`diag_set` and `return -1` in alter.cc triggers.

Part of #4247

Some functions called from triggers won't be cleared from
exceptions within this patchset. They are wrapped in try..catch
blocks.

Part of #4247

Trigger function returning type is changed from void to int and
any non-zero value means the trigger was processed with an error.
A trigger can still raise an error - there is no more refactoring
except obvious `diag_raise();' --> `return -1;' replacement.

Prerequisites: #4247

Currently, the test shows all the data contained in the spaces
_func, _user and _space. This was added 6 years ago and is no
longer needed. In addition, this is inconvenient, as some data
changes every time bootstrap.snap is created. Due to this, we must
update the test result file every time we generate bootstrap.snap.
This patch removes the display of this data from the test.

Before the patch there was a race in replication
password configuration. It was possible that a replica
connects to a master with a custom password before
that password is actually set. The replica treated the
error as critical and exited.

But in fact it is not critical. Replica even can
withstand absence of a user and keeps reconnecting.
Wrong password situation arises from the same problem
of non atomic configuration and is fixed the same -
keep reconnect attempts if the password was wrong.

Closes #4550

Closes #4519

@TarantoolBot document
Title: key_def.new() accept both 'field' and 'fieldno'

Before the patch key_def.new() took an index part
array as it is returned in <index_object>.parts: each
part should include 'type', 'fieldno', and what else
.parts element contains.

But it was not possible to create a key_def from an
index definition - the array passed to
<space_object>.create_index() 'parts' argument. Because
key_def.new() didn't recognize 'field' option. That
might be useful, when a key_def is needed on a remote
client, where a space object and its indexes do not
exist. And it would be strange to force a user to
create them just so as he would be able to access

    <net_box connection>.space.<space_name>.
        index.<index_name>.parts

As well as it would be crutchy to make a user manually
replace 'field' with 'fieldno' in its index definition
just to create a key_def.

Additionally, an ability to pass an index definition
to a key_def constructor makes the API more symmetric.

Note, that it still is not 100% symmetric, because a
user can't pass field names to the key_def
constructor. A space is needed for that anyway.

The previous patch introduced a way to set box.cfg options
in a strict order, even on a reconfiguration. It was used to set
listen before replication.
The same order problem existed for replication settings. A user
could do

    box.cfg{
        replication_connect_quorum = 0,
        replication = {...}
    }

and expect, that due to quorum 0 the cfg() will return
immediately. But actually the behaviour was undefined - due to
arbitrary order of keys in a Lua table, replication could be
applied before quorum.

The patch makes all replication settings be applied before
replication.

Follow up #4433
Part of #3760

Before the patch the nil UUID was ignored and a new random one
was generated. This was because internally box treats nil UUID
as its absence.

Now a user will see an explicit message that nil UUID is a
reserved value.

Closes #4282

Added Ubuntu 19.10 Eoan Ermine into CI.

Close #4583

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>