The SWIM unit tests code with the fake events and time does lots
of forbidden things: it manually invokes pending watcher
callbacks; manages global time without a kernel; puts not
existing descriptors into the loop. These foul blows open the
gates to the full control over IO events, descriptors, virtual
time. Hundreds of virtual seconds pass in milliseconds in
reality, it makes SWIM unit tests fast despite complex logic.

All these actions does not affect the loop until yield. On yield
a scheduler fiber wakes up and

    1) infinitely generates EV_READ on not existing descriptors
       because a kernel considers them closed;

    2) manual pending callbacks invocation asserts, because it is
       not allowed for non-scheduler fibers.

To avoid these problems a new isolated loop is created, not
visible for the scheduler. Here the fake events library can rack
and ruin whatever it wants.

Needed for #4250

When a member is deleted, pointer at its struct tt_uuid becomes
invalid. This commit makes member:uuid() create a new tt_uuid
object, cached for subsequent calls.

Travis-ci failed on brew install python2 due to it was
already installed on OSX 10.13 Sierra, so it needs
to be installed without fails on its exists.

Follow up #4254

Up to this point, the is_temporary flag has been set for surrogate space
definitions used to transfer meta-information during compilation stage
of DML queries (CREATE TABLE/INDEX etc).

At some point, it became possible to use spaces created in Lua in
SQL statements. Since it is allowed to create temporary spaces in
Lua, not all temporary spaces in SQL are surrogate wrappers. To separate
real temporary spaces (i.e. which came from space cache) from surrogate
ones, is_ephemeral flag is now set in such spaces instead of
is_temporary. Note that there can't be mess between these flags since
ephemeral spaces can exist only during execution of VDBE bytecode.
Also note that flag is required only for debugging facilities.

Close #4139

Fixed swim headers in addition to commit:
88892f13 ('swim: fix build on FreeBSD')

Included unistd.h header to fix the following errors:

In file included from src/lib/swim/swim_transport_udp.c:31:
src/lib/swim/swim_transport.h:61:1:
    error: unknown type name 'ssize_t'; did you mean 'size_t'?
ssize_t
^~~~~~~
size_t
/usr/include/netinet6/in6.h:698:18: note: 'size_t' declared here
typedef __size_t        size_t;
                        ^
In file included from src/lib/swim/swim_transport_udp.c:31:
src/lib/swim/swim_transport.h:66:1:
    error: unknown type name 'ssize_t'; did you mean 'size_t'?
ssize_t
^~~~~~~
size_t
/usr/include/netinet6/in6.h:698:18: note: 'size_t' declared here
typedef __size_t        size_t;

Closes #4050

Enabled use_unix_sockets_iproto option to use unix sockets
iproto provides the new way to handle the problem with
'Address already in use' error. It lets test-run appoint
unix sockets for LISTEN environment variable values.

Before this change the TcpPortDispatcher was used to
eliminate the race condition when two workers trying to use
the same port: the idea was that each worker used its own
ports range. Really these ports could race with client ports
(from, say, net.box or replication), which typically didn't
use bind() and so bound to a random available port (despite
any dispatched ranges) and could produce 'Address already in
use' error.

Closes: #4008

Removed skip flag file to switch on the testing of the
http_client test. Enabled http_client test on OSX,
fixed missing of the python2 symlink. Removed the subtest
on '595 error return' from 'error' suite, due to it may
hang forever. To enable test on travis-ci reverted commit:

1d7285c4 ('Disable flaky http_client.test.lua')

Closes #4254

This is the regression from a7c855e5
(net.box: fetch '_vcollation' sysview into the module). The result file
was corrupted, because the test sometimes produces incorrect result in
the test case for on_connect / on_disconnect triggers. This issue is
tracked by #4273.

Follow up #3941.

Fetch "_vcollation" sysview to show collation name instead collation id.

Closes #3941

Add "_vcollation" sysview to read it from net.box. This
sysview is always readable, except when a user doesn't have
"public" role.

Needed for #3941

@TarantoolBot document
Title: box.space._vcollation

_vcollation is a system space that represents a virtual view.
The structure of its tuples is identical to that of _collation.
Tuples of this sysview is always readable, except when the user
doesn't have "public" role.

Follow-up #4196

It is legacy flag and isn't used anymore.

Before this patch, the existence of space was checked for the
CREATE TABLE or CREATE VIEW statements during the parsing. If the
space already exists, an error has been set and cleanup is
performed. But, if the statement contained 'IF NOT EXIST', the
cleanup was performed, but the error was not set. Meanwhile,
create_foreign_key() assumes that if create_table_def->new_space
is NULL, then we are dealing with ALTER TABLE statement. This in
turn false, since ctd->new_space is nullified also in case of
already existing table. This causes an assertion or a segmentation
fault when creating a foreign key during space creation.

This patch moves this check to VDBE. Parsing now is always
processed till the end as in case space doesn’t exist.

Closes #4196

After compacting runs, we first mark them as dropped (VY_LOG_DROP_RUN),
then try to delete their files unless they are needed for recovery from
the checkpoint, and finally mark them as not needed in the vylog
(VY_LOG_FORGET_RUN). There's a potential race sitting here: the problem
is the garbage collector might kick in after files are dropped, but
before they are marked as not needed. If this happens, there will be
runs that have two VY_LOG_FORGET_RUN records, which will break recovery:

  Run XX is forgotten, but not registered

The following patches make the race more likely to happen so let's
eliminate it by making the garbage collector the only one who can mark
runs as not needed (i.e. write VY_LOG_FORGET_RUN record). There will
be no warnings, because the garbage collector silently ignores ENOENT
errors, see vy_gc().

Another good thing about this patch is that now we never yield inside
a vylog transaction, which makes it easier to remove the vylog latch
blocking implementation of transactional DDL.

Closes #3691

@TarantoolBot document
Title: check constraint for Lua space

The check constraint is a type of integrity constraint which
specifies a requirement that must be met by tuple before it
is inserted into space. The constraint result must be predictable.
Expression in check constraint must be <boolean value expression>
I.e. return boolean result.

Now it is possible to create ck constraints only for empty space
having format. Constraint expression is a string that defines
relations between top-level tuple fields.
Take into account that all names are converted to an uppercase
before resolve(like SQL does), use \" sign for names of fields
that were created not with SQL.

The check constraints are fired on insertion to the Lua space
together with Lua space triggers. The execution order of
ck constraints checks and space triggers follows their creation
sequence.

Note: this patch changes the CK constraints execution order for
SQL. Previously check of CK constraints integrity was fired before
tuple is formed; meanwhile now they are implemented as NoSQL before
replace triggers, which are fired right before tuple insertion.
In turn, type casts are performed earlier than msgpack
serialization. You should be careful with functions that use
field types in your check constrains (like typeof()).

Consider following situation:
```
 box.execute("CREATE TABLE t2(id  INT primary key,
                              x INTEGER CHECK (x > 1));")
 box.execute("INSERT INTO t2 VALUES(3, 1.1)")
```
the last operation would fail because 1.1 is silently
cast to integer 1 which is not greater than 1.

To create a new CK constraint for a space, use
space:create_check_constraint method. All space constraints are
shown in space.ck_constraint table. To drop ck constraint,
use :drop method.

Example:
```
s1 = box.schema.create_space('test1')
pk = s1:create_index('pk')
ck = s1:create_check_constraint('physics', 'X < Y')
s1:insert({2, 1}) -- fail
ck:drop()
```

To perform ck constraints tests before insert or update space
operation, we use precompiled VDBE machine associated with
each ck constraint, that is executed in on_replace trigger.
Each ck constraint VDBE code consists of
1) prologue code that maps new(or updated) tuple via binding,
2) ck constraint code generated by CK constraint AST.

In case of ck constraint error the tuple insert/replace operation
is aborted and ck constraint error is handled as diag message.

Needed for #3691

This patch introduces a new system space to persist check
constraints. The format of the new system space is

_ck_constraint (space id = 364)
[<space id> UINT, <constraint name> STR,
 <is_deferred>BOOL, <language>STR, <code>STR]

A CK constraint is local for a space, so every pair
<space id, CK name> is unique
(it is also the PK in the _ck_constraint space).

After insertion into this space, a new instance describing check
constraint is created. Check constraint holds an exspression AST.
While space features some check constraints, it isn't allowed to
be dropped. The :drop() space method firstly deletes all check
constraints and then removes an entry from the _space.

Because the space alter, the index alter and the space truncate
operations cause space recreation process, a new
RebuildCkConstrains object is introduced. This alter object
compiles a new ck constraint object, replaces and removes
an existent instances atomically (but if the assembly of some
ck constraint object fails, nothing is changed).
In fact, in scope of this patch we don't really need to recreate
a ck_constraint object in such situations (it is enough to patch
space_def pointer in AST tree like we did it before, but we are
going to recompile a VDBE that represents ck constraint in
further patches, and that operation is not safe).

The main motivation for these changes is an ability to support
ADD CHECK CONSTRAINT operation in the future. CK constraints are
easier to manage as self-sustained objects: such change is
managed with atomic insertion(unlike the current architecture).

Finally, the xfer optimization is disabled now if some space have
ck constraints. In following patches this xfer optimisation
becomes impossible, so there is no reason to rewrite this code
now.

Needed for #3691

Refactored OP_Column instruction with a new vdbe_field_ref class.
The vdbe_field_ref is a reusable object that speeds-up field
access for given tuple or tuple data.
Introduced OP_Fetch opcode that uses vdbe_field_ref given as a
first argument. This opcode makes possible to perform binding
of a new tuple to an existent VDBE without decoding it's fields.

Needed for #3691

This preparatory refactoring is necessary to simplify the process
of introducing a new OP_Fetch statement in the next patch.
- got rid of useless sMem local variable
- got rid of useless payloadSize in VdbeCursor structure

Needed for #3691

A new sql_bind_ptr routine allows to bing a generic pointer to
VDBE variable. This change is required to pass a tuple_fetcher
representing a new tuple to the check constraint VDBE.

Needed for #3961

The sql_flags is a parser parameter that describes how to parse
the SQL request, determines general behaviour: like whether
foreign keys are handled as deferred or not etc. But now this
information is taken from the global user session object.

When we need to run the parser with some other parameters, it is
necessary to change global session object, which may lead to
unpredictable consequences in general case.
Introduced a new parser and vdbe field sql_flags which is
responsible for SQL parsing results.

Needed for #3691

The SQL_NullCallback flag is never set now so it redundant.
Let's get red if it because we are going to pass
user_session->sql_flags to parser and use it instead of
session instance.

Needed for #3691

After making memtx space format check non-blocking, move the appropriate
vinyl test case to engine suite. Introduce a new errinj,
ERRINJ_CHECK_FORMAT_DELAY, to unify the test case for both engines.

Follow-up #3976

Just like index build, space check in memtx stalls event loop for the
whole check time. Add occasional yields, and an on_replace trigger,
which checks format for tuples inserted while space format check is in
progress.

Follow-up #3976

@TarantoolBot document
Title: memtx now checks space format in background

There is no event loop stall when memtx engine checks space format
anymore. You may insert tuples into a space, while its new format is
being checked. If the tuples don't match the new format, format change
will be aborted.

Currently to determine current source path, user has to do it
by getting current source info from the stack and parse it manually

This commit adds couple little helpers for this task called
`debug.sourcefile()` and `debug.sourcedir()`,
which returns a path to the file being executed.

The path is relative to the current working directory.
If such path cannot be determined, '.' is returned (i.e. interactive
mode). There is an exception if you are running script with an absolute
path. In that case, absolute path will be returned too.

```bash
tarantool /absolute/path/to/the/script.lua
```

There are also handy shortcuts for that functions: `debug.__file__` and
`debug.__dir__`

@TarantoolBot document
Title: Document new debug helper methods

There are two more helpers in debug module `sourcefile()` and
`sourcedir()`. This two helpers returns relative paths to source file
and directory respectively.

There are also `debug.__file__` and `debug.__dir__` which are just handy
ways to call `sourcefile()` and `sourcedir()`.

It should be mentioned that it is only possible to determine real path
to the file, if the function was defined in a some lua file.
`loadstring` may be exception here, since lua will store whole string
in debug info.

There is also a possible pitfall because of tail call optimization, so
if the function `fn` is defined inside `myapp/init.lua` like this:

```lua
function fn()
    return debug.sourcefile()
end
```

It would not be possible to determine file path correctly because that
function would not appear in a stacktrace. Even worse, it may return
wrong result, i.e. actual path where "parent" function was defined.

To force the interpreter to avoit this kind of optimizations you may use
parenthesis like this:

```lua
function fn()
    return (debug.sourcefile())
end
```

Also both `sourcefile` and `sourcedir` functions have optional level
argument, which allows set the level of the call stack function should
examine for source path. By default, 2 is used.

Example:
```
[001] box/on_replace.test.lua                                         [ pass ]
[001] box/on_replace.test.lua                                         [ fail ]
[001]
[001] Test failed! Result content mismatch:
[001] --- box/on_replace.result	Tue Jun  4 10:44:39 2019
[001] +++ box/on_replace.reject	Tue Jun  4 10:44:50 2019
[001] @@ -4,7 +4,7 @@
[001]  -- test c and lua triggers: must return only lua triggers
[001]  #box.space._space:on_replace()
[001]  ---
[001] -- 0
[001] +- 2
[001]  ...
```
This happened because we forgot to remove triggers set on _space.
Clear them below the test case.

This commit fixes a regression introduced by commit
5ab0763b
(pass operation type to lua triggers)

When passing operation type to the trigger, we relied on a corresponding
xrow_header, which would later be written to the log. When before_replace
triggers are fired, there is no row connected to the txn statement yet,
so we faked one to pass operations. We, however, didn't account for
temporary spaces, which never have a row associated with a corresponding
txn stmt. This lead to a segfault on an attemt to use on_replace triggers
with temporary spaces.

Add a fake row for temporary spaces to pass operation type in on_replace
triggers and add some tests.

Closes #4266

This reverts commit 6600dc1b.

Pushed by mistake before all changes.

Since DDL is triggered by the admin, it can be deliberately initiated
when the workload is known to be low. Throttling it along with DML
requests would only cause exasperation in this case. So we don't apply
disk-based rate limit to DDL. This should be fine, because the
disk-based limit is set rather strictly to let the workload some space
to grow, see vy_regulator_update_rate_limit(), and in contrast to the
memory-based limit, exceeding the disk-based limit doesn't result in
abrupt stalls - it may only lead to a gradual accumulation of disk space
usage and read latency.

Closes #4238

Fixed swim headers in addition to commit

Closes #4050

Since we have implemented memtx background index build, the
corresponding vinyl test cases are now also suitable for memtx,
so move them to engine suite so that both engines are tested.
Also add some tests to check that an ongoing index build is aborted in
case a tuple violating unique constraint or format of the new index is
inserted.
Add some error injections to unify appropriate memtx/vinyl tests.

Closes #3976

Memtx index build used to stall event loop for all the build period.
Add occasional yields so that the loop is not blocked for too long.
Also make index build set on_replace triggers so that concurrent
replaces are also correctly handled during the build.

Part of #3976

@TarantoolBot document
Title: memtx indices are now built in background

Memtx engine no longer stalls the event loop during an index build.
You may insert tuples into a space while an index build is in progress:
the tuples will be correctly added to the new index. If such tuple
violates the new indexes unique constraint or doesn't match the new
index format, the index build will be aborted.

If a key isn't found in the tuple cache, we fetch it from a run file. In
this case disk read and page decompression is done by a reader thread,
however key lookup in the fetched page is still performed by the tx
thread. Since pages are immutable, this could as well be done by the
reader thread, which would allow us to save some precious CPU cycles for
tx.

Close #4257

To handle fiber cancellation during page read we need to pin all objects
referenced by vy_page_read_task. Currently, there's the only such
object, vy_run. It has reference counting so pinning it is trivial.
However, to move page lookup to a reader thread, we need to also
reference key def, tuple format, and key. Format and key have reference
counting, but key def doesn't - we typically copy it. Copying it in this
case is too heavy.

Actually, cancelling a fiber manually or on timeout while it's reading
disk doesn't make much sense with PCIE attached flash drives. It used to
be reasonable with rotating disks, since a rotating disk controller
could retry reading a block indefinitely on read failure. It is still
relevant to Network Attached Storage. On the other hand, NAS has never
been tested, and what isn't tested, can and should be removed. For
complex SQL queries we'll be forced to rethink timeout handling anyway.

That being said, let's simply drop this functionality.

Page reading code is intermixed with the reader thread selection in the
same function, which makes it difficult to extend the former. So let's
introduce a helper function encapsulating a call on behalf of a reader
thread.

Since a page read task references the source run file, we don't need to
pass page info by value.

This function is a part of the run iterator API so we can't use it in
a reader thread. Let's make it an independent helper. As a good side
effect, we can now reuse it in the slice stream implementation.

Since commit f42596a2 we've started
to test if a fiber is cancelled in coio_wait. This may hang interactive
console with simple fiber.kill(fiber.self()) call.

Sane users don't do such tricks and this affects interactive console
mode only but still to be on a safe side lets exit if someone occasionally
killed the console fiber.

Notes:
 - such exit may ruine terminals settings and one need
   to reset it after;
 - the issue happens on interactive console only
   so I didn't find a way for automatic test and
   tested manually.

This warning breaks -Werror -O2 build on GCC 9.1.