C struct error objects can be created directly only in C.
C-side increments their reference counter when pushes to the Lua
stack.

It is not going to be so convenient soon. error_unpack() function
will be used in netbox to decode error object via Lua FFI.

Such error object will have 0 refs and no Lua GC callback
established. Because it won't be pushed on Lua stack natually,
from Lua C. To make such errors alive their reference counter
will be incremented and error_unref() will be set as GC callback.

Follow up for #4398

Co-authored-by: Vladislav <Shpilevoy&lt;v.shpilevoy@tarantool.org>

Closes #4398

@TarantoolBot document
Title: Error objects encoding in MessagePack

Until now an error sent over IProto, or serialized into
MessagePack was turned into a string consisting of the error
message. As a result, all other error object attributes were lost,
including type of the object. On client side seeing a string it
was not possible to tell whether the string is a real string, or
it is a serialized error.

To deal with that the error objects encoding is reworked from the
scratch. Now, when session setting `error_marshaling_enabled` is
true, all fibers of that session will encode error objects as a
new MP_EXT type - MP_ERROR (0x03).

```
    +--------+----------+========+
    | MP_EXT | MP_ERROR | MP_MAP |
    +--------+----------+========+

    MP_ERROR: <MP_MAP> {
        MP_ERROR_STACK: <MP_ARRAY> [
            <MP_MAP> {
                MP_ERROR_TYPE: <MP_STR>,
                MP_ERROR_FILE: <MP_STR>,
                MP_ERROR_LINE: <MP_UINT>,
                MP_ERROR_MESSAGE: <MP_STR>,
                MP_ERROR_ERRNO: <MP_UINT>,
                MP_ERROR_CODE: <MP_UINT>,
                MP_ERROR_FIELDS: <MP_MAP> {
                    <MP_STR>: ...,
                    <MP_STR>: ...,
                    ...
                },
                ...
            },
            ...
        ]
    }
```

On the top level there is a single key: `MP_ERROR_STACK = 0x00`.
More keys can be added in future, and a client should ignore all
unknown keys to keep compatibility with new versions.

Every error in the stack is a map with the following keys:
* `MP_ERROR_TYPE = 0x00` - error type. This is what is visible in
  `<error_object>.base_type` field;
* `MP_ERROR_FILE = 0x01` - file name from `<error_object>.trace`;
* `MP_ERROR_LINE = 0x02` - line from `<error_object>.trace`;
* `MP_ERROR_MESSAGE = 0x03` - error message from
  `<error_object>.message`;
* `MP_ERROR_ERRNO = 0x04` - errno saved at the moment of the error
  creation. Visible in `<error_object>.errno`;
* `MP_ERROR_CODE = 0x05` - error code. Visible in
  `<error_object>.code` and in C function `box_error_code()`.
* `MP_ERROR_FIELDS = 0x06` - additional fields depending on error
  type. For example, AccessDenied error type stores here fields
  `access_type`, `object_type`, `object_name`. Connector's code
  should ignore unknown keys met here, and be ready, that for some
  existing errors new fields can be added, old can be dropped.

Lua C module 'msgpack' supports registration of custom extension
decoders for MP_EXT values. That is needed to make 'msgpack' not
depending on any modules which use it.

So far the only box-related extension were tuples - struct tuple
cdata needed to be encoded as an array.

That is going to change in next commits, where struct error cdata
appears, also depending on box. So the decoder can't be located
in src/box/lua/tuple.c. It is moved to a more common place -
src/box/lua/init.c.

Needed for #4398

We want to have a transparent marshalling through net.box
for errors. To do this, we need to recreate the error
on the client side with the same parameters as on the server.
For convenience, we update AccessDeniedError constructor
which has pointers to static strings and add the XlogGapError
constructor that does not require vclock.

Needed for #4398

Errors are encoded as a string when serialized to MessagePack to
be sent over IProto or when just saved into a buffer via Lua
modules msgpackffi and msgpack.

That is not very useful on client-side, because most of the error
metadata is lost: code, type, trace - everything except the
message.

Next commits are going to dedicate a new MP_EXT type to error
objects so as everything could be encoded, and on client side it
would be possible to restore types.

But this is a breaking change in case some users use old
connectors when work with newer Tarantool instances. So to smooth
the upgrade there is a new session setting -
'error_marshaling_enabled'.

By default it is false. When it is true, all fibers of the given
session will serialize error objects as MP_EXT.

Co-authored-by: Vladislav <Shpilevoy&lt;v.shpilevoy@tarantool.org>

Needed for #4398

Session settings are stored in a monolithic array. Submodules
can define a range of settings in there. For example, SQL. It
occupies settings from 0 to 8. There is a second array of only
SQL settings in build.c, of the same size, and it uses the same
indexes.

But if something will be added before SQL settings, so as they
won't start from 0, it will break the equal indexes assumption.
SQL should normalize all setting identifiers by
SESSION_SETTING_SQL_BEGIN.

Co-authored-by: Vladislav <Shpilevoy&lt;v.shpilevoy@tarantool.org>

Part of #4398

@TarantoolBot document
Title: Custom error types for Lua errors

Errors can be created in 2 ways: `box.error.new()` and `box.error()`.

Both used to take either `code, reason, <reason string args>` or
`{code = code, reason = reason, ...}` arguments.

Now in the first option instead of code a user can specify a
string as its own error type. In the second option a user can
specify both code and type. For example:

```Lua
box.error('MyErrorType', 'Message')
box.error({type = 'MyErrorType', code = 1024, reason = 'Message'})
```
Or no-throw version:
```Lua
box.error.new('MyErrorType', 'Message')
box.error.new({type = 'MyErrorType', code = 1024, reason = 'Message'})
```
When a custom type is specified, it is shown in `err.type`
attribute. When it is not specified, `err.type` shows one of
built-in errors such as 'ClientError', 'OurOfMemory', etc.

Name length limit on the custom type is 63 bytes. All what is
longer is truncated.

Original error type can be checked using `err.base_type` member,
although normally it should not be used. For user-defined types
base type is 'CustomError'.

For example:
```
tarantool> e = box.error.new({type = 'MyErrorType', code = 1024, reason = 'Message'})
---
...

tarantool> e:unpack()
---
- code: 1024
  trace:
  - file: '[string "e = box.error.new({type = ''MyErrorType'', code..."]'
    line: 1
  type: MyErrorType
  custom_type: MyErrorType
  message: Message
  base_type: CustomError
...
```

As stated in the 'OS/X AND DARWIN BUGS' section of the libev
documentation [1], kqueue() and poll() have known problems on Mac OS, so
the library uses select() on Mac OS (it is the build time default). The
library however uses the trick to overcome 1024 fds limit: libev sets
the undocumented macro _DARWIN_UNLIMITED_SELECT, which enables linking
against select() implementation without the limit.

The magic macro stops working at some point around Mac OS 10.10 (see
[2]), because it was defined after <sys/time.h> inclusion.  For recent
Mac OS versions the macro has effect only when it is defined before
<sys/time.h> inclusion.

The macro definition was [moved][3] in libev 4.25. Excerpt from the
changelog [4]:

 | 4.25 Fri Dec 21 07:49:20 CET 2018
 | <...>
 | - move the darwin select workaround higher in ev.c, as newer versions of
 |   darwin managed to break their broken select even more.

More proper fix would be updating of libev to a newer version, however I
would postpone it until a moment when we'll have a time to properly test
everything with a new version of the library.

[1]: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#OS_X_AND_DARWIN_BUGS
[2]: http://lists.schmorp.de/pipermail/libev/2018q2/002788.html
[3]: http://cvs.schmorp.de/libev/ev.c?r1=1.482&r2=1.483
[4]: http://cvs.schmorp.de/libev/Changes?view=markup



Fixes #3867
Fixes #4673

Investigated-by: Maria Khaydich <maria.khaydich@tarantool.org>
Co-authored-by: Maria Khaydich <maria.khaydich@tarantool.org>

Fixes #4031

Co-developed-by: Cyrill Gorcunov <gorcunov@gmail.com>
Co-developed-by: Igor Munkin <imun@tarantool.org>
Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Igor Munkin <imun@tarantool.org>

@TarantoolBot document
Title: popen module

```
Overview
========

Tarantool supports execution of external programs similarly to well
known Python's `subprocess` or Ruby's `Open3`. Note though the `popen`
module does not match one to one to the helpers these languages provide
and provides only basic functions. The popen object creation is
implemented via `vfork()` system call which means the caller thread is
blocked until execution of a child process begins.

Module functions
================

The `popen` module provides two functions to create that named popen
object: `popen.shell` which is the similar to libc `popen` syscall and
`popen.new` to create popen object with more specific options.

`popen.shell(command[, mode]) -> handle, err`
---------------------------------------------

Execute a shell command.

@param command  a command to run, mandatory
@param mode     communication mode, optional
                'w'    to use ph:write()
                'r'    to use ph:read()
                'R'    to use ph:read({stderr = true})
                nil    inherit parent's std* file descriptors

Several mode characters can be set together: 'rw', 'rRw', etc.

This function is just shortcut for popen.new({command}, opts)
with opts.{shell,setsid,group_signal} set to `true` and
and opts.{stdin,stdout,stderr} set based on `mode` parameter.

All std* streams are inherited from parent by default if it is
not changed using mode: 'r' for stdout, 'R' for stderr, 'w' for
stdin.

Raise an error on incorrect parameters:

- IllegalParams: incorrect type or value of a parameter.

Return a popen handle on success.

Return `nil, err` on a failure.
@see popen.new() for possible reasons.

Example:

 | local popen = require('popen')
 |
 | -- Run the program and save its handle.
 | local ph = popen.shell('date', 'r')
 |
 | -- Read program's output, strip trailing newline.
 | local date = ph:read():rstrip()
 |
 | -- Free resources. The process is killed (but 'date'
 | -- exits itself anyway).
 | ph:close()
 |
 | print(date)

Execute 'sh -c date' command, read the output and close the
popen object.

Unix defines a text file as a sequence of lines, each ends
with the newline symbol. The same convention is usually
applied for a text output of a command (so when it is
redirected to a file, the file will be correct).

However internally an application usually operates on
strings, which are NOT newline terminated (e.g. literals
for error messages). The newline is usually added right
before a string is written to the outside world (stdout,
console or log). :rstrip() in the example above is shown
for this sake.

`popen.new(argv[, opts]) -> handle, err`
----------------------------------------

Execute a child program in a new process.

@param argv  an array of a program to run with
             command line options, mandatory;
             absolute path to the program is required
             when @a opts.shell is false (default)

@param opts  table of options

@param opts.stdin   action on STDIN_FILENO
@param opts.stdout  action on STDOUT_FILENO
@param opts.stderr  action on STDERR_FILENO

File descriptor actions:

    popen.opts.INHERIT  (== 'inherit') [default]
                        inherit the fd from the parent
    popen.opts.DEVNULL  (== 'devnull')
                        open /dev/null on the fd
    popen.opts.CLOSE    (== 'close')
                        close the fd
    popen.opts.PIPE     (== 'pipe')
                        feed data from/to the fd to parent
                        using a pipe

@param opts.env  a table of environment variables to
                 be used inside a process; key is a
                 variable name, value is a variable
                 value.
                 - when is not set then the current
                   environment is inherited;
                 - if set to an empty table then the
                   environment will be dropped
                 - if set then the environment will be
                   replaced

@param opts.shell            (boolean, default: false)
       true                  run a child process via
                             'sh -c "${opts.argv}"'
       false                 call the executable directly

@param opts.setsid           (boolean, default: false)
       true                  run the program in a new
                             session
       false                 run the program in the
                             tarantool instance's
                             session and process group

@param opts.close_fds        (boolean, default: true)
       true                  close all inherited fds from a
                             parent
       false                 don't do that

@param opts.restore_signals  (boolean, default: true)
       true                  reset all signal actions
                             modified in parent's process
       false                 inherit changed actions

@param opts.group_signal     (boolean, default: false)
       true                  send signal to a child process
                             group (only when opts.setsid is
                             enabled)
       false                 send signal to a child process
                             only

@param opts.keep_child       (boolean, default: false)
       true                  don't send SIGKILL to a child
                             process at freeing (by :close()
                             or Lua GC)
       false                 send SIGKILL to a child process
                             (or a process group if
                             opts.group_signal is enabled) at
                             :close() or collecting of the
                             handle by Lua GC

The returned handle provides :close() method to explicitly
release all occupied resources (including the child process
itself if @a opts.keep_child is not set). However if the
method is not called for a handle during its lifetime, the
same freeing actions will be triggered by Lua GC.

It is recommended to use opts.setsid + opts.group_signal
if a child process may spawn its own childs and they all
should be killed together.

Note: A signal will not be sent if the child process is
already dead: otherwise we might kill another process that
occupies the same PID later. This means that if the child
process dies before its own childs, the function will not
send a signal to the process group even when opts.setsid and
opts.group_signal are set.

Use os.environ() to pass copy of current environment with
several replacements (see example 2 below).

Raise an error on incorrect parameters:

- IllegalParams: incorrect type or value of a parameter.
- IllegalParams: group signal is set, while setsid is not.

Return a popen handle on success.

Return `nil, err` on a failure. Possible reasons:

- SystemError: dup(), fcntl(), pipe(), vfork() or close()
               fails in the parent process.
- SystemError: (temporary restriction) the parent process
               has closed stdin, stdout or stderr.
- OutOfMemory: unable to allocate the handle or a temporary
               buffer.

Example 1:

 | local popen = require('popen')
 |
 | local ph = popen.new({'/bin/date'}, {
 |     stdout = popen.opts.PIPE,
 | })
 | local date = ph:read():rstrip()
 | ph:close()
 | print(date) -- Thu 16 Apr 2020 01:40:56 AM MSK

Execute 'date' command, read the result and close the
popen object.

Example 2:

 | local popen = require('popen')
 |
 | local env = os.environ()
 | env['FOO'] = 'bar'
 |
 | local ph = popen.new({'echo "${FOO}"'}, {
 |     stdout = popen.opts.PIPE,
 |     shell = true,
 |     env = env,
 | })
 | local res = ph:read():rstrip()
 | ph:close()
 | print(res) -- bar

It is quite similar to the previous one, but sets the
environment variable and uses shell builtin 'echo' to
show it.

Example 3:

 | local popen = require('popen')
 |
 | local ph = popen.new({'echo hello >&2'}, { -- !!
 |     stderr = popen.opts.PIPE,              -- !!
 |     shell = true,
 | })
 | local res = ph:read({stderr = true}):rstrip()
 | ph:close()
 | print(res) -- hello

This example demonstrates how to capture child's stderr.

Example 4:

 | local function call_jq(input, filter)
 |     -- Start jq process, connect to stdin, stdout and stderr.
 |     local jq_argv = {'/usr/bin/jq', '-M', '--unbuffered', filter}
 |     local ph, err = popen.new(jq_argv, {
 |         stdin = popen.opts.PIPE,
 |         stdout = popen.opts.PIPE,
 |         stderr = popen.opts.PIPE,
 |     })
 |     if ph == nil then return nil, err end
 |
 |     -- Write input data to child's stdin and send EOF.
 |     local ok, err = ph:write(input)
 |     if not ok then return nil, err end
 |     ph:shutdown({stdin = true})
 |
 |     -- Read everything until EOF.
 |     local chunks = {}
 |     while true do
 |         local chunk, err = ph:read()
 |         if chunk == nil then
 |             ph:close()
 |             return nil, err
 |         end
 |         if chunk == '' then break end -- EOF
 |         table.insert(chunks, chunk)
 |     end
 |
 |     -- Read diagnostics from stderr if any.
 |     local err = ph:read({stderr = true})
 |     if err ~= '' then
 |         ph:close()
 |         return nil, err
 |     end
 |
 |     -- Glue all chunks, strip trailing newline.
 |     return table.concat(chunks):rstrip()
 | end

Demonstrates how to run a stream program (like `grep`, `sed`
and so), write to its stdin and read from its stdout.

The example assumes that input data are small enough to fit
a pipe buffer (typically 64 KiB, but depends on a platform
and its configuration). It will stuck in :write() for large
data. How to handle this case: call :read() in a loop in
another fiber (start it before a first :write()).

If a process writes large text to stderr, it may fill out
stderr pipe buffer and stuck in write(2, ...). So we need
to read stderr in a separate fiber to handle this case.

Handle methods
==============

`popen_handle:read([opts]) -> str, err`
---------------------------------------

Read data from a child peer.

@param handle        handle of a child process
@param opts          an options table
@param opts.stdout   whether to read from stdout, boolean
                     (default: true)
@param opts.stderr   whether to read from stderr, boolean
                     (default: false)
@param opts.timeout  time quota in seconds
                     (default: 100 years)

Read data from stdout or stderr streams with @a timeout.
By default it reads from stdout. Set @a opts.stderr to
`true` to read from stderr.

It is not possible to read from stdout and stderr both in
one call. Set either @a opts.stdout or @a opts.stderr.

Raise an error on incorrect parameters or when the fiber is
cancelled:

- IllegalParams:    incorrect type or value of a parameter.
- IllegalParams:    called on a closed handle.
- IllegalParams:    opts.stdout and opts.stderr are set both
- IllegalParams:    a requested IO operation is not supported
                    by the handle (stdout / stderr is not
                    piped).
- IllegalParams:    attempt to operate on a closed file
                    descriptor.
- FiberIsCancelled: cancelled by an outside code.

Return a string on success, an empty string at EOF.

Return `nil, err` on a failure. Possible reasons:

- SocketError: an IO error occurs at read().
- TimedOut:    @a timeout quota is exceeded.
- OutOfMemory: no memory space for a buffer to read into.
- LuajitError: ("not enough memory"): no memory space for
               the Lua string.

`popen_handle:write(str[, opts]) -> str, err`
---------------------------------------------

Write data to a child peer.

@param handle        a handle of a child process
@param str           a string to write
@param opts          table of options
@param opts.timeout  time quota in seconds
                     (default: 100 years)

Write string @a str to stdin stream of a child process.

The function may yield forever if a child process does
not read data from stdin and a pipe buffer becomes full.
Size of this buffer depends on a platform. Use
@a opts.timeout when unsure.

When @a opts.timeout is not set, the function blocks
(yields the fiber) until all data is written or an error
happened.

Raise an error on incorrect parameters or when the fiber is
cancelled:

- IllegalParams:    incorrect type or value of a parameter.
- IllegalParams:    called on a closed handle.
- IllegalParams:    string length is greater then SSIZE_MAX.
- IllegalParams:    a requested IO operation is not supported
                    by the handle (stdin is not piped).
- IllegalParams:    attempt to operate on a closed file
                    descriptor.
- FiberIsCancelled: cancelled by an outside code.

Return `true` on success.

Return `nil, err` on a failure. Possible reasons:

- SocketError: an IO error occurs at write().
- TimedOut:    @a timeout quota is exceeded.

`popen_handle:shutdown(opts) -> true`
------------------------------------------

Close parent's ends of std* fds.

@param handle        handle of a child process
@param opts          an options table
@param opts.stdin    close parent's end of stdin, boolean
@param opts.stdout   close parent's end of stdout, boolean
@param opts.stderr   close parent's end of stderr, boolean

The main reason to use this function is to send EOF to
child's stdin. However parent's end of stdout / stderr
may be closed too.

The function does not fail on already closed fds (idempotence).
However it fails on attempt to close the end of a pipe that was
never exist. In other words, only those std* options that
were set to popen.opts.PIPE at a handle creation may be used
here (for popen.shell: 'r' corresponds to stdout, 'R' to stderr
and 'w' to stdin).

The function does not close any fds on a failure: either all
requested fds are closed or neither of them.

Example:

 | local popen = require('popen')
 |
 | local ph = popen.shell('sed s/foo/bar/', 'rw')
 | ph:write('lorem foo ipsum')
 | ph:shutdown({stdin = true})
 | local res = ph:read()
 | ph:close()
 | print(res) -- lorem bar ipsum

Raise an error on incorrect parameters:

- IllegalParams:  an incorrect handle parameter.
- IllegalParams:  called on a closed handle.
- IllegalParams:  neither stdin, stdout nor stderr is choosen.
- IllegalParams:  a requested IO operation is not supported
                  by the handle (one of std* is not piped).

Return `true` on success.

`popen_handle:terminate() -> ok, err`
-------------------------------------

Send SIGTERM signal to a child process.

@param handle  a handle carries child process to terminate

The function only sends SIGTERM signal and does NOT
free any resources (popen handle memory and file
descriptors).

@see popen_handle:signal() for errors and return values.

`popen_handle:kill() -> ok, err`
--------------------------------

Send SIGKILL signal to a child process.

@param handle  a handle carries child process to kill

The function only sends SIGKILL signal and does NOT
free any resources (popen handle memory and file
descriptors).

@see popen_handle:signal() for errors and return values.

`popen_handle:signal(signo) -> ok, err`
---------------------------------------

Send signal to a child process.

@param handle  a handle carries child process to be signaled
@param signo   signal number to send

When opts.setsid and opts.group_signal are set on the handle
the signal is sent to the process group rather than to the
process. @see popen.new() for details about group
signaling.

Note: The module offers popen.signal.SIG* constants, because
some signals have different numbers on different platforms.

Raise an error on incorrect parameters:

- IllegalParams:    an incorrect handle parameter.
- IllegalParams:    called on a closed handle.

Return `true` if signal is sent.

Return `nil, err` on a failure. Possible reasons:

- SystemError: a process does not exists anymore

               Aside of a non-exist process it is also
               returned for a zombie process or when all
               processes in a group are zombies (but
               see note re Mac OS below).

- SystemError: invalid signal number

- SystemError: no permission to send a signal to
               a process or a process group

               It is returned on Mac OS when a signal is
               sent to a process group, where a group leader
               is zombie (or when all processes in it
               are zombies, don't sure).

               Whether it may appear due to other
               reasons is unclear.

`popen_handle:info() -> res`
----------------------------

Return information about popen handle.

@param handle  a handle of a child process

Raise an error on incorrect parameters:

- IllegalParams: an incorrect handle parameter.
- IllegalParams: called on a closed handle.

Return information about the handle in the following
format:

    {
        pid = <number> or <nil>,
        command = <string>,
        opts = <table>,
        status = <table>,
        stdin = one-of(
            popen.stream.OPEN   (== 'open'),
            popen.stream.CLOSED (== 'closed'),
            nil,
        ),
        stdout = one-of(
            popen.stream.OPEN   (== 'open'),
            popen.stream.CLOSED (== 'closed'),
            nil,
        ),
        stderr = one-of(
            popen.stream.OPEN   (== 'open'),
            popen.stream.CLOSED (== 'closed'),
            nil,
        ),
    }

`pid` is a process id of the process when it is alive,
otherwise `pid` is nil.

`command` is a concatenation of space separated arguments
that were passed to execve(). Multiword arguments are quoted.
Quotes inside arguments are not escaped.

`opts` is a table of handle options in the format of
popen.new() `opts` parameter. `opts.env` is not shown here,
because the environment variables map is not stored in a
handle.

`status` is a table that represents a process status in the
following format:

    {
        state = one-of(
            popen.state.ALIVE    (== 'alive'),
            popen.state.EXITED   (== 'exited'),
            popen.state.SIGNALED (== 'signaled'),
        )

        -- Present when `state` is 'exited'.
        exit_code = <number>,

        -- Present when `state` is 'signaled'.
        signo = <number>,
        signame = <string>,
    }

`stdin`, `stdout`, `stderr` reflect status of parent's end
of a piped stream. When a stream is not piped the field is
not present (`nil`). When it is piped, the status may be
one of the following:

- popen.stream.OPEN    (== 'open')
- popen.stream.CLOSED  (== 'closed')

The status may be changed from 'open' to 'closed'
by :shutdown({std... = true}) call.

Example 1 (tarantool console):

 | tarantool> require('popen').new({'/usr/bin/touch', '/tmp/foo'})
 | ---
 | - command: /usr/bin/touch /tmp/foo
 |   status:
 |     state: alive
 |   opts:
 |     stdout: inherit
 |     stdin: inherit
 |     group_signal: false
 |     keep_child: false
 |     close_fds: true
 |     restore_signals: true
 |     shell: false
 |     setsid: false
 |     stderr: inherit
 |   pid: 9499
 | ...

Example 2 (tarantool console):

 | tarantool> require('popen').shell('grep foo', 'wrR')
 | ---
 | - stdout: open
 |   command: sh -c 'grep foo'
 |   stderr: open
 |   status:
 |     state: alive
 |   stdin: open
 |   opts:
 |     stdout: pipe
 |     stdin: pipe
 |     group_signal: true
 |     keep_child: false
 |     close_fds: true
 |     restore_signals: true
 |     shell: true
 |     setsid: true
 |     stderr: pipe
 |   pid: 10497
 | ...

`popen_handle:wait() -> res`
----------------------------

Wait until a child process get exited or signaled.

@param handle  a handle of process to wait

Raise an error on incorrect parameters or when the fiber is
cancelled:

- IllegalParams:    an incorrect handle parameter.
- IllegalParams:    called on a closed handle.
- FiberIsCancelled: cancelled by an outside code.

Return a process status table (the same as ph.status and
ph.info().status). @see popen_handle:info() for the format
of the table.

`popen_handle:close() -> ok, err`
---------------------------------

Close a popen handle.

@param handle  a handle to close

Basically it kills a process using SIGKILL and releases all
resources assosiated with the popen handle.

Details about signaling:

- The signal is sent only when opts.keep_child is not set.
- The signal is sent only when a process is alive according
  to the information available on current even loop iteration.
  (There is a gap here: a zombie may be signaled; it is
  harmless.)
- The signal is sent to a process or a grocess group depending
  of opts.group_signal. (@see lbox_popen_new() for details of
  group signaling).

Resources are released disregarding of whether a signal
sending succeeds: fds are closed, memory is released,
the handle is marked as closed.

No operation is possible on a closed handle except
:close(), which always successful on closed handle
(idempotence).

Raise an error on incorrect parameters:

- IllegalParams: an incorrect handle parameter.

The function may return `true` or `nil, err`, but it always
frees the handle resources. So any return value usually
means success for a caller. The return values are purely
informational: it is for logging or same kind of reporting.

Possible diagnostics (don't consider them as errors):

- SystemError: no permission to send a signal to
               a process or a process group

               This diagnostics may appear due to
               Mac OS behaviour on zombies when
               opts.group_signal is set,
               @see lbox_popen_signal().

               Whether it may appear due to other
               reasons is unclear.

Always return `true` when a process is known as dead (say,
after ph:wait()): no signal will be send, so no 'failure'
may appear.

Handle fields
=============

- popen_handle.pid
- popen_handle.command
- popen_handle.opts
- popen_handle.status
- popen_handle.stdin
- popen_handle.stdout
- popen_handle.stderr

See popen_handle:info() for description of those fields.

Module constants
================

- popen.opts
  - INHERIT (== 'inherit')
  - DEVNULL (== 'devnull')
  - CLOSE   (== 'close')
  - PIPE    (== 'pipe')

- popen.signal
  - SIGTERM (== 9)
  - SIGKILL (== 15)
  - ...

- popen.state
  - ALIVE    (== 'alive')
  - EXITED   (== 'exited')
  - SIGNALED (== 'signaled')

- popen.stream
  - OPEN    (== 'open')
  - CLOSED  (== 'closed')
```

(cherry picked from commit 34c2789b48eadd36da742f7f761a998889e70544)

The function still set a diagnostics when a signal sending fails and
returns -1, but it is purely informational result: for logging or so. It
reflects notes about dealing with failures in Linux's `man 2 close`:

 | Note, however, that a failure return should be used only for
 | diagnostic purposes <...> or remedial purposes <...>.
 |
 | <...> Linux  kernel always releases the file descriptor early in the
 | close operation, freeing it for reuse; the steps that may return an
 | error <...> occur only later in the close operation.
 |
 | Many other implementations similarly always close the file descriptor
 | <...> even if they subsequently report an error on return from
 | close(). POSIX.1 is currently silent on this point, but there are
 | plans to mandate this behavior in the next major release of the
 | standard.

When kill or killpg returns EPERM a caller usually unable to overcome it
somehow: retrying is not sufficient here. So there are no needs to keep
the handle: a caller refuses the handle and don't want to perform any
other operation on it.  The open engine do its best to kill a child
process or a process group, but when it is not possible, just set the a
diagnostic and free handle resources anyway.

Left comments about observed Mac OS behaviour regarding killing a
process group, where all processes are zombies (or just when a process
group leader is zombie, don't sure): it gives EPERM instead of ESRCH
from killpg(). This result should not surprise a user, so it should be
documented. See [1] for another description of the problem (I don't find
any official information about this).

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1329528



Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>
(cherry picked from commit 56a8c346ecb0581300a63c6e677d8a4672ff1f95)

After 2.2.0-633-gaa0964ae1 ('net.box: fix schema fetching from 1.10/2.1
servers') net.box expects that _vcollation system view exists on a
tarantool server of 2.2.1+ version. This is however not always so: a
server may be run on a new version of tarantool, but work on a schema of
an old version.

The situation with non last schema is usual for replication cluster in
process of upgrading: all instances run on the new version of tarantool
first (no auto-upgrade is performed by tarantools in a cluster). Then
box.schema.upgrade() should be called, but the instances should be
operable even before the call.

Before the commit net.box was unable to connect a server if it is run on
a schema without _vcollation system view (say, 2.1.3), but the server
executable is of 2.2.1 version or newer.

Note: I trim tests from the commit to polish them a bit more, but
include the fix itself to 2.4.1 release.

Follows up #4307
Fixes #4691

This changeset makes possible to run luajit-tap tests requiring
libraries implemented in C:
* symlink to luajit test is created on configuration phase instead of
  build one.
* introduced a CMake function for building shared libraries required for
  luajit tests.

Furthermore this commit enables CMake build for the following luajit-tap
tests:
* gh-4427-ffi-sandwich
* lj-flush-on-trace

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Reviewed-by: Sergey Ostanevich <sergos@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

- jit: abort trace execution on JIT mode change
- jit: abort trace recording and execution for C API

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

Found that some package builds failed on the mistake in CMakeLists.txt
file, the failed packages and test builds were:
- CentOS 6
- CentOS 7
- Ubuntu 14.04
and static build based on Dockerfile.

The core of the issue is that CMake 2 does not support line continuation
with backslash.

The commit fixes the regression from
7b443650 ('feedback: add cmake option to
disable the daemon').

Follow up #3308

There is a complaint that the feedback daemon is a 'spying' tool
and because of that can't be used on Gentoo. Its default disabled
option also is not acceptable, the daemon should be eliminated
completely.

The patch introduces cmake option ENABLE_FEEDBACK_DAEMON. It is
ON by default. When set to OFF, all feedback daemon's code is not
included into the binary, its configuration options disappear.

Closes #3308

box.cfg() works in two stages, when called first time - boot the
instance using box_cfg() C++ function, and then configure it.
During booting all non-dynamic parameters are read. Dynamic are
configured mostly afterwards.

Normally there should be a yield between box_cfg() C++ call and
dynamic parameters configuration. It is used by box.ctl.wait_ro()
and box.ctl.wait_rw() Lua calls to catch the instance in read-only
state always before read-write state.

In theory a user should be able to call box.ctl.wait_ro() and
box.ctl.wait_rw() in one fiber, box.cfg() in another, and these
waits would be unblocked one after another.

It works fine now, but only because of, surprisingly, the feedback
daemon. The daemon creates a yield after C++ box_cfg() is
finished, but dynamic parameters are still being applied in
load_cfg.lua. That gives time to catch box.ctl.wait_ro() event.

The thing is that dynamic parameters configuration includes the
daemon's options too. When 'feedback_enable' option is installed
to true, the daemon is started using fiber.create(). That creates
a yield, and gives time to box.ctl.wait_ro() fibers to handle the
event.

When the daemon is disabled or removed, like it is going to happen
in #3308, this trick does not work, and box.ctl.wait_ro() started
before box.cfg() is never triggered.

It could be tested on app-tap/cfg.test.lua with

    box.cfg{}

changed to

    box.cfg{feedback_enabled = false}

Then the test would hang. A test is not patched here, because the
feedback is going to be optionally removed in a next commit, and
the test would become flaky depending on build options.

Needed for #3308

Feedback daemon's code was located in two files:
box/lua/feedback_daemon.lua and box/lua/schema.lua. That makes
it harder to eliminate the daemon at cmake configuration time.

Now all its code is in one place, in feedback_daemon.lua. Disable
of the daemon's code now is a matter of excluding the Lua file
from source code.

Part of #3308

Box built-in modules, such as session, tuple, schema, etc, were
loaded using luaL_loadbuffer() + lua_call(). Error of the former
call was handled properly with a panic message describing the
problem.

But if lua_call() failed, it resulted into 'unknown exception' in
main.cc. Not very helpful. Now it is lua_pcall(), and the error
message is included into the panic() message. That helps in debug,
when something is being changed in the box modules.

On Linux x86_64 `ssize_t` is 64 bit, while `int` is 32 bit wide (at
least typically). Let's return `ssize_t` from popen_write_timeout() to
prevent data loss.

Part of #4031

Reported-by: Cyrill Gorcunov <gorcunov@gmail.com>
Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

It was easy to misinterpret popen_may_io() contract. In fact, I made
this mistake recently and want to clarify how the function should be
called.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

The function popen_shutdown() checks whether std{in,out,err} was piped
and closes the parent's end. A user should have ability to send EOF for
child's stdin for stream programs like `grep`. It is better when there
is a function that encapsulates proper checks, error messages and the
actual actions.

This commit in particular reverts
1ef95b99 ('popen: remove redundant fd
check before perform IO'), because now the check is meaningful: an fd
may become closed before the whole popen handle will be deleted.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

Popen backend errors should be meaningful for a user of the popen Lua
API, because otherwise we'll need to map backend errors into Lua API
errors. Those particular failures can't appear when the functions are
called from the Lua API, but it is good to keep all error messages in
one style.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

Even when ..._SETSID and ..._GROUP_SIGNAL are set, we unable to safely
kill a process group after the child process we spawned becomes died. So
we don't do that.

The behaviour seems to be indefeasible part of Unix process group
design. The best that we can do here is describe those details in the
documentation comment.

NB: It seems that pid namespaces allow to overcome this problem, however
it is the Linux specific feature, so we unlikely will use them.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

It is convenient to have a formal description of an API during
development and when writing a documentation. I plan to use those
contracts when I will write an API description for the future popen Lua
module.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

fcntl(2) lists flags that can be set using F_SETFL: O_CLOEXEC is not
included there. F_SETFD should be used to set close-on-exec.

Parent's end of pipes are closed explicitly in a child process anyway.
However this change fixes closing of the copy of a logger fd. See commit
07a07b3c ('popen: decouple logger fd
from stderr') for more information why this file descriptor was
introduced.

Part of #4031.

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

For debugging purposes.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

Of course it is still not fair shell-style quoting: at least we should
also escape quotes inside arguments. But it gives correct output for
most of typical commands and has straightforward implementation.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

The change 'popen: require popen handle to be non-NULL' makes
popen_stat() function always successful. There is no reason to return a
success / failure result.

See the previous similar patch: 'popen: remove retval from
popen_state()'.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

See the previous similar commits:

* popen: add missed diag_set() in popen IO functions
* popen: add missed diag_set in popen_signal/delete

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

This information may be useful for debuggging.

Part of #4031

Acked-by: Cyrill Gorcunov <gorcunov@gmail.com>

Currently popen_delete kills all children process.
Moreover we use popen_delete on tarantool exit.

Alexander pointed out that keep children running
even if tarantool is exited is still needed.

Part of #4031

Reported-by: Alexander Turenko <alexander.turenko@tarantool.org>
Acked-by: Alexander Turenko <alexander.turenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

As Alexander pointed out this might be useful
for running a pipe of programs inside shell
(i.e. popen.shell('foo | bar | baz', 'r')).

Part of #4031

Reported-by: Alexander Turenko <alexander.turenko@tarantool.org>
Acked-by: Alexander Turenko <alexander.turenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Before this patch, if an ephemeral space was used during INSERT or
REPLACE, the inserted values were sorted by the first column,
since this was the first part of the index. This can lead to an
error when using the AUTOINCREMENT feature, since changing the
order of the inserted value can change the value inserted instead
of NULL. To avoid this, the patch makes the rowid of the inserted
row in the ephemeral space the only part of the ephemeral space
index.

Closes #4256

This patch specifies field types in ephemeral space format in SQL.
Prior to this patch, all fields had a SCALAR field type.

This patch allows us to not use the primary index to obtain field
types, since now the ephemeral space has field types in the
format. This allows us to change the structure of the primary
index, which helps to solve the issue #4256. In addition, since we
can now set the field types of the ephemeral space, we can use
this feature to set the field types according to the left value of
the IN operator. This will fix issue #4692.

Needed for #4256
Needed for #4692
Closes #3841

This patch allows to set field types and names in ephemeral space
formats.

Needed for #4256
Needed for #4692
Part of #3841

relay_schedule_pending_gc() is executed after relay status update,
which made perfect sense before we've introduced local spaces rework, making
local space operations use a special instance id: 0.
Relay status update is performed only when the remote instance has
reported a bigger vclock, than its previous one. However, we may have an
entire WAL file filled with local space changes, in which case the
changes won't be transmitted to replica, and it will report the same
vclock as before, postponing the scheduled gc until a non-local row is
created on master.

Fix this by reordering relay_schedule_pending_gc() and relay status
update. In case nothing new is added to pending_gc queue and replica
clock is not updated, relay_schedule_pending_gc() will exit on the first
loop iteration, so it doesn't add an overhead.

Also make relay_schedule_pending_gc() use vclock_compare_ignore0() instead
of plain vclock_compare().

Follow-up #4114

Prior to this patch, due to the absence of the '\0' character at
the end of the BLOB, it was possible to get an error or incorrect
result when using CAST() from BLOB to INTEGER or UNSIGNED. This
has now been fixed, but the maximum length of a BLOB that could be
cast to INTEGER or UNSIGNED was limited to 12287 bytes.

Examples of wrong CAST() from BLOB to INTEGER:

CREATE TABLE t (i INT PRIMARY KEY, a VARBINARY, b INT, c INT);
INSERT INTO t VALUES (1, X'33', 0x33, 0x00), (2, X'34', 0x41, 0);

Example of wrong result:

SELECT CAST(a AS INTEGER) FROM t WHERE i = 1;

Result: 33

Example of error during CAST():

SELECT CAST(a AS INTEGER) FROM t WHERE i = 2;

Result: 'Type mismatch: can not convert varbinary to integer'

Closes #4766

Prior to this patch, STRING, which contains the DOUBLE value,
could be implicitly cast to INTEGER. This was done by converting
STRING to DOUBLE and then converting this DOUBLE value to INTEGER.
This may affect the accuracy of CAST(), so it was forbidden. It
is worth noting that these changes will not affect the comparison,
since the implicit cast in this case has different mechanics.

Example:
box.execute("CREATE TABLE t(i INT PRIMARY KEY);")

Before patch:
box.execute("INSERT INTO t VALUES ('111.1');")
box.execute("SELECT * FROM t;")
Result: 111

After patch:
box.execute("INSERT INTO t VALUES ('1.1');")
Result: 'Type mismatch: can not convert 1.1 to integer'

box.execute("INSERT INTO t VALUES ('1.0');")
Result: 'Type mismatch: can not convert 1.0 to integer'

box.execute("INSERT INTO t VALUES ('1.');")
Result: 'Type mismatch: can not convert 1. to integer'

@TarantoolBot document
Title: disallow cast from STRING contains DOUBLE to INTEGER

After the last two patches, explicit and implicit casting from the
string containing DOUBLE to INTEGER directly will be prohibited.
The user must use the explicit cast to DOUBLE before the explicit
or implicit cast to INTEGER. The reason for this is that before
these patches, such STRINGs were implicitly cast to DOUBLE, and
then this DOUBLE was implicitly or explicitly cast to INTEGER.
Because of this, the result of such a cast may differ from what
the user expects, and the user may not know why.

It is worth noting that these changes will not affect the
comparison, since the implicit cast in this case has different
mechanics.

Example for implicit cast:

box.execute("CREATE TABLE t(i INT PRIMARY KEY);")
-- Does not work anymore:
box.execute("INSERT INTO t VALUES ('1.1');")
-- Right way:
box.execute("INSERT INTO t VALUES (CAST('1.1' AS DOUBLE));")

Example for explicit cast:

-- Does not work anymore:
box.execute("SELECT CAST('1.1' AS INTEGER);")
-- Right way:
box.execute("SELECT CAST(CAST('1.1' AS DOUBLE) AS INTEGER);")

Prior to this patch, STRING, which contains the DOUBLE value,
could be cast to INTEGER. This was done by converting STRING to
DOUBLE and then converting this DOUBLE value to INTEGER. This may
affect the accuracy of CAST(), so it was forbidden.

Before patch:
box.execute("SELECT CAST('111.1' as INTEGER);")
Result: 111

After patch:
box.execute("SELECT CAST('1.1' as INTEGER);")
Result: 'Type mismatch: can not convert 1.1 to integer'

box.execute("SELECT CAST('1.0' as INTEGER);")
Result: 'Type mismatch: can not convert 1.0 to integer'

box.execute("SELECT CAST('1.' as INTEGER);")
Result: 'Type mismatch: can not convert 1. to integer'

Moved sources tarball creation from travis-ci to gitlab-ci,
moved its jobs for sources packing and sources deploying.

Close #4895