box_check_config() didn't check memtx_memory and vinyl_memory
upper bound. As a result, it was possible to set memory size
higher than what the quota allows as maximum.

That worked only when box.cfg() was called first time, because
quota_init() does not check its value. Subsequent box.cfg() calls
use quota_set(), which aborts the program if a size is too big.
Only in debug mode. In release quota_set() also worked with any
sizes.

Closes #4705

Reviewed-by: Igor Munkin <imun@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

It was calling box_check_sql_cache_size() assuming that it
throws. But it returns 0/-1.

Reviewed-by: Igor Munkin <imun@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

Use bit_count_u32() instead of plain __builtin_popcount() for filter
size, just as we do for vclock size now.

We're using an unsigned int to hold vclock map, but there is no
guarantee that unsigned int will be 4 bytes in size to fit all the 32
vclock components. So use uint32_t instead and add an alias to it
vclock_map_t.

We need to set a thread cancellation guard, because
another thread may cancel the current thread at a
really bad time (messages flush, mutex lock)

Fixes: #4127

Users keep complaining about too short fiber name. New limit is
255, should be enough for any sane name.

Closes #4394

Reviewed-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

@TarantoolBot document
Title: fiber.name length limit.

It was 32, now it is 255. Besides, it seems like `fiber.name`
`{truncate = true}` option is not documented.

By default, if a new name is too long, `fiber.name(new_name)`
fails with an exception. To make it always succeed there is an
option 'truncate': `fiber.name(new_name, {truncate = true})`. It
truncates the name to the max length if it is too long.

Box.cfg{listen = 0} automatically chooses a port. But it was
impossible to obtain a real port the instance is bound to.

An ability to see a real port may help to make test-run more
robust, because it won't depend on which ports are free, and
won't need to pre-choose them in advance.

Now box.info.listen shows a real address, or nil when listen is
turned off. Also a real address is logged instead of the dummy
0-port one.

Closes #4620

@TarantoolBot document
Title: box.info.listen - real address

New value in box.info - listen. It is a real address to which the
instance was bound. For example, if box.cfg.listen was set with
a zero port, box.info.listen will show a real port. The address
is stored as a string:

    - unix/:<path> for UNIX domain sockets;
    - <ip>:<port> for IPv4;
    - [ip]:<port> for IPv6.

If the instance does not listen anything, box.info.listen is nil.

This patch introduces "current" function for sequences.
It returns the last retrieved value of specified sequence or
throws an error if no value has been generated yet.

This patch partially reverts 3ff1f1e3
(box: remove sequence_get) here similar function "get" was removed
to avoid possible misleading with "currval" function of PosgreSQL
that returns the last obtained value of the sequence in the scope
of current session. In contrast "current" returns the last globally
retrieved value of the sequence.

Closes #4752

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

@TarantoolBot document
Title: sequence:current()

This patch introduces "current" function for sequences.
It returns the last retrieved value of specified sequence or
throws an error if no value has been generated yet ("next"
has not been called yet or right after "reset" is called).

Lua:

Example:

```lua
sq = box.schema.sequence.create('test')
---
...
sq:current()
---
- error: Sequence 'test' is not started
...
sq:next()
---
- 1
...
sq:current()
---
- 1
...
sq:set(42)
---
...
sq:current()
---
- 42
...
sq:reset()
---
...
sq:current()  -- error
---
- error: Sequence 'test' is not started
...
```

C API:

```C
int
box_sequence_current(uint32_t seq_id, int64_t *result);
```

Where:
  * seq_id - sequence identifier;
  * result - pointer to a variable where the current sequence
  value will be stored on success.

Returns 0 on success and -1 otherwise. In case of an error user
could get it via `box_error_last()`.

Absence of the body in the unprepare response forces users to perform
additional checks to avoid errors. Adding an empty body fixes this problem.

Closes #4769

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

In #4684 it was found that box.tuple.* contained some private
functions: bless(), encode(), and is().

Bless() and encode() didn't make any sense for a user, so they
were hidden into box.internal.tuple.*.

But box.tuple.is() is actually a useful thing. It is harnessed in
the tests a lot, and is likely to be already used by customers,
because it is available in box.tuple.* for a long time. It is a
matter of time when someone will open a doc ticket saying that
box.tuple.is() is not documented. The patch makes it legally
public.

Follow-up #4684

@TarantoolBot document
Title: box.tuple.is()
```Lua
box.tuple.is(object)
```
A function to check whether a given object is a tuple cdata
object. Returns true or false. Never raises nor returns an error.

box.tuple.bless, .encode, and .is are internal. Their
behaviour is not documented, and they may omit some checks for the
sake of speed, and can crash if used without thinking.

Nonetheless, despite they are not documented, curious users could
notice them in box.tuple.* output via autocompletion, for example.
And they could try to use them. This is not ok.

box.tuple.bless() being called by a user leads either to a crash,
or to a leak (because it is basically tuple reference counter
increment).

box.tuple.encode() is kind of a wrapper around msgpack, and users
should not touch it. It may change, may be removed. And is just
makes no sense except some rare cases in schema.lua.

bless() and encode() were used in schema.lua only, so the patch
simply moves them to box.internal.tuple.

box.tuple.is() is kept as is, because
- this is used in the tests a lot;
- it is totally safe;
- that function actually makes sense, and some users could have
  already started using it.

There is no a test, since nothing to test - bless() is not
available for users anymore (assuming no one is brave enough to
rely on box.internal).

Closes #4684

Due to os specifics we can't call setsid after vfork on macos
(vfork is not longer a part of posix btw). Instead we can use
ioctl to clear the session, then initiate a new process group.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

This helps to identify if something gone
wrong inside a child process.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

On error path we arm diag with error but strictly
speaking some users (such as unit tests) do not have
to access diag for logging.

Thus log it explicitly for debug sake.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

This is part of posix standart.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

We need it inside popen engine to be able to print
errors from inside of a child process.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

This code is called from C, so it shouldn't throw.

Closes #4753

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

CREATE TABLE used to check column name duplicates before going to
box. But it is not necessary, because the same check is done by
box.

Reviewed-by: Nikita Pettik <korablev@tarantool.org>

Original port_tuple in C has 'int size;' field. It was
'size_t size' in Lua. Since sizeof(size_t) usually is
8, and sizeof(int) is 4, this was a really bad typo.

When tarantool tries to recover rtree from a snapshot and memtx_memory value
is lower than it has been when the snapshot was created, server suffers from
segmentation fault. This happens because there is no out of memory error
handling in rtree lib. In another words, we do not check the result of
malloc operation.
The execution flow in case of recovery uses different way and the secondary
keys are build in batches. That way has no checks and reservations.
The patch adds memtx_rtree_index_reserve implementation to make sure that any
memory allocation in rtree will fail. Although this gives us no additional
optimization as in case of memtx_tree, the memory reservation prevents
tarantool from segmentation fault. If there is not enough memory to be reserved
server will fail gently with the "Failed to allocate" error message.

Closes #4619

Despite what was stated in the documentation, netbox.connect was not always
equivalent to netbox.self. In particular, they converted tuple to different
types - table and cdata respectively.

The patch fixes the issue and covers all cases where netbox.self and connect
perform conversion of types - e.g., for box.error.

Closes #4513

The patch fixes a bug for the commit b0f588f6 where statistics on
box.execute was collected twice. This happened because
sql_prepare_and_execute called sql_execute under the hood, so there's
no need to do rmean_collect in both of them.

Follow-up #4756

All lua types feature check, push and is functions. We expose lua_checktuple
for full consistency.

Closes #2553

TMPDIR is an environment variable used to tell what a directory
should be used to create temporary files. It is described in the
POSIX standard, and should be used by programs creating temporary
files.

Closes #4794

@TarantoolBot document
Title: fio.tempdir() $TMPDIR

fio.tempdir() stores created temporary directory into /tmp by
default. This can be changed by setting TMPDIR environment
variable. Before starting Tarantool, or at runtime by
os.setenv().

libcurl has a built-in threaded resolver used for asynchronous DNS
requests, however, when DNS server is slow to respond, the request still
hangs tarantool until it is finished. The reason is that curl calls
thread_join on the resolving thread internally upon timeout, making the
calling thread hang until resolution has ended.
Use c-ares as an asynchronous resolver instead to eliminate the problem.

Closes #4591

It was possible to leak user password through setting 'replication'
configuration option in first box.cfg invocation. This happened due
to unconditional logging in load_cfg function. The patch introduces
conditional logging.

Closes #4493

Extend <ALTER TABLE> statement to drop table constraints by their
names.

Closes #4120

@TarantoolBot document
Title: Drop table constraints in SQL
Now, it is possible to drop table constraints (PRIMARY KEY,
UNIQUE, FOREIGN KEY, CHECK) using
<ALTER TABLE table_name DROP CONSTRAINT constraint_name> statement
by their names.

For example:

tarantool> box.execute([[CREATE TABLE test (
                             a INTEGER PRIMARY KEY,
                             b INTEGER,
                             CONSTRAINT cnstr CHECK (a >= 0)
                        );]])
---
- row_count: 1
...

tarantool> box.execute('ALTER TABLE test DROP CONSTRAINT cnstr;')
---
- row_count: 1
...

The same for all the other constraints.

Remove function box_index_by_name() from parser to avoid selects
during parsing. Add the ability to choose index during VDBE code
compilation which will be used to find the tuple to drop from a
system space.

Needed for #4120

Clarify the error message for better user handling. Add the name
of space where the constraint under dropping wasn't founded.

Part of #4120

When checking wheter rejoin is needed, replica loops through all the
instances in box.cfg.replication, which makes it believe that there is a
master holding files, needed by it, since it accounts itself just like
all other instances.
So make replica skip itself when finding an instance which holds files
needed by it, and determining whether rebootstrap is needed.

We already have a working test for the issue, it missed the issue due to
replica.lua replication settings. Fix replica.lua to optionally include
itself in box.cfg.replication, so that the corresponding test works
correctly.

Closes #4759

We have a mechanism for restoring rows originating from an instance that
suffered a sudden power loss: remote masters resend the isntance's rows
received before a certain point in time, defined by remote master vclock
at the moment of subscribe.
However, this is useful only on initial replication configuraiton, when
an instance has just recovered, so that it can receive what it has
relayed but haven't synced to disk.
In other cases, when an instance is operating normally and master-master
replication is configured, the mechanism described above may lead to
instance re-applying instance's own rows, coming from a master it has just
subscribed to.
To fix the problem do not relay rows coming from a remote instance, if
the instance has already recovered.

Closes #4739

Add a filter for relay to skip rows coming from unwanted instances.
A list of instance ids whose rows replica doesn't want to fetch is encoded
together with SUBSCRIBE request after a freshly introduced flag IPROTO_ID_FILTER.

Filtering rows is needed to prevent an instance from fetching its own
rows from a remote master, which is useful on initial configuration and
harmful on resubscribe.

Prerequisite #4739, #3294

@TarantoolBot document

Title: document new binary protocol key and subscribe request changes

Add key `IPROTO_ID_FILTER = 0x51` to the internals reference.
This is an optional key used in SUBSCRIBE request followed by an array
of ids of instances whose rows won't be relayed to the replica.

SUBSCRIBE request is supplemented with an optional field of the
following structure:
```
+====================+
|      ID_FILTER     |
|   0x51 : ID LIST   |
| MP_INT : MP_ARRRAY |
|                    |
+====================+
```
The field is encoded only when the id list is not empty.

There is an assertion in vclock_follow `lsn > prev_lsn`, which doesn't
fire in release builds, of course. Let's at least warn the user on an
attempt to write a record with a duplicate or otherwise broken lsn, and
not follow such an lsn.

Follow-up #4739

is_orphan status check is needed by applier in order to tell relay
whether to send the instance's own rows back or not.

Prerequisite #4739

Fix for commit f58cb606 ('popen:
introduce a backend engine').

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>

In the patch we introduce popen backend engine which provides
a way to execute external programs and communicate with their
stdin/stdout/stderr streams.

It is possible to run a child process with:

 a) completely closed stdX descriptors
 b) provide /dev/null descriptors to appropritate stdX
 c) pass new transport into a child (currently we use
    pipes for this sake, but may extend to tty/sockets)
 d) inherit stdX from a parent, iow do nothing

On tarantool start we create @popen_pids_map hash which maps
created processes PIDs to popen_handle structure, this structure
keeps everything needed to control and communicate with the children.
The hash will allow us to find a hild process quickly from inside
of a signal handler.

Each handle links into @popen_head list, which is need to be able
to destory children processes on exit procedure (ie when we exit
tarantool and need to cleanup the resources used).

Every new process is born by vfork() call - we can't use fork()
because of at_fork() handlers in libeio which cause deadlocking
in internal mutex usage. Thus the caller waits until vfork()
finishes its work and runs exec (or exit with error).

Because children processes are running without any limitations
they can exit by self or can be killed by some other third side
(say user of a hw node), we need to watch their state which is
done by setting a hook with ev_child_start() helper. This helper
allows us to catch SIGCHLD when a child get exited/signaled
and unregister it from a pool or currently running children.
Note the libev wait() reaps child zomby by self. Another
interesting detail is that libev catches signal in async way
but our SIGCHLD hook is called in sync way before child reap.

This engine provides the following API:
 - popen_init
	to initialize engine
 - popen_free
	to finalize engine and free all reasources
	allocated so far
 - popen_new
	to create a new child process and start it
 - popen_delete
	to release resources occupied and
	terminate a child process
 - popen_stat
	to fetch statistics about a child process
 - popen_command
	to fetch command line string formerly used
	on the popen object creation
 - popen_write_timeout
	to write data into child's stdin with
	timeout
 - popen_read_timeout
	to read data from child's stdout/stderr
	with timeout
 - popen_state
	to fetch state (alive, exited or killed) and
	exit code of a child process
 - popen_state_str
	to get state of a child process in string
	form, for Lua usage mostly
 - popen_send_signal
	to send signal to a child process (for
	example to kill it)

Known issues to fix in next series:

 - environment variables for non-linux systems do not support
   inheritance for now due to lack of testing on my side;

 - for linux base systems we use popen2 system call passing
   O_CLOEXEC flag so that two concurrent popen_create calls
   would not affect each other with pipes inheritance (while
   currently we don't have a case where concurrent calls could
   be done as far as I know, still better to be on a safe side
   from the beginning);

 - there are some files (such as xlog) which tarantool opens
   for own needs without setting O_CLOEXEC flag and it get
   propagated to a children process; for linux based systems
   we use close_inherited_fds helper which walks over opened
   files of a process and close them. But for other targets
   like MachO or FreeBSD this helper just zapped simply because
   I don't have such machines to experimant with; we should
   investigate this moment in more details later once base
   code is merged in;

 - need to consider a case where we will be using piping for
   descriptors (for example we might be writting into stdin
   of a child from another pipe, for this sake we could use
   splice() syscall which gonna be a way faster than copying
   data inside kernel between process). Still the question
   is -- do we really need it? Since we use interanal flags
   in popen handle this should not be a big problem to extend
   this interfaces;

   this particular feature is considered to have a very low
   priority but I left it here just to not forget.

Part-of #4031

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

There is no reason to hide functions. In particular
we will use these helpers in popen code.

Part-of #4031

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Calling prepare and execute did not update corresponding request statistics
in the box.stat table. This happened because methods that collect stats were
never called where they should have been.

Closes #4756

Error message on granted privileges was not flexible and
did not distinguish between universal or any other privileges
leaving either 'nil' or simply '' at the end.

Closes #714