vy_build_insert_tuple() processes insertion into secondary indexes being
created. It contains yield points during which in-memory level of LSM
tree may change (for example rotate owing to triggered dump). So after
yield point it is required to fetch from LSM struct pointer to mem again
to operate on valid metadata. This patch updates pointer to mem after
mentioned yield point.

Closes #4810

Cleaned up the tests code according to the Lua style guide:
- made scoped variables local
- added os.exit call at the end of the test chunk
- adjusted the messages in test:ok calls

Currently the journal provides only one method -- write,
which implies a callback to trigger upon write completion
(in contrary with 1.10 series where all commits were
processing in synchronous way).

Lets make difference between sync and async writes more
notable: provide journal::write_async method which runs
completion function once entry is written, in turn
journal:write handle transaction in synchronous way.

Redesing notes:

1) The callback for async write set once in journal
   creation. There is no need to carry callback in
   every journal entry. This allows us to save some
   memory;

2) txn_commit and txn_commit_async call txn_rollback
   where appropriate;

3) no need to call journal_entry_complete on sync
   writes anymore;

4) wal_write_in_wal_mode_none is too long, renamed
   to wal_write_none;

5) wal engine use async writes internally but it is
   transparent to callers.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

This makes code easier to read and allows to reuse
txn allocation in sync\async writes.

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Otherwise we won't be able to make a rollback in case
of journal_entry_new allocation failure.

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

We gonna diverge sync and async code flow thus lets
make txn_commit to not use txn_commit_async.

Fixes #4776

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

We will use this function inside wal engine right
after journal redesign is complete.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

To reuse in sync trancastion once journal
redesign is complete.

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

For unification sake, we will handle nop transactions
via common helper for both sync and async cases.

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

To reflect the fact tha we're don't waiting for
transaction to complete but rely on completion
callback.

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

We already have a "get" function in this header which is
named in_txt(). Having both get/set in one place should
be more consistent.

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

There is no need for several instances of
recovery journal variable. Lets make it
statically allocated inside recovery_journal_create
routine.

And drop the inline annotation because there is
absolutely no need for this routine being inline.

Suggested-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Allows to eliminate code duplication.

Acked-by: Konstantin Osipov <kostja.osipov@gmail.com>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

box_check_config() didn't check memtx_memory and vinyl_memory
upper bound. As a result, it was possible to set memory size
higher than what the quota allows as maximum.

That worked only when box.cfg() was called first time, because
quota_init() does not check its value. Subsequent box.cfg() calls
use quota_set(), which aborts the program if a size is too big.
Only in debug mode. In release quota_set() also worked with any
sizes.

Closes #4705

Reviewed-by: Igor Munkin <imun@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

It was calling box_check_sql_cache_size() assuming that it
throws. But it returns 0/-1.

Reviewed-by: Igor Munkin <imun@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

Use bit_count_u32() instead of plain __builtin_popcount() for filter
size, just as we do for vclock size now.

We're using an unsigned int to hold vclock map, but there is no
guarantee that unsigned int will be 4 bytes in size to fit all the 32
vclock components. So use uint32_t instead and add an alias to it
vclock_map_t.

We need to set a thread cancellation guard, because
another thread may cancel the current thread at a
really bad time (messages flush, mutex lock)

Fixes: #4127

Users keep complaining about too short fiber name. New limit is
255, should be enough for any sane name.

Closes #4394

Reviewed-by: Cyrill Gorcunov <gorcunov@gmail.com>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

@TarantoolBot document
Title: fiber.name length limit.

It was 32, now it is 255. Besides, it seems like `fiber.name`
`{truncate = true}` option is not documented.

By default, if a new name is too long, `fiber.name(new_name)`
fails with an exception. To make it always succeed there is an
option 'truncate': `fiber.name(new_name, {truncate = true})`. It
truncates the name to the max length if it is too long.

Box.cfg{listen = 0} automatically chooses a port. But it was
impossible to obtain a real port the instance is bound to.

An ability to see a real port may help to make test-run more
robust, because it won't depend on which ports are free, and
won't need to pre-choose them in advance.

Now box.info.listen shows a real address, or nil when listen is
turned off. Also a real address is logged instead of the dummy
0-port one.

Closes #4620

@TarantoolBot document
Title: box.info.listen - real address

New value in box.info - listen. It is a real address to which the
instance was bound. For example, if box.cfg.listen was set with
a zero port, box.info.listen will show a real port. The address
is stored as a string:

    - unix/:<path> for UNIX domain sockets;
    - <ip>:<port> for IPv4;
    - [ip]:<port> for IPv6.

If the instance does not listen anything, box.info.listen is nil.

This patch introduces "current" function for sequences.
It returns the last retrieved value of specified sequence or
throws an error if no value has been generated yet.

This patch partially reverts 3ff1f1e3
(box: remove sequence_get) here similar function "get" was removed
to avoid possible misleading with "currval" function of PosgreSQL
that returns the last obtained value of the sequence in the scope
of current session. In contrast "current" returns the last globally
retrieved value of the sequence.

Closes #4752

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

@TarantoolBot document
Title: sequence:current()

This patch introduces "current" function for sequences.
It returns the last retrieved value of specified sequence or
throws an error if no value has been generated yet ("next"
has not been called yet or right after "reset" is called).

Lua:

Example:

```lua
sq = box.schema.sequence.create('test')
---
...
sq:current()
---
- error: Sequence 'test' is not started
...
sq:next()
---
- 1
...
sq:current()
---
- 1
...
sq:set(42)
---
...
sq:current()
---
- 42
...
sq:reset()
---
...
sq:current()  -- error
---
- error: Sequence 'test' is not started
...
```

C API:

```C
int
box_sequence_current(uint32_t seq_id, int64_t *result);
```

Where:
  * seq_id - sequence identifier;
  * result - pointer to a variable where the current sequence
  value will be stored on success.

Returns 0 on success and -1 otherwise. In case of an error user
could get it via `box_error_last()`.

Absence of the body in the unprepare response forces users to perform
additional checks to avoid errors. Adding an empty body fixes this problem.

Closes #4769

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

It is needed for performing iproto tests in Lua.

Needed for #4769

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

Since this build flag has been removed as a result of reverting the
tarantool/luajit@d4e985a, its definition in the corresponding Tarantool
cmake file is irrelevant.

Furthermore, considering the breakage faced in #4770 the following tests
are introduced:
* the check whether space __pairs metamethod is set to space.pairs to
  create a Lua Fun iterator that handles __pairs manually underneath.
* the check whether pairs builtin behaviour doesn't change when __pairs
  is set e.g. on space object.

Follow-up #4560
Closes #4770

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

In #4684 it was found that box.tuple.* contained some private
functions: bless(), encode(), and is().

Bless() and encode() didn't make any sense for a user, so they
were hidden into box.internal.tuple.*.

But box.tuple.is() is actually a useful thing. It is harnessed in
the tests a lot, and is likely to be already used by customers,
because it is available in box.tuple.* for a long time. It is a
matter of time when someone will open a doc ticket saying that
box.tuple.is() is not documented. The patch makes it legally
public.

Follow-up #4684

@TarantoolBot document
Title: box.tuple.is()
```Lua
box.tuple.is(object)
```
A function to check whether a given object is a tuple cdata
object. Returns true or false. Never raises nor returns an error.

box.tuple.bless, .encode, and .is are internal. Their
behaviour is not documented, and they may omit some checks for the
sake of speed, and can crash if used without thinking.

Nonetheless, despite they are not documented, curious users could
notice them in box.tuple.* output via autocompletion, for example.
And they could try to use them. This is not ok.

box.tuple.bless() being called by a user leads either to a crash,
or to a leak (because it is basically tuple reference counter
increment).

box.tuple.encode() is kind of a wrapper around msgpack, and users
should not touch it. It may change, may be removed. And is just
makes no sense except some rare cases in schema.lua.

bless() and encode() were used in schema.lua only, so the patch
simply moves them to box.internal.tuple.

box.tuple.is() is kept as is, because
- this is used in the tests a lot;
- it is totally safe;
- that function actually makes sense, and some users could have
  already started using it.

There is no a test, since nothing to test - bless() is not
available for users anymore (assuming no one is brave enough to
rely on box.internal).

Closes #4684

Basic test for popen engine

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Due to os specifics we can't call setsid after vfork on macos
(vfork is not longer a part of posix btw). Instead we can use
ioctl to clear the session, then initiate a new process group.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

This helps to identify if something gone
wrong inside a child process.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

On error path we arm diag with error but strictly
speaking some users (such as unit tests) do not have
to access diag for logging.

Thus log it explicitly for debug sake.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

This is part of posix standart.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

We need it inside popen engine to be able to print
errors from inside of a child process.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

This code is called from C, so it shouldn't throw.

Closes #4753

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

CREATE TABLE used to check column name duplicates before going to
box. But it is not necessary, because the same check is done by
box.

Reviewed-by: Nikita Pettik <korablev@tarantool.org>

Use 'filename' grep_log() option instead of custom log search.

Reviewed-by: Alexander Turenko <alexander.turenko@tarantool.org>

Add filename option to grep_log() (#106).

The option can be used when it is not possible to grab 'box.cfg.log'
value from a tarantool instance: say, when the instance is stopped or
crashed.

Usage:

 | local test_run = require('test_run').new()
 | test_run:grep_log(node, what, bytes, {filename = <...>})

Original port_tuple in C has 'int size;' field. It was
'size_t size' in Lua. Since sizeof(size_t) usually is
8, and sizeof(int) is 4, this was a really bad typo.

When tarantool tries to recover rtree from a snapshot and memtx_memory value
is lower than it has been when the snapshot was created, server suffers from
segmentation fault. This happens because there is no out of memory error
handling in rtree lib. In another words, we do not check the result of
malloc operation.
The execution flow in case of recovery uses different way and the secondary
keys are build in batches. That way has no checks and reservations.
The patch adds memtx_rtree_index_reserve implementation to make sure that any
memory allocation in rtree will fail. Although this gives us no additional
optimization as in case of memtx_tree, the memory reservation prevents
tarantool from segmentation fault. If there is not enough memory to be reserved
server will fail gently with the "Failed to allocate" error message.

Closes #4619

Despite what was stated in the documentation, netbox.connect was not always
equivalent to netbox.self. In particular, they converted tuple to different
types - table and cdata respectively.

The patch fixes the issue and covers all cases where netbox.self and connect
perform conversion of types - e.g., for box.error.

Closes #4513