We aren't planning to support all iterator types for _sequence read
views, but once we implement a Lua API for read views, the user will
still be able to create a read view of this space. So let's replace
the assertions with a graceful error.

Needed for https://github.com/tarantool/tarantool-ee/issues/197

NO_DOC=internal
NO_TEST=ee
NO_CHANGELOG=internal

To implement all iterator types for memtx tree and hash index read
views, we need to have the key definition. We can't just use the key
definition stored in the index directly, because, despite the fact that
a read view keeps a reference the index, its index definition may still
be changed by alter (when compatible changes happen).

To solve this problem, let's store a copy of the index definition in
memtx tree and hash index read views. The index definition also contains
the index name, which will be useful for exporting the read view to Lua.

Needed for https://github.com/tarantool/tarantool-ee/issues/197

NO_DOC=internal
NO_TEST=ee
NO_CHANGELOG=internal

To make a memtx snapshot, we use the create_snapshot_iterator index
method. The method creates a 'frozen' iterator over an index - changes
done to the index after the iterator was created don't affect the
iterator output. Also, the iterator is safe to use from any thread.
This API works just fine for snapshots, but it's too limited to allow
creation of user read views so we need to rework it.

To make the existing snapshot infrastructure suitable for user read
views, this commit replaces the create_snapshot_iterator method with
create_read_view. The new method returns an index_read_view object,
which has the API similar to the read-only API of an index. A read view
object may only be created and destroyed in the tx thread, but it may be
used in any thread.

Currently, index_read_view has the only method - create_iterator, which
takes iterator type and key and returns an index_read_view_iterator
object. The iterator type and key arguments are ignored and we always
assume the iterator type to be ITER_ALL (asserted), but later on we will
fix this and also add a method to look up a tuple by key.

Closes #7194

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Since commit f167c1af ("memtx: decompress tuples in snapshot
iterator") a snapshot iterator may allocate the result tuple on the
fiber region - the caller is supposed to clean the region after usage.
So we don't need to store the tuple in sequence_data_iterator anymore -
we can allocate it on the fiber region instead, which is simpler and
more straightforward.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Currently, there's no notion of a LIGHT hash table read view per se -
one can create an iterator over a regular hash table and then "freeze"
it. This works just fine for snapshotting and joining replicas, but this
spartan API doesn't let us implement user read views, because to do that
we need to do lookups and create iterators over a frozen hash table as
many times as we want, not just once.

So this patch introduces a concept of LIGHT(view), which contains a
frozen image of a LIGHT(core) and implements a subset of non-modifying
LIGHT(core) methods:

 - LIGHT(view_count)
 - LIGHT(view_find)
 - LIGHT(view_find_key)
 - LIGHT(view_get)
 - LIGHT(view_iterator_begin)
 - LIGHT(view_iterator_key)
 - LIGHT(view_iterator_get_and_next)

Note, LIGHT(core) and LIGHT(view) share LIGHT(iterator), because
iterator methods (begin, key, get_and_next) take LIGHT(core) or
LIGHT(view). The LIGHT(iterator) now contains only a hash table slot.

We could also implement the rest of non-modifying methods, but didn't do
that, because they are not needed to implement user read views:

 - LIGHT(random)
 - LIGHT(selfcheck)

To create a LIGHT(view) from a LIGHT(core), one is supposed to call
LIGHT(view_create). If a LIGHT(view) is no longer needed, it should be
destroyed with LIGHT(view_destroy).

Old methods used for creating frozen iterators were dropped:

 - LIGHT(iterator_freeze)
 - LIGHT(iterator_destroy)

To avoid code duplication, we factored out the common part of
LIGHT(core) and LIGHT(view) into a new structure, named LIGHT(common).
Basically, the new structure contains all LIGHT(core) members except
matras, which is stored in LIGHT(core). The difference between
LIGHT(view) and LIGHT(core) is that the latter stores matras_view
instead of matras. The common part contains pointers to matras and
matras_view, which are used by internal implementation to look up
LIGHT(record).

All internal methods now take LIGHT(common) instead of LIGHT(core).
For all public methods that are implemented both for LIGHT(core) and
LIGHT(view), we have the common implementation defined in _impl suffixed
private function, which is called by the corresponding public functions.

To ensure that a modifying method isn't called on LIGHT(common) object
corresponding to a LIGHT(view) because of a bug in the LIGHT code, we
added !matras_is_read_view_created assertion to LIGHT(touch_record),
LIGHT(prepare_first_insert), and LIGHT(grow).

Closes #7192

NO_DOC=refactoring
NO_CHANGELOG=refactoring

`VERSION` files in small subproject and in tarantool are treated as C++
standard library on a filesystem with case-insensitive names. So we have
to delete the root of tarantool project from `include_directories` in
tarantool CMake. Also we have to change `include_directories` in tarantool
CMake from the root of `small` project to `include` subfolder in `small`
project.
Closes #6076

This patch introduces "current" function for sequences.
It returns the last retrieved value of specified sequence or
throws an error if no value has been generated yet.

This patch partially reverts 3ff1f1e3
(box: remove sequence_get) here similar function "get" was removed
to avoid possible misleading with "currval" function of PosgreSQL
that returns the last obtained value of the sequence in the scope
of current session. In contrast "current" returns the last globally
retrieved value of the sequence.

Closes #4752

Reviewed-by: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

@TarantoolBot document
Title: sequence:current()

This patch introduces "current" function for sequences.
It returns the last retrieved value of specified sequence or
throws an error if no value has been generated yet ("next"
has not been called yet or right after "reset" is called).

Lua:

Example:

```lua
sq = box.schema.sequence.create('test')
---
...
sq:current()
---
- error: Sequence 'test' is not started
...
sq:next()
---
- 1
...
sq:current()
---
- 1
...
sq:set(42)
---
...
sq:current()
---
- 42
...
sq:reset()
---
...
sq:current()  -- error
---
- error: Sequence 'test' is not started
...
```

C API:

```C
int
box_sequence_current(uint32_t seq_id, int64_t *result);
```

Where:
  * seq_id - sequence identifier;
  * result - pointer to a variable where the current sequence
  value will be stored on success.

Returns 0 on success and -1 otherwise. In case of an error user
could get it via `box_error_last()`.

Memtx iterators never fail, that's why the snapshot iterator interface
doesn't support failures. However, once we introduce snapshot iterator
support for vinyl, we will need a way to handle errors in the API.

Currently, if an INSERT is executed inside SQL trigger and it
results in generated autoincrement identifiers, ones will be
displayed as a result of the statement. This is wrong, since we
are not able to divide IDs obtained into those that belong to the
table mentioned in the statement and those that do not belong to
this table. This has been fixed by adding a new argument to
OP_IdxInsert. In case the argument is not 0, recently generated
identifier is retrieved and saved into the list, which is held in
VDBE itself. Note that from now we don't save autoincremented
value to VDBE right in sequence_next() - this operation is moved
to OP_IdxInsert. So that, VDBE can be removed from struct txn.

For example:
box.execute('CREATE TABLE t1 (i INT PRIMARY KEY AUTOINCREMENT);')
box.execute('CREATE TABLE t2 (i INT PRIMARY KEY AUTOINCREMENT);')
box.execute('CREATE TRIGGER r AFTER INSERT ON t1 FOR EACH ROW '..
            'BEGIN INSERT INTO t2 VALUES (null); END')
box.execute('INSERT INTO t2 VALUES (100);')
box.execute('INSERT INTO t1 VALUES (NULL), (NULL), (NULL);')

Result should be:
---
- autoincrement_ids:
  - 1
  - 2
  - 3
  row_count: 3
...

Closes #4188

To implement transactional DDL, we must make sure that in-memory schema
is updated synchronously with system space updates, i.e. on_replace, not
on_commit.

Note, to do this in case of the sequence cache, we have to rework the
way sequences are exported to Lua - make on_alter_sequence similar to
how on_alter_space and on_alter_func triggers are implemented.

According to documentation some JDBC functions have an ability to
return all ids that were generated in executed INSERT statement.
This patch gives a way to implement such functionality. After
this patch all ids autogenerated during VDBE execution will be
saved and returned through IPROTO.

Closes #2618

Allow define access privileges for all spaces, functions and sequences.
Read and write privileges are supported for spaces, execute privilege
for sequences. Privilege granting and revoking might be done through old api
without object identification:
  box.schema.user.grant("guest", "read", "space")

Prerequisite #945

* Add following behavior:
Owner of object can't utilize her own objects if she has not usage
access.
* Change access checks of space, sequence, function objects
Similar checks of other objects are performed in alter.cc.

Closes gh-3089

* Add box_on_access_denied API method
* Modify access error handlers in order to call the mentioned trigger
* Add new type of error - AccessDeniedError

Related #2911 "add audit log triggers"

Delete specifc access denied error code (ER_FUNCTION_ACCESS_DENIED,
ER_SPACE_ACCESS_DENIED, ER_FUNC_ACCESS_DENIED) and always
ER_ACCESS_DENIED code, which now contains object name and type

Pass operation type (create, drop, grant, revoke) to ER_ACCESS_DENIED.
Add a helper function schema_find_name() to schema.[h,cc].

In scope of gh-2911 "add triggers for audit log".

Heavily edited by @kostja

Add system privileges 'session' and 'usage'

* 'session' privilege lets user connect to database server
* 'usage' privilege lets user use his/her rights on database objects
* Both privileges are assigned to all users by default.

Implementation details:
* system privileges are special grant rights to 'universe'.
Therefore, they can be granted only by admin. Because of this fact,
during creation or deletion of user, we have to switch to 'admin' to
grant or revoke these rights.

Important changes:
* changed bootstrap.snap due to need to start admin with new privileges
* added auto upgrade script for 1.7.7

Fixes gh-2898.

With contributions by @kostja.

box.session.su() changes both user and effective user right now.
Changing only the session user seems to be rather difficult:
we need to keep the object allocated somewhere, and keeping
in mind request multiplexor in iproto, with which many requests
can share the same session, it can only be Lua stack.

While at it, change current_user() to effective_user() to
make it less ambiguous.

Since commit c0a30d17 ("sequence: speed up allocation of sequence
data tuple during checkpoint") sequence_data_iterator_next() does not
throw an exception, so the whole thing can be moved to sequence.c.

 - Add a new schema object class, 'sequence', which can be passed to
   box.schema.{grant,revoke} to grant access to a sequence.
 - To use a sequence (i.e. call next(), set(), or reset() on it), the
   user must either be the owner of the sequence or has a permission to
   write to it.
 - This also holds for sequences attached to spaces: the user can only
   insert a tuple to a space if he has a write access to the sequence
   attached to the space. UPDATE/DELETE do not require access to a
   sequence.
 - Automatically generated sequences are special: it can be used for
   auto increment (by passing nil to INSERT) even if the user was not
   explicitly granted access to it - it is enough to have write access
   to the space.

See #389

The value returned by space.auto_increment() is not guaranteed to be
monotonically growing, because this method uses max+1 for the next
generated value, which can be less than a value generated before in case
the max key is deleted. Besides, calling max() may be expensive, e.g. it
requires a disk access in case of Vinyl. So this patch implements auto
increment based on persistent sequences.

To be used for auto increment, a sequence needs to be attached to a
space. To attach an existing sequence to a space, pass 'sequence=ID' in
index options when creating or modifying the primary key. Here ID is the
name or the numeric identifier of the sequence to attach. Note,
sequences can be attached only to spaces with integer primary key.
After a sequence is attached, it will be used for auto increment when
the user passes 'nil' for the primary key value. If the user passes a
value different from 'nil', the value will be used to update the
attached sequence.

To detach a sequence from the space it was attached to, either drop the
primary key of the space or alter the primary key with 'sequence=false'.

It is not necessary to pre-create a sequence to use it for auto
increment - a sequence can be automatically generated. To generate a
sequence automatically, pass 'sequence=true' in index options.
Automatically generated sequences are named '_auto_ID', where ID is the
space id, and removed when the space is dropped or the sequence is
detached.

A sequence cannot be dropped as long as it is attached to any spaces.

Closes #389

Currently, _sequence_data is written to the snapshot the same way as any
other memtx space - the snapshot iterator simply walks over the space
tuples and dumps them one-by-one. This is OK now, because _sequence_data
is always up-to-date. However, after the next patch, its contents may
become stale - due to technical reasons we won't update _sequence_data
if a sequence is used for auto increment in a space. To make sure we
still snapshot sequence values properly, let's introduce a special
iterator for this space that walks over the sequence cache instead of
the space contents.

Needed for #389

In contrast to PostgreSQL, this method doesn't make sense in Tarantool,
because we don't have sessions.

This patch implements a new object type, persistent sequences. Sequences
are created with function box.schema.sequence.create(name, options).
Options include min/max, start value, increment, cache size, just like
in Postgresql, although 'cache' is ignored for now. All sequences can be
accessed via box.sequence.<name>, similarly to spaces. To generate a
sequence value, use seq:next() method. To retrieve the last generated
value, use seq:get(). A sequence value can also be reset to the start
value or to any other value using seq:reset() and seq:set() methods.

Needed for #389

Introduce an API for transaction ordering. Different
implementations of the API are to be used in case of local
wal recovery, wal_mode=NONE, and wal_mode=write/fsync.

* update test-run to a version which works with the
new server as well as with the old one.
* move lock/unlock facilities to an own file
* do not attempt to play with hot standby in an empty
  WAL dir
* lock wal dir, not snap dir.

Update test-run, review fixes.

 * implement and use functions for getting MessagePack data
   of a tuple

 * rename struct tuple.data to tuple.raw,
   and struct vy_stmt.data to vy_stmt.raw

Need for #1572

* Extract high-level IPROTO_CALL code to box.cc
* Move execute_c_call to func.cc
* Prepare src/lua/call.cc to rewrite in C

Incompatible changes:

* #300 logic was reverted. An attempt to call an unexisting Lua function
  without universal execute permissions now cause "access denied" message
  instead of "function does not exist".

  Since there is no cache for Lua procedures, existence of a function can be
  checked only by attempt to execute it. Caller **must** have a permission
  to execute function in order to execute functions. It's obvious, isn't it?

  Anyway, this patch doesn't affect user experience with using stored
  procedures. The two steps still must be performed to allow Lua calls
  using the binary protocol:
    1. A function must be defined in Lua
    2. A function must be declared in box.schema and caller must have
       execute permission for this object or for entire "universe".
  The order actually doesn't matter - the both steps must be done.

Voted by team

Please be careful - only _ex versions throw OutOfMemory in lib/small.
All other function must be checked for NULL return!

Affected prefixes:

 * ibuf_
 * obuf_
 * region_
 * mempool_
 * smalloc_

Introduce a layer of wrappers for a number of internal box functions,
dealing with accss to spaces and indexes.

These wrappers:
    * don't throw exceptions
    * have a common prefix box_
    * are exported in the server development headers

Rewrite Lua C bindings to use the public API described above.
Rewrite Lua FFI bindings to do the same.
Add test.

Extract the remaining places which access tuple->data
directly into tuple_to_port.cc file, which will have full
access to all tuple formats and all destinations to which
a tuple may need to be converted.

In this file it'll be possible to perform a conversion
efficiently, at the same time, it will be the only place
of cross-dependency between all tuple formats and all
conversion destinations.