This patch introduces a new space option, group_id, which defines how
the space is replicated. If it is 0 (default), the space is replicated
throughout the entire cluster. If it is 1, the space is replica local,
i.e. all changes made to it are invisible to other replicas in the
cluster. Currently, no other value is permitted, but in future we will
use this option for setting up arbitrary replication groups in a
cluster. The option can only be set on space creation and cannot be
altered.

Since the concept of replication groups hasn't been established yet,
group_id isn't exposed to Lua. Instead, we use is_local flag, both in
box.schema.space.create arguments and in box.space output.

Technically, to support this feature, we introduce a new header key,
IPROTO_GROUP_ID, which is set to the space group id for all rows
corresponding to a space, both in xlog and in snap. Relay won't send
snapshot rows whose group_id is 1. As for xlog rows, they are
transformed to IPROTO_NOP so as to promote vclock on replicas without
any actual data modification.

The feature is currently supported for memtx spaces only, but it should
be easy to implement it for vinyl spaces as well.

Closes #3443

@TarantoolBot document
Title: Document new space option - is_local
If a space is created with is_local flag set in options, changes made to
the space will be persisted, but won't be replicated.

If a vinyl iterator is passed to another fiber, it may trigger a
use-after-free bug, because the tx it's using may be destroyed
while it's reading the disk. So let's explicitly ban that.

Closes #3394

Fixes commit a09c04bf ("test: fix a sporadic failure of
replication/gc.test").

The test was failing due to a race condition between gc and relay
threads on the master: replica could have acknoledged the last xlog
before it was stopped, in which case gc would correctly delete it and
the total number of xlogs would be equal to 2, not 3.

We typically prefix all boolean variables with 'is_', so let's rename
space_opts::temporary to is_temporary for consistency.

While we are at it, let's also rename tuple_format::temporary to
is_temporary and use space_is_temporary() helper wherever we have
a space pointer.

A NOP request has no body, but since it is treated as DML, we still
encode a zero-size map for its body. This complicates conversion of
local requests to NOP in relay as we can't omit xrow_encode_dml (see
the next patch), so let's allow DML requests to be bodiless.

Needed for #3443

Currently, IPROTO_NOP can only be generated by a before_replace trigger,
when it returns the old tuple thus turning the original operation into a
NOP. In such a case we know the space id and we write it to the request
body. This allows us to dispatch NOP requests via DML route.

As a part of replica local spaces feature, we will substitute requests
operating on local spaces with NOP in relay in order to promote vclock
on replicas without actual data modification. Since space_id is stored
in request body, sending it to replicas would mean decoding the request
body in relay, which is an overkill. To avoid that, let's separate NOP
and DML paths and remove space_id from NOP requests.

Needed for #3443

If pk_def passed to index_def_new() is not NULL, the function will merge
it with the given key_def to create index cmp_def, no matter if the
index is primary or secondary. When an index is altered, we call
index_def_new() to create the new definition, passing the primary key
definition of the altered space for pk_def. If it is the primary index
that is altered, we will pass the definition of the old primary index
and index_def_new() will happily merge it with the new index definition,
resulting in invalid index_def::cmp_def. This doesn't affect memtx, as
memtx doesn't use cmp_def for unique indexes, but it does affect vinyl
in a peculiar way:

  tarantool> _ = box.schema.space.create('test', {engine = 'vinyl'})
  ---
  ...

  tarantool> _ = box.space.test:create_index('pk')
  ---
  ...

  tarantool> _ = box.space.test.index.pk:alter{parts = {2, 'unsigned'}}
  ---
  ...

  tarantool> _ = box.space.test:replace{1, 1}
  ---
  ...

  tarantool> _ = box.space.test:replace{2, 1}
  ---
  ...

  tarantool> box.space.test:select()
  ---
  - - [1, 1]
    - [2, 1]
  ...

(expected: [2, 1])

Fix this by making index_def_new() merge key_def with pk_def only for
secondary indexes.

Closes #3508

There was no check for create privilege when creating a sequence.
Added one, and modified the tests accordingly.

Net.box usage for console is deprecated in 1.10,
replaced it with console.

Closes: #3490

Schema version is used by both clients and internal modules to check
whether there vere any updates in spaces and indices. While clients
only need to be notified when there is a noticeable change, e.g.
space is removed, internal components also need to be notified when
something like space:truncate() happens, because even though this
operation doesn't change space id or any of its indices, it creates a
new space object, so all the pointers to the old object have to be updated.
Currently both clients and internals share the same schema version, which
leads to unnecessary updates on the client side.

Fix this by implementing 2 separate counters for internal and public use:
schema_state gets updated on every change, including recreation of the same
space object, while schema_version is updated only when there are noticable
changes for the clients. Introduce a new AlterOp to alter.cc to update
public schema_version.
Now all the internals reference schema_state, while all the clients use
schema_version. box.iternal.schema_version() returns schema_version
(the public one).

Closes: #3414

Add a missing CREATE access check. Update tests.
Update a comment.

Fix a security issue when a user who had read/write access
to system spaces could create any object, even while lacking
CREATE privilege. The issue was caused by a misleading access
check in access_check_ddl which would grant access to the owner
of the object. But in case of CREATE the owner of the object
is the effective user alraedy, so CREATE access was always granted.

In case of CREATE, ignore the definer user id in access_check_ddl() - it
is irrelevant, since we create a *new* object.

Update tests.

In scope of gh-945

Add a test case.
Remove trailing spaces.

Add propagation to luarocks of --only-server, --server keys.

Closes #2640

There are some hacks to know the instance was run by tarantoolctl,
none of them are too reliable, though. This patch introduces 2
environment variables set by tarantoolctl for the instance to
know when it's being run or restarted.

Closes: #3215

@TarantoolBot document
Title: tarantoolctl: document setting environment variables
tarantoolctl sets the `TARANTOOLCTL` environment variable when starting
an instance, and sets the `TARANTOOL_RESTARTED' environment variable
when restarting.

Fixed FreeBSD build: there were conflicting types bitset
declared in lib/bitset and _cpuset.h that is the part of
pthread_np.h used on FreeBSD.

Resolves #3046.

All box exceptions belong to box/error.h. Let's move XlogGapError there
as well. This will facilitate conversion of recovery.cc to C when we
finally get to it. While we are at it, let's also move BuildXlogError
function declaration from diag.h to box/error.h, closer to its
definition.

In order to determine whether we need to rebootstrap the instance on
startup, we need to know its vclock. To find it out, we are planning to
scan the last xlog file before proceeding to local recovery, but this
means in case rebootstrap is not required we will scan the last xlog
twice, which is sub-optimal. To speed up this procedure, let's create a
new empty xlog before shutting down the server and reopen it after
restart.

Needed for #461

After read-only flag is dropped, a test space
is created successfully and on next launch creation
will fail since it is not droppped.
Drop the space.

Closes #3507

Currently, if the last WAL in the directory happens to be corrupted or
empty so that we don't recover anything from it, recovery clock will be
that of the last record of the previous WAL. If the previous WAL happens
to have a gap at the end, the next WAL will be created between the last
WAL (empty one) and the next to last (with a gap at the end), breaking
the file order in the WAL directory. That said, we must promote recovery
clock even if we don't recover anything from a WAL.

Currently, the lsn gap check is rather sloppy: when we open an xlog file
for recovery, we check that its vclock equals the vclock of the last
replayed row (see recover_remaining_wals), so if there were WAL write
errors at the end of an xlog file, we will report a false-positive gap
error (because wal doesn't rollback lsn counter). Let's use PrevVclock
xlog meta key introduced earlier to improve the check.

Currently, a cursor that has never been opened and a cursor that was
properly closed share the same state, XLOG_CURSOR_CLOSED. Let's add a
new state, XLOG_CURSOR_UNINITIALIZED, so that we can differentiate
between those two. This new state will be used by the next patch.

This patch adds a new key to xlog header, PrevVclock, which contains the
vclock of the previous xlog file in the xlog directory. It is set by
xdir_create_xlog() to the last vclock in xdir::index. The new key is
only present in XLOG files (it doesn't make sense for SNAP or VYLOG
anyway). It will be used to make the check for xlog gaps more thorough.

Garbage collection doesn't distinguish consumers which need checkpoint
files, such as backup, and the ones, who only need WALS, such as
replicas. A disconnected replica will 'hold' all checkpoint files, created
after it got unsynchronised, even though it doesn't need them, which may
lead to disk space shortage. To fix this, we store consumer's type, and
treat consumers differently during garbage collection: now only the old
WALS are stored for replicas, and old checkpoints are stored for backup,
if any. Also changed the tests to check updated garbage collection correctly.

Closes #3444

box.session.su() set effective user to user
after its execution, which made nested calls
to it not work. Fixed this by saving current
effective user and recovering from the save
after sudo execution. This opened up a bug in
box.schema.user.drop(): it has unnecessary
check for privelege PRIV_REVOKE, which never
gets granted to anyone but admin. Also fixed
this by adding one extra box.session.su() call.

Closes #3090, #3492

In 1.9 it was possible to have a vinylless configuration with
vinyl_memory=0, allow to do this in 1.10 by adjusting sanity
checks for vinyl_memory and memtx_memory. Now banning only
negative values.
memtx_memory check was changed for consistency, trying to
set memtx_memory to 0 fails anyways.
Also added a test to check that vinyl_memory can actually
be set to 0.

Closes: #3468

Before this patch, memtx would silently roll back a multi-statement
transaction on yield, switching the session to autocommit mode.

It would do nothing in case yield happened in a sub-statement
in auto-commit mode.

This could lead to nasty/painful to debug side-effects in
malformed Lua programs.

Fix by adding a special transaction state - aborted, and enter
this state in case of implicit yield.

Check for what happens when a sub-statement yields.
Check that yield trigger is removed by a rollback.

Fixes gh-2631
Fixes gh-2528

fiber->on_yield triggers were not invoked in fiber_call(),
which meant that memtx transaction was not rolled back by
fiber.create().

Fixes gh-3493

Part of #3328

Bug: During parsing http headers, long headers names are truncated
to zero length, but values are not ignored.

Fix this with adding parameter  max_header_name_length to http request.
If header name is bigger than this value, header name is truncated to
this length. Default value of max_header_name_length is 32.

Do some refactoring with renaming long names in http_parser.

Closes #3451