- Factor out local_recovery() from box_cfg_xc(). Make it setup
   replication and handle local recovery and hot standby cases.
 - Move replication setup in case of initial bootstrap from box_cfg_xc()
   to bootstrap() to make bootstrap() consistent with local_recovery().
 - Move initial snapshot creation from bootstrap() to bootsrap_master()
   and bootstrap_from_master().

Needed for #461

box_sync_replication() can now be called before recovery, right after
box_listen(). This is a step toward detecting if the instance fell too
much behind its peers in the cluster and so needs to be rebootstrapped.

Needed for #461

In order to find out if the current instance fell too much behind its
peers in the cluster and so needs to be re-bootstrapped we need to
connect it to remote peers before proceeding to local recovery. The
problem is box.cfg.replication may have an entry corresponding to the
instance itself so before connecting we have to start listening to
incoming connections. So this patch moves the call to box_listen()
before recoery is started unless the instance in hot standby mode.
It also folds box_bind() into box_listen() as it is no longer needed
as a separate function.

Needed for #461

In order to find out if the current instance fell too much behind its
peers in the cluster and so needs to be rebootstrapped, we need to know
its vclock before we start local recovery. To do that, let's scan the
most recent xlog. In future, we can optimize that by either storing end
vclock in xlog eof marker or by making a new xlog on server stop.

Needed for #461

Introduce vclock_is_set() helper and use it on xlog_meta::prev_vclock
instead.

Follow-up ac90b498 ("xlog: store prev vclock in xlog header").

Store replication group id in vylog and ignore spaces whose group_id
equals GROUP_LOCAL when relaying initial join rows.

Folow-up #3443

Follow-up #3443

Replica local spaces can't cause replication conflicts so we don't
really need to respect box.cfg.read_only for them (they are similar
to temporary spaces in this regard).

Follow-up #3443

Next checkpoint time is set by the formula:
 period = self.checkpoint_interval + offset,
where offset is defined as follow:
 offset = random % self.checkpoint_interval

So offset must be calculated again if at least the new
interval is less than the old one.

Closes #3370

vy_mem_iterator_next is as effecient as the current implementation of
vy_point_lookup_scan_mem, because it doesn't copy statements anymore
(see commit 1e1c1fdb vinyl: make read iterator always return newest
tuple version). Let's use it instead of open-coding vy_mem tree lookup.

Now it is possible to specify a number in exponential
form via all formats allowed by json standard.
json.decode('{"remained_amount":2.0e+3}')
json.decode('{"remained_amount":2.0E+3}')
json.decode('{"remained_amount":2e+3}')
json.decode('{"remained_amount":2E+3}')     <-- fixed

Closes #3514.

This patch introduces a new space option, group_id, which defines how
the space is replicated. If it is 0 (default), the space is replicated
throughout the entire cluster. If it is 1, the space is replica local,
i.e. all changes made to it are invisible to other replicas in the
cluster. Currently, no other value is permitted, but in future we will
use this option for setting up arbitrary replication groups in a
cluster. The option can only be set on space creation and cannot be
altered.

Since the concept of replication groups hasn't been established yet,
group_id isn't exposed to Lua. Instead, we use is_local flag, both in
box.schema.space.create arguments and in box.space output.

Technically, to support this feature, we introduce a new header key,
IPROTO_GROUP_ID, which is set to the space group id for all rows
corresponding to a space, both in xlog and in snap. Relay won't send
snapshot rows whose group_id is 1. As for xlog rows, they are
transformed to IPROTO_NOP so as to promote vclock on replicas without
any actual data modification.

The feature is currently supported for memtx spaces only, but it should
be easy to implement it for vinyl spaces as well.

Closes #3443

@TarantoolBot document
Title: Document new space option - is_local
If a space is created with is_local flag set in options, changes made to
the space will be persisted, but won't be replicated.

If a vinyl iterator is passed to another fiber, it may trigger a
use-after-free bug, because the tx it's using may be destroyed
while it's reading the disk. So let's explicitly ban that.

Closes #3394

Fixes commit a09c04bf ("test: fix a sporadic failure of
replication/gc.test").

The test was failing due to a race condition between gc and relay
threads on the master: replica could have acknoledged the last xlog
before it was stopped, in which case gc would correctly delete it and
the total number of xlogs would be equal to 2, not 3.

We typically prefix all boolean variables with 'is_', so let's rename
space_opts::temporary to is_temporary for consistency.

While we are at it, let's also rename tuple_format::temporary to
is_temporary and use space_is_temporary() helper wherever we have
a space pointer.

A NOP request has no body, but since it is treated as DML, we still
encode a zero-size map for its body. This complicates conversion of
local requests to NOP in relay as we can't omit xrow_encode_dml (see
the next patch), so let's allow DML requests to be bodiless.

Needed for #3443

Currently, IPROTO_NOP can only be generated by a before_replace trigger,
when it returns the old tuple thus turning the original operation into a
NOP. In such a case we know the space id and we write it to the request
body. This allows us to dispatch NOP requests via DML route.

As a part of replica local spaces feature, we will substitute requests
operating on local spaces with NOP in relay in order to promote vclock
on replicas without actual data modification. Since space_id is stored
in request body, sending it to replicas would mean decoding the request
body in relay, which is an overkill. To avoid that, let's separate NOP
and DML paths and remove space_id from NOP requests.

Needed for #3443

If pk_def passed to index_def_new() is not NULL, the function will merge
it with the given key_def to create index cmp_def, no matter if the
index is primary or secondary. When an index is altered, we call
index_def_new() to create the new definition, passing the primary key
definition of the altered space for pk_def. If it is the primary index
that is altered, we will pass the definition of the old primary index
and index_def_new() will happily merge it with the new index definition,
resulting in invalid index_def::cmp_def. This doesn't affect memtx, as
memtx doesn't use cmp_def for unique indexes, but it does affect vinyl
in a peculiar way:

  tarantool> _ = box.schema.space.create('test', {engine = 'vinyl'})
  ---
  ...

  tarantool> _ = box.space.test:create_index('pk')
  ---
  ...

  tarantool> _ = box.space.test.index.pk:alter{parts = {2, 'unsigned'}}
  ---
  ...

  tarantool> _ = box.space.test:replace{1, 1}
  ---
  ...

  tarantool> _ = box.space.test:replace{2, 1}
  ---
  ...

  tarantool> box.space.test:select()
  ---
  - - [1, 1]
    - [2, 1]
  ...

(expected: [2, 1])

Fix this by making index_def_new() merge key_def with pk_def only for
secondary indexes.

Closes #3508

There was no check for create privilege when creating a sequence.
Added one, and modified the tests accordingly.

Net.box usage for console is deprecated in 1.10,
replaced it with console.

Closes: #3490

Schema version is used by both clients and internal modules to check
whether there vere any updates in spaces and indices. While clients
only need to be notified when there is a noticeable change, e.g.
space is removed, internal components also need to be notified when
something like space:truncate() happens, because even though this
operation doesn't change space id or any of its indices, it creates a
new space object, so all the pointers to the old object have to be updated.
Currently both clients and internals share the same schema version, which
leads to unnecessary updates on the client side.

Fix this by implementing 2 separate counters for internal and public use:
schema_state gets updated on every change, including recreation of the same
space object, while schema_version is updated only when there are noticable
changes for the clients. Introduce a new AlterOp to alter.cc to update
public schema_version.
Now all the internals reference schema_state, while all the clients use
schema_version. box.iternal.schema_version() returns schema_version
(the public one).

Closes: #3414

Add a missing CREATE access check. Update tests.
Update a comment.

Fix a security issue when a user who had read/write access
to system spaces could create any object, even while lacking
CREATE privilege. The issue was caused by a misleading access
check in access_check_ddl which would grant access to the owner
of the object. But in case of CREATE the owner of the object
is the effective user alraedy, so CREATE access was always granted.

In case of CREATE, ignore the definer user id in access_check_ddl() - it
is irrelevant, since we create a *new* object.

Update tests.

In scope of gh-945

Add a test case.
Remove trailing spaces.

Add propagation to luarocks of --only-server, --server keys.

Closes #2640

There are some hacks to know the instance was run by tarantoolctl,
none of them are too reliable, though. This patch introduces 2
environment variables set by tarantoolctl for the instance to
know when it's being run or restarted.

Closes: #3215

@TarantoolBot document
Title: tarantoolctl: document setting environment variables
tarantoolctl sets the `TARANTOOLCTL` environment variable when starting
an instance, and sets the `TARANTOOL_RESTARTED' environment variable
when restarting.

Fixed FreeBSD build: there were conflicting types bitset
declared in lib/bitset and _cpuset.h that is the part of
pthread_np.h used on FreeBSD.

Resolves #3046.

All box exceptions belong to box/error.h. Let's move XlogGapError there
as well. This will facilitate conversion of recovery.cc to C when we
finally get to it. While we are at it, let's also move BuildXlogError
function declaration from diag.h to box/error.h, closer to its
definition.

In order to determine whether we need to rebootstrap the instance on
startup, we need to know its vclock. To find it out, we are planning to
scan the last xlog file before proceeding to local recovery, but this
means in case rebootstrap is not required we will scan the last xlog
twice, which is sub-optimal. To speed up this procedure, let's create a
new empty xlog before shutting down the server and reopen it after
restart.

Needed for #461

After read-only flag is dropped, a test space
is created successfully and on next launch creation
will fail since it is not droppped.
Drop the space.

Closes #3507

Currently, if the last WAL in the directory happens to be corrupted or
empty so that we don't recover anything from it, recovery clock will be
that of the last record of the previous WAL. If the previous WAL happens
to have a gap at the end, the next WAL will be created between the last
WAL (empty one) and the next to last (with a gap at the end), breaking
the file order in the WAL directory. That said, we must promote recovery
clock even if we don't recover anything from a WAL.