Need to update tests as with fixup in upstrem
commit baf636a74b4b6d055d93e2d01366d6097eb82d90
Author: Tina Müller <cpan2@tinita.de>
Date:   Thu Jun 14 19:27:04 2018 +0200

The closing single quote needs to be indented...
if it's on its own line.

Closes #3275.

Closes #3275.

Found by @ImeevMA

 - Factor out local_recovery() from box_cfg_xc(). Make it setup
   replication and handle local recovery and hot standby cases.
 - Move replication setup in case of initial bootstrap from box_cfg_xc()
   to bootstrap() to make bootstrap() consistent with local_recovery().
 - Move initial snapshot creation from bootstrap() to bootsrap_master()
   and bootstrap_from_master().

Needed for #461

box_sync_replication() can now be called before recovery, right after
box_listen(). This is a step toward detecting if the instance fell too
much behind its peers in the cluster and so needs to be rebootstrapped.

Needed for #461

In order to find out if the current instance fell too much behind its
peers in the cluster and so needs to be re-bootstrapped we need to
connect it to remote peers before proceeding to local recovery. The
problem is box.cfg.replication may have an entry corresponding to the
instance itself so before connecting we have to start listening to
incoming connections. So this patch moves the call to box_listen()
before recoery is started unless the instance in hot standby mode.
It also folds box_bind() into box_listen() as it is no longer needed
as a separate function.

Needed for #461

In order to find out if the current instance fell too much behind its
peers in the cluster and so needs to be rebootstrapped, we need to know
its vclock before we start local recovery. To do that, let's scan the
most recent xlog. In future, we can optimize that by either storing end
vclock in xlog eof marker or by making a new xlog on server stop.

Needed for #461

Introduce vclock_is_set() helper and use it on xlog_meta::prev_vclock
instead.

Follow-up ac90b498 ("xlog: store prev vclock in xlog header").

This opcode was used when Vdbe had to store key_def in P4 for
OP_OpenRead/Write, so P4 was occupied and OP_LoadPtr was used to
store space pointer in another opcode. But now P4 is free for
OpenRead/Write and space pointer can be stored here.

Alongside, some useless key_def duplications are removed.

Made vdbe_emit_open_cursor calls consistent:
now it uses index id everywhere.
This required to change a way to detect that
VDBE has openned Read cursor to specified table
in vdbe_has_table_read to write result of insert
in temp table if required.

As sqlite3NestedParse became totaly useless, let's
remove unreacheble code and all mentioning.

Resolves #3496.

Now we manually generate AST structures to drop outdated
stats from _sql_stat1 and _sql_stat4 spaces instead of
starting sqlite3NestedParse. This function becomes totally
useless and could be removed.

Part of #3496.

Store replication group id in vylog and ignore spaces whose group_id
equals GROUP_LOCAL when relaying initial join rows.

Folow-up #3443

Follow-up #3443

Replica local spaces can't cause replication conflicts so we don't
really need to respect box.cfg.read_only for them (they are similar
to temporary spaces in this regard).

Follow-up #3443

Next checkpoint time is set by the formula:
 period = self.checkpoint_interval + offset,
where offset is defined as follow:
 offset = random % self.checkpoint_interval

So offset must be calculated again if at least the new
interval is less than the old one.

Closes #3370

vy_mem_iterator_next is as effecient as the current implementation of
vy_point_lookup_scan_mem, because it doesn't copy statements anymore
(see commit 1e1c1fdb vinyl: make read iterator always return newest
tuple version). Let's use it instead of open-coding vy_mem tree lookup.

Now it is possible to specify a number in exponential
form via all formats allowed by json standard.
json.decode('{"remained_amount":2.0e+3}')
json.decode('{"remained_amount":2.0E+3}')
json.decode('{"remained_amount":2e+3}')
json.decode('{"remained_amount":2E+3}')     <-- fixed

Closes #3514.

This patch introduces a new space option, group_id, which defines how
the space is replicated. If it is 0 (default), the space is replicated
throughout the entire cluster. If it is 1, the space is replica local,
i.e. all changes made to it are invisible to other replicas in the
cluster. Currently, no other value is permitted, but in future we will
use this option for setting up arbitrary replication groups in a
cluster. The option can only be set on space creation and cannot be
altered.

Since the concept of replication groups hasn't been established yet,
group_id isn't exposed to Lua. Instead, we use is_local flag, both in
box.schema.space.create arguments and in box.space output.

Technically, to support this feature, we introduce a new header key,
IPROTO_GROUP_ID, which is set to the space group id for all rows
corresponding to a space, both in xlog and in snap. Relay won't send
snapshot rows whose group_id is 1. As for xlog rows, they are
transformed to IPROTO_NOP so as to promote vclock on replicas without
any actual data modification.

The feature is currently supported for memtx spaces only, but it should
be easy to implement it for vinyl spaces as well.

Closes #3443

@TarantoolBot document
Title: Document new space option - is_local
If a space is created with is_local flag set in options, changes made to
the space will be persisted, but won't be replicated.

If a vinyl iterator is passed to another fiber, it may trigger a
use-after-free bug, because the tx it's using may be destroyed
while it's reading the disk. So let's explicitly ban that.

Closes #3394

Fixes commit a09c04bf ("test: fix a sporadic failure of
replication/gc.test").

The test was failing due to a race condition between gc and relay
threads on the master: replica could have acknoledged the last xlog
before it was stopped, in which case gc would correctly delete it and
the total number of xlogs would be equal to 2, not 3.

We typically prefix all boolean variables with 'is_', so let's rename
space_opts::temporary to is_temporary for consistency.

While we are at it, let's also rename tuple_format::temporary to
is_temporary and use space_is_temporary() helper wherever we have
a space pointer.

A NOP request has no body, but since it is treated as DML, we still
encode a zero-size map for its body. This complicates conversion of
local requests to NOP in relay as we can't omit xrow_encode_dml (see
the next patch), so let's allow DML requests to be bodiless.

Needed for #3443

Currently, IPROTO_NOP can only be generated by a before_replace trigger,
when it returns the old tuple thus turning the original operation into a
NOP. In such a case we know the space id and we write it to the request
body. This allows us to dispatch NOP requests via DML route.

As a part of replica local spaces feature, we will substitute requests
operating on local spaces with NOP in relay in order to promote vclock
on replicas without actual data modification. Since space_id is stored
in request body, sending it to replicas would mean decoding the request
body in relay, which is an overkill. To avoid that, let's separate NOP
and DML paths and remove space_id from NOP requests.

Needed for #3443

If pk_def passed to index_def_new() is not NULL, the function will merge
it with the given key_def to create index cmp_def, no matter if the
index is primary or secondary. When an index is altered, we call
index_def_new() to create the new definition, passing the primary key
definition of the altered space for pk_def. If it is the primary index
that is altered, we will pass the definition of the old primary index
and index_def_new() will happily merge it with the new index definition,
resulting in invalid index_def::cmp_def. This doesn't affect memtx, as
memtx doesn't use cmp_def for unique indexes, but it does affect vinyl
in a peculiar way:

  tarantool> _ = box.schema.space.create('test', {engine = 'vinyl'})
  ---
  ...

  tarantool> _ = box.space.test:create_index('pk')
  ---
  ...

  tarantool> _ = box.space.test.index.pk:alter{parts = {2, 'unsigned'}}
  ---
  ...

  tarantool> _ = box.space.test:replace{1, 1}
  ---
  ...

  tarantool> _ = box.space.test:replace{2, 1}
  ---
  ...

  tarantool> box.space.test:select()
  ---
  - - [1, 1]
    - [2, 1]
  ...

(expected: [2, 1])

Fix this by making index_def_new() merge key_def with pk_def only for
secondary indexes.

Closes #3508

There was no check for create privilege when creating a sequence.
Added one, and modified the tests accordingly.

Net.box usage for console is deprecated in 1.10,
replaced it with console.

Closes: #3490

Schema version is used by both clients and internal modules to check
whether there vere any updates in spaces and indices. While clients
only need to be notified when there is a noticeable change, e.g.
space is removed, internal components also need to be notified when
something like space:truncate() happens, because even though this
operation doesn't change space id or any of its indices, it creates a
new space object, so all the pointers to the old object have to be updated.
Currently both clients and internals share the same schema version, which
leads to unnecessary updates on the client side.

Fix this by implementing 2 separate counters for internal and public use:
schema_state gets updated on every change, including recreation of the same
space object, while schema_version is updated only when there are noticable
changes for the clients. Introduce a new AlterOp to alter.cc to update
public schema_version.
Now all the internals reference schema_state, while all the clients use
schema_version. box.iternal.schema_version() returns schema_version
(the public one).

Closes: #3414