Currently, IPROTO_NOP can only be generated by a before_replace trigger,
when it returns the old tuple thus turning the original operation into a
NOP. In such a case we know the space id and we write it to the request
body. This allows us to dispatch NOP requests via DML route.

As a part of replica local spaces feature, we will substitute requests
operating on local spaces with NOP in relay in order to promote vclock
on replicas without actual data modification. Since space_id is stored
in request body, sending it to replicas would mean decoding the request
body in relay, which is an overkill. To avoid that, let's separate NOP
and DML paths and remove space_id from NOP requests.

Needed for #3443

If pk_def passed to index_def_new() is not NULL, the function will merge
it with the given key_def to create index cmp_def, no matter if the
index is primary or secondary. When an index is altered, we call
index_def_new() to create the new definition, passing the primary key
definition of the altered space for pk_def. If it is the primary index
that is altered, we will pass the definition of the old primary index
and index_def_new() will happily merge it with the new index definition,
resulting in invalid index_def::cmp_def. This doesn't affect memtx, as
memtx doesn't use cmp_def for unique indexes, but it does affect vinyl
in a peculiar way:

  tarantool> _ = box.schema.space.create('test', {engine = 'vinyl'})
  ---
  ...

  tarantool> _ = box.space.test:create_index('pk')
  ---
  ...

  tarantool> _ = box.space.test.index.pk:alter{parts = {2, 'unsigned'}}
  ---
  ...

  tarantool> _ = box.space.test:replace{1, 1}
  ---
  ...

  tarantool> _ = box.space.test:replace{2, 1}
  ---
  ...

  tarantool> box.space.test:select()
  ---
  - - [1, 1]
    - [2, 1]
  ...

(expected: [2, 1])

Fix this by making index_def_new() merge key_def with pk_def only for
secondary indexes.

Closes #3508

There was no check for create privilege when creating a sequence.
Added one, and modified the tests accordingly.

Net.box usage for console is deprecated in 1.10,
replaced it with console.

Closes: #3490

Schema version is used by both clients and internal modules to check
whether there vere any updates in spaces and indices. While clients
only need to be notified when there is a noticeable change, e.g.
space is removed, internal components also need to be notified when
something like space:truncate() happens, because even though this
operation doesn't change space id or any of its indices, it creates a
new space object, so all the pointers to the old object have to be updated.
Currently both clients and internals share the same schema version, which
leads to unnecessary updates on the client side.

Fix this by implementing 2 separate counters for internal and public use:
schema_state gets updated on every change, including recreation of the same
space object, while schema_version is updated only when there are noticable
changes for the clients. Introduce a new AlterOp to alter.cc to update
public schema_version.
Now all the internals reference schema_state, while all the clients use
schema_version. box.iternal.schema_version() returns schema_version
(the public one).

Closes: #3414

Add a missing CREATE access check. Update tests.
Update a comment.

Fix a security issue when a user who had read/write access
to system spaces could create any object, even while lacking
CREATE privilege. The issue was caused by a misleading access
check in access_check_ddl which would grant access to the owner
of the object. But in case of CREATE the owner of the object
is the effective user alraedy, so CREATE access was always granted.

In case of CREATE, ignore the definer user id in access_check_ddl() - it
is irrelevant, since we create a *new* object.

Update tests.

In scope of gh-945

Add a test case.
Remove trailing spaces.

Add propagation to luarocks of --only-server, --server keys.

Closes #2640

There are some hacks to know the instance was run by tarantoolctl,
none of them are too reliable, though. This patch introduces 2
environment variables set by tarantoolctl for the instance to
know when it's being run or restarted.

Closes: #3215

@TarantoolBot document
Title: tarantoolctl: document setting environment variables
tarantoolctl sets the `TARANTOOLCTL` environment variable when starting
an instance, and sets the `TARANTOOL_RESTARTED' environment variable
when restarting.

Fixed FreeBSD build: there were conflicting types bitset
declared in lib/bitset and _cpuset.h that is the part of
pthread_np.h used on FreeBSD.

Resolves #3046.

All box exceptions belong to box/error.h. Let's move XlogGapError there
as well. This will facilitate conversion of recovery.cc to C when we
finally get to it. While we are at it, let's also move BuildXlogError
function declaration from diag.h to box/error.h, closer to its
definition.

In order to determine whether we need to rebootstrap the instance on
startup, we need to know its vclock. To find it out, we are planning to
scan the last xlog file before proceeding to local recovery, but this
means in case rebootstrap is not required we will scan the last xlog
twice, which is sub-optimal. To speed up this procedure, let's create a
new empty xlog before shutting down the server and reopen it after
restart.

Needed for #461

After read-only flag is dropped, a test space
is created successfully and on next launch creation
will fail since it is not droppped.
Drop the space.

Closes #3507

Currently, if the last WAL in the directory happens to be corrupted or
empty so that we don't recover anything from it, recovery clock will be
that of the last record of the previous WAL. If the previous WAL happens
to have a gap at the end, the next WAL will be created between the last
WAL (empty one) and the next to last (with a gap at the end), breaking
the file order in the WAL directory. That said, we must promote recovery
clock even if we don't recover anything from a WAL.

Currently, the lsn gap check is rather sloppy: when we open an xlog file
for recovery, we check that its vclock equals the vclock of the last
replayed row (see recover_remaining_wals), so if there were WAL write
errors at the end of an xlog file, we will report a false-positive gap
error (because wal doesn't rollback lsn counter). Let's use PrevVclock
xlog meta key introduced earlier to improve the check.

Currently, a cursor that has never been opened and a cursor that was
properly closed share the same state, XLOG_CURSOR_CLOSED. Let's add a
new state, XLOG_CURSOR_UNINITIALIZED, so that we can differentiate
between those two. This new state will be used by the next patch.

This patch adds a new key to xlog header, PrevVclock, which contains the
vclock of the previous xlog file in the xlog directory. It is set by
xdir_create_xlog() to the last vclock in xdir::index. The new key is
only present in XLOG files (it doesn't make sense for SNAP or VYLOG
anyway). It will be used to make the check for xlog gaps more thorough.

Garbage collection doesn't distinguish consumers which need checkpoint
files, such as backup, and the ones, who only need WALS, such as
replicas. A disconnected replica will 'hold' all checkpoint files, created
after it got unsynchronised, even though it doesn't need them, which may
lead to disk space shortage. To fix this, we store consumer's type, and
treat consumers differently during garbage collection: now only the old
WALS are stored for replicas, and old checkpoints are stored for backup,
if any. Also changed the tests to check updated garbage collection correctly.

Closes #3444

box.session.su() set effective user to user
after its execution, which made nested calls
to it not work. Fixed this by saving current
effective user and recovering from the save
after sudo execution. This opened up a bug in
box.schema.user.drop(): it has unnecessary
check for privelege PRIV_REVOKE, which never
gets granted to anyone but admin. Also fixed
this by adding one extra box.session.su() call.

Closes #3090, #3492

In 1.9 it was possible to have a vinylless configuration with
vinyl_memory=0, allow to do this in 1.10 by adjusting sanity
checks for vinyl_memory and memtx_memory. Now banning only
negative values.
memtx_memory check was changed for consistency, trying to
set memtx_memory to 0 fails anyways.
Also added a test to check that vinyl_memory can actually
be set to 0.

Closes: #3468

Before this patch, memtx would silently roll back a multi-statement
transaction on yield, switching the session to autocommit mode.

It would do nothing in case yield happened in a sub-statement
in auto-commit mode.

This could lead to nasty/painful to debug side-effects in
malformed Lua programs.

Fix by adding a special transaction state - aborted, and enter
this state in case of implicit yield.

Check for what happens when a sub-statement yields.
Check that yield trigger is removed by a rollback.

Fixes gh-2631
Fixes gh-2528

fiber->on_yield triggers were not invoked in fiber_call(),
which meant that memtx transaction was not rolled back by
fiber.create().

Fixes gh-3493

Part of #3328

Bug: During parsing http headers, long headers names are truncated
to zero length, but values are not ignored.

Fix this with adding parameter  max_header_name_length to http request.
If header name is bigger than this value, header name is truncated to
this length. Default value of max_header_name_length is 32.

Do some refactoring with renaming long names in http_parser.

Closes #3451

Bug: Header parser validates http status line and besides saving http
status, saves valid characters to header name, which is wrong.

Fix this with skipping status line after validation without saving it as
a header.

In scope of #3451

If tarantool is stopped while writing a snapshot or a vinyl run file,
inprogress files will never be removed. Fix this by collecting those
files on recovery completion.

Original patch by @IlyaMarkovMipt. Reworked by @locker.

Closes #3406

In order to log only about files that were actually removed change log
messages from "removing <name of file>" to "removed <name of file>" in
vy_run_remove_files and xdir_collect_garbage functions.

Needed for #3406

A minor follow up on the fix for gh-3452 (http.client timeout bug)

Current implementation of http.client relies on fiber_cond which is set
after the request was registered and doesn't consider the fact that
response may be handled before the set of fiber_cond.

So we may have the following situation:
1. Register request in libcurl(curl_multi_add_handle in curl_execute).
2. Receive and process response, fiber_cond_signal on cond_var which no
one waits.
3. fiber_cond_wait on cond which is already signaled. Wait until timeout
is fired.

In this case user have to wait timeout, though data was received
earlier.

Fix this with adding extra flag in_progress to curl_request struct.
Set this flag true before registering request in libcurl and set it
false when request is finished before fiber_cond_signal.
When in_progress flag is false, don't wait on cond variable.

Add 1 error injection.

Closes #3452