The function is called only by the journal when write is finished.

Besides, it may not complete the transaction. In case of
synchronous replication it is not enough for completion. It means,
it can't have 'complete' in its name.

Also the function is never used out of txn.c, so it is removed
from txn.h and is now static.

The patch is a preparation for not spaming "too long WAL write" on
synchronous transactions, because it is simply misleading.

Part of #5139

We should never trust the request data it can
carry anything, thus lets make sure we're not
decoding some trash into a string.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

The size should be matched to the real size of a buffer,
otherwise it is a room for mistake. Same time make sure
we're not overriding the buffer.

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

Found that the previously fixed vinyl/gh.test.lua test in commit:

  94dc5bdd ('test: gh test hangs after gh-4957-too-many-upserts')

with adding fiber.sleep(1) workaround to avoid of raise from the
previously run vinyl/gh-4957-too-many-upserts.test.lua test can be
changed in the other way. The new change from one side will leave
the found issue untouched to be able to resolve it within opened
issue in github. And from the other side it will let the test-run
tool to be able to avoid of this issue using fragile list feature
to save the stability of testing due to found issue is flaky and
can be passed on reruns.

The current fix changes the forever waiting loop to especially
created for such situations test_run:wait_cond() routine which has
timeout in it to avoid of hanging the test till global timeout will
occure. It will let the testing to be continued even after the fail.

Needed for #5141

Added restart the current server to resolve the issue #5141 which
reproduced in test:

  vinyl/gh-5141-invalid-vylog-file.test.lua

Added test-run filter on box.snapshot error message:

  'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'

to avoid of printing changing data in results file to be able to use
its checksums in fragile list of test-run to rerun it as flaky issue.

Part of #5141

Created the stable reproducer for the issue #5141:

  box.snapshot()
  ---
 -- ok
 +- error: 'Invalid VYLOG file: Slice <NUM> deleted but not registered'
  ...

flaky occured in vinyl/ suite tests if running after the test:

  vinyl/gh-4957-too-many-upserts.test.lua

as new standalone test:

  vinyl/gh-5141-invalid-vylog-file.test.lua

based on test:

  vinyl/gh-4957-too-many-upserts.test.lua

Due to issue not reproduced on FreeBSD 12, then test was blocked with:

  vinyl/gh-5141-invalid-vylog-file.skipcond

Needed for #5141

Added test-run filter on box.snapshot error message:

  'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'

to avoid of printing changing data in results file to be able to use
its checksums in fragile list of test-run to rerun it as flaky issue.
Also added checksums to fragile list for the following tests:

  vinyl/iterator.test.lua                       gh-5141
  vinyl/snapshot.test.lua                       gh-4984

Needed for #5141
Needed for #4984

Sometimes it is convenient to use default compiler on CentOS 7.
Added test job which uses for compiling default compiler files:

  CC=/usr/bin/gcc
  CXX=/usr/bin/g++

Closes #4941

Added ENABLE_WERROR flag to build options to enable Werror.

Part of #4941

Building with gcc-4.8.5 on CentOS 7 found issue:

cd /source/build/usr/src/debug/tarantool-2.6.0.144/test/unit && /usr/bin/g++ ... -Wp,-D_FORTIFY_SOURCE=2 ... -O2 ... -O0 -o CMakeFiles/fiber.test.dir/fiber.cc.o -c /source/build/usr/src/debug/tarantool-2.6.0.144/test/unit/fiber.cc
In file included from /usr/include/inttypes.h:25:0,
                 from /source/build/usr/src/debug/tarantool-2.6.0.144/src/lib/small/small/region.h:34,
                 from /source/build/usr/src/debug/tarantool-2.6.0.144/src/lib/core/memory.h:33,
                 from /source/build/usr/src/debug/tarantool-2.6.0.144/test/unit/fiber.cc:1:
/usr/include/features.h:330:4: error: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Werror=cpp]
 # warning _FORTIFY_SOURCE requires compiling with optimization (-O)

It happened because _FORTIFY_SOURCE=2 flag needed -O[1|2] optimization,
but latest set in command was -O0. To fix it removed not needed '-O0'
optimization from test/unit/CmakeLists.txt file. This optimization
became unneeded after the commit:

  aa78a941 ("test/uint: fiber")

when the test was completely rewritten.

Needed for #4941

Store *.reject files in ${BUILD}/test/var/rejects/<...>/ instead of
${SOURCE}/test/<...>/.

The past approach leads to problems with testing, when the out of source
build is used and sources are on a read-only filesystem. The main
problem is when a test fails, but it is marked as fragile and should be
run again. The test fail assumes storing of the .reject file and the
testing fails on attempt to write to the read-only filesystem. The
re-run is not performed so.

Side effect for the in-source build: since the test/var/ directory is
gitignored, the *.reject files will not shown in `git status` output
anymore.

https://github.com/tarantool/test-run/pull/209

Follows up #4874

Closes #5071

@TarantoolBot document
Title: memtx: varbinary supported in bitset indexes
Now it is possible to create bitset indexes on the fields of varbinary type,
e.g.: s:create_index('b', {type = 'bitset', parts = {1, "varbinary"}})

* test: fix warnings spotted by luacheck

Previously accepted formats of index parts:
parts = {field1, type1, field2, type2}, or
parts = {{field1, type1, ...}, {field2, type2, ...}}

Now it is allowed to write without extra brace if there is one part only:
parts = {field1, type1, ...}

Closes #2866

To run Jepsen tests in different configurations we need to parametrize run
script by options, so lein options and number of nodes passed with environment
variables. By default script runs testing with Tarantool built from latest
commit.

Added these configurations:

- single instance
- single instance with enabled TXM
- cluster with enabled Raft
- cluster with enabled Raft and TXM

Closes #5437

When the connection was not established yet, netbox reported empty error
while executing a remote request.

Closes #4787

Tx stories must be linked into correct double-linked list.
Preserve it.

Part of #5423

There was a mess in tuple refernce in TX history.
Now it was remade in the following asumptions:
 * a clean tuple belongs to space, and the space implicitly holds
a reference to the tuple.
 * a dirty tuple belongs to TX manager and a reference is held
in the corresponding story.

Closes #5423

When an instance is configured as candidate, it has a leader death timer
ticking constantly to schedule an election as soon as leader disappears.
When the instance receives the leader's heartbeat, it resets the timer
to its initial value.

When being a voter, the instance ignores heartbeats, since it has
nothing to wait for. So its timer must be stopped. Otherwise it'll try
to schedule a new election and fail.

Stop the timer on transition from candidate to voter.

When a node becomes a leader, it restarts relay recovery cursors
to re-send all the data since the last acked row.

But during recovery restart the relay lost the trigger, which used
to update GC state in TX thread.

The patch preserves the trigger.

Follow up for #5433

When a Raft node is elected as a leader, it should resend all its
data to the followers from the last acked vclock. Because while
the node was not a leader, the other instances ignored all the
changes from it.

The resending is done via restart of the recovery cursor in the
relay thread. When the cursor was restarted, it used the last
acked vclock to find the needed xlog file. But it didn't set the
local LSN component, which was 0 (replicas don't send it).

When in reality the component was not zero, the recovery cursor
still tried to find the oldest xlog file having the first local
row. And it couldn't. The first created local row may be gone long
time ago.

The patch makes the restart keep the local LSN component
unchanged, as it was used by the previous recovery cursor, before
the restart.

Closes #5433

  box/net.box_incorrect_iterator_gh-841.test.lua	gh-5434
  replication/election_basic.test.lua			gh-5368
  replication/election_qsync.test.lua			gh-5430
  replication/election_qsync_stress.test.lua		gh-5395
  replication/gh-4402-info-errno.test.lua		gh-5366
  replication/gh-5426-election-on-off.test.lua		gh-5433
  wal_off/snapshot_stress.test.lua			gh-5431

Test replication/election_qsync_stress.test.lua flaky fails and even
often on FreeBSD 12. It has stable reproducer shown in the issue. To
stabilize the gitlab testing blocked this test from running on
FreeBSD 12 with skip condition file till it will be fixed.

Part of #5395

Since LuaJIT provides public C API symbols that are used in the final
executable, the linker may not just throw it away.
Nevertheless for future compatibility all symbols from LuaJIT public API
should be added to exports.h file.

Follows up #5187

Since LuaJIT provides extended LuaC API introduced in the scope of
5a61e1ab54b5c66bfebd836db1ac47996611e065 ('misc: add C and Lua API for
platform metrics') corresponding header should be provided along with
other Tarantool development files.

Follows up #5187

Since 9fba29ab (memtx: introduce tuple
compare hint) memtx tree key data (indexes) started to contain extra 8
bytes as a hint. Now it is optional and can be turned off in an index
options with "hint = false" entry.

Closes #4927

@TarantoolBot document
Title: memtx: optional tuple compare hints
Update the documentation for an index creation to reflect that there is
now an option to turn off hints for the index.

It is needed for the further c++ implementation in memtx_tree.cc. To
see the file diff properly it should not be renamed and reworked
in one commit. Some not c++ comparable casts were fixed.

Prerequisites: #4927

* misc: add lmisclib.h to lua.hpp header
* build: fix build for Windows-like systems
* test: fix warning for old gcc at testgetmetrics.c

To make vinyl/upgrade tests hierarchy clean and follow more extendible
folder structure (like in xlog/upgrade), let's a bit reorganize upgrade
test.

Follow-up #5107

It was disabled since 9cf66ac0
Let's remove it to avoid test contamination folder.

This patch enables "ephemeral index" optimization (which is called
Automatic Indexes in the original SQLite implementation). The main idea
behind the optimization is to create temporary (ephemeral) space that lasts
only the duration of single SQL statement. Such space stores intermediate
execution results. Since in Tarantool index can't exist without space
(in contrast to SQLite) we call this optimization "ephemeral index".
It may turn out beneficial due to the fact that cost of creating and
filling space with single index is (NlogN). So if we lookup data more
than logN times during the query execution, ephemeral index may give as
better timings (space full-scan is still O(N)).

As a rule "ephemeral index" space relates to one of spaces used in query
and contains all or fewer fields than the original space. The order of
fields in ephemeral space may be different from order in the original space.
In addition to the mentioned fields, ephemeral index has a rowid -
additional unsigned field with non-decreasing values which allows to
store not unique values.
Example:

CREATE TABLE t1 (i INT PRIMARY KEY, a INT, b INT, c INT);
CREATE TABLE t2 (i INT PRIMARY KEY, a INT, b INT, c INT);
...
SELECT t1.a, t2.b FROM t1, t2 WHERE t1.b = t2.c

In case the planner decides to use an ephemeral index and chooses t1 as
the source space, an ephemeral index with three fields will be created
and will contain the values from t1.b, t1.a, and rowid in that order.
If t2 is chosen as the source space, an ephemeral index will contain the
values from t2.c, t2.b, and rowid.

This optimization can also be applied to store results of subquery
evaluation.

Closes #4933

Co-authored-by: Mergen Imeev <imeevma@gmail.com>
Co-authored-by: Timur Safin <tsafin@tarantool.org>
Reviewed-by: Nikita Pettik <korablev@tarantool.org>

Found that if the previos test like gh-4957-too-many-upserts.test.lua
leaves some garbage on the current Tarantool server, then the next
running gh.test.lua test hangs. To resolve it decided to use workaround
and to set delay before the test start with fiber.sleep(1) to be able
to get out of one loop iteration.

Closes #5141

Adding forgotten resource cleanup to the
`test_tuple_validate_formatted` test, to make ASAN
happy again.

Closes #5432
Relates to #5384

To ease the further maintenance of the tests located in LuaJIT repo,
particular hardcoded paths are replaced with the root path to test
directory within LuaJIT third party submodule.

Closes #5425

Reviewed-by: Timur Safin <tsafin@tarantool.org>
Reviewed-by: Alexander V. Tikhonov <avtikhon@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

* test: introduce the root CMakeLists.txt

This patch looks into tarantool repo files, extracts
download url and adds it into feedback report.

Currently only centos, fedora (yum) and ubuntu (apt) are
supported.

Closes #5281

When running test without using test-run, i.e:

```
tarantool test/box-tap/feedback_daemon.test.lua
```

test would fail on second run, since database was not removed.