prbuf is partitioned ring buffer. The main property of the buffer is
that it can be recovered from raw memory. To achieve this buffer saves
metadata before each stored entry. For further details see source code.

NO_DOC=<Private data structure>
NO_CHANGELOG=<No user visible changes>

Currently `tarantoolctl rocks --help` generate such help message:

NAME
	/usr/bin/tarantoolctl - LuaRocks main command-line interface

SYNOPSIS
	/usr/bin/tarantoolctl [<flags...>] [VAR=VALUE]...

This is wrong.

This patch makes the output look like this:

NAME
	/usr/bin/tarantoolctl rocks - LuaRocks main command-line interface

SYNOPSIS
	/usr/bin/tarantoolctl rocks [<flags...>] [VAR=VALUE]...

NO_DOC=bugfix

This patch updates libicu version to 71.1.
See changelog for details (https://icu.unicode.org/download/71).
The main reason to do that is a bug in datetime parsing that was
found using icu-date module (tarantool/icu-date#27)
and was fixed in linicu upstream
(https://unicode-org.atlassian.net/browse/ICU-21802).

NO_DOC=build
NO_TEST=build

* memprof: enrich symtab when new trace is compiled
* memprof: substitute long proto names with aliases
* memprof: enrich symtab when meeting new prototype
* memprof: add symbol generations

Closes #5815

NO_DOC=LuaJIT submodule bump
NO_TEST=LuaJIT submodule bump

Unlimited number of make jobs may lead to OOM. We should use
`make -j $(nproc)` instead of `make -j` on Linux. Note, on OSX
and FreeBSD the `nproc` command is unavailable out of the box.
So using `sysctl -n hw.ncpu` instead.

Closes #7020

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

We have recently turned off retrying test runs after a failure for
improving developer experience on using test-run on local machines.
Now to enable tests auto-rerun on CI let's pass TEST_RUN_RETRIES=3
to environment.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Bump test-run to new version with the following improvements:

- Add option to tune test run retries after failure [1]
- Set 0 as default value for retries after failure [1]
- Add an option to show test-run's environment vars [2]
- Prettify `test-run.py --help` output [3]

[1] https://github.com/tarantool/test-run/pull/336
[2] https://github.com/tarantool/test-run/pull/326
[3] https://github.com/tarantool/test-run/pull/337

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

This patch introduces memtx_tx_region and memtx_tx_mempool:
engineers must use only these proxies to collect statistics.

Also this patch introduces box.stat.memtx.mvcc - the way to
get memtx mvcc memory statistics.

Closes #6150

@TarantoolBot document
Title: Memtx MVCC memory monitoring

Introduce memtx MVCC memory monitoring. One can get it with
box.stat.memtx.tx() method or use index to access a particular
statistic. The statistics format:

txn:
  statements:
    max: 0
    avg: 0
    total: 0
  user:
    max: 0
    avg: 0
    total: 0
  system:
    max: 0
    avg: 0
    total: 0
mvcc:
  trackers:
    max: 0
    avg: 0
    total: 0
  conflicts:
    max: 0
    avg: 0
    total: 0
  tuples:
    tracking:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    used:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    read_view:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0

Since transaction can allocate objects on its own, we need to track it
to collect memory statistics of memtx txm. Txn's use only region
allocator so this patch introduces tx_region methods - txns should use
it instead of region methods to collect allocation statistics.

Part of #6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes
NO_TEST=already covered

Since a transaction has been prepared its garbage (produced stories
and replaced tuples) cannot be deleted because they are recognized
as used in read-view even if there are no transactions in read-view.
This problem makes it difficult to test the memory monitoring system,
so this patch solves the problem.

Close #6635
Part of #6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring

It is difficult to monitor memory consumption for each transcation
when some functions allocating on txn's region do not have any information
about owner of the region. This patch solve the problem.

Part of #6150

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring

An opportunity to call garbage collector of memtx transaction
manager manually allows to understand which garbage cannot be freed.
This knowledge can help us to improve garbage collector. Also this
opportunity makes it easier to test memtx mvcc memory monitoring.

Part of #6150

NO_DOC=internal feature
NO_CHANGELOG=internal feature
NO_TEST=internal feature

There is no information about owner of txn_stmt when it is
being allocated. It makes difficult to track memory
consumption for each transaction. That is why we need to
pass transaction itself to txn_stmt_new() instead of its region.

Part of #6150

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring

Commit/rollback triggers are run asynchronously, upon receiving the
write status from WAL. We can't run them in the original fiber that
submitted the WAL request, because it would open a time window between
writing a transaction to WAL and committing it in tx, which could lead
to violating the cascading rolback principles. As a result,
commit/rollback triggers run with admin privileges.

Let's fix this issue by temporarily setting session and credentials to
the original fiber's for running commit/rollback triggers.

Closes #7005

NO_DOC=bugfix

Tarantool used to crash if launched with multiple -e or -l options without
a space between the option and the value, e.g.: `tarantool -ea -eb`.
It happened because optv[] was allocated for argc==3 items, while 4
options were written to it after parsing (-e, a, -e, b).
This patch allocates optv[] for the maximum possible number of options:
(argc - 1) * 2.

Closes #5747

NO_DOC=bugfix

In commit c1c77782
("replication: fix bootstrap failing with ER_READONLY")
seek_once was changed to seek_wait. Seek_once returned a non-negative int
on success and -1 if failed, seek_wait returns True on success and
False if failed. Therefore no need to compare it to 0 anymore.

NO_DOC=Minor fix in test
NO_CHANGELOG=Minor fix in test

The option will be used in the Enterprise version to configure audit
log event filter.

NO_DOC=ee
NO_CHANGELOG=ee

The functions are not used since commit 1a6ad79e ("lua/cfg: drop
unused log methods").

NO_DOC=drop unused code
NO_TEST=drop unused code
NO_CHANGELOG=drop unused code

This patch fixes an issue with the implicit cast of INTEGER and DECIMAL
values to DOUBLE when they are passed as the first argument to the
ROUND() function.

Closes #6988

@TarantoolBot document
Title: ROUND() now properly supports INTEGER and DECIMAL

INTEGER and DECIMAL values passed as the first argument now will not be
cast to DOUBLE and the result will be of the same type as the first
argument. Also, the default type for the ROUND() is now DECIMAL.

We need to include EXTRA_CORE_INCLUDE_DIRS in order to build unit tests.
Including it into test/CMakeLists.txt is useless. Moreover, it may break
EE build in case EXTRA_CORE_INCLUDE_DIRS includes a system directory
that contains a file that has the same name as one of our internal
headers.

See https://github.com/tarantool/tarantool-ee/issues/71

NO_DOC=cmake
NO_TEST=cmake
NO_CHANGELOG=cmake

The option will be used in the Enterprise version to configure audit
log format.

NO_DOC=ee
NO_CHANGELOG=ee

The function doesn't modify neither the array passed to it, nor the
strings in the array so it should use const.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

@TarantoolBot document
Title: Document transaction isolation settings

All transactions in tarantool see only committed changes, but is
a loose concept since the commit of a transaction is long-term
act and there's a dilemma of choosing a timepoint at which the
changes must be seen by other transactions.

When enabled, mvcc engine allows some options on this field.
To be  specific we introduced two levels of isolaton:
* read-committed: all transaction that have started committing
(box.commit() was called) are visible.
* read-confirmed: all transaction that have finished committing
(box.commit() returnd) are visible.

The first level is good for RW transactions since it allows to
avoid some conflicts, while the second is good for RO transactions
since it allows to get tuples that a definitely was perisisten on
disk.

There's also a default option that allows mvcc engine to decide:
* best-effort: do the best depending of which actions the
transaction has made.

For autocommit transaction (one-statement actions without explicit
box.begin/commit calls) an obvious rule is apllied: RO (select,
get, count etc) transactions are done with read-confirmed; all
other (replace, delete, etc) - read-committed.

If a transaction has an explicit box.begin call, the level can be
specified in the following way:
box.begin({tnx_isolation = 'default'})
box.begin({txn_isolation = 'read-committed'})
box.begin({txn_isolation = 'read-confirmed'})
box.begin({tnx_isolation = 'best-effort'})

One can similarly set the level in net.box stream begin() method.

If not specified (or if 'default' was passed) the level is taken
from configuration. One can set default level in box.cfg:
box.cfg({default_txn_isolation = 'read-committed'})
box.cfg({default_txn_isolation = 'read-confirmed'})
box.cfg({default_tnx_isolation = 'best-effort'})

The default option is 'best-effort'.

In iproto the level is specified by IPROTO_TNX_ISOLATION key in
the body of IPROTO_BEGIN request. The value is the one of the
following enum values:
enum txn_isolation_level {
	/** Take isolation level from global default level. */
	TXN_ISOLATION_DEFAULT = 0,
	/** Allow to read committed, but not confirmed changes. */
	TXN_ISOLATION_READ_COMMITTED = 1,
	/** Allow to read only confirmed changes. */
	TXN_ISOLATION_READ_CONFIRMED = 2,
	/** Determine isolation level automatically. */
	TXN_ISOLATION_BEST_EFFORD = 3,
};

NO_TEST=no changes
Closes #6930

Introduce new option IPROTO_TXN_ISOLATION (0x59) in the body of
IPROTO_BEGIN request, so a user can set isolation level similar
to box.begin in lua.

The value must be one of the following integers:
enum txn_isolation_level {
	/** Take isolation level from global default level. */
	TXN_ISOLATION_DEFAULT,
	/** Allow to read committed, but not confirmed changes. */
	TXN_ISOLATION_READ_COMMITTED,
	/** Allow to read only confirmed changes. */
	TXN_ISOLATION_READ_CONFIRMED,
	/** Determine isolation level automatically. */
	TXN_ISOLATION_BEST_EFFORT,
};

Support the new option in net.box.

Part of #6930
NO_DOC=see later commits
NO_CHANGELOG=see later commits

Now memtx TX manager tries to determine the best isolation level
by itself. There could be two options:
* READ_COMMITTED, when the transaction see changes of other tx
that are committed but not yet confirmed (written to WAL)
* READ_CONFIRMED, when the transaction see only confirmed changes.

Introduce a simple way to specify the isolation level explicitly:
box.begin{tx_isolation = 'default'} - the same as box.begin().
box.begin{tx_isolation = 'read-committed'} - READ_COMMITTED.
box.begin{tx_isolation = 'read-confirmed'} - READ_CONFIRMED.
box.begin{tx_isolation = 'best-effort'} - old automatic way.

Intrduce a bit more complex but potentially faster way to set
isolation level, like that:
my_level = box.tnx_isolation_level.READ_COMMITTED
..
box.begin{tx_isolation = my_level}

For simplicity of implementation also support symmetric values as
'READ_COMMITTED' and box.tnx_isolation_level['read-committed'].

Introduce a new box.cfg option - default_tx_isolation, that is
used as a default when a transaction is started. The option is
dynamic and possible values are the same as in box.begin, except
'default' which is meaningless.

In addition to string value the corresponding numeric values can
be used in both box.begin and box.cfg.

Part of #6930
NO_DOC=see later commits
NO_CHANGELOG=see later commits

When a transaction is started without specifying isolation level
(which is impossible now) the transactional manager must choose
the transaction level automatically, that means that is must
detemine whether the transaction can see other prepared changes
or not. The best effort that we can made is to check if current
transaction is read-only or not. For read-only transactions
there are hope and fear that it will remain read-only, and the
best choice is not to see prepared changes. But if the transaction
has DML statements - it must see prepared changed.

Note that a read-only transaction can became read-write if it make
a DML statement. But if a transaction ignores some other prepared
change and then makes a DML, there are no other options except
abort that transaction - it could not be serialized anymore.

Part of #6930
Closes #6246
NO_DOC=see later commits

This flag actually describes isolation level:
is_prepared_ok == true - READ_COMMITTED,
is_prepared_ok == false - READ_CONFIRMED.

Now it is always calculated as tnx != NULL.
Let memtx_tx_tuple_clarify to calculate it by itself.

No logical changes.

Part of #6930
NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring

No logical changes.

Part of #6930
NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring

Internally space's indexes can containt dirty tuples that are
invisible for user. That's why there's special adjustment in
space:count() that substracts number of invisible tuple in the
space.

By a mistake that check thought that all prepared statements are
visible, which is wrong for autocommit reads, like standalone
space:count() without explicit transaction.

Fix it by using common for all reads practice: ignore prepared
statements if current transaction is NULL.

Closes #6421
NO_DOC=bugfix

Memtx TX manager stores a pointer to deleting statement in prepared
story. This pointer is set in two cases:
1. a statement deletes (or overwrites) a tuple
2. a story becomes prepared while other inprogress TX overwrites it

By design a tuple can be deleted only by primary index, the case
when a transaction overwrites somehting in secondary index but does
not overwrite the same tuple in primary index is prohibited. That's
why the pointer (to deleting statement) must be set by and only by
the next statement in the primary index chain.

By mistake the pointer is set also in second index chain analysis
after reordering which led to unexpected state of a story.

The patch removes the problem.

Closes #6452
NO_DOC=bugfix

We used to use very ugly and tricky approach to check that passed years,
months and days were not exceeding supported range of values. Now we have
introduced to `c-dt` library the new function `dt_from_ymd_checked` for
that purpose (i.e. check that values are valid, and construct dt from
them). So rewrite/simplify Lua code to use that entry as
`tnt_dt_from_ymd_checked`.

Part of #6731

NO_DOC=refactoring
NO_CHANGELOG=refactoring

We have introduced new parse functions, and modified relevant
external submodules to support huge years formats.

@TarantoolBot document
Title: Document datetime.parse

We support relaxed ISO-8601/RFC339 date formats. Consider RFC-3339 as a
profile of a strict ISO-8601, not major extension. We use c-dt, which is
ISO-8601 conformant in such way that we accept superset of ISO-8601
strings, with a few reasonable extensions.

Few examples for further clarification:

Years
=====

| Basic | Extended | ... | ISO-8601? |
|-------|-----------|-----|-----------|
| `20121224` | `2012-12-24` | Calendar date | yes |
| `2012359` | `2012-359` | Ordinal date | yes |
| `2012W521` | `2012-W52-1` | Week date | yes |
| `2012Q485` | `2012-Q4-85` | Quarter date |  |

Modified [`c-dt` parser supports extended years](tarantool/c-dt#1)
values, which are outside of standard [0001 .. 9999] range

| Tarantool extended values |   |
|------------------------------|---|
| `0000-01-01` | `-0001-12-31` |
| `5879611-07-11` | `-5879610-06-22` |

Time
====

| Basic | Extended |
|-------|-----------|
| `T12` | N/A |
| `T1230` | `T12:30` |
| `T123045` | `T12:30:45` |
| `T123045.123456789` | `T12:30:45.123456789` |
| `T123045,123456789` | `T12:30:45,123456789` |

The time designator `T` may be omitted. This is exactly how RFC3339 is
different to strict ISO-8601.

Timezone
========

We do not (yet) support names for standard time-zones (like `MSK`, or
`EST`, or `America/New_York` or whatever) we only support numeric offsets
at the moment.

In a form

| Basic | Extended |
|-------|-----------|
| `Z` | N/A |
| `+hh` | N/A |
| `-hhmm` | `-hh:mm` |
| `GMT+h` |  |
| `GMT-hhmm` |  |
| `GMT+h:mm | `GMT+hh:mm` |
| `UTC-h:mm` | `UTC-hh:mm` |

And relaxed ISO-8601 format is compositions of all relaxed subformats (i.e.
date, time, and timezone) putting them together as:

> {relaxed-date} ( `[Tt ]` {relaxed-time} ( `' '` {lenient-time-zone})? )?

NO_TEST=documentaton update
Closes #6731

* Default parse
  - new c-dt version used which handles extended years range
    while parse relaxed iso8601 gformat strings;
  - family of functions like dt_from_ymd_checked functions
    added to the new c-dt version, now used by conversion code
    to properly handle validation of a 32-bit boundary values;
  - datetime_parse_full() modified to properly handle huge years values;
  - added tests for extended years range.

* strptime-like parse
  - properly handle longer than 4 years values, negative values,
    and handle zulu suffix, which may be generated by Tarantool
    stringization routines;

Part of #6731

NO_DOC=internal
NO_CHANGELOG=internal

Part of #6731

NO_DOC=internal
NO_CHANGELOG=internal

To parse date/time strings using format string we use
`strptime()` implementation from FreeBSD, which is
modified to use our `struct datetime` data structure.

List of supported format has been extended to include
`%f` which is flag used whenever you need to process
nanoseconds part of datetime value.

```
tarantool> T = date.parse('Thu Jan  1 03:00:00 1970', {format = '%c'})

tarantool> T
- 1970-01-01T03:00:00Z

tarantool> T = date.parse('12/31/2020', {format = '%m/%d/%y'})

tarantool> T
- 2020-12-31T00:00:00Z

tarantool> T = date.parse('1970-01-01T03:00:00.125000000+0300',
                          {format = '%FT%T.%f%z'})

tarantool> T
- 1970-01-01T03:00:00.125+0300
```

Part of #6731

NO_DOC=internal
NO_CHANGELOG=internal

Datetime module provides parse function to create
datetime object given input string.

`datetime.parse` function expect 1 required argument - which is
input string, and set of optional parameters passed as table
in 2nd argument.

Allowed attributes in this optional table are:
* `format` - should be either 'iso8601', 'rfc3339' or `strptime`-like
  format string. [strptime format will be added as part of next
  commit];
* `tzoffset` - to redefine offset of input string value, if there
  is no timezone provided.
* `tz` - human-readable, Olson database, timezone identifier, e.g.
  'Europe/Moscow'. Not yet implemented in this commit.

```
tarantool> T = date.parse('1970-01-01T00:00:00Z')

tarantool> T
- 1970-01-01T00:00:00Z

tarantool> T = date.parse('1970-01-01T00:00:00',
                          {format = 'iso8601', tzoffset = 180})

tarantool> T
- 1970-01-01T00:00:00+0300

tarantool> T = date.parse('2017-12-27T18:45:32.999999-05:00',
                          {format = 'rfc3339'})

tarantool> T
- 2017-12-27T18:45:32.999999-0500
```

Implemented as per RFC https://hackmd.io/@Mons/S1Vfc_axK#%D0%AD%D1%82%D0%B0%D0%BF-3-%D0%9F%D0%B0%D1%80%D1%81%D0%B8%D0%BD%D0%B3-%D0%B4%D0%B0%D1%82-%D0%BF%D0%BE-%D1%84%D0%BE%D1%80%D0%BC%D0%B0%D1%82%D1%83

Part of #6731

NO_DOC=internal
NO_CHANGELOG=internal

Previously, sysbench benchmark was running inside Docker on CI.
According to the performance testing best practices it is a quite
doubtful approach. So let's switch to a more traditional practice:
run on hardware.

Closes tarantool/tarantool-qa#158

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

When the master is just starting up it's possible for replica's JOIN
request to arrive right in time to bypass ER_LOADING check (after master
is fully recovered) but still fail due to ER_READONLY: box.cfg.read_only
is only read and set after box_cfg() (its C part) returns.

In this case the joining replica simply exits with an error and doesn't
retry JOIN.

Let's fix that. Make ER_READONLY a recoverable error and let replica
retry joining after receiving ER_READONLY.

Anonymous nodes relied on ER_READONLY to forbid replication from
anonymous to normal replicas. That check doesn't work anymore.
So introduce explicit checks banning replication from anonymous nodes.

Note, there were some alternatives to this fix.

First of all, theoretically, we could stop firing ER_LOADING later,
after box_cfg() is complete. This solution wouldn't work because it
would lead to deadlocks: the nodes would be stuck in replicaset_sync(),
because each of them rejects replication with ER_LOADING.

Another solution would be to read the real box.cfg.read_only value
earlier, in order to allow replication right after the node finishes
recovery.
This would also be bad, because we should never let a node become
writeable before box.cfg is finished. Even after local_recovery is
complete, the node should stay read-only until it synchronizes with
other replicas.

That said, neither of the two alternatives fit, so the solution with
retrying JOIN on ER_READONLY was chosen.

Since the bug is fixed, re-enable the test in which it was discovered:
replication-py/init_storage.test.py

Also, remove replication/ddl.test.lua from fragile list, since this bug
was the only reason for its fragility.

Closes #5337
Closes #6966

NO_DOC=minor bugfix

The source file will be added to the Enterprise repository.

NO_DOC=cmake
NO_TEST=cmake
NO_CHANGELOG=cmake