Currently, there is a fundamental logical inconsistency with read-view and
conflicted states of transactions.

Conflicted transactions see all prepared changes (e.g., #7238), because
they are handled differently than read-view ones. At the same time, one
does not know the state of the transaction until `box.commit` is called.

A similar problem arises with read-view transactions: if such transactions
do any DML statements, they are de-facto conflicted, but this will only be
determined at preparation stage:
https://github.com/tarantool/tarantool/blob/79245573dabf3c1eb4eb904fd80ee84270360476/src/box/txn.c#L1006-L1013

Fix this inconsistency by the following changes:
1. Conflict "read-view" transactions on attempt to perform DML statements
immediately — guarantee this with an assertion at preparation stage.
2. Make conflicted transactions unconditionally throw "Transaction has been
aborted by conflict" error on any CRUD operations (including read-only
ones) until they are either rolled back (which will return no error) or
committed (which will return the same error).

Closes #7238
Closes #7239
Closes #7240

@TarantoolBot document
Title: new  behaviour of "conflicted" transactions

"Conflicted" transactions now return "Transaction aborted by conflicted"
error on any CRUD operations (including read-only ones), until they are
either rolled back (which will return no error) or committed (which will
return the same error).

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

Regular expression now works on versions: alpha, beta, rc and so on.

NO_DOC=bugfix
NO_TEST=bugfix
NO_CHANGELOG=bugfix

Tabs were replaced with spaces to bypass checkpatch.

NO_DOC=bugfix
NO_TEST=bugfix
NO_CHANGELOG=bugfix

The `cmp_tuple` helper function is broken - it assumes that all tuple
fields, including the payload, are numeric. It isn't true - the payload
field is either nil or string. This results in a false-positive test
failure:

```
error: '[string "function cmp_tuple(t1, t2)     for i = 1, PAY..."]:1:
       attempt to compare nil with string'
```

Closes #6336

NO_DOC=test
NO_CHANGELOG=test

Bump test-run to new version with the following improvements:

  - Fix issue with not detecting successful server start [1]

[1] tarantool/test-run#343

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

With the default shutdown timeout of 3 seconds, a test that leaves
behind asynchronous requests would still pass, but it would take longer
to finish, because the server instance started by Tarantool would have
to wait for the dangling requests to complete. Setting the timeout to
infinity will result in a hang, making us fix the test.

Infinite timeout is also good for catching bugs like #7225 and #7256.

We don't set the timeout for diff and TAP tests because those are
deprecated and shouldn't be used for writing new tests. Nevertheless,
I manually checked that none of them hangs if the timeout is set to
infinity.

Closes #6820

NO_DOC=test
NO_CHANGELOG=test

If a socket fd is shared by a child process, closing it in the parent
will not shut down the underlying connection. As a result, the server
may hang executing the graceful shutdown protocol. Fix this problem by
explicitly shutting down the connection socket fd before closing it.

This is a recommended way to terminate a Unix socket connection, see
http://www.faqs.org/faqs/unix-faq/socket/#:~:text=2.6.%20%20When%20should%20I%20use%20shutdown()%3F

Closes #7256

NO_DOC=bug fix

It's possible to wakeup a fiber, which is waiting for WAL write
completion, using Tarantool C API. This results in an error like:
```
main/118/lua F> Journal result code -1 can't be converted to an error
```

This patch introduces a flag, which is set when WAL write is
finished, that allows fibers to yield until the flag is set.

Closes #6506

NO_DOC=bugfix

Bump test-run to new version with the following improvements:

  - Fail *.test.py tests in case of server start errors [1]

[1] tarantool/test-run#333

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

This patch fixes format building when an ephemeral space was used in
ORDER BY and ORDER BY uses at least two variables from the list of
selected columns.

Closes #7042

NO_DOC=Bugfix

There was an assertion failure when inserting  a decimal into an index
which contained double Inf or NaN.

The reason for that was never checking decimal_from_*() return values,
and decimal_from_double() not being able to handle NaN or Inf, because
these values are not representable in decimal numbers.

Start handling decimal_from_<type> return values and fix decimal
comparison with Inf, NaN.

Closes #6377

NO_DOC=bugfix

To reduce the chance to encounter the tarantool/test-run#141 issue in
replication-py/swim tests, let's switch to using unix sockets instead
of TCP ports for tarantool console.

NO_DOC=testing stuff
NO_TEST=testing stuff
NO_CHANGELOG=testing stuff

Currently it's possible to wakeup a fiber, which is waiting for `cbus_call`
completion, using Tarantool C API. This will cause a misleading `TimedOut`
error. This patch reworks `cbus_call` in such a way that it yields until
a completion flag is set.

Part of #7166

NO_DOC=refactoring
NO_CHANGELOG=refactoring

Part of #7166

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Simplify/shorten `interval_to_string()` implementation.

Part of #7045

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Do not even try to make more readable output of secs/nsec,
but rather report them as is, without any [de]normalization.

Not the prior way:
```
tarantool> dt.interval.new{min=1, sec=59, nsec=2e9+1}
--
- +1 minutes, 61.000000001 seconds
...
```

But instead as:
```
tarantool> dt.interval.new{min=1, sec=59, nsec=2e9+1}
--
- +1 minutes, 59 seconds, 2000000001 nanoseconds
...
```

Closes #7045

NO_DOC=internal

The graceful shutdown protocol works as follows:

 1. The server sends a shutdown request (the box.shutdown event) to all
    its clients that subscribed to it.
 2. Upon receiving a shutdown request, a client is supposed to close its
    connection.
 3. The server waits for all clients subscribed to box.shutdown event to
    exit.
 4. The server exits.

In net.box, the box.shutdown event is processed by `remote._callback`.
The problem is it may occur that `remote._callback` is garbage collected
while the `remote` object isn't. If this happens, the shutdown request
will never get processed, and the server won't exit until the `remote`
object is garbage collected, which may take forever.

Let's fix this issue by breaking the worker loop if we see that the
callback was garbage collected.

Closes #7225

NO_DOC=bug fix

nixos (and probably some other distributives) place zoneinfo directory
not in /usr/share (in /etc for example). TZDIR is set accordingly.
Currently zoneinfo is looked for in /usr/share, disregarding TZDIR env
variable.

This commit adds compile definition for TZDIR if such env variable is
defined. This fixes zoneinfo lookup for nixos.

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

This reverts commit 33830978.

Follow-up #6477

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

This reverts commit 9d1f9f0e.

Follow-up #6477

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Two things we need to do to fix build with OpenSSL 3.0:

1. Use EVP_MAC_* functions instead of HMAC_*
   https://www.openssl.org/docs/man3.0/man3/HMAC_CTX_new.html

2. Load the Legacy provider to enable legacy algorithms, such as MD4
   https://wiki.openssl.org/index.php/OpenSSL_3.0#Programming_in_OpenSSL_3.0

Closes #6477

NO_DOC=build fix
NO_TEST=build fix
NO_CHANGELOG=build fix

We redefine ssl_init and ssl_free in the EE build, because we need to do
some extra work there. Currently, it's fine to duplicate the bulk of the
OpenSSL library initialization code between EE and CE repositories, but
with the introduction of OpenSSL 3.0 it's going to become more
complicated so duplicating would look bad. Let's move the common code to
ssl_init_impl() and ssl_free_impl() helper functions.

Needed for #6477

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

ERR_error_string adds some extra information that depends on the OpenSSL
library version (code, module, method). This information says nothing to
the end user, and it results in different test results after updating to
OpenSSL 3.0. Let's use ERR_reason_error_string instead, which just
prints a human-readable error message.

Part of #6477

NO_DOC=minor change in error message
NO_CHANGELOG=minor change in error message

openssl_err_str is used for reporting OpenSSL errors. It calls
crypto_ERR_* functions using FFI. There's a typo in the code:
ffi.crypto_ERR_error_string is used instead of ffi.C.*.

We don't normally step on this, because OpenSSL doesn't return errors
in our configuration, but if it did for some reason (e.g. a cipher was
disabled in the library), we'd get a confusing error message.

NO_DOC=bug fix
NO_TEST=occur only on internal error
NO_CHANGELOG=occur only on internal error

Added 'tarantool/actions/cleanup' action to each job which uses
self-hosted runners.

The action cleans workspace directory of self-hosted runner after
previous run. The main reason to add this action is 'Need a single
revision' error [1] caused by a conflict of submodule versions,
the standard 'actions/checkout' action fails with this error. It's a
well-known problem and related issue [2] is still opened.

[1] https://github.com/tarantool/tarantool-qa/issues/145
[2] https://github.com/actions/checkout/issues/418

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Closes tarantool/tarantool-qa#145

This patch introduces not thread-safe low resolution
monotonic clock, based on interval timer. It should
be used only by thread that initialized it.

Part of #6085

NO_CHANGELOG=internal feature
NO_DOC=internal feature

Since the use of sigprocmask() is unspecified in a multithreaded
process we should use pthread_sigmask() instead. This patch
replaces all the sigprocmask calls with pthread analogue.

NO_TEST=refactoring
NO_CHANGELOG=refactoring
NO_DOC=refactoring

To ensure that regular and GC64 jobs in packaging workflows don't
overwrite artifacts of each other, we need to use a different artifact
name per job.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

In PR #7090 we forgot to update the path to gather failure artifacts for
many packaging workflows. Now it is fixed.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

GitHub suggests to look on the document, when it is present. The banner
is shown for visitors of issues and pull requests pages. We can use this
feature to highlight typical stones on the contributor's road.

My goal is to leave suggestions for typical points of a confusion in a
short document (don't bury into details) and leave links for further
reading.

I hope we should move most of content of the 'open a pull request'
section to a wiki page and possibly merge it with [1] and [2]. It
requires more work and I would consider it as a future step toward good
developer guidelines.

[1]: https://github.com/tarantool/tarantool/wiki/Code-review-procedure
[2]: https://www.tarantool.io/en/doc/latest/dev_guide/developer_guidelines

NO_CHANGELOG=not a user visible change
NO_DOC=nothing requires an update on the website
NO_TEST=it is purely documentation change

Introduce helpers for each of our custom extension types.
The helpers will be used by mp_check() to validate extension contents
and make sure no malformed data is accepted by tarantool.

Closes #6857

NO_DOC=no user visible changes

The inject remote method is used in testing quite a lot. For example,
when someone needs to pass arbitrary MsgPack to Tarantool.

The current way to use it is:
conn:_request(netbox._method.inject, opts, nil, nil, custom_msgpack)

This is quite long and ugly, so let's add a helper:
conn:_inject(custom_msgpack, opts)

Also, while we're at it, lets fix the following issue.
The _inject remote method is barely usable without knowing which sync
net.box will use for the next request.

On the one hand, the user has to encode some custom sync to the request
he's injecting.
On the other hand, net.box doesn't parse the custom sync and always uses
its own pre-generated one to wait for the request response.

So the user has to pick the correct sync value, which net.box uses
internally.

Let's make life simpler and introduce _next_sync method, which returns
the next sync, which will be used by net.box.

In-scope-of #6857
Closes #7177

NO_DOC=internal change
NO_CHANGELOG=internal change
NO_TEST=tested implicitly in next commit

A Tarantool server is supposed to send a greeting message right after
accepting a new client so the first thing an applier does after
connecting to the master is reads the greeting. It does this without
timeouts. The problem is that if by mistake we connect to a wrong
instance, which doesn't send anything to clients, the applier will hang
forever (until the remote closes the socket), without logging any
errors.

This may happen even with a valid Tarantool instance - if SSL encryption
is enabled on the master, but not on the client, because the SSL
protocol assumes that the client initiates a connection by writing to
the socket first (before the server).

Let's add a timeout to the operation reading the greeting. The timeout
is set to replication_disconnect_timeout(), after which a connection is
broken if the master doesn't send heartbeats for that long. Note, we
don't add a timeout to other read/write operations issued to initiate a
replication connection, because if we received a greeting and it's
valid, then the master is likely to be fine.

Closes #7204

NO_DOC=bug

The upgrade test worked incorrectly: the instances WAL directory contained
xlog with the latest schema version and the snapshot contained an older
schema version (2.7.1) than required (2.9.1), whereas there only need to
be a snapshot with 2.9.1 schema version — fix this descrepancy.

Also, schema upgrade is performed automatically: remove the redundant
`box.schema.upgrade` call.

NO_CHANGELOG=test fix
NO_DOC=test fix
NO_DOC=test fix

Commit b18dd47f ("Introduce backtrace=true option to fiber.info()")
introduced a way to skip backtraces in fiber.info() calls.
Commit 9da7e03e ("fiber: introduce fiber_o:info() and fiber_o:csw()")
introduced `options` function for fiber object however it ignored
passed options.
This patch fixed it. Currently fiber:info({backtrace = false})
returns info without backtrace.

Closes #7210

NO_DOC=bugfix

- Ubuntu Groovy Gorilla (20.10)
- Ubuntu Hirsute Hippo (21.04)

See for details: https://wiki.ubuntu.com/Releases

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

- Fedora 30
- Fedora 31
- Fedora 32
- Fedora 33

See for details: https://docs.fedoraproject.org/en-US/releases/eol

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

FreeBSD Tarantool port has some problems if build uses cmake'
ExternalProject_Add which refers to the github repository.
Sumodule works better - so switching to using it.

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

Previous version of tzcode/CMakeLists.txt leaves untracked files under
source directory src/lib/tzcode/tz/. We have changed slightly an approach
used to checkout and build IANA github sources under build/ directory,
not in ${PROJECT_BINARY_DIR} (which may be pointing not to out-of-source
build directory, but rather to the repository root itself, if insource
build used).

Closes #7173

NO_CHANGELOG=infra
NO_DOC=infra
NO_TEST=infra